[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[nongnu] elpa/parseclj a71e57df4d 080/185: Rename clj-lex to parseclj-le
From: |
ELPA Syncer |
Subject: |
[nongnu] elpa/parseclj a71e57df4d 080/185: Rename clj-lex to parseclj-lex |
Date: |
Tue, 28 Dec 2021 14:05:20 -0500 (EST) |
branch: elpa/parseclj
commit a71e57df4daa450801d4fbeac23cbd7cee09d557
Author: Arne Brasseur <arne@arnebrasseur.net>
Commit: Arne Brasseur <arne@arnebrasseur.net>
Rename clj-lex to parseclj-lex
---
Cask | 2 +-
DESIGN.md | 6 +-
clj-ast.el | 6 +-
clj-edn.el | 4 +-
clj-lex.el => parseclj-lex.el | 170 ++++++++++++------------
parseclj.el | 24 ++--
test/clj-lex-test.el | 295 ------------------------------------------
test/parseclj-lex-test.el | 295 ++++++++++++++++++++++++++++++++++++++++++
8 files changed, 401 insertions(+), 401 deletions(-)
diff --git a/Cask b/Cask
index d6fa83afea..a906ff2929 100644
--- a/Cask
+++ b/Cask
@@ -4,7 +4,7 @@
(package-file "parseclj.el")
-(files "clj-lex.el"
+(files "parseclj-lex.el"
"clj-edn.el"
"clj-ast.el")
diff --git a/DESIGN.md b/DESIGN.md
index 3a31a5e242..0b52f2401b 100644
--- a/DESIGN.md
+++ b/DESIGN.md
@@ -45,11 +45,11 @@ The implementation is implemented in three parts: a lexer,
a parser, and multipl
The *lexer* turns the input text, a buffer, into tokens, data structures
representing a single syntactical unit, such as a symbol, a number, or a
delimiter like "(", ")", "#{", or "#_".
-In parseclj the lexer is a single function `clj-lex-next` which can be called
repeatedly to get a sequence of tokens. `clj-lex-next` returns the token at
"point" (i.e. the Emacs cursor position), and moves point to after the token.
+In parseclj the lexer is a single function `parseclj-lex-next` which can be
called repeatedly to get a sequence of tokens. `parseclj-lex-next` returns the
token at "point" (i.e. the Emacs cursor position), and moves point to after the
token.
A *token* is a association list (list of cons cells), with keys `:token-type`,
`:form`, `:position`, and optionally `:error-type`.
-Note: we don't add line/column numbers to the token, the consumer can add
these if needed based on the position of point before calling `clj-lex-next`.
+Note: we don't add line/column numbers to the token, the consumer can add
these if needed based on the position of point before calling
`parseclj-lex-next`.
Example:
@@ -123,7 +123,7 @@ Tokens can be recognized by the `:token-type` key, which
must always come first
The parser is again a single function `parseclj-parse`. It is a higher order
function, with much of the final result determined by the `reduce-leaf` and
`reduce-branch` functions passed in as arguments.
-`parseclj-parse` internally operates by using a stack. This stack contains
tokens (as returned by `clj-lex-next`), and reduced values.
+`parseclj-parse` internally operates by using a stack. This stack contains
tokens (as returned by `parseclj-lex-next`), and reduced values.
`reduce-leaf` is a two-argument function. It takes the current value of the
stack, and a token, and returns an updated stack, typically by parsing the
token to a value and pushing that value onto the stack.
diff --git a/clj-ast.el b/clj-ast.el
index 96e90d5bb0..f5d6d2465e 100644
--- a/clj-ast.el
+++ b/clj-ast.el
@@ -34,17 +34,17 @@
(apply 'a-list ':node-type type ':position position kvs))
(defun clj-ast--reduce-leaf (stack token)
- (if (eq (clj-lex-token-type token) :whitespace)
+ (if (eq (parseclj-lex-token-type token) :whitespace)
stack
(cons
- (parseclj--make-node (clj-lex-token-type token) (a-get token 'pos)
+ (parseclj--make-node (parseclj-lex-token-type token) (a-get token 'pos)
':form (a-get token 'form)
':value (parseclj--leaf-token-value token))
stack)))
(defun clj-ast--reduce-branch (stack opener-token children)
(let* ((pos (a-get opener-token 'pos))
- (type (clj-lex-token-type opener-token))
+ (type (parseclj-lex-token-type opener-token))
(type (cl-case type
(:lparen :list)
(:lbracket :vector)
diff --git a/clj-edn.el b/clj-edn.el
index a1de242a41..393dd6783c 100644
--- a/clj-edn.el
+++ b/clj-edn.el
@@ -42,13 +42,13 @@ changes the behavior of the EDN reader. Instead pass your
own
handlers as an optional argument to the reader functions.")
(defun clj-edn-reduce-leaf (stack token)
- (if (member (clj-lex-token-type token) (list :whitespace :comment))
+ (if (member (parseclj-lex-token-type token) (list :whitespace :comment))
stack
(cons (parseclj--leaf-token-value token) stack)))
(defun clj-edn-reduce-branch (tag-readers)
(lambda (stack opener-token children)
- (let ((token-type (clj-lex-token-type opener-token)))
+ (let ((token-type (parseclj-lex-token-type opener-token)))
(if (member token-type '(:root :discard))
stack
(cons
diff --git a/clj-lex.el b/parseclj-lex.el
similarity index 55%
rename from clj-lex.el
rename to parseclj-lex.el
index f85818ecfb..78c89f17b4 100644
--- a/clj-lex.el
+++ b/parseclj-lex.el
@@ -1,4 +1,4 @@
-;;; clj-lex.el --- Clojure/EDN Lexer
+;;; parseclj-lex.el --- Clojure/EDN Lexer
;; Copyright (C) 2017 Arne Brasseur
@@ -25,7 +25,7 @@
;; A reader for EDN data files and parser for Clojure source files.
-(defun clj-lex-token (type form pos &rest args)
+(defun parseclj-lex-token (type form pos &rest args)
`((type . ,type)
(form . ,form)
(pos . ,pos)
@@ -33,17 +33,17 @@
(cons (car pair) (cadr pair)))
(seq-partition args 2))))
-(defun clj-lex-token-type (token)
+(defun parseclj-lex-token-type (token)
(and (listp token)
(cdr (assq 'type token))))
-(defun clj-lex-token? (token)
+(defun parseclj-lex-token? (token)
(and (listp token)
(consp (car token))
(eq 'type (caar token))
(not (listp (cdar token)))))
-(defun clj-lex-at-whitespace? ()
+(defun parseclj-lex-at-whitespace? ()
(let ((char (char-after (point))))
(or (equal char ?\ )
(equal char ?\t)
@@ -51,44 +51,44 @@
(equal char ?\r)
(equal char ?,))))
-(defun clj-lex-at-eof? ()
+(defun parseclj-lex-at-eof? ()
(eq (point) (point-max)))
-(defun clj-lex-whitespace ()
+(defun parseclj-lex-whitespace ()
(let ((pos (point)))
- (while (clj-lex-at-whitespace?)
+ (while (parseclj-lex-at-whitespace?)
(right-char))
- (clj-lex-token :whitespace
+ (parseclj-lex-token :whitespace
(buffer-substring-no-properties pos (point))
pos)))
-(defun clj-lex-skip-digits ()
+(defun parseclj-lex-skip-digits ()
(while (and (char-after (point))
(<= ?0 (char-after (point)))
(<= (char-after (point)) ?9))
(right-char)))
-(defun clj-lex-skip-number ()
+(defun parseclj-lex-skip-number ()
;; [\+\-]?\d+\.\d+
(when (member (char-after (point)) '(?+ ?-))
(right-char))
- (clj-lex-skip-digits)
+ (parseclj-lex-skip-digits)
(when (eq (char-after (point)) ?.)
(right-char))
- (clj-lex-skip-digits))
+ (parseclj-lex-skip-digits))
-(defun clj-lex-number ()
+(defun parseclj-lex-number ()
(let ((pos (point)))
- (clj-lex-skip-number)
+ (parseclj-lex-skip-number)
;; 10110r2 or 4.3e+22
(when (member (char-after (point)) '(?E ?e ?r))
(right-char))
- (clj-lex-skip-number)
+ (parseclj-lex-skip-number)
;; trailing M
(when (eq (char-after (point)) ?M)
@@ -100,26 +100,26 @@
(and (member char '(?. ?* ?+ ?! ?- ?_ ?? ?$ ?& ?= ?<
?> ?/)))))
(progn
(right-char)
- (clj-lex-token :lex-error
+ (parseclj-lex-token :lex-error
(buffer-substring-no-properties pos (point))
pos
'error-type :invalid-number-format))
- (clj-lex-token :number
+ (parseclj-lex-token :number
(buffer-substring-no-properties pos (point))
pos)))))
-(defun clj-lex-digit? (char)
+(defun parseclj-lex-digit? (char)
(and char (<= ?0 char) (<= char ?9)))
-(defun clj-lex-at-number? ()
+(defun parseclj-lex-at-number? ()
(let ((char (char-after (point))))
- (or (clj-lex-digit? char)
+ (or (parseclj-lex-digit? char)
(and (member char '(?- ?+ ?.))
- (clj-lex-digit? (char-after (1+ (point))))))))
+ (parseclj-lex-digit? (char-after (1+ (point))))))))
-(defun clj-lex-symbol-start? (char &optional alpha-only)
+(defun parseclj-lex-symbol-start? (char &optional alpha-only)
"Symbols begin with a non-numeric character and can contain
alphanumeric characters and . * + ! - _ ? $ % & = < >. If -, + or
. are the first character, the second character (if any) must be
@@ -133,77 +133,77 @@ behavior."
(and (<= ?A char) (<= char ?Z))
(and (not alpha-only) (member char '(?. ?* ?+ ?! ?- ?_ ??
?$ ?% ?& ?= ?< ?> ?/))))))))
-(defun clj-lex-symbol-rest? (char)
- (or (clj-lex-symbol-start? char)
- (clj-lex-digit? char)
+(defun parseclj-lex-symbol-rest? (char)
+ (or (parseclj-lex-symbol-start? char)
+ (parseclj-lex-digit? char)
(eq ?: char)
(eq ?# char)))
-(defun clj-lex-get-symbol-at-point (pos)
+(defun parseclj-lex-get-symbol-at-point (pos)
"Return the symbol at point."
- (while (clj-lex-symbol-rest? (char-after (point)))
+ (while (parseclj-lex-symbol-rest? (char-after (point)))
(right-char))
(buffer-substring-no-properties pos (point)))
-(defun clj-lex-symbol ()
+(defun parseclj-lex-symbol ()
(let ((pos (point)))
(right-char)
- (let ((sym (clj-lex-get-symbol-at-point pos)))
+ (let ((sym (parseclj-lex-get-symbol-at-point pos)))
(cond
- ((equal sym "nil") (clj-lex-token :nil "nil" pos))
- ((equal sym "true") (clj-lex-token :true "true" pos))
- ((equal sym "false") (clj-lex-token :false "false" pos))
- (t (clj-lex-token :symbol sym pos))))))
+ ((equal sym "nil") (parseclj-lex-token :nil "nil" pos))
+ ((equal sym "true") (parseclj-lex-token :true "true" pos))
+ ((equal sym "false") (parseclj-lex-token :false "false" pos))
+ (t (parseclj-lex-token :symbol sym pos))))))
-(defun clj-lex-string ()
+(defun parseclj-lex-string ()
(let ((pos (point)))
(right-char)
- (while (not (or (equal (char-after (point)) ?\") (clj-lex-at-eof?)))
+ (while (not (or (equal (char-after (point)) ?\") (parseclj-lex-at-eof?)))
(if (equal (char-after (point)) ?\\)
(right-char 2)
(right-char)))
(if (equal (char-after (point)) ?\")
(progn
(right-char)
- (clj-lex-token :string (buffer-substring-no-properties pos (point))
pos))
- (clj-lex-token :lex-error (buffer-substring-no-properties pos (point))
pos))))
+ (parseclj-lex-token :string (buffer-substring-no-properties pos
(point)) pos))
+ (parseclj-lex-token :lex-error (buffer-substring-no-properties pos
(point)) pos))))
-(defun clj-lex-lookahead (n)
+(defun parseclj-lex-lookahead (n)
(buffer-substring-no-properties (point) (min (+ (point) n) (point-max))))
-(defun clj-lex-character ()
+(defun parseclj-lex-character ()
(let ((pos (point)))
(right-char)
(cond
- ((equal (clj-lex-lookahead 3) "tab")
+ ((equal (parseclj-lex-lookahead 3) "tab")
(right-char 3)
- (clj-lex-token :character (buffer-substring-no-properties pos (point))
pos))
+ (parseclj-lex-token :character (buffer-substring-no-properties pos
(point)) pos))
- ((equal (clj-lex-lookahead 5) "space")
+ ((equal (parseclj-lex-lookahead 5) "space")
(right-char 5)
- (clj-lex-token :character (buffer-substring-no-properties pos (point))
pos))
+ (parseclj-lex-token :character (buffer-substring-no-properties pos
(point)) pos))
- ((equal (clj-lex-lookahead 6) "return")
+ ((equal (parseclj-lex-lookahead 6) "return")
(right-char 6)
- (clj-lex-token :character (buffer-substring-no-properties pos (point))
pos))
+ (parseclj-lex-token :character (buffer-substring-no-properties pos
(point)) pos))
- ((equal (clj-lex-lookahead 7) "newline")
+ ((equal (parseclj-lex-lookahead 7) "newline")
(right-char 7)
- (clj-lex-token :character (buffer-substring-no-properties pos (point))
pos))
+ (parseclj-lex-token :character (buffer-substring-no-properties pos
(point)) pos))
((equal (char-after (point)) ?u)
(right-char 5)
- (clj-lex-token :character (buffer-substring-no-properties pos (point))
pos))
+ (parseclj-lex-token :character (buffer-substring-no-properties pos
(point)) pos))
((equal (char-after (point)) ?o)
(right-char 4)
- (clj-lex-token :character (buffer-substring-no-properties pos (point))
pos))
+ (parseclj-lex-token :character (buffer-substring-no-properties pos
(point)) pos))
(t
(right-char)
- (clj-lex-token :character (buffer-substring-no-properties pos (point))
pos)))))
+ (parseclj-lex-token :character (buffer-substring-no-properties pos
(point)) pos)))))
-(defun clj-lex-keyword ()
+(defun parseclj-lex-keyword ()
(let ((pos (point)))
(right-char)
(when (equal (char-after (point)) ?:) ;; same-namespace keyword
@@ -211,70 +211,70 @@ behavior."
(if (equal (char-after (point)) ?:) ;; three colons in a row => lex-error
(progn
(right-char)
- (clj-lex-token :lex-error (buffer-substring-no-properties pos
(point)) pos 'error-type :invalid-keyword))
+ (parseclj-lex-token :lex-error (buffer-substring-no-properties pos
(point)) pos 'error-type :invalid-keyword))
(progn
- (while (or (clj-lex-symbol-rest? (char-after (point)))
+ (while (or (parseclj-lex-symbol-rest? (char-after (point)))
(equal (char-after (point)) ?#))
(right-char))
- (clj-lex-token :keyword (buffer-substring-no-properties pos (point))
pos)))))
+ (parseclj-lex-token :keyword (buffer-substring-no-properties pos
(point)) pos)))))
-(defun clj-lex-comment ()
+(defun parseclj-lex-comment ()
(let ((pos (point)))
(goto-char (line-end-position))
(when (equal (char-after (point)) ?\n)
(right-char))
- (clj-lex-token :comment (buffer-substring-no-properties pos (point)) pos)))
+ (parseclj-lex-token :comment (buffer-substring-no-properties pos (point))
pos)))
-(defun clj-lex-next ()
- (if (clj-lex-at-eof?)
- (clj-lex-token :eof nil (point))
+(defun parseclj-lex-next ()
+ (if (parseclj-lex-at-eof?)
+ (parseclj-lex-token :eof nil (point))
(let ((char (char-after (point)))
(pos (point)))
(cond
- ((clj-lex-at-whitespace?)
- (clj-lex-whitespace))
+ ((parseclj-lex-at-whitespace?)
+ (parseclj-lex-whitespace))
((equal char ?\()
(right-char)
- (clj-lex-token :lparen "(" pos))
+ (parseclj-lex-token :lparen "(" pos))
((equal char ?\))
(right-char)
- (clj-lex-token :rparen ")" pos))
+ (parseclj-lex-token :rparen ")" pos))
((equal char ?\[)
(right-char)
- (clj-lex-token :lbracket "[" pos))
+ (parseclj-lex-token :lbracket "[" pos))
((equal char ?\])
(right-char)
- (clj-lex-token :rbracket "]" pos))
+ (parseclj-lex-token :rbracket "]" pos))
((equal char ?{)
(right-char)
- (clj-lex-token :lbrace "{" pos))
+ (parseclj-lex-token :lbrace "{" pos))
((equal char ?})
(right-char)
- (clj-lex-token :rbrace "}" pos))
+ (parseclj-lex-token :rbrace "}" pos))
- ((clj-lex-at-number?)
- (clj-lex-number))
+ ((parseclj-lex-at-number?)
+ (parseclj-lex-number))
- ((clj-lex-symbol-start? char)
- (clj-lex-symbol))
+ ((parseclj-lex-symbol-start? char)
+ (parseclj-lex-symbol))
((equal char ?\")
- (clj-lex-string))
+ (parseclj-lex-string))
((equal char ?\\)
- (clj-lex-character))
+ (parseclj-lex-character))
((equal char ?:)
- (clj-lex-keyword))
+ (parseclj-lex-keyword))
((equal char ?\;)
- (clj-lex-comment))
+ (parseclj-lex-comment))
((equal char ?#)
(right-char)
@@ -282,22 +282,22 @@ behavior."
(cond
((equal char ?{)
(right-char)
- (clj-lex-token :set "#{" pos))
+ (parseclj-lex-token :set "#{" pos))
((equal char ?_)
(right-char)
- (clj-lex-token :discard "#_" pos))
- ((clj-lex-symbol-start? char t)
+ (parseclj-lex-token :discard "#_" pos))
+ ((parseclj-lex-symbol-start? char t)
(right-char)
- (clj-lex-token :tag (concat "#" (clj-lex-get-symbol-at-point (1+
pos))) pos))
+ (parseclj-lex-token :tag (concat "#"
(parseclj-lex-get-symbol-at-point (1+ pos))) pos))
(t
- (while (not (or (clj-lex-at-whitespace?)
- (clj-lex-at-eof?)))
+ (while (not (or (parseclj-lex-at-whitespace?)
+ (parseclj-lex-at-eof?)))
(right-char))
- (clj-lex-token :lex-error (buffer-substring-no-properties pos
(point)) pos 'error-type :invalid-hashtag-dispatcher)))))
+ (parseclj-lex-token :lex-error (buffer-substring-no-properties pos
(point)) pos 'error-type :invalid-hashtag-dispatcher)))))
(t
(concat ":(" (char-to-string char)))))))
-(provide 'clj-lex)
+(provide 'parseclj-lex)
-;;; clj-lex.el ends here
+;;; parseclj-lex.el ends here
diff --git a/parseclj.el b/parseclj.el
index fe3dcb0b5a..d15ea4c00b 100644
--- a/parseclj.el
+++ b/parseclj.el
@@ -33,7 +33,7 @@
(require 'cl-lib)
(require 'a)
-(require 'clj-lex)
+(require 'parseclj-lex)
(require 'clj-edn)
(require 'clj-ast)
@@ -58,8 +58,8 @@
(member (a-get node ':node-type) parseclj--leaf-tokens))
(defun parseclj--is-open-prefix? (el)
- (and (member (clj-lex-token-type el) '(:discard :tag))
- (clj-lex-token? el)))
+ (and (member (parseclj-lex-token-type el) '(:discard :tag))
+ (parseclj-lex-token? el)))
;; The EDN spec is not clear about wether \u0123 and \o012 are supported in
;; strings. They are described as character literals, but not as string escape
@@ -101,7 +101,7 @@
(t first-char))))
(defun parseclj--leaf-token-value (token)
- (cl-case (clj-lex-token-type token)
+ (cl-case (parseclj-lex-token-type token)
(:number (string-to-number (alist-get 'form token)))
(:nil nil)
(:true t)
@@ -115,21 +115,21 @@
;;; Shift-Reduce Parser
(defun parseclj--find-opener (stack closer-token)
- (cl-case (clj-lex-token-type closer-token)
+ (cl-case (parseclj-lex-token-type closer-token)
(:rparen :lparen)
(:rbracket :lbracket)
- (:rbrace (clj-lex-token-type
- (seq-find (lambda (token) (member (clj-lex-token-type token)
'(:lbrace :set))) stack)))))
+ (:rbrace (parseclj-lex-token-type
+ (seq-find (lambda (token) (member (parseclj-lex-token-type
token) '(:lbrace :set))) stack)))))
(defun parseclj--reduce-coll (stack closer-token reduceN)
"Reduce collection based on the top of the stack"
(let ((opener-type (parseclj--find-opener stack closer-token))
(coll nil))
(while (and stack
- (not (eq (clj-lex-token-type (car stack)) opener-type)))
+ (not (eq (parseclj-lex-token-type (car stack)) opener-type)))
(push (pop stack) coll))
- (if (eq (clj-lex-token-type (car stack)) opener-type)
+ (if (eq (parseclj-lex-token-type (car stack)) opener-type)
(let ((node (pop stack)))
(funcall reduceN stack node coll))
;; Syntax error
@@ -140,12 +140,12 @@
(defun parseclj-parse (reduce-leaf reduce-branch)
(let ((stack nil))
- (while (not (eq (clj-lex-token-type (setq token (clj-lex-next))) :eof))
+ (while (not (eq (parseclj-lex-token-type (setq token (parseclj-lex-next)))
:eof))
;; (message "STACK: %S" stack)
;; (message "TOKEN: %S\n" token)
;; Reduce based on the top item on the stack (collections)
- (let ((token-type (clj-lex-token-type token)))
+ (let ((token-type (parseclj-lex-token-type token)))
(cond
((member token-type parseclj--leaf-tokens) (setf stack (funcall
reduce-leaf stack token)))
((member token-type parseclj--closer-tokens) (setf stack
(parseclj--reduce-coll stack token reduce-branch)))
@@ -154,7 +154,7 @@
;; Reduce based on top two items on the stack (special prefixed elements)
(seq-let [top lookup] stack
(when (and (parseclj--is-open-prefix? lookup)
- (not (clj-lex-token? top))) ;; top is fully reduced
+ (not (parseclj-lex-token? top))) ;; top is fully reduced
(setf stack (funcall reduce-branch (cddr stack) lookup (list
top))))))
;; reduce root
diff --git a/test/clj-lex-test.el b/test/clj-lex-test.el
deleted file mode 100644
index ae8325b741..0000000000
--- a/test/clj-lex-test.el
+++ /dev/null
@@ -1,295 +0,0 @@
-;;; clj-lex-test.el --- Unit tests for the lexer
-
-;; Copyright (C) 2017 Arne Brasseur
-
-;; Author: Arne Brasseur <arne@arnebrasseur.net>
-
-;; This file is not part of GNU Emacs.
-
-;; This file is free software; you can redistribute it and/or modify
-;; it under the terms of the GNU General Public License as published by
-;; the Free Software Foundation; either version 3, or (at your option)
-;; any later version.
-
-;; This file is distributed in the hope that it will be useful,
-;; but WITHOUT ANY WARRANTY; without even the implied warranty of
-;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-;; GNU General Public License for more details.
-
-;; You should have received a copy of the GNU General Public License
-;; along with GNU Emacs; see the file COPYING. If not, write to
-;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
-;; Boston, MA 02110-1301, USA.
-
-;;; Commentary
-
-;; Unit tests for the lexer
-
-;;; Code
-
-(require 'ert)
-(require 'clj-lex)
-
-(ert-deftest clj-lex-test-next ()
- (with-temp-buffer
- (insert "()")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :lparen) (form . "(") (pos . 1))))
- (should (equal (clj-lex-next) '((type . :rparen) (form . ")") (pos . 2))))
- (should (equal (clj-lex-next) '((type . :eof) (form . nil) (pos . 3)))))
-
- (with-temp-buffer
- (insert "123")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :number)
- (form . "123")
- (pos . 1)))))
-
- (with-temp-buffer
- (insert "123e34M")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :number)
- (form . "123e34M")
- (pos . 1)))))
-
- (with-temp-buffer
- (insert "123x")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :lex-error "123x" 1
'error-type :invalid-number-format))))
-
- (with-temp-buffer
- (insert " \t \n")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :whitespace) (form . " \t \n")
(pos . 1)))))
-
- (with-temp-buffer
- (insert "nil")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :nil) (form . "nil") (pos . 1)))))
-
- (with-temp-buffer
- (insert "true")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :true) (form . "true") (pos .
1)))))
-
- (with-temp-buffer
- (insert "false")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :false) (form . "false") (pos .
1)))))
-
- (with-temp-buffer
- (insert "hello-world")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :symbol) (form . "hello-world")
(pos . 1)))))
-
- (with-temp-buffer
- (insert "-hello-world")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :symbol) (form . "-hello-world")
(pos . 1)))))
-
- (with-temp-buffer
- (insert "foo#")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :symbol) (form . "foo#") (pos .
1)))))
-
- (with-temp-buffer
- (insert "#inst")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :tag) (form . "#inst") (pos .
1)))))
-
- (with-temp-buffer
- (insert "#qualified/tag")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :tag) (form . "#qualified/tag")
(pos . 1)))))
-
- (with-temp-buffer
- (insert "\\newline\\return\\space\\tab\\a\\b\\c")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :character "\\newline" 1)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\return" 9)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\space" 16)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\tab" 22)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\a" 26)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\b" 28)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\c" 30))))
-
- (with-temp-buffer
- (insert "\\newline\\return\\space\\tab\\a\\b\\c")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :character "\\newline" 1)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\return" 9)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\space" 16)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\tab" 22)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\a" 26)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\b" 28)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\c" 30))))
-
- (with-temp-buffer
- (insert "\\u0078\\o170")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :character "\\u0078" 1)))
- (should (equal (clj-lex-next) (clj-lex-token :character "\\o170" 7))))
-
- (with-temp-buffer
- (insert "\"\\u0078\\o170\"")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :string "\"\\u0078\\o170\""
1))))
-
- (with-temp-buffer
- (insert ":hello-world")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :keyword ":hello-world" 1))))
-
- (with-temp-buffer
- (insert ":hello/world")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :keyword ":hello/world" 1))))
-
- (with-temp-buffer
- (insert "::hello-world")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :keyword "::hello-world" 1))))
-
- (with-temp-buffer
- (insert ":::hello-world")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :lex-error ":::" 1
'error-type :invalid-keyword))))
-
- (with-temp-buffer
- (insert "[123]")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :lbracket "[" 1)))
- (should (equal (clj-lex-next) (clj-lex-token :number "123" 2)))
- (should (equal (clj-lex-next) (clj-lex-token :rbracket "]" 5))))
-
- (with-temp-buffer
- (insert "{:count 123}")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :lbrace "{" 1)))
- (should (equal (clj-lex-next) (clj-lex-token :keyword ":count" 2)))
- (should (equal (clj-lex-next) (clj-lex-token :whitespace " " 8)))
- (should (equal (clj-lex-next) (clj-lex-token :number "123" 9)))
- (should (equal (clj-lex-next) (clj-lex-token :rbrace "}" 12))))
-
- (with-temp-buffer
- (insert "#{:x}")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :set "#{" 1)))
- (should (equal (clj-lex-next) (clj-lex-token :keyword ":x" 3)))
- (should (equal (clj-lex-next) (clj-lex-token :rbrace "}" 5))))
-
- (with-temp-buffer
- (insert "(10 #_11 12 #_#_ 13 14)")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :lparen "(" 1)))
- (should (equal (clj-lex-next) (clj-lex-token :number "10" 2)))
- (should (equal (clj-lex-next) (clj-lex-token :whitespace " " 4)))
- (should (equal (clj-lex-next) (clj-lex-token :discard "#_" 5)))
- (should (equal (clj-lex-next) (clj-lex-token :number "11" 7)))
- (should (equal (clj-lex-next) (clj-lex-token :whitespace " " 9)))
- (should (equal (clj-lex-next) (clj-lex-token :number "12" 10)))
- (should (equal (clj-lex-next) (clj-lex-token :whitespace " " 12)))
- (should (equal (clj-lex-next) (clj-lex-token :discard "#_" 13)))
- (should (equal (clj-lex-next) (clj-lex-token :discard "#_" 15)))
- (should (equal (clj-lex-next) (clj-lex-token :whitespace " " 17)))
- (should (equal (clj-lex-next) (clj-lex-token :number "13" 18)))
- (should (equal (clj-lex-next) (clj-lex-token :whitespace " " 20)))
- (should (equal (clj-lex-next) (clj-lex-token :number "14" 21)))
- (should (equal (clj-lex-next) (clj-lex-token :rparen ")" 23)))))
-
-(ert-deftest clj-lex-test-at-number? ()
- (dolist (str '("123" ".9" "+1" "0" "-456"))
- (with-temp-buffer
- (insert str)
- (goto-char 1)
- (should (equal (clj-lex-at-number?) t))))
-
- (dolist (str '("a123" "$.9" "+/1" "++0" "-"))
- (with-temp-buffer
- (insert str)
- (goto-char 1)
- (should (equal (clj-lex-at-number?) nil)))))
-
-(ert-deftest clj-lex-test-token ()
- (should (equal (clj-lex-token :whitespace ",,," 10)
- '((type . :whitespace)
- (form . ",,,")
- (pos . 10)))))
-
-(ert-deftest clj-lex-test-digit? ()
- (should (equal (clj-lex-digit? ?0) t))
- (should (equal (clj-lex-digit? ?5) t))
- (should (equal (clj-lex-digit? ?9) t))
- (should (equal (clj-lex-digit? ?a) nil))
- (should (equal (clj-lex-digit? ?-) nil)))
-
-(ert-deftest clj-lex-test-symbol-start? ()
- (should (equal (clj-lex-symbol-start? ?0) nil))
- (should (equal (clj-lex-symbol-start? ?a) t))
- (should (equal (clj-lex-symbol-start? ?A) t))
- (should (equal (clj-lex-symbol-start? ?.) t))
- (should (equal (clj-lex-symbol-start? ?. t) nil))
- (should (equal (clj-lex-symbol-start? ?~) nil))
- (should (equal (clj-lex-symbol-start? ? ) nil)))
-
-(ert-deftest clj-lex-test-symbol-rest? ()
- (should (equal (clj-lex-symbol-rest? ?0) t))
- (should (equal (clj-lex-symbol-rest? ?a) t))
- (should (equal (clj-lex-symbol-rest? ?A) t))
- (should (equal (clj-lex-symbol-rest? ?.) t))
- (should (equal (clj-lex-symbol-rest? ?~) nil))
- (should (equal (clj-lex-symbol-rest? ? ) nil)))
-
-(ert-deftest clj-lex-test-get-symbol-at-point ()
- (with-temp-buffer
- (insert "a-symbol")
- (goto-char 1)
- (should (equal (clj-lex-get-symbol-at-point 1) "a-symbol"))
- (should (equal (point) 9))))
-
-(ert-deftest clj-lex-test-invalid-tag ()
- (with-temp-buffer
- (insert "#.not-a-tag")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :lex-error) (form . "#.not-a-tag")
(pos . 1) (error-type . :invalid-hashtag-dispatcher)))))
-
- (with-temp-buffer
- (insert "#-not-a-tag")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :lex-error) (form . "#-not-a-tag")
(pos . 1) (error-type . :invalid-hashtag-dispatcher)))))
-
- (with-temp-buffer
- (insert "#+not-a-tag")
- (goto-char 1)
- (should (equal (clj-lex-next) '((type . :lex-error) (form . "#+not-a-tag")
(pos . 1) (error-type . :invalid-hashtag-dispatcher))))))
-
-(ert-deftest clj-lex-test-string ()
- (with-temp-buffer
- (insert "\"abc\"")
- (goto-char 1)
- (should (equal (clj-lex-string) (clj-lex-token :string "\"abc\"" 1))))
-
- (with-temp-buffer
- (insert "\"abc")
- (goto-char 1)
- (should (equal (clj-lex-string) (clj-lex-token :lex-error "\"abc" 1))))
-
- (with-temp-buffer
- (insert "\"abc\\\"\"")"abc\""
- (goto-char 1)
- (should (equal (clj-lex-string) (clj-lex-token :string "\"abc\\\"\"" 1)))))
-
-(ert-deftest clj-lex-test-tag ()
- (with-temp-buffer
- (insert "#inst")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :tag "#inst" 1))))
-
- (with-temp-buffer
- (insert "#foo/bar")
- (goto-char 1)
- (should (equal (clj-lex-next) (clj-lex-token :tag "#foo/bar" 1)))))
-
-(provide 'clj-lex-test)
-
-;;; clj-lex-test.el ends here
diff --git a/test/parseclj-lex-test.el b/test/parseclj-lex-test.el
new file mode 100644
index 0000000000..56e5c195cb
--- /dev/null
+++ b/test/parseclj-lex-test.el
@@ -0,0 +1,295 @@
+;;; parseclj-lex-test.el --- Unit tests for the lexer
+
+;; Copyright (C) 2017 Arne Brasseur
+
+;; Author: Arne Brasseur <arne@arnebrasseur.net>
+
+;; This file is not part of GNU Emacs.
+
+;; This file is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3, or (at your option)
+;; any later version.
+
+;; This file is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs; see the file COPYING. If not, write to
+;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+;; Boston, MA 02110-1301, USA.
+
+;;; Commentary
+
+;; Unit tests for the lexer
+
+;;; Code
+
+(require 'ert)
+(require 'parseclj-lex)
+
+(ert-deftest parseclj-lex-test-next ()
+ (with-temp-buffer
+ (insert "()")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :lparen) (form . "(") (pos .
1))))
+ (should (equal (parseclj-lex-next) '((type . :rparen) (form . ")") (pos .
2))))
+ (should (equal (parseclj-lex-next) '((type . :eof) (form . nil) (pos .
3)))))
+
+ (with-temp-buffer
+ (insert "123")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :number)
+ (form . "123")
+ (pos . 1)))))
+
+ (with-temp-buffer
+ (insert "123e34M")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :number)
+ (form . "123e34M")
+ (pos . 1)))))
+
+ (with-temp-buffer
+ (insert "123x")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :lex-error "123x" 1
'error-type :invalid-number-format))))
+
+ (with-temp-buffer
+ (insert " \t \n")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :whitespace) (form . " \t
\n") (pos . 1)))))
+
+ (with-temp-buffer
+ (insert "nil")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :nil) (form . "nil") (pos .
1)))))
+
+ (with-temp-buffer
+ (insert "true")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :true) (form . "true") (pos .
1)))))
+
+ (with-temp-buffer
+ (insert "false")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :false) (form . "false") (pos
. 1)))))
+
+ (with-temp-buffer
+ (insert "hello-world")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :symbol) (form .
"hello-world") (pos . 1)))))
+
+ (with-temp-buffer
+ (insert "-hello-world")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :symbol) (form .
"-hello-world") (pos . 1)))))
+
+ (with-temp-buffer
+ (insert "foo#")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :symbol) (form . "foo#") (pos
. 1)))))
+
+ (with-temp-buffer
+ (insert "#inst")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :tag) (form . "#inst") (pos .
1)))))
+
+ (with-temp-buffer
+ (insert "#qualified/tag")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :tag) (form .
"#qualified/tag") (pos . 1)))))
+
+ (with-temp-buffer
+ (insert "\\newline\\return\\space\\tab\\a\\b\\c")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character
"\\newline" 1)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character
"\\return" 9)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character
"\\space" 16)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\tab"
22)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\a"
26)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\b"
28)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\c"
30))))
+
+ (with-temp-buffer
+ (insert "\\newline\\return\\space\\tab\\a\\b\\c")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character
"\\newline" 1)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character
"\\return" 9)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character
"\\space" 16)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\tab"
22)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\a"
26)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\b"
28)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\c"
30))))
+
+ (with-temp-buffer
+ (insert "\\u0078\\o170")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character
"\\u0078" 1)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :character "\\o170"
7))))
+
+ (with-temp-buffer
+ (insert "\"\\u0078\\o170\"")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :string
"\"\\u0078\\o170\"" 1))))
+
+ (with-temp-buffer
+ (insert ":hello-world")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :keyword
":hello-world" 1))))
+
+ (with-temp-buffer
+ (insert ":hello/world")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :keyword
":hello/world" 1))))
+
+ (with-temp-buffer
+ (insert "::hello-world")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :keyword
"::hello-world" 1))))
+
+ (with-temp-buffer
+ (insert ":::hello-world")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :lex-error ":::" 1
'error-type :invalid-keyword))))
+
+ (with-temp-buffer
+ (insert "[123]")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :lbracket "[" 1)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :number "123" 2)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :rbracket "]" 5))))
+
+ (with-temp-buffer
+ (insert "{:count 123}")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :lbrace "{" 1)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :keyword ":count"
2)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :whitespace " " 8)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :number "123" 9)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :rbrace "}" 12))))
+
+ (with-temp-buffer
+ (insert "#{:x}")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :set "#{" 1)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :keyword ":x" 3)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :rbrace "}" 5))))
+
+ (with-temp-buffer
+ (insert "(10 #_11 12 #_#_ 13 14)")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :lparen "(" 1)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :number "10" 2)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :whitespace " " 4)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :discard "#_" 5)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :number "11" 7)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :whitespace " " 9)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :number "12" 10)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :whitespace " "
12)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :discard "#_" 13)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :discard "#_" 15)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :whitespace " "
17)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :number "13" 18)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :whitespace " "
20)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :number "14" 21)))
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :rparen ")" 23)))))
+
+(ert-deftest parseclj-lex-test-at-number? ()
+ (dolist (str '("123" ".9" "+1" "0" "-456"))
+ (with-temp-buffer
+ (insert str)
+ (goto-char 1)
+ (should (equal (parseclj-lex-at-number?) t))))
+
+ (dolist (str '("a123" "$.9" "+/1" "++0" "-"))
+ (with-temp-buffer
+ (insert str)
+ (goto-char 1)
+ (should (equal (parseclj-lex-at-number?) nil)))))
+
+(ert-deftest parseclj-lex-test-token ()
+ (should (equal (parseclj-lex-token :whitespace ",,," 10)
+ '((type . :whitespace)
+ (form . ",,,")
+ (pos . 10)))))
+
+(ert-deftest parseclj-lex-test-digit? ()
+ (should (equal (parseclj-lex-digit? ?0) t))
+ (should (equal (parseclj-lex-digit? ?5) t))
+ (should (equal (parseclj-lex-digit? ?9) t))
+ (should (equal (parseclj-lex-digit? ?a) nil))
+ (should (equal (parseclj-lex-digit? ?-) nil)))
+
+(ert-deftest parseclj-lex-test-symbol-start? ()
+ (should (equal (parseclj-lex-symbol-start? ?0) nil))
+ (should (equal (parseclj-lex-symbol-start? ?a) t))
+ (should (equal (parseclj-lex-symbol-start? ?A) t))
+ (should (equal (parseclj-lex-symbol-start? ?.) t))
+ (should (equal (parseclj-lex-symbol-start? ?. t) nil))
+ (should (equal (parseclj-lex-symbol-start? ?~) nil))
+ (should (equal (parseclj-lex-symbol-start? ? ) nil)))
+
+(ert-deftest parseclj-lex-test-symbol-rest? ()
+ (should (equal (parseclj-lex-symbol-rest? ?0) t))
+ (should (equal (parseclj-lex-symbol-rest? ?a) t))
+ (should (equal (parseclj-lex-symbol-rest? ?A) t))
+ (should (equal (parseclj-lex-symbol-rest? ?.) t))
+ (should (equal (parseclj-lex-symbol-rest? ?~) nil))
+ (should (equal (parseclj-lex-symbol-rest? ? ) nil)))
+
+(ert-deftest parseclj-lex-test-get-symbol-at-point ()
+ (with-temp-buffer
+ (insert "a-symbol")
+ (goto-char 1)
+ (should (equal (parseclj-lex-get-symbol-at-point 1) "a-symbol"))
+ (should (equal (point) 9))))
+
+(ert-deftest parseclj-lex-test-invalid-tag ()
+ (with-temp-buffer
+ (insert "#.not-a-tag")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :lex-error) (form .
"#.not-a-tag") (pos . 1) (error-type . :invalid-hashtag-dispatcher)))))
+
+ (with-temp-buffer
+ (insert "#-not-a-tag")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :lex-error) (form .
"#-not-a-tag") (pos . 1) (error-type . :invalid-hashtag-dispatcher)))))
+
+ (with-temp-buffer
+ (insert "#+not-a-tag")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) '((type . :lex-error) (form .
"#+not-a-tag") (pos . 1) (error-type . :invalid-hashtag-dispatcher))))))
+
+(ert-deftest parseclj-lex-test-string ()
+ (with-temp-buffer
+ (insert "\"abc\"")
+ (goto-char 1)
+ (should (equal (parseclj-lex-string) (parseclj-lex-token :string "\"abc\""
1))))
+
+ (with-temp-buffer
+ (insert "\"abc")
+ (goto-char 1)
+ (should (equal (parseclj-lex-string) (parseclj-lex-token :lex-error
"\"abc" 1))))
+
+ (with-temp-buffer
+ (insert "\"abc\\\"\"")"abc\""
+ (goto-char 1)
+ (should (equal (parseclj-lex-string) (parseclj-lex-token :string
"\"abc\\\"\"" 1)))))
+
+(ert-deftest parseclj-lex-test-tag ()
+ (with-temp-buffer
+ (insert "#inst")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :tag "#inst" 1))))
+
+ (with-temp-buffer
+ (insert "#foo/bar")
+ (goto-char 1)
+ (should (equal (parseclj-lex-next) (parseclj-lex-token :tag "#foo/bar"
1)))))
+
+(provide 'parseclj-lex-test)
+
+;;; parseclj-lex-test.el ends here
- [nongnu] elpa/parseclj 61513d2c99 164/185: Merge pull request #27 from clojure-emacs/shebang-and-symbolic-values, (continued)
- [nongnu] elpa/parseclj 61513d2c99 164/185: Merge pull request #27 from clojure-emacs/shebang-and-symbolic-values, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj e6bce85062 159/185: Update the CHANGELOG, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 1f7fe675ae 172/185: Merge pull request #31 from clojure-emacs/remove-a-el-part-2, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj a8c4cf30fb 185/185: Correctly bump versions, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 7ae887b1de 037/185: Refactor clj-parse.el, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj a83816010e 039/185: Refactor clj-parse-test., ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 9d4c0b7b4b 041/185: Merge pull request #2 from volrath/tag-support, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 879ac980a8 056/185: Treat numbers with trailing symbol characters as lex errors, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 2781f0cd7f 088/185: Vocab chage: closer/opener => closing-token/opening-token, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 4fc37462ab 073/185: Make Travis use Cask, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj a71e57df4d 080/185: Rename clj-lex to parseclj-lex,
ELPA Syncer <=
- [nongnu] elpa/parseclj 168027fed5 094/185: Merge pull request #7 from lambdaisland/reorganize-package, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 45bd6a7431 109/185: Clean up tests, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 1f8e449897 105/185: Simplify error messages, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj f362018ff1 112/185: Silence the byte-compiler about some unused vars., ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 70804992ee 122/185: Merge pull request #14 from lambdaisland/parseclj-lex-symbol-fix, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 19ca5f5bd0 121/185: Stick to the previous Travis Trusty image, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj b2550e6456 133/185: Add docstring for `parseclj-lex--string-value`, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 13059d8529 138/185: Minor checkdoc fixes, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj 49c715ed22 148/185: For prefix-2 elements: pass children to reduce in right order, ELPA Syncer, 2021/12/28
- [nongnu] elpa/parseclj dc0d165b0a 152/185: Merge pull request #23 from clojure-emacs/read-one, ELPA Syncer, 2021/12/28