Introduce rustc_lexer
The idea here is to make a reusable library out of the existing rust-lexer, by separating out pure lexing and rustc-specific concerns, like spans, error reporting an interning. So, rustc_lexer operates directly on `&str`, produces simple tokens which are a pair of type-tag and a bit of original text, and does not report errors, instead storing them as flags on the token.
This commit is contained in:
@@ -53,10 +53,10 @@ LL | 0o;
|
||||
| ^^
|
||||
|
||||
error: expected at least one digit in exponent
|
||||
--> $DIR/lex-bad-numeric-literals.rs:12:8
|
||||
--> $DIR/lex-bad-numeric-literals.rs:12:5
|
||||
|
|
||||
LL | 1e+;
|
||||
| ^
|
||||
| ^^^
|
||||
|
||||
error: hexadecimal float literal is not supported
|
||||
--> $DIR/lex-bad-numeric-literals.rs:13:5
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
error: unterminated raw string
|
||||
--> $DIR/raw-byte-string-eof.rs:2:6
|
||||
--> $DIR/raw-byte-string-eof.rs:2:5
|
||||
|
|
||||
LL | br##"a"#;
|
||||
| ^ unterminated raw string
|
||||
| ^ unterminated raw string
|
||||
|
|
||||
= note: this raw string should be terminated with `"##`
|
||||
|
||||
|
||||
@@ -11,10 +11,10 @@ LL | br"é";
|
||||
LL | br"é";
|
||||
| ^
|
||||
|
||||
error: found invalid character; only `#` is allowed in raw string delimitation: ~
|
||||
error: found invalid character; only `#` is allowed in raw string delimitation: ~
|
||||
--> $DIR/raw-byte-string-literals.rs:6:5
|
||||
|
|
||||
LL | br##~"a"~##;
|
||||
LL | br##~"a"~##;
|
||||
| ^^^^^
|
||||
|
||||
error: aborting due to 3 previous errors
|
||||
|
||||
@@ -2,7 +2,7 @@ error: found invalid character; only `#` is allowed in raw string delimitation:
|
||||
--> $DIR/raw-str-delim.rs:2:5
|
||||
|
|
||||
LL | r#~"#"~#
|
||||
| ^^
|
||||
| ^^^
|
||||
|
||||
error: aborting due to previous error
|
||||
|
||||
|
||||
Reference in New Issue
Block a user