Rename `libsyntax` to `librustc_ast`
This was the last rustc crate that wasn't following the `rustc_*` naming convention.
Follow-up to https://github.com/rust-lang/rust/pull/67763.
parser: `token` -> `normalized_token`, `nonnormalized_token` -> `token`
So, after https://github.com/rust-lang/rust/pull/69006, its follow-ups and an attempt to remove `Parser::prev_span` I came to the conclusion that the unnormalized token and its span is what you want in most cases, so it should be default.
Normalization only makes difference in few cases where we are checking against `token::Ident` or `token::Lifetime` specifically.
This PR uses `normalized_token` for those cases.
Using normalization explicitly means that people writing code should remember about `NtIdent` and `NtLifetime` in general. (That is alleviated by the fact that `token.ident()` and `fn parse_ident_*` are already written.)
Remembering about `NtIdent`, was, however, already the case, kind of, because the implicit normalization was performed only for the current/previous token, but not for things like `look_ahead`.
As a result, most of token classification methods in `token.rs` already take `NtIdent` into account (this PR fixes a few pre-existing minor mistakes though).
The next step is removing `normalized(_prev)_token` entirely and replacing it with `token.ident()` (mostly) and `token.normalize()` (occasionally).
I want to make it a separate PR for that and run it though perf.
`normalized_token` filled on every bump has both a potential to avoid repeated normalization, and to do unnecessary work in advance (it probably doesn't matter anyway, the normalization is very cheap).
r? @Centril
parser: Simplify treatment of macro variables in `Parser::bump`
Follow-up to https://github.com/rust-lang/rust/pull/69006.
Token normalization for `$ident` and `$lifetime` is merged directly into `bump`.
Special "unknown macro variable" diagnostic for unexpected `$`s is removed as preventing legal code from compiling (as a result `bump` also doesn't call itself recursively anymore and can't make `prev_token` inconsistent).
r? @Centril
parse: fuse associated and extern items up to defaultness
Language changes:
- The grammar of extern `type` aliases is unified with associated ones, and becomes:
```rust
TypeItem = "type" ident generics {":" bounds}? where_clause {"=" type}? ";" ;
```
Semantic restrictions (`ast_validation`) are added to forbid any parameters in `generics`, any bounds in `bounds`, and any predicates in `where_clause`, as well as the presence of a type expression (`= u8`).
(Work still remains to fuse this with free `type` aliases, but this can be done later.)
- The grammar of constants and static items (free, associated, and extern) now permits the absence of an expression, and becomes:
```rust
GlobalItem = {"const" {ident | "_"} | "static" "mut"? ident} {"=" expr}? ";" ;
```
- A semantic restriction is added to enforce the presence of the expression (the body).
- A semantic restriction is added to reject `const _` in associated contexts.
Together, these changes allow us to fuse the grammar of associated items and extern items up to `default`ness which is the main goal of the PR.
-----------------------
We are now very close to fully fusing the entirely of item parsing and their ASTs. To progress further, we must make a decision: should we parse e.g. `default use foo::bar;` and whatnot? Accepting that is likely easiest from a parsing perspective, as it does not require using look-ahead, but it is perhaps not too onerous to only accept it for `fn`s (and all their various qualifiers), `const`s, `static`s, and `type`s.
r? @petrochenkov
