Simplify the macro hygiene algorithm
This PR removes renaming from the hygiene algorithm and treats differently marked identifiers as unequal.
This change makes the scope of identifiers in `macro_rules!` items empty. That is, identifiers in `macro_rules!` definitions do not inherit any semantics from the `macro_rules!`'s scope.
Since `macro_rules!` macros are items, the scope of their identifiers "should" be the same as that of other items; in particular, the scope should contain only items. Since all items are unhygienic today, this would mean the scope should be empty.
However, the scope of an identifier in a `macro_rules!` statement today is the scope that the identifier would have if it replaced the `macro_rules!` (excluding anything unhygienic, i.e. locals only).
To continue to support this, this PR tracks the scope of each `macro_rules!` and uses it in `resolve` to ensure that an identifier expanded from a `macro_rules!` gets a chance to resolve to the locals in the `macro_rules!`'s scope.
This PR is a pure refactoring. After this PR,
- `syntax::ext::expand` is much simpler.
- We can expand macros in any order without causing problems for hygiene (needed for macro modularization).
- We can deprecate or remove today's `macro_rules!` scope easily.
- Expansion performance improves by 25%, post-expansion memory usage decreases by ~5%.
- Expanding a block is no longer quadratic in the number of `let` statements (fixes#10607).
r? @nrc
Fix expansion performance regression
**syntax-[breaking-change] cc #31645**
This fixes#34630 by reverting commit 5bf7970 of PR #33943, which landed in #34424.
By removing the `Rc<_>` wrapping around `Delimited` and `SequenceRepetition` in `TokenTree`, 5bf7970 made cloning `TokenTree`s more expensive. While this had no measurable performance impact on the compiler's crates, it caused an order of magnitude performance regression on some macro-heavy code in the wild. I believe this is due to clones of `TokenTree`s in `macro_parser.rs` and/or `macro_rules.rs`.
r? @nrc
Forbid type parameters and global paths in macro invocations
Fixes#28558.
This is a [breaking-change]. For example, the following would break:
```rust
macro_rules! m { () => { () } }
fn main() {
m::<T>!(); // Type parameters are no longer allowed in macro invocations
::m!(); // Global paths are no longer allowed in macro invocations
}
```
Any breakage can be fixed by removing the type parameters or the leading `::` (respectively).
r? @eddyb
To allow these braced macro invocation, this PR removes the optional expression from `ast::Block` and instead uses a `StmtKind::Expr` at the end of the statement list.
Currently, braced macro invocations in blocks can expand into statements (and items) except when they are last in a block, in which case they can only expand into expressions.
For example,
```rust
macro_rules! make_stmt {
() => { let x = 0; }
}
fn f() {
make_stmt! {} //< This is OK...
let x = 0; //< ... unless this line is commented out.
}
```
Fixes#34418.
syntax-[breaking-change] cc #31645
(Only breaking because ast::TokenTree is now tokenstream::TokenTree.)
This pull request refactors TokenTrees into their own file as src/libsyntax/tokenstream.rs, moving them out of src/libsyntax/ast.rs, in order to prepare for an accompanying TokenStream implementation (per RFC 1566).
This PR refactors the 'errors' part of libsyntax into its own crate (librustc_errors). This is the first part of a few refactorings to simplify error reporting and potentially support more output formats (like a standardized JSON output and possibly an --explain mode that can work with the user's code), though this PR stands on its own and doesn't assume further changes.
As part of separating out the errors crate, I have also refactored the code position portion of codemap into its own crate (libsyntax_pos). While it's helpful to have the common code positions in a separate crate for the new errors crate, this may also enable further simplifications in the future.
**syntax-[breaking-change]** cc #31645
New `TraitItemKind::Macro` variant
This change adds support for macro expansion inside trait items by adding the new `TraitItemKind::Macro` and associated parsing code.
Revert a change in the scope of macros imported from crates to fix a regression
Fixes#34212.
The regression was caused by #34032, which changed the scope of macros imported from extern crates to match the scope of macros imported from modules.
r? @nrc
