Remove token::{Open,Close}Delim.

By replacing them with `{Open,Close}{Param,Brace,Bracket,Invisible}`.

PR #137902 made `ast::TokenKind` more like `lexer::TokenKind` by
replacing the compound `BinOp{,Eq}(BinOpToken)` variants with fieldless
variants `Plus`, `Minus`, `Star`, etc. This commit does a similar thing
with delimiters. It also makes `ast::TokenKind` more similar to
`parser::TokenType`.

This requires a few new methods:
- `TokenKind::is_{,open_,close_}delim()` replace various kinds of
  pattern matches.
- `Delimiter::as_{open,close}_token_kind` are used to convert
  `Delimiter` values to `TokenKind`.

Despite these additions, it's a net reduction in lines of code. This is
because e.g. `token::OpenParen` is so much shorter than
`token::OpenDelim(Delimiter::Parenthesis)` that many multi-line forms
reduce to single line forms. And many places where the number of lines
doesn't change are still easier to read, just because the names are
shorter, e.g.:
```
-   } else if self.token != token::CloseDelim(Delimiter::Brace) {
+   } else if self.token != token::CloseBrace {
```
This commit is contained in:
Nicholas Nethercote
2025-04-16 16:13:50 +10:00
parent a15cce2690
commit bf8ce32558
30 changed files with 456 additions and 498 deletions

View File

@@ -1,7 +1,7 @@
use rustc_ast::ptr::P;
use rustc_ast::token::NtExprKind::*;
use rustc_ast::token::NtPatKind::*;
use rustc_ast::token::{self, Delimiter, InvisibleOrigin, MetaVarKind, NonterminalKind, Token};
use rustc_ast::token::{self, InvisibleOrigin, MetaVarKind, NonterminalKind, Token};
use rustc_errors::PResult;
use rustc_span::{Ident, kw};
@@ -69,13 +69,13 @@ impl<'a> Parser<'a> {
| token::Ident(..)
| token::NtIdent(..)
| token::NtLifetime(..)
| token::OpenDelim(Delimiter::Invisible(InvisibleOrigin::MetaVar(_))) => true,
| token::OpenInvisible(InvisibleOrigin::MetaVar(_)) => true,
_ => token.can_begin_type(),
},
NonterminalKind::Block => match &token.kind {
token::OpenDelim(Delimiter::Brace) => true,
token::OpenBrace => true,
token::NtLifetime(..) => true,
token::OpenDelim(Delimiter::Invisible(InvisibleOrigin::MetaVar(k))) => match k {
token::OpenInvisible(InvisibleOrigin::MetaVar(k)) => match k {
MetaVarKind::Block
| MetaVarKind::Stmt
| MetaVarKind::Expr { .. }
@@ -94,9 +94,7 @@ impl<'a> Parser<'a> {
},
NonterminalKind::Path | NonterminalKind::Meta => match &token.kind {
token::PathSep | token::Ident(..) | token::NtIdent(..) => true,
token::OpenDelim(Delimiter::Invisible(InvisibleOrigin::MetaVar(kind))) => {
may_be_ident(*kind)
}
token::OpenInvisible(InvisibleOrigin::MetaVar(kind)) => may_be_ident(*kind),
_ => false,
},
NonterminalKind::Pat(pat_kind) => token.can_begin_pattern(pat_kind),
@@ -105,7 +103,7 @@ impl<'a> Parser<'a> {
_ => false,
},
NonterminalKind::TT | NonterminalKind::Item | NonterminalKind::Stmt => {
!matches!(token.kind, token::CloseDelim(_))
token.kind.close_delim().is_none()
}
}
}