Remove token::{Open,Close}Delim.

By replacing them with `{Open,Close}{Param,Brace,Bracket,Invisible}`.

PR #137902 made `ast::TokenKind` more like `lexer::TokenKind` by
replacing the compound `BinOp{,Eq}(BinOpToken)` variants with fieldless
variants `Plus`, `Minus`, `Star`, etc. This commit does a similar thing
with delimiters. It also makes `ast::TokenKind` more similar to
`parser::TokenType`.

This requires a few new methods:
- `TokenKind::is_{,open_,close_}delim()` replace various kinds of
  pattern matches.
- `Delimiter::as_{open,close}_token_kind` are used to convert
  `Delimiter` values to `TokenKind`.

Despite these additions, it's a net reduction in lines of code. This is
because e.g. `token::OpenParen` is so much shorter than
`token::OpenDelim(Delimiter::Parenthesis)` that many multi-line forms
reduce to single line forms. And many places where the number of lines
doesn't change are still easier to read, just because the names are
shorter, e.g.:
```
-   } else if self.token != token::CloseDelim(Delimiter::Brace) {
+   } else if self.token != token::CloseBrace {
```
This commit is contained in:
Nicholas Nethercote
2025-04-16 16:13:50 +10:00
parent a15cce2690
commit bf8ce32558
30 changed files with 456 additions and 498 deletions

View File

@@ -1,4 +1,3 @@
use ast::token::Delimiter;
use rustc_ast::{
self as ast, AttrVec, DUMMY_NODE_ID, GenericBounds, GenericParam, GenericParamKind, TyKind,
WhereClause, token,
@@ -437,7 +436,7 @@ impl<'a> Parser<'a> {
if let Some(struct_) = struct_
&& self.may_recover()
&& self.token == token::OpenDelim(Delimiter::Parenthesis)
&& self.token == token::OpenParen
{
snapshot = Some((struct_, self.create_snapshot_for_diagnostic()));
};
@@ -548,7 +547,7 @@ impl<'a> Parser<'a> {
matches!(t.kind, token::Gt | token::Comma | token::Colon | token::Eq)
// Recovery-only branch -- this could be removed,
// since it only affects diagnostics currently.
|| matches!(t.kind, token::Question)
|| t.kind == token::Question
})
|| self.is_keyword_ahead(start + 1, &[kw::Const]))
}