Make TokenStream less recursive.
`TokenStream` is currently recursive in *two* ways: - the `TokenTree` variant contains a `ThinTokenStream`, which can contain a `TokenStream`; - the `TokenStream` variant contains a `Vec<TokenStream>`. The latter is not necessary and causes significant complexity. This commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`. This reduces complexity significantly. In particular, `StreamCursor` is eliminated, and `Cursor` becomes much simpler, consisting now of just a `TokenStream` and an index. The commit also removes the `Extend` impl for `TokenStream`, because it is only used in tests. (The commit also removes those tests.) Overall, the commit reduces the number of lines of code by almost 200.
This commit is contained in:
@@ -2914,7 +2914,7 @@ impl<'a> Parser<'a> {
|
||||
TokenTree::Delimited(
|
||||
frame.span,
|
||||
frame.delim,
|
||||
frame.tree_cursor.original_stream().into(),
|
||||
frame.tree_cursor.stream.into(),
|
||||
)
|
||||
},
|
||||
token::CloseDelim(_) | token::Eof => unreachable!(),
|
||||
|
||||
Reference in New Issue
Block a user