Clean `rustc/parse/src/lexer` to improve maintainability
This PR refactors the lexer code to improve maintainability and eliminate code duplication.
In the first commit, I improve the error handling:
- rename `make_unclosed_delims_error` to more appropriate `make_mismatched_closing_delims_errors`
- changes return type from Option<Diag> to `Vec<Diag>` to avoid lengthy vec processing at `lex_token_trees`
- use `splice` instead of `extend` to make the logic clearer, since `errs` sounds more generic and better suited as a return value
In the second commit, I replace the magic number 5 with UNCLOSED_DELIMITER_SHOW_LIMIT constant.
In the third commit, I moves `eof_err` function below parsing logic for better code flow.
In the forth one, I extract `calculate_spacing` function to eliminate duplicate spacing logic between `bump` and `bump_minimal` functions.
r? compiler
In optimized builds GVN gets rid of these already, but in `opt-level=0` we actually make `alloca`s for this, which particularly impacts `?`-style things that use actually-only-one-variant types like this.
Implement AST visitors using a derive macro.
AST visitors are large and error-prone beasts. This PR attempts to write them using a derive macro.
The design uses three traits: `Visitor`, `Visitable`, `Walkable`.
- `Visitor` is the trait implemented by downstream crates, it lists `visit_stuff` methods, which call `Walkable::walk_ref` by default;
- `Walkable` is derived using the macro, the generated `walk_ref` method calls `Visitable::visit` on each component;
- `Visitable` is implemented by `common_visitor_and_walkers` macro, to call the proper `Visitor::visit_stuff` method if it exists, to call `Walkable::walk_ref` if there is none.
I agree this is quite a lot of spaghetti macros. I'm open to suggestions on how to reduce the amount of boilerplate code.
If this PR is accepted, I believe the same design can be used for the HIR visitor.
This function can cause false negatives if used incorrectly
(usually "do any of the doc fragments come from a macro" is
the wrong question to ask), and thus it is unused.
Certain LLVM intrinsics, such as `llvm.wasm.throw`, can unwind. Marking
them as nounwind causes us to skip cleanup of locals and optimize out
`catch_unwind` under inlining or when `llvm.wasm.throw` is used directly
by user code.
The motivation for forcibly marking llvm.* as nounwind is no longer
present: most intrinsics are linked as `extern "C"` or other
non-unwinding ABIs, so we won't codegen `invoke` for them anyway.
Rename `tests/{assembly,codegen}` into `tests/{assembly,codegen}-llvm` and ignore these testsuites if configured backend doesn't match
Follow-up of https://github.com/rust-lang/rust/pull/144125.
This PR changes `compiletest` so that `asm` tests are only run if they match the current codegen backend. To better reflect it, I renamed the `tests/ui/asm` folder into `tests/ui/asm-llvm`. Like that, we can add new asm tests for other backends if we want without needing to add extra code to `compiletest`.
Next step will be to use the new code annotations added in rust-lang/rust#144125 to ignore ui tests failing in cg_gcc until it's fixed on our side.
cc `@antoyo` `@oli-obk`
r? `@Kobzol`
Rather than adding `get_unused_rule` to the `TTMacroExpander` trait, put
it on the concrete `MacroRulesMacroExpander`, and downcast to that type
via `Any` in order to call it.
Suggested-by: Vadim Petrochenkov <vadim.petrochenkov@gmail.com>
The previous manual parsing of `serde_json::Value` was a lot of
complicated code and extremely error-prone. It was full of janky
behavior like sometimes ignoring type errors, sometimes erroring for
type errors, sometimes warning for type errors, and sometimes just
ICEing for type errors (the icing on the top).
Additionally, many of the error messages about allowed values were out
of date because they were in a completely different place than the
FromStr impls. Overall, the system caused confusion for users.
I also found the old deserialization code annoying to read. Whenever a
`key!` invocation was found, one had to first look for the right macro
arm, and no go to definition could help.
This PR replaces all this manual parsing with a 2-step process involving
serde.
First, the string is parsed into a `TargetSpecJson` struct. This struct
is a 1:1 representation of the spec JSON. It already parses all the
enums and is very simple to read and write.
Then, the fields from this struct are copied into the actual `Target`.
The reason for this two-step process instead of just serializing into a
`Target` is because of a few reasons
1. There are a few transformations performed between the two formats
2. The default logic is implemented this way. Otherwise all the default
field values would have to be spelled out again, which is
suboptimal. With this logic, they fall out naturally, because
everything in the json struct is an `Option`.
Overall, the mapping is pretty simple, with the vast majority of fields
just doing a 1:1 mapping that is captured by two macros. I have
deliberately avoided making the macros generic to keep them simple.
All the `FromStr` impls now have the error message right inside them,
which increases the chance of it being up to date. Some "`from_str`"
impls were turned into proper `FromStr` impls to support this.
The new code is much less involved, delegating all the JSON parsing
logic to serde, without any manual type matching.
This change introduces a few breaking changes for consumers. While it is
possible to use this format on stable, it is very much subject to
change, so breaking changes are expected. The hope is also that because
of the way stricter behavior, breaking changes are easier to deal with,
as they come with clearer error messages.
1. Invalid types now always error, everywhere. Previously, they would
sometimes error, and sometimes just be ignored (which meant the users
JSON was still broken, just silently!)
2. This now makes use of `deny_unknown_fields` instead of just warning
on unused fields, which was done previously. Serde doesn't make it
easy to get such warning behavior, which was the primary reason that
this now changed. But I think error behavior is very reasonable too.
If someone has random stale fields in their JSON, it is likely
because these fields did something at some point but no longer do,
and the user likely wants to be informed of this so they can figure
out what to do.
This is also relevant for the future. If we remove a field but
someone has it set, it probably makes sense for them to take a look
whether they need this and should look for alternatives, or whether
they can just delete it. Overall, the JSON is made more explicit.
This is the only expected breakage, but there could also be small
breakage from small mistakes. All targets roundtrip though, so it can't
be anything too major.
fix handling of base address for TypeId allocations
This fixes the problems discovered by ````@theemathas```` in https://github.com/rust-lang/rust/pull/142789:
- const-eval would sometimes consider TypeId pointers to be null
- the type ID is different in Miri than in regular executions
Both boil down to the same issue: the TypeId "allocation" has a guaranteed 0 base address, but const-eval assumes it was non-zero (like normal allocations) and Miri randomized it (like normal allocations).
r? ````@oli-obk````
Mitigate `#[align]` name resolution ambiguity regression with a rename
Mitigates beta regression rust-lang/rust#143834 after a beta backport.
### Background on the beta regression
The name resolution regression arises due to rust-lang/rust#142507 adding a new feature-gated built-in attribute named `#[align]`. However, unfortunately even [introducing new feature-gated unstable built-in attributes can break user code](https://www.github.com/rust-lang/rust/issues/134963) such as
```rs
macro_rules! align {
() => {
/* .. */
};
}
pub(crate) use align; // `use` here becomes ambiguous
```
### Mitigation approach
This PR renames `#[align]` to `#[rustc_align]` to mitigate the beta regression by:
1. Undoing the introduction of a new built-in attribute with a common name, i.e. `#[align]`.
2. Renaming `#[align]` to `#[rustc_align]`. The renamed attribute being `rustc_align` will not introduce new stable breakages, as attributes beginning with `rustc` are reserved and perma-unstable. This does mean existing nightly code using `fn_align` feature will additionally need to specify `#![feature(rustc_attrs)]`.
This PR is very much a short-term mitigation to alleviate time pressure from having to fully fix the current limitation of inevitable name resolution regressions that would arise from adding any built-in attributes. Long-term solutions are discussed in [#t-lang > namespacing macro attrs to reduce conflicts with new adds](https://rust-lang.zulipchat.com/#narrow/channel/213817-t-lang/topic/namespacing.20macro.20attrs.20to.20reduce.20conflicts.20with.20new.20adds/with/529249622).
### Alternative mitigation options
[Various mitigation options were considered during the compiler triage meeting](https://github.com/rust-lang/rust/issues/143834#issuecomment-3084415277), and those consideration are partly reproduced here:
- Reverting the PR doesn't seem very minimal/trivial, and carries risks of its own.
- Rename to a less-common but aim-to-stabilization name is itself not safe nor convenient, because (1) that risks introducing new regressions (i.e. ambiguity against the new name), and (2) lang would have to FCP the new name hastily for the mitigation to land timely and have a chance to be backported. This also makes the path towards stabilization annoying.
- Rename the attribute to a rustc attribute, which will be perma-unstable and does not cause new ambiguities in stable code.
- This alleviates the time pressure to address *this* regression, or for lang to have to rush an FCP for some new name that can still break user code.
- This avoids backing out a whole implementation.
### Review advice
This PR is best reviewed commit-by-commit.
- Commit 1 adds a test `tests/ui/attributes/fn-align-nameres-ambiguity-143834.rs` which demonstrates the current name resolution regression re. `align`. This test fails against current master.
- Commit 2 carries out the renames and test reblesses. Notably, commit 2 will cause `tests/ui/attributes/fn-align-nameres-ambiguity-143834.rs` to change from fail (nameres regression) to pass.
This PR, if the approach still seems acceptable, will need a beta-backport to address the beta regression.
rustc_public: de-StableMIR-ize
This PR updates relevant docs about StableMIR, basically just rewording StableMIR/SMIR to rustc_public/rustc_public's IR.
The README.md in the `rustc_public` crate is out-dated. I plan to rewrite it after we fork rustc_public into its own repository.
This PR doesn't change the fact that we still use `-Z unpretty=stable-mir` as a rustc parameter for printing the IR, since I feel it's a bit verbose and weird if we use `-Z unpretty=rustc-public-ir`. I was wondering if we can have a short and easy alias for rustc_public's IR.
gpu offload host code generation
r? ghost
This will generate most of the host side code to use llvm's offload feature.
The first PR will only handle automatic mem-transfers to and from the device.
So if a user calls a kernel, we will copy inputs back and forth, but we won't do the actual kernel launch.
Before merging, we will use LLVM's Info infrastructure to verify that the memcopies match what openmp offloa generates in C++. `LIBOMPTARGET_INFO=-1 ./my_rust_binary` should print that a memcpy to and later from the device is happening.
A follow-up PR will generate the actual device-side kernel which will then do computations on the GPU.
A third PR will implement manual host2device and device2host functionality, but the goal is to minimize cases where a user has to overwrite our default handling due to performance issues.
I'm trying to get a full MVP out first, so this just recognizes GPU functions based on magic names. The final frontend will obviously move this over to use proper macros, like I'm already doing it for the autodiff work.
This work will also be compatible with std::autodiff, so one can differentiate GPU kernels.
Tracking:
- https://github.com/rust-lang/rust/issues/131513
```
error[E0507]: cannot move out of `f`, a captured variable in an `FnMut` closure
--> $DIR/borrowck-call-is-borrow-issue-12224.rs:57:13
|
LL | let mut f = move |g: Box<dyn FnMut(isize)>, b: isize| {
| ----- captured outer variable
...
LL | f(Box::new(|a| {
| --- captured by this `FnMut` closure
LL |
LL | foo(f);
| ^ move occurs because `f` has type `{closure@$DIR/borrowck-call-is-borrow-issue-12224.rs:52:17: 52:58}`, which does not implement the `Copy` trait
```
instead of
```
error[E0507]: cannot move out of `f`, a captured variable in an `FnMut` closure
--> $DIR/borrowck-call-is-borrow-issue-12224.rs:57:13
|
LL | let mut f = move |g: Box<dyn FnMut(isize)>, b: isize| {
| _________-----___-
| | |
| | captured outer variable
LL | | let _ = s.len();
LL | | };
| |_____- move occurs because `f` has type `{closure@$DIR/borrowck-call-is-borrow-issue-12224.rs:52:17: 52:58}`, which does not implement the `Copy` trait
LL | f(Box::new(|a| {
| --- captured by this `FnMut` closure
LL |
LL | foo(f);
| ^ `f` is moved here
```
Account not only for `fn` parameters when moving non-`Copy` values into closure, but also for let bindings.
```
error[E0507]: cannot move out of `bar`, a captured variable in an `FnMut` closure
--> $DIR/borrowck-move-by-capture.rs:9:29
|
LL | let bar: Box<_> = Box::new(3);
| --- ------ move occurs because `bar` has type `Box<isize>`, which does not implement the `Copy` trait
| |
| captured outer variable
LL | let _g = to_fn_mut(|| {
| -- captured by this `FnMut` closure
LL | let _h = to_fn_once(move || -> isize { *bar });
| ^^^^^^^^^^^^^^^^ ---- variable moved due to use in closure
| |
| `bar` is moved here
|
help: consider cloning the value before moving it into the closure
|
LL ~ let value = bar.clone();
LL ~ let _h = to_fn_once(move || -> isize { value });
|
```
```
error[E0507]: cannot move out of `y`, a captured variable in an `Fn` closure
--> $DIR/unboxed-closures-move-upvar-from-non-once-ref-closure.rs:12:9
|
LL | let y = vec![format!("World")];
| - ---------------------- move occurs because `y` has type `Vec<String>`, which does not implement the `Copy` trait
| |
| captured outer variable
LL | call(|| {
| -- captured by this `Fn` closure
LL | y.into_iter();
| ^ ----------- `y` moved due to this method call
| |
| `y` is moved here
|
note: `into_iter` takes ownership of the receiver `self`, which moves `y`
--> $SRC_DIR/core/src/iter/traits/collect.rs:LL:COL
help: you can `clone` the value and consume it, but this might not be your desired behavior
|
LL | <Vec<String> as Clone>::clone(&y).into_iter();
| +++++++++++++++++++++++++++++++ +
help: consider cloning the value if the performance cost is acceptable
|
LL | y.clone().into_iter();
| ++++++++
```
When encountering a non-`Copy` value that is moved into a closure which is coming directly from a fn parameter, point at the parameter's type when mentioning it is not `Copy`.
Before:
```
error[E0507]: cannot move out of `foo`, a captured variable in an `Fn` closure
--> f111.rs:14:25
|
13 | fn do_stuff(foo: Option<Foo>) {
| --- captured outer variable
14 | require_fn_trait(|| async {
| -- ^^^^^ `foo` is moved here
| |
| captured by this `Fn` closure
15 | if foo.map_or(false, |f| f.foo()) {
| ---
| |
| variable moved due to use in coroutine
| move occurs because `foo` has type `Option<Foo>`, which does not implement the `Copy` trait
```
After:
```
error[E0507]: cannot move out of `foo`, a captured variable in an `Fn` closure
--> f111.rs:14:25
|
13 | fn do_stuff(foo: Option<Foo>) {
| --- ----------- move occurs because `foo` has type `Option<Foo>`, which does not implement the `Copy` trait
| |
| captured outer variable
14 | require_fn_trait(|| async {
| -- ^^^^^ `foo` is moved here
| |
| captured by this `Fn` closure
15 | if foo.map_or(false, |f| f.foo()) {
| --- variable moved due to use in coroutine
```
