generate elf symbol version in raw-dylib
For link names like `aaa@bbb`, it generates a symbol named `aaa` and a version named `bbb`.
For link names like `aaa\0bbb`, `aaa@`@bbb`` or `aa@bb@cc`, it emits errors.
It adds a test that the executable is linked with glibc using raw-dylib.
cc rust-lang/rust#135694
Use serde for target spec json deserialize
The previous manual parsing of `serde_json::Value` was a lot of complicated code and extremely error-prone. It was full of janky behavior like sometimes ignoring type errors, sometimes erroring for type errors, sometimes warning for type errors, and sometimes just ICEing for type errors (the icing on the top).
Additionally, many of the error messages about allowed values were out of date because they were in a completely different place than the FromStr impls. Overall, the system caused confusion for users.
I also found the old deserialization code annoying to read. Whenever a `key!` invocation was found, one had to first look for the right macro arm, and no go to definition could help.
This PR replaces all this manual parsing with a 2-step process involving serde.
First, the string is parsed into a `TargetSpecJson` struct. This struct is a 1:1 representation of the spec JSON. It already parses all the enums and is very simple to read and write.
Then, the fields from this struct are copied into the actual `Target`. The reason for this two-step process instead of just serializing into a `Target` is because of a few reasons
1. There are a few transformations performed between the two formats
2. The default logic is implemented this way. Otherwise all the default field values would have to be spelled out again, which is suboptimal. With this logic, they fall out naturally, because everything in the json struct is an `Option`.
Overall, the mapping is pretty simple, with the vast majority of fields just doing a 1:1 mapping that is captured by two macros. I have deliberately avoided making the macros generic to keep them simple.
All the `FromStr` impls now have the error message right inside them, which increases the chance of it being up to date. Some "`from_str`" impls were turned into proper `FromStr` impls to support this.
The new code is much less involved, delegating all the JSON parsing logic to serde, without any manual type matching.
This change introduces a few breaking changes for consumers. While it is possible to use this format on stable, it is very much subject to change, so breaking changes are expected. The hope is also that because of the way stricter behavior, breaking changes are easier to deal with, as they come with clearer error messages.
1. Invalid types now always error, everywhere. Previously, they would sometimes error, and sometimes just be ignored (which meant the users JSON was still broken, just silently!)
2. This now makes use of `deny_unknown_fields` instead of just warning on unused fields, which was done previously. Serde doesn't make it easy to get such warning behavior, which was the primary reason that this now changed. But I think error behavior is very reasonable too. If someone has random stale fields in their JSON, it is likely because these fields did something at some point but no longer do, and the user likely wants to be informed of this so they can figure out what to do.
This is also relevant for the future. If we remove a field but someone has it set, it probably makes sense for them to take a look whether they need this and should look for alternatives, or whether they can just delete it. Overall, the JSON is made more explicit.
This is the only expected breakage, but there could also be small breakage from small mistakes. All targets roundtrip though, so it can't be anything too major.
fixesrust-lang/rust#144153
don't link to the nightly version of the Edition Guide in stable lints
As reported in rust-lang/rust#143557 for `rust_2024_incompatible_pat`, most future-Edition-incompatibility lints link to the nightly version of the Edition Guide; the lints were written before their respective Editions (and their guides) stabilized. But now that Rusts 2021 and 2024 are stable, these lints are emitted on stable versions of the compiler, where it makes more sense to present users with links that don't say "nightly" in them.
This does not change the link for `rust_2024_incompatible_pat`. That's handled in rust-lang/rust#144006.
Ensure we codegen the main fn
This fixes two bugs. The one that was identified in the linked issue is that when we have a `main` function, mono collection didn't consider it as an extra collection root.
The other is that since CGU partitioning doesn't know about the call edges between the entrypoint functions, naively it can put them in different CGUs and mark them all as internal. Which would result in LLVM just deleting all of them. There was an existing hack to exclude `lang = "start"` from internalization, which I've extended to include `main`.
Fixes https://github.com/rust-lang/rust/issues/144052
don't link to the nightly version of the Edition Guide in stable lints
As reported in rust-lang/rust#143557 for `rust_2024_incompatible_pat`, most future-Edition-incompatibility lints link to the nightly version of the Edition Guide; the lints were written before their respective Editions (and their guides) stabilized. But now that Rusts 2021 and 2024 are stable, these lints are emitted on stable versions of the compiler, where it makes more sense to present users with links that don't say "nightly" in them.
This does not change the link for `rust_2024_incompatible_pat`. That's handled in rust-lang/rust#144006.
Various refactors to the LTO handling code (part 2)
Continuing from https://github.com/rust-lang/rust/pull/143388 this removes a bit of dead code and moves the LTO symbol export calculation from individual backends to cg_ssa.
We lost the following comment during refactorings:
The current code for niche-filling relies on variant indices instead of actual discriminants, so enums with explicit discriminants (RFC 2363) would misbehave.
resolve: Make disambiguators for underscore bindings module-local (take 2)
The difference with https://github.com/rust-lang/rust/pull/144013 can be seen in the second commit.
Now we just keep a separate disambiguator counter in every `Module`, instead of a global counter in `Resolver`.
This will be ok for parallel import resolution because we'll need to lock the module anyway when updating `resolutions` and other fields in it.
And for external modules the disabmiguator could be just passed as an argument to `define_extern`, without using any cells or locks, once https://github.com/rust-lang/rust/pull/143884 lands.
Unblocks https://github.com/rust-lang/rust/pull/143884.
Enforce that PR CI jobs are a subset of Auto CI jobs modulo carve-outs
### Background
Currently, it is possible for a PR with red PR-only CI to pass Auto CI, then all subsequent PR CI runs will be red until that is fixed, even in completely unrelated PRs. For instance, this happened with PR-CI-only Spellcheck (rust-lang/rust#144183).
See more discussions at [#t-infra > Spellcheck workflow now fails on all PRs (tree bad?)](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Spellcheck.20workflow.20now.20fails.20on.20all.20PRs.20.28tree.20bad.3F.29/with/529769404).
### CI invariant: PR CI jobs are a subset of Auto CI jobs modulo carve-outs
To prevent red PR CI in completely unrelated subsequent PRs and PR CI runs, we need to maintain an invariant that **PR CI jobs are a subset of Auto CI jobs modulo carve-outs**.
This is **not** a "strict" subset relationship: some jobs necessarily have to differ under PR CI and Auto CI environments, at least in the current setup. Still, we can try to enforce a weaker "subset modulo carve-outs" relationship between CI jobs and their corresponding Auto jobs. For instance:
- `x86_64-gnu-tools` will have `auto`-only env vars like `DEPLOY_TOOLSTATES_JSON: toolstates-linux.json`.
- `tidy` will want to `continue_on_error: true` in PR CI to allow for more "useful" compilation errors to also be reported, whereas it should be `continue_on_error: false` in Auto CI to prevent wasting Auto CI resources.
The **carve-outs** are:
1. `env` variables.
2. `continue_on_error`.
We enforce this invariant through `citool`, so only affects job definitions that are handled by `citool`. Notably, this is not sufficient *alone* to address the CI-only Spellcheck issue (rust-lang/rust#144183). To carry out this enforcement, we modify `citool` to auto-register PR jobs as Auto jobs with `continue_on_error` overridden to `false` **unless** there's an overriding Auto job for the PR job of the same name that only differs by the permitted **carve-outs**.
### Addressing the Spellcheck PR-only CI issue
Note that Spellcheck currently does not go through `citool` or `bootstrap`, and is its own GitHub Actions workflow. To actually address the PR-CI-only Spellcheck issue (rust-lang/rust#144183), and carry out the subset-modulo-carve-outs enforcement universally, this PR additionally **removes the current Spellcheck implementation** (a separate GitHub Actions Workflow). That is incompatible with Homu unless we do some hacks in the main CI workflow.
This effectively partially reverts rust-lang/rust#134006 (the separate workflow part, not the tidy extra checks component), but is not prejudice against relanding the `typos`-based spellcheck in another implementation that goes through the usual bootstrap CI workflow so that it does work with Homu. The `typos`-based spellcheck seems to have a good false-positive rate.
Closesrust-lang/rust#144183.
---
r? infra-ci
For example, transmuting between `bool` and `Ordering` doesn't need two `assume`s because one range is a superset of the other.
Multiple are still used for things like `char` <-> `NonZero<u32>`, which overlap but where neither fully contains the other.
bootstrap: add package.json and package-lock.json to dist tarball
this ensures that js-related tests can still be run from within such a dist tarball.
followup to rust-lang/rust#142924
r? ```````@Kobzol```````
coverage: Enlarge empty spans during MIR instrumentation, not codegen
This re-lands the part of rust-lang/rust#140847 that was (hopefully) not responsible for the coverage-instrumentation regressions that caused that PR to be reverted.
---
Enlarging empty spans was historically performed during MIR instrumentation, but had to be moved to codegen as part of larger changes in rust-lang/rust#134497, leading to the status quo. But now there should be no reason not to move that step back to its more logical home in instrumentaion.
mbe: Use concrete type for `get_unused_rule`
Rather than adding `get_unused_rule` to the `TTMacroExpander` trait, put
it on the concrete `MacroRulesMacroExpander`, and downcast to that type
via `Any` in order to call it.
Suggested-by: Vadim Petrochenkov <vadim.petrochenkov@gmail.com>
r? ```````@petrochenkov```````
Clean `rustc/parse/src/lexer` to improve maintainability
This PR refactors the lexer code to improve maintainability and eliminate code duplication.
In the first commit, I improve the error handling:
- rename `make_unclosed_delims_error` to more appropriate `make_mismatched_closing_delims_errors`
- changes return type from Option<Diag> to `Vec<Diag>` to avoid lengthy vec processing at `lex_token_trees`
- use `splice` instead of `extend` to make the logic clearer, since `errs` sounds more generic and better suited as a return value
In the second commit, I replace the magic number 5 with UNCLOSED_DELIMITER_SHOW_LIMIT constant.
In the third commit, I moves `eof_err` function below parsing logic for better code flow.
In the forth one, I extract `calculate_spacing` function to eliminate duplicate spacing logic between `bump` and `bump_minimal` functions.
r? compiler
Fix broken TLS destructors on 32-bit win7
Fixesrust-lang/rust#141300
On the 32-bit win7 target, we use OS TLS instead of native TLS, due to issues with how the OS handles alignment. Unfortunately, this caused issues due to the TLS destructors not running, causing memory leaks among other problems.
On Windows, to support OS TLS, the TlsAlloc family of function is used by Rust. This function does not support TLS destructors at all. However, rust has some code to emulate those destructors, by leveraging the TLS support functionality found in the MSVC CRT (specifically, in tlssup.c of the CRT).
To use this functionality, the user must do two things:
1. They must put the address to their callback in a section between `.CRT$XLB` and `.CRT$XLY`.
2. They must add a reference to `_tls_used` (or `__tls_used` on x86) to make sure the TLS support code in tlssup.c isn't garbage collected by the linker.
Prior to this commit, this second bit wasn't being done properly by the Rust TLS support code. Instead of adding a reference to _tls_used, it instead had a reference to its own callback to prevent it from getting GC'd by the linker. While this is _also_ necessary, not having a reference on _tls_used made the entire support non-functional.
This commit reworks the code to:
1. Add an unconditional `#[used]` attribute on the CALLBACK, which should be enough to prevent it from getting GC'd by the linker.
2. Add a reference to `_tls_used`, which should pull the TLS support code into the Rust programs and not let it be GC'd by the linker.
Move `std_detect` into stdlib
This PR moves the `std_detect` crate from `stdarch` to be a part of rust-lang/rust instead.
The first commit actually moves the whole directory from the stdarch Josh subtree, so that git blame history is kept intact. Then I had to make a few changes to appease `tidy`.
The most complex thing here is porting the tests. We can't have `std_detect` both in r-l/r and stdarch, because they could get desynchronized, so we have to perform the move more or less "atomically", which means that we also have to port all the existing `std_detect` tests from the `stdarch` repository.
The stdarch repo runs the following `std_detect` tests:
### Build
The `build-std-detect.sh` script (e2b6512aed/ci/build-std-detect.sh) builds `std_detect` using the nightly compiler for several targets. This will be subsumed by normal `x build library` on our Tier 1/2 targets. However, the stdarch repository also tests the following targets:
- aarch64-unknown-freebsd
- armv6-unknown-freebsd
- powerpc-unknown-freebsd
- powerpc64-unknown-freebsd
- aarch64-unknown-openbsd
Which we don't build/test on our CI currently. I think we have mostly two options here:
1) Ignore these targets
2) Create a special CI job that will build stage 1 rustc and then cross-compile std (or just the `std_detect` crate?) for these targets.
### Documentation
The `dox.sh` script (3fec5adcd5/ci/dox.sh) builds and documents `std_detect` for several targets. All of them are Tier 2/we have `dist-` jobs for them, so I think that we can just skip this and let our normal CI subsume it?
### Tests
The `run.sh` script (1b201cec2c/ci/run.sh) runs `cargo test` on `std_detect` with a bunch of variations of feature flags. This will be subsumed by `x test library` in our CI. The only problem is that `stdarch` runs these tests for a ludicrous number of targets:
```
- tuple: i686-unknown-linux-gnu
- tuple: x86_64-unknown-linux-gnu
- tuple: arm-unknown-linux-gnueabihf
- tuple: armv7-unknown-linux-gnueabihf
- tuple: aarch64-unknown-linux-gnu
- tuple: aarch64_be-unknown-linux-gnu
- tuple: riscv32gc-unknown-linux-gnu
- tuple: riscv64gc-unknown-linux-gnu
- tuple: powerpc-unknown-linux-gnu
- tuple: powerpc64-unknown-linux-gnu
- tuple: powerpc64le-unknown-linux-gnu
- tuple: s390x-unknown-linux-gnu
- tuple: i586-unknown-linux-gnu
- tuple: nvptx64-nvidia-cuda
- tuple: thumbv6m-none-eabi
- tuple: thumbv7m-none-eabi
- tuple: thumbv7em-none-eabi
- tuple: thumbv7em-none-eabihf
- tuple: loongarch64-unknown-linux-gnu
- tuple: wasm32-wasip1
- tuple: x86_64-apple-darwin
- tuple: x86_64-apple-ios-macabi
- tuple: aarch64-apple-darwin
- tuple: aarch64-apple-ios-macabi
- tuple: x86_64-pc-windows-msvc
- tuple: i686-pc-windows-msvc
- tuple: aarch64-pc-windows-msvc
- tuple: x86_64-pc-windows-gnu
- tuple: aarch64-unknown-linux-gnu
- tuple: aarch64_be-unknown-linux-gnu
- tuple: armv7-unknown-linux-gnueabihf
- tuple: loongarch64-unknown-linux-gnu
- tuple: powerpc-unknown-linux-gnu
- tuple: powerpc64-unknown-linux-gnu
- tuple: powerpc64le-unknown-linux-gnu
- tuple: riscv32gc-unknown-linux-gnu
- tuple: riscv64gc-unknown-linux-gnu
- tuple: s390x-unknown-linux-gnu
- tuple: x86_64-unknown-linux-gnu
- tuple: aarch64-apple-darwin
- tuple: aarch64-apple-ios-macabi
```
We definitely do not run *tests* for all of these targets on our CI.
# Outcome
We have decided to just subsume std_detect tests by our normal test suite for now, and not create a separate CI job. Therefore, this PR performs the following changes in target testing for `std_detect`:
The following T3 targets would go from "build" to "nothing":
```
aarch64-unknown-freebsd (T3)
armv6-unknown-freebsd (T3)
powerpc-unknown-freebsd (T3)
powerpc64-unknown-freebsd (T3)
aarch64-unknown-openbsd (T3)
```
The following T3 targets would go from "test" to "nothing":
```
aarch64_be-unknown-linux-gnu (T3)
riscv32gc-unknown-liux-gnu (T3)
```
The following T2 targets would go from "test" to "build":
```
arm-unknown-linux-gnueabihf (T2)
armv7-unknown-linux-gnueabihf (T2)
riscv64gc-unknown-linux-gnu (T2)
powerpc-unknown-linux-gnu (T2)
powerpc64-unknown-linux-gnu (T2)
powerpc64le-unknown-linux-gnu (T2)
s390x-unknown-linux-gnu (T2)
i586-unknown-linux-gnu (T2)
loongarch64-unknown-linux-gnu (T2)
wasm32-wasip1 (T2)
x86_64-apple-ios-macabi (T2)
aarch64-apple-ios-macabi (T2)
aarch64-pc-windows-msvc (T2)
armv7-unknown-linux-gnueabihf (T2)
loongarch64-unknown-linux-gnu (T2)
powerpc-unknown-linux-gnu (T2)
```
I have confirmed in https://github.com/rust-lang/stdarch/pull/1873 that the current version of this PR would pass stdarch's CI testsuite.
r? `@ghost`
try-job: armhf-gnu
try-job: arm-android
- Use EFI_TCP4_GET_MODE_DATA to be able to query for ttl, nodelay,
peer_addr and socket_addr.
- peer_addr is needed for implementation of `accept`.
Signed-off-by: Ayush Singh <ayush@beagleboard.org>
Implement AST visitors using a derive macro.
AST visitors are large and error-prone beasts. This PR attempts to write them using a derive macro.
The design uses three traits: `Visitor`, `Visitable`, `Walkable`.
- `Visitor` is the trait implemented by downstream crates, it lists `visit_stuff` methods, which call `Walkable::walk_ref` by default;
- `Walkable` is derived using the macro, the generated `walk_ref` method calls `Visitable::visit` on each component;
- `Visitable` is implemented by `common_visitor_and_walkers` macro, to call the proper `Visitor::visit_stuff` method if it exists, to call `Walkable::walk_ref` if there is none.
I agree this is quite a lot of spaghetti macros. I'm open to suggestions on how to reduce the amount of boilerplate code.
If this PR is accepted, I believe the same design can be used for the HIR visitor.
