The main loop has a *very* complex condition, which includes two
mentions of `codegen_state`. The body of the loop then immediately
switches on the `codegen_state`.
I find it easier to understand if it's a `loop` and we check for exit
conditions after switching on `codegen_state`. We end up with a tiny bit
of code duplication, but it's clear that (a) we never exit in the
`Ongoing` case, (b) we exit in the `Completed` state only if several
things are true (and there's interaction with LTO there), and (c) we
exit in the `Aborted` state if a couple of things are true. Also, the
exit conditions are all simple conjunctions.
This loop condition involves `codegen_state`, `work_items`, and
`running_with_own_token`. But the body of the loop cannot modify
`codegen_state`, so repeatedly checking it is unnecessary.
`CodegenContext` is immutable except for the `worker` field - we clone
`CodegenContext` in multiple places, changing the `worker` field each
time. It's simpler to move the `worker` field out of `CodegenContext`.
It took me some time to understand how the main thread can lend a
jobserver token to an LLVM thread. This commit renames a couple of
things to make it clearer.
- Rename the `LLVMing` variant as `Lending`, because that is a clearer
description of what is happening.
- Rename `running` as `running_with_own_token`, which makes it clearer
that there might be one additional LLVM thread running (with a loaned
token). Also add a comment to its definition.
And rename the `Compiled` variant as `Finished`, because that name makes
it clearer there is nothing left to do, contrasting nicely with the
`Needs*` variants.
Support `--print KIND=PATH` command line syntax
As is already done for `--emit KIND=PATH` and `-L KIND=PATH`.
In the discussion of #110785, it was pointed out that `--print KIND=PATH` is nicer than trying to apply the single global `-o` path to `--print`'s output, because in general there can be multiple print requests within a single rustc invocation, and anyway `-o` would already be used for a different meaning in the case of `link-args` and `native-static-libs`.
I am interested in using `--print cfg=PATH` in Buck2. Currently Buck2 works around the lack of support for `--print KIND=PATH` by [indirecting through a Python wrapper script](d43cf3a51a/prelude/rust/tools/get_rustc_cfg.py) to redirect rustc's stdout into the location dictated by the build system.
From skimming Cargo's usages of `--print`, it definitely seems like it would benefit from `--print KIND=PATH` too. Currently it is working around the lack of this by inserting `--crate-name=___ --print=crate-name` so that it can look for a line containing `___` as a delimiter between the 2 other `--print` informations it actually cares about. This is commented as a "HACK" and "abuse". 31eda6f7c3/src/cargo/core/compiler/build_context/target_info.rs (L242) (FYI `@weihanglo` as you dealt with this recently in https://github.com/rust-lang/cargo/pull/11633.)
Mentioning reviewers active in #110785: `@fee1-dead` `@jyn514` `@bjorn3`
Resurrect: rustc_llvm: Add a -Z `print-codegen-stats` option to expose LLVM statistics.
This resurrects PR https://github.com/rust-lang/rust/pull/104000, which has sat idle for a while. And I want to see the effect of stack-move optimizations on LLVM (like https://reviews.llvm.org/D153453) :).
I have applied the changes requested by `@oli-obk` and `@nagisa` https://github.com/rust-lang/rust/pull/104000#discussion_r1014625377 and https://github.com/rust-lang/rust/pull/104000#discussion_r1014642482 in the latest commits.
r? `@oli-obk`
-----
LLVM has a neat [statistics](https://llvm.org/docs/ProgrammersManual.html#the-statistic-class-stats-option) feature that tracks how often optimizations kick in. It's very handy for optimization work. Since we expose the LLVM pass timings, I thought it made sense to expose the LLVM statistics too.
-----
(Edit: fix broken link
(Edit2: fix segmentation fault and use malloc
If `rustc` is built with
```toml
[llvm]
assertions = true
```
Then you can see like
```
rustc +stage1 -Z print-codegen-stats -C opt-level=3 tmp.rs
===-------------------------------------------------------------------------===
... Statistics Collected ...
===-------------------------------------------------------------------------===
3 aa - Number of MayAlias results
193 aa - Number of MustAlias results
531 aa - Number of NoAlias results
...
```
And the current default build emits only
```
$ rustc +stage1 -Z print-codegen-stats -C opt-level=3 tmp.rs
===-------------------------------------------------------------------------===
... Statistics Collected ...
===-------------------------------------------------------------------------===
$
```
This might be better to emit the message to tell assertion flag necessity, but now I can't find how to do that...
Verify that all crate sources are in sync
This ensures that rustc will not attempt to link against a cdylib as if it is a rust dylib when an rlib for the same crate is available. Previously rustc didn't actually check if any further formats of a crate which has been loaded are of the same version and if they are actually valid. This caused a cdylib to be interpreted as rust dylib as soon as the corresponding rlib was loaded. As cdylibs don't export any rust symbols, linking would fail if rustc decides to link against the cdylib rather than the rlib.
Two crates depended on the previous behavior by separately compiling a test crate as both rlib and dylib. These have been changed to capture their original spirit to the best of my ability while still working when rustc verifies that all crates are in sync. It is unlikely that build systems depend on the current behavior and in any case we are taking a lot of measures to ensure that any change to either the source or the compilation options (including crate type) results in rustc rejecting it as incompatible. We merely didn't do this check here for now obsolete perf reasons.
Fixes https://github.com/rust-lang/rust/issues/10786
Fixes https://github.com/rust-lang/rust/issues/82151
Fixes https://github.com/rust-lang/rust/issues/82972
Closes https://github.com/bevy-cheatbook/bevy-cheatbook/issues/114
Implement rust-lang/compiler-team#578.
When an ICE is encountered on nightly releases, the new rustc panic
handler will also write the contents of the backtrace to disk. If any
`delay_span_bug`s are encountered, their backtrace is also added to the
file. The platform and rustc version will also be collected.
LLVM has a neat [statistics] feature that tracks how often optimizations kick
in. It's very handy for optimization work. Since we expose the LLVM pass
timings, I thought it made sense to expose the LLVM statistics too.
[statistics]: https://llvm.org/docs/ProgrammersManual.html#the-statistic-class-stats-option
De-duplicate consecutive libs when printing native-static-libs
This PR adds a de-duplicate step just before printing the `native-static-libs`.
This step de-duplicates all the consecutive libs based only on the relevant comparison elements (this exclude spans, ast elements, ...).
Fixes https://github.com/rust-lang/rust/issues/113209
make MCP510 behavior opt-in to avoid conflicts between the CLI and target flavors
Fixes#113597, which contains more details on how this happens through the code, and showcases an unexpected `Gnu(Cc::Yes, Lld::Yes)` flavor.
#112910 added support to use `lld` when the flavor requests it, but didn't explicitly do so only when using `-Clink-self-contained=+linker` or one of the unstable `-Clinker-flavor`s.
The problem: some targets have a `lld` linker and flavor, e.g. `thumbv6m-none-eabi` from that issue. Users can override the linker but there are no linker flavors precise enough to describe the linker opting out of lld: when using `-Clinker=arm-none-eabi-gcc`, we infer this is a `Cc::Yes` linker flavor, but the `lld` component is unknown and therefore defaulted to the target's linker flavor, `Lld::Yes`.
<details>
<summary>Walkthrough of how this happens</summary>
The linker flavor used is a mix between what can be inferred from the CLI (`-C linker`) and the target's default linker flavor:
- there is no linker flavor on the CLI (and that also offers another workaround on nightly: `-C linker-flavor=gnu-cc -Zunstable-options`), so it will have to be inferred [from here](5dac6b320b/compiler/rustc_codegen_ssa/src/back/link.rs (L1334-L1336)) to [here](5dac6b320b/compiler/rustc_codegen_ssa/src/back/link.rs (L1321-L1327)).
- in [`infer_linker_hints`](5dac6b320b/compiler/rustc_target/src/spec/mod.rs (L320-L352)) `-C linker=arm-none-eabi-gcc` infers a `Some(Cc::Yes)` cc hint, and no hint about lld.
- the target's `linker_flavor` is combined in `with_cli_hints` with these hints. We have our `Cc::Yes`, but there is no hint about lld, [so the target's flavor `lld` component is used](5dac6b320b/compiler/rustc_target/src/spec/mod.rs (L356-L358)). It's [`Gnu(Cc::No, Lld::Yes)`](993deaa0bf/compiler/rustc_target/src/spec/thumb_base.rs (L35)).
- so we now have our `Gnu(Cc::Yes, Lld::Yes)` flavor
</details>
This results in a `Gnu(Cc::Yes, Lld::Yes)` flavor on a non-lld linker, causing an additional unexpected `-fuse-ld=lld` argument to be passed.
I don't know if this target defaulting to `rust-lld` is expected, but until MCP510's new linker flavor are stable, when people will be able to describe their linker/flavor accurately, this PR keeps the stable behavior of not doing anything when the linker/flavor on the CLI unexpectedly conflict with the target's.
I've tested this on a `no_std` `-C linker=arm-none-eabi-gcc -C link-arg=-nostartfiles --target thumbv6m-none-eabi` example, trying to simulate one of `cortex-m`'s test mentioned in issue #113597 (I don't know how to build a local complete `thumbv6m-none-eabi` toolchain to run the exact test), and checked that `-fuse-lld` was indeed gone and the error disappeared.
r? `````@petrochenkov`````
PR #95604 introduced a "synthetic object file to ensure all exported and
used symbols participate in the linking". One constraint on this file is
that for MIPS-based targets, its architecture-specific ELF flags must be
the same as all other object files passed to the linker. That's enforced
by LLD, here:
https://github.com/llvm/llvm-project/blob/llvmorg-16.0.6/lld/ELF/Arch/MipsArchTree.cpp#L77
The current approach to determining e_flags for 32-bit was implemented
in PR #96930, which links to this issue that summarizes the problem well:
https://github.com/ayrtonm/psx-sdk-rs/issues/9
> ... the temporary object file is created with an e_flags which is
> invalid for 32-bit MIPS targets. The main issue is that it omits the ABI
> bits (EF_MIPS_ABI_O32) which implies it uses the N64 ABI.
To enable the N32 MIPS ABI (which succeeded O32), this patch enables
setting the synthetic object's ABI based on the target "llvm-abiname"
field, if it's given; otherwise, the O32 ABI is assumed for 32-bit MIPS
targets.
More information about the N32 ABI can be found here:
https://web.archive.org/web/20160121005457/http://techpubs.sgi.com/library/manuals/2000/007-2816-005/pdf/007-2816-005.pdf
Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML
This PR adds an option for `rustc` to emit LLVM optimization remarks to a set of YAML files, which can then be digested by existing tools, like https://github.com/OfekShilon/optview2. When `-Cremark-dir` is passed, and remarks are enabled (`-Cremark=all`), the remarks will be now written to the specified directory, **instead** of being printed to standard error output. The files are named based on the CGU from which they are being generated.
Currently, the remarks are written using the LLVM streaming machinery, directly in the diagnostics handler. It seemed easier than going back to Rust and then form there back to C++ to use the streamer from the diagnostics handler. But there are many ways to implement this, of course, so I'm open to suggestions :)
I included some comments with questions into the code. Also, I'm not sure how to test this.
r? `@tmiasko`
Implement most of MCP510
This implements most of what remains to be done for MCP510:
- turns `-C link-self-contained` into a `+`/`-` list of components, like `-C link-self-contained=+linker,+crto,+libc,+unwind,+sanitizers,+mingw`. The scaffolding is present for all these expected components to be implemented and stabilized in the future on their own time. This PR only handles the `-Zgcc-ld=lld` subset of these link-self-contained components as `-Clink-self-contained=+linker`
- handles `-C link-self-contained=y|n` as-is today, for compatibility with `rustc_codegen_ssa:🔙🔗:self_contained`'s [explicit opt-in and opt-out](9eee230cd0/compiler/rustc_codegen_ssa/src/back/link.rs (L1671-L1676)).
- therefore supports our plan to opt out of `rust-lld` (when it's enabled by default) even for current `-Clink-self-contained` users, with e.g. `-Clink-self-contained -Clink-self-contained=-linker`
- turns `add_gcc_ld_path` into its expected final form, by using the `-C link-self-contained=+linker` CLI flag, and whether the `LinkerFlavor` has the expected `Cc::Yes` and `Lld::Yes` shape (this is not yet the case in practice for any CLI linker flavor)
- makes the [new clean linker flavors](https://github.com/rust-lang/rust/pull/96827#issuecomment-1208441595) selectable in the CLI in addition to the legacy ones, in order to opt-in to using `cc` and `lld` to emulate `-Zgcc-ld=lld`
- ensure the new `-C link-self-contained` components, and `-C linker-flavor`s are unstable, and require `-Z unstable-options` to be used
The up-to-date set of flags for the future stable CLI version of `-Zgcc-ld=lld` is currently: `-Clink-self-contained=+linker -Clinker-flavor=gnu-lld-cc -Zunstable-options`.
It's possible we'll also need to do something for distros that don't ship `rust-lld`, but maybe there are already no tool search paths to be added to `cc` in this situation anyways.
r? `@petrochenkov`
loongarch: Fix ELF header flags
This patch changes the ELF header flags so that the ABI matches the floating-point features. It also updates the link to the new official documentation.
Fix unset e_flags in ELF files generated for AVR targets
Closes#106576
~~Sort-of blocked by gimli-rs/object#500~~ (merged)
I'm not sure whether the list of AVR CPU names is okay here. Maybe it could be moved out-of-line to improve the readability of the function.
For non-incremental builds on Unix, currently all the thread names look
like `opt regex.f10ba03eb5ec7975-cgu.0`. But they are truncated by
`pthread_setname` to `opt regex.f10ba`, hiding the numeric suffix that
distinguishes them. This is really annoying when using a profiler like
Samply.
This commit changes these thread names to a form like `opt cgu.0`, which
is much better.
The codegen main loop has two bools, `codegen_done` and
`codegen_aborted`. There are only three valid combinations: `(false,
false)`, `(true, false)`, `(true, true)`.
This commit replaces them with a single tri-state enum, which makes
things clearer.
`Message` is an enum with multiple variants. Four of those variants map
directly onto the four variants of `WorkItemResult`. This commit reduces
those four `Message` variants to a single variant containing a
`WorkItemResult`. This requires increasing `WorkItemResult`'s visibility
to `pub(crate)` visibility, but `WorkItem` and `Message` can also have
their visibility reduced to `pub(crate)`.
This change avoids some boilerplate enum translation code, and makes
`Message` easier to understand.
`Message` is an enum with multiple variants, for messages sent to the
coordinator thread. *Except* for `Message::CodegenItem`, which is
entirely disjoint, being for messages sent from the coordinator thread
to the main thread.
This commit move `Message::CodegenItem` into a separate type,
`CguMessage`, which makes the code much clearer.
