Make `#[used]` work when linking with `ld64`
To make `#[used]` work in static libraries, we use the `symbols.o` trick introduced in https://github.com/rust-lang/rust/pull/95604.
However, the linker shipped with Xcode, ld64, works a bit differently from other linkers; in particular, [it completely ignores undefined symbols by themselves](https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L2455-L2468), and only consider them if they have relocations (something something atoms something fixups, I don't know the details).
So to make the `symbols.o` file work on ld64, we need to actually insert a relocation. That's kinda cumbersome to do though, since the relocation must be valid, and hence must point to a valid piece of machine code, and is hence very architecture-specific.
Fixes https://github.com/rust-lang/rust/issues/133491, see that for investigation.
---
Another option would be to pass `-u _foo` to the final linker invocation. This has the problem that `-u` causes the linker to not be able to dead-strip the symbol, which is undesirable. (If we did this, we would possibly also want to do it by putting the arguments in a file by itself, and passing that file via ``@`,` e.g. ``@undefined_symbols.txt`,` similar to https://github.com/rust-lang/rust/issues/52699, though that [is only supported since Xcode 12](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes#Linking), and I'm not sure we wanna bump that).
Various other options that are probably all undesirable as they affect link time performance:
- Pass `-all_load` to the linker.
- Pass `-ObjC` to the linker (the Objective-C support in the linker has different code paths that load more of the binary), and instrument the binaries that contain `#[used]` symbols.
- Pass `-force_load` to libraries that contain `#[used]` symbols.
Failed attempt: Embed `-u _foo` in the object file with `LC_LINKER_OPTION`, akin to https://github.com/rust-lang/rust/issues/121293. Doesn't work, both because `ld64` doesn't read that from archive members unless it already has a reason to load the member (which is what this PR is trying to make it do), and because `ld64` only support the `-l`, `-needed-l`, `-framework` and `-needed_framework` flags in there.
---
TODO:
- [x] Support all Apple architectures.
- [x] Ensure that this works regardless of the actual type of the symbol.
- [x] Write up more docs.
- [x] Wire up a few proper tests.
`@rustbot` label O-apple
Misc. `rustc_codegen_ssa` cleanups 🧹
Just a bunch of stuff I found while reading the crate's code.
Each commit can stand on its own.
Maybe r? `@Noratrieb` because I saw you did some similar cleanups on these files a while ago? (feel free to re-assign, I'm just guessing)
Pass through of target features to llvm-bitcode-linker and handling them
When using the llvm-bitcode-linker (`linker-flavor=llbc`) target-features are not passed through and are not handled by it.
The llvm-bitcode-linker is mainly used as a self contained linker to link llvm bitcode for the nvptx64 target. It uses `llvm-link`, `opt` and `llc` internally. To produce a `.ptx` file of a specific ptx-version it is necessary to pass the version to llc with the `--mattr` option. Without explicitly setting it, the emitted `.ptx`-version is the minimum supported version of the `--target-cpu`.
I would like to be able to explicitly set the ptx version as [some llvm problems only occur in earlier `.ptx`-versions](https://github.com/llvm/llvm-project/issues/112998).
Therefore this pull request adds support for passing target features to llvm-bitcode-linker and handling them.
I was not quite sure if adding these features to `rustc_target/src/target_features.rs` is necessary or not. If so I will gladly add these.
r? ``@kjetilkjeka``
This commit removes the `avr-unknown-gnu-atmega328` target and replaces
it with a more generic `avr-none` variant that must be specialized with
the `-C target-cpu` flag (e.g. `-C target-cpu=atmega328p`).
The .ptx version produced by llc can be specified by passing it with --mattr. Currently it is not possible to specify the .ptx version with -Ctarget-feature because these are not passed through to llvm-bitcode-linker and handled by it. This commit adds both.
--target-feature and -mattr are passed with equals to mitigate issues when the value starts with a - (minus).
- Don't show environment variables. Seeing PATH is almost never useful, and it can be extremely long.
- For .rlibs in the sysroot, replace crate hashes with a `"-*"` string. This will expand to the full crate name when pasted into the shell.
- Move `.rlib` to outside the glob.
- Abbreviate the sysroot path to `<sysroot>` wherever it appears in the arguments.
This also adds an example of the linker output as a run-make test. Currently it only runs on x86_64-unknown-linux-gnu, because each platform has its own linker arguments. So that it's stable across machines, pass BUILD_ROOT as an argument through compiletest through to run-make tests.
- Only use linker-flavor=gnu-cc if we're actually going to compare the output. It doesn't exist on MacOS.
These cannot be silenced with a CLI flag, and are not useful to warn
about. They can still be viewed for debugging purposes using
`RUSTC_LOG=rustc_codegen_ssa:🔗:back`.
If we build the standard library with wasm-eh then we need to link
with `-fwasm-exceptions` even if we compile with `panic=abort`
Without this change, linking a `panic=abort` crate fails with:
`undefined symbol: __cpp_exception`.
Followup to #131830.
Add support for wasm exception handling to Emscripten target
This is a draft because we need some additional setting for the Emscripten target to select between the old exception handling and the new exception handling. I don't know how to add a setting like that, would appreciate advice from Rust folks. We could maybe choose to use the new exception handling if `Ctarget-feature=+exception-handling` is passed? I tried this but I get errors from llvm so I'm not doing it right.
Pass the arch rather than full target name to windows_registry::find_tool
The full target name can be anything with custom target specs. Passing just the arch wasn't possible before cc 1.2, but is now thanks to https://github.com/rust-lang/cc-rs/pull/1285.
try-job: i686-msvc
When `-Cstrip` was changed to use the bundled rust-objcopy instead of
/usr/bin/strip on OSX, strip-like arguments were preserved.
But strip and objcopy are, while being the same binary, different, they
have different defaults depending on which binary they are.
Notably, strip strips everything by default, and objcopy doesn't strip
anything by default.
Additionally, `-S` actually means `--strip-all`, so debuginfo stripped
everything and symbols didn't strip anything.
We now correctly pass `--strip-debug` and `--strip-all`.
Reduce the amount of explicit FatalError.raise()
Instead use dcx.abort_if_error() or guar.raise_fatal() instead. These guarantee that an error actually happened previously and thus we don't silently abort.
Instead use dcx.abort_if_error() or guar.raise_fatal() instead. These
guarantee that an error actually happened previously and thus we don't
silently abort.
`rustc_span::symbol` defines some things that are re-exported from
`rustc_span`, such as `Symbol` and `sym`. But it doesn't re-export some
closely related things such as `Ident` and `kw`. So you can do `use
rustc_span::{Symbol, sym}` but you have to do `use
rustc_span::symbol::{Ident, kw}`, which is inconsistent for no good
reason.
This commit re-exports `Ident`, `kw`, and `MacroRulesNormalizedIdent`,
and changes many `rustc_span::symbol::` qualifiers in `compiler/` to
`rustc_span::`. This is a 200+ net line of code reduction, mostly
because many files with two `use rustc_span` items can be reduced to
one.
don't show the full linker args unless `--verbose` is passed
the linker arguments can be *very* long, especially for crates with many dependencies. often they are not useful. omit them unless the user specifically requests them.
split out from https://github.com/rust-lang/rust/pull/119286. fixes https://github.com/rust-lang/rust/issues/109979.
r? `@bjorn3`
try-build: i686-mingw
the linker arguments can be *very* long, especially for crates with many dependencies. some parts of them are not very useful. unless specifically requested:
- omit object files specific to the current invocation
- fold rlib files into a single braced argument (in shell expansion format)
this shortens the output significantly without removing too much information.
It is treated as a map already. This is using FxIndexMap rather than
UnordMap because the latter doesn't provide an api to pick a single
value iff all values are equal, which each_linked_rlib depends on.
[AIX] Pass -bnoipath when adding rust upstream dynamic crates
Unlike ELF linkers, AIX doesn't feature `DT_SONAME` to override
the dependency name when outputing a shared library, which is something
we rely on for dylib crates.
See for reference:
bc145cec45/compiler/rustc_codegen_ssa/src/back/linker.rs (L464))
Thus, `ld` on AIX will use the full path to shared libraries as the dependency if passed it
by default unless `noipath` is passed, so pass it here so we don't end up with full path dependencies
for dylib crates.
It was inconsistently done (sometimes even within a single function) and
most of the rest of the compiler uses fatal errors instead, which need
to be caught using catch_with_exit_code anyway. Using fatal errors
instead of ErrorGuaranteed everywhere in the driver simplifies things a
bit.
Properly pass linker arguments that contain commas
When linking with the system C compiler, we sometimes want to forward certain arguments unchanged to the linker. This can be done with `-Wl,arg1,arg2` or `-Xlinker arg1 -Xlinker arg2`. `-Wl` is used when possible, since it is more compact, but it does not support commas in the argument itself - in those cases, we need to use `-Xlinker`, and that is what this PR implements.
This also fixes using sanitizers on macOS with `-Clinker-flavor=ld`, as those were previously manually using `-Wl`/`-Xlinker` (probably since the support wasn't present in the `link_args` function).
Note that there has been [a previous PR for this](https://github.com/rust-lang/rust/pull/38798), but it only implemented this in certain cases when passing `-rpath`.
r? compiler
Using `cc_args` panics when using `-Clinker-flavor=ld`, because the
arguments are in a form tailored for `-Clinker-flavor=gcc`.
So instead, we use `link_args` and let that wrap the arguments with the
appropriate `-Wl` or `-Xlinker` when needed.
[AIX] Add option -X32_64 to the "strip" command
The AIX `strip` utility requires option `-X` to specify the object mode. This patch adds the `-X32_64` option to the `strip` command so that it can handle both 32-bit and 64-bit objects. The parameter `option` of function `strip_symbols_with_external_utility`, previously a single string, has been changed to `options`, an array of string slices, to accommodate multiple `strip` options.
Over in Zed we've noticed that loading crates for a large-ish workspace can take almost 200ms. We've pinned it down to how rustc searches for paths, as it performs a linear search over the list of candidate paths. In our case the candidate list had about 20k entries which we had to iterate over for each dependency being loaded.
This commit introduces a simple FilesIndex that's just a sorted Vec under the hood. Since crates are looked up by both prefix and suffix, we perform a range search on said Vec (which constraints the search space based on prefix) and follow up with a linear scan of entries with matching suffixes.
FilesIndex is also pre-filtered before any queries are performed using available target information; query prefixes/sufixes are based on the target we are compiling for, so we can remove entries that can never match up front.
Overall, this commit brings down build time for us in dev scenarios by about 6%.
100ms might not seem like much, but this is a constant cost that each of our workspace crates has to pay, even when said crate is miniscule.
Add a default implementation for CodegenBackend::link
As a side effect this should add raw-dylib support to cg_gcc as the default ArchiveBuilderBuilder that is used implements create_dll_import_lib. I haven't tested if the raw-dylib support actually works however.
After link_binary the temporary files referenced by CodegenResults are
deleted, so calling link_binary again with the same CodegenResults
should not be allowed.
On arm64, Cranelift used to produce object files that don't work with
lld. This has since been fixed. The GCC backend should always produce
object files that work with lld unless lld for whatever reason drops GCC
support. Most of the other more niche backends don't use cg_ssa's linker
code at all. If they do and don't work with lld, they can always disable
lld usage using a cli argument.
Without this commit using cg_clif is by default in a non-trivial amount
of cases a perf regression on Linux due to ld.bfd being a fair bit
slower than lld. It is possible to explicitly enable it without this
commit, but most users are unlikely to do this.
Set "symbol name" in raw-dylib import libraries to the decorated name
`windows-rs` received a bug report that mixing raw-dylib generated and the Windows SDK import libraries was causing linker failures: <https://github.com/microsoft/windows-rs/issues/3285>
The root cause turned out to be #124958, that is we are not including the decorated name in the import library and so the import name type is also not being correctly set.
This change modifies the generation of import libraries to set the "symbol name" to the fully decorated name and correctly marks the import as being data vs function.
Note that this also required some changes to how the symbol is named within Rust: for MSVC we now need to use the decorated name but for MinGW we still need to use partially decorated (or undecorated) name.
Fixes#124958
Passing i686 MSVC and MinGW build: <https://github.com/rust-lang/rust/actions/runs/11000433888?pr=130586>
r? `@ChrisDenton`
The target name can be anything with custom target specs. Matching on
fields inside the target spec is much more robust than matching on the
target name.
The OS version depends on the deployment target environment variables,
the access of which we want to move to later in the compilation pipeline
that has access to more information, for example `env_depinfo`.
Apple: Do not specify an SDK version in `rlib` object files
This was added in https://github.com/rust-lang/rust/pull/114114, but is unnecessary, since it ends up being overwritten when linking anyhow, and it feels wrong to embed some arbitrary SDK version in here. The object files produced by LLVM also do not set this, and the tooling shows `n/a` when it's `0`, so it seems to genuinely be optional in object files.
I've also added a test for the different places the SDK version shows up, and documented a bit more in the code how SDK versions work.
See https://github.com/rust-lang/rust/issues/129432 for the bigger picture.
Tested with (excludes the same few targets as in https://github.com/rust-lang/rust/pull/130435):
```console
./x test tests/run-make/apple-sdk-version --target aarch64-apple-darwin,aarch64-apple-ios,aarch64-apple-ios-macabi,aarch64-apple-ios-sim,aarch64-apple-tvos,aarch64-apple-tvos-sim,aarch64-apple-visionos,aarch64-apple-visionos-sim,aarch64-apple-watchos,aarch64-apple-watchos-sim,arm64_32-apple-watchos,armv7k-apple-watchos,armv7s-apple-ios,x86_64-apple-darwin,x86_64-apple-ios,x86_64-apple-ios-macabi,x86_64-apple-tvos,x86_64-apple-watchos-sim,x86_64h-apple-darwin
IPHONEOS_DEPLOYMENT_TARGET=10.0 ./x test tests/run-make/apple-sdk-version --target=i386-apple-ios
```
CC `@BlackHoleFox,` you [originally commented on these values](https://github.com/rust-lang/rust/pull/114114#discussion_r1300599445).
`@rustbot` label O-apple
