fix: remove the check of lld not supporting @response-file
In LLVM v9, lld has supported `@response-file.`
LLVM v9 was released on 2019-09-19.
The check was added back to 2018-03-14 (1.26.0) via 04442af18b.
It has been more than five years, and we ship our own lld regardlessly.
This should be happily removed.
See also:
* <bb12396f91>
* <https://reviews.llvm.org/D63024>
metadata: Ignore sysroot when doing the manual native lib search in rustc
This is the opposite alternative to https://github.com/rust-lang/rust/pull/138170 and another way to make native library search consistent between rustc and linker.
This way the directory list searched by rustc is still a prefix of the directory list considered by linker, but it's a shorter prefix than in #138170.
We can include the sysroot directories into rustc's search again later if the issues with #138170 are resolved, it will be a backward compatible change.
This may break some code doing weird things on unstable rustc, or tier 2-3 targets, like bundling `libunwind.a` or sanitizers into something.
Note that this doesn't affect shipped `libc.a`, because it lives in `self-contained` directories in sysroot, and `self-contained` sysroot is already not included into the rustc's search. All libunwind and sanitizer libs should be moved to `self-contained` sysroot too eventually.
With the consistent search directory list between rustc and linker we can make rustc own the native library search (at least for static libs) and use linker search only as a fallback (like in #123436). This will allow addressing issues like https://github.com/rust-lang/rust/pull/132394 once and for all on all targets.
r? ``@bjorn3``
In LLVM v9, lld has supported @response-file
LLVM v9 was released on 2019-09-19.
And the check was added back to 2018-03-14 (1.26.0) via 04442af18b.
It has been more than five years, and we ship our own lld regardlessly.
This should be happily removed.
See also:
* <bb12396f91>
* <https://reviews.llvm.org/D63024>
Continuing the work from #137350.
Removes the unused methods: `expect_variant`, `expect_field`,
`expect_foreign_item`.
Every method gains a `hir_` prefix.
compiler: Use `size_of` from the prelude instead of imported
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the prelude instead of importing or qualifying them. Apply this change across the compiler.
These functions were added to all preludes in Rust 1.80.
r? ``@compiler-errors``
self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux
To help stabilization, this PR disables an LLD optimization on x64 linux with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](https://github.com/dtolnay/linkme/issues/63) or on nightly, need to disable lld.
More information about all this, and the historical differences, can be found in:
- https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order
- https://lld.llvm.org/ELF/start-stop-gc
This optimization has [no visible impact](https://github.com/rust-lang/rust/pull/137685#issuecomment-2686116312) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable.
I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe ````@Kobzol```` will look into it if they have time.
r? ````@petrochenkov````
Add verbatim linker to AIXLinker
This adds support for the "verbatim" native link modifier on AIX, will successfully pass the `native-link-modifier-verbatim-linker test case`
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the
prelude instead of importing or qualifying them.
These functions were added to all preludes in Rust 1.80.
Clean up various LLVM FFI things in codegen_llvm
cc ```@ZuseZ4``` I touched some autodiff parts
The major change of this PR is [bfd88ce](bfd88cead0) which makes `CodegenCx` generic just like `GenericBuilder`
The other commits mostly took advantage of the new feature of making extern functions safe, but also just used some wrappers that were already there and shrunk unsafe blocks.
best reviewed commit-by-commit
Support raw-dylib link kind on ELF
raw-dylib is a link kind that allows rustc to link against a library without having any library files present.
This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future.
This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.
I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27
Tracking issue: #135694
r? bjorn3
try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: test-various
Fix link failure on AVR (incompatible ISA error)
Fixes#137739. A reproducer of the issue is present there. I believe the root cause was introducing the avr-none target (which has no CPU by default) while also trying to get the ISA revision from the target spec. This commit uses the `target-cpu` option instead, which is already required to be present for the target.
r? compiler
cc ``@Patryk27``
The embedded bitcode should always be prepared for LTO/ThinLTO
Fixes#115344. Fixes#117220.
There are currently two methods for generating bitcode that used for LTO. One method involves using `-C linker-plugin-lto` to emit object files as bitcode, which is the typical setting used by cargo. The other method is through `-C embed-bitcode=yes`.
When using with `-C embed-bitcode=yes -C lto=no`, we run a complete non-LTO LLVM pipeline to obtain bitcode, then the bitcode is used for LTO. We run the Call Graph Profile Pass twice on the same module.
This PR is doing something similar to LLVM's `buildFatLTODefaultPipeline`, obtaining the bitcode for embedding after running `buildThinLTOPreLinkDefaultPipeline`.
r? nikic
Fixes#137739. A reproducer of the issue is present there. I believe the
root cause was introducing the avr-none target (which has no CPU by
default) and trying to get the ISA revision from there. This commit
uses the `target-cpu` option instead, which is already required to be
present for the target.
Co-authored-by: tones111 <tones111@users.noreply.github.com>
raw-dylib is a link kind that allows rustc to link against a library
without having any library files present.
This currently only exists on Windows. rustc will take all the symbols
from raw-dylib link blocks and put them in an import library, where they
can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient
to have this same functionality. Not having the libraries present at
build-time can be convenient for several reasons, especially
cross-compilation. With raw-dylib, code linking against a library can be
cross-compiled without needing to have these libraries available on the
build machine. If the libc crate makes use of this, it would allow
cross-compilation without having any libc available on the build
machine. This is not yet possible with this implementation, at least
against libc's like glibc that use symbol versioning.
The raw-dylib kind could be extended with support for symbol versioning
in the future.
This implementation is very experimental and I have not tested it very
well. I have tested it for a toy example and the lz4-sys crate, where it
was able to successfully link a binary despite not having a
corresponding library at build-time.
Make `#[used]` work when linking with `ld64`
To make `#[used]` work in static libraries, we use the `symbols.o` trick introduced in https://github.com/rust-lang/rust/pull/95604.
However, the linker shipped with Xcode, ld64, works a bit differently from other linkers; in particular, [it completely ignores undefined symbols by themselves](https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L2455-L2468), and only consider them if they have relocations (something something atoms something fixups, I don't know the details).
So to make the `symbols.o` file work on ld64, we need to actually insert a relocation. That's kinda cumbersome to do though, since the relocation must be valid, and hence must point to a valid piece of machine code, and is hence very architecture-specific.
Fixes https://github.com/rust-lang/rust/issues/133491, see that for investigation.
---
Another option would be to pass `-u _foo` to the final linker invocation. This has the problem that `-u` causes the linker to not be able to dead-strip the symbol, which is undesirable. (If we did this, we would possibly also want to do it by putting the arguments in a file by itself, and passing that file via ``@`,` e.g. ``@undefined_symbols.txt`,` similar to https://github.com/rust-lang/rust/issues/52699, though that [is only supported since Xcode 12](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes#Linking), and I'm not sure we wanna bump that).
Various other options that are probably all undesirable as they affect link time performance:
- Pass `-all_load` to the linker.
- Pass `-ObjC` to the linker (the Objective-C support in the linker has different code paths that load more of the binary), and instrument the binaries that contain `#[used]` symbols.
- Pass `-force_load` to libraries that contain `#[used]` symbols.
Failed attempt: Embed `-u _foo` in the object file with `LC_LINKER_OPTION`, akin to https://github.com/rust-lang/rust/issues/121293. Doesn't work, both because `ld64` doesn't read that from archive members unless it already has a reason to load the member (which is what this PR is trying to make it do), and because `ld64` only support the `-l`, `-needed-l`, `-framework` and `-needed_framework` flags in there.
---
TODO:
- [x] Support all Apple architectures.
- [x] Ensure that this works regardless of the actual type of the symbol.
- [x] Write up more docs.
- [x] Wire up a few proper tests.
`@rustbot` label O-apple
Misc. `rustc_codegen_ssa` cleanups 🧹
Just a bunch of stuff I found while reading the crate's code.
Each commit can stand on its own.
Maybe r? `@Noratrieb` because I saw you did some similar cleanups on these files a while ago? (feel free to re-assign, I'm just guessing)
Pass through of target features to llvm-bitcode-linker and handling them
When using the llvm-bitcode-linker (`linker-flavor=llbc`) target-features are not passed through and are not handled by it.
The llvm-bitcode-linker is mainly used as a self contained linker to link llvm bitcode for the nvptx64 target. It uses `llvm-link`, `opt` and `llc` internally. To produce a `.ptx` file of a specific ptx-version it is necessary to pass the version to llc with the `--mattr` option. Without explicitly setting it, the emitted `.ptx`-version is the minimum supported version of the `--target-cpu`.
I would like to be able to explicitly set the ptx version as [some llvm problems only occur in earlier `.ptx`-versions](https://github.com/llvm/llvm-project/issues/112998).
Therefore this pull request adds support for passing target features to llvm-bitcode-linker and handling them.
I was not quite sure if adding these features to `rustc_target/src/target_features.rs` is necessary or not. If so I will gladly add these.
r? ``@kjetilkjeka``
This commit removes the `avr-unknown-gnu-atmega328` target and replaces
it with a more generic `avr-none` variant that must be specialized with
the `-C target-cpu` flag (e.g. `-C target-cpu=atmega328p`).
The .ptx version produced by llc can be specified by passing it with --mattr. Currently it is not possible to specify the .ptx version with -Ctarget-feature because these are not passed through to llvm-bitcode-linker and handled by it. This commit adds both.
--target-feature and -mattr are passed with equals to mitigate issues when the value starts with a - (minus).
Export kernel descriptor for amdgpu kernels
The host runtime (HIP or HSA) expects a kernel descriptor object for each kernel in the ELF file. The amdgpu LLVM backend generates the object. It is created as a symbol with the name of the kernel plus a `.kd` suffix.
Add it to the exported symbols in the linker script, so that it can be found.
For reference, the symbol is created here in LLVM: d5457e4c16/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp (L966)
I wrote [a test](6a9115b121) for this as well, I’ll add that once the target is merged and working.
With this, all PRs to get working code for amdgpu are open (this + the target + the two patches adding addrspacecasts for alloca and global variables).
Tracking issue: #135024
r? `@workingjubilee`
The host runtime (HIP or HSA) expects a kernel descriptor object for
each kernel in the ELF file. The amdgpu LLVM backend generates the
object. It is created as a symbol with the name of the kernel plus a
`.kd` suffix.
Add it to the exported symbols in the linker script, so that it can be
found.
