Make `-Z gcc-ld=lld` work for Apple targets
`-Z gcc-ld=lld` was introduced in #85961. It does not work on Macos because lld needs be either named `ld64` or passed `-flavor darwin` as the first two arguments in order to select the Mach-O flavor. Rust invokes cc (=clang) on Macos for linking which calls `ld` as linker binary and not `ld64`, so just creating an `ld64` binary and modifying the search path with `-B` does not work.
In order to solve this patch does:
* Set the `lld_flavor` for all Apple-derived targets to `LldFlavor::Ld64`. As far as I can see this actually works towards fixing `-Xlinker=rust-lld` as all those targets use the Mach-O object format.
* Copy/hardlink rust-lld to the gcc-ld subdirectory as ld64 next to ld.
* If `-Z gcc-ld=lld` is used and the target lld flavor is Ld64 add `-fuse-ld=/path/to/ld64` to the linker invocation.
Fixes#86945.
Link the static libraries with "-bundle" modifier from upstream rust crate
right after linking this rust crate. Some linker such as GNU linker
`ld.bdf` treat order of linking as order of dependency. After this change,
static libraries with "-bundle" modifier is linked in the same order as
"+bundle" modifier. So we can change the value of "bundle" modifier without
causing linking error.
Don't use gc-sections with profile-generate.
When building with profile-generate don't call gc_sections as this can
can sometimes strip out profile data. This missing information in the
prof files can then result in missing functions when using the profile
information.
#78226
r? `@Mark-Simulacrum`
When building with profile-generate request that metadata is kept
during the gc_sections call, as this can sometimes strip out profile
data.
This missing information in the prof files can then result in missing
functions when using the profile information.
Add suggestions for "undefined reference" link errors
This adds a suggestion for "undefined reference to ..." linking errors to install or specify the location to an external library. Since there is no defined error format for linkers, we just check if there was a failure and if that failure contains the string `undefined reference to`. This also makes it impossible to test this, since the output depends on the system linker. The output now looks like:
```
error: linking with `cc` failed: exit status: 1
|
= note: "cc" "-m64" "linking_failure.linking_failure.7rcbfp3g-cgu.0.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.1.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.2.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.3.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.4.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.5.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.6.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.7.rcgu.o" "linking_failure.linking_failure.7rcbfp3g-cgu.8.rcgu.o" "linking_failure.53u64zklswtfazes.rcgu.o" "-Wl,--as-needed" "-L" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-Wl,-Bstatic" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-01ce3ba5c629d02f.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-f1f2102409186354.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-1e8b6b56a999f838.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-d0e93eb4e14f1d19.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-1d7e39d75d082b43.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-ade42e945045b261.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-1a65064fccf4ebc1.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-4d699c310fdfe72d.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-1cafa68a696ec800.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e9f1c8c4dab2f046.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-ecc1a743be25c7f7.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-e074031c4b66b6b6.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9aa6ed9f1d3bfd53.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-7862bf96c2250ca0.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-f02ce0dc7895b5fd.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-3af9c60917570521.rlib" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-ca16fc7bb3645684.rlib" "-Wl,--end-group" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d8e1a5b7299604cc.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/smit/rustc-dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "linking_failure" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs"
= note: /usr/bin/ld: linking_failure.linking_failure.7rcbfp3g-cgu.3.rcgu.o: in function `linking_failure::main':
linking_failure.7rcbfp3g-cgu.3:(.text._ZN15linking_failure4main17h52b6e3052e444479E+0x3): undefined reference to `doesnt_exist_thiwthwfyl'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
= help: some `extern` functions couldn't be found; you may need to install or specify the path to some dependencies
= note: use the -L flag to specify the library lookup path
= note: use the cargo:rustc-link-search directive to specify the library lookup path with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-search)
error: aborting due to previous error
```
ignore test if rust-lld not found
create ld -> rust-lld symlink at build time instead of run time
for testing in ci
copy instead of symlinking
remove linux check
test for linker, suggestions from bjorn3
fix overly restrictive lld matcher
use -Zgcc-ld flag instead of -Clinker-flavor
refactor code adding lld to gcc path
revert ci changes
suggestions from petrochenkov
rename gcc_ld to gcc-ld in dirs
Remove the install prefix from the rpath set when using -Crpath
It was broken anyway for rustup installs and nobody seems to have noticed.
Fixes https://github.com/rust-lang/rust/issues/82392
Unify duplicate linker_and_flavor methods in rustc_codegen_{cranelift,ssa}.
The two methods were exactly the same so this removes the cranelift copy. This will help make sure both they don't get out of sync.
Driver improvements
This PR contains a couple of cleanups for the driver and a few small improvements for the custom codegen backend interface. It also implements `--version` and `-Cpasses=list` support for custom codegen backends.
- Combine all native library arguments together, to simplify potential support for library deduplication and similar things
- Split arguments into order-independent and order-dependent, to define more precisely what (pre,post,late)-link-args mean
Tweak wasm_base target spec to indicate linker is not GNU and update linker inferring logic for wasm-ld.
Reported via [Zulip](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/wasi.20linker.20unknown.20argument.3A.20--as-needed): we try passing `--as-needed` to the linker if it's GNU ld which `wasm-ld` is not. Usually this isn't an issue for wasm as we would use the WasmLd linker driver but because the linker in question (`wasm32-unknown-wasi-wasm-ld`) ended with `-ld` our linker inferring [logic](f64503eb55/compiler/rustc_codegen_ssa/src/back/link.rs (L957-L1040)) used the `GccLinker` implementations. (UPD: The linker inferring logic actually didn't apply in this case because the linker is actually invoked through gcc in the reported issue. But it's still worth updating the logic I think.)
This change then has 2 parts:
1. Update wasm_base target spec to indicate `linker_is_gnu: false` plus a few additions of `target.is_like_wasm` to handle flags `wasm-ld` does in fact support.
2. Improve the linker detection logic to properly determine the correct flavor of wasm linker we're using when we can.
We need to add the new `target.is_like_wasm` branches to handle the case where the "linker" used could be something like clang which would then under the hood call wasm-ld.
Preserve metadata w/ Solaris-like linkers.
#84468 moved the `-zignore` linker flag from the `gc_sections` method to `add_as_needed` which is more accurate but Solaris-style linkers will also end up removing an unreferenced ELF sections [1]. This had the unfortunate side effect of causing the `.rustc` section (which has the metada) to be removed which could cause issues when trying to link against the resulting crates or use proc macros.
Since the `-zignore` is positional, we fix this by moving the metadata objects to before the flag.
[1] Specifically a section is considered unreferenced if:
* The section is allocatable
* No other sections bind to (relocate) to this section
* The section provides no global symbols
https://docs.oracle.com/cd/E19683-01/817-3677/6mj8mbtbs/index.html#chapter4-19
Partial support for raw-dylib linkage
First cut of functionality for issue #58713: add support for `#[link(kind = "raw-dylib")]` on `extern` blocks in lib crates compiled to .rlib files. Does not yet support `#[link_name]` attributes on functions, or the `#[link_ordinal]` attribute, or `#[link(kind = "raw-dylib")]` on `extern` blocks in bin crates; I intend to publish subsequent PRs to fill those gaps. It's also not yet clear whether this works for functions in `extern "stdcall"` blocks; I also intend to investigate that shortly and make any necessary changes as a follow-on PR.
This implementation calls out to an LLVM function to construct the actual `.idata` sections as temporary `.lib` files on disk and then links those into the generated .rlib.
BPF target support
This adds `bpfel-unknown-none` and `bpfeb-unknown-none`, two new no_std targets that generate little and big endian BPF. The approach taken is very similar to the cuda target, where `TargetOptions::obj_is_bitcode` is enabled and code generation is done by the linker.
I added the targets to `dist-various-2`. There are [some tests](https://github.com/alessandrod/bpf-linker/tree/main/tests/assembly) in bpf-linker and I'm planning to add more. Those are currently not ran as part of rust CI.
This does not yet support #[link_name] attributes on functions, the #[link_ordinal]
attribute, #[link(kind = "raw-dylib")] on extern blocks in bin crates, or
stdcall functions on 32-bit x86.
This commit updates how rustc compiler metadata is stored in rlibs.
Previously metadata was stored as a raw file that has the same format as
`--emit metadata`. After this commit, however, the metadata is encoded
into a small object file which has one section which is the contents of
the metadata.
The motivation for this commit is to fix a common case where #83730
arises. The problem is that when rustc crates a `dylib` crate type it
needs to include entire rlib files into the dylib, so it passes
`--whole-archive` (or the equivalent) to the linker. The problem with
this, though, is that the linker will attempt to read all files in the
archive. If the metadata file were left as-is (today) then the linker
would generate an error saying it can't read the file. The previous
solution was to alter the rlib just before linking, creating a new
archive in a temporary directory which has the metadata file removed.
This problem from before this commit is now removed if the metadata file
is stored in an object file that the linker can read. The only caveat we
have to take care of is to ensure that the linker never actually
includes the contents of the object file into the final output. We apply
similar tricks as the `.llvmbc` bytecode sections to do this.
This involved changing the metadata loading code a bit, namely updating
some of the LLVM C APIs used to use non-deprecated ones and fiddling
with the lifetimes a bit to get everything to work out. Otherwise though
this isn't intended to be a functional change really, only that metadata
is stored differently in archives now.
This should end up fixing #83730 because by default dylibs will no
longer have their rlib dependencies "altered" meaning that
split-debuginfo will continue to have valid paths pointing at the
original rlibs. (note that we still "alter" rlibs if LTO is enabled to
remove Rust object files and we also "alter" for the #[link(cfg)]
feature, but that's rarely used).
Closes#83730
For extern providers, both provide and provide_extern are called.
wasm_import_module_map is already provided in provide, so it doesn't
need to be provided in provide_extern.
