raw-dylib is a link kind that allows rustc to link against a library
without having any library files present.
This currently only exists on Windows. rustc will take all the symbols
from raw-dylib link blocks and put them in an import library, where they
can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient
to have this same functionality. Not having the libraries present at
build-time can be convenient for several reasons, especially
cross-compilation. With raw-dylib, code linking against a library can be
cross-compiled without needing to have these libraries available on the
build machine. If the libc crate makes use of this, it would allow
cross-compilation without having any libc available on the build
machine. This is not yet possible with this implementation, at least
against libc's like glibc that use symbol versioning.
The raw-dylib kind could be extended with support for symbol versioning
in the future.
This implementation is very experimental and I have not tested it very
well. I have tested it for a toy example and the lz4-sys crate, where it
was able to successfully link a binary despite not having a
corresponding library at build-time.
Make `#[used]` work when linking with `ld64`
To make `#[used]` work in static libraries, we use the `symbols.o` trick introduced in https://github.com/rust-lang/rust/pull/95604.
However, the linker shipped with Xcode, ld64, works a bit differently from other linkers; in particular, [it completely ignores undefined symbols by themselves](https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L2455-L2468), and only consider them if they have relocations (something something atoms something fixups, I don't know the details).
So to make the `symbols.o` file work on ld64, we need to actually insert a relocation. That's kinda cumbersome to do though, since the relocation must be valid, and hence must point to a valid piece of machine code, and is hence very architecture-specific.
Fixes https://github.com/rust-lang/rust/issues/133491, see that for investigation.
---
Another option would be to pass `-u _foo` to the final linker invocation. This has the problem that `-u` causes the linker to not be able to dead-strip the symbol, which is undesirable. (If we did this, we would possibly also want to do it by putting the arguments in a file by itself, and passing that file via ``@`,` e.g. ``@undefined_symbols.txt`,` similar to https://github.com/rust-lang/rust/issues/52699, though that [is only supported since Xcode 12](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes#Linking), and I'm not sure we wanna bump that).
Various other options that are probably all undesirable as they affect link time performance:
- Pass `-all_load` to the linker.
- Pass `-ObjC` to the linker (the Objective-C support in the linker has different code paths that load more of the binary), and instrument the binaries that contain `#[used]` symbols.
- Pass `-force_load` to libraries that contain `#[used]` symbols.
Failed attempt: Embed `-u _foo` in the object file with `LC_LINKER_OPTION`, akin to https://github.com/rust-lang/rust/issues/121293. Doesn't work, both because `ld64` doesn't read that from archive members unless it already has a reason to load the member (which is what this PR is trying to make it do), and because `ld64` only support the `-l`, `-needed-l`, `-framework` and `-needed_framework` flags in there.
---
TODO:
- [x] Support all Apple architectures.
- [x] Ensure that this works regardless of the actual type of the symbol.
- [x] Write up more docs.
- [x] Wire up a few proper tests.
`@rustbot` label O-apple
This commit removes the `avr-unknown-gnu-atmega328` target and replaces
it with a more generic `avr-none` variant that must be specialized with
the `-C target-cpu` flag (e.g. `-C target-cpu=atmega328p`).
Replace some u64 hashes with Hash64
I introduced the Hash64 and Hash128 types in https://github.com/rust-lang/rust/pull/110083, essentially as a mechanism to prevent hashes from landing in our leb128 encoding paths. If you just have a u64 or u128 field in a struct then derive Encodable/Decodable, that number gets leb128 encoding. So if you need to store a hash or some other value which behaves very close to a hash, don't store it as a u64.
This reverts part of https://github.com/rust-lang/rust/pull/117603, which turned an encoded Hash64 into a u64.
Based on https://github.com/rust-lang/rust/pull/110083, I don't expect this to be perf-sensitive on its own, though I expect that it may help stabilize some of the small rmeta size fluctuations we currently see in perf reports.
Export kernel descriptor for amdgpu kernels
The host runtime (HIP or HSA) expects a kernel descriptor object for each kernel in the ELF file. The amdgpu LLVM backend generates the object. It is created as a symbol with the name of the kernel plus a `.kd` suffix.
Add it to the exported symbols in the linker script, so that it can be found.
For reference, the symbol is created here in LLVM: d5457e4c16/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp (L966)
I wrote [a test](6a9115b121) for this as well, I’ll add that once the target is merged and working.
With this, all PRs to get working code for amdgpu are open (this + the target + the two patches adding addrspacecasts for alloca and global variables).
Tracking issue: #135024
r? `@workingjubilee`
The host runtime (HIP or HSA) expects a kernel descriptor object for
each kernel in the ELF file. The amdgpu LLVM backend generates the
object. It is created as a symbol with the name of the kernel plus a
`.kd` suffix.
Add it to the exported symbols in the linker script, so that it can be
found.
- Use `object` based test logic instead of processing `nm`
human-readable textual output.
- Try to expand test coverage to not be limited to only linux + x86_64.
Co-authored-by: binarycat <binarycat@envs.net>
Target option to require explicit cpu
Some targets have many different CPUs and no generic CPU that can be used as a default. For these targets, the user needs to explicitly specify a CPU through `-C target-cpu=`.
Add an option for targets and an error message if no CPU is set.
This affects the proposed amdgpu and avr targets.
amdgpu tracking issue: #135024
AVR MCP: https://github.com/rust-lang/compiler-team/issues/800
Remove -Zinline-in-all-cgus and clean up tests/codegen-units/
Implementation of https://github.com/rust-lang/compiler-team/issues/814
I've taken some liberties with cleaning up the CGU partitioning tests, because that's the only place this flag was used and also mattered. I've often fought a lot with the contents of `tests/codegen-units` and it has never been clear to me when a test failure indicates a problem with my changes as opposed to a test just needing to be manually blessed. Hopefully the combination of the new README, new comments, and using `-Zprint-mono-items=lazy` in the partitioning tests improves that.
I've also deleted some of the `tests/run-make/sepcomp` tests. I think all the "sepcomp" tests have been obviated for years by better-designed (less flaky, clearer failures) test suites, but here I'm just deleting the ones I'm confident in.
This needs more time to bake before we turn it on. Turning it on early risks people silencing the warning indefinitely, before we have the chance to make it less noisy.
- Don't show environment variables. Seeing PATH is almost never useful, and it can be extremely long.
- For .rlibs in the sysroot, replace crate hashes with a `"-*"` string. This will expand to the full crate name when pasted into the shell.
- Move `.rlib` to outside the glob.
- Abbreviate the sysroot path to `<sysroot>` wherever it appears in the arguments.
This also adds an example of the linker output as a run-make test. Currently it only runs on x86_64-unknown-linux-gnu, because each platform has its own linker arguments. So that it's stable across machines, pass BUILD_ROOT as an argument through compiletest through to run-make tests.
- Only use linker-flavor=gnu-cc if we're actually going to compare the output. It doesn't exist on MacOS.
tests: Port `jobserver-error` to rmake.rs
Part of #121876.
This PR ports `tests/run-make/jobserver-error` to rmake.rs, and is basically #128789 slightly adjusted.
The complexity involved here is mostly how to get `/dev/null/` piping to fd 3 working with std `Command`, whereas with a shell this is much easier (as is evident with the `Makefile` version).
Supersedes #128789.
This PR is co-authored with `@Oneirical` and `@coolreader18.`
try-job: aarch64-gnu
try-job: i686-gnu-1
try-job: x86_64-gnu-debug
try-job: x86_64-gnu-llvm-18-1
Add test for checking used glibc symbols
This test checks that we do not use too new glibc symbols in the compiler on x64 GNU Linux, in order not to break our [glibc promises](https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html).
One thing that isn't solved in the PR yet is to make sure that this test will only run on `dist` CI, more specifically on the `dist-x86_64-linux` runner, in the opt-dist post-optimization tests (it can fail elsewhere, that doesn't matter). Any suggestions on how to do that are welcome.
Fixes: https://github.com/rust-lang/rust/issues/134037
r? `@jieyouxu`
tests: delete `cat-and-grep-sanity-check`
Part of #121876.
All remaining `Makefile`s have open PRs that do not rely on platform `cat` or `grep` or the `cat-and-grep` script.
remove support for the (unstable) #[start] attribute
As explained by `@Noratrieb:`
`#[start]` should be deleted. It's nothing but an accidentally leaked implementation detail that's a not very useful mix between "portable" entrypoint logic and bad abstraction.
I think the way the stable user-facing entrypoint should work (and works today on stable) is pretty simple:
- `std`-using cross-platform programs should use `fn main()`. the compiler, together with `std`, will then ensure that code ends up at `main` (by having a platform-specific entrypoint that gets directed through `lang_start` in `std` to `main` - but that's just an implementation detail)
- `no_std` platform-specific programs should use `#![no_main]` and define their own platform-specific entrypoint symbol with `#[no_mangle]`, like `main`, `_start`, `WinMain` or `my_embedded_platform_wants_to_start_here`. most of them only support a single platform anyways, and need cfg for the different platform's ways of passing arguments or other things *anyways*
`#[start]` is in a super weird position of being neither of those two. It tries to pretend that it's cross-platform, but its signature is a total lie. Those arguments are just stubbed out to zero on ~~Windows~~ wasm, for example. It also only handles the platform-specific entrypoints for a few platforms that are supported by `std`, like Windows or Unix-likes. `my_embedded_platform_wants_to_start_here` can't use it, and neither could a libc-less Linux program.
So we have an attribute that only works in some cases anyways, that has a signature that's a total lie (and a signature that, as I might want to add, has changed recently, and that I definitely would not be comfortable giving *any* stability guarantees on), and where there's a pretty easy way to get things working without it in the first place.
Note that this feature has **not** been RFCed in the first place.
*This comment was posted [in May](https://github.com/rust-lang/rust/issues/29633#issuecomment-2088596042) and so far nobody spoke up in that issue with a usecase that would require keeping the attribute.*
Closes https://github.com/rust-lang/rust/issues/29633
try-job: x86_64-gnu-nopt
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: test-various
