This commit works around the newly-introduced LLVM shared library.
This is needed such that llvm-config run from
librustc_llvm's build script can correctly locate it's own LLVM, not the
one in stage0/lib. The LLVM build system uses the DT_RUNPATH/RUNPATH
header within the llvm-config binary, which we want to use, but because
Cargo always adds the host compiler's "libdir" (stage0/lib in our
case) to the dynamic linker's search path, we weren't properly finding
the freshly-built LLVM in llvm/lib. By restoring the environment
variable setting the search path to what bootstrap sees, the problem is
resolved and librustc_llvm correctly links and finds the appropriate
LLVM.
Several run-make-fulldeps tests are also updated with similar handling.
Provide the option to use libc++ even on all platforms
This is the default on platforms which use libc++ as the default C++
library but this option allows using libc++ on others as well.
When building a distributed compiler on Linux where we use ThinLTO to
create the LLVM shared object this commit switches the compiler to
dynamically linking that LLVM artifact instead of statically linking to
LLVM. The primary goal here is to reduce CI compile times, avoiding two+
ThinLTO builds of all of LLVM. By linking dynamically to LLVM we'll
reuse the one ThinLTO step done by LLVM's build itself.
Lots of discussion about this change can be found [here] and down. A
perf run will show whether this is worth it or not!
[here]: https://github.com/rust-lang/rust/pull/53245#issuecomment-417015334
When building a distributed compiler on Linux where we use ThinLTO to
create the LLVM shared object this commit switches the compiler to
dynamically linking that LLVM artifact instead of statically linking to
LLVM. The primary goal here is to reduce CI compile times, avoiding two+
ThinLTO builds of all of LLVM. By linking dynamically to LLVM we'll
reuse the one ThinLTO step done by LLVM's build itself.
Lots of discussion about this change can be found [here] and down. A
perf run will show whether this is worth it or not!
[here]: https://github.com/rust-lang/rust/pull/53245#issuecomment-417015334
This was an accidental regression from #56092, but for `no_std` targets
being built and distributed we want to be sure to activate the
compiler-builtins `mem` feature which demangles important memory-related
intrinsics.
Ever since we added a Cargo-based build system for the compiler the
standard library has always been a little special, it's never been able
to depend on crates.io crates for runtime dependencies. This has been a
result of various limitations, namely that Cargo doesn't understand that
crates from crates.io depend on libcore, so Cargo tries to build crates
before libcore is finished.
I had an idea this afternoon, however, which lifts the strategy
from #52919 to directly depend on crates.io crates from the standard
library. After all is said and done this removes a whopping three
submodules that we need to manage!
The basic idea here is that for any crate `std` depends on it adds an
*optional* dependency on an empty crate on crates.io, in this case named
`rustc-std-workspace-core`. This crate is overridden via `[patch]` in
this repository to point to a local crate we write, and *that* has a
`path` dependency on libcore.
Note that all `no_std` crates also depend on `compiler_builtins`, but if
we're not using submodules we can publish `compiler_builtins` to
crates.io and all crates can depend on it anyway! The basic strategy
then looks like:
* The standard library (or some transitive dep) decides to depend on a
crate `foo`.
* The standard library adds
```toml
[dependencies]
foo = { version = "0.1", features = ['rustc-dep-of-std'] }
```
* The crate `foo` has an optional dependency on `rustc-std-workspace-core`
* The crate `foo` has an optional dependency on `compiler_builtins`
* The crate `foo` has a feature `rustc-dep-of-std` which activates these
crates and any other necessary infrastructure in the crate.
A sample commit for `dlmalloc` [turns out to be quite simple][commit].
After that all `no_std` crates should largely build "as is" and still be
publishable on crates.io! Notably they should be able to continue to use
stable Rust if necessary, since the `rename-dependency` feature of Cargo
is soon stabilizing.
As a proof of concept, this commit removes the `dlmalloc`,
`libcompiler_builtins`, and `libc` submodules from this repository. Long
thorns in our side these are now gone for good and we can directly
depend on crates.io! It's hoped that in the long term we can bring in
other crates as necessary, but for now this is largely intended to
simply make it easier to manage these crates and remove submodules.
This should be a transparent non-breaking change for all users, but one
possible stickler is that this almost for sure breaks out-of-tree
`std`-building tools like `xargo` and `cargo-xbuild`. I think it should
be relatively easy to get them working, however, as all that's needed is
an entry in the `[patch]` section used to build the standard library.
Hopefully we can work with these tools to solve this problem!
[commit]: 28ee12db81
This commit replaces many usages of `File::open` and reading or writing
with `fs::read_to_string`, `fs::read` and `fs::write`. This reduces code
complexity, and will improve performance for most reads, since the
functions allocate the buffer to be the size of the file.
I believe that this commit will not impact behavior in any way, so some
matches will check the error kind in case the file was not valid UTF-8.
Some of these cases may not actually care about the error.
This commit removes all jemalloc related submodules, configuration, etc,
from the bootstrap, from the standard library, and from the compiler.
This will be followed up with a change to use jemalloc specifically as
part of rustc on blessed platforms.
remove useless format!()s
remove redundant field names in a few struct initializations
pass slice instead of a vector to a function
use is_empty() instead of comparisons to .len()
No functional change intended.
* Make it influence the behavior of the compiled rustc, rather than
just the rustc build system. That is, if verify_llvm_ir=true,
even manual invocations of the built rustc will verify LLVM IR.
* Enable verification of LLVM IR in CI, for non-deploy and
deploy-alt builds. This is similar to how LLVM assertions are
handled.
Currently we ship sanitizer libraries as they're built, but these names
unfortunately conflict with the names of the sanitizer libraries
installed on the system. If a crate, for example, links in C code that
wants to use the system sanitizer and the Rust code doesn't use
sanitizers at all, then using `cargo` may accidentally pull in the
Rust-installed sanitizer library due to a conflict in names.
This change is intended to be entirely transparent for Rust users of
sanitizers, it should only hopefully improve our story with other users!
Closes#54134
Clean up dependency tracking in Rustbuild [2/2]
Make `clear_if_dirty` calls in `Builder::cargo` with stamp dependencies for the given Mode.
Continuation of #50904
Ref issue #50509
r? @Mark-Simulacrum
This commit updates the debuginfo that is encoded in all of our released
artifacts by default. Currently it has paths like `/checkout/src/...` but these
are a little inconsistent and have changed over time. This commit instead
attempts to actually define the file paths in our debuginfo to be consistent
between releases.
All debuginfo paths are now intended to be `/rustc/$sha` where `$sha` is the git
sha of the released compiler. Sub-paths are all paths into the git repo at that
`$sha`.
This allows clearing it out and building it separately from the
compiler. Since it's essentially a different and separate crate this
makes sense to do, each cargo invocation should generally happen in its
own directory.
Previously we'd attempt to recompile them and that would fail since
we've essentially not built the entire compiler yet, or we're faking
that fact. This commit should make us ignore the codegen backend build
as well.
Unlike the other compile steps, there is no CodegenBackendLink step that
we run here, because that is done later as a part of assembling the
final compiler and as an explicit function call.
rename rustc's lld to rust-lld
to not shadow the system installed LLD when linking with LLD.
Before:
- `-C linker=lld -Z linker-flavor=ld.lld` uses rustc's LLD
- It's not possible to use a system installed LLD that's named `lld`
With this commit:
- `-C linker=rust-lld -Z linker-flavor=ld.lld` uses rustc's LLD
- `-C linker=lld -Z linker-flavor=ld.lld` uses the system installed LLD
we don't offer guarantees about the availability of LLD in the rustc sysroot so we can rename the tool as long as we don't break the wasm32-unknown-unknown target which depends on it.
r? @alexcrichton we discussed this before
Previously, the --keep-stage argument would only function for compilers
that were depended on by future stages. For example, if trying to build
a stage 1 compiler you could --keep-stage 0 to avoid re-building the
stage 0 compiler. However, this is often not what users want in
practice.
The new implementation essentially skips builds all higher stages of the
compiler, so an argument of 1 to keep-stage will skip rebuilds of the
libraries, just linking them into the sysroot. This is unlikely to work
well in cases where metadata or similar changes have been made, but is
likely fine otherwise.
This change is somewhat untested, but since it shouldn't have any effect
except with --keep-stage, I don't see that as a large problem.
Previously Cargo would hardlink all the dependencies into the "root" as
foo.dll and the `toplevel` array would get populated with these, but
that's no longer the case. Instead, cargo will only do this for the
final artifacts/final libraries.
Rustbuild is updated to continue looping through the artifacts mentioned
instead of early-returning. This should fix the bug.
@alexcrichton found the cause of this and suggested this fix.
to not shadow the system installed LLD when linking with LLD.
Before:
- `-C linker=lld -Z linker-flavor=ld.lld` uses rustc's LLD
- It's not possible to use a system installed LLD that's named `lld`
With this commit:
- `-C linker=rust-lld -Z linker-flavor=ld.lld` uses rustc's LLD
- `-C linker=lld -Z linker-flavor=ld.lld` uses the system installed LLD
Switch to bootstrapping from 1.27
It's possible the Float trait could be removed from core, but I couldn't tell whether it was intended to be removed or not. @SimonSapin may be able to comment more here; we can presumably also do that in a follow up PR as this one is already quite large.
In #49289, rustc was changed to emit metadata for binaries, which made
it so that the librustc.rmeta file created when compiling librustc was
overwritten by the rustc-main compilation. This commit renames the
rustc-main binary to avoid this problem.
https://github.com/rust-lang/cargo/issues/5524 has also been filed to
see if Cargo can learn to warn on this situation instead of leaving it
for the user to debug.
