rust-lang/rust
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Auto merge of #144303 - Kobzol:bootstrap-tool-cleanup, r=jieyouxu

Browse Source 
Consolidate staging for `rustc_private` tools

This PR continues bootstrap refactoring, this time by consolidating staging for `Mode::ToolRustc` tools. This refactoring was in the critical path of refactoring `test`/`dist`/`clippy`/`doc` steps, and getting rid of the rmeta/rlib sysroot copy, because tools are pervasive and they are being used for a lot of things in bootstrap.

The main idea is to explicitly model the fact that a stage N `Mode::ToolRustc` tool always works with two different compilers:
- Stage N-1 rustc (`build_compiler`) builds stage N rustc (`target_compiler`)
- Rlib artifacts from stage N rustc are copied to the sysroot of stage N-1 rustc
- Stage N-1 rustc builds the (stage N) tool itself, the tool links to the rlib artifacts of the stage N rustc

Before, the code often used `compiler`, which meant sometimes the build compiler, sometimes the target compiler, and sometimes neither (looking at you, `download-rustc`). This is especially annoying when you get to a situation where you have an install step that invokes a dist step that invokes a tool build step, where *some* compiler is being propagated through, without it being clear what does that compiler represent. This refactoring hopefully makes that clearer and more explicit. It also gets rid of a few `builder.ensure(Rustc(...))` calls within bootstrap, which is always nice.

`Rustdoc` needs to be handled a bit specially, because it acts as a compiler itself, I documented that in the changes.

It wasn't practical to do these refactorings in multiple PRs, so I did it all in one PR. The meat of the change is 9ee6d1c1ed112c3dcfb5684b33772b136df0dca3.

I tested manually that `x build rustdoc` and `x build miri` still works even with `download-rustc`, although I cannot promise any extra support for `download-rustc`, IMO we will just have to reimplement it from scratch in a different way.

As usually, I did some drive-by refactorings to bootstrap, trying to document and clarify things, add more step metadata and tests.

Since these changes broke Cargo, which was incorrectly using `Mode::ToolRustc`, I also changed cargo to `ToolTarget` in this PR.

Best reviewed commit-by-commit (note that I renamed `link_compiler` to `target_compiler`, in accordance to the rest of bootstrap, in the last commit).

r? `@jieyouxu`

try-job: x86_64-gnu-aux
try-job: x86_64-msvc-ext1
This commit is contained in:
bors
2025-08-05 11:34:14 +00:00
parent 0f35336396 ac28b5b93a
commit 213d946a38
12 changed files with 741 additions and 450 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
src/bootstrap/src/core/build_steps/compile.rs
View File

@@ -19,7 +19,7 @@ use serde_derive::Deserialize;
use tracing::{instrument, span};
use crate::core::build_steps::gcc::{Gcc, add_cg_gcc_cargo_flags};
use crate::core::build_steps::tool::{SourceType, copy_lld_artifacts};
use crate::core::build_steps::tool::{RustcPrivateCompilers, SourceType, copy_lld_artifacts};
use crate::core::build_steps::{dist, llvm};
use crate::core::builder;
use crate::core::builder::{
@@ -1131,7 +1131,7 @@ impl Step for Rustc {
cargo.env("RUSTC_BOLT_LINK_FLAGS", "1");
}
let _guard = builder.msg_sysroot_tool(
let _guard = builder.msg_rustc_tool(
Kind::Build,
build_compiler.stage,
format_args!("compiler artifacts{}", crate_description(&self.crates)),
@@ -1544,9 +1544,8 @@ impl Step for RustcLink {
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct CodegenBackend {
pub target: TargetSelection,
pub compiler: Compiler,
pub backend: CodegenBackendKind,
compilers: RustcPrivateCompilers,
backend: CodegenBackendKind,
}
fn needs_codegen_config(run: &RunConfig<'_>) -> bool {
@@ -1610,8 +1609,11 @@ impl Step for CodegenBackend {
}
run.builder.ensure(CodegenBackend {
target: run.target,
compiler: run.builder.compiler(run.builder.top_stage, run.build_triple()),
compilers: RustcPrivateCompilers::new(
run.builder,
run.builder.top_stage,
run.target,
),
backend: backend.clone(),
});
}
@@ -1624,20 +1626,17 @@ impl Step for CodegenBackend {
name = "CodegenBackend::run",
skip_all,
fields(
compiler = ?self.compiler,
target = ?self.target,
backend = ?self.target,
compilers = ?self.compilers,
backend = ?self.backend,
),
),
)]
fn run(self, builder: &Builder<'_>) {
let compiler = self.compiler;
let target = self.target;
let backend = self.backend;
let target = self.compilers.target();
let build_compiler = self.compilers.build_compiler();
builder.ensure(Rustc::new(compiler, target));
if builder.config.keep_stage.contains(&compiler.stage) {
if builder.config.keep_stage.contains(&build_compiler.stage) {
trace!("`keep-stage` requested");
builder.info(
"WARNING: Using a potentially old codegen backend. \
@@ -1648,17 +1647,11 @@ impl Step for CodegenBackend {
return;
}
let compiler_to_use = builder.compiler_for(compiler.stage, compiler.host, target);
if compiler_to_use != compiler {
builder.ensure(CodegenBackend { compiler: compiler_to_use, target, backend });
return;
}
let out_dir = builder.cargo_out(compiler, Mode::Codegen, target);
let out_dir = builder.cargo_out(build_compiler, Mode::Codegen, target);
let mut cargo = builder::Cargo::new(
builder,
compiler,
build_compiler,
Mode::Codegen,
SourceType::InTree,
target,
@@ -1679,8 +1672,13 @@ impl Step for CodegenBackend {
let tmp_stamp = BuildStamp::new(&out_dir).with_prefix("tmp");
let _guard =
builder.msg_build(compiler, format_args!("codegen backend {}", backend.name()), target);
let _guard = builder.msg_rustc_tool(
Kind::Build,
build_compiler.stage,
format_args!("codegen backend {}", backend.name()),
build_compiler.host,
target,
);
let files = run_cargo(builder, cargo, vec![], &tmp_stamp, vec![], false, false);
if builder.config.dry_run() {
return;
@@ -1700,10 +1698,17 @@ impl Step for CodegenBackend {
f.display()
);
}
let stamp = build_stamp::codegen_backend_stamp(builder, compiler, target, &backend);
let stamp = build_stamp::codegen_backend_stamp(builder, build_compiler, target, &backend);
let codegen_backend = codegen_backend.to_str().unwrap();
t!(stamp.add_stamp(codegen_backend).write());
}
fn metadata(&self) -> Option<StepMetadata> {
Some(
StepMetadata::build(&self.backend.crate_name(), self.compilers.target())
.built_by(self.compilers.build_compiler()),
)
}
}
/// Creates the `codegen-backends` folder for a compiler that's about to be
@@ -2190,8 +2195,10 @@ impl Step for Assemble {
continue;
}
builder.ensure(CodegenBackend {
compiler: build_compiler,
target: target_compiler.host,
compilers: RustcPrivateCompilers::from_build_and_target_compiler(
build_compiler,
target_compiler,
),
backend: backend.clone(),
});
}

99
src/bootstrap/src/core/build_steps/dist.rs
View File

@@ -20,7 +20,7 @@ use object::read::archive::ArchiveFile;
use tracing::instrument;
use crate::core::build_steps::doc::DocumentationFormat;
use crate::core::build_steps::tool::{self, Tool};
use crate::core::build_steps::tool::{self, RustcPrivateCompilers, Tool};
use crate::core::build_steps::vendor::{VENDOR_DIR, Vendor};
use crate::core::build_steps::{compile, llvm};
use crate::core::builder::{Builder, Kind, RunConfig, ShouldRun, Step, StepMetadata};
@@ -425,19 +425,20 @@ impl Step for Rustc {
.as_ref()
.is_none_or(|tools| tools.iter().any(|tool| tool == "rustdoc"))
{
let rustdoc = builder.rustdoc(compiler);
let rustdoc = builder.rustdoc_for_compiler(compiler);
builder.install(&rustdoc, &image.join("bin"), FileType::Executable);
}
let ra_proc_macro_srv_compiler =
builder.compiler_for(compiler.stage, builder.config.host_target, compiler.host);
builder.ensure(compile::Rustc::new(ra_proc_macro_srv_compiler, compiler.host));
let compilers = RustcPrivateCompilers::from_build_compiler(
builder,
ra_proc_macro_srv_compiler,
compiler.host,
);
if let Some(ra_proc_macro_srv) = builder.ensure_if_default(
tool::RustAnalyzerProcMacroSrv {
compiler: ra_proc_macro_srv_compiler,
target: compiler.host,
},
tool::RustAnalyzerProcMacroSrv::from_compilers(compilers),
builder.kind,
) {
let dst = image.join("libexec");
@@ -1172,7 +1173,7 @@ impl Step for PlainSourceTarball {
#[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)]
pub struct Cargo {
pub compiler: Compiler,
pub build_compiler: Compiler,
pub target: TargetSelection,
}
@@ -1188,7 +1189,7 @@ impl Step for Cargo {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Cargo {
compiler: run.builder.compiler_for(
build_compiler: run.builder.compiler_for(
run.builder.top_stage,
run.builder.config.host_target,
run.target,
@@ -1198,12 +1199,10 @@ impl Step for Cargo {
}
fn run(self, builder: &Builder<'_>) -> Option<GeneratedTarball> {
let compiler = self.compiler;
let build_compiler = self.build_compiler;
let target = self.target;
builder.ensure(compile::Rustc::new(compiler, target));
let cargo = builder.ensure(tool::Cargo { compiler, target });
let cargo = builder.ensure(tool::Cargo::from_build_compiler(build_compiler, target));
let src = builder.src.join("src/tools/cargo");
let etc = src.join("src/etc");
@@ -1228,7 +1227,7 @@ impl Step for Cargo {
#[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)]
pub struct RustAnalyzer {
pub compiler: Compiler,
pub build_compiler: Compiler,
pub target: TargetSelection,
}
@@ -1244,7 +1243,7 @@ impl Step for RustAnalyzer {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(RustAnalyzer {
compiler: run.builder.compiler_for(
build_compiler: run.builder.compiler_for(
run.builder.top_stage,
run.builder.config.host_target,
run.target,
@@ -1254,12 +1253,11 @@ impl Step for RustAnalyzer {
}
fn run(self, builder: &Builder<'_>) -> Option<GeneratedTarball> {
let compiler = self.compiler;
let target = self.target;
let compilers =
RustcPrivateCompilers::from_build_compiler(builder, self.build_compiler, self.target);
builder.ensure(compile::Rustc::new(compiler, target));
let rust_analyzer = builder.ensure(tool::RustAnalyzer { compiler, target });
let rust_analyzer = builder.ensure(tool::RustAnalyzer::from_compilers(compilers));
let mut tarball = Tarball::new(builder, "rust-analyzer", &target.triple);
tarball.set_overlay(OverlayKind::RustAnalyzer);
@@ -1270,9 +1268,9 @@ impl Step for RustAnalyzer {
}
}
#[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)]
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct Clippy {
pub compiler: Compiler,
pub build_compiler: Compiler,
pub target: TargetSelection,
}
@@ -1288,7 +1286,7 @@ impl Step for Clippy {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Clippy {
compiler: run.builder.compiler_for(
build_compiler: run.builder.compiler_for(
run.builder.top_stage,
run.builder.config.host_target,
run.target,
@@ -1298,16 +1296,15 @@ impl Step for Clippy {
}
fn run(self, builder: &Builder<'_>) -> Option<GeneratedTarball> {
let compiler = self.compiler;
let target = self.target;
builder.ensure(compile::Rustc::new(compiler, target));
let compilers =
RustcPrivateCompilers::from_build_compiler(builder, self.build_compiler, target);
// Prepare the image directory
// We expect clippy to build, because we've exited this step above if tool
// state for clippy isn't testing.
let clippy = builder.ensure(tool::Clippy { compiler, target });
let cargoclippy = builder.ensure(tool::CargoClippy { compiler, target });
let clippy = builder.ensure(tool::Clippy::from_compilers(compilers));
let cargoclippy = builder.ensure(tool::CargoClippy::from_compilers(compilers));
let mut tarball = Tarball::new(builder, "clippy", &target.triple);
tarball.set_overlay(OverlayKind::Clippy);
@@ -1319,9 +1316,9 @@ impl Step for Clippy {
}
}
#[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)]
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct Miri {
pub compiler: Compiler,
pub build_compiler: Compiler,
pub target: TargetSelection,
}
@@ -1337,7 +1334,7 @@ impl Step for Miri {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Miri {
compiler: run.builder.compiler_for(
build_compiler: run.builder.compiler_for(
run.builder.top_stage,
run.builder.config.host_target,
run.target,
@@ -1354,15 +1351,12 @@ impl Step for Miri {
return None;
}
let compiler = self.compiler;
let target = self.target;
let compilers =
RustcPrivateCompilers::from_build_compiler(builder, self.build_compiler, self.target);
let miri = builder.ensure(tool::Miri::from_compilers(compilers));
let cargomiri = builder.ensure(tool::CargoMiri::from_compilers(compilers));
builder.ensure(compile::Rustc::new(compiler, target));
let miri = builder.ensure(tool::Miri { compiler, target });
let cargomiri = builder.ensure(tool::CargoMiri { compiler, target });
let mut tarball = Tarball::new(builder, "miri", &target.triple);
let mut tarball = Tarball::new(builder, "miri", &self.target.triple);
tarball.set_overlay(OverlayKind::Miri);
tarball.is_preview(true);
tarball.add_file(&miri.tool_path, "bin", FileType::Executable);
@@ -1466,9 +1460,9 @@ impl Step for CodegenBackend {
}
}
#[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)]
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct Rustfmt {
pub compiler: Compiler,
pub build_compiler: Compiler,
pub target: TargetSelection,
}
@@ -1484,7 +1478,7 @@ impl Step for Rustfmt {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Rustfmt {
compiler: run.builder.compiler_for(
build_compiler: run.builder.compiler_for(
run.builder.top_stage,
run.builder.config.host_target,
run.target,
@@ -1494,14 +1488,13 @@ impl Step for Rustfmt {
}
fn run(self, builder: &Builder<'_>) -> Option<GeneratedTarball> {
let compiler = self.compiler;
let target = self.target;
let compilers =
RustcPrivateCompilers::from_build_compiler(builder, self.build_compiler, self.target);
builder.ensure(compile::Rustc::new(compiler, target));
let rustfmt = builder.ensure(tool::Rustfmt::from_compilers(compilers));
let cargofmt = builder.ensure(tool::Cargofmt::from_compilers(compilers));
let rustfmt = builder.ensure(tool::Rustfmt { compiler, target });
let cargofmt = builder.ensure(tool::Cargofmt { compiler, target });
let mut tarball = Tarball::new(builder, "rustfmt", &target.triple);
let mut tarball = Tarball::new(builder, "rustfmt", &self.target.triple);
tarball.set_overlay(OverlayKind::Rustfmt);
tarball.is_preview(true);
tarball.add_file(&rustfmt.tool_path, "bin", FileType::Executable);
@@ -1548,7 +1541,7 @@ impl Step for Extended {
let mut built_tools = HashSet::new();
macro_rules! add_component {
($name:expr => $step:expr) => {
if let Some(tarball) = builder.ensure_if_default($step, Kind::Dist) {
if let Some(Some(tarball)) = builder.ensure_if_default($step, Kind::Dist) {
tarballs.push(tarball);
built_tools.insert($name);
}
@@ -1568,12 +1561,12 @@ impl Step for Extended {
add_component!("rust-docs" => Docs { host: target });
add_component!("rust-json-docs" => JsonDocs { host: target });
add_component!("cargo" => Cargo { compiler, target });
add_component!("rustfmt" => Rustfmt { compiler, target });
add_component!("rust-analyzer" => RustAnalyzer { compiler, target });
add_component!("cargo" => Cargo { build_compiler: compiler, target });
add_component!("rustfmt" => Rustfmt { build_compiler: compiler, target });
add_component!("rust-analyzer" => RustAnalyzer { build_compiler: compiler, target });
add_component!("llvm-components" => LlvmTools { target });
add_component!("clippy" => Clippy { compiler, target });
add_component!("miri" => Miri { compiler, target });
add_component!("clippy" => Clippy { build_compiler: compiler, target });
add_component!("miri" => Miri { build_compiler: compiler, target });
add_component!("analysis" => Analysis { compiler, target });
add_component!("rustc-codegen-cranelift" => CodegenBackend {
compiler: builder.compiler(stage, target),

16
src/bootstrap/src/core/build_steps/doc.rs
View File

@@ -149,7 +149,7 @@ impl<P: Step> Step for RustbookSrc<P> {
let mut rustbook_cmd = builder.tool_cmd(Tool::Rustbook);
if let Some(compiler) = self.rustdoc_compiler {
let mut rustdoc = builder.rustdoc(compiler);
let mut rustdoc = builder.rustdoc_for_compiler(compiler);
rustdoc.pop();
let old_path = env::var_os("PATH").unwrap_or_default();
let new_path =
@@ -365,7 +365,7 @@ impl Step for Standalone {
}
let html = out.join(filename).with_extension("html");
let rustdoc = builder.rustdoc(compiler);
let rustdoc = builder.rustdoc_for_compiler(compiler);
if up_to_date(&path, &html)
&& up_to_date(&footer, &html)
&& up_to_date(&favicon, &html)
@@ -463,7 +463,7 @@ impl Step for Releases {
let html = out.join("releases.html");
let tmppath = out.join("releases.md");
let inpath = builder.src.join("RELEASES.md");
let rustdoc = builder.rustdoc(compiler);
let rustdoc = builder.rustdoc_for_compiler(compiler);
if !up_to_date(&inpath, &html)
|| !up_to_date(&footer, &html)
|| !up_to_date(&favicon, &html)
@@ -811,7 +811,7 @@ impl Step for Rustc {
let compiler = builder.compiler(stage, builder.config.host_target);
builder.std(compiler, builder.config.host_target);
let _guard = builder.msg_sysroot_tool(
let _guard = builder.msg_rustc_tool(
Kind::Doc,
stage,
format!("compiler{}", crate_description(&self.crates)),
@@ -901,6 +901,10 @@ impl Step for Rustc {
builder.open_in_browser(index);
}
}
fn metadata(&self) -> Option<StepMetadata> {
Some(StepMetadata::doc("rustc", self.target).stage(self.stage))
}
}
macro_rules! tool_doc {
@@ -1018,6 +1022,10 @@ macro_rules! tool_doc {
})?
}
}
fn metadata(&self) -> Option<StepMetadata> {
Some(StepMetadata::doc(stringify!($tool), self.target))
}
}
}
}

10
src/bootstrap/src/core/build_steps/install.rs
View File

@@ -215,13 +215,13 @@ install!((self, builder, _config),
};
Cargo, alias = "cargo", Self::should_build(_config), only_hosts: true, {
let tarball = builder
.ensure(dist::Cargo { compiler: self.compiler, target: self.target })
.ensure(dist::Cargo { build_compiler: self.compiler, target: self.target })
.expect("missing cargo");
install_sh(builder, "cargo", self.compiler.stage, Some(self.target), &tarball);
};
RustAnalyzer, alias = "rust-analyzer", Self::should_build(_config), only_hosts: true, {
if let Some(tarball) =
builder.ensure(dist::RustAnalyzer { compiler: self.compiler, target: self.target })
builder.ensure(dist::RustAnalyzer { build_compiler: self.compiler, target: self.target })
{
install_sh(builder, "rust-analyzer", self.compiler.stage, Some(self.target), &tarball);
} else {
@@ -232,12 +232,12 @@ install!((self, builder, _config),
};
Clippy, alias = "clippy", Self::should_build(_config), only_hosts: true, {
let tarball = builder
.ensure(dist::Clippy { compiler: self.compiler, target: self.target })
.ensure(dist::Clippy { build_compiler: self.compiler, target: self.target })
.expect("missing clippy");
install_sh(builder, "clippy", self.compiler.stage, Some(self.target), &tarball);
};
Miri, alias = "miri", Self::should_build(_config), only_hosts: true, {
if let Some(tarball) = builder.ensure(dist::Miri { compiler: self.compiler, target: self.target }) {
if let Some(tarball) = builder.ensure(dist::Miri { build_compiler: self.compiler, target: self.target }) {
install_sh(builder, "miri", self.compiler.stage, Some(self.target), &tarball);
} else {
// Miri is only available on nightly
@@ -257,7 +257,7 @@ install!((self, builder, _config),
};
Rustfmt, alias = "rustfmt", Self::should_build(_config), only_hosts: true, {
if let Some(tarball) = builder.ensure(dist::Rustfmt {
compiler: self.compiler,
build_compiler: self.compiler,
target: self.target
}) {
install_sh(builder, "rustfmt", self.compiler.stage, Some(self.target), &tarball);

2
src/bootstrap/src/core/build_steps/perf.rs
View File

@@ -157,7 +157,7 @@ Consider setting `rust.debuginfo-level = 1` in `bootstrap.toml`."#);
if let Some(opts) = args.cmd.shared_opts()
&& opts.profiles.contains(&Profile::Doc)
{
builder.ensure(Rustdoc { compiler });
builder.ensure(Rustdoc { target_compiler: compiler });
}
let sysroot = builder.ensure(Sysroot::new(compiler));

14
src/bootstrap/src/core/build_steps/run.rs
View File

@@ -9,7 +9,7 @@ use clap_complete::{Generator, shells};
use crate::core::build_steps::dist::distdir;
use crate::core::build_steps::test;
use crate::core::build_steps::tool::{self, SourceType, Tool};
use crate::core::build_steps::tool::{self, RustcPrivateCompilers, SourceType, Tool};
use crate::core::build_steps::vendor::{Vendor, default_paths_to_vendor};
use crate::core::builder::{Builder, Kind, RunConfig, ShouldRun, Step};
use crate::core::config::TargetSelection;
@@ -135,13 +135,13 @@ impl Step for Miri {
}
// This compiler runs on the host, we'll just use it for the target.
let target_compiler = builder.compiler(stage, target);
let miri_build = builder.ensure(tool::Miri { compiler: target_compiler, target });
// Rustc tools are off by one stage, so use the build compiler to run miri.
let compilers = RustcPrivateCompilers::new(builder, stage, target);
let miri_build = builder.ensure(tool::Miri::from_compilers(compilers));
let host_compiler = miri_build.build_compiler;
// Get a target sysroot for Miri.
let miri_sysroot = test::Miri::build_miri_sysroot(builder, target_compiler, target);
let miri_sysroot =
test::Miri::build_miri_sysroot(builder, compilers.target_compiler(), target);
// # Run miri.
// Running it via `cargo run` as that figures out the right dylib path.
@@ -465,8 +465,8 @@ impl Step for Rustfmt {
std::process::exit(1);
}
let compiler = builder.compiler(stage, host);
let rustfmt_build = builder.ensure(tool::Rustfmt { compiler, target: host });
let compilers = RustcPrivateCompilers::new(builder, stage, host);
let rustfmt_build = builder.ensure(tool::Rustfmt::from_compilers(compilers));
let mut rustfmt = tool::prepare_tool_cargo(
builder,

242
src/bootstrap/src/core/build_steps/test.rs
View File

@@ -4,10 +4,12 @@
//! However, this contains ~all test parts we expect people to be able to build and run locally.
use std::collections::HashSet;
use std::env::split_paths;
use std::ffi::{OsStr, OsString};
use std::path::{Path, PathBuf};
use std::{env, fs, iter};
use build_helper::exit;
#[cfg(feature = "tracing")]
use tracing::instrument;
@@ -17,7 +19,10 @@ use crate::core::build_steps::gcc::{Gcc, add_cg_gcc_cargo_flags};
use crate::core::build_steps::llvm::get_llvm_version;
use crate::core::build_steps::run::get_completion_paths;
use crate::core::build_steps::synthetic_targets::MirOptPanicAbortSyntheticTarget;
use crate::core::build_steps::tool::{self, COMPILETEST_ALLOW_FEATURES, SourceType, Tool};
use crate::core::build_steps::tool::{
self, COMPILETEST_ALLOW_FEATURES, RustcPrivateCompilers, SourceType, Tool, ToolTargetBuildMode,
get_tool_target_compiler,
};
use crate::core::build_steps::toolstate::ToolState;
use crate::core::build_steps::{compile, dist, llvm};
use crate::core::builder::{
@@ -29,8 +34,8 @@ use crate::core::config::flags::{Subcommand, get_completion};
use crate::utils::build_stamp::{self, BuildStamp};
use crate::utils::exec::{BootstrapCommand, command};
use crate::utils::helpers::{
self, LldThreads, add_rustdoc_cargo_linker_args, dylib_path, dylib_path_var, linker_args,
linker_flags, t, target_supports_cranelift_backend, up_to_date,
self, LldThreads, add_dylib_path, add_rustdoc_cargo_linker_args, dylib_path, dylib_path_var,
linker_args, linker_flags, t, target_supports_cranelift_backend, up_to_date,
};
use crate::utils::render_tests::{add_flags_and_try_run_tests, try_run_tests};
use crate::{CLang, CodegenBackendKind, DocTests, GitRepo, Mode, PathSet, debug, envify};
@@ -226,7 +231,7 @@ impl Step for HtmlCheck {
/// order to test cargo.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Cargotest {
stage: u32,
build_compiler: Compiler,
host: TargetSelection,
}
@@ -239,7 +244,19 @@ impl Step for Cargotest {
}
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Cargotest { stage: run.builder.top_stage, host: run.target });
if run.builder.top_stage == 0 {
eprintln!(
"ERROR: running cargotest with stage 0 is currently unsupported. Use at least stage 1."
);
exit!(1);
}
// We want to build cargo stage N (where N == top_stage), and rustc stage N,
// and test both of these together.
// So we need to get a build compiler stage N-1 to build the stage N components.
run.builder.ensure(Cargotest {
build_compiler: run.builder.compiler(run.builder.top_stage - 1, run.target),
host: run.target,
});
}
/// Runs the `cargotest` tool as compiled in `stage` by the `host` compiler.
@@ -247,9 +264,19 @@ impl Step for Cargotest {
/// This tool in `src/tools` will check out a few Rust projects and run `cargo
/// test` to ensure that we don't regress the test suites there.
fn run(self, builder: &Builder<'_>) {
let compiler = builder.compiler(self.stage, self.host);
builder.ensure(compile::Rustc::new(compiler, compiler.host));
let cargo = builder.ensure(tool::Cargo { compiler, target: compiler.host });
// cargotest's staging has several pieces:
// consider ./x test cargotest --stage=2.
//
// The test goal is to exercise a (stage 2 cargo, stage 2 rustc) pair through a stage 2
// cargotest tool.
// To produce the stage 2 cargo and cargotest, we need to do so with the stage 1 rustc and std.
// Importantly, the stage 2 rustc being tested (`tested_compiler`) via stage 2 cargotest is
// the rustc built by an earlier stage 1 rustc (the build_compiler). These are two different
// compilers!
let cargo =
builder.ensure(tool::Cargo::from_build_compiler(self.build_compiler, self.host));
let tested_compiler = builder.compiler(self.build_compiler.stage + 1, self.host);
builder.std(tested_compiler, self.host);
// Note that this is a short, cryptic, and not scoped directory name. This
// is currently to minimize the length of path on Windows where we otherwise
@@ -262,23 +289,28 @@ impl Step for Cargotest {
cmd.arg(&cargo.tool_path)
.arg(&out_dir)
.args(builder.config.test_args())
.env("RUSTC", builder.rustc(compiler))
.env("RUSTDOC", builder.rustdoc(compiler));
.env("RUSTC", builder.rustc(tested_compiler))
.env("RUSTDOC", builder.rustdoc_for_compiler(tested_compiler));
add_rustdoc_cargo_linker_args(
&mut cmd,
builder,
compiler.host,
tested_compiler.host,
LldThreads::No,
compiler.stage,
tested_compiler.stage,
);
cmd.delay_failure().run(builder);
}
fn metadata(&self) -> Option<StepMetadata> {
Some(StepMetadata::test("cargotest", self.host).stage(self.build_compiler.stage + 1))
}
}
/// Runs `cargo test` for cargo itself.
/// We label these tests as "cargo self-tests".
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Cargo {
stage: u32,
build_compiler: Compiler,
host: TargetSelection,
}
@@ -295,35 +327,33 @@ impl Step for Cargo {
}
fn make_run(run: RunConfig<'_>) {
// If stage is explicitly set or not lower than 2, keep it. Otherwise, make sure it's at least 2
// as tests for this step don't work with a lower stage.
let stage = if run.builder.config.is_explicit_stage() || run.builder.top_stage >= 2 {
run.builder.top_stage
} else {
2
};
run.builder.ensure(Cargo { stage, host: run.target });
run.builder.ensure(Cargo {
build_compiler: get_tool_target_compiler(
run.builder,
ToolTargetBuildMode::Build(run.target),
),
host: run.target,
});
}
/// Runs `cargo test` for `cargo` packaged with Rust.
fn run(self, builder: &Builder<'_>) {
let stage = self.stage;
// When we do a "stage 1 cargo self-test", it means that we test the stage 1 rustc
// using stage 1 cargo. So we actually build cargo using the stage 0 compiler, and then
// run its tests against the stage 1 compiler (called `tested_compiler` below).
builder.ensure(tool::Cargo::from_build_compiler(self.build_compiler, self.host));
if stage < 2 {
eprintln!("WARNING: cargo tests on stage {stage} may not behave well.");
eprintln!("HELP: consider using stage 2");
}
let compiler = builder.compiler(stage, self.host);
let cargo = builder.ensure(tool::Cargo { compiler, target: self.host });
let compiler = cargo.build_compiler;
let tested_compiler = builder.compiler(self.build_compiler.stage + 1, self.host);
builder.std(tested_compiler, self.host);
// We also need to build rustdoc for cargo tests
// It will be located in the bindir of `tested_compiler`, so we don't need to explicitly
// pass its path to Cargo.
builder.rustdoc_for_compiler(tested_compiler);
let cargo = tool::prepare_tool_cargo(
builder,
compiler,
Mode::ToolRustc,
self.build_compiler,
Mode::ToolTarget,
self.host,
Kind::Test,
Self::CRATE_PATH,
@@ -340,7 +370,25 @@ impl Step for Cargo {
// Forcibly disable tests using nightly features since any changes to
// those features won't be able to land.
cargo.env("CARGO_TEST_DISABLE_NIGHTLY", "1");
cargo.env("PATH", path_for_cargo(builder, compiler));
// Configure PATH to find the right rustc. NB. we have to use PATH
// and not RUSTC because the Cargo test suite has tests that will
// fail if rustc is not spelled `rustc`.
cargo.env("PATH", bin_path_for_cargo(builder, tested_compiler));
// The `cargo` command configured above has dylib dir path set to the `build_compiler`'s
// libdir. That causes issues in cargo test, because the programs that cargo compiles are
// incorrectly picking that libdir, even though they should be picking the
// `tested_compiler`'s libdir. We thus have to override the precedence here.
let mut existing_dylib_paths = cargo
.get_envs()
.find(|(k, _)| *k == OsStr::new(dylib_path_var()))
.and_then(|(_, v)| v)
.map(|value| split_paths(value).collect::<Vec<PathBuf>>())
.unwrap_or_default();
existing_dylib_paths.insert(0, builder.rustc_libdir(tested_compiler));
add_dylib_path(existing_dylib_paths, &mut cargo);
// Cargo's test suite uses `CARGO_RUSTC_CURRENT_DIR` to determine the path that `file!` is
// relative to. Cargo no longer sets this env var, so we have to do that. This has to be the
// same value as `-Zroot-dir`.
@@ -352,7 +400,7 @@ impl Step for Cargo {
crates: vec!["cargo".into()],
target: self.host.triple.to_string(),
host: self.host.triple.to_string(),
stage,
stage: self.build_compiler.stage + 1,
},
builder,
);
@@ -364,8 +412,7 @@ impl Step for Cargo {
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct RustAnalyzer {
stage: u32,
host: TargetSelection,
compilers: RustcPrivateCompilers,
}
impl Step for RustAnalyzer {
@@ -378,19 +425,18 @@ impl Step for RustAnalyzer {
}
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Self { stage: run.builder.top_stage, host: run.target });
run.builder.ensure(Self {
compilers: RustcPrivateCompilers::new(
run.builder,
run.builder.top_stage,
run.builder.host_target,
),
});
}
/// Runs `cargo test` for rust-analyzer
fn run(self, builder: &Builder<'_>) {
let stage = self.stage;
let host = self.host;
let compiler = builder.compiler(stage, host);
let compiler = tool::get_tool_rustc_compiler(builder, compiler);
// We don't need to build the whole Rust Analyzer for the proc-macro-srv test suite,
// but we do need the standard library to be present.
builder.ensure(compile::Rustc::new(compiler, host));
let host = self.compilers.target();
let workspace_path = "src/tools/rust-analyzer";
// until the whole RA test suite runs on `i686`, we only run
@@ -398,7 +444,7 @@ impl Step for RustAnalyzer {
let crate_path = "src/tools/rust-analyzer/crates/proc-macro-srv";
let mut cargo = tool::prepare_tool_cargo(
builder,
compiler,
self.compilers.build_compiler(),
Mode::ToolRustc,
host,
Kind::Test,
@@ -425,8 +471,7 @@ impl Step for RustAnalyzer {
/// Runs `cargo test` for rustfmt.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Rustfmt {
stage: u32,
host: TargetSelection,
compilers: RustcPrivateCompilers,
}
impl Step for Rustfmt {
@@ -438,36 +483,39 @@ impl Step for Rustfmt {
}
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Rustfmt { stage: run.builder.top_stage, host: run.target });
run.builder.ensure(Rustfmt {
compilers: RustcPrivateCompilers::new(
run.builder,
run.builder.top_stage,
run.builder.host_target,
),
});
}
/// Runs `cargo test` for rustfmt.
fn run(self, builder: &Builder<'_>) {
let stage = self.stage;
let host = self.host;
let compiler = builder.compiler(stage, host);
let tool_result = builder.ensure(tool::Rustfmt { compiler, target: self.host });
let compiler = tool_result.build_compiler;
let tool_result = builder.ensure(tool::Rustfmt::from_compilers(self.compilers));
let build_compiler = tool_result.build_compiler;
let target = self.compilers.target();
let mut cargo = tool::prepare_tool_cargo(
builder,
compiler,
build_compiler,
Mode::ToolRustc,
host,
target,
Kind::Test,
"src/tools/rustfmt",
SourceType::InTree,
&[],
);
let dir = testdir(builder, compiler.host);
let dir = testdir(builder, target);
t!(fs::create_dir_all(&dir));
cargo.env("RUSTFMT_TEST_DIR", dir);
cargo.add_rustc_lib_path(builder);
run_cargo_test(cargo, &[], &[], "rustfmt", host, builder);
run_cargo_test(cargo, &[], &[], "rustfmt", target, builder);
}
}
@@ -542,12 +590,14 @@ impl Step for Miri {
}
// This compiler runs on the host, we'll just use it for the target.
let target_compiler = builder.compiler(stage, host);
let compilers = RustcPrivateCompilers::new(builder, stage, host);
// Build our tools.
let miri = builder.ensure(tool::Miri { compiler: target_compiler, target: host });
let miri = builder.ensure(tool::Miri::from_compilers(compilers));
// the ui tests also assume cargo-miri has been built
builder.ensure(tool::CargoMiri { compiler: target_compiler, target: host });
builder.ensure(tool::CargoMiri::from_compilers(compilers));
let target_compiler = compilers.target_compiler();
// We also need sysroots, for Miri and for the host (the latter for build scripts).
// This is for the tests so everything is done with the target compiler.
@@ -599,7 +649,7 @@ impl Step for Miri {
cargo.env("MIRI_TEST_TARGET", target.rustc_target_arg());
{
let _guard = builder.msg_sysroot_tool(Kind::Test, stage, "miri", host, target);
let _guard = builder.msg_rustc_tool(Kind::Test, stage, "miri", host, target);
let _time = helpers::timeit(builder);
cargo.run(builder);
}
@@ -615,7 +665,7 @@ impl Step for Miri {
cargo.args(["tests/pass", "tests/panic"]);
{
let _guard = builder.msg_sysroot_tool(
let _guard = builder.msg_rustc_tool(
Kind::Test,
stage,
"miri (mir-opt-level 4)",
@@ -692,7 +742,7 @@ impl Step for CargoMiri {
// Finally, run everything.
let mut cargo = BootstrapCommand::from(cargo);
{
let _guard = builder.msg_sysroot_tool(Kind::Test, stage, "cargo-miri", host, target);
let _guard = builder.msg_rustc_tool(Kind::Test, stage, "cargo-miri", host, target);
let _time = helpers::timeit(builder);
cargo.run(builder);
}
@@ -762,7 +812,7 @@ NOTE: if you're sure you want to do this, please open an issue as to why. In the
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Clippy {
host: TargetSelection,
compilers: RustcPrivateCompilers,
}
impl Step for Clippy {
@@ -775,23 +825,30 @@ impl Step for Clippy {
}
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Clippy { host: run.target });
run.builder.ensure(Clippy {
compilers: RustcPrivateCompilers::new(
run.builder,
run.builder.top_stage,
run.builder.host_target,
),
});
}
/// Runs `cargo test` for clippy.
fn run(self, builder: &Builder<'_>) {
let stage = builder.top_stage;
let host = self.host;
let host = self.compilers.target();
// We need to carefully distinguish the compiler that builds clippy, and the compiler
// that is linked into the clippy being tested. `target_compiler` is the latter,
// and it must also be used by clippy's test runner to build tests and their dependencies.
let target_compiler = builder.compiler(stage, host);
let compilers = self.compilers;
let target_compiler = compilers.target_compiler();
let tool_result = builder.ensure(tool::Clippy { compiler: target_compiler, target: host });
let tool_compiler = tool_result.build_compiler;
let tool_result = builder.ensure(tool::Clippy::from_compilers(compilers));
let build_compiler = tool_result.build_compiler;
let mut cargo = tool::prepare_tool_cargo(
builder,
tool_compiler,
build_compiler,
Mode::ToolRustc,
host,
Kind::Test,
@@ -800,9 +857,10 @@ impl Step for Clippy {
&[],
);
cargo.env("RUSTC_TEST_SUITE", builder.rustc(tool_compiler));
cargo.env("RUSTC_LIB_PATH", builder.rustc_libdir(tool_compiler));
let host_libs = builder.stage_out(tool_compiler, Mode::ToolRustc).join(builder.cargo_dir());
cargo.env("RUSTC_TEST_SUITE", builder.rustc(build_compiler));
cargo.env("RUSTC_LIB_PATH", builder.rustc_libdir(build_compiler));
let host_libs =
builder.stage_out(build_compiler, Mode::ToolRustc).join(builder.cargo_dir());
cargo.env("HOST_LIBS", host_libs);
// Build the standard library that the tests can use.
@@ -831,8 +889,7 @@ impl Step for Clippy {
cargo.add_rustc_lib_path(builder);
let cargo = prepare_cargo_test(cargo, &[], &[], host, builder);
let _guard =
builder.msg_sysroot_tool(Kind::Test, tool_compiler.stage, "clippy", host, host);
let _guard = builder.msg_rustc_tool(Kind::Test, build_compiler.stage, "clippy", host, host);
// Clippy reports errors if it blessed the outputs
if cargo.allow_failure().run(builder) {
@@ -846,10 +903,7 @@ impl Step for Clippy {
}
}
fn path_for_cargo(builder: &Builder<'_>, compiler: Compiler) -> OsString {
// Configure PATH to find the right rustc. NB. we have to use PATH
// and not RUSTC because the Cargo test suite has tests that will
// fail if rustc is not spelled `rustc`.
fn bin_path_for_cargo(builder: &Builder<'_>, compiler: Compiler) -> OsString {
let path = builder.sysroot(compiler).join("bin");
let old_path = env::var_os("PATH").unwrap_or_default();
env::join_paths(iter::once(path).chain(env::split_paths(&old_path))).expect("")
@@ -884,7 +938,7 @@ impl Step for RustdocTheme {
.env("RUSTC_SYSROOT", builder.sysroot(self.compiler))
.env("RUSTDOC_LIBDIR", builder.sysroot_target_libdir(self.compiler, self.compiler.host))
.env("CFG_RELEASE_CHANNEL", &builder.config.channel)
.env("RUSTDOC_REAL", builder.rustdoc(self.compiler))
.env("RUSTDOC_REAL", builder.rustdoc_for_compiler(self.compiler))
.env("RUSTC_BOOTSTRAP", "1");
cmd.args(linker_args(builder, self.compiler.host, LldThreads::No, self.compiler.stage));
@@ -1043,7 +1097,11 @@ impl Step for RustdocGUI {
let mut cmd = builder.tool_cmd(Tool::RustdocGUITest);
let out_dir = builder.test_out(self.target).join("rustdoc-gui");
build_stamp::clear_if_dirty(builder, &out_dir, &builder.rustdoc(self.compiler));
build_stamp::clear_if_dirty(
builder,
&out_dir,
&builder.rustdoc_for_compiler(self.compiler),
);
if let Some(src) = builder.config.src.to_str() {
cmd.arg("--rust-src").arg(src);
@@ -1059,7 +1117,7 @@ impl Step for RustdocGUI {
cmd.arg("--jobs").arg(builder.jobs().to_string());
cmd.env("RUSTDOC", builder.rustdoc(self.compiler))
cmd.env("RUSTDOC", builder.rustdoc_for_compiler(self.compiler))
.env("RUSTC", builder.rustc(self.compiler));
add_rustdoc_cargo_linker_args(
@@ -1095,7 +1153,7 @@ impl Step for RustdocGUI {
}
let _time = helpers::timeit(builder);
let _guard = builder.msg_sysroot_tool(
let _guard = builder.msg_rustc_tool(
Kind::Test,
self.compiler.stage,
"rustdoc-gui",
@@ -1737,7 +1795,7 @@ NOTE: if you're sure you want to do this, please open an issue as to why. In the
// If we're using `--stage 0`, we should provide the bootstrap cargo.
builder.initial_cargo.clone()
} else {
builder.ensure(tool::Cargo { compiler, target: compiler.host }).tool_path
builder.ensure(tool::Cargo::from_build_compiler(compiler, compiler.host)).tool_path
};
cmd.arg("--cargo-path").arg(cargo_path);
@@ -1756,7 +1814,7 @@ NOTE: if you're sure you want to do this, please open an issue as to why. In the
|| mode == "rustdoc-json"
|| suite == "coverage-run-rustdoc"
{
cmd.arg("--rustdoc-path").arg(builder.rustdoc(compiler));
cmd.arg("--rustdoc-path").arg(builder.rustdoc_for_compiler(compiler));
}
if mode == "rustdoc-json" {
@@ -2272,7 +2330,7 @@ impl BookTest {
// mdbook just executes a binary named "rustdoc", so we need to update
// PATH so that it points to our rustdoc.
let mut rustdoc_path = builder.rustdoc(compiler);
let mut rustdoc_path = builder.rustdoc_for_compiler(compiler);
rustdoc_path.pop();
let old_path = env::var_os("PATH").unwrap_or_default();
let new_path = env::join_paths(iter::once(rustdoc_path).chain(env::split_paths(&old_path)))
@@ -2595,7 +2653,7 @@ fn run_cargo_test<'a>(
let mut cargo = prepare_cargo_test(cargo, libtest_args, crates, target, builder);
let _time = helpers::timeit(builder);
let _group = description.into().and_then(|what| {
builder.msg_sysroot_tool(Kind::Test, compiler.stage, what, compiler.host, target)
builder.msg_rustc_tool(Kind::Test, compiler.stage, what, compiler.host, target)
});
#[cfg(feature = "build-metrics")]

395
src/bootstrap/src/core/build_steps/tool.rs
View File

@@ -71,13 +71,9 @@ impl Builder<'_> {
) -> Option<gha::Group> {
match mode {
// depends on compiler stage, different to host compiler
Mode::ToolRustc => self.msg_sysroot_tool(
kind,
build_stage,
format_args!("tool {tool}"),
*host,
*target,
),
Mode::ToolRustc => {
self.msg_rustc_tool(kind, build_stage, format_args!("tool {tool}"), *host, *target)
}
// doesn't depend on compiler, same as host compiler
_ => self.msg(kind, build_stage, format_args!("tool {tool}"), *host, *target),
}
@@ -90,11 +86,8 @@ impl Builder<'_> {
pub struct ToolBuildResult {
/// Artifact path of the corresponding tool that was built.
pub tool_path: PathBuf,
/// Compiler used to build the tool. For non-`ToolRustc` tools this is equal to `target_compiler`.
/// For `ToolRustc` this is one stage before of the `target_compiler`.
/// Compiler used to build the tool.
pub build_compiler: Compiler,
/// Target compiler passed to `Step`.
pub target_compiler: Compiler,
}
impl Step for ToolBuild {
@@ -108,22 +101,15 @@ impl Step for ToolBuild {
///
/// This will build the specified tool with the specified `host` compiler in
/// `stage` into the normal cargo output directory.
fn run(mut self, builder: &Builder<'_>) -> ToolBuildResult {
fn run(self, builder: &Builder<'_>) -> ToolBuildResult {
let target = self.target;
let mut tool = self.tool;
let path = self.path;
let target_compiler = self.build_compiler;
self.build_compiler = if self.mode == Mode::ToolRustc {
get_tool_rustc_compiler(builder, self.build_compiler)
} else {
self.build_compiler
};
match self.mode {
Mode::ToolRustc => {
// If compiler was forced, its artifacts should have been prepared earlier.
if !self.build_compiler.is_forced_compiler() {
// FIXME: remove this, it's only needed for download-rustc...
if !self.build_compiler.is_forced_compiler() && builder.download_rustc() {
builder.std(self.build_compiler, self.build_compiler.host);
builder.ensure(compile::Rustc::new(self.build_compiler, target));
}
@@ -184,8 +170,7 @@ impl Step for ToolBuild {
Kind::Build,
self.mode,
self.tool,
// A stage N tool is built with the stage N-1 compiler.
self.build_compiler.stage + 1,
self.build_compiler.stage,
&self.build_compiler.host,
&self.target,
);
@@ -216,7 +201,7 @@ impl Step for ToolBuild {
.join(format!("lib{tool}.rlib")),
};
ToolBuildResult { tool_path, build_compiler: self.build_compiler, target_compiler }
ToolBuildResult { tool_path, build_compiler: self.build_compiler }
}
}
}
@@ -346,27 +331,6 @@ pub fn prepare_tool_cargo(
cargo
}
/// Handle stage-off logic for `ToolRustc` tools when necessary.
pub(crate) fn get_tool_rustc_compiler(
builder: &Builder<'_>,
target_compiler: Compiler,
) -> Compiler {
if target_compiler.is_forced_compiler() {
return target_compiler;
}
if builder.download_rustc() && target_compiler.stage == 1 {
// We shouldn't drop to stage0 compiler when using CI rustc.
return builder.compiler(1, builder.config.host_target);
}
// Similar to `compile::Assemble`, build with the previous stage's compiler. Otherwise
// we'd have stageN/bin/rustc and stageN/bin/$rustc_tool be effectively different stage
// compilers, which isn't what we want. Rustc tools should be linked in the same way as the
// compiler it's paired with, so it must be built with the previous stage compiler.
builder.compiler(target_compiler.stage.saturating_sub(1), builder.config.host_target)
}
/// Determines how to build a `ToolTarget`, i.e. which compiler should be used to compile it.
/// The compiler stage is automatically bumped if we need to cross-compile a stage 1 tool.
pub enum ToolTargetBuildMode {
@@ -719,15 +683,21 @@ impl Step for RemoteTestServer {
}
}
#[derive(Debug, Clone, Hash, PartialEq, Eq, Ord, PartialOrd)]
/// Represents `Rustdoc` that either comes from the external stage0 sysroot or that is built
/// locally.
/// Rustdoc is special, because it both essentially corresponds to a `Compiler` (that can be
/// externally provided), but also to a `ToolRustc` tool.
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct Rustdoc {
/// This should only ever be 0 or 2.
/// We sometimes want to reference the "bootstrap" rustdoc, which is why this option is here.
pub compiler: Compiler,
/// If the stage of `target_compiler` is `0`, then rustdoc is externally provided.
/// Otherwise it is built locally.
pub target_compiler: Compiler,
}
impl Step for Rustdoc {
type Output = ToolBuildResult;
/// Path to the built rustdoc binary.
type Output = PathBuf;
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
@@ -736,26 +706,25 @@ impl Step for Rustdoc {
}
fn make_run(run: RunConfig<'_>) {
run.builder
.ensure(Rustdoc { compiler: run.builder.compiler(run.builder.top_stage, run.target) });
run.builder.ensure(Rustdoc {
target_compiler: run.builder.compiler(run.builder.top_stage, run.target),
});
}
fn run(self, builder: &Builder<'_>) -> ToolBuildResult {
let target_compiler = self.compiler;
fn run(self, builder: &Builder<'_>) -> Self::Output {
let target_compiler = self.target_compiler;
let target = target_compiler.host;
// If stage is 0, we use a prebuilt rustdoc from stage0
if target_compiler.stage == 0 {
if !target_compiler.is_snapshot(builder) {
panic!("rustdoc in stage 0 must be snapshot rustdoc");
}
return ToolBuildResult {
tool_path: builder.initial_rustdoc.clone(),
build_compiler: target_compiler,
target_compiler,
};
return builder.initial_rustdoc.clone();
}
// If stage is higher, we build rustdoc instead
let bin_rustdoc = || {
let sysroot = builder.sysroot(target_compiler);
let bindir = sysroot.join("bin");
@@ -767,10 +736,7 @@ impl Step for Rustdoc {
// If CI rustc is enabled and we haven't modified the rustdoc sources,
// use the precompiled rustdoc from CI rustc's sysroot to speed up bootstrapping.
if builder.download_rustc()
&& target_compiler.stage > 0
&& builder.rust_info().is_managed_git_subrepository()
{
if builder.download_rustc() && builder.rust_info().is_managed_git_subrepository() {
let files_to_track = &["src/librustdoc", "src/tools/rustdoc", "src/rustdoc-json-types"];
// Check if unchanged
@@ -783,12 +749,7 @@ impl Step for Rustdoc {
let bin_rustdoc = bin_rustdoc();
builder.copy_link(&precompiled_rustdoc, &bin_rustdoc, FileType::Executable);
return ToolBuildResult {
tool_path: bin_rustdoc,
build_compiler: target_compiler,
target_compiler,
};
return bin_rustdoc;
}
}
@@ -804,9 +765,10 @@ impl Step for Rustdoc {
extra_features.push("jemalloc".to_string());
}
let ToolBuildResult { tool_path, build_compiler, target_compiler } =
builder.ensure(ToolBuild {
build_compiler: target_compiler,
let compilers = RustcPrivateCompilers::from_target_compiler(builder, target_compiler);
let tool_path = builder
.ensure(ToolBuild {
build_compiler: compilers.build_compiler,
target,
// Cargo adds a number of paths to the dylib search path on windows, which results in
// the wrong rustdoc being executed. To avoid the conflicting rustdocs, we name the "tool"
@@ -819,38 +781,41 @@ impl Step for Rustdoc {
allow_features: "",
cargo_args: Vec::new(),
artifact_kind: ToolArtifactKind::Binary,
});
})
.tool_path;
// don't create a stage0-sysroot/bin directory.
if target_compiler.stage > 0 {
if builder.config.rust_debuginfo_level_tools == DebuginfoLevel::None {
// Due to LTO a lot of debug info from C++ dependencies such as jemalloc can make it into
// our final binaries
compile::strip_debug(builder, target, &tool_path);
}
let bin_rustdoc = bin_rustdoc();
builder.copy_link(&tool_path, &bin_rustdoc, FileType::Executable);
ToolBuildResult { tool_path: bin_rustdoc, build_compiler, target_compiler }
} else {
ToolBuildResult { tool_path, build_compiler, target_compiler }
if builder.config.rust_debuginfo_level_tools == DebuginfoLevel::None {
// Due to LTO a lot of debug info from C++ dependencies such as jemalloc can make it into
// our final binaries
compile::strip_debug(builder, target, &tool_path);
}
let bin_rustdoc = bin_rustdoc();
builder.copy_link(&tool_path, &bin_rustdoc, FileType::Executable);
bin_rustdoc
}
fn metadata(&self) -> Option<StepMetadata> {
Some(
StepMetadata::build("rustdoc", self.compiler.host)
// rustdoc is ToolRustc, so stage N rustdoc is built by stage N-1 rustc
// FIXME: make this stage deduction automatic somehow
// FIXME: log the compiler that actually built ToolRustc steps
.stage(self.compiler.stage.saturating_sub(1)),
StepMetadata::build("rustdoc", self.target_compiler.host)
.stage(self.target_compiler.stage),
)
}
}
/// Builds the cargo tool.
/// Note that it can be built using a stable compiler.
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct Cargo {
pub compiler: Compiler,
pub target: TargetSelection,
build_compiler: Compiler,
target: TargetSelection,
}
impl Cargo {
/// Returns `Cargo` that will be **compiled** by the passed compiler, for the given
/// `target`.
pub fn from_build_compiler(build_compiler: Compiler, target: TargetSelection) -> Self {
Self { build_compiler, target }
}
}
impl Step for Cargo {
@@ -865,7 +830,10 @@ impl Step for Cargo {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(Cargo {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.config.host_target),
build_compiler: get_tool_target_compiler(
run.builder,
ToolTargetBuildMode::Build(run.target),
),
target: run.target,
});
}
@@ -873,19 +841,28 @@ impl Step for Cargo {
fn run(self, builder: &Builder<'_>) -> ToolBuildResult {
builder.build.require_submodule("src/tools/cargo", None);
builder.std(self.build_compiler, self.target);
builder.ensure(ToolBuild {
build_compiler: self.compiler,
build_compiler: self.build_compiler,
target: self.target,
tool: "cargo",
mode: Mode::ToolRustc,
mode: Mode::ToolTarget,
path: "src/tools/cargo",
source_type: SourceType::Submodule,
extra_features: Vec::new(),
allow_features: "",
// Cargo is compilable with a stable compiler, but since we run in bootstrap,
// with RUSTC_BOOTSTRAP being set, some "clever" build scripts enable specialization
// based on this, which breaks stuff. We thus have to explicitly allow these features
// here.
allow_features: "min_specialization,specialization",
cargo_args: Vec::new(),
artifact_kind: ToolArtifactKind::Binary,
})
}
fn metadata(&self) -> Option<StepMetadata> {
Some(StepMetadata::build("cargo", self.target).built_by(self.build_compiler))
}
}
/// Represents a built LldWrapper, the `lld-wrapper` tool itself, and a directory
@@ -1066,8 +1043,13 @@ impl Step for WasmComponentLd {
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct RustAnalyzer {
pub compiler: Compiler,
pub target: TargetSelection,
compilers: RustcPrivateCompilers,
}
impl RustAnalyzer {
pub fn from_compilers(compilers: RustcPrivateCompilers) -> Self {
Self { compilers }
}
}
impl RustAnalyzer {
@@ -1086,15 +1068,16 @@ impl Step for RustAnalyzer {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(RustAnalyzer {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.config.host_target),
target: run.target,
compilers: RustcPrivateCompilers::new(run.builder, run.builder.top_stage, run.target),
});
}
fn run(self, builder: &Builder<'_>) -> ToolBuildResult {
let build_compiler = self.compilers.build_compiler;
let target = self.compilers.target();
builder.ensure(ToolBuild {
build_compiler: self.compiler,
target: self.target,
build_compiler,
target,
tool: "rust-analyzer",
mode: Mode::ToolRustc,
path: "src/tools/rust-analyzer",
@@ -1105,16 +1088,29 @@ impl Step for RustAnalyzer {
artifact_kind: ToolArtifactKind::Binary,
})
}
fn metadata(&self) -> Option<StepMetadata> {
Some(
StepMetadata::build("rust-analyzer", self.compilers.target())
.built_by(self.compilers.build_compiler),
)
}
}
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct RustAnalyzerProcMacroSrv {
pub compiler: Compiler,
pub target: TargetSelection,
compilers: RustcPrivateCompilers,
}
impl RustAnalyzerProcMacroSrv {
pub fn from_compilers(compilers: RustcPrivateCompilers) -> Self {
Self { compilers }
}
}
impl Step for RustAnalyzerProcMacroSrv {
type Output = Option<ToolBuildResult>;
type Output = ToolBuildResult;
const DEFAULT: bool = true;
const ONLY_HOSTS: bool = true;
@@ -1131,15 +1127,14 @@ impl Step for RustAnalyzerProcMacroSrv {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure(RustAnalyzerProcMacroSrv {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.config.host_target),
target: run.target,
compilers: RustcPrivateCompilers::new(run.builder, run.builder.top_stage, run.target),
});
}
fn run(self, builder: &Builder<'_>) -> Option<ToolBuildResult> {
fn run(self, builder: &Builder<'_>) -> Self::Output {
let tool_result = builder.ensure(ToolBuild {
build_compiler: self.compiler,
target: self.target,
build_compiler: self.compilers.build_compiler,
target: self.compilers.target(),
tool: "rust-analyzer-proc-macro-srv",
mode: Mode::ToolRustc,
path: "src/tools/rust-analyzer/crates/proc-macro-srv-cli",
@@ -1152,7 +1147,7 @@ impl Step for RustAnalyzerProcMacroSrv {
// Copy `rust-analyzer-proc-macro-srv` to `<sysroot>/libexec/`
// so that r-a can use it.
let libexec_path = builder.sysroot(self.compiler).join("libexec");
let libexec_path = builder.sysroot(self.compilers.target_compiler).join("libexec");
t!(fs::create_dir_all(&libexec_path));
builder.copy_link(
&tool_result.tool_path,
@@ -1160,7 +1155,14 @@ impl Step for RustAnalyzerProcMacroSrv {
FileType::Executable,
);
Some(tool_result)
tool_result
}
fn metadata(&self) -> Option<StepMetadata> {
Some(
StepMetadata::build("rust-analyzer-proc-macro-srv", self.compilers.target())
.built_by(self.compilers.build_compiler),
)
}
}
@@ -1302,7 +1304,92 @@ impl Step for LibcxxVersionTool {
}
}
macro_rules! tool_extended {
/// Represents which compilers are involved in the compilation of a tool
/// that depends on compiler internals (`rustc_private`).
/// Their compilation looks like this:
///
/// - `build_compiler` (stage N-1) builds `target_compiler` (stage N) to produce .rlibs
/// - These .rlibs are copied into the sysroot of `build_compiler`
/// - `build_compiler` (stage N-1) builds `<tool>` (stage N)
/// - `<tool>` links to .rlibs from `target_compiler`
///
/// Eventually, this could also be used for .rmetas and check builds, but so far we only deal with
/// normal builds here.
#[derive(Copy, Clone, Debug, Hash, PartialEq, Eq)]
pub struct RustcPrivateCompilers {
/// Compiler that builds the tool and that builds `target_compiler`.
build_compiler: Compiler,
/// Compiler to which .rlib artifacts the tool links to.
/// The host target of this compiler corresponds to the target of the tool.
target_compiler: Compiler,
}
impl RustcPrivateCompilers {
/// Create compilers for a `rustc_private` tool with the given `stage` and for the given
/// `target`.
pub fn new(builder: &Builder<'_>, stage: u32, target: TargetSelection) -> Self {
let build_compiler = Self::build_compiler_from_stage(builder, stage);
// This is the compiler we'll link to
// FIXME: make 100% sure that `target_compiler` was indeed built with `build_compiler`...
let target_compiler = builder.compiler(build_compiler.stage + 1, target);
Self { build_compiler, target_compiler }
}
pub fn from_build_and_target_compiler(
build_compiler: Compiler,
target_compiler: Compiler,
) -> Self {
Self { build_compiler, target_compiler }
}
/// Create rustc tool compilers from the build compiler.
pub fn from_build_compiler(
builder: &Builder<'_>,
build_compiler: Compiler,
target: TargetSelection,
) -> Self {
let target_compiler = builder.compiler(build_compiler.stage + 1, target);
Self { build_compiler, target_compiler }
}
/// Create rustc tool compilers from the target compiler.
pub fn from_target_compiler(builder: &Builder<'_>, target_compiler: Compiler) -> Self {
Self {
build_compiler: Self::build_compiler_from_stage(builder, target_compiler.stage),
target_compiler,
}
}
fn build_compiler_from_stage(builder: &Builder<'_>, stage: u32) -> Compiler {
assert!(stage > 0);
if builder.download_rustc() && stage == 1 {
// We shouldn't drop to stage0 compiler when using CI rustc.
builder.compiler(1, builder.config.host_target)
} else {
builder.compiler(stage - 1, builder.config.host_target)
}
}
pub fn build_compiler(&self) -> Compiler {
self.build_compiler
}
pub fn target_compiler(&self) -> Compiler {
self.target_compiler
}
/// Target of the tool being compiled
pub fn target(&self) -> TargetSelection {
self.target_compiler.host
}
}
/// Creates a step that builds an extended `Mode::ToolRustc` tool
/// and installs it into the sysroot of a corresponding compiler.
macro_rules! tool_rustc_extended {
(
$name:ident {
path: $path:expr,
@@ -1316,8 +1403,15 @@ macro_rules! tool_extended {
) => {
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct $name {
pub compiler: Compiler,
pub target: TargetSelection,
compilers: RustcPrivateCompilers,
}
impl $name {
pub fn from_compilers(compilers: RustcPrivateCompilers) -> Self {
Self {
compilers,
}
}
}
impl Step for $name {
@@ -1326,7 +1420,7 @@ macro_rules! tool_extended {
const ONLY_HOSTS: bool = true;
fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_> {
should_run_tool_build_step(
should_run_extended_rustc_tool(
run,
$tool_name,
$path,
@@ -1336,17 +1430,15 @@ macro_rules! tool_extended {
fn make_run(run: RunConfig<'_>) {
run.builder.ensure($name {
compiler: run.builder.compiler(run.builder.top_stage, run.builder.config.host_target),
target: run.target,
compilers: RustcPrivateCompilers::new(run.builder, run.builder.top_stage, run.target),
});
}
fn run(self, builder: &Builder<'_>) -> ToolBuildResult {
let Self { compiler, target } = self;
run_tool_build_step(
let Self { compilers } = self;
build_extended_rustc_tool(
builder,
compiler,
target,
compilers,
$tool_name,
$path,
None $( .or(Some(&$add_bins_to_sysroot)) )?,
@@ -1356,18 +1448,16 @@ macro_rules! tool_extended {
}
fn metadata(&self) -> Option<StepMetadata> {
// FIXME: refactor extended tool steps to make the build_compiler explicit,
// it is offset by one now for rustc tools
Some(
StepMetadata::build($tool_name, self.target)
.built_by(self.compiler.with_stage(self.compiler.stage.saturating_sub(1)))
StepMetadata::build($tool_name, self.compilers.target())
.built_by(self.compilers.build_compiler)
)
}
}
}
}
fn should_run_tool_build_step<'a>(
fn should_run_extended_rustc_tool<'a>(
run: ShouldRun<'a>,
tool_name: &'static str,
path: &'static str,
@@ -1391,39 +1481,38 @@ fn should_run_tool_build_step<'a>(
)
}
#[expect(clippy::too_many_arguments)] // silence overeager clippy lint
fn run_tool_build_step(
fn build_extended_rustc_tool(
builder: &Builder<'_>,
compiler: Compiler,
target: TargetSelection,
compilers: RustcPrivateCompilers,
tool_name: &'static str,
path: &'static str,
add_bins_to_sysroot: Option<&[&str]>,
add_features: Option<fn(&Builder<'_>, TargetSelection, &mut Vec<String>)>,
cargo_args: Option<&[&'static str]>,
) -> ToolBuildResult {
let target = compilers.target();
let mut extra_features = Vec::new();
if let Some(func) = add_features {
func(builder, target, &mut extra_features);
}
let ToolBuildResult { tool_path, build_compiler, target_compiler } =
builder.ensure(ToolBuild {
build_compiler: compiler,
target,
tool: tool_name,
mode: Mode::ToolRustc,
path,
extra_features,
source_type: SourceType::InTree,
allow_features: "",
cargo_args: cargo_args.unwrap_or_default().iter().map(|s| String::from(*s)).collect(),
artifact_kind: ToolArtifactKind::Binary,
});
let build_compiler = compilers.build_compiler;
let ToolBuildResult { tool_path, .. } = builder.ensure(ToolBuild {
build_compiler,
target,
tool: tool_name,
mode: Mode::ToolRustc,
path,
extra_features,
source_type: SourceType::InTree,
allow_features: "",
cargo_args: cargo_args.unwrap_or_default().iter().map(|s| String::from(*s)).collect(),
artifact_kind: ToolArtifactKind::Binary,
});
let target_compiler = compilers.target_compiler;
if let Some(add_bins_to_sysroot) = add_bins_to_sysroot
&& !add_bins_to_sysroot.is_empty()
&& target_compiler.stage > 0
{
let bindir = builder.sysroot(target_compiler).join("bin");
t!(fs::create_dir_all(&bindir));
@@ -1435,25 +1524,25 @@ fn run_tool_build_step(
// Return a path into the bin dir.
let path = bindir.join(exe(tool_name, target_compiler.host));
ToolBuildResult { tool_path: path, build_compiler, target_compiler }
ToolBuildResult { tool_path: path, build_compiler }
} else {
ToolBuildResult { tool_path, build_compiler, target_compiler }
ToolBuildResult { tool_path, build_compiler }
}
}
tool_extended!(Cargofmt {
tool_rustc_extended!(Cargofmt {
path: "src/tools/rustfmt",
tool_name: "cargo-fmt",
stable: true,
add_bins_to_sysroot: ["cargo-fmt"]
});
tool_extended!(CargoClippy {
tool_rustc_extended!(CargoClippy {
path: "src/tools/clippy",
tool_name: "cargo-clippy",
stable: true,
add_bins_to_sysroot: ["cargo-clippy"]
});
tool_extended!(Clippy {
tool_rustc_extended!(Clippy {
path: "src/tools/clippy",
tool_name: "clippy-driver",
stable: true,
@@ -1464,7 +1553,7 @@ tool_extended!(Clippy {
}
}
});
tool_extended!(Miri {
tool_rustc_extended!(Miri {
path: "src/tools/miri",
tool_name: "miri",
stable: false,
@@ -1472,13 +1561,13 @@ tool_extended!(Miri {
// Always compile also tests when building miri. Otherwise feature unification can cause rebuilds between building and testing miri.
cargo_args: &["--all-targets"],
});
tool_extended!(CargoMiri {
tool_rustc_extended!(CargoMiri {
path: "src/tools/miri/cargo-miri",
tool_name: "cargo-miri",
stable: false,
add_bins_to_sysroot: ["cargo-miri"]
});
tool_extended!(Rustfmt {
tool_rustc_extended!(Rustfmt {
path: "src/tools/rustfmt",
tool_name: "rustfmt",
stable: true,

4
src/bootstrap/src/core/builder/cargo.rs
View File

@@ -508,7 +508,7 @@ impl Builder<'_> {
}
_ => panic!("doc mode {mode:?} not expected"),
};
let rustdoc = self.rustdoc(compiler);
let rustdoc = self.rustdoc_for_compiler(compiler);
build_stamp::clear_if_dirty(self, &my_out, &rustdoc);
}
@@ -822,7 +822,7 @@ impl Builder<'_> {
}
let rustdoc_path = match cmd_kind {
Kind::Doc | Kind::Test | Kind::MiriTest => self.rustdoc(compiler),
Kind::Doc | Kind::Test | Kind::MiriTest => self.rustdoc_for_compiler(compiler),
_ => PathBuf::from("/path/to/nowhere/rustdoc/not/required"),
};

38
src/bootstrap/src/core/builder/mod.rs
View File

@@ -16,6 +16,7 @@ use tracing::instrument;
pub use self::cargo::{Cargo, cargo_profile_var};
pub use crate::Compiler;
use crate::core::build_steps::compile::{Std, StdLink};
use crate::core::build_steps::tool::RustcPrivateCompilers;
use crate::core::build_steps::{
check, clean, clippy, compile, dist, doc, gcc, install, llvm, run, setup, test, tool, vendor,
};
@@ -1543,8 +1544,11 @@ You have to build a stage1 compiler for `{}` first, and then use it to build a s
.map(|entry| entry.path())
}
pub fn rustdoc(&self, compiler: Compiler) -> PathBuf {
self.ensure(tool::Rustdoc { compiler }).tool_path
/// Returns a path to `Rustdoc` that "belongs" to the `target_compiler`.
/// It can be either a stage0 rustdoc or a locally built rustdoc that *links* to
/// `target_compiler`.
pub fn rustdoc_for_compiler(&self, target_compiler: Compiler) -> PathBuf {
self.ensure(tool::Rustdoc { target_compiler })
}
pub fn cargo_clippy_cmd(&self, run_compiler: Compiler) -> BootstrapCommand {
@@ -1560,10 +1564,13 @@ You have to build a stage1 compiler for `{}` first, and then use it to build a s
return cmd;
}
let _ =
self.ensure(tool::Clippy { compiler: run_compiler, target: self.build.host_target });
let cargo_clippy = self
.ensure(tool::CargoClippy { compiler: run_compiler, target: self.build.host_target });
// FIXME: double check that `run_compiler`'s stage is what we want to use
let compilers =
RustcPrivateCompilers::new(self, run_compiler.stage, self.build.host_target);
assert_eq!(run_compiler, compilers.target_compiler());
let _ = self.ensure(tool::Clippy::from_compilers(compilers));
let cargo_clippy = self.ensure(tool::CargoClippy::from_compilers(compilers));
let mut dylib_path = helpers::dylib_path();
dylib_path.insert(0, self.sysroot(run_compiler).join("lib"));
@@ -1575,11 +1582,14 @@ You have to build a stage1 compiler for `{}` first, and then use it to build a s
pub fn cargo_miri_cmd(&self, run_compiler: Compiler) -> BootstrapCommand {
assert!(run_compiler.stage > 0, "miri can not be invoked at stage 0");
let compilers =
RustcPrivateCompilers::new(self, run_compiler.stage, self.build.host_target);
assert_eq!(run_compiler, compilers.target_compiler());
// Prepare the tools
let miri =
self.ensure(tool::Miri { compiler: run_compiler, target: self.build.host_target });
let cargo_miri =
self.ensure(tool::CargoMiri { compiler: run_compiler, target: self.build.host_target });
let miri = self.ensure(tool::Miri::from_compilers(compilers));
let cargo_miri = self.ensure(tool::CargoMiri::from_compilers(compilers));
// Invoke cargo-miri, make sure it can find miri and cargo.
let mut cmd = command(cargo_miri.tool_path);
cmd.env("MIRI", &miri.tool_path);
@@ -1604,7 +1614,7 @@ You have to build a stage1 compiler for `{}` first, and then use it to build a s
// equivalently to rustc.
.env("RUSTDOC_LIBDIR", self.rustc_libdir(compiler))
.env("CFG_RELEASE_CHANNEL", &self.config.channel)
.env("RUSTDOC_REAL", self.rustdoc(compiler))
.env("RUSTDOC_REAL", self.rustdoc_for_compiler(compiler))
.env("RUSTC_BOOTSTRAP", "1");
cmd.arg("-Wrustdoc::invalid_codeblock_attributes");
@@ -1728,11 +1738,11 @@ You have to build a stage1 compiler for `{}` first, and then use it to build a s
/// Ensure that a given step is built *only if it's supposed to be built by default*, returning
/// its output. This will cache the step, so it's safe (and good!) to call this as often as
/// needed to ensure that all dependencies are build.
pub(crate) fn ensure_if_default<T, S: Step<Output = Option<T>>>(
pub(crate) fn ensure_if_default<T, S: Step<Output = T>>(
&'a self,
step: S,
kind: Kind,
) -> S::Output {
) -> Option<S::Output> {
let desc = StepDescription::from::<S>(kind);
let should_run = (desc.should_run)(ShouldRun::new(self, desc.kind));
@@ -1744,7 +1754,7 @@ You have to build a stage1 compiler for `{}` first, and then use it to build a s
}
// Only execute if it's supposed to run as default
if desc.default && should_run.is_really_default() { self.ensure(step) } else { None }
if desc.default && should_run.is_really_default() { Some(self.ensure(step)) } else { None }
}
/// Checks if any of the "should_run" paths is in the `Builder` paths.

288
src/bootstrap/src/core/builder/tests.rs
View File

@@ -279,13 +279,6 @@ mod defaults {
first(cache.all::<tool::ErrorIndex>()),
&[tool::ErrorIndex { compiler: Compiler::new(1, a) }]
);
// docs should be built with the stage0 compiler, not with the stage0 artifacts.
// recall that rustdoc is off-by-one: `stage` is the compiler rustdoc is _linked_ to,
// not the one it was built by.
assert_eq!(
first(cache.all::<tool::Rustdoc>()),
&[tool::Rustdoc { compiler: Compiler::new(1, a) },]
);
}
}
@@ -337,12 +330,6 @@ mod dist {
first(builder.cache.all::<tool::ErrorIndex>()),
&[tool::ErrorIndex { compiler: Compiler::new(1, a) }]
);
// This is actually stage 1, but Rustdoc::run swaps out the compiler with
// stage minus 1 if --stage is not 0. Very confusing!
assert_eq!(
first(builder.cache.all::<tool::Rustdoc>()),
&[tool::Rustdoc { compiler: Compiler::new(2, a) },]
);
}
}
@@ -569,36 +556,6 @@ fn test_is_builder_target() {
}
}
#[test]
fn test_get_tool_rustc_compiler() {
let mut config = configure("build", &[], &[]);
config.download_rustc_commit = None;
let build = Build::new(config);
let builder = Builder::new(&build);
let target_triple_1 = TargetSelection::from_user(TEST_TRIPLE_1);
let compiler = Compiler::new(2, target_triple_1);
let expected = Compiler::new(1, target_triple_1);
let actual = tool::get_tool_rustc_compiler(&builder, compiler);
assert_eq!(expected, actual);
let compiler = Compiler::new(1, target_triple_1);
let expected = Compiler::new(0, target_triple_1);
let actual = tool::get_tool_rustc_compiler(&builder, compiler);
assert_eq!(expected, actual);
let mut config = configure("build", &[], &[]);
config.download_rustc_commit = Some("".to_owned());
let build = Build::new(config);
let builder = Builder::new(&build);
let compiler = Compiler::new(1, target_triple_1);
let expected = Compiler::new(1, target_triple_1);
let actual = tool::get_tool_rustc_compiler(&builder, compiler);
assert_eq!(expected, actual);
}
/// When bootstrap detects a step dependency cycle (which is a bug), its panic
/// message should show the actual steps on the stack, not just several copies
/// of `Any { .. }`.
@@ -657,7 +614,7 @@ mod snapshot {
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustdoc 0 <host>
[build] rustdoc 1 <host>
");
}
@@ -680,10 +637,10 @@ mod snapshot {
[build] rustc 2 <host> -> std 2 <host>
[build] rustc 1 <host> -> std 1 <target1>
[build] rustc 2 <host> -> std 2 <target1>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[build] llvm <target1>
[build] rustc 1 <host> -> rustc 2 <target1>
[build] rustdoc 1 <target1>
[build] rustdoc 2 <target1>
");
}
@@ -762,6 +719,23 @@ mod snapshot {
");
}
#[test]
fn build_compiler_codegen_backend() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx
.config("build")
.args(&["--set", "rust.codegen-backends=['llvm', 'cranelift']"])
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 0 <host> -> rustc_codegen_cranelift 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustdoc 1 <host>
"
);
}
#[test]
fn build_compiler_tools() {
let ctx = TestCtx::new();
@@ -780,7 +754,7 @@ mod snapshot {
[build] rustc 1 <host> -> LldWrapper 2 <host>
[build] rustc 1 <host> -> LlvmBitcodeLinker 2 <host>
[build] rustc 2 <host> -> std 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
"
);
}
@@ -809,7 +783,7 @@ mod snapshot {
[build] rustc 1 <host> -> rustc 2 <target1>
[build] rustc 1 <host> -> LldWrapper 2 <target1>
[build] rustc 1 <host> -> LlvmBitcodeLinker 2 <target1>
[build] rustdoc 1 <target1>
[build] rustdoc 2 <target1>
"
);
}
@@ -998,7 +972,7 @@ mod snapshot {
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustdoc 0 <host>
[build] rustdoc 1 <host>
[doc] std 1 <host> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
");
}
@@ -1026,6 +1000,15 @@ mod snapshot {
");
}
#[test]
fn build_cargo() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("build")
.paths(&["cargo"])
.render_steps(), @"[build] rustc 0 <host> -> cargo 1 <host>");
}
#[test]
fn dist_default_stage() {
let ctx = TestCtx::new();
@@ -1047,7 +1030,7 @@ mod snapshot {
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[doc] std 2 <host> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[build] rustc 2 <host> -> std 2 <host>
[build] rustc 0 <host> -> LintDocs 1 <host>
@@ -1089,7 +1072,7 @@ mod snapshot {
[build] rustc 1 <host> -> LldWrapper 2 <host>
[build] rustc 1 <host> -> WasmComponentLd 2 <host>
[build] rustc 1 <host> -> LlvmBitcodeLinker 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[doc] std 2 <host> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[build] rustc 2 <host> -> std 2 <host>
[build] rustc 0 <host> -> LintDocs 1 <host>
@@ -1097,16 +1080,19 @@ mod snapshot {
[dist] docs <host>
[doc] std 2 <host> crates=[]
[dist] mingw <host>
[build] rustc 1 <host> -> rust-analyzer-proc-macro-srv 2 <host>
[build] rustc 0 <host> -> GenerateCopyright 1 <host>
[dist] rustc <host>
[dist] rustc 1 <host> -> std 1 <host>
[dist] src <>
[build] rustc 0 <host> -> rustfmt 1 <host>
[build] rustc 0 <host> -> cargo-fmt 1 <host>
[build] rustc 0 <host> -> clippy-driver 1 <host>
[build] rustc 0 <host> -> cargo-clippy 1 <host>
[build] rustc 0 <host> -> miri 1 <host>
[build] rustc 0 <host> -> cargo-miri 1 <host>
[build] rustc 1 <host> -> cargo 2 <host>
[build] rustc 1 <host> -> rust-analyzer 2 <host>
[build] rustc 1 <host> -> rustfmt 2 <host>
[build] rustc 1 <host> -> cargo-fmt 2 <host>
[build] rustc 1 <host> -> clippy-driver 2 <host>
[build] rustc 1 <host> -> cargo-clippy 2 <host>
[build] rustc 1 <host> -> miri 2 <host>
[build] rustc 1 <host> -> cargo-miri 2 <host>
");
}
@@ -1125,7 +1111,7 @@ mod snapshot {
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[doc] std 2 <host> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[doc] std 2 <target1> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[build] rustc 2 <host> -> std 2 <host>
@@ -1162,7 +1148,7 @@ mod snapshot {
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[doc] std 2 <host> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[build] rustc 2 <host> -> std 2 <host>
[build] rustc 0 <host> -> LintDocs 1 <host>
@@ -1176,7 +1162,7 @@ mod snapshot {
[dist] rustc <host>
[build] llvm <target1>
[build] rustc 1 <host> -> rustc 2 <target1>
[build] rustdoc 1 <target1>
[build] rustdoc 2 <target1>
[dist] rustc <target1>
[dist] rustc 1 <host> -> std 1 <host>
[dist] src <>
@@ -1199,7 +1185,7 @@ mod snapshot {
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[doc] std 2 <host> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[doc] std 2 <target1> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[build] rustc 2 <host> -> std 2 <host>
@@ -1217,7 +1203,7 @@ mod snapshot {
[dist] rustc <host>
[build] llvm <target1>
[build] rustc 1 <host> -> rustc 2 <target1>
[build] rustdoc 1 <target1>
[build] rustdoc 2 <target1>
[dist] rustc <target1>
[dist] rustc 1 <host> -> std 1 <host>
[dist] rustc 1 <host> -> std 1 <target1>
@@ -1241,7 +1227,7 @@ mod snapshot {
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[doc] std 2 <target1> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[build] rustc 2 <host> -> std 2 <host>
[build] rustc 0 <host> -> RustInstaller 1 <host>
@@ -1273,7 +1259,7 @@ mod snapshot {
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustc 1 <host> -> WasmComponentLd 2 <host>
[build] rustdoc 1 <host>
[build] rustdoc 2 <host>
[doc] std 2 <target1> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
[build] rustc 2 <host> -> std 2 <host>
[build] rustc 1 <host> -> std 1 <target1>
@@ -1286,17 +1272,20 @@ mod snapshot {
[build] llvm <target1>
[build] rustc 1 <host> -> rustc 2 <target1>
[build] rustc 1 <host> -> WasmComponentLd 2 <target1>
[build] rustdoc 1 <target1>
[build] rustdoc 2 <target1>
[build] rustc 1 <host> -> rust-analyzer-proc-macro-srv 2 <target1>
[build] rustc 0 <host> -> GenerateCopyright 1 <host>
[dist] rustc <target1>
[dist] rustc 1 <host> -> std 1 <target1>
[dist] src <>
[build] rustc 0 <host> -> rustfmt 1 <target1>
[build] rustc 0 <host> -> cargo-fmt 1 <target1>
[build] rustc 0 <host> -> clippy-driver 1 <target1>
[build] rustc 0 <host> -> cargo-clippy 1 <target1>
[build] rustc 0 <host> -> miri 1 <target1>
[build] rustc 0 <host> -> cargo-miri 1 <target1>
[build] rustc 1 <host> -> cargo 2 <target1>
[build] rustc 1 <host> -> rust-analyzer 2 <target1>
[build] rustc 1 <host> -> rustfmt 2 <target1>
[build] rustc 1 <host> -> cargo-fmt 2 <target1>
[build] rustc 1 <host> -> clippy-driver 2 <target1>
[build] rustc 1 <host> -> cargo-clippy 2 <target1>
[build] rustc 1 <host> -> miri 2 <target1>
[build] rustc 1 <host> -> cargo-miri 2 <target1>
[build] rustc 1 <host> -> LlvmBitcodeLinker 2 <target1>
");
}
@@ -1610,6 +1599,58 @@ mod snapshot {
steps.assert_contains_fuzzy(StepMetadata::build("rustc", host));
}
#[test]
fn test_cargo_stage_1() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("test")
.path("cargo")
.render_steps(), @r"
[build] rustc 0 <host> -> cargo 1 <host>
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustdoc 1 <host>
[build] rustdoc 0 <host>
");
}
#[test]
fn test_cargo_stage_2() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("test")
.path("cargo")
.stage(2)
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> cargo 2 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustc 2 <host> -> std 2 <host>
[build] rustdoc 2 <host>
[build] rustdoc 1 <host>
");
}
#[test]
fn test_cargotest() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("test")
.path("cargotest")
.render_steps(), @r"
[build] rustc 0 <host> -> cargo 1 <host>
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 0 <host> -> CargoTest 1 <host>
[build] rustdoc 1 <host>
[test] cargotest 1 <host>
");
}
#[test]
fn doc_library() {
let ctx = TestCtx::new();
@@ -1619,7 +1660,7 @@ mod snapshot {
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustdoc 0 <host>
[build] rustdoc 1 <host>
[doc] std 1 <host> crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]
");
}
@@ -1633,7 +1674,7 @@ mod snapshot {
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustdoc 0 <host>
[build] rustdoc 1 <host>
[doc] std 1 <host> crates=[core]
");
}
@@ -1648,7 +1689,7 @@ mod snapshot {
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustdoc 0 <host>
[build] rustdoc 1 <host>
[doc] std 1 <host> crates=[core]
");
}
@@ -1678,7 +1719,7 @@ mod snapshot {
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustdoc 0 <host>
[build] rustdoc 1 <host>
[doc] std 1 <host> crates=[alloc,core]
");
}
@@ -1694,10 +1735,105 @@ mod snapshot {
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustdoc 0 <host>
[build] rustdoc 1 <host>
[doc] std 1 <target1> crates=[alloc,core]
");
}
#[test]
fn doc_compiler_stage_0() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("doc")
.path("compiler")
.stage(0)
.render_steps(), @r"
[build] rustdoc 0 <host>
[build] llvm <host>
[doc] rustc 0 <host>
");
}
#[test]
fn doc_compiler_stage_1() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("doc")
.path("compiler")
.stage(1)
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustdoc 1 <host>
[doc] rustc 1 <host>
");
}
#[test]
fn doc_compiler_stage_2() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("doc")
.path("compiler")
.stage(2)
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustc 2 <host> -> std 2 <host>
[build] rustdoc 2 <host>
[doc] rustc 2 <host>
");
}
#[test]
fn doc_compiletest_stage_0() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("doc")
.path("src/tools/compiletest")
.stage(0)
.render_steps(), @r"
[build] rustdoc 0 <host>
[doc] Compiletest <host>
");
}
#[test]
fn doc_compiletest_stage_1() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("doc")
.path("src/tools/compiletest")
.stage(1)
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustdoc 1 <host>
[doc] Compiletest <host>
");
}
#[test]
fn doc_compiletest_stage_2() {
let ctx = TestCtx::new();
insta::assert_snapshot!(
ctx.config("doc")
.path("src/tools/compiletest")
.stage(2)
.render_steps(), @r"
[build] llvm <host>
[build] rustc 0 <host> -> rustc 1 <host>
[build] rustc 1 <host> -> std 1 <host>
[build] rustc 1 <host> -> rustc 2 <host>
[build] rustc 2 <host> -> std 2 <host>
[build] rustdoc 2 <host>
[doc] Compiletest <host>
");
}
}
struct ExecutedSteps {

20
src/bootstrap/src/lib.rs
View File

@@ -1105,17 +1105,6 @@ impl Build {
self.msg(Kind::Doc, compiler.stage, what, compiler.host, target.into())
}
#[must_use = "Groups should not be dropped until the Step finishes running"]
#[track_caller]
fn msg_build(
&self,
compiler: Compiler,
what: impl Display,
target: impl Into<Option<TargetSelection>>,
) -> Option<gha::Group> {
self.msg(Kind::Build, compiler.stage, what, compiler.host, target)
}
/// Return a `Group` guard for a [`Step`] that is built for each `--stage`.
///
/// [`Step`]: crate::core::builder::Step
@@ -1162,20 +1151,21 @@ impl Build {
#[must_use = "Groups should not be dropped until the Step finishes running"]
#[track_caller]
fn msg_sysroot_tool(
fn msg_rustc_tool(
&self,
action: impl Into<Kind>,
stage: u32,
build_stage: u32,
what: impl Display,
host: TargetSelection,
target: TargetSelection,
) -> Option<gha::Group> {
let action = action.into().description();
let msg = |fmt| format!("{action} {what} {fmt}");
let msg = if host == target {
msg(format_args!("(stage{stage} -> stage{}, {target})", stage + 1))
msg(format_args!("(stage{build_stage} -> stage{}, {target})", build_stage + 1))
} else {
msg(format_args!("(stage{stage}:{host} -> stage{}:{target})", stage + 1))
msg(format_args!("(stage{build_stage}:{host} -> stage{}:{target})", build_stage + 1))
};
self.group(&msg)
}