Integrate jobserver support to parallel codegen

This commit integrates the `jobserver` crate into the compiler. The crate was
previously integrated in to Cargo as part of rust-lang/cargo#4110. The purpose
here is to two-fold:

* Primarily the compiler can cooperate with Cargo on parallelism. When you run
  `cargo build -j4` then this'll make sure that the entire build process between
  Cargo/rustc won't use more than 4 cores, whereas today you'd get 4 rustc
  instances which may all try to spawn lots of threads.

* Secondarily rustc/Cargo can now integrate with a foreign GNU `make` jobserver.
  This means that if you call cargo/rustc from `make` or another
  jobserver-compatible implementation it'll use foreign parallelism settings
  instead of creating new ones locally.

As the number of parallel codegen instances in the compiler continues to grow
over time with the advent of incremental compilation it's expected that this'll
become more of a problem, so this is intended to nip concurrent concerns in the
bud by having all the tools to cooperate!

Note that while rustc has support for itself creating a jobserver it's far more
likely that rustc will always use the jobserver configured by Cargo. Cargo today
will now set a jobserver unconditionally for rustc to use.
This commit is contained in:
Alex Crichton
2017-06-15 07:08:18 -07:00
parent 03198da2ad
commit 201f06988f
19 changed files with 512 additions and 309 deletions

View File

@@ -9,6 +9,9 @@ path = "lib.rs"
test = false
bench = false
[dev-dependencies]
rand = { path = "../librand" }
[[test]]
name = "coretests"
path = "../libcore/tests/lib.rs"