Some users of the build system change the git sha on every build due to
utilizing git to push changes to a remote server. This allows them to
simply configure that away instead of depending on custom patches to
rustbuild.
This introduces a slight change in behavior, where we unilaterally
respect the --host and --target parameters passed for all sanity
checking and runtime configuration.
When copying libstd for the stage 2 compiler, the builder ignores the
configured libdir/libdir_relative configuration parameters. This causes
the compiler to fail to find libstd, which cause any tools built with the
stage 2 compiler to fail.
To fix this, make the copy steps of rustbuild aware of the libdir_relative
parameter when the stage >= 2. Also update the dist target to be aware of
the new location of libstd.
Autogenerate stubs and SUMMARY.md in the unstable book
Removes a speed bump in compiler development by autogenerating stubs for features in the unstable book. See #42454 for discussion.
The PR contains three commits, separated in order to make review easy:
* The first commit converts the tidy tool from a binary crate to a crate that contains both a library and a binary. In the second commit, we'll use the tidy library
* The second and main commit introduces autogeneration of SUMMARY.md and feature stub files
* The third commit turns off the tidy lint that checks for features without a stub, and removes the stub files. A separate commit due to the large number of files touched
Members of the doc team who wish to document some features can either do this (where `$rustsrc` is the root of the rust repo git checkout):
1. cd to `$rustsrc/src/tools/unstable-book-gen` and then do `cargo run $rustsrc/src $rustsrc/src/doc/unstable-book` to put the stubs into the unstable book
2. cd to `$rustsrc` and run `git ls-files --others --exclude-standard` to list the newly added stubs
3. choose a file to edit, then `git add` it and `git commit`
4. afterwards, remove all changes by the tool by doing `git --reset hard` and `git clean -f`
Or they can do this:
1. remove the comment marker in `src/tools/tidy/src/unstable_book.rs` line 122
2. run `./x.py test src/tools/tidy` to list the unstable features which only have stubs
3. revert the change in 1
3. document one of the chosen unstable features
The changes done by this PR also allow for further development:
* tidy obtains information about tracking issues. We can now forbid differing tracking issues between differing `#![unstable]` annotations. I haven't done this but plan to in a future PR
* we now have a general framework for generating stuff for the unstable book at build time. Further changes can autogenerate a list of the API a given library feature exposes.
The old way to simply click through the documentation after it has been uploaded to rust-lang.org works as well.
r? @nagisa
Fixes#42454
This option forwards to each `cargo test` invocation, and applies the
same logic across all test steps to keep going after failures. At the
end, a brief summary line reports how many commands failed, if any.
Note that if a test program fails to even start at all, or if an
auxiliary build command related to testing fails, these are still left
to stop everything right away.
Fixes#40219.
* Bring back colors on Travis, which was disabled since #39036.
Append --color=always to cargo when running in CI environment.
* Removed `set -x` in the shell scripts. The `retry` function already
prints which command it is running, add `-x` just add noise to the
output.
* Support travis_fold/travis_time. Matching pairs of these allow Travis CI
to collapse the output in between. This greatly cut down the unnecessary
"successful" output one need to scroll through before finding the failed
statement.
I noticed these while reading through the build system
documentation. They're hardly worth fixing, but I'm also using this to
get my feet wet with the rustc contribution system.
Add an option to run rustbuild on low priority on Windows and Unix
This is a resurrection of #40776, combining their Windows setup with an additional setup on Unix to set the program group's *nice*ness to +10 (low-but-not-lowest priority, mirroring the priority in the Windows setup) when the `low_priority` option is on.
This is a resurrection of #40776, combining their Windows setup with an
additional setup on Unix to set the program group's niceness to +10
(low-but-not-lowest priority) when the `low_priority` option is on.
When -Z profile is passed, the GCDAProfiling LLVM pass is added
to the pipeline, which uses debug information to instrument the IR.
After compiling with -Z profile, the $(OUT_DIR)/$(CRATE_NAME).gcno
file is created, containing initial profiling information.
After running the program built, the $(OUT_DIR)/$(CRATE_NAME).gcda
file is created, containing branch counters.
The created *.gcno and *.gcda files can be processed using
the "llvm-cov gcov" and "lcov" tools. The profiling data LLVM
generates does not faithfully follow the GCC's format for *.gcno
and *.gcda files, and so it will probably not work with other tools
(such as gcov itself) that consume these files.
We've got a freshly minted beta compiler, let's update to use that on nightly!
This has a few other changes associated with it as well
* A bump to the rustc version number (to 1.19.0)
* Movement of the `cargo` and `rls` submodules to their "proper" location in
`src/tools/{cargo,rls}`. Now that Cargo workspaces support the `exclude`
option this can work.
* Updates of the `cargo` and `rls` submodules to their master branches.
* Tweak to the `src/stage0.txt` format to be more amenable for Cargo version
numbers. On the beta channel Cargo will bootstrap from a different version
than rustc (e.g. the version numbers are different), so we need different
configuration for this.
* Addition of `dev` as a readable key in the `src/stage0.txt` format. If present
then stage0 compilers are downloaded from `dev-static.rust-lang.org` instead
of `static.rust-lang.org`. This is added to accomodate our updated release
process with Travis and AppVeyor.
Currently our slowest test suite on android, run-pass, takes over 5 times longer
than the x86_64 component (~400 -> ~2200s). Typically QEMU emulation does indeed
add overhead, but not 5x for this kind of workload. One of the slowest parts of
the Android process is that *compilation* happens serially. Tests themselves
need to run single-threaded on the emulator (due to how the test harness works)
and this forces the compiles themselves to be single threaded.
Now Travis gives us more than one core per machine, so it'd be much better if we
could take advantage of them! The emulator itself is still fundamentally
single-threaded, but we should see a nice speedup by sending binaries for it to
run much more quickly.
It turns out that we've already got all the tools to do this in-tree. The
qemu-test-{server,client} that are in use for the ARM Linux testing are a
perfect match for the Android emulator. This commit migrates the custom adb
management code in compiletest/rustbuild to the same qemu-test-{server,client}
implementation that ARM Linux uses.
This allows us to lift the parallelism restriction on the compiletest test
suites, namely run-pass. Consequently although we'll still basically run the
tests themselves in single threaded mode we'll be able to compile all of them in
parallel, keeping the pipeline much more full and using more cores for the work
at hand. Additionally the architecture here should be a bit speedier as it
should have less overhead than adb which is a whole new process on both the host
and the emulator!
Locally on an 8 core machine I've seen the run-pass test suite speed up from
taking nearly an hour to only taking 6 minutes. I don't think we'll see quite a
drastic speedup on Travis but I'm hoping this change can place the Android tests
well below 2 hours instead of just above 2 hours.
Because the client/server here are now repurposed for more than just QEMU,
they've been renamed to `remote-test-{server,client}`.
Note that this PR does not currently modify how debuginfo tests are executed on
Android. While parallelizable it wouldn't be quite as easy, so that's left to
another day. Thankfully that test suite is much smaller than the run-pass test
suite.
As a final fix I discovered that the ARM and Android test suites were actually
running all library unit tests (e.g. stdtest, coretest, etc) twice. I've
corrected that to only run tests once which should also give a nice boost in
overall cycle time here.
