Previously, the thread name (&str) was converted to a CString in the
new thread, but outside unwind::try, causing a panic to continue into FFI.
This patch changes that behaviour, so that the panic instead happens
in the parent thread (where panic infrastructure is properly set up),
not the new thread.
This could potentially be a breaking change for architectures who don't
support thread names.
Signed-off-by: David Henningsson <diwic@ubuntu.com>
Automated conversion using the untry tool [1] and the following command:
```
$ find -name '*.rs' -type f | xargs untry
```
at the root of the Rust repo.
[1]: https://github.com/japaric/untry
This code was currently only exercised on OSX, but this applies the same method
of subtraction used on Linux which doesn't have the same overflow issues.
Note that this currently includes no tests, but that's because this is only
visible with debug assertions enabled. Soon, however, I'll enable debug
assertions on all auto builds on the bots so we should get testing for this.
Closes#32268
Removes all unstable and deprecated APIs prior to the 1.8 release. All APIs that
are deprecated in the 1.8 release are sticking around for the rest of this
cycle.
Some notable changes are:
* The `dynamic_lib` module was moved into `rustc_back` as the compiler still
relies on a few bits and pieces.
* The `DebugTuple` formatter now special-cases an empty struct name with only
one field to append a trailing comma.
Semantically there's actually no reason for us to spawn threads as part of the
call to `wait_with_output`, and that's generally an incredibly heavyweight
operation for just reading a few bytes (especially when stderr probably rarely
has bytes!). An equivalent operation in terms of what's implemented today would
be to just drain both pipes of all contents and then call `wait` on the child
process itself.
On Unix we can implement this through some convenient use of the `select`
function, whereas on Windows we can make use of overlapped I/O. Note that on
Windows this requires us to use named pipes instead of anonymous pipes, but
they're semantically the same under the hood.
For example if `Command::output` or `Command::status` is used then stdin is just
immediately closed. Add an option for this so as an optimization we can avoid
creating pipes entirely.
This should help reduce the number of active file descriptors when spawning
processes on Unix and the number of active handles on Windows.
This pushes the implementation detail of proxying `read_to_end` through to
`read_to_end_uninitialized` all the way down to the `FileDesc` and `Handle`
implementations on Unix/Windows. This way intermediate layers will also be able
to take advantage of this optimized implementation.
This commit also adds the optimized implementation for `ChildStdout` and
`ChildStderr`.
Squashed 10 commits:
1) The main cause of the problem is that libstd/os/mod.rs treats emscripten targets as an alias of linux targets, whereas liblibc treats emscripten targets as musl-compliant, so it gets a slightly different struct stat64 defined.
This commit adds conditional compilation checks to use the correct timestamp format on fs metadata functions in the case of compiling to emscripten targets.
2) Update previous commit to comply with rust formatting standards.
Removed tab characters, remove trailing whitespaces.
3) Move emscripten changes into their own file under libstd/os/emscripten
Put libstd/os/linux/fs back to the way it was.
4) Cannot use stat.st_ctim on emscripten to get created time.
5) Remove compile-time conditionals for target_env = musl, it looks like musl builds compile fine already.
6) Undone some formatting changes that are no longer needed,
Removed some more target_env="musl" compilation checks that I missed from my previous commit.
7) upgrade to liblibc e19309c, it fixes the differences in the musl stat and stat64 definitions.
8) Undo the compile-time checks to check for emscripten (or musl targets) in the FileAttr struct.
No longer needed after updating liblibc to e19309c.
9) Change the MetadataExt implementation of emscripten fs.rs module to match the changes in new liblibc.
10) remove a stray return statement, should have been removed in the previous commit.
Also back out keepalive support for TCP since the API is perhaps not
actually what we want. You can't read the interval on Windows, and
we should probably separate the functionality of turning keepalive on
and overriding the interval.
Android should use 64-bit LFS symbols for `lseek` and `ftruncate`, lest
those offset parameters suffer a lossy cast down to a 32-bit `off_t`.
Unlike GNU/Linux, Android's `stat`, `dirent`, and related functions are
always 64-bit LFS compatible, and `open` already implies `O_LARGEFILE`,
so all those don't need to follow Linux. It might be nice to unify them
anyway, but those other LFS symbols aren't present in API 18 bionic.
r? @alexcrichton
Some struct members have a slighty different name on NetBSD. This has been
fixed in the libc crate, but not in libstd.
This also removes `st_spare` from MetadataExt, since it is private field
reserved for future use.
Android should use 64-bit LFS symbols for `lseek` and `ftruncate`, lest
those offset parameters suffer a lossy cast down to a 32-bit `off_t`.
Unlike GNU/Linux, Android's `stat`, `dirent`, and related functions are
always 64-bit LFS compatible, and `open` already implies `O_LARGEFILE`,
so all those don't need to follow Linux. It might be nice to unify them
anyway, but those other LFS symbols aren't present in API 18 bionic.
r? @alexcrichton
Remove alternate stack with sigaltstack before unmaping it.
Also reuse existing signal stack if already set, this is especially
useful when working with sanitizers that configure alternate stack
themselves.
This change depends on SS_DISABLE recently introduced in libc crate and updates
this git submodule accordingly.
This commit is an implementation of [RFC 1415][rfc] which deprecates all types
in the `std::os::*::raw` modules.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1415-trim-std-os.md
Many of the types in these modules don't actually have a canonical platform
representation, for example the definition of `stat` on 32-bit Linux will change
depending on whether C code is compiled with LFS support or not. Unfortunately
the current types in `std::os::*::raw` are billed as "compatible with C", which
in light of this means it isn't really possible.
To make matters worse, platforms like Android sometimes define these types as
*smaller* than the way they're actually represented in the `stat` structure
itself. This means that when methods like `DirEntry::ino` are called on Android
the result may be truncated as we're tied to returning a `ino_t` type, not the
underlying type.
The commit here incorporates two backwards-compatible components:
* Deprecate all `raw` types that aren't in `std::os::raw`
* Expand the `std::os::*::fs::MetadataExt` trait on all platforms for method
accessors of all fields. The fields now returned widened types which are the
same across platforms (consistency across platforms is not required, however,
it's just convenient).
and two also backwards-incompatible components:
* Change the definition of all `std::os::*::raw` type aliases to
correspond to the newly widened types that are being returned on each
platform.
* Change the definition of `std::os::*::raw::stat` on Linux to match the LFS
definitions rather than the standard ones.
The breaking changes here will specifically break code that assumes that `libc`
and `std` agree on the definition of `std::os::*::raw` types, or that the `std`
types are faithful representations of the types in C. An [audit] has been
performed of crates.io to determine the fallout which was determined two be
minimal, with the two found cases of breakage having been fixed now.
[audit]: https://github.com/rust-lang/rfcs/pull/1415#issuecomment-180645582
---
Ok, so after all that, we're finally able to support LFS on Linux! This commit
then simultaneously starts using `stat64` and friends on Linux to ensure that we
can open >4GB files on 32-bit Linux. Yay!
Closes#28978Closes#30050Closes#31549
This commit is an implementation of [RFC 1415][rfc] which deprecates all types
in the `std::os::*::raw` modules.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1415-trim-std-os.md
Many of the types in these modules don't actually have a canonical platform
representation, for example the definition of `stat` on 32-bit Linux will change
depending on whether C code is compiled with LFS support or not. Unfortunately
the current types in `std::os::*::raw` are billed as "compatible with C", which
in light of this means it isn't really possible.
To make matters worse, platforms like Android sometimes define these types as
*smaller* than the way they're actually represented in the `stat` structure
itself. This means that when methods like `DirEntry::ino` are called on Android
the result may be truncated as we're tied to returning a `ino_t` type, not the
underlying type.
The commit here incorporates two backwards-compatible components:
* Deprecate all `raw` types that aren't in `std::os::raw`
* Expand the `std::os::*::fs::MetadataExt` trait on all platforms for method
accessors of all fields. The fields now returned widened types which are the
same across platforms (consistency across platforms is not required, however,
it's just convenient).
and two also backwards-incompatible components:
* Change the definition of all `std::os::*::raw` type aliases to
correspond to the newly widened types that are being returned on each
platform.
* Change the definition of `std::os::*::raw::stat` on Linux to match the LFS
definitions rather than the standard ones.
The breaking changes here will specifically break code that assumes that `libc`
and `std` agree on the definition of `std::os::*::raw` types, or that the `std`
types are faithful representations of the types in C. An [audit] has been
performed of crates.io to determine the fallout which was determined two be
minimal, with the two found cases of breakage having been fixed now.
[audit]: https://github.com/rust-lang/rfcs/pull/1415#issuecomment-180645582
---
Ok, so after all that, we're finally able to support LFS on Linux! This commit
then simultaneously starts using `stat64` and friends on Linux to ensure that we
can open >4GB files on 32-bit Linux. Yay!
Closes#28978Closes#30050Closes#31549
This series of commits adds the initial implementation of a new build system for
the compiler and standard library based on Cargo. The high-level architecture
now looks like:
1. The `./configure` script is run with `--enable-rustbuild` and other standard
configuration options.
2. A `Makefile` is generate which proxies commands to the new build system.
3. The new build system has a Python script entry point which manages
downloading both a Rust and Cargo nightly. This initial script also manages
building the build system itself (which is written in Rust).
4. The build system, written in rust and called `bootstrap`, architects how to
call `cargo` and manages building all native libraries and such.
One might reasonably ask "why rewrite the build system?", which is a good
question! The Rust project has used Makefiles for as long as I can remember at
least, and while ugly and difficult to use are undeniably robust as they contain
years worth of tweaking and tuning for working on as many platforms in as many
situation as possible. The rationale behind this PR, however is:
* The makefiles are impenetrable to all but a few people on this
planet. This means that contributions to the build system are almost
nonexistent, and furthermore if a build system change is needed it's
incredibly difficult to figure out how to do so. This hindrance prevents us
from doing some "perhaps fancier" things we may wish to do in make.
* Our build system, while portable, is unfortunately not infinitely portable
everywhere. For example the recently-introduced MSVC target is quite unlikely
to have `make` installed by default (e.g. it requires building inside of an
MSYS2 shell currently). Conversely, the portability of make comes at a cost of
crazy and weird hacks to work around all sorts of versions of software
everywhere, especially when it comes to the configure script and makefiles.
By rewriting this logic in one of the most robust platforms there is, Rust,
we get to assuage all of these worries for free!
* There's a standard tool to build Rust crates, Cargo, but the standard library
and compiler don't use it. This means that they cannot benefit easily from the
crates.io ecosystem, nor can the ecosystem benefit from a standard way to
build this repository itself. Moving to Cargo should help assuage both of
these needs. This has the added benefit of making the compiler more
approachable for newbies as working on the compiler will just happen to be
working on a large Cargo project, all the same standard tools and tricks will
apply.
* There's a huge amount of portability information in the main distribution, for
example around cross compiling, compiling on new OSes, etc. Pushing this logic
into standard crates (like `gcc`) enables the community to immediately benefit
from new build logic.
Despite these benefits, it's going to be a long road to actually replace our
current build system. This PR is just the beginning and doesn't implement the
full suite of functionality as the current one, but there are many more to
follow! The current implementation strategy hopes to look like:
1. Land a second build system in-tree that can be itereated on an and
contributed to. This will not be used just yet in terms of gating new commits
to the repo.
2. Over time, bring the second build system to feature parity with the old build
system, start setting up CI for both build systems.
3. At some point in the future, switch the default to the new build system, but
keep the old one around.
4. At some further point in the future, delete the entire old build system.
---
Alright, so with all that out of the way, here's some more info on this PR
itself. The inital build system here is contained in the `src/bootstrap`
directory and just adds the necessary minimum bits to bootstrap the compiler
itself. There is currently no support for building documentation, running tests,
or installing, but the implemented support is:
* Compiling LLVM with `cmake` instead of `./configure` + `make`. The LLVM
project is removing their autotools build system, so we'd have to make this
transition eventually anyway.
* Compiling compiler-rt with `cmake` as well (for the same rationale as above).
* Adding `Cargo.toml` to map out the dependency graph to all crates, and also
adding `build.rs` files where appropriate. For example `alloc_jemalloc` has a
script to build jemalloc, `flate` has a script to build `miniz.c`, `std` will
build `libbacktrace`, etc.
* Orchestrating all the calls to `cargo` to build the standard distribution,
following the normal bootstrapping process. This also tracks dependencies
between steps to ensure cross-compilation targets happen as well.
* Configuration is intended to eventually be done through a `config.toml` file,
so support is implemented for this. The most likely vector of configuration
for now, however, is likely through `config.mk` (what `./configure` emits), so
the build system currently parses this information.
There's still quite a few steps left to do, and I'll open up some follow-up
issues (as well as a tracking issue) for this migration, but hopefully this is a
great start to get going! This PR is currently tested on all the
Windows/Linux/OSX triples for x86\_64 and x86, but more portability is always
welcome!
---
Future functionality left to implement
* [ ] Re-verify that multi-host builds work
* [ ] Verify android build works
* [ ] Verify iOS build work (mostly compiler-rt)
* [ ] Verify sha256 and ideally gpg of downloaded nightly compiler and nightly rustc
* [ ] Implement testing -- this is a huge bullet point with lots of sub-bullets
* [ ] Build and generate documentation (plus the various tools we have in-tree)
* [ ] Move various src/etc scripts into Rust -- not sure how this interacts with `make` build system
* [ ] Implement `make install` - like testing this is also quite massive
* [x] Deduplicate version information with makefiles
Have all Cargo-built crates pass `--cfg cargobuild` and then add appropriate
`#[cfg]` definitions to all crates to avoid linking anything if this is passed.
This should help allow libstd to compile with both the makefiles and with Cargo.
This commit implements the `exec` function proposed in [RFC 1359][rfc] which is
a function on the `CommandExt` trait to execute all parts of a `Command::spawn`
without the `fork` on Unix. More details on the function itself can be found in
the comments in the commit.
[rfc]: https://github.com/rust-lang/rfcs/pull/1359
cc #31398
Most of this is platform-specific anyway, and we generally have to jump through
fewer hoops to do the equivalent operation on Windows. One benefit for Windows
today is that this new structure avoids an extra `DuplicateHandle` when creating
pipes. For Unix, however, the behavior should be the same.
Note that this is just a pure refactoring, no functionality was added or
removed.
On Unix we have to be careful to not call `waitpid` twice, but we don't have to
be careful on Windows due to the way process handles work there. As a result the
cached `Option<ExitStatus>` is only necessary on Unix, and it's also just an
implementation detail of the Unix module.
At the same time. also update some code in `kill` on Unix to avoid a wonky
waitpid with WNOHANG. This was added in 0e190b9a to solve #13124, but the
`signal(0)` method is not supported any more so there's no need to for this
workaround. I believe that this is no longer necessary as it's not really doing
anything.
This is a Unix-specific function which adds the ability to register a closure to
run pre-exec to configure the child process as required (note that these
closures are run post-fork).
cc #31398
* Build up the argp/envp pointers while the `Command` is being constructed
rather than only when `spawn` is called. This will allow better sharing of
code between fork/exec paths.
* Rename `child_after_fork` to `exec` and have it only perform the exec half of
the spawning. This also means the return type has changed to `io::Error`
rather than `!` to represent errors that happen.
