Add support for test suites emulated in QEMU
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
This is much nicer for callers who want to short-circuit real I/O errors
with `?`, because they can write this
if let Some(status) = foo.try_wait()? {
...
} else {
...
}
instead of this
match foo.try_wait() {
Ok(status) => {
...
}
Err(err) if err.kind() == io::ErrorKind::WouldBlock => {
...
}
Err(err) => return Err(err),
}
The original design of `try_wait` was patterned after the `Read` and
`Write` traits, which support both blocking and non-blocking
implementations in a single API. But since `try_wait` is never blocking,
it makes sense to optimize for the non-blocking case.
Tracking issue: https://github.com/rust-lang/rust/issues/38903
Use less syscalls in `FileDesc::set_{nonblocking,cloexec}`
Only set the flags if they differ from what the OS reported, use
`FIONBIO` to atomically set the non-blocking IO flag on Linux.
Add peek APIs to std::net
Adds "peek" APIs to `std::net` sockets, including:
- `UdpSocket.peek()`
- `UdpSocket.peek_from()`
- `TcpStream.peek()`
These methods enable socket reads without side-effects. That is, repeated calls to `peek()` return identical data. This is accomplished by providing the POSIX flag `MSG_PEEK` to the underlying socket read operations.
This also moves the current implementation of `recv_from` out of the platform-independent `sys_common` and into respective `sys/windows` and `sys/unix` implementations. This allows for more platform-dependent implementations where necessary.
Fixes#38980
Support unprivileged symlink creation in Windows
Symlink creation on Windows has in the past basically required admin; it’s being opened up a bit in the Creators Update, so that at least people who have put their computers into Developer Mode will be able to create symlinks without special privileges. (It’s unclear from what Microsoft has said whether Developer Mode will be required in the final Creators Update release, but sadly I expect it still will be, so this *still* won’t be as helpful as I’d like.)
Because of compatibility concerns, they’ve hidden this new functionality behind a new flag in the CreateSymbolicLink dwFlags: `SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE`. So we add this flag in order to join the party.
Sources:
- https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/ is the official announcement (search for CreateSymbolicLink)
- https://news.ycombinator.com/item?id=13096354 on why the new flag.
Expand documentation of process::exit and exec
Show a conventional way to use process::exit when destructors are considered important and also
mention that the same caveats wrt destructors apply to exec as well.
These methods enable socket reads without side-effects. That is,
repeated calls to peek() return identical data. This is accomplished
by providing the POSIX flag MSG_PEEK to the underlying socket read
operations.
This also moves the current implementation of recv_from out of the
platform-independent sys_common and into respective sys/windows and
sys/unix implementations. This allows for more platform-dependent
implementations.
Use less syscalls in `anon_pipe()`
Save a `ENOSYS` failure from `pipe2` and don't try again.
Use `cvt` instead of `cvt_r` for `pipe2` - `EINTR` is not an error
`pipe2` can return.
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
Make backtraces work on Windows GNU targets again.
This is done by adding a function that can return a filename
to pass to backtrace_create_state. The filename is obtained in
a safe way by first getting the filename, locking the file so it can't
be moved, and then getting the filename again and making sure it's the same.
See: https://github.com/rust-lang/rust/pull/37359#issuecomment-260123399
Issue: #33985
Note though that this isn't that pretty...
I had to implement a `WideCharToMultiByte` wrapper function to convert to the ANSI code page. This will work better than only allowing ASCII provided that the ANSI code page is set to the user's local language, which is often the case.
Also, please make sure that I didn't break the Unix build.
std: Stabilize APIs for the 1.16.0 release
This commit applies the stabilization/deprecations of the 1.16.0 release, as
tracked by the rust-lang/rust issue tracker and the final-comment-period tag.
The following APIs were stabilized:
* `VecDeque::truncate`
* `VecDeque::resize`
* `String::insert_str`
* `Duration::checked_{add,sub,div,mul}`
* `str::replacen`
* `SocketAddr::is_ipv{4,6}`
* `IpAddr::is_ipv{4,6}`
* `str::repeat`
* `Vec::dedup_by`
* `Vec::dedup_by_key`
* `Result::unwrap_or_default`
* `<*const T>::wrapping_offset`
* `<*mut T>::wrapping_offset`
* `CommandExt::creation_flags` (on Windows)
* `File::set_permissions`
* `String::split_off`
The following APIs were deprecated
* `EnumSet` - replaced with other ecosystem abstractions, long since unstable
Closes#27788Closes#35553Closes#35774Closes#36436Closes#36949Closes#37079Closes#37087Closes#37516Closes#37827Closes#37916Closes#37966Closes#38080
This commit applies the stabilization/deprecations of the 1.16.0 release, as
tracked by the rust-lang/rust issue tracker and the final-comment-period tag.
The following APIs were stabilized:
* `VecDeque::truncate`
* `VecDeque::resize`
* `String::insert_str`
* `Duration::checked_{add,sub,div,mul}`
* `str::replacen`
* `SocketAddr::is_ipv{4,6}`
* `IpAddr::is_ipv{4,6}`
* `str::repeat`
* `Vec::dedup_by`
* `Vec::dedup_by_key`
* `Result::unwrap_or_default`
* `<*const T>::wrapping_offset`
* `<*mut T>::wrapping_offset`
* `CommandExt::creation_flags` (on Windows)
* `File::set_permissions`
* `String::split_off`
The following APIs were deprecated
* `EnumSet` - replaced with other ecosystem abstractions, long since unstable
Closes#27788Closes#35553Closes#35774Closes#36436Closes#36949Closes#37079Closes#37087Closes#37516Closes#37827Closes#37916Closes#37966Closes#38080
This is done by adding a function that can return a filename
to pass to backtrace_create_state. The filename is obtained in
a safe way by first getting the filename, locking the file so it can't
be moved, and then getting the filename again and making sure it's the same.
See: https://github.com/rust-lang/rust/pull/37359#issuecomment-260123399
Issue: #33985
Show a conventional way to use process::exit when destructors are considered important and also
mention that the same caveats wrt destructors apply to exec as well.
std: Add a nonblocking `Child::try_wait` method
This commit adds a new method to the `Child` type in the `std::process` module
called `try_wait`. This method is the same as `wait` except that it will not
block the calling thread and instead only attempt to collect the exit status. On
Unix this means that we call `waitpid` with the `WNOHANG` flag and on Windows it
just means that we pass a 0 timeout to `WaitForSingleObject`.
Currently it's possible to build this method out of tree, but it's unfortunately
tricky to do so. Specifically on Unix you essentially lose ownership of the pid
for the process once a call to `waitpid` has succeeded. Although `Child` tracks
this state internally to be resilient to multiple calls to `wait` or a `kill`
after a successful wait, if the child is waited on externally then the state
inside of `Child` is not updated. This means that external implementations of
this method must be extra careful to essentially not use a `Child`'s methods
after a call to `waitpid` has succeeded (even in a nonblocking fashion).
By adding this functionality to the standard library it should help canonicalize
these external implementations and ensure they can continue to robustly reuse
the `Child` type from the standard library without worrying about pid ownership.
Symlink creation on Windows has in the past basically required admin;
it’s being opened up a bit in the Creators Update, so that at least
people who have put their computers into Developer Mode will be able to
create symlinks without special privileges. (Microsoft are being
cautious about it all; the Developer Mode requirement makes it so that
it this won’t be as helpful as I’d like, but it’s still an improvement
over requiring admin.)
Because of compatibility concerns, they’ve hidden this new functionality
behind a new flag in the CreateSymbolicLink dwFlags:
SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE. So we add this flag in
order to join the party.
Older Windows doesn’t like this new flag, though, so if we encounter
ERROR_INVALID_PARAMETER we try again without the new flag.
Sources:
- https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/
is the official announcement (search for CreateSymbolicLink)
- https://news.ycombinator.com/item?id=13096354 on why the new flag.
- https://twitter.com/richturn_ms/status/818167548269051905 confirming
that Developer Mode will still be required.
This commit adds a new method to the `Child` type in the `std::process` module
called `try_wait`. This method is the same as `wait` except that it will not
block the calling thread and instead only attempt to collect the exit status. On
Unix this means that we call `waitpid` with the `WNOHANG` flag and on Windows it
just means that we pass a 0 timeout to `WaitForSingleObject`.
Currently it's possible to build this method out of tree, but it's unfortunately
tricky to do so. Specifically on Unix you essentially lose ownership of the pid
for the process once a call to `waitpid` has succeeded. Although `Child` tracks
this state internally to be resilient to multiple calls to `wait` or a `kill`
after a successful wait, if the child is waited on externally then the state
inside of `Child` is not updated. This means that external implementations of
this method must be extra careful to essentially not use a `Child`'s methods
after a call to `waitpid` has succeeded (even in a nonblocking fashion).
By adding this functionality to the standard library it should help canonicalize
these external implementations and ensure they can continue to robustly reuse
the `Child` type from the standard library without worrying about pid ownership.
This commit fixes a mistake introduced in #31618 where overlapped handles were
leaked to child processes on Windows. On Windows once a handle is in overlapped
mode it should always have I/O executed with an instance of `OVERLAPPED`. Most
child processes, however, are not prepared to have their stdio handles in
overlapped mode as they don't use `OVERLAPPED` on reads/writes to the handle.
Now we haven't had any odd behavior in Rust up to this point, and the original
bug was introduced almost a year ago. I believe this is because it turns out
that if you *don't* pass an `OVERLAPPED` then the system will [supply one for
you][link]. In this case everything will go awry if you concurrently operate on
the handle. In Rust, however, the stdio handles are always locked, and there's
no way to not use them unlocked in libstd. Due to that change we've always had
synchronized access to these handles, which means that Rust programs typically
"just work".
Conversely, though, this commit fixes the test case included, which exhibits
behavior that other programs Rust spawns may attempt to execute. Namely, the
stdio handles may be concurrently used and having them in overlapped mode wreaks
havoc.
[link]: https://blogs.msdn.microsoft.com/oldnewthing/20121012-00/?p=6343Closes#38811
Add socket timeout and ttl support in `sys::redox`
This adds support for `read_timeout`, `write_timeout`, and `ttl` on `TcpStream`, `TcpListener`, and `UdpSocket` in the `sys::redox` module.
The DNS lookup has been set to use a 5 second timeout by default.
std: Clamp max read/write sizes on Unix
Turns out that even though all these functions take a `size_t` they don't
actually work that well with anything larger than the maximum value of
`ssize_t`, the return value. Furthermore it looks like OSX rejects any
read/write requests larger than `INT_MAX - 1`. Handle all these cases by just
clamping the maximum size of a read/write on Unix to a platform-specific value.
Closes#38590
Ctrl-Z returns from Stdin.read() when reading from the console on Windows
Fixes#19914.
Fixes read(), read_to_string(), read_to_end(), etc.
r? @alexcrichton
Turns out that even though all these functions take a `size_t` they don't
actually work that well with anything larger than the maximum value of
`ssize_t`, the return value. Furthermore it looks like OSX rejects any
read/write requests larger than `INT_MAX - 1`. Handle all these cases by just
clamping the maximum size of a read/write on Unix to a platform-specific value.
Closes#38590
Removes magenta build warning.
Small bug fix to remove an unused type in the magenta process code that causes build failures for magenta's rustc.
r? @alexcrichton
@tedsta @raphlinus
Redox Cross Compilation
I will admit - there are things here that I wish I did not have to do. This completes the ability to create a cross compiler from the rust repository for `x86_64-unknown-redox`. I will document this PR with inline comments explaining some things.
[View this gist to see how a cross compiler is built](https://gist.github.com/jackpot51/6680ad973986e84d69c79854249f2b7e)
Prior discussion of a smaller change is here: https://github.com/rust-lang/rust/pull/38366
