Merge Apple `std::os` extensions modules into `std::os::darwin`
The functionality available on Apple platforms are very similar, and were (basically) duplicated for each platform.
This PR rectifies that by merging the code into one module.
Ultimately, I've done this to fix `./x build library --target=aarch64-apple-tvos,aarch64-apple-watchos,aarch64-apple-visionos`, as that currently fails because of dead code warnings.
Publically exposing these to tvOS/watchOS/visionOS targets is considered in https://github.com/rust-lang/rust/pull/123723, but that seems to be dragging out, and in any case I think it makes sense to do the refactor separately from stabilization.
r? libs
Fixes https://github.com/rust-lang/rust/issues/121640 and https://github.com/rust-lang/rust/issues/124825.
The functionality available on Apple platforms are very similar, and
were duplicated for each platform.
Additionally, this fixes a warning when compiling the standard library
for tvOS, watchOS and visionOS by marking the corresponding code as
dead code.
Use ManuallyDrop in BufWriter::into_parts
The fact that `mem::forget` takes by value means that it interacts very poorly with Stacked Borrows; generally users think of calling it as a no-op, but in Stacked Borrows, the field retagging tends to cause surprise tag invalidation.
Remove memory leaks in doctests in `core`, `alloc`, and `std`
cc `@RalfJung` https://github.com/rust-lang/rust/issues/126067https://github.com/rust-lang/miri/issues/3670
Should be no actual *documentation* changes[^1], all added/modified lines in the doctests are hidden with `#`,
This PR splits the existing memory leaks in doctests in `core`, `alloc`, and `std` into two general categories:
1. "Non-focused" memory leaks that are incidental to the thing being documented, and/or are easy to remove, i.e. they are only there because preventing the leak would make the doctest less clear and/or concise.
- These doctests simply have a comment like `# // Prevent leaks for Miri.` above the added line that removes the memory leak.
- [^2]Some of these would perhaps be better as part of the public documentation part of the doctest, to clarify that a memory leak can happen if it is not otherwise mentioned explicitly in the documentation (specifically the ones in `(A)Rc::increment_strong_count(_in)`).
2. "Focused" memory leaks that are intentional and documented, and/or are possibly fragile to remove.
- These doctests have a `# // FIXME` comment above the line that removes the memory leak, with a note that once `-Zmiri-disable-leak-check` can be applied at test granularity, these tests should be "un-unleakified" and have `-Zmiri-disable-leak-check` enabled.
- Some of these are possibly fragile (e.g. unleaking the result of `Vec::leak`) and thus should definitely not be made part of the documentation.
This should be all of the leaks currently in `core` and `alloc`. I only found one leak in `std`, and it was in the first category (excluding the modules `@RalfJung` mentioned in https://github.com/rust-lang/rust/issues/126067 , and reducing the number of iterations of [one test](https://github.com/rust-lang/rust/blob/master/library/std/src/sync/once_lock.rs#L49-L94) from 1000 to 10)
[^1]: assuming [^2] is not added
[^2]: backlink
Windows: Add experimental support for linking std-required system DLLs using raw-dylib
For Windows, this allows std to define system imports without needing the user to have import libraries. It's intended for this to become the default.
For now it's an experimental feature so it can be tested using build-std.
fix interleaved output in the default panic hook when multiple threads panic simultaneously
previously, we only held a lock for printing the backtrace itself. since all threads were printing to the same file descriptor, that meant random output in the default panic hook from one thread would be interleaved with the backtrace from another. now, we hold the lock for the full duration of the hook, and the output is ordered.
---
i noticed some odd things while working on this you may or may not already be aware of.
- libbacktrace is included as a submodule instead of a normal rustc crate, and as a result uses `cfg(backtrace_in_std)` instead of a more normal `cfg(feature = "rustc-dep-of-std")`. probably this is left over from before rust used a cargo-based build system?
- the default panic handler uses `trace_unsynchronized`, etc, in `sys::backtrace::print`. as a result, the lock only applies to concurrent *panic handlers*, not concurrent *threads*. in other words, if another, non-panicking, thread tried to print a backtrace at the same time as the panic handler, we may have UB, especially on windows.
- we have the option of changing backtrace to enable locking when `backtrace_in_std` is set so we can reuse their lock instead of trying to add our own.
Guard against calling `libc::exit` multiple times on Linux.
Mitigates (but does not fix) #126600 by ensuring only one thread which calls Rust `exit` actually calls `libc::exit`, and all other callers of Rust `exit` block.
previously, we only held a lock for printing the backtrace itself. since all threads were printing to the same file descriptor, that meant random output in the default panic hook would be interleaved with the backtrace. now, we hold the lock for the full duration of the hook, and the output is ordered.
Use pidfd_spawn for faster process spawning when a PidFd is requested
glibc 2.39 added `pidfd_spawnp` and `pidfd_getpid` which makes it possible to get pidfds while staying on the CLONE_VFORK path.
verified that vfork gets used with strace:
```
$ strace -ff -e pidfd_open,clone3,openat,execve,waitid,close ./x test std --no-doc -- pidfd
[...]
[pid 2820532] clone3({flags=CLONE_VM|CLONE_PIDFD|CLONE_VFORK|CLONE_CLEAR_SIGHAND, pidfd=0x7b7f885fec6c, exit_signal=SIGCHLD, stack=0x7b7f88aff000, stack_size=0x9000}strace: Process 2820533 attached
<unfinished ...>
[pid 2820533] execve("/home/the8472/bin/sleep", ["sleep", "1000"], 0x7ffdd0e268d8 /* 107 vars */) = -1 ENOENT (No such file or directory)
[pid 2820533] execve("/home/the8472/.cargo/bin/sleep", ["sleep", "1000"], 0x7ffdd0e268d8 /* 107 vars */) = -1 ENOENT (No such file or directory)
[pid 2820533] execve("/usr/local/bin/sleep", ["sleep", "1000"], 0x7ffdd0e268d8 /* 107 vars */) = -1 ENOENT (No such file or directory)
[pid 2820533] execve("/usr/bin/sleep", ["sleep", "1000"], 0x7ffdd0e268d8 /* 107 vars */ <unfinished ...>
[pid 2820532] <... clone3 resumed> => {pidfd=[3]}, 88) = 2820533
[pid 2820533] <... execve resumed>) = 0
[pid 2820532] openat(AT_FDCWD, "/proc/self/fdinfo/3", O_RDONLY|O_CLOEXEC) = 4
[pid 2820532] close(4) = 0
```
Tracking issue: #82971
std: Set `has_reliable_f16` to false for MIPS targets in build.rs
This PR makes std tests link for MIPS again (they broke with https://github.com/rust-lang/rust/pull/126608) by avoiding the following link errors. Step-by-step instructions on how to reproduce these errors in docker can be found below.
std.9e27ea-cgu.12:(.text._ZN3std3num8test_num17edc3E+0x38): undefined reference to `__gnu_f2h_ieee'
std.9e27ea-cgu.12:(.text._ZN3std3num8test_num17hdc3E+0x38): undefined reference to `__gnu_h2f_ieee'
This PR just adds one line of config in existing f16 infrastructure. It also disables four doctests that fails with the same link errors.
## Step-by-step to reproduce linking error
1. Prepare:
```sh
docker run -it ubuntu:24.10
apt update && apt install -y \
libc6-mips-cross \
libc6-mipsel-cross \
libc6-mips64-cross \
libc6-mips64el-cross \
gcc-mips-linux-gnu \
gcc-mipsel-linux-gnu \
gcc-mips64-linux-gnuabi64 \
gcc-mips64el-linux-gnuabi64 \
git curl python3 build-essential
git clone --depth 1 https://github.com/rust-lang/rust.git
cd rust
```
2. Try to link std tests for any of these 4 MIPS targets by running any one of these commands:
```sh
CC_mips_unknown_linux_gnu=mips-linux-gnu-gcc \
CARGO_TARGET_MIPS_UNKNOWN_LINUX_GNU_LINKER=mips-linux-gnu-gcc \
./x test library/std --target mips-unknown-linux-gnu
CC_mipsel_unknown_linux_gnu=mipsel-linux-gnu-gcc \
CARGO_TARGET_MIPSEL_UNKNOWN_LINUX_GNU_LINKER=mipsel-linux-gnu-gcc \
./x test library/std --target mipsel-unknown-linux-gnu
CC_mips64_unknown_linux_gnuabi64=mips64-linux-gnuabi64-gcc \
CARGO_TARGET_MIPS64_UNKNOWN_LINUX_GNUABI64_LINKER=mips64-linux-gnuabi64-gcc \
./x test library/std --target mips64-unknown-linux-gnuabi64
CC_mips64el_unknown_linux_gnuabi64=mips64el-linux-gnuabi64-gcc \
CARGO_TARGET_MIPS64EL_UNKNOWN_LINUX_GNUABI64_LINKER=mips64el-linux-gnuabi64-gcc \
./x test library/std --target mips64el-unknown-linux-gnuabi64
```
### Expected
No link error. After this PR there are no link errors.
### Actual
```
error: linking with `mips-linux-gnu-gcc` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "mips-linux-gnu-gcc" "/tmp/rustcEtKsay/symbols.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.00.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.01.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.02.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.03.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.04.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.05.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.06.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.07.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.08.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.09.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.10.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.11.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.12.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.13.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.14.rcgu.o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.15.rcgu.o" "-Wl,--as-needed" "-L" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps" "-L" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/release/deps" "-L" "/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/mips-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/librand_xorshift-deb32232a867c543.rlib" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/librand-5a391600dce9d98f.rlib" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/librand_core-a11cfba3d86c5298.rlib" "/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/mips-unknown-linux-gnu/lib/libtest-65b05caf5a9b99a4.rlib" "/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/mips-unknown-linux-gnu/lib/libgetopts-ba692b2f798aef60.rlib" "/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/mips-unknown-linux-gnu/lib/libunicode_width-20ec8b475126cb0b.rlib" "/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/mips-unknown-linux-gnu/lib/librustc_std_workspace_std-c17f739fee51cc86.rlib" "-L" "/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/mips-unknown-linux-gnu/lib" "-Wl,-Bdynamic" "-lstd-124ee57a4c00deda" "-Wl,-Bstatic" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/libcompiler_builtins-bd55a137b89bc81f.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/mips-unknown-linux-gnu/lib" "-o" "/rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin" "-Wl,-rpath,$ORIGIN/../lib"
= note: /usr/lib/gcc-cross/mips-linux-gnu/12/../../../../mips-linux-gnu/bin/ld: /rust/build/x86_64-unknown-linux-gnu/stage1-std/mips-unknown-linux-gnu/release/deps/std-1cffa50fa8c43b63.std.9ee227e919a554fa-cgu.12.rcgu.o: in function `std::num::test_num':
std.9ee227e919a554fa-cgu.12:(.text._ZN3std3num8test_num17haed2ea710c1afdc3E+0x38): undefined reference to `__gnu_f2h_ieee'
/usr/lib/gcc-cross/mips-linux-gnu/12/../../../../mips-linux-gnu/bin/ld: std.9ee227e919a554fa-cgu.12:(.text._ZN3std3num8test_num17haed2ea710c1afdc3E+0x3c): undefined reference to `__gnu_f2h_ieee'
/usr/lib/gcc-cross/mips-linux-gnu/12/../../../../mips-linux-gnu/bin/ld: std.9ee227e919a554fa-cgu.12:(.text._ZN3std3num8test_num17haed2ea710c1afdc3E+0x44): undefined reference to `__gnu_h2f_ieee'
...
collect2: error: ld returned 1 exit status
error: could not compile `std` (lib test) due to 1 previous error
```
clarify `sys::unix::fd::FileDesc::drop` comment
closes#66876
simply clarifies some resource-relevant things regarding the `close` syscall to reduce the amount of search needed in other parts of the web.
once_lock: make test not take as long in Miri
Allocating 1000 list elements takes a while (`@zachs18` reported >5min), so let's reduce the iteration count when running in Miri. Unfortunately due to this clever `while let i @ 0..LEN =` thing, the count needs to be a constants, and constants cannot be shadowed, so we need to use another trick to hide the `cfg!(miri)` from the docs. (I think this loop condition may be a bit too clever, it took me a bit to decipher. Ideally this would be `while let i = ... && i < LEN`, but that is not stable yet.)
Improve std::Path's Hash quality by avoiding prefix collisions
This adds a bit rotation to the already existing state so that the same sequence of characters chunked at different offsets into separate path components results in different hashes.
The tests are from #127255Closes#127254
