Commit Graph

606 Commits

Author SHA1 Message Date
Aria Beingessner
28576e9c51 mark FIXMES for all the places found that are probably offset_from 2022-03-29 20:18:28 -04:00
Aria Beingessner
b608df8277 revert changes that cast functions to raw pointers, portability hazard 2022-03-29 20:18:27 -04:00
Alexis Beingessner
09395f626b Make some linux/unix APIs better conform to strict provenance.
This largely makes the stdlib conform to strict provenance on Ubuntu.
Some hairier things have been left alone for now.
2022-03-29 20:18:27 -04:00
Marcus Calhoun-Lopez
8c18844324 Fix build on i686-apple-darwin systems
On 32-bit systems, fdopendir is called `_fdopendir$INODE64$UNIX2003`.
On 64-bit systems, fdopendir is called `_fdopendir$INODE64`.
2022-03-28 12:52:14 -07:00
Marcus Calhoun-Lopez
c2d5c64132 Fix build on i686-apple-darwin systems
Replace `target_arch = "x86_64"` with `not(target_arch = "aarch64")` so that i686-apple-darwin systems dynamically choose implementation.
2022-03-28 12:52:14 -07:00
Mara Bos
c9ae3fe68f Explicitly use CLOCK_MONOTONIC in futex_wait.
Instant might be changed to use CLOCK_BOOTTIME at some point.
2022-03-24 11:11:31 +01:00
Mara Bos
23badeb4cb Make Timespec available in sys::unix. 2022-03-24 11:11:03 +01:00
Mara Bos
87299298d9 Use FUTEX_WAIT_BITSET rather than FUTEX_WAIT on Linux. 2022-03-24 09:51:48 +01:00
Mara Bos
da4ef044c1 Spin before blocking in Mutex::lock. 2022-03-23 14:58:44 +01:00
Mara Bos
7f26adeac1 Replace Linux Mutex and Condvar with futex based ones. 2022-03-23 14:58:44 +01:00
Mara Bos
73d63488e4 Add futex_wake_all. 2022-03-23 14:53:59 +01:00
Mara Bos
4fbd71c943 Return timeout status in futex_wait. 2022-03-23 14:53:59 +01:00
bors
36748cf814 Auto merge of #95173 - m-ou-se:sys-locks-module, r=dtolnay
Move std::sys::{mutex, condvar, rwlock} to std::sys::locks.

This cleans up the the std::sys modules a bit by putting the locks in a single module called `locks` rather than spread over the three modules `mutex`, `condvar`, and `rwlock`. This makes it easier to organise lock implementations, which helps with https://github.com/rust-lang/rust/issues/93740.
2022-03-23 06:01:48 +00:00
Mara Bos
ac6996345d Move pthread locks to own module. 2022-03-21 15:51:25 +01:00
Matthias Krüger
acb7ed141b Rollup merge of #94749 - RalfJung:remove-dir-all-miri, r=cuviper
remove_dir_all: use fallback implementation on Miri

Fixes https://github.com/rust-lang/miri/issues/1966

The new implementation requires `openat`, `unlinkat`, and `fdopendir`. These cannot easily be shimmed in Miri since libstd does not expose APIs corresponding to them. So for now it is probably easiest to just use the fallback code in Miri. Nobody should run Miri as root anyway...
2022-03-20 09:14:58 +01:00
Dylan DPC
3545003b29 Rollup merge of #93858 - krallin:process-process_group, r=dtolnay
Add a `process_group` method to UNIX `CommandExt`

- Tracking issue: #93857
- RFC: https://github.com/rust-lang/rfcs/pull/3228

Add a `process_group` method to `std::os::unix::process::CommandExt` that
allows setting the process group id (i.e. calling `setpgid`) in the child, thus
enabling users to set process groups while leveraging the `posix_spawn` fast
path.
2022-03-19 14:50:24 +01:00
Dylan DPC
a87590e34e Rollup merge of #92612 - atopia:update-lib-l4re, r=dtolnay
Update stdlib for the l4re target

This PR contains the work by ``@humenda`` and myself to update standard library support for the x86_64-unknown-l4re-uclibc tier 3 target, split out from  humenda/rust as requested in #85967. The changes have been rebased on current master and updated in follow up commits by myself. The publishing of the changes is authorized and preferred by the original author. To preserve attribution, when standard library changes were introduced as part of other changes to the compiler, I have kept the changes concerning the standard library and altered the commit messages as indicated. Any incompatibilities have been remedied in follow up commits, so that the PR as a whole should result in a clean update of the target.
2022-03-19 02:01:59 +01:00
wcampbell
b1f3179804 feat: Add use of bool::then in sys/unix/process
Remove else { None } in favor of using bool::then()
2022-03-17 19:12:09 -04:00
Thomas Orozco
b628497b7c Add a process_group method to UNIX CommandExt 2022-03-14 14:33:41 +00:00
T-O-R-U-S
72a25d05bf Use implicit capture syntax in format_args
This updates the standard library's documentation to use the new syntax. The
documentation is worthwhile to update as it should be more idiomatic
(particularly for features like this, which are nice for users to get acquainted
with). The general codebase is likely more hassle than benefit to update: it'll
hurt git blame, and generally updates can be done by folks updating the code if
(and when) that makes things more readable with the new format.

A few places in the compiler and library code are updated (mostly just due to
already having been done when this commit was first authored).
2022-03-10 10:23:40 -05:00
Benjamin Lamowski
bc199b5778 add as_raw() method to L4Re's Socket mock
Minimally comply with with #87329 to avoid breaking tests on L4Re.
2022-03-09 11:53:27 +01:00
Benjamin Lamowski
997dc5899a adapt L4Re network interface mock to #87329
Copy the relevant trait implementations from the Unix default.
2022-03-09 11:53:27 +01:00
Benjamin Lamowski
c0dc41f5ff L4Re does not support sanitizing standard streams
L4Re provides limited POSIX support which includes support for
standard I/O streams, and a limited implementation of the standard file
handling API. However, because as a capability based OS it strives to
only make a local view available to each application, there are
currently no standardized special files like /dev/null that could serve
to sanitize closed standard FDs.

For now, skip any attempts to sanitize standard streams until a more
complete POSIX runtime is available.
2022-03-09 11:53:27 +01:00
Benjamin Lamowski
898f379817 drop unused libc imports on L4Re
As a capability-based microkernel OS, L4Re only has incomplete support
for POSIX APIs, in particular it does not implement UIDs and GIDs.
2022-03-09 11:53:27 +01:00
Sebastian Humenda
11b717647e fix return value of LookupHost::port()
[Benjamin Lamowski: Reworded commit message after split commit.]
2022-03-09 11:53:27 +01:00
Sebastian Humenda
7a74d28c38 fix return values in L4Re networking stub
[Benjamin Lamowski: Reworded commit message after split commit.]
2022-03-09 11:53:27 +01:00
bors
163c207fc2 Auto merge of #94750 - cuviper:dirent64_min, r=joshtriplett
unix: reduce the size of DirEntry

On platforms where we call `readdir` instead of `readdir_r`, we store
the name as an allocated `CString` for variable length. There's no point
carrying around a full `dirent64` with its fixed-length `d_name` too.
2022-03-09 02:17:58 +00:00
Dylan DPC
a67b6299b4 Rollup merge of #94724 - cuviper:rmdirall-cstr, r=Dylan-DPC
unix: Avoid name conversions in `remove_dir_all_recursive`

Each recursive call was creating an `OsString` for a `&Path`, only for
it to be turned into a `CString` right away. Instead we can directly
pass `.name_cstr()`, saving two allocations each time.
2022-03-08 22:44:00 +01:00
Josh Stone
e8b9ba84be unix: reduce the size of DirEntry
On platforms where we call `readdir` instead of `readdir_r`, we store
the name as an allocated `CString` for variable length. There's no point
carrying around a full `dirent64` with its fixed-length `d_name` too.
2022-03-08 13:36:01 -08:00
Ralf Jung
2a2b212ea3 remove_dir_all: use fallback implementation on Miri 2022-03-08 16:26:10 -05:00
Josh Stone
ef3e33bd16 unix: Avoid name conversions in remove_dir_all_recursive
Each recursive call was creating an `OsString` for a `&Path`, only for
it to be turned into a `CString` right away. Instead we can directly
pass `.name_cstr()`, saving two allocations each time.
2022-03-07 18:51:53 -08:00
Fausto
776be7e73e promot debug_assert to assert 2022-03-07 15:48:35 -05:00
bors
2631aeef82 Auto merge of #94272 - tavianator:readdir-reclen-for-real, r=cuviper
fs: Don't dereference a pointer to a too-small allocation

ptr::addr_of!((*ptr).field) still requires ptr to point to an
appropriate allocation for its type.  Since the pointer returned by
readdir() can be smaller than sizeof(struct dirent), we need to entirely
avoid dereferencing it as that type.

Link: https://github.com/rust-lang/miri/pull/1981#issuecomment-1048278492
Link: https://github.com/rust-lang/rust/pull/93459#discussion_r795089971
2022-03-07 04:48:23 +00:00
fee1-dead
8ea3f236dc Rollup merge of #94649 - ChrisDenton:unix-absolute-fix, r=Dylan-DPC
Unix path::absolute: Fix leading "." component

Testing leading `.` and `..` components were missing from the unix tests.

This PR adds them and fixes the leading `.` case. It also fixes the test cases so that they do an exact comparison.

This problem reported by ``@axetroy``
2022-03-06 22:35:31 +11:00
Chris Denton
e8b7371a23 Unix path::absolute: Fix leading "." component
Testing leading `.` and `..` components were missing from the unix tests.
2022-03-05 17:57:12 +00:00
Ralf Jung
51b4ea2ba1 do not attempt to open cgroup files under Miri 2022-03-05 11:23:25 -05:00
Dylan DPC
3e1e9b4866 Rollup merge of #94446 - rusticstuff:remove_dir_all-illumos-fix, r=cuviper
UNIX `remove_dir_all()`: Try recursing first on the slow path

This only affects the _slow_ code path - if there is no `dirent.d_type` or if it is `DT_UNKNOWN`.

POSIX specifies that calling `unlink()` or `unlinkat(..., 0)` on a directory is allowed to succeed:
> The _path_ argument shall not name a directory unless the process has appropriate privileges and the implementation supports using _unlink()_ on directories.

This however can cause dangling inodes requiring an fsck e.g. on Illumos UFS, so we have to avoid that in the common case. We now just try to recurse into it first and unlink() if we can't open it as a directory.

The other two commits integrate the Macos x86-64 implementation reducing redundancy. Split into two commits for better reviewing.

Fixes #94335.
2022-03-05 04:46:37 +01:00
Hans Kratz
735f60c34f Integrate macos x86-64 remove_dir_all() impl. Step 2: readd 2022-03-04 13:47:50 +01:00
Hans Kratz
41b4423cdf Integrate macos x86-64 remove_dir_all() impl. Step 1: remove 2022-03-04 13:47:36 +01:00
Hans Kratz
e427333071 remove_dir_all(): try recursing first instead of trying to unlink()
This only affects the `slow` code path, if there is no `dirent.d_type` or if
the type is `DT_UNKNOWN`.

POSIX specifies that calling `unlink()` or `unlinkat(..., 0)` on a directory can
succeed:
> "The _path_ argument shall not name a directory unless the process has
> appropriate privileges and the implementation supports using _unlink()_ on
> directories."
This however can cause orphaned directories requiring an fsck e.g. on Illumos
UFS, so we have to avoid that in the common case. We now just try to recurse
into it first and unlink() if we can't open it as a directory.
2022-03-04 13:33:35 +01:00
Matthias Krüger
a638f50d8d Rollup merge of #92697 - the8472:cgroups, r=joshtriplett
Use cgroup quotas for calculating `available_parallelism`

Automated tests for this are possible but would require a bunch of assumptions. It requires root + a recent kernel, systemd and maybe docker. And even then it would need a helper binary since the test has to run in a separate process.

Limitations

* only supports cgroup v2 and assumes it's mounted under `/sys/fs/cgroup`
* procfs must be available
* the quota gets mixed into `sched_getaffinity`, so if the latter doesn't work then quota information gets ignored too

Manually tested via

```
// spawn a new cgroup scope for the current user
$ sudo systemd-run -p CPUQuota="300%" --uid=$(id -u) -tdS

// quota.rs
#![feature(available_parallelism)]
fn main() {
    println!("{:?}", std:🧵:available_parallelism()); // prints Ok(3)
}
```

strace:

```
sched_getaffinity(3041643, 32, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]) = 32
openat(AT_FDCWD, "/proc/self/cgroup", O_RDONLY|O_CLOEXEC) = 3
statx(0, NULL, AT_STATX_SYNC_AS_STAT, STATX_ALL, NULL) = -1 EFAULT (Bad address)
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0444, stx_size=0, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "0::/system.slice/run-u31477.serv"..., 128) = 36
read(3, "", 92)                         = 0
close(3)                                = 0
statx(AT_FDCWD, "/sys/fs/cgroup/system.slice/run-u31477.service/cgroup.controllers", AT_STATX_SYNC_AS_STAT, STATX_ALL, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0444, stx_size=0, ...}) = 0
openat(AT_FDCWD, "/sys/fs/cgroup/system.slice/run-u31477.service/cpu.max", O_RDONLY|O_CLOEXEC) = 3
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=0, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "300000 100000\n", 20)          = 14
read(3, "", 6)                          = 0
close(3)                                = 0
openat(AT_FDCWD, "/sys/fs/cgroup/system.slice/cpu.max", O_RDONLY|O_CLOEXEC) = 3
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=0, ...}) = 0
lseek(3, 0, SEEK_CUR)                   = 0
read(3, "max 100000\n", 20)             = 11
read(3, "", 9)                          = 0
close(3)                                = 0
openat(AT_FDCWD, "/sys/fs/cgroup/cpu.max", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
sched_getaffinity(0, 128, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47]) = 40
```

r? ```````@joshtriplett```````
cc ```````@yoshuawuyts```````

Tracking issue and previous discussion: #74479
2022-03-03 20:01:43 +01:00
Dylan DPC
c9dc44be24 Rollup merge of #93663 - sunfishcode:sunfishcode/as-raw-name, r=joshtriplett
Rename `BorrowedFd::borrow_raw_fd` to `BorrowedFd::borrow_raw`.

Also, rename `BorrowedHandle::borrow_raw_handle` and
`BorrowedSocket::borrow_raw_socket` to `BorrowedHandle::borrow_raw` and
`BorrowedSocket::borrow_raw`.

This is just a minor rename to reduce redundancy in the user code calling
these functions, and to eliminate an inessential difference between
`BorrowedFd` code and `BorrowedHandle`/`BorrowedSocket` code.

While here, add a simple test exercising `BorrowedFd::borrow_raw_fd`.

r? ``````@joshtriplett``````
2022-03-03 01:09:10 +01:00
The 8472
af6d2ed245 hardcode /sys/fs/cgroup instead of doing a lookup via mountinfo
this avoids parsing mountinfo which can be huge on some systems and
something might be emulating cgroup fs for sandboxing reasons which means
it wouldn't show up as mountpoint

additionally the new implementation operates on a single pathbuffer, reducing allocations
2022-03-03 00:43:46 +01:00
The 8472
bac5523ea0 Use cgroup quotas for calculating available_parallelism
Manually tested via


```
// spawn a new cgroup scope for the current user
$ sudo systemd-run -p CPUQuota="300%" --uid=$(id -u) -tdS


// quota.rs
#![feature(available_parallelism)]
fn main() {
    println!("{:?}", std:🧵:available_parallelism()); // prints Ok(3)
}
```


Caveats

* cgroup v1 is ignored
* funky mountpoints (containing spaces, newlines or control chars) for cgroupfs will not be handled correctly since that would require unescaping /proc/self/mountinfo
  The escaping behavior of procfs seems to be undocumented. systemd and docker default to `/sys/fs/cgroup` so it should be fine for most systems.
* quota will be ignored when `sched_getaffinity` doesn't work
* assumes procfs is mounted under `/proc` and cgroupfs mounted and readable somewhere in the directory tree
2022-03-03 00:43:45 +01:00
Josh Triplett
335c9609c6 Provide C FFI types via core::ffi, not just in std
The ability to interoperate with C code via FFI is not limited to crates
using std; this allows using these types without std.

The existing types in `std::os::raw` become type aliases for the ones in
`core::ffi`. This uses type aliases rather than re-exports, to allow the
std types to remain stable while the core types are unstable.

This also moves the currently unstable `NonZero_` variants and
`c_size_t`/`c_ssize_t`/`c_ptrdiff_t` types to `core::ffi`, while leaving
them unstable.
2022-03-01 17:16:05 -08:00
Tavian Barnes
478cf8b3a4 fs: Don't dereference a pointer to a too-small allocation
ptr::addr_of!((*ptr).field) still requires ptr to point to an
appropriate allocation for its type.  Since the pointer returned by
readdir() can be smaller than sizeof(struct dirent), we need to entirely
avoid dereferencing it as that type.

Link: https://github.com/rust-lang/miri/pull/1981#issuecomment-1048278492
Link: https://github.com/rust-lang/rust/pull/93459#discussion_r795089971
2022-02-23 09:51:02 -05:00
David Carlier
f810314bc6 solarish current_exe using libc call directly 2022-02-20 08:53:18 +00:00
bors
1f4681ad7a Auto merge of #91673 - ChrisDenton:path-absolute, r=Mark-Simulacrum
`std::path::absolute`

Implements #59117 by adding a `std::path::absolute` function that creates an absolute path without reading the filesystem. This is intended to be a drop-in replacement for [`std::fs::canonicalize`](https://doc.rust-lang.org/std/fs/fn.canonicalize.html) in cases where it isn't necessary to resolve symlinks. It can be used on paths that don't exist or where resolving symlinks is unwanted. It can also be used to avoid circumstances where `canonicalize` might otherwise fail.

On Windows this is a wrapper around [`GetFullPathNameW`](https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfullpathnamew). On Unix it partially implements the POSIX [pathname resolution](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_13) specification, stopping just short of actually resolving symlinks.
2022-02-13 12:03:52 +00:00
The8472
9d8ef11607 make Instant::{duration_since, elapsed, sub} saturating and remove workarounds
This removes all mutex/atomics based workarounds for non-monotonic clocks and makes the previously panicking methods saturating instead.

Effectively this moves the monotonization from `Instant` construction to the comparisons.

This has some observable effects, especially on platforms without monotonic clocks:

* Incorrectly ordered Instant comparisons no longer panic. This may hide some programming errors until someone actually looks at the resulting `Duration`
* `checked_duration_since` will now return `None` in more cases. Previously it only happened when one compared instants obtained in the wrong order or
  manually created ones. Now it also does on backslides.

The upside is reduced complexity and lower overhead of `Instant::now`.
2022-02-13 01:04:55 +01:00
Yuki Okushi
a898b31662 Rename to InvalidFilename 2022-02-10 23:49:27 +09:00