* "while until either" could also be changed to "for a while until either", but I just deleted "while".
* fixed sentence with incorrect "at" and "has/have".
* linked [*currently allocated*] similar to other methods.
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the
prelude instead of importing or qualifying them.
These functions were added to all preludes in Rust 1.80.
Const stability checks v2
The const stability system has served us well ever since `const fn` were first stabilized. It's main feature is that it enforces *recursive* validity -- a stable const fn cannot internally make use of unstable const features without an explicit marker in the form of `#[rustc_allow_const_fn_unstable]`. This is done to make sure that we don't accidentally expose unstable const features on stable in a way that would be hard to take back. As part of this, it is enforced that a `#[rustc_const_stable]` can only call `#[rustc_const_stable]` functions. However, some problems have been coming up with increased usage:
- It is baffling that we have to mark private or even unstable functions as `#[rustc_const_stable]` when they are used as helpers in regular stable `const fn`, and often people will rather add `#[rustc_allow_const_fn_unstable]` instead which was not our intention.
- The system has several gaping holes: a private `const fn` without stability attributes whose inherited stability (walking up parent modules) is `#[stable]` is allowed to call *arbitrary* unstable const operations, but can itself be called from stable `const fn`. Similarly, `#[allow_internal_unstable]` on a macro completely bypasses the recursive nature of the check.
Fundamentally, the problem is that we have *three* disjoint categories of functions, and not enough attributes to distinguish them:
1. const-stable functions
2. private/unstable functions that are meant to be callable from const-stable functions
3. functions that can make use of unstable const features
Functions in the first two categories cannot use unstable const features and they can only call functions from the first two categories.
This PR implements the following system:
- `#[rustc_const_stable]` puts functions in the first category. It may only be applied to `#[stable]` functions.
- `#[rustc_const_unstable]` by default puts functions in the third category. The new attribute `#[rustc_const_stable_indirect]` can be added to such a function to move it into the second category.
- `const fn` without a const stability marker are in the second category if they are still unstable. They automatically inherit the feature gate for regular calls, it can now also be used for const-calls.
Also, all the holes mentioned above have been closed. There's still one potential hole that is hard to avoid, which is when MIR building automatically inserts calls to a particular function in stable functions -- which happens in the panic machinery. Those need to be manually marked `#[rustc_const_stable_indirect]` to be sure they follow recursive const stability. But that's a fairly rare and special case so IMO it's fine.
The net effect of this is that a `#[unstable]` or unmarked function can be constified simply by marking it as `const fn`, and it will then be const-callable from stable `const fn` and subject to recursive const stability requirements. If it is publicly reachable (which implies it cannot be unmarked), it will be const-unstable under the same feature gate. Only if the function ever becomes `#[stable]` does it need a `#[rustc_const_unstable]` or `#[rustc_const_stable]` marker to decide if this should also imply const-stability.
Adding `#[rustc_const_unstable]` is only needed for (a) functions that need to use unstable const lang features (including intrinsics), or (b) `#[stable]` functions that are not yet intended to be const-stable. Adding `#[rustc_const_stable]` is only needed for functions that are actually meant to be directly callable from stable const code. `#[rustc_const_stable_indirect]` is used to mark intrinsics as const-callable and for `#[rustc_const_unstable]` functions that are actually called from other, exposed-on-stable `const fn`. No other attributes are required.
Also see the updated dev-guide at https://github.com/rust-lang/rustc-dev-guide/pull/2098.
I think in the future we may want to tweak this further, so that in the hopefully common case where a public function's const-stability just exactly mirrors its regular stability, we never have to add any attribute. But right now, once the function is stable this requires `#[rustc_const_stable]`.
### Open question
There is one point I could see we might want to do differently, and that is putting `#[rustc_const_unstable]` functions (but not intrinsics) in category 2 by default, and requiring an extra attribute for `#[rustc_const_not_exposed_on_stable]` or so. This would require a bunch of extra annotations, but would have the advantage that turning a `#[rustc_const_unstable]` into `#[rustc_const_stable]` will never change the way the function is const-checked. Currently, we often discover in the const stabilization PR that a function needs some other unstable const things, and then we rush to quickly deal with that. In this alternative universe, we'd work towards getting rid of the `rustc_const_not_exposed_on_stable` before stabilization, and once that is done stabilization becomes a trivial matter. `#[rustc_const_stable_indirect]` would then only be used for intrinsics.
I think I like this idea, but might want to do it in a follow-up PR, as it will need a whole bunch of annotations in the standard library. Also, we probably want to convert all const intrinsics to the "new" form (`#[rustc_intrinsic]` instead of an `extern` block) before doing this to avoid having to deal with two different ways of declaring intrinsics.
Cc `@rust-lang/wg-const-eval` `@rust-lang/libs-api`
Part of https://github.com/rust-lang/rust/issues/129815 (but not finished since this is not yet sufficient to safely let us expose `const fn` from hashbrown)
Fixes https://github.com/rust-lang/rust/issues/131073 by making it so that const-stable functions are always stable
try-job: test-various
Fundamentally, we have *three* disjoint categories of functions:
1. const-stable functions
2. private/unstable functions that are meant to be callable from const-stable functions
3. functions that can make use of unstable const features
This PR implements the following system:
- `#[rustc_const_stable]` puts functions in the first category. It may only be applied to `#[stable]` functions.
- `#[rustc_const_unstable]` by default puts functions in the third category. The new attribute `#[rustc_const_stable_indirect]` can be added to such a function to move it into the second category.
- `const fn` without a const stability marker are in the second category if they are still unstable. They automatically inherit the feature gate for regular calls, it can now also be used for const-calls.
Also, several holes in recursive const stability checking are being closed.
There's still one potential hole that is hard to avoid, which is when MIR
building automatically inserts calls to a particular function in stable
functions -- which happens in the panic machinery. Those need to *not* be
`rustc_const_unstable` (or manually get a `rustc_const_stable_indirect`) to be
sure they follow recursive const stability. But that's a fairly rare and special
case so IMO it's fine.
The net effect of this is that a `#[unstable]` or unmarked function can be
constified simply by marking it as `const fn`, and it will then be
const-callable from stable `const fn` and subject to recursive const stability
requirements. If it is publicly reachable (which implies it cannot be unmarked),
it will be const-unstable under the same feature gate. Only if the function ever
becomes `#[stable]` does it need a `#[rustc_const_unstable]` or
`#[rustc_const_stable]` marker to decide if this should also imply
const-stability.
Adding `#[rustc_const_unstable]` is only needed for (a) functions that need to
use unstable const lang features (including intrinsics), or (b) `#[stable]`
functions that are not yet intended to be const-stable. Adding
`#[rustc_const_stable]` is only needed for functions that are actually meant to
be directly callable from stable const code. `#[rustc_const_stable_indirect]` is
used to mark intrinsics as const-callable and for `#[rustc_const_unstable]`
functions that are actually called from other, exposed-on-stable `const fn`. No
other attributes are required.
Things like `padding_needed_for` are current implemented being super careful to handle things like `Layout::size` potentially being `usize::MAX`.
But now that 95295 has happened, that's no longer a concern. It's possible to add two `Layout::size`s together without risking overflow now.
So take advantage of that to remove a bunch of checked math that's not actually needed. For example, the round-up-and-add-next-size in `extend` doesn't need any overflow checks at all, just the final check for compatibility with the alignment.
(And while I was doing that I made it all unstably const, because there's nothing in `Layout` that's fundamentally runtime-only.)
The previous definition (accidentally) disallowed the implementation of
stack-based allocators whose memory would become invalid once the
lifetime of the allocator type ended.
This also ensures the validity of the following blanket implementation:
```rust
impl<A: Allocator> Allocator for &'_ A {}
```
This allows LLVM to optimize comparisons to zero before & after the
multiplication into one, saving on code size and eliminating an (always
true) branch from most Vec allocations.
Fix docs for `alloc::realloc`
Fixes#108546.
Corrects the docs for `alloc::realloc` to bring the safety constraints into line with `Layout::from_size_align_unchecked`'s constraints.
Clarify that copied allocators must behave the same
Currently, the safety documentation for `Allocator` says that a cloned or moved allocator must behave the same as the original. However, it does not specify that a copied allocator must behave the same, and it's possible to construct an allocator that permits being moved or cloned, but sometimes produces a new allocator when copied.
<details>
<summary>Contrived example which results in a Miri error</summary>
```rust
#![feature(allocator_api, once_cell, strict_provenance)]
use std::{
alloc::{AllocError, Allocator, Global, Layout},
collections::HashMap,
hint,
marker::PhantomPinned,
num::NonZeroUsize,
pin::Pin,
ptr::{addr_of, NonNull},
sync::{LazyLock, Mutex},
};
mod source_allocator {
use super::*;
// `SourceAllocator` has 3 states:
// - invalid value: is_cloned == false, source != self.addr()
// - source value: is_cloned == false, source == self.addr()
// - cloned value: is_cloned == true
pub struct SourceAllocator {
is_cloned: bool,
source: usize,
_pin: PhantomPinned,
}
impl SourceAllocator {
// Returns a pinned source value (pointing to itself).
pub fn new_source() -> Pin<Box<Self>> {
let mut b = Box::new(Self {
is_cloned: false,
source: 0,
_pin: PhantomPinned,
});
b.source = b.addr();
Box::into_pin(b)
}
fn addr(&self) -> usize {
addr_of!(*self).addr()
}
// Invalid values point to source 0.
// Source values point to themselves.
// Cloned values point to their corresponding source.
fn source(&self) -> usize {
if self.is_cloned || self.addr() == self.source {
self.source
} else {
0
}
}
}
// Copying an invalid value produces an invalid value.
// Copying a source value produces an invalid value.
// Copying a cloned value produces a cloned value with the same source.
impl Copy for SourceAllocator {}
// Cloning an invalid value produces an invalid value.
// Cloning a source value produces a cloned value with that source.
// Cloning a cloned value produces a cloned value with the same source.
impl Clone for SourceAllocator {
fn clone(&self) -> Self {
if self.is_cloned || self.addr() != self.source {
*self
} else {
Self {
is_cloned: true,
source: self.source,
_pin: PhantomPinned,
}
}
}
}
static SOURCE_MAP: LazyLock<Mutex<HashMap<NonZeroUsize, usize>>> =
LazyLock::new(Default::default);
// SAFETY: Wraps `Global`'s methods with additional tracking.
// All invalid values share blocks with each other.
// Each source value shares blocks with all cloned values pointing to it.
// Cloning an allocator always produces a compatible allocator:
// - Cloning an invalid value produces another invalid value.
// - Cloning a source value produces a cloned value pointing to it.
// - Cloning a cloned value produces another cloned value with the same source.
// Moving an allocator always produces a compatible allocator:
// - Invalid values remain invalid when moved.
// - Source values cannot be moved, since they are always pinned to the heap.
// - Cloned values keep the same source when moved.
unsafe impl Allocator for SourceAllocator {
fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError> {
let mut map = SOURCE_MAP.lock().unwrap();
let block = Global.allocate(layout)?;
let block_addr = block.cast::<u8>().addr();
map.insert(block_addr, self.source());
Ok(block)
}
unsafe fn deallocate(&self, block: NonNull<u8>, layout: Layout) {
let mut map = SOURCE_MAP.lock().unwrap();
let block_addr = block.addr();
// SAFETY: `block` came from an allocator that shares blocks with this allocator.
if map.remove(&block_addr) != Some(self.source()) {
hint::unreachable_unchecked()
}
Global.deallocate(block, layout)
}
}
}
use source_allocator::SourceAllocator;
// SAFETY: `alloc1` and `alloc2` must share blocks.
unsafe fn test_same(alloc1: &SourceAllocator, alloc2: &SourceAllocator) {
let ptr = alloc1.allocate(Layout:🆕:<i32>()).unwrap();
alloc2.deallocate(ptr.cast(), Layout:🆕:<i32>());
}
fn main() {
let orig = &*SourceAllocator::new_source();
let orig_cloned1 = &orig.clone();
let orig_cloned2 = &orig.clone();
let copied = &{ *orig };
let copied_cloned1 = &copied.clone();
let copied_cloned2 = &copied.clone();
unsafe {
test_same(orig, orig_cloned1);
test_same(orig_cloned1, orig_cloned2);
test_same(copied, copied_cloned1);
test_same(copied_cloned1, copied_cloned2);
test_same(orig, copied); // error
}
}
```
</details>
This could result in issues in the future for algorithms that specialize on `Copy` types. Right now, nothing in the standard library that depends on `Allocator + Clone` is susceptible to this issue, but I still think it would make sense to specify that copying an allocator is always as valid as cloning it.
Constify remaining `Layout` methods
Makes the methods on `Layout` that aren't yet unstably const, under the same feature and issue, #67521. Most of them required no changes, only non-trivial change is probably constifying `ValidAlignment` which may affect #102072
