Enable Vec's calloc optimization for Option<NonZero>
Someone on discord noticed that `vec![None::<NonZeroU32>; N]` wasn't getting the optimization, so here's a PR 🙃
We can certainly do this in the standard library because we know for sure this is ok, but I think it's also a necessary consequence of documented guarantees like those in https://doc.rust-lang.org/std/option/#representation and https://doc.rust-lang.org/core/num/struct.NonZeroU32.html
It feels weird to do this without adding a test, but I wasn't sure where that would belong. Is it worth adding codegen tests for these?
Add `TrustedRandomAccess` specialization for `Vec::extend()`
This should do roughly the same as the `TrustedLen` specialization but result in less IR by using `__iterator_get_unchecked`
instead of `Iterator::for_each`
Conflicting specializations are manually prioritized by grouping them under yet another helper trait.
For certain sorts of systems, programming, it's deemed essential that
all allocation failures be explicitly handled where they occur. For
example, see Linus Torvald's opinion in [1]. Merely not calling global
panic handlers, or always `try_reserving` first (for vectors), is not
deemed good enough, because the mere presence of the global OOM handlers
is burdens static analysis.
One option for these projects to use rust would just be to skip `alloc`,
rolling their own allocation abstractions. But this would, in my
opinion be a real shame. `alloc` has a few `try_*` methods already, and
we could easily have more. Features like custom allocator support also
demonstrate and existing to support diverse use-cases with the same
abstractions.
A natural way to add such a feature flag would a Cargo feature, but
there are currently uncertainties around how std library crate's Cargo
features may or not be stable, so to avoid any risk of stabilizing by
mistake we are going with a more low-level "raw cfg" token, which
cannot be interacted with via Cargo alone.
Note also that since there is no notion of "default cfg tokens" outside
of Cargo features, we have to invert the condition from
`global_oom_handling` to to `not(no_global_oom_handling)`. This breaks
the monotonicity that would be important for a Cargo feature (i.e.
turning on more features should never break compatibility), but it
doesn't matter for raw cfg tokens which are not intended to be
"constraint solved" by Cargo or anything else.
To support this use-case we create a new feature, "global-oom-handling",
on by default, and put the global OOM handler infra and everything else
it that depends on it behind it. By default, nothing is changed, but
users concerned about global handling can make sure it is disabled, and
be confident that all OOM handling is local and explicit.
For this first iteration, non-flat collections are outright disabled.
`Vec` and `String` don't yet have `try_*` allocation methods, but are
kept anyways since they can be oom-safely created "from parts", and we
hope to add those `try_` methods in the future.
[1]: https://lore.kernel.org/lkml/CAHk-=wh_sNLoz84AUUzuqXEsYH35u=8HV3vK-jbRbJ_B-JjGrg@mail.gmail.com/
Fix double-drop in `Vec::from_iter(vec.into_iter())` specialization when items drop during panic
This fixes the double-drop but it leaves a behavioral difference compared to the default implementation intact: In the default implementation the source and the destination vec are separate objects, so they get dropped separately. Here they share an allocation and the latter only exists as a pointer into the former. So if dropping the former panics then this fix will leak more items than the default implementation would. Is this acceptable or should the specialization also mimic the default implementation's drops-during-panic behavior?
Fixes#83618
`@rustbot` label T-libs-impl
may not -> might not
may not -> might not
"may not" has two possible meanings:
1. A command: "You may not stay up past your bedtime."
2. A fact that's only sometimes true: "Some cities may not have bike lanes."
In some cases, the meaning is ambiguous: "Some cars may not have snow
tires." (do the cars *happen* to not have snow tires, or is it
physically impossible for them to have snow tires?)
This changes places where the standard library uses the "description of
fact" meaning to say "might not" instead.
This is just `std::vec` for now - if you think this is a good idea I can
convert the rest of the standard library.
"may not" has two possible meanings:
1. A command: "You may not stay up past your bedtime."
2. A fact that's only sometimes true: "Some cities may not have bike lanes."
In some cases, the meaning is ambiguous: "Some cars may not have snow
tires." (do the cars *happen* to not have snow tires, or is it
physically impossible for them to have snow tires?)
This changes places where the standard library uses the "description of
fact" meaning to say "might not" instead.
This is just `std::vec` for now - if you think this is a good idea I can
convert the rest of the standard library.
This allows the optimizer to turn certain iterator pipelines such as
```rust
let vec = vec![0usize; 100];
vec.into_iter().map(|e| e as isize).collect::<Vec<_>>()
```
into a noop.
The optimization only applies when iterator sources are `T: Copy`
since `impl TrustedRandomAccess for IntoIter<T>`.
No such requirement applies to the output type (`Iterator::Item`).
Vec::dedup_by optimization
Now `Vec::dedup_by` drops items in-place as it goes through them.
From my benchmarks, it is around 10% faster when T is small, with no major regression when otherwise.
I used `ptr::copy` instead of conditional `ptr::copy_nonoverlapping`, because the latter had some weird performance issues on my ryzen laptop (it was 50% slower on it than on intel/sandybridge laptop)
It would be good if someone was able to reproduce these results.
Previously vec's len was updated only after full copy, making the method
leak if T::clone panic!s.
This commit makes `Vec::extend_from_within` (or, more accurately, it's
`T: Clone` specialization) update vec's len on every iteration, fixing
the issue.
`T: Copy` specialization was not affected by the issue b/c it doesn't
call user specified code (as, e.g. `T::clone`), and instead calls
`ptr::copy_nonoverlapping`.
Revert `Vec::spare_capacity_mut` impl to prevent pointers invalidation
The implementation was changed in #79015.
Later it was [pointed out](https://github.com/rust-lang/rust/issues/81944#issuecomment-782849785) that the implementation invalidates pointers to the buffer (initialized elements) by creating a unique reference to the buffer. This PR reverts the implementation.
r? ```@RalfJung```
Turn may_have_side_effect into an associated constant
The `may_have_side_effect` is an implementation detail of `TrustedRandomAccess`
trait. It describes if obtaining an iterator element may have side effects. It
is currently implemented as an associated function.
Turn `may_have_side_effect` into an associated constant. This makes the
value immediately available to the optimizer.
Almost all safety comments are of the form `// SAFETY:`,
so normalize the rest and fix a few of them that should
have been a `/// # Safety` section instead.
Furthermore, make `tidy` only allow the uppercase form. While
currently `tidy` only checks `core`, it is a good idea to prevent
`core` from drifting to non-uppercase comments, so that later
we can start checking `alloc` etc. too.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Improve design of `assert_len`
It was discussed in the [tracking issue](https://github.com/rust-lang/rust/issues/76393#issuecomment-761765448) that `assert_len`'s name and usage are confusing. This PR improves them based on a suggestion by ``@scottmcm`` in that issue.
I also improved the documentation to make it clearer when you might want to use this method.
Old example:
```rust
let range = range.assert_len(slice.len());
```
New example:
```rust
let range = range.ensure_subset_of(..slice.len());
```
Fixes#81157
The `may_have_side_effect` is an implementation detail of `TrustedRandomAccess`
trait. It describes if obtaining an iterator element may have side effects. It
is currently implemented as an associated function.
Turn `may_have_side_effect` into an associated constant. This makes the
value immediately available to the optimizer.
Fix doc test for Vec::retain(), now passes clippy::eval_order_dependence
Doc test for Vec::retain() works correctly but is flagged by clippy::eval_order_dependence. Fix avoids the issue by using an iterator instead of an index.
Optimize Vec::retain
Use `copy_non_overlapping` instead of `swap` to reduce memory writes, like what we've done in #44355 and `String::retain`.
#48065 already tried to do this optimization but it is reverted in #67300 due to bad codegen of `DrainFilter::drop`.
This PR re-implement the drop-then-move approach. I did a [benchmark](https://gist.github.com/oxalica/3360eec9376f22533fcecff02798b698) on small-no-drop, small-need-drop, large-no-drop elements with different predicate functions. It turns out that the new implementation is >20% faster in average for almost all cases. Only 2/24 cases are slower by 3% and 5%. See the link above for more detail.
I think regression in may-panic cases is due to drop-guard preventing some optimization. If it's permitted to leak elements when predicate function of element's `drop` panic, the new implementation should be almost always faster than current one.
I'm not sure if we should leak on panic, since there is indeed an issue (#52267) complains about it before.
