Make Tree Borrows Provenance GC compact the tree
Follow-up on #3833 and #3835. In these PRs, the TB GC was fixed to no longer cause a stack overflow. One test that motivated it was the test `fill::horizontal_line` in [`tiny-skia`](https://github.com/RazrFalcon/tiny-skia). But not causing stack overflows was not a large improvents, since it did not fix the fundamental issue: The tree was too large. The test now ran, but it required gigabytes of memory and hours of time (only for it to be OOM-killed 🤬), whereas it finishes within 24 seconds in Stacked Borrows. With this merged, it finishes in about 40 seconds under TB.
The problem in that test was that it used [`slice::chunked`](https://doc.rust-lang.org/std/primitive.slice.html#method.chunks) to iterate a slice in chunks. That iterator is written to reborrow at each call to `next`, which creates a linear tree with a bunch of intermediary nodes, which also fragments the `RangeMap` for that allocation.
The solution is to now compact the tree, so that these interior nodes are removed. Care is taken to not remove nodes that are protected, or that otherwise restrict their children.
I am currently only 99% sure that this is sound, and I do also think that this could compact even more. So `@Vanille-N` please also have a look at whether I got the compacting logic right.
For a more visual comparison, [here is a gist](https://gist.github.com/JoJoDeveloping/ae4a7f7c29335a4c233ef42d2f267b01) of what the tree looks like at one point during that test, with and without compacting.
This new GC requires a different iteration order during accesses (since the current one can make the error messages non-deterministic), so it is rebased on top of #3843 and requires that PR to be merged first.
LLVM uses the word "code" to refer to a particular kind of coverage mapping.
This unrelated usage of the word is confusing, and makes it harder to introduce
types whose names correspond to the LLVM classification of coverage kinds.
Get rid of `predicates_defined_on`
This is the uncontroversial part of #129532. This simply inlines the `predicates_defined_on` into into `predicates_of`. Nothing should change here logically.
This particular target does not expand into much code, so it is a good
first candidate to see if we could keep this in the CI.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
This change will also remove the current warnings in the build due to
`rustfmt` not being available (for `bindgen` output):
error: 'rustfmt' is not installed for the custom toolchain 'local'.
note: this is a custom toolchain, which cannot use `rustup component add`
help: if you built this toolchain from source, and used `rustup toolchain link`, then you may be able to build the component with `x.py`
Failed to run rustfmt: Internal rustfmt error (non-fatal, continuing)
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Apparently tools like `rustfmt` require it in order to find the right
`librustc_driver.so` without extra tweaks.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Stop storing a special inner body for the coroutine by-move body for async closures
...and instead, just synthesize an item which is treated mostly normally by the MIR pipeline.
This PR does a few things:
* We synthesize a new `DefId` for the by-move body of a closure, which has its `mir_built` fed with the output of the `ByMoveBody` MIR transformation, and some other relevant queries.
* This has the `DefKind::ByMoveBody`, which we use to distinguish it from "real" bodies (that come from HIR) which need to be borrowck'd. Introduce `TyCtxt::is_synthetic_mir` to skip over `mir_borrowck` which is called by `mir_promoted`; borrowck isn't really possible to make work ATM since it heavily relies being called on a body generated from HIR, and is redundant by the construction of the by-move-body.
* Remove the special `PassManager` hacks for handling the inner `by_move_body` stored within the coroutine's mir body. Instead, this body is fed like a regular MIR body, so it's goes through all of the `tcx.*_mir` stages normally (build -> promoted -> ...etc... -> optimized) ✨.
* Remove the `InstanceKind::ByMoveBody` shim, since now we have a "regular" def id, we can just use `InstanceKind::Item`. This also allows us to remove the corresponding hacks from codegen, such as in `fn_sig_for_fn_abi` ✨.
Notable remarks:
* ~~I know it's kind of weird to be using `DefKind::Closure` here, since it's not a distinct closure but just a new MIR body. I don't believe it really matters, but I could also use a different `DefKind`... maybe one that we could use for synthetic MIR bodies in general?~~ edit: We're doing this now.
Support blocking for epoll
This PR enabled epoll to have blocking operation.
The changes introduced by this PR are:
- Refactored part of the logic in ``epoll_wait`` to ``blocking_epoll_callback``
- Added a new field ``thread_ids`` in ``Epoll`` for blocked thread ids
- Added a new ``BlockReason::Epoll``
Follow-up on #3833 and #3835. In these PRs, the TB GC was fixed to no
longer cause a stack overflow. One test that motivated it was the test
`fill::horizontal_line` in `tiny_skia`. But not causing stack overflows
was not a large improvents, since it did not fix the fundamental issue:
The tree was too large. The test now ran, but it required gigabytes of
memory and hours of time, whereas it finishes within seconds in Stacked
Borrows.
The problem in that test was that it used [`slice::chunked`](https://doc.rust-lang.org/std/primitive.slice.html#method.chunks) to iterate
a slice in chunks. That iterator is written to reborrow at each call to
`next`, which creates a linear tree with a bunch of intermediary nodes,
which also fragments the `RangeMap` for that allocation.
The solution is to now compact the tree, so that these interior nodes
are removed. Care is taken to not remove nodes that are protected, or
that otherwise restrict their children.
Make TB tree traversal bottom-up
In preparation for #3837, the tree traversal needs to be made bottom-up, because the current top-down tree traversal, coupled with that PR's changes to the garbage collector, can introduce non-deterministic error messages if the GC removes a parent tag of the accessed tag that would have triggered the error first.
This is a breaking change for the diagnostics emitted by TB. The implemented semantics stay the same.
The main `arbitrary_self_types` feature gate will shortly be reused for
a new version of arbitrary self types which we are amending per [this
RFC](https://github.com/rust-lang/rfcs/blob/master/text/3519-arbitrary-self-types-v2.md).
The main amendments are:
* _do_ support `self` types which can't safely implement `Deref`
* do _not_ support generic `self` types
* do _not_ support raw pointers as `self` types.
This PR relates to the last of those bullet points: this strips pointer
support from the current `arbitrary_self_types` feature.
We expect this to cause some amount of breakage for crates using this
unstable feature to allow raw pointer self types. If that's the case, we
want to know about it, and we want crate authors to know of the upcoming
changes.
For now, this can be resolved by adding the new
`arbitrary_self_types_pointers` feature to such crates. If we determine
that use of raw pointers as self types is common, then we may maintain
that as an unstable feature even if we come to stabilize the rest of the
`arbitrary_self_types` support in future. If we don't hear that this PR
is causing breakage, then perhaps we don't need it at all, even behind
an unstable feature gate.
[Tracking issue](https://github.com/rust-lang/rust/issues/44874)
This is [step 4 of the plan outlined here](https://github.com/rust-lang/rust/issues/44874#issuecomment-2122179688)
A partial stabilization that only affects:
- AllocType<T>::new_uninit
- AllocType<T>::assume_init
- AllocType<[T]>::new_uninit_slice
- AllocType<[T]>::assume_init
where "AllocType" is Box, Rc, or Arc
