Commit Graph

102 Commits

Author SHA1 Message Date
Jubilee
33eb552ceb Rollup merge of #143410 - scottmcm:redo-transmute-again, r=RalfJung,workingjubilee
Block SIMD in transmute_immediate; delete `OperandValueKind`

Vectors have been causing me problems for years in this code, for example https://github.com/rust-lang/rust/pull/110021#discussion_r1160975086 and https://github.com/rust-lang/rust/pull/143194

See conversation in <https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Is.20transmuting.20a.20.60T.60.20to.20.60Tx1.60.20.28one-element.20SIMD.20vector.29.20UB.3F/near/526262799>.

By blocking SIMD in `transmute_immediate` it can be simplified to just take the `Scalar`s involved -- the backend types can be gotten from those `Scalar`s, rather than needing to be passed.  And there's an assert added to ICE it if it does get hit.

Accordingly, this changes `rvalue_creates_operand` to not send SIMD transmutes through the operand path, but to always go through memory instead, like they did back before rust-lang/rust#108442.

And thanks to those changes, I could also remove the `OperandValueKind` type that I added back then which `@RalfJung` rightly considers pretty sketchy.

cc `@folkertdev` `@workingjubilee` from the zulip conversation too
2025-07-04 23:26:24 -07:00
Scott McMurray
4e615272bf Address PR feedback 2025-07-04 12:29:27 -07:00
Scott McMurray
e0c54c3a9b Rename transmute_immediatetransmute_scalar 2025-07-04 10:35:51 -07:00
Scott McMurray
caeacba78a Allow all MIR Aggregates to take the operand path (if layout permits) 2025-07-03 22:53:19 -07:00
Scott McMurray
5292554337 Block SIMD in transmute_immediate; delete OperandValueKind
See conversation in <https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Is.20transmuting.20a.20.60T.60.20to.20.60Tx1.60.20.28one-element.20SIMD.20vector.29.20UB.3F/near/526262799>.
2025-07-03 22:23:15 -07:00
Florian Sextl
1c25bfba9d move discr=varid check to layout_sanity_check 2025-06-28 14:47:27 +02:00
Scott McMurray
e4f196a7b4 CodeGen: rework Aggregate implemention for rvalue_creates_operand cases
Another refactor pulled out from 138759

The previous implementation I'd written here based on `index_by_increasing_offset` is complicated to follow and difficult to extend to non-structs.

This changes the implementation, without actually changing any codegen (thus no test changes either), to be more like the existing `extract_field` (<2b0274c71d/compiler/rustc_codegen_ssa/src/mir/operand.rs (L345-L425)>) in that it allows setting a particular field directly.

Notably I've found this one much easier to get right, in particular because having the `OperandRef<Result<V, Scalar>>` gives a really useful thing to include in ICE messages if something did happen to go wrong.
2025-06-17 18:59:22 -07:00
Matthias Krüger
29954af23a Rollup merge of #142044 - workingjubilee:document-operandvalue-pair, r=scottmcm
compiler: Document the offset invariant of `OperandValue::Pair`

A subtle invariant of `OperandValue::Pair` that came up during review and was found to be undocumented.

Visible in code like this:
4b27a04cc8/compiler/rustc_codegen_ssa/src/mir/operand.rs (L376-L392)
2025-06-06 00:58:45 +02:00
Jubilee Young
64df9e3c8a compiler: Document the offset invariant of OperandValue::Pair 2025-06-04 16:42:56 -07:00
Scott McMurray
ee9901e65c Change tag_field to FieldIdx in Variants::Multiple
It was already available as a generic parameter anyway, and it's not like we'll ever put a tag in the 5-billionth field.
2025-06-03 23:42:21 -07:00
Scott McMurray
502f7f9c24 Address PR feedback 2025-04-07 18:12:06 -07:00
Scott McMurray
1f06a6a252 Tell LLVM about impossible niche tags 2025-04-05 19:54:47 -07:00
Scott McMurray
2b15dd1ddd Add more comments to discriminant calculations. 2025-03-13 00:39:18 -07:00
Scott McMurray
143f39362a Don't alloca just to look at a discriminant
Today we're making LLVM do a bunch of extra work for every enum you match on, even trivial stuff like `Option<bool>`.  Let's not.
2025-03-12 00:56:43 -07:00
Matthias Krüger
70b9968d1e Rollup merge of #137894 - compiler-errors:no-scalar-pair-opt, r=oli-obk
Revert "store ScalarPair via memset when one side is undef and the other side can be memset"

cc #137892
reverts #135335

r? oli-obk
2025-03-03 20:47:12 +01:00
Michael Goulet
a59a8f9e75 Revert "Auto merge of #135335 - oli-obk:push-zxwssomxxtnq, r=saethlin"
This reverts commit a7a6c64a65, reversing
changes made to ebbe63891f.
2025-03-02 18:52:48 +00:00
Ralf Jung
aac65f562b rename BackendRepr::Vector → SimdVector 2025-02-28 17:17:45 +01:00
Michael Goulet
3d5438accd Fix binding mode problems 2025-02-22 00:13:19 +00:00
Scott McMurray
da77b39f05 Refactor OperandRef::extract_field to prep for 838 2025-02-20 22:26:24 -08:00
Zachary S
7ba3d7b54e Remove BackendRepr::Uninhabited, replaced with an uninhabited: bool field in LayoutData.
Also update comments that refered to BackendRepr::Uninhabited.
2025-02-20 13:27:32 -06:00
Scott McMurray
6f9cfd694d Rework OperandRef::extract_field to stop calling to_immediate_scalar on things which are already immediates
That means it stops trying to truncate things that are already `i1`s.
2025-02-19 12:03:40 -08:00
Oli Scherer
8876cf7181 Also generate undef scalars and scalar pairs 2025-01-21 08:22:15 +00:00
Jubilee Young
7086dd83cc compiler: rustc_abi::Abi => BackendRepr
The initial naming of "Abi" was an awful mistake, conveying wrong ideas
about how psABIs worked and even more about what the enum meant.
It was only meant to represent the way the value would be described to
a codegen backend as it was lowered to that intermediate representation.
It was never meant to mean anything about the actual psABI handling!
The conflation is because LLVM typically will associate a certain form
with a certain ABI, but even that does not hold when the special cases
that actually exist arise, plus the IR annotations that modify the ABI.

Reframe `rustc_abi::Abi` as the `BackendRepr` of the type, and rename
`BackendRepr::Aggregate` as `BackendRepr::Memory`. Unfortunately, due to
the persistent misunderstandings, this too is now incorrect:
- Scattered ABI-relevant code is entangled with BackendRepr
- We do not always pre-compute a correct BackendRepr that reflects how
  we "actually" want this value to be handled, so we leave the backend
  interface to also inject various special-cases here
- In some cases `BackendRepr::Memory` is a "real" aggregate, but in
  others it is in fact using memory, and in some cases it is a scalar!

Our rustc-to-backend lowering code handles this sort of thing right now.
That will eventually be addressed by lifting duplicated lowering code
to either rustc_codegen_ssa or rustc_target as appropriate.
2024-10-29 14:56:00 -07:00
Jubilee Young
839cf1c1a4 compiler: Factor rustc_target::abi out of cg_ssa 2024-10-08 18:24:56 -07:00
Urgau
018ba0528f Use wide pointers consistenly across the compiler 2024-10-04 14:06:48 +02:00
Michael Goulet
c682aa162b Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
Matthias Krüger
21313d7947 Rollup merge of #130457 - nnethercote:cleanup-codegen-traits, r=bjorn3
Cleanup codegen traits

The traits governing codegen are quite complicated and hard to follow. This PR cleans them up a bit.

r? `@bjorn3`
2024-09-18 17:49:43 +02:00
Nicholas Nethercote
cd3da000c0 Clean up formatting.
Reflow overly long comments, plus some minor whitespace improvements.
2024-09-17 16:24:35 +10:00
Nicholas Nethercote
bdacdfe95f Minimize visibilities.
This makes it much clearer which things are used outside the crate.
2024-09-17 16:24:33 +10:00
Nicholas Nethercote
a8d22eb39e Rename supertraits of CodegenMethods.
Supertraits of `BuilderMethods` are all called `XyzBuilderMethods`.
Supertraits of `CodegenMethods` are all called `XyzMethods`. This commit
changes the latter to `XyzCodegenMethods`, for consistency.
2024-09-17 10:24:43 +10:00
Nicholas Nethercote
540fcc617a Move some supertraits outward.
Specifically, put them where they are genuinely required, i.e. the
outermost place they can be.
2024-09-17 10:24:43 +10:00
Guillaume Gomez
7c6dca9050 Rollup merge of #128978 - compiler-errors:assert-matches, r=jieyouxu
Use `assert_matches` around the compiler more

It's a useful assertion, especially since it actually prints out the LHS.
2024-08-12 17:09:19 +02:00
Michael Goulet
c361c924a0 Use assert_matches around the compiler 2024-08-11 12:25:39 -04:00
James Barford-Evans
27ca35aa1b const vector passed to codegen 2024-08-08 11:15:03 +01:00
Nicholas Nethercote
84ac80f192 Reformat use declarations.
The previous commit updated `rustfmt.toml` appropriately. This commit is
the outcome of running `x fmt --all` with the new formatting options.
2024-07-29 08:26:52 +10:00
Jubilee Young
ce7b069fd8 compiler: Never debug_assert in codegen
The gains in performance are not worth the costs in correctness.
This is partly because the gains are zero and the costs are unknown.
2024-07-20 00:16:44 -07:00
Scott McMurray
459ce3f6bb Add an intrinsic for ptr::metadata 2024-05-28 09:28:51 -07:00
Nicholas Nethercote
8ae0694fcb Remove #[macro_use] extern crate tracing from rustc_codegen_ssa. 2024-05-23 18:02:40 +10:00
Scott McMurray
dcab06d7d2 Unify Rvalue::Aggregate paths in cg_ssa 2024-05-11 21:22:51 -07:00
Scott McMurray
9be16ebe89 Refactoring after the PlaceValue addition
I added `PlaceValue` in 123775, but kept that one line-by-line simple because it touched so many places.

This goes through to add more helpers & docs, and change some `PlaceRef` to `PlaceValue` where the type didn't need to be included.

No behaviour changes.
2024-05-10 20:09:37 -07:00
bors
6e1d94708a Auto merge of #123886 - scottmcm:more-rvalue-operands, r=matthewjasper
Avoid `alloca`s in codegen for simple `mir::Aggregate` statements

The core idea here is to remove the abstraction penalty of simple newtypes in codegen.

Even something simple like constructing a
```rust
#[repr(transparent)] struct Foo(u32);
```
forces an `alloca` to be generated in nightly right now.

Certainly LLVM can optimize that away, but it would be nice if it didn't have to.

Quick example:
```rust
#[repr(transparent)]
pub struct Transparent32(u32);

#[no_mangle]
pub fn make_transparent(x: u32) -> Transparent32 {
    let a = Transparent32(x);
    a
}
```
on nightly we produce <https://rust.godbolt.org/z/zcvoM79ae>
```llvm
define noundef i32 `@make_transparent(i32` noundef %x) unnamed_addr #0 {
  %a = alloca i32, align 4
  store i32 %x, ptr %a, align 4
  %0 = load i32, ptr %a, align 4, !noundef !3
  ret i32 %0
}
```
but after this PR we produce
```llvm
define noundef i32 `@make_transparent(i32` noundef %x) unnamed_addr #0 {
start:
  ret i32 %x
}
```
(even before the optimizer runs).
2024-05-10 20:17:22 +00:00
Michael Goulet
d50c2b0a52 Make builtin_deref just return a Ty 2024-05-09 22:55:00 -04:00
Scott McMurray
c38f75c21f Make SSA aggregates without needing an alloca 2024-05-08 20:38:04 -07:00
Nicholas Nethercote
99e036bd21 Remove extern crate rustc_middle from numerous crates. 2024-04-29 14:50:45 +10:00
bors
29a56a3b1c Auto merge of #122053 - erikdesjardins:alloca, r=nikic
Stop using LLVM struct types for alloca

The alloca type has no semantic meaning, only the size (and alignment, but we specify it explicitly) matter. Using `[N x i8]` is a more direct way to specify that we want `N` bytes, and avoids relying on LLVM's struct layout. It is likely that a future LLVM version will change to an untyped alloca representation.

Split out from #121577.

r? `@ghost`
2024-04-24 03:00:44 +00:00
Erik Desjardins
f4426c189f use [N x i8] for alloca types 2024-04-11 21:42:35 -04:00
Scott McMurray
3596098823 Put PlaceValue into OperandValue::Ref, rather than 3 tuple fields 2024-04-11 00:10:10 -07:00
Scott McMurray
89502e584b Make PlaceRef hold a PlaceValue for the non-layout fields (like OperandRef does) 2024-04-11 00:10:10 -07:00
Scott McMurray
b5376ba601 Remove my scalar_copy_backend_type optimization attempt
I added this back in 111999, but I no longer think it's a good idea
- It had to get scaled back to only power-of-two things to not break a bunch of targets
- LLVM seems to be getting better at memcpy removal anyway
- Introducing vector instructions has seemed to sometimes (115515) make autovectorization worse

So this removes it from the codegen crates entirely, and instead just tries to use <https://doc.rust-lang.org/nightly/nightly-rustc/rustc_codegen_ssa/traits/builder/trait.BuilderMethods.html#method.typed_place_copy> instead of direct `memcpy` so things will still use load/store for immediates.
2024-04-09 08:51:32 -07:00
Ralf Jung
f391c0793b only set noalias on Box with the global allocator 2024-03-05 15:03:33 +01:00