Commit Graph

383 Commits

Author SHA1 Message Date
Joseph Richey
5c294cedc0 Move from an "asm" flag to a "no-asm" feature flag (#386)
* Use a no-asm feature instead of an asm feature

This works better as core/alloc/std have trouble supporting default
featues in this crate.

Signed-off-by: Joe Richey <joerichey@google.com>

* Have no-asm disable arm assembly intrinsics

Signed-off-by: Joe Richey <joerichey@google.com>
2020-11-09 09:24:25 -06:00
Joseph Richey
53daa3c593 Use REP MOVSB/STOSB when the ERMSB feature is present (#392)
* Reorganize mem functions

This reduces the amount of platform-specific code

Signed-off-by: Joe Richey <joerichey@google.com>

* Use ERMSB implementations if the feature is set

Signed-off-by: Joe Richey <joerichey@google.com>

* Add non-aligned benchmarks

Signed-off-by: Joe Richey <joerichey@google.com>
2020-11-03 08:57:08 -06:00
Robert Jördens
a97fe5f1d3 math: add {fmin,fmax}{f,} for thumb*-none-eabi* (#389)
These are exposed in core::f32

close #354 
c.f. rust-lang/rust#62729
Patch from @whitequark (https://paste.debian.net/1168430/)
2020-10-26 11:37:02 -05:00
Joseph Richey
34e35d74b6 Use REP MOVSQ/STOSQ on x86_64 (#365)
* mem: Move mem* functions to separate directory

Signed-off-by: Joe Richey <joerichey@google.com>

* memcpy: Create separate memcpy.rs file

Signed-off-by: Joe Richey <joerichey@google.com>

* benches: Add benchmarks for mem* functions

This allows comparing the "normal" implementations to the
implementations provided by this crate.

Signed-off-by: Joe Richey <joerichey@google.com>

* mem: Add REP MOVSB/STOSB implementations

The assembly generated seems correct:
    https://rust.godbolt.org/z/GGnec8

Signed-off-by: Joe Richey <joerichey@google.com>

* mem: Add documentations for REP string insturctions

Signed-off-by: Joe Richey <joerichey@google.com>

* Use quad-word rep string instructions

Signed-off-by: Joe Richey <joerichey@google.com>

* Prevent panic when compiled in debug mode

Signed-off-by: Joe Richey <joerichey@google.com>

* Add tests for mem* functions

Signed-off-by: Joe Richey <joerichey@google.com>

* Add build/test with the "asm" feature

Signed-off-by: Joe Richey <joerichey@google.com>

* Add byte length to Bencher

Signed-off-by: Joe Richey <joerichey@google.com>
2020-10-24 10:58:04 -05:00
pca006132
f9bf5fee78 Use weak linkage for aeabi memory functions (#385) 2020-10-14 11:10:38 -05:00
Aaron Kutch
d2a2525035 Hide macros and functions
These macros and functions are only in the public interface for testing purposes or because of `#[macro_export]` pollution
2020-10-02 23:26:12 -05:00
Aaron Kutch
756a9dea00 Construct signed division functions differently 2020-10-02 22:55:31 -05:00
Aaron Kutch
1dfa14595e Remove unneeded code from asymmetric.rs
Rebenchmarking this showed that perf changed for the worse only on really low end CPUs
2020-10-02 16:35:55 -05:00
Amanieu d'Antras
9c3ea6ad97 Merge pull request #332 from AaronKutch/issue-265 2020-09-03 20:52:23 +01:00
Amanieu d'Antras
2df3b161f6 Merge pull request #372 from xiaoyuxlu/uefi_x86_target_probestack
Add uefi arch x86 probestack support
2020-09-03 20:27:14 +01:00
Aaron Kutch
92f0680743 Add __divmodti4 2020-08-29 18:02:57 -05:00
Aaron Kutch
ffb386369b Use unreachable_unchecked 2020-08-14 15:31:56 -05:00
Aaron Kutch
11ae780481 Change inlining to favor three underlying division functions 2020-08-14 15:31:56 -05:00
Aaron Kutch
d1c8673332 Use specialized-div-rem 1.0.0 for division algorithms 2020-08-14 15:31:36 -05:00
Xiaoyu Lu
0a957ae73a Add uefi arch x86 probestack support
1. In UEFI x86 arch, probestack need triple underscore.

2.  In UEFI, probestack function do things like _chkstk(in MSVC).
    MSVC x32's _chkstk and cygwin/mingw's _alloca adjust %esp themselves
    MSVC x64's __chkstk and cygwin/mingw's ___chkstk_ms do not adjust
    %rsp themselves.
    But current probestack doesn't adjust esp. And LLVM doesn't generate
    sub %eax, %esp after probestack. So we adjust esp in probestack like
    MSVC x32's _chkstk.
2020-08-14 16:26:37 +08:00
Max Audron
66f5688d60 add 32 bit shift instructions
* add 32 bit shift instructions to src/int/shift.rs
  __ashlsi3
  __ashrsi3
  __lshrsi3

* add int_impl! for 16 bit numbers and large_int! for i32 and u32

* add tests in testcrate/build.rs
2020-08-13 12:46:28 +02:00
Aaron Kutch
5386117b97 Remove unused code 2020-07-28 13:46:51 -05:00
Aaron Kutch
adbc0b24c5 Remove erroneous aapcs_on_arm and add maybe_use_optimized_c_shim 2020-07-28 13:46:51 -05:00
Aaron Kutch
3fda53a90d replace old soft division code with new functions 2020-07-28 13:46:50 -05:00
Aaron Kutch
981e803acd regularize the location and documentation of division functions 2020-07-28 13:46:50 -05:00
Aaron Kutch
7652f288d3 Improve __clzsi2 performance (#366) 2020-07-28 13:09:18 -05:00
Joseph Richey
d59b2875b5 lint: Allow improper_ctypes_definitions (#364)
https://github.com/rust-lang/rust/pull/72700 caused the existing
`allow(improper_ctypes)` guard to stop working, we now need
`allow(improper_ctypes_definitions)` instead.

We keep the old one to avoid any issues with older nightlies.

Signed-off-by: Joe Richey <joerichey@google.com>
2020-07-08 09:07:19 -05:00
jethrogb
96e3a08719 Manually patch ret instruction for LVI (#359)
Co-authored-by: Jethro Beekman <jethro@fortanix.com>
2020-06-01 10:55:42 -05:00
Alex Crichton
e4293adbe1 Add back in unsafe for bootstrapping
And add an `#[allow]` for now to appease stage0
2020-05-29 13:43:21 -07:00
Alex Crichton
63982a3b9e Expand wasm32 testing on CI (#360)
* Expand wasm32 testing on CI

Run the full `run.sh` test script to get full assertions, including that
nothing in the wasm compiler-builtins is panicking. Unfortunately it's
currently panicking, so this is good to weed out!

* Update libm
2020-05-29 14:38:29 -05:00
Tomasz Miąsko
ca827eb234 Use crate visibility for traits (#358)
Co-authored-by: Tomasz Miąsko <tomasz.miasko@gmail.com>
2020-05-26 15:12:10 -05:00
Alex Crichton
0e69cc8817 Switch to using llvm_asm! instead of asm! (#351)
* Switch to using `llvm_asm!` instead of `asm!`

* Run rustfmt

* Fix how LTO is specified on nightly
2020-04-29 15:30:10 -05:00
Tomasz Miąsko
ef99c37aae Place intrinsics in individual object files (#349)
Co-authored-by: Tomasz Miąsko <tomasz.miasko@gmail.com>
2020-04-10 17:00:50 -05:00
Tyler Mandry
6677b55fcb Set probestack visibility to hidden on ELF targets (#340) 2020-02-11 22:21:12 -08:00
Daniel Frampton
8647190d6c Don't modify the intrinsic abi for aarch64 windows (#337) 2020-01-14 15:28:10 -06:00
Adam Schwalm
328332003d Add separate rust_probestack definition for uefi (#335)
This is necessary because the Mach-O definition must have the
triple underscore, but the UEFI one must not.
2020-01-06 10:22:30 -06:00
Runji Wang
304028b2ed Fix compile error on x86_64-unknown-uefi target (#331)
* fix compile error on x86_64-unknown-uefi target

* Fix tests on nightly
2019-12-10 09:02:14 -08:00
Tyler Mandry
d511d48b97 Add control flow information to __rust_probestack (#328) 2019-12-06 08:51:42 -06:00
Alex Crichton
eb58e12abf Gate atomic intrinsics on presence of instructions (#324)
Don't emit the intrinsics for platforms which don't actually have the
instructions to do atomic loads/stores.

Closes #322
2019-11-11 12:38:50 -06:00
Alex Crichton
3f473cd3f3 Allow FFI-unsafe warnings for u128/i128 (#323)
* Allow FFI-unsafe warnings for u128/i128

Handle new warnings on nightly, and we shouldn't need to worry about
these with compiler-builtins since this is tied to a particular compiler.

* Clean up crate attributes

* No need for stability marker
* Rustdoc docs not used for this crate
* Remove old build-system related cruft from rustc itself.

* Run `cargo fmt`
2019-11-11 12:19:10 -06:00
Oliver Scherer
bff186bef3 Emit _fltused on uefi targets as a short-term workaround (#317)
* Emit `_fltused` on `uefi` targets as a short-term workaround

* Remove stray docker container
2019-11-07 13:04:11 -06:00
Ian Kronquist
af3d2e0a3c Implement bcmp (#315)
As of LLVM 9.0, certain calls to memcmp may be converted to bcmp, which I guess
could save a single subtraction on some architectures. [1]

bcmp is just like memcmp except instead of returning the difference between the
two differing bytes, it returns non-zero instead. As such, memcmp is a valid
implementation of bcmp.

If we care about size, bcmp should just call memcmp.
If we care about speed, we can change bcmp to look like this instead:
```rust
pub unsafe extern "C" fn bcmp(s1: *const u8, s2: *const u8, n: usize) -> i32 {
    let mut i = 0;
    while i < n {
        let a = *s1.offset(i as isize);
        let b = *s2.offset(i as isize);
        if a != b {
            return 1;
        }
        i += 1;
    }
    0
}
```

In this PR I do not address any changes which may or may not be needed for arm
aebi as I lack proper test hardware.

[1]: https://releases.llvm.org/9.0.0/docs/ReleaseNotes.html#noteworthy-optimizations
2019-09-30 11:02:47 -05:00
Benjamin Saunders
01a0524d8c Tidy up unordered elementwise atomic memory intrinsics 2019-08-23 10:20:09 -07:00
Benjamin Saunders
1cc3d32874 Implement LLVM's elementwise unordered atomic memory intrinsics
Allows uses of intrinsics of the form
llvm.(memcpy|memmove|memset).element.unordered.atomic.* to be linked.
2019-08-21 18:21:58 -07:00
Alex Crichton
0f6042793a Merge pull request #306 from da-x/probestack-frame-pointer
probestack: add frame pointers for easier traceback
2019-07-24 09:04:02 -05:00
Dan Aloni
84e8cc74c5 probestack: add frame pointers for easier traceback
This turns the following backtrace,

```
>> bt
 #0  0x0000555555576f73 in __rust_probestack () at /cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.14/src/probestack.rs:55
Backtrace stopped: Cannot access memory at address 0x7fffff7fedf0
```

To this:

```
>>> bt
 #0  0x0000555555574e47 in __rust_probestack ()
 #1  0x00005555555595ba in test::main ()
 #2  0x00005555555594f3 in std::rt::lang_start::{{closure}} ()
 #3  0x0000555555561ae3 in std::panicking::try::do_call ()
 #4  0x000055555556595a in __rust_maybe_catch_panic ()
 #5  0x000055555555af9b in std::rt::lang_start_internal ()
 #6  0x00005555555594d5 in std::rt::lang_start ()
 #7  0x000055555555977b in main ()
```
2019-07-24 10:18:30 +03:00
Alex Crichton
3c36beb31b Merge pull request #302 from bjorn3/no_128bit_lang_items
Replace {u,i}128_* lang items with __rust_{u,i}128_* unmangled functions
2019-07-19 09:44:00 -05:00
bjorn3
8bd3e4d5a1 Replace {u,i}128_* lang items with __rust_{u,i}128_* unmangled functions
The -Zlower-128bit-ops feature is completely broken, as libcore needs
those lang items to compile with this feature, but they are only
provided by compiler_builtins, which itself depends on libcore.
According to rust-lang/rust#58969 the feature never got finished.

This commit removes the associated lang items and replaces them with
normal unmangled functions, when there is no existing intrinsic. This
makes it easier for alternative codegen backends to implement 128bit
integer support.
2019-07-13 11:38:35 +02:00
Ralf Jung
29ea38c623 remove unused imports 2019-07-13 11:00:15 +02:00
Ralf Jung
f4bc012b26 avoid ptr::write which might panic in debug mode 2019-07-13 10:55:54 +02:00
varkor
3fe09ab411 Update libm for fmin/fmax/fminf/fmaxf 2019-06-05 23:41:18 +01:00
Alex Crichton
f567dbb36b Remove the need for #[cfg] in #[use_c_shim_if]
This commit tweaks the implementation of the synthetic
`#[use_c_shim_if]` attribute, renaming it to
`#[maybe_use_optimized_c_shim]` in the process. This no longer requires
specifying a `#[cfg]` clause indicating when the optimized intrinsic
should be used, but rather this is inferred and printed from the build
script.

The build script will now print out appropriate `#[cfg]` directives for
rustc to indicate what intrinsics it's compiling. This should remove the
need for us to keep the build script and the source in sync, but rather
the build script can simply take care of everything.
2019-05-15 12:59:31 -07:00
Alex Crichton
b2cfc3a4f1 Run rustfmt over everything 2019-05-14 14:40:38 -07:00
Alex Crichton
6ddcff1475 Fix __divsi3 and __udivsi3 on thumbv6m targets
This commit fixes a bug accidentally introduced in #285 where some
lingering references remained to `#[cfg(thumbv6m)]` but this, since the
historical revert, was renamed to `#[cfg(thumb_1)]`. This caused on the
thumbv6m platform for the intrinsics to be accidentally omitted because
the build script didn't actually compile them but the Rust code thought
the C code was in use.

After correcting the `#[cfg]` statements the CI configuration for the
`thumb*` family of targets was all updated. The support for xargo
testing was removed from `run.sh` since it had long since bitrotted, and
the script was updated to simply build the intrinsics example to attempt
to link for each of these targets. This in turn exposed the bug locally
and allowed to confirm a fix once the `#[cfg]` statements were
corrected.

cc rust-lang/rust#60782
2019-05-14 12:26:09 -07:00
Alex Crichton
a41077e11b Merge pull request #286 from alexcrichton/fix-dupe
Fix duplicate symbol __clzsi2
2019-05-02 17:17:52 -05:00