Commit Graph

222 Commits

Author SHA1 Message Date
Nicholas Nethercote
6341935a13 Remove extern crate tracing from numerous crates. 2024-04-30 16:47:49 +10:00
Nicholas Nethercote
f3e05d1609 Remove extern crate rustc_data_structures from rustc_query_system. 2024-04-29 15:40:29 +10:00
Nicholas Nethercote
4814fd0a4b Remove extern crate rustc_macros from numerous crates. 2024-04-29 10:21:54 +10:00
Michael Woerister
6146a51f17 Add more context to the forbidden dep-graph read ICE error message. 2024-04-22 14:54:28 +02:00
Michael Woerister
c373ec07c4 Improve ICE message for forbidden dep-graph reads. 2024-04-22 12:11:07 +02:00
Matthias Krüger
f9ca213510 remove some things that do not need to be 2024-04-11 21:09:52 +02:00
bors
4563f70c3b Auto merge of #122070 - Zoxc:dep-edges-from-previous, r=cjgillot
Encode dep graph edges directly from the previous graph when promoting

This encodes dep graph edges directly from the previous graph when promoting nodes from a previous session, avoiding allocations / copies.

~~Based on https://github.com/rust-lang/rust/pull/122064 and https://github.com/rust-lang/rust/pull/116375.~~

<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.4177s</td><td align="right">0.4072s</td><td align="right">💚  -2.52%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.1430s</td><td align="right">0.1420s</td><td align="right"> -0.69%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.3106s</td><td align="right">0.3038s</td><td align="right">💚  -2.19%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.5823s</td><td align="right">0.5688s</td><td align="right">💚  -2.33%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:unchanged</td><td align="right">1.3992s</td><td align="right">1.3692s</td><td align="right">💚  -2.14%</td></tr><tr><td>Total</td><td align="right">2.8528s</td><td align="right">2.7910s</td><td align="right">💚  -2.17%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9803s</td><td align="right">💚  -1.97%</td></tr></table>
2024-04-05 11:11:17 +00:00
Vadim Petrochenkov
b40ea03f8a rustc_index: Add a ZERO constant to index types
It is commonly used.
2024-04-03 19:06:22 +03:00
John Kåre Alsaker
aa9c9a36c0 Add some comments and do some renames 2024-03-23 20:23:25 +01:00
John Kåre Alsaker
6119763e19 Encode dep graph edges directly from the previous graph when promoting 2024-03-23 20:03:55 +01:00
Jubilee
f54350a9ba Rollup merge of #122245 - saethlin:check-dep-graph-size, r=petrochenkov
Detect truncated DepGraph files

I suspect that the following issues are caused by truncated incr comp files:

* https://github.com/rust-lang/rust/issues/120582
* https://github.com/rust-lang/rust/issues/121499
* https://github.com/rust-lang/rust/issues/122210

We fail with an allocation failure or capacity overflow in this case because we assume that the ending bytes of an DepGraph file are the lengths of arrays. If the file has somehow been truncated then the ending bytes are probably some of our varint encoding, which tries to eliminate zero bytes, so interpreting a random 8 bytes as an array length has a very high chance of producing a byte capacity over `isize::MAX`.

Now theoretically since https://github.com/rust-lang/rust/pull/119510 merged I have fixed the out-of-disk issues and yet in https://github.com/rust-lang/rust/issues/120894#issuecomment-1945126700 I still see some decoding failures that look like out-of-disk ICEs, for example https://crater-reports.s3.amazonaws.com/beta-1.77-1/beta-2024-02-10/gh/scottfones.aoc_2022/log.txt

So this PR should ensure that we get an ICE that clearly identifies if the file in question is truncated.
2024-03-12 09:04:00 -07:00
Ben Kimock
11f8866ca8 Detect truncated incr comp files 2024-03-09 15:43:50 -05:00
John Kåre Alsaker
9707e103ea Avoid the double lock around EncoderState 2024-03-06 04:40:39 +01:00
John Kåre Alsaker
a2499bdfbe Remove profiling from intern_node 2024-03-06 04:31:56 +01:00
John Kåre Alsaker
29cc76f0fc Add a profiler reference to GraphEncoder 2024-03-06 04:17:17 +01:00
Matthias Krüger
47bf8a6c28 Rollup merge of #121401 - eltociear:patch-25, r=nnethercote
Fix typo in serialized.rs

accomodate -> accommodate
2024-02-22 18:09:54 +01:00
Nicholas Nethercote
46f4983356 Adjust the has_errors* methods.
Currently `has_errors` excludes lint errors. This commit changes it to
include lint errors.

The motivation for this is that for most places it doesn't matter
whether lint errors are included or not. But there are multiple places
where they must be includes, and only one place where they must not be
included. So it makes sense for `has_errors` to do the thing that fits
the most situations, and the new `has_errors_excluding_lint_errors`
method in the one exceptional place.

The same change is made for `err_count`. Annoyingly, this requires the
introduction of `err_count_excluding_lint_errs` for one place, to
preserve existing error printing behaviour. But I still think the change
is worthwhile overall.
2024-02-22 08:03:47 +11:00
Ikko Eltociear Ashimine
00234f0f68 Fix typo in serialized.rs
accomodate -> accommodate
2024-02-22 00:33:23 +09:00
Michael Goulet
6b2a8249c1 Remove dead args from functions 2024-02-02 22:47:26 +00:00
Nicholas Nethercote
1f9fa2305a Tweak error counting.
We have several methods indicating the presence of errors, lint errors,
and delayed bugs. I find it frustrating that it's very unclear which one
you should use in any particular spot. This commit attempts to instill a
basic principle of "use the least general one possible", because that
reflects reality in practice -- `has_errors` is the least general one
and has by far the most uses (esp. via `abort_if_errors`).

Specifics:
- Add some comments giving some usage guidelines.
- Prefer `has_errors` to comparing `err_count` to zero.
- Remove `has_errors_or_span_delayed_bugs` because it's a weird one: in
  the cases where we need to count delayed bugs, we should really be
  counting lint errors as well.
- Rename `is_compilation_going_to_fail` as
  `has_errors_or_lint_errors_or_span_delayed_bugs`, for consistency with
  `has_errors` and `has_errors_or_lint_errors`.
- Change a few other `has_errors_or_lint_errors` calls to `has_errors`,
  as per the "least general" principle.

This didn't turn out to be as neat as I hoped when I started, but I
think it's still an improvement.
2024-01-22 10:14:01 +11:00
John Kåre Alsaker
862011e1ca Avoid code generation for ThinVec<Diagnostic>'s destructor in the query system 2024-01-20 13:43:05 +01:00
Camille GILLOT
c6f83b8ff6 Inline 2 functions that appear in dep-graph profiles. 2024-01-14 12:57:13 +00:00
klensy
56173611d6 don't reexport atomic::ordering via rustc_data_structures, use std import 2024-01-06 15:01:10 +03:00
Nicholas Nethercote
8a9db25459 Remove more Session methods that duplicate DiagCtxt methods. 2023-12-24 08:17:47 +11:00
Nicholas Nethercote
f6aa418c9f Rename many DiagCtxt and EarlyDiagCtxt locals. 2023-12-18 16:06:22 +11:00
Nicholas Nethercote
09af8a667c Rename Session::span_diagnostic as Session::dcx. 2023-12-18 16:06:21 +11:00
Jubilee
9e872b7cd8 Rollup merge of #118933 - nnethercote:cleanup-errors-even-more, r=compiler-errors
Cleanup errors handlers even more

A sequel to #118587.

r? `@compiler-errors`
2023-12-14 16:07:48 -08:00
Nicholas Nethercote
9a78412511 Split Handler::emit_diagnostic in two.
Currently, `emit_diagnostic` takes `&mut self`.

This commit changes it so `emit_diagnostic` takes `self` and the new
`emit_diagnostic_without_consuming` function takes `&mut self`.

I find the distinction useful. The former case is much more common, and
avoids a bunch of `mut` and `&mut` occurrences. We can also restrict the
latter with `pub(crate)` which is nice.
2023-12-15 10:13:12 +11:00
Matthias Krüger
d707461a1a clippy::complexity fixes
filter_map_identity
 needless_bool
 search_is_some
 unit_arg
 map_identity
 needless_question_mark
 derivable_impls
2023-12-12 19:28:13 +01:00
surechen
40ae34194c remove redundant imports
detects redundant imports that can be eliminated.

for #117772 :

In order to facilitate review and modification, split the checking code and
removing redundant imports code into two PR.
2023-12-10 10:56:22 +08:00
Nicholas Nethercote
2c337a072c Rename HandlerInner::delayed_span_bugs as HandlerInner::span_delayed_bugs.
For reasons similar to the previous commit.
2023-12-02 09:01:34 +11:00
Ben Kimock
fbaa24ee35 Call FileEncoder::finish in rmeta encoding 2023-11-22 22:49:22 -05:00
Nicholas Nethercote
3ef9d4d0ed Replace custom_encodable with encodable.
By default, `newtype_index!` types get a default `Encodable`/`Decodable`
impl. You can opt out of this with `custom_encodable`. Opting out is the
opposite to how Rust normally works with autogenerated (derived) impls.

This commit inverts the behaviour, replacing `custom_encodable` with
`encodable` which opts into the default `Encodable`/`Decodable` impl.
Only 23 of the 59 `newtype_index!` occurrences need `encodable`.

Even better, there were eight crates with a dependency on
`rustc_serialize` just from unused default `Encodable`/`Decodable`
impls. This commit removes that dependency from those eight crates.
2023-11-22 18:37:14 +11:00
Nilstrieb
21a870515b Fix clippy::needless_borrow in the compiler
`x clippy compiler -Aclippy::all -Wclippy::needless_borrow --fix`.

Then I had to remove a few unnecessary parens and muts that were exposed
now.
2023-11-21 20:13:40 +01:00
Nicholas Nethercote
8f669f558a Reduce exposure of things. 2023-11-16 16:49:22 +11:00
Jubilee
975d042d4c Rollup merge of #116534 - cjgillot:no-dep-tasks, r=davidtwco
Remove -Zdep-tasks.

This option is not useful any more, we can use `tracing` and `RUSTC_LOG` to debug the dep-graph.
2023-10-28 01:07:35 -07:00
gvozdvmozgu
bb67e0f47b fix broken link: update incremental compilation url 2023-10-22 07:20:36 -07:00
Camille GILLOT
e960d0e751 Remove -Zdep-tasks. 2023-10-08 12:23:54 +00:00
John Kåre Alsaker
1806efe7f2 Move DepKind to rustc_query_system and define it as u16 2023-09-21 17:06:14 +02:00
bors
3223b0b5e8 Auto merge of #115542 - saethlin:fileencoder-is-bufwriter, r=WaffleLapkin
Simplify/Optimize FileEncoder

FileEncoder is basically a BufWriter except that it exposes access to the not-written-to-yet region of the buffer so that some users can write directly to the buffer. This strategy is awesome because it lets us avoid calling memcpy for small copies, but the previous strategy was based on the writer accessing a `&mut [MaybeUninit<u8>; N]` and returning a `&[u8]` which is an API which currently mandates the use of unsafe code, making that interface in general not that appealing.

So this PR cleans up the FileEncoder implementation and builds on that general idea of direct buffer access in order to prevent `memcpy` calls in a few key places when encoding the dep graph and rmeta tables. The interface used here is now 100% safe, but with the caveat that internally we need to avoid trusting the number of bytes that the provided function claims to have written.

The original primary objective of this PR was to clean up the FileEncoder implementation so that the fix for the following issues would be easy to implement. The fix for these issues is to correctly update self.buffered even when writes fail, which I think it's easy to verify manually is now done, because all the FileEncoder methods are small.

Fixes https://github.com/rust-lang/rust/issues/115298
Fixes https://github.com/rust-lang/rust/issues/114671
Fixes https://github.com/rust-lang/rust/issues/114045
Fixes https://github.com/rust-lang/rust/issues/108100
Fixes https://github.com/rust-lang/rust/issues/106787
2023-09-20 21:47:54 +00:00
Ben Kimock
6cee6b0bde PR feedback 2023-09-20 16:49:13 -04:00
John Kåre Alsaker
f8ad88be81 Use UnhashMap for the index 2023-09-12 08:59:37 +02:00
John Kåre Alsaker
734e5a1fbd Encode the number of dep kinds encountered in the dep graph 2023-09-12 08:51:39 +02:00
John Kåre Alsaker
8c5bc990cc Store a index per dep node kind 2023-09-12 08:46:17 +02:00
Ben Kimock
01e9798148 Reimplement FileEncoder with a small-write optimization 2023-09-10 23:37:51 -04:00
John Kåre Alsaker
8fc160b742 Add optimized lock methods for Sharded 2023-09-08 08:48:44 +02:00
Ben Kimock
469dc8f0fa Add comments with the same level of detail as the PR description 2023-09-06 21:15:03 -04:00
Ben Kimock
94fe18f84b Use a specialized varint + bitpacking scheme for DepGraph encoding 2023-09-04 12:16:50 -04:00
John Kåre Alsaker
81220c0ace Keep SHARDS fixed instead of a function of cfg!(parallel_compiler) 2023-08-16 10:00:25 +02:00
bjorn3
980143b50c Pass WorkProductMap to build_dep_graph instead of FxIndexMap
Constructing an FxIndexMap is useless work as the iteration order never
matters.
2023-08-13 16:07:48 +00:00