Implement another internal lints
cc #49509
This adds ~~two~~ one internal lint~~s~~:
1. LINT_PASS_IMPL_WITHOUT_MACRO: Make sure, that the `{declare,impl}_lint_pass` macro is used to implement lint passes. cc #59669
2. ~~USAGE_OF_TYCTXT_AND_SPAN_ARGS: item 2 on the list in #49509~~
~~With 2. I wasn't sure, if this lint should be applied everywhere. That means a careful review of 0955835 would be great. Also 73fb9b4 allows this lint on some functions. Should I also apply this lint there?~~
TODO (not directly relevant for review):
- [ ] https://github.com/rust-lang/rust/pull/59316#discussion_r280186517 (not sure yet, if this works or how to query for `rustc_private`, since it's not in [`Features`](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/feature_gate/struct.Features.html) 🤔 cc @eddyb)
- [x] https://github.com/rust-lang/rust/pull/61735#discussion_r292389870
- [x] Check explicitly for the `{declare,impl}_lint_pass!` macros
r? @oli-obk
* Move fs::create_dir_all calls into DocFS to provide a clean
extension point if async extension there is needed.
* Convert callsites of create_dir_all to ensure_dir to reduce syscalls.
* Convert fs::write usage to DocFS.write
(which also removes a lot of try_err! usage for easier reading)
* Convert File::create calls to use Vec buffers and then DocFS.write
in order to consistently reduce syscalls as well, make
deferring to threads cleaner and avoid leaving dangling content if
writing to existing files....
* Convert OpenOptions usage similarly - I could find no discussion on
the use of create_new for that one output file vs all the other
files render creates, if link redirection attacks are a concern
DocFS will provide a good central point to introduce systematic
create_new usage. (fs::write/File::create is vulnerable to link
redirection attacks).
* DocFS::write defers to rayon for IO on Windows producing a modest
speedup: before this patch on my development workstation:
$ time cargo +mystg1 doc -p winapi:0.3.7
Documenting winapi v0.3.7
Finished dev [unoptimized + debuginfo] target(s) in 6m 11s
real 6m11.734s
Afterwards:
$ time cargo +mystg1 doc -p winapi:0.3.7
Compiling winapi v0.3.7
Documenting winapi v0.3.7
Finished dev [unoptimized + debuginfo] target(s) in 49.53s
real 0m49.643s
I haven't measured how much time is in the compilation logic vs in the
IO and outputting etc, but this takes it from frustating to tolerable
for me, at least for now.
rustdoc: add option to calculate "documentation coverage"
This PR adds a new flag to rustdoc, `--show-coverage`. When passed, this flag will make rustdoc count the number of items in a crate with documentation instead of generating docs. This count will be output as a table of each file in the crate, like this (when run on my crate `egg-mode`):
```
+-------------------------------------+------------+------------+------------+
| File | Documented | Total | Percentage |
+-------------------------------------+------------+------------+------------+
| src/auth.rs | 16 | 16 | 100.0% |
| src/common/mod.rs | 1 | 1 | 100.0% |
| src/common/response.rs | 9 | 9 | 100.0% |
| src/cursor.rs | 24 | 24 | 100.0% |
| src/direct/fun.rs | 6 | 6 | 100.0% |
| src/direct/mod.rs | 41 | 41 | 100.0% |
| src/entities.rs | 50 | 50 | 100.0% |
| src/error.rs | 27 | 27 | 100.0% |
| src/lib.rs | 1 | 1 | 100.0% |
| src/list/fun.rs | 19 | 19 | 100.0% |
| src/list/mod.rs | 22 | 22 | 100.0% |
| src/media/mod.rs | 27 | 27 | 100.0% |
| src/place/fun.rs | 8 | 8 | 100.0% |
| src/place/mod.rs | 35 | 35 | 100.0% |
| src/search.rs | 26 | 26 | 100.0% |
| src/service.rs | 74 | 74 | 100.0% |
| src/stream/mod.rs | 49 | 49 | 100.0% |
| src/tweet/fun.rs | 15 | 15 | 100.0% |
| src/tweet/mod.rs | 73 | 73 | 100.0% |
| src/user/fun.rs | 24 | 24 | 100.0% |
| src/user/mod.rs | 87 | 87 | 100.0% |
+-------------------------------------+------------+------------+------------+
| Total | 634 | 634 | 100.0% |
+-------------------------------------+------------+------------+------------+
```
Trait implementations are not counted because by default they "inherit" the docs from the trait, even though an impl can override those docs. Similarly, inherent impl blocks are not counted at all, because for the majority of cases such docs are not useful. (The usual pattern for inherent impl blocks is to throw all the methods on a type into a single impl block. Any docs you would put on that block would be better served on the type itself.)
In addition, `--show-coverage` can be combined with `--document-private-items` to get the coverage counts for everything in the crate, not just public items.
The coverage calculation is implemented as a late pass and two new sets of passes which strip out most of the work that rustdoc otherwise does when generating docs. The is because after the new pass is executed, rustdoc immediately closes instead of going on to generate documentation.
Many examples of coverage calculations have been included as `rustdoc-ui` tests.
r? @rust-lang/rustdoc
Rustdoc remove old style files
Reopening of #56577 (which I can't seem to reopen...).
I made the flag unstable so with this change, what was blocking the PR is now gone I assume.
rustdoc: overhaul code block lexing errors
Fixes#53919.
This PR moves the reporting of code block lexing errors from rendering time to an early pass, so we can use the compiler's error reporting mechanisms. This dramatically improves the diagnostics in this situation: we now de-emphasize the lexing errors as a note under a warning that has a span and suggestion instead of just emitting errors at the top level.
Additionally, this PR generalizes the markdown -> source span calculation function, which should allow other rustdoc warnings to use better spans in the future.
Last, the PR makes sure that the code block is always emitted in the docs, even if it fails to highlight correctly.
Of note:
- The new pass unfortunately adds another pass over the docs to gather the doc blocks for syntax-checking. I wonder if this could be combined with the pass that looks for testable blocks? I'm not familiar with that code, so I don't know how feasible that is.
- `pulldown_cmark` doesn't make it easy to find the spans of the code blocks, so the code that calculates the spans is a little nasty. It works for all the test cases I threw at it, but I wouldn't be surprised if an edge case would break it. Should have a thorough review.
- This PR worsens the state of #56885, since those certain fatal lexing errors are now emitted before docs get generated at all.
This commit completely removes usage of the `panictry!` macro from
outside libsyntax. The macro causes parse errors to be fatal, so using
it in libsyntax_ext caused parse failures *within* a syntax extension to
be fatal, which is probably not intended.
Furthermore, this commit adds spans to diagnostics emitted by empty
extensions if they were missing, à la #56491.
Add index page argument
@Mark-Simulacrum: I might need some help from you: in bootstrap, I want to add an argument (a new flag added into `rustdoc`) in order to generate the current index directly when `rustdoc` is documenting the `std` lib. However, my change in `bootstrap` didn't do it and I assume it must be moved inside the `Std` struct. But there, I don't see how to pass it to `rustdoc` through `cargo`. Did I miss anything?
r? @QuietMisdreavus
add -Zui-testing to rustdoc
Before we depend on the `rustdoc-ui` tests some more, let's make rustdoc act the same as the compiler when they're actually being executed.
constraints:
- clean/inline.rs needs this map to fill in traits when inlining
- fold.rs needs this map to allow passes to fold trait items
- html/render.rs needs this map to seed the Cache.traits map of all
known traits
The first two are the real problem, since `DocFolder` only operates on
`clean::Crate` but `clean/inline.rs` only sees the `DocContext`. The
introduction of early passes means that these two now exist at the same
time, so they need to share ownership of the map. Even better, the use
of `Crate` in a rustc thread pool means that it needs to be Sync, so it
can't use `Lrc<Lock>` to manually activate thread-safety.
`parking_lot` is reused from elsewhere in the tree to allow use of its
`ReentrantMutex`, as the relevant parts of rustdoc are still
single-threaded and this allows for easier use in that context.
