The unifying theme for this commit is weak, admittedly. I put together a
list of "expensive" functions when I originally proposed this whole
effort, but nobody's cared about that criterion. Still, it's a decent
way to bite off a not-too-big chunk of work.
Given the grab bag nature of this commit, the messages I used vary quite
a bit.
Add #[must_use] to from_value conversions
I added two methods to the list myself. Clippy did not flag them because they take `mut` args, but neither modifies their argument.
```rust
core::str const unsafe fn from_utf8_unchecked_mut(v: &mut [u8]) -> &mut str;
std::ffi::CString unsafe fn from_raw(ptr: *mut c_char) -> CString;
```
I put a custom note on `from_raw`:
```rust
#[must_use = "call `drop(from_raw(ptr))` if you intend to drop the `CString`"]
pub unsafe fn from_raw(ptr: *mut c_char) -> CString {
```
Parent issue: #89692
r? ``@joshtriplett``
Optimize File::read_to_end and read_to_string
Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity.
`std::fs::read` and `std::fs::read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the documentation for both the `Read` trait and the `Read::read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as `impl Read` or `dyn Read` will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked `unsafe`.
- `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added `@camelid's` suggested wording from https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502.
r? `@joshtriplett`
Reading a file into an empty vector or string buffer can incur
unnecessary `read` syscalls and memory re-allocations as the buffer
"warms up" and grows to its final size. This is perhaps a necessary evil
with generic readers, but files can be read in smarter by checking the
file size and reserving that much capacity.
`std::fs::read` and `read_to_string` already perform this optimization:
they open the file, reads its metadata, and call `with_capacity` with
the file size. This ensures that the buffer does not need to be resized
and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls
`file.read_to_end` or `file.read_to_string` they do not get this
optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of
examples of this. They're found all over in unit tests, which isn't a
big deal, but there are also several real instances in the compiler and
in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the `Read` trait and the `read_to_end` method both show
this exact pattern as examples of how to use readers. What this says to
me is that this shouldn't be solved by simply fixing the instances of it
in this codebase. If it's here it's certain to be prevalent in the wider
Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and
`read_to_string` directly on `File`. This way it's no longer a minor
footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as
a bare `Read` constraint or via a `dyn Read` trait object will benefit.
For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to
`self.inner`'s methods. That way it can call `File`'s optimized
implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked
`unsafe`.
- `File::read_to_string` being more efficient means that the performance
note for `io::read_to_string` can be softened. I've added @camelid's
suggested wording from:
https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
Fix read_to_end to not grow an exact size buffer
If you know how much data to expect and use `Vec::with_capacity` to pre-allocate a buffer of that capacity, `Read::read_to_end` will still double its capacity. It needs some space to perform a read, even though that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading into a small "probe" buffer instead. If that read returns `0` then it's confirmed that the buffer was the perfect size. If it doesn't, the probe buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in `initial_buffer_size`.
Curiously, there was a unit test that specifically checked that `Read::read_to_end` *does* over-allocate. I removed that test, too.
Fix spacing of links in inline code.
Similar to #80733, but the focus is different. This PR eliminates all occurrences of pieced-together inline code blocks like [`Box`]`<`[`Option`]`<T>>` and replaces them with good-looking ones (using HTML-syntax), like <code>[Box]<[Option]\<T>></code>. As far as I can tell, I should’ve found all of these in the standard library (regex search with `` r"`\]`|`\[`" ``) \[except for in `core::convert` where I’ve noticed other things in the docs that I want to fix in a separate PR]. In particular, unlike #80733, I’ve added almost no new instance of inline code that’s broken up into multiple links (or some link and some link-free part). I also added tooltips (the stuff in quotes for the markdown link listings) in places that caught my eye, but that’s by no means systematic, just opportunistic.
[Box]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[`Box`]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[Option]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
[`Option`]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
Context: I got annoyed by repeatedly running into new misformatted inline code while reading the standard library docs. I know that once issue #83997 (and/or related ones) are resolved, these changes become somewhat obsolete, but I fail to notice much progress on that end right now.
r? `@jyn514`
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync}
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::string
----------
Fix spacing for links inside code blocks in alloc::vec
----------
Fix spacing for links inside code blocks in core::option
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in core::result
----------
Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll}
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path}
----------
Fix spacing for links inside code blocks in std::{collections, time}
----------
Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str}
----------
Fix spacing for links inside code blocks, and improve link tooltips in std::ffi
----------
Fix spacing for links inside code blocks, and improve a few link tooltips
in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}}
----------
Fix typo in link to `into` for `OsString` docs
----------
Remove tooltips that will probably become redundant in the future
----------
Apply suggestions from code review
Replacing `…std/primitive.reference.html` paths with just `reference`
Co-authored-by: Joshua Nelson <github@jyn.dev>
----------
Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
If you know how much data to expect and use `Vec::with_capacity` to
pre-allocate a buffer of that capacity, `Read::read_to_end` will still
double its capacity. It needs some space to perform a read, even though
that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into
memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading
into a small "probe" buffer instead. If that read returns `0` then it's
confirmed that the buffer was the perfect size. If it doesn't, the probe
buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be
removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the
previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in
`initial_buffer_size`.
Curiously, there was a unit test that specifically checked that
`Read::read_to_end` *does* over-allocate. I removed that test, too.
Stabilise BufWriter::into_parts
The FCP for this has already completed, in #80690.
This was just blocked on #85901 (which changed the name), which is now merged. The original stabilisation MR was #84770 but that has a lot of noise in it, and I also accidentally deleted the branch while trying to tidy up. So here is a new MR. Sorry for the noise.
Closes#80690
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Replace read_to_string with read_line in Stdin example
The current example results in infinitely reading from stdin, which can confuse newcomers trying to read from stdin.
(`@razmag` encountered this while learning the language from the docs)
impl Default, Copy, Clone for std::io::Sink and Empty
The omission of `Sink: Default` is causing me a slight inconvenience in a test harness. There seems little reason for this and `Empty` not to be `Clone` and `Copy` too.
I have made all three of these insta-stable, because:
AIUI `Copy` can only be derived, and I was not able to find any examples of how to unstably derive it. I think it is probably not possible.
I hunted through the git history for precedent and found
> 79b8ad84c8
> Implement `Copy` for `IoSlice`
> https://github.com/rust-lang/rust/pull/69403
which was also insta-stable.
Fix may not to appropriate might not or must not
I went through and changed occurrences of `may not` to be more explicit with `might not` and `must not`.
I looked in stdlib and as @BurntSushi thought, `raw` is generally
used for raw pointers, or other hazardous kinds of thing. stdlib does
not have `into_parts` apart from the one I added to `IntoInnerError`.
I did an ad-hoc search of the rustdocs for my current game project
Otter, which includes quite a large number of dependencies.
`into_parts` seems heavily used for things quite like this.
So change this name.
Suggested-by: Andrew Gallant <jamslam@gmail.com>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
I didn't notice the submodule, which means I failed to re-export this
to make it actually-public.
Reported-by: Andrew Gallant <jamslam@gmail.com>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
