Skip to content

Commit 299a13b

Browse files
committed
sembr tests/running.md
1 parent 2ae4b0b commit 299a13b

File tree

1 file changed

+76
-75
lines changed

1 file changed

+76
-75
lines changed

src/doc/rustc-dev-guide/src/tests/running.md

Lines changed: 76 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
11
# Running tests
22

3-
You can run the entire test collection using `x`. But note that running the
4-
*entire* test collection is almost never what you want to do during local
5-
development because it takes a really long time. For local development, see the
6-
subsection after on how to run a subset of tests.
3+
You can run the entire test collection using `x`.
4+
But note that running the *entire* test collection is almost never what you want to do during local
5+
development because it takes a really long time.
6+
For local development, see the subsection after on how to run a subset of tests.
77

88
<div class="warning">
99

10-
Running plain `./x test` will build the stage 1 compiler and then run the whole
11-
test suite. This not only includes `tests/`, but also `library/`, `compiler/`,
10+
Running plain `./x test` will build the stage 1 compiler and then run the whole test suite.
11+
This not only includes `tests/`, but also `library/`, `compiler/`,
1212
`src/tools/` package tests and more.
1313

1414
You usually only want to run a subset of the test suites (or even a smaller set
15-
of tests than that) which you expect will exercise your changes. PR CI exercises
16-
a subset of test collections, and merge queue CI will exercise all of the test
15+
of tests than that) which you expect will exercise your changes.
16+
PR CI exercises a subset of test collections, and merge queue CI will exercise all of the test
1717
collection.
1818

1919
</div>
@@ -22,8 +22,8 @@ collection.
2222
./x test
2323
```
2424

25-
The test results are cached and previously successful tests are `ignored` during
26-
testing. The stdout/stderr contents as well as a timestamp file for every test
25+
The test results are cached and previously successful tests are `ignored` during testing.
26+
The stdout/stderr contents as well as a timestamp file for every test
2727
can be found under `build/<target-tuple>/test/` for the given
2828
`<target-tuple>`. To force-rerun a test (e.g. in case the test runner fails to
2929
notice a change) you can use the `--force-rerun` CLI option.
@@ -40,18 +40,17 @@ notice a change) you can use the `--force-rerun` CLI option.
4040
4141
## Running a subset of the test suites
4242

43-
When working on a specific PR, you will usually want to run a smaller set of
44-
tests. For example, a good "smoke test" that can be used after modifying rustc
43+
When working on a specific PR, you will usually want to run a smaller set of tests.
44+
For example, a good "smoke test" that can be used after modifying rustc
4545
to see if things are generally working correctly would be to exercise the `ui`
4646
test suite ([`tests/ui`]):
4747

4848
```text
4949
./x test tests/ui
5050
```
5151

52-
Of course, the choice of test suites is
53-
somewhat arbitrary, and may not suit the task you are doing. For example, if you
54-
are hacking on debuginfo, you may be better off with the debuginfo test suite:
52+
Of course, the choice of test suites is somewhat arbitrary, and may not suit the task you are doing.
53+
For example, if you are hacking on debuginfo, you may be better off with the debuginfo test suite:
5554

5655
```text
5756
./x test tests/debuginfo
@@ -77,8 +76,8 @@ Likewise, you can test a single file by passing its path:
7776
./x test tests/ui/const-generics/const-test.rs
7877
```
7978

80-
`x` doesn't support running a single tool test by passing its path yet. You'll
81-
have to use the `--test-args` argument as described
79+
`x` doesn't support running a single tool test by passing its path yet.
80+
You'll have to use the `--test-args` argument as described
8281
[below](#running-an-individual-test).
8382

8483
```text
@@ -97,8 +96,8 @@ have to use the `--test-args` argument as described
9796
./x test --stage 0 library/std
9897
```
9998

100-
Note that this only runs tests on `std`; if you want to test `core` or other
101-
crates, you have to specify those explicitly.
99+
Note that this only runs tests on `std`;
100+
if you want to test `core` or other crates, you have to specify those explicitly.
102101

103102
### Run the tidy script and tests on the standard library
104103

@@ -117,8 +116,8 @@ you avoid having to run tests for components you did not change at all.
117116

118117
<div class="warning">
119118

120-
Note that bors only runs the tests with the full stage 2 build; therefore, while
121-
the tests **usually** work fine with stage 1, there are some limitations.
119+
Note that bors only runs the tests with the full stage 2 build;
120+
therefore, while the tests **usually** work fine with stage 1, there are some limitations.
122121

123122
</div>
124123

@@ -128,8 +127,8 @@ the tests **usually** work fine with stage 1, there are some limitations.
128127
./x test --stage 2
129128
```
130129

131-
<div class="warning">
132-
You almost never need to do this; CI will run these tests for you.
130+
<div class="warning"> You almost never need to do this;
131+
CI will run these tests for you.
133132
</div>
134133

135134
## Run unit tests on the compiler/library
@@ -140,7 +139,8 @@ You may want to run unit tests on a specific file with following:
140139
./x test compiler/rustc_data_structures/src/thin_vec/tests.rs
141140
```
142141

143-
But unfortunately, it's impossible. You should invoke the following instead:
142+
But unfortunately, it's impossible.
143+
You should invoke the following instead:
144144

145145
```text
146146
./x test compiler/rustc_data_structures/ --test-args thin_vec
@@ -149,24 +149,23 @@ But unfortunately, it's impossible. You should invoke the following instead:
149149
## Running an individual test
150150

151151
Another common thing that people want to do is to run an **individual test**,
152-
often the test they are trying to fix. As mentioned earlier, you may pass the
153-
full file path to achieve this, or alternatively one may invoke `x` with the
154-
`--test-args` option:
152+
often the test they are trying to fix.
153+
As mentioned earlier, you may pass the
154+
full file path to achieve this, or alternatively one may invoke `x` with the `--test-args` option:
155155

156156
```text
157157
./x test tests/ui --test-args issue-1234
158158
```
159159

160160
Under the hood, the test runner invokes the standard Rust test runner (the same
161161
one you get with `#[test]`), so this command would wind up filtering for tests
162-
that include "issue-1234" in the name. Thus, `--test-args` is a good way to run
163-
a collection of related tests.
162+
that include "issue-1234" in the name.
163+
Thus, `--test-args` is a good way to run a collection of related tests.
164164

165165
## Passing arguments to `rustc` when running tests
166166

167167
It can sometimes be useful to run some tests with specific compiler arguments,
168-
without using `RUSTFLAGS` (during development of unstable features, with `-Z`
169-
flags, for example).
168+
without using `RUSTFLAGS` (during development of unstable features, with `-Z` flags, for example).
170169

171170
This can be done with `./x test`'s `--compiletest-rustc-args` option, to pass
172171
additional arguments to the compiler when building the tests.
@@ -176,8 +175,7 @@ additional arguments to the compiler when building the tests.
176175
If you have changed the compiler's output intentionally, or you are making a new
177176
test, you can pass `--bless` to the test subcommand.
178177

179-
As an example,
180-
if some tests in `tests/ui` are failing, you can run this command:
178+
As an example, if some tests in `tests/ui` are failing, you can run this command:
181179

182180
```text
183181
./x test tests/ui --bless
@@ -192,8 +190,9 @@ just like when running the tests without the `--bless` flag.
192190
There are a few options for running tests:
193191

194192
* `bootstrap.toml` has the `rust.verbose-tests` option. If `false`, each test will
195-
print a single dot (the default). If `true`, the name of every test will be
196-
printed. This is equivalent to the `--quiet` option in the [Rust test
193+
print a single dot (the default).
194+
If `true`, the name of every test will be printed.
195+
This is equivalent to the `--quiet` option in the [Rust test
197196
harness](https://doc.rust-lang.org/rustc/tests/).
198197
* The environment variable `RUST_TEST_THREADS` can be set to the number of
199198
concurrent threads to use for testing.
@@ -202,24 +201,24 @@ There are a few options for running tests:
202201

203202
Pass UI tests now have three modes, `check-pass`, `build-pass` and `run-pass`.
204203
When `--pass $mode` is passed, these tests will be forced to run under the given
205-
`$mode` unless the directive `//@ ignore-pass` exists in the test file. For
206-
example, you can run all the tests in `tests/ui` as `check-pass`:
204+
`$mode` unless the directive `//@ ignore-pass` exists in the test file.
205+
For example, you can run all the tests in `tests/ui` as `check-pass`:
207206

208207
```text
209208
./x test tests/ui --pass check
210209
```
211210

212-
By passing `--pass $mode`, you can reduce the testing time. For each mode,
213-
please see [Controlling pass/fail
211+
By passing `--pass $mode`, you can reduce the testing time.
212+
For each mode, please see [Controlling pass/fail
214213
expectations](ui.md#controlling-passfail-expectations).
215214

216215
## Running tests with different "compare modes"
217216

218-
UI tests may have different output depending on certain "modes" that the
219-
compiler is in. For example, when using the Polonius mode, a test `foo.rs` will
217+
UI tests may have different output depending on certain "modes" that the compiler is in.
218+
For example, when using the Polonius mode, a test `foo.rs` will
220219
first look for expected output in `foo.polonius.stderr`, falling back to the
221-
usual `foo.stderr` if not found. The following will run the UI test suite in
222-
Polonius mode:
220+
usual `foo.stderr` if not found.
221+
The following will run the UI test suite in Polonius mode:
223222

224223
```text
225224
./x test tests/ui --compare-mode=polonius
@@ -229,26 +228,28 @@ See [Compare modes](compiletest.md#compare-modes) for more details.
229228

230229
## Running tests manually
231230

232-
Sometimes it's easier and faster to just run the test by hand. Most tests are
233-
just `.rs` files, so after [creating a rustup
231+
Sometimes it's easier and faster to just run the test by hand.
232+
Most tests are just `.rs` files, so after [creating a rustup
234233
toolchain](../building/how-to-build-and-run.md#creating-a-rustup-toolchain), you
235234
can do something like:
236235

237236
```text
238237
rustc +stage1 tests/ui/issue-1234.rs
239238
```
240239

241-
This is much faster, but doesn't always work. For example, some tests include
240+
This is much faster, but doesn't always work.
241+
For example, some tests include
242242
directives that specify specific compiler flags, or which rely on other crates,
243243
and they may not run the same without those options.
244244

245245
## Running tests on a remote machine
246246

247247
Tests may be run on a remote machine (e.g. to test builds for a different
248-
architecture). This is done using `remote-test-client` on the build machine to
248+
architecture).
249+
This is done using `remote-test-client` on the build machine to
249250
send test programs to `remote-test-server` running on the remote machine.
250-
`remote-test-server` executes the test programs and sends the results back to
251-
the build machine. `remote-test-server` provides *unauthenticated remote code
251+
`remote-test-server` executes the test programs and sends the results back to the build machine.
252+
`remote-test-server` provides *unauthenticated remote code
252253
execution* so be careful where it is used.
253254

254255
To do this, first build `remote-test-server` for the remote machine, e.g. for
@@ -258,13 +259,12 @@ RISC-V
258259
./x build src/tools/remote-test-server --target riscv64gc-unknown-linux-gnu
259260
```
260261

261-
The binary will be created at
262-
`./build/host/stage2-tools/$TARGET_ARCH/release/remote-test-server`. Copy this
263-
over to the remote machine.
262+
The binary will be created at `./build/host/stage2-tools/$TARGET_ARCH/release/remote-test-server`.
263+
Copy this over to the remote machine.
264264

265265
On the remote machine, run the `remote-test-server` with the `--bind
266-
0.0.0.0:12345` flag (and optionally `-v` for verbose output). Output should look
267-
like this:
266+
0.0.0.0:12345` flag (and optionally `-v` for verbose output).
267+
Output should look like this:
268268

269269
```text
270270
$ ./remote-test-server -v --bind 0.0.0.0:12345
@@ -273,12 +273,13 @@ listening on 0.0.0.0:12345!
273273
```
274274

275275
Note that binding the server to 0.0.0.0 will allow all hosts able to reach your
276-
machine to execute arbitrary code on your machine. We strongly recommend either
276+
machine to execute arbitrary code on your machine.
277+
We strongly recommend either
277278
setting up a firewall to block external access to port 12345, or to use a more
278279
restrictive IP address when binding.
279280

280-
You can test if the `remote-test-server` is working by connecting to it and
281-
sending `ping\n`. It should reply `pong`:
281+
You can test if the `remote-test-server` is working by connecting to it and sending `ping\n`.
282+
It should reply `pong`:
282283

283284
```text
284285
$ nc $REMOTE_IP 12345
@@ -287,8 +288,8 @@ pong
287288
```
288289

289290
To run tests using the remote runner, set the `TEST_DEVICE_ADDR` environment
290-
variable then use `x` as usual. For example, to run `ui` tests for a RISC-V
291-
machine with the IP address `1.2.3.4` use
291+
variable then use `x` as usual.
292+
For example, to run `ui` tests for a RISC-V machine with the IP address `1.2.3.4` use
292293

293294
```text
294295
export TEST_DEVICE_ADDR="1.2.3.4:12345"
@@ -315,34 +316,34 @@ run "/tmp/work/test1018/a"
315316
[...]
316317
```
317318

318-
Tests are built on the machine running `x` not on the remote machine. Tests
319-
which fail to build unexpectedly (or `ui` tests producing incorrect build
319+
Tests are built on the machine running `x` not on the remote machine.
320+
Tests which fail to build unexpectedly (or `ui` tests producing incorrect build
320321
output) may fail without ever running on the remote machine.
321322

322323
## Testing on emulators
323324

324-
Some platforms are tested via an emulator for architectures that aren't readily
325-
available. For architectures where the standard library is well supported and
325+
Some platforms are tested via an emulator for architectures that aren't readily available.
326+
For architectures where the standard library is well supported and
326327
the host operating system supports TCP/IP networking, see the above instructions
327328
for testing on a remote machine (in this case the remote machine is emulated).
328329

329-
There is also a set of tools for orchestrating running the tests within the
330-
emulator. Platforms such as `arm-android` and `arm-unknown-linux-gnueabihf` are
331-
set up to automatically run the tests under emulation on GitHub Actions. The
332-
following will take a look at how a target's tests are run under emulation.
330+
There is also a set of tools for orchestrating running the tests within the emulator.
331+
Platforms such as `arm-android` and `arm-unknown-linux-gnueabihf` are
332+
set up to automatically run the tests under emulation on GitHub Actions.
333+
The following will take a look at how a target's tests are run under emulation.
333334

334-
The Docker image for [armhf-gnu] includes [QEMU] to emulate the ARM CPU
335-
architecture. Included in the Rust tree are the tools [remote-test-client] and
335+
The Docker image for [armhf-gnu] includes [QEMU] to emulate the ARM CPU architecture.
336+
Included in the Rust tree are the tools [remote-test-client] and
336337
[remote-test-server] which are programs for sending test programs and libraries
337-
to the emulator, and running the tests within the emulator, and reading the
338-
results. The Docker image is set up to launch `remote-test-server` and the
338+
to the emulator, and running the tests within the emulator, and reading the results.
339+
The Docker image is set up to launch `remote-test-server` and the
339340
build tools use `remote-test-client` to communicate with the server to
340341
coordinate running tests (see [src/bootstrap/src/core/build_steps/test.rs]).
341342

342-
To run on the iOS/tvOS/watchOS/visionOS simulator, we can similarly treat it as
343-
a "remote" machine. A curious detail here is that the network is shared between
344-
the simulator instance and the host macOS, so we can use the local loopback
345-
address `127.0.0.1`. Something like the following should work:
343+
To run on the iOS/tvOS/watchOS/visionOS simulator, we can similarly treat it as a "remote" machine.
344+
A curious detail here is that the network is shared between
345+
the simulator instance and the host macOS, so we can use the local loopback address `127.0.0.1`.
346+
Something like the following should work:
346347

347348
```sh
348349
# Build the test server for the iOS simulator:

0 commit comments

Comments
 (0)