Merge pull request #2637 from rust-lang/tshepang/sembr

sembr tests/running.md and improve it a bit

Author: tshepang

src/doc/rustc-dev-guide/ci/sembr/src/main.rs

@@ -137,6 +137,7 @@ fn lengthen_lines(content: &str, limit: usize) -> String {
     let mut new_content = content.clone();
     let mut new_n = 0;
     let mut in_code_block = false;
+    let mut in_html_div = false;
     let mut skip_next = false;
     for (n, line) in content.iter().enumerate() {
         if skip_next {
@@ -150,6 +151,17 @@ fn lengthen_lines(content: &str, limit: usize) -> String {
             in_code_block = !in_code_block;
             continue;
         }
+        if line.trim_start().starts_with("<div") {
+            in_html_div = true;
+            continue;
+        }
+        if line.trim_start().starts_with("</div") {
+            in_html_div = false;
+            continue;
+        }
+        if in_html_div {
+            continue;
+        }
         if ignore(line, in_code_block) || REGEX_SPLIT.is_match(line) {
             continue;
         }
@@ -215,9 +227,15 @@ fn test_prettify() {
     let original = "\
 do not split
 short sentences
+<div class='warning'>
+a bit of text inside
+</div>
 ";
     let expected = "\
 do not split short sentences
+<div class='warning'>
+a bit of text inside
+</div>
 ";
     assert_eq!(expected, lengthen_lines(original, 50));
 }

src/doc/rustc-dev-guide/src/tests/running.md

@@ -1,19 +1,19 @@
 # Running tests
 
-You can run the entire test collection using `x`. But note that running the
-*entire* test collection is almost never what you want to do during local
-development because it takes a really long time. For local development, see the
-subsection after on how to run a subset of tests.
+You can run the entire test collection using `x`.
+But note that running the *entire* test collection is almost never what you want to do during local
+development because it takes a really long time.
+For local development, see the subsection after on how to run a subset of tests.
 
 <div class="warning">
 
-Running plain `./x test` will build the stage 1 compiler and then run the whole
-test suite. This not only includes `tests/`, but also `library/`, `compiler/`,
+Running plain `./x test` will build the stage 1 compiler and then run the whole test suite.
+This not only includes `tests/`, but also `library/`, `compiler/`,
 `src/tools/` package tests and more.
 
 You usually only want to run a subset of the test suites (or even a smaller set
-of tests than that) which you expect will exercise your changes. PR CI exercises
-a subset of test collections, and merge queue CI will exercise all of the test
+of tests than that) which you expect will exercise your changes.
+PR CI exercises a subset of test collections, and merge queue CI will exercise all of the test
 collection.
 
 </div>
@@ -22,8 +22,8 @@ collection.
 ./x test
 ```
 
-The test results are cached and previously successful tests are `ignored` during
-testing. The stdout/stderr contents as well as a timestamp file for every test
+The test results are cached and previously successful tests are `ignored` during testing.
+The stdout/stderr contents as well as a timestamp file for every test
 can be found under `build/<target-tuple>/test/` for the given
 `<target-tuple>`. To force-rerun a test (e.g. in case the test runner fails to
 notice a change) you can use the `--force-rerun` CLI option.
@@ -40,18 +40,17 @@ notice a change) you can use the `--force-rerun` CLI option.
 
 ## Running a subset of the test suites
 
-When working on a specific PR, you will usually want to run a smaller set of
-tests. For example, a good "smoke test" that can be used after modifying rustc
+When working on a specific PR, you will usually want to run a smaller set of tests.
+For example, a good "smoke test" that can be used after modifying rustc
 to see if things are generally working correctly would be to exercise the `ui`
 test suite ([`tests/ui`]):
 
 ```text
 ./x test tests/ui
 ```
 
-Of course, the choice of test suites is
-somewhat arbitrary, and may not suit the task you are doing. For example, if you
-are hacking on debuginfo, you may be better off with the debuginfo test suite:
+Of course, the choice of test suites is somewhat arbitrary, and may not suit the task you are doing.
+For example, if you are hacking on debuginfo, you may be better off with the debuginfo test suite:
 
 ```text
 ./x test tests/debuginfo
@@ -77,8 +76,8 @@ Likewise, you can test a single file by passing its path:
 ./x test tests/ui/const-generics/const-test.rs
 ```
 
-`x` doesn't support running a single tool test by passing its path yet. You'll
-have to use the `--test-args` argument as described
+`x` doesn't support running a single tool test by passing its path yet.
+You'll have to use the `--test-args` argument as described
 [below](#running-an-individual-test).
 
 ```text
@@ -97,8 +96,8 @@ have to use the `--test-args` argument as described
 ./x test --stage 0 library/std
 ```
 
-Note that this only runs tests on `std`; if you want to test `core` or other
-crates, you have to specify those explicitly.
+Note that this only runs tests on `std`;
+if you want to test `core` or other crates, you have to specify those explicitly.
 
 ### Run the tidy script and tests on the standard library
 
@@ -117,8 +116,8 @@ you avoid having to run tests for components you did not change at all.
 
 <div class="warning">
 
-Note that bors only runs the tests with the full stage 2 build; therefore, while
-the tests **usually** work fine with stage 1, there are some limitations.
+Note that bors only runs the tests with the full stage 2 build;
+therefore, while the tests **usually** work fine with stage 1, there are some limitations.
 
 </div>
 
@@ -129,7 +128,8 @@ the tests **usually** work fine with stage 1, there are some limitations.
 ```
 
 <div class="warning">
-You almost never need to do this; CI will run these tests for you.
+You almost never need to do this;
+CI will run these tests for you.
 </div>
 
 ## Run unit tests on the compiler/library
@@ -140,7 +140,8 @@ You may want to run unit tests on a specific file with following:
 ./x test compiler/rustc_data_structures/src/thin_vec/tests.rs
 ```
 
-But unfortunately, it's impossible. You should invoke the following instead:
+But unfortunately, it's impossible.
+You should invoke the following instead:
 
 ```text
 ./x test compiler/rustc_data_structures/ --test-args thin_vec
@@ -149,24 +150,23 @@ But unfortunately, it's impossible. You should invoke the following instead:
 ## Running an individual test
 
 Another common thing that people want to do is to run an **individual test**,
-often the test they are trying to fix. As mentioned earlier, you may pass the
-full file path to achieve this, or alternatively one may invoke `x` with the
-`--test-args` option:
+often the test they are trying to fix.
+As mentioned earlier, you may pass the
+full file path to achieve this, or alternatively one may invoke `x` with the `--test-args` option:
 
 ```text
 ./x test tests/ui --test-args issue-1234
 ```
 
 Under the hood, the test runner invokes the standard Rust test runner (the same
 one you get with `#[test]`), so this command would wind up filtering for tests
-that include "issue-1234" in the name. Thus, `--test-args` is a good way to run
-a collection of related tests.
+that include "issue-1234" in the name.
+Thus, `--test-args` is a good way to run a collection of related tests.
 
 ## Passing arguments to `rustc` when running tests
 
 It can sometimes be useful to run some tests with specific compiler arguments,
-without using `RUSTFLAGS` (during development of unstable features, with `-Z`
-flags, for example).
+without using `RUSTFLAGS` (during development of unstable features, with `-Z` flags, for example).
 
 This can be done with `./x test`'s `--compiletest-rustc-args` option, to pass
 additional arguments to the compiler when building the tests.
@@ -176,8 +176,7 @@ additional arguments to the compiler when building the tests.
 If you have changed the compiler's output intentionally, or you are making a new
 test, you can pass `--bless` to the test subcommand.
 
-As an example,
-if some tests in `tests/ui` are failing, you can run this command:
+As an example, if some tests in `tests/ui` are failing, you can run this command:
 
 ```text
 ./x test tests/ui --bless
@@ -192,8 +191,9 @@ just like when running the tests without the `--bless` flag.
 There are a few options for running tests:
 
 * `bootstrap.toml` has the `rust.verbose-tests` option. If `false`, each test will
-  print a single dot (the default). If `true`, the name of every test will be
-  printed. This is equivalent to the `--quiet` option in the [Rust test
+  print a single dot (the default).
+  If `true`, the name of every test will be printed.
+  This is equivalent to the `--quiet` option in the [Rust test
   harness](https://doc.rust-lang.org/rustc/tests/).
 * The environment variable `RUST_TEST_THREADS` can be set to the number of
   concurrent threads to use for testing.
@@ -202,24 +202,24 @@ There are a few options for running tests:
 
 Pass UI tests now have three modes, `check-pass`, `build-pass` and `run-pass`.
 When `--pass $mode` is passed, these tests will be forced to run under the given
-`$mode` unless the directive `//@ ignore-pass` exists in the test file. For
-example, you can run all the tests in `tests/ui` as `check-pass`:
+`$mode` unless the directive `//@ ignore-pass` exists in the test file.
+For example, you can run all the tests in `tests/ui` as `check-pass`:
 
 ```text
 ./x test tests/ui --pass check
 ```
 
-By passing `--pass $mode`, you can reduce the testing time. For each mode,
-please see [Controlling pass/fail
+By passing `--pass $mode`, you can reduce the testing time.
+For each mode, please see [Controlling pass/fail
 expectations](ui.md#controlling-passfail-expectations).
 
 ## Running tests with different "compare modes"
 
-UI tests may have different output depending on certain "modes" that the
-compiler is in. For example, when using the Polonius mode, a test `foo.rs` will
+UI tests may have different output depending on certain "modes" that the compiler is in.
+For example, when using the Polonius mode, a test `foo.rs` will
 first look for expected output in `foo.polonius.stderr`, falling back to the
-usual `foo.stderr` if not found. The following will run the UI test suite in
-Polonius mode:
+usual `foo.stderr` if not found.
+The following will run the UI test suite in Polonius mode:
 
 ```text
 ./x test tests/ui --compare-mode=polonius
@@ -229,66 +229,68 @@ See [Compare modes](compiletest.md#compare-modes) for more details.
 
 ## Running tests manually
 
-Sometimes it's easier and faster to just run the test by hand. Most tests are
-just `.rs` files, so after [creating a rustup
+Sometimes it's easier and faster to just run the test by hand.
+Most tests are just `.rs` files, so after [creating a rustup
 toolchain](../building/how-to-build-and-run.md#creating-a-rustup-toolchain), you
 can do something like:
 
 ```text
 rustc +stage1 tests/ui/issue-1234.rs
 ```
 
-This is much faster, but doesn't always work. For example, some tests include
+This is much faster, but doesn't always work.
+For example, some tests include
 directives that specify specific compiler flags, or which rely on other crates,
 and they may not run the same without those options.
 
 ## Running tests on a remote machine
 
 Tests may be run on a remote machine (e.g. to test builds for a different
-architecture). This is done using `remote-test-client` on the build machine to
+architecture).
+This is done using `remote-test-client` on the build machine to
 send test programs to `remote-test-server` running on the remote machine.
-`remote-test-server` executes the test programs and sends the results back to
-the build machine. `remote-test-server` provides *unauthenticated remote code
+`remote-test-server` executes the test programs and sends the results back to the build machine.
+`remote-test-server` provides *unauthenticated remote code
 execution* so be careful where it is used.
 
-To do this, first build `remote-test-server` for the remote machine, e.g. for
-RISC-V
+To do this, first build `remote-test-server` for the remote machine
+(using RISC-V as an example):
 
 ```text
 ./x build src/tools/remote-test-server --target riscv64gc-unknown-linux-gnu
 ```
 
-The binary will be created at
-`./build/host/stage2-tools/$TARGET_ARCH/release/remote-test-server`. Copy this
-over to the remote machine.
+The binary will be created at `./build/host/stage2-tools/$TARGET_ARCH/release/remote-test-server`.
+Copy this over to the remote machine.
 
 On the remote machine, run the `remote-test-server` with the `--bind
-0.0.0.0:12345` flag (and optionally `-v` for verbose output). Output should look
-like this:
+0.0.0.0:12345` flag (and optionally `--verbose` flag).
+Output should look like this:
 
-```text
-$ ./remote-test-server -v --bind 0.0.0.0:12345
+```console
+$ ./remote-test-server --verbose --bind 0.0.0.0:12345
 starting test server
 listening on 0.0.0.0:12345!
 ```
 
 Note that binding the server to 0.0.0.0 will allow all hosts able to reach your
-machine to execute arbitrary code on your machine. We strongly recommend either
+machine to execute arbitrary code on your machine.
+We strongly recommend either
 setting up a firewall to block external access to port 12345, or to use a more
 restrictive IP address when binding.
 
-You can test if the `remote-test-server` is working by connecting to it and
-sending `ping\n`. It should reply `pong`:
+You can test if the `remote-test-server` is working by connecting to it and sending `ping\n`.
+It should reply `pong`:
 
-```text
+```console
 $ nc $REMOTE_IP 12345
 ping
 pong
 ```
 
 To run tests using the remote runner, set the `TEST_DEVICE_ADDR` environment
-variable then use `x` as usual. For example, to run `ui` tests for a RISC-V
-machine with the IP address `1.2.3.4` use
+variable then use `x` as usual.
+For example, to run `ui` tests for a RISC-V machine with the IP address `1.2.3.4` use
 
 ```text
 export TEST_DEVICE_ADDR="1.2.3.4:12345"
@@ -315,34 +317,34 @@ run "/tmp/work/test1018/a"
 [...]
 ```
 
-Tests are built on the machine running `x` not on the remote machine. Tests
-which fail to build unexpectedly (or `ui` tests producing incorrect build
+Tests are built on the machine running `x` not on the remote machine.
+Tests which fail to build unexpectedly (or `ui` tests producing incorrect build
 output) may fail without ever running on the remote machine.
 
 ## Testing on emulators
 
-Some platforms are tested via an emulator for architectures that aren't readily
-available. For architectures where the standard library is well supported and
+Some platforms are tested via an emulator for architectures that aren't readily available.
+For architectures where the standard library is well supported and
 the host operating system supports TCP/IP networking, see the above instructions
 for testing on a remote machine (in this case the remote machine is emulated).
 
-There is also a set of tools for orchestrating running the tests within the
-emulator. Platforms such as `arm-android` and `arm-unknown-linux-gnueabihf` are
-set up to automatically run the tests under emulation on GitHub Actions. The
-following will take a look at how a target's tests are run under emulation.
+There is also a set of tools for orchestrating running the tests within the emulator.
+Platforms such as `arm-android` and `arm-unknown-linux-gnueabihf` are
+set up to automatically run the tests under emulation on GitHub Actions.
+The following will take a look at how a target's tests are run under emulation.
 
-The Docker image for [armhf-gnu] includes [QEMU] to emulate the ARM CPU
-architecture. Included in the Rust tree are the tools [remote-test-client] and
+The Docker image for [armhf-gnu] includes [QEMU] to emulate the ARM CPU architecture.
+Included in the Rust tree are the tools [remote-test-client] and
 [remote-test-server] which are programs for sending test programs and libraries
-to the emulator, and running the tests within the emulator, and reading the
-results.  The Docker image is set up to launch `remote-test-server` and the
+to the emulator, and running the tests within the emulator, and reading the results.
+The Docker image is set up to launch `remote-test-server` and the
 build tools use `remote-test-client` to communicate with the server to
 coordinate running tests (see [src/bootstrap/src/core/build_steps/test.rs]).
 
-To run on the iOS/tvOS/watchOS/visionOS simulator, we can similarly treat it as
-a "remote" machine. A curious detail here is that the network is shared between
-the simulator instance and the host macOS, so we can use the local loopback
-address `127.0.0.1`. Something like the following should work:
+To run on the iOS/tvOS/watchOS/visionOS simulator, we can similarly treat it as a "remote" machine.
+A curious detail here is that the network is shared between
+the simulator instance and the host macOS, so we can use the local loopback address `127.0.0.1`.
+Something like the following should work:
 
 ```sh
 # Build the test server for the iOS simulator:
@@ -384,7 +386,7 @@ On the [wasm32-wasip1 target support page] a minimum version is specified that y
 Some cmake commands that take a while and give a lot of very concerning c++ warnings...
 Then, in `bootstrap.toml`, point to the sysroot like so:
 
-```
+```toml
 [target.wasm32-wasip1]
 wasi-root = "<wasi-sdk location>/build/sysroot/install/share/wasi-sysroot"
 ```