Auto merge of rust-lang#3590 - RalfJung:miri-test-target, r=RalfJung

make MIRI_TEST_TARGET and RUSTC_BLESS entirely an internal thing

They are just used to communicate between `./miri test`/`./x.py test miri` and the test harness, but should not be used by users.

Author: bors

src/tools/miri/CONTRIBUTING.md

@@ -78,8 +78,8 @@ You can (cross-)run the entire test suite using:
 ```
 
 `./miri test FILTER` only runs those tests that contain `FILTER` in their filename (including the
-base directory, e.g. `./miri test fail` will run all compile-fail tests). Multiple filters 
-are supported: `./miri test FILTER1 FILTER2`.
+base directory, e.g. `./miri test fail` will run all compile-fail tests). Multiple filters are
+supported: `./miri test FILTER1 FILTER2` runs all tests that contain either string.
 
 #### Fine grained logging
 
@@ -139,9 +139,8 @@ and then you can use it as if it was installed by `rustup` as a component of the
 in the `miri` toolchain's sysroot to prevent conflicts with other toolchains.
 The Miri binaries in the `cargo` bin directory (usually `~/.cargo/bin`) are managed by rustup.
 
-There's a test for the cargo wrapper in the `test-cargo-miri` directory; run
-`./run-test.py` in there to execute it. Like `./miri test`, this respects the
-`MIRI_TEST_TARGET` environment variable to execute the test for another target.
+There's a test for the cargo wrapper in the `test-cargo-miri` directory; run `./run-test.py` in
+there to execute it. You can pass `--target` to execute the test for another target.
 
 ### Using a modified standard library
 
@@ -287,3 +286,41 @@ https. Add the following to your `.gitconfig`:
 [url "git@github.com:"]
     pushInsteadOf = https://github.com/
 ```
+
+## Internal environment variables
+
+The following environment variables are *internal* and must not be used by
+anyone but Miri itself. They are used to communicate between different Miri
+binaries, and as such worth documenting:
+
+* `CARGO_EXTRA_FLAGS` is understood by `./miri` and passed to all host cargo invocations.
+* `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to
+  actually not interpret the code but compile it like rustc would. With `target`, Miri sets
+  some compiler flags to prepare the code for interpretation; with `host`, this is not done.
+  This environment variable is useful to be sure that the compiled `rlib`s are compatible
+  with Miri.
+* `MIRI_CALLED_FROM_SETUP` is set during the Miri sysroot build,
+  which will re-invoke `cargo-miri` as the `rustc` to use for this build.
+* `MIRI_CALLED_FROM_RUSTDOC` when set to any value tells `cargo-miri` that it is
+  running as a child process of `rustdoc`, which invokes it twice for each doc-test
+  and requires special treatment, most notably a check-only build before interpretation.
+  This is set by `cargo-miri` itself when running as a `rustdoc`-wrapper.
+* `MIRI_CWD` when set to any value tells the Miri driver to change to the given
+  directory after loading all the source files, but before commencing
+  interpretation. This is useful if the interpreted program wants a different
+  working directory at run-time than at build-time.
+* `MIRI_LOCAL_CRATES` is set by `cargo-miri` to tell the Miri driver which
+  crates should be given special treatment in diagnostics, in addition to the
+  crate currently being compiled.
+* `MIRI_ORIG_RUSTDOC` is set and read by different phases of `cargo-miri` to remember the
+  value of `RUSTDOC` from before it was overwritten.
+* `MIRI_REPLACE_LIBRS_IF_NOT_TEST` when set to any value enables a hack that helps bootstrap
+  run the standard library tests in Miri.
+* `MIRI_TEST_TARGET` is set by `./miri test` (and `./x.py test miri`) to tell the test harness about
+  the chosen target.
+* `MIRI_VERBOSE` when set to any value tells the various `cargo-miri` phases to
+  perform verbose logging.
+* `MIRI_HOST_SYSROOT` is set by bootstrap to tell `cargo-miri` which sysroot to use for *host*
+  operations.
+* `RUSTC_BLESS` is set by `./miri test` (and `./x.py test miri`) to indicate bless-mode to the test
+  harness.

src/tools/miri/README.md

@@ -463,50 +463,14 @@ by all intended entry points, i.e. `cargo miri` and `./miri {test,run}`):
 * `MIRI_SYSROOT` indicates the sysroot to use. When using `cargo miri`, this skips the automatic
   setup -- only set this if you do not want to use the automatically created sysroot. When invoking
   `cargo miri setup`, this indicates where the sysroot will be put.
-* `MIRI_TEST_TARGET` (recognized by `./miri {test,run}`) indicates which target
-  architecture to test against.  The `--target` flag may be used for the same
-  purpose.
 * `MIRI_TEST_THREADS` (recognized by `./miri test`): set the number of threads to use for running tests.
   By default, the number of cores is used.
 * `MIRI_NO_STD` makes sure that the target's sysroot is built without libstd. This allows testing
   and running no_std programs. (Miri has a heuristic to detect no-std targets based on the target
   name; this environment variable is only needed when that heuristic fails.)
-* `RUSTC_BLESS` (recognized by `./miri test` and `cargo-miri-test/run-test.py`): overwrite all
-  `stderr` and `stdout` files instead of checking whether the output matches.
 * `MIRI_SKIP_UI_CHECKS` (recognized by `./miri test`): don't check whether the
   `stderr` or `stdout` files match the actual output.
 
-The following environment variables are *internal* and must not be used by
-anyone but Miri itself. They are used to communicate between different Miri
-binaries, and as such worth documenting:
-
-* `MIRI_BE_RUSTC` can be set to `host` or `target`. It tells the Miri driver to
-  actually not interpret the code but compile it like rustc would. With `target`, Miri sets
-  some compiler flags to prepare the code for interpretation; with `host`, this is not done.
-  This environment variable is useful to be sure that the compiled `rlib`s are compatible
-  with Miri.
-* `MIRI_CALLED_FROM_SETUP` is set during the Miri sysroot build,
-  which will re-invoke `cargo-miri` as the `rustc` to use for this build.
-* `MIRI_CALLED_FROM_RUSTDOC` when set to any value tells `cargo-miri` that it is
-  running as a child process of `rustdoc`, which invokes it twice for each doc-test
-  and requires special treatment, most notably a check-only build before interpretation.
-  This is set by `cargo-miri` itself when running as a `rustdoc`-wrapper.
-* `MIRI_CWD` when set to any value tells the Miri driver to change to the given
-  directory after loading all the source files, but before commencing
-  interpretation. This is useful if the interpreted program wants a different
-  working directory at run-time than at build-time.
-* `MIRI_LOCAL_CRATES` is set by `cargo-miri` to tell the Miri driver which
-  crates should be given special treatment in diagnostics, in addition to the
-  crate currently being compiled.
-* `MIRI_ORIG_RUSTDOC` is set and read by different phases of `cargo-miri` to remember the
-  value of `RUSTDOC` from before it was overwritten.
-* `MIRI_REPLACE_LIBRS_IF_NOT_TEST` when set to any value enables a hack that helps bootstrap
-  run the standard library tests in Miri.
-* `MIRI_VERBOSE` when set to any value tells the various `cargo-miri` phases to
-  perform verbose logging.
-* `MIRI_HOST_SYSROOT` is set by bootstrap to tell `cargo-miri` which sysroot to use for *host*
-  operations.
-
 [testing-miri]: CONTRIBUTING.md#testing-the-miri-driver
 
 ## Miri `extern` functions

src/tools/miri/ci/ci.sh

@@ -31,24 +31,26 @@ time ./miri build --all-targets # the build that all the `./miri test` below wil
 endgroup
 
 # Run tests. Recognizes these variables:
-# - MIRI_TEST_TARGET: the target to test. Empty for host target.
+# - TEST_TARGET: the target to test. Empty for host target.
 # - GC_STRESS: if non-empty, run the GC stress test for the main test suite.
 # - MIR_OPT: if non-empty, re-run test `pass` tests with mir-opt-level=4
 # - MANY_SEEDS: if set to N, run the "many-seeds" tests N times
 # - TEST_BENCH: if non-empty, check that the benchmarks all build
 # - CARGO_MIRI_ENV: if non-empty, set some env vars and config to potentially confuse cargo-miri
 function run_tests {
-  if [ -n "${MIRI_TEST_TARGET-}" ]; then
-    begingroup "Testing foreign architecture $MIRI_TEST_TARGET"
+  if [ -n "${TEST_TARGET-}" ]; then
+    begingroup "Testing foreign architecture $TEST_TARGET"
+    TARGET_FLAG="--target $TEST_TARGET"
   else
     begingroup "Testing host architecture"
+    TARGET_FLAG=""
   fi
 
   ## ui test suite
   if [ -n "${GC_STRESS-}" ]; then
-    time MIRIFLAGS="${MIRIFLAGS-} -Zmiri-provenance-gc=1" ./miri test
+    time MIRIFLAGS="${MIRIFLAGS-} -Zmiri-provenance-gc=1" ./miri test $TARGET_FLAG
   else
-    time ./miri test
+    time ./miri test $TARGET_FLAG
   fi
 
   ## advanced tests
@@ -59,17 +61,17 @@ function run_tests {
     # them. Also error locations change so we don't run the failing tests.
     # We explicitly enable debug-assertions here, they are disabled by -O but we have tests
     # which exist to check that we panic on debug assertion failures.
-    time MIRIFLAGS="${MIRIFLAGS-} -O -Zmir-opt-level=4 -Cdebug-assertions=yes" MIRI_SKIP_UI_CHECKS=1 ./miri test tests/{pass,panic}
+    time MIRIFLAGS="${MIRIFLAGS-} -O -Zmir-opt-level=4 -Cdebug-assertions=yes" MIRI_SKIP_UI_CHECKS=1 ./miri test $TARGET_FLAG tests/{pass,panic}
   fi
   if [ -n "${MANY_SEEDS-}" ]; then
     # Also run some many-seeds tests.
     time for FILE in tests/many-seeds/*.rs; do
-      ./miri run "--many-seeds=0..$MANY_SEEDS" "$FILE"
+      ./miri run "--many-seeds=0..$MANY_SEEDS" $TARGET_FLAG "$FILE"
     done
   fi
   if [ -n "${TEST_BENCH-}" ]; then
     # Check that the benchmarks build and run, but only once.
-    time HYPERFINE="hyperfine -w0 -r1" ./miri bench
+    time HYPERFINE="hyperfine -w0 -r1" ./miri bench $TARGET_FLAG
   fi
 
   ## test-cargo-miri
@@ -91,7 +93,7 @@ function run_tests {
     echo 'build.rustc-wrapper = "thisdoesnotexist"' > .cargo/config.toml
   fi
   # Run the actual test
-  time ${PYTHON} test-cargo-miri/run-test.py
+  time ${PYTHON} test-cargo-miri/run-test.py $TARGET_FLAG
   # Clean up
   unset RUSTC MIRI
   rm -rf .cargo
@@ -100,17 +102,18 @@ function run_tests {
 }
 
 function run_tests_minimal {
-  if [ -n "${MIRI_TEST_TARGET-}" ]; then
-    begingroup "Testing MINIMAL foreign architecture $MIRI_TEST_TARGET: only testing $@"
+  if [ -n "${TEST_TARGET-}" ]; then
+    begingroup "Testing MINIMAL foreign architecture $TEST_TARGET: only testing $@"
+    TARGET_FLAG="--target $TEST_TARGET"
   else
-    echo "run_tests_minimal requires MIRI_TEST_TARGET to be set"
+    echo "run_tests_minimal requires TEST_TARGET to be set"
     exit 1
   fi
 
-  time ./miri test "$@"
+  time ./miri test $TARGET_FLAG "$@"
 
   # Ensure that a small smoke test of cargo-miri works.
-  time cargo miri run --manifest-path test-cargo-miri/no-std-smoke/Cargo.toml --target ${MIRI_TEST_TARGET-$HOST_TARGET}
+  time cargo miri run --manifest-path test-cargo-miri/no-std-smoke/Cargo.toml $TARGET_FLAG
 
   endgroup
 }
@@ -126,33 +129,33 @@ case $HOST_TARGET in
     # Extra tier 1
     # With reduced many-seed count to avoid spending too much time on that.
     # (All OSes and ABIs are run with 64 seeds at least once though via the macOS runner.)
-    MANY_SEEDS=16 MIRI_TEST_TARGET=i686-unknown-linux-gnu run_tests
-    MANY_SEEDS=16 MIRI_TEST_TARGET=aarch64-unknown-linux-gnu run_tests
-    MANY_SEEDS=16 MIRI_TEST_TARGET=x86_64-apple-darwin run_tests
-    MANY_SEEDS=16 MIRI_TEST_TARGET=x86_64-pc-windows-gnu run_tests
+    MANY_SEEDS=16 TEST_TARGET=i686-unknown-linux-gnu run_tests
+    MANY_SEEDS=16 TEST_TARGET=aarch64-unknown-linux-gnu run_tests
+    MANY_SEEDS=16 TEST_TARGET=x86_64-apple-darwin run_tests
+    MANY_SEEDS=16 TEST_TARGET=x86_64-pc-windows-gnu run_tests
     ;;
   aarch64-apple-darwin)
     # Host (tier 2)
     GC_STRESS=1 MIR_OPT=1 MANY_SEEDS=64 TEST_BENCH=1 CARGO_MIRI_ENV=1 run_tests
     # Extra tier 1
-    MANY_SEEDS=64 MIRI_TEST_TARGET=i686-pc-windows-gnu run_tests
-    MANY_SEEDS=64 MIRI_TEST_TARGET=x86_64-pc-windows-msvc CARGO_MIRI_ENV=1 run_tests
+    MANY_SEEDS=64 TEST_TARGET=i686-pc-windows-gnu run_tests
+    MANY_SEEDS=64 TEST_TARGET=x86_64-pc-windows-msvc CARGO_MIRI_ENV=1 run_tests
     # Extra tier 2
-    MIRI_TEST_TARGET=arm-unknown-linux-gnueabi run_tests
-    MIRI_TEST_TARGET=s390x-unknown-linux-gnu run_tests # big-endian architecture of choice
+    TEST_TARGET=arm-unknown-linux-gnueabi run_tests
+    TEST_TARGET=s390x-unknown-linux-gnu run_tests # big-endian architecture of choice
     # Partially supported targets (tier 2)
     VERY_BASIC="integer vec string btreemap" # common things we test on all of them (if they have std), requires no target-specific shims
     BASIC="$VERY_BASIC hello hashmap alloc align" # ensures we have the shims for stdout and basic data structures
-    MIRI_TEST_TARGET=x86_64-unknown-freebsd run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
-    MIRI_TEST_TARGET=i686-unknown-freebsd   run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
-    MIRI_TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
-    MIRI_TEST_TARGET=x86_64-pc-solaris run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
-    MIRI_TEST_TARGET=aarch64-linux-android  run_tests_minimal $VERY_BASIC hello panic/panic
-    MIRI_TEST_TARGET=wasm32-wasi run_tests_minimal $VERY_BASIC wasm
-    MIRI_TEST_TARGET=wasm32-unknown-unknown run_tests_minimal $VERY_BASIC wasm
-    MIRI_TEST_TARGET=thumbv7em-none-eabihf run_tests_minimal no_std
+    TEST_TARGET=x86_64-unknown-freebsd run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
+    TEST_TARGET=i686-unknown-freebsd   run_tests_minimal $BASIC panic/panic concurrency/simple atomic threadname libc-mem libc-misc libc-random libc-time fs env num_cpus
+    TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
+    TEST_TARGET=x86_64-pc-solaris run_tests_minimal $VERY_BASIC hello panic/panic concurrency/simple pthread-sync libc-mem libc-misc libc-random
+    TEST_TARGET=aarch64-linux-android  run_tests_minimal $VERY_BASIC hello panic/panic
+    TEST_TARGET=wasm32-wasi run_tests_minimal $VERY_BASIC wasm
+    TEST_TARGET=wasm32-unknown-unknown run_tests_minimal $VERY_BASIC wasm
+    TEST_TARGET=thumbv7em-none-eabihf run_tests_minimal no_std
     # Custom target JSON file
-    MIRI_TEST_TARGET=tests/avr.json MIRI_NO_STD=1 run_tests_minimal no_std
+    TEST_TARGET=tests/avr.json MIRI_NO_STD=1 run_tests_minimal no_std
     ;;
   i686-pc-windows-msvc)
     # Host
@@ -162,7 +165,7 @@ case $HOST_TARGET in
     # Extra tier 1
     # We really want to ensure a Linux target works on a Windows host,
     # and a 64bit target works on a 32bit host.
-    MIRI_TEST_TARGET=x86_64-unknown-linux-gnu run_tests
+    TEST_TARGET=x86_64-unknown-linux-gnu run_tests
     ;;
   *)
     echo "FATAL: unknown host target: $HOST_TARGET"