rustbuild: Both stage0 and stage1 std are rebuilt spuriously

[petrochenkov]

I'm not sure when exactly it happens, but it reproduces roughly like this:

• Rebase my working branch on upstream/master.
• Run ./x.py test --stage 1 src/test/ui --test-args mytest, the build starts from stage0 std or later and succeeds.
• Run ./x.py test --stage 1 src/test/ui --test-args mytest again, should be a trivial build, but the build starts from stage0 std or stage1 std again and succeeds.
• Run ./x.py test --stage 1 src/test/ui --test-args mytest again, trivial build happens.

The target is x86_64-pc-windows-gnu.
The issue exists for several weeks, so it's a regression.
cc @Mark-Simulacrum

[petrochenkov]

Reproduction script:

./x.py test --stage 1 src/test/ui --test-args mytest
rm -rf build/x86_64-pc-windows-gnu/{stage1*,stage0-sysroot}
./x.py test --stage 1 src/test/ui --test-args mytest
./x.py test --stage 1 src/test/ui --test-args mytest -vvvvvv

You might have to run it multiple times to cause the spurious rebuild in the last command.

[petrochenkov]

For some reason backtrace v0.3.40 is getting recompiled:

Building stage1 std artifacts (x86_64-pc-windows-gnu -> x86_64-pc-windows-gnu)
running: "C:\\msys64\\home\\we\\rust\\build\\x86_64-pc-windows-gnu\\stage0\\bin\\cargo.exe" "build" "-Zconfig-profile" "--target" "x86_64-pc-windows-gnu" "-Zbinary-dep-depinfo" "-j" "8" "-v" "-v" "-v" "-v" "-v" "--release" "--features" "panic-unwind backtrace compiler-builtins-c" "--manifest-path" "C:\\msys64\\home\\we\\rust\\src/libtest/Cargo.toml" "--message-format" "json-render-diagnostics"
       Fresh cc v1.0.49
       Fresh core v0.0.0 (C:\msys64\home\we\rust\src\libcore)
       Fresh autocfg v0.1.6
       Fresh rustc-std-workspace-core v1.99.0 (C:\msys64\home\we\rust\src\tools\rustc-std-workspace-core)
       Fresh compiler_builtins v0.1.22
       Fresh libc v0.2.66
       Fresh cfg-if v0.1.8
       Fresh alloc v0.0.0 (C:\msys64\home\we\rust\src\liballoc)
       Fresh backtrace-sys v0.1.32
       Fresh rustc-demangle v0.1.16
       Fresh panic_abort v0.0.0 (C:\msys64\home\we\rust\src\libpanic_abort)
       Fresh rustc-std-workspace-alloc v1.99.0 (C:\msys64\home\we\rust\src\tools\rustc-std-workspace-alloc)
       Fresh unwind v0.0.0 (C:\msys64\home\we\rust\src\libunwind)
   Compiling backtrace v0.3.40
       Fresh hashbrown v0.6.2
       Fresh panic_unwind v0.0.0 (C:\msys64\home\we\rust\src\libpanic_unwind)

[petrochenkov]

cc @alexcrichton

[Mark-Simulacrum]

I think @eddyb was hitting this at some point as well, though we weren't able to determine quite why.

[petrochenkov]

I haven't encountered these rebuilds before, but recently (since December?) they happen quite often, like every second or third build, and that's pretty inconvenient given the long builds.

Perhaps something changed in backtrace itself?

[Mark-Simulacrum]

I don't think we've changed backtrace itself recently.

It would perhaps be handy to check if this still happens before 87540bd (which changed how/when we clear out std's target directory). However, I don't expect it would have, as that would cause all of std to be rebuilt, not just backtrace. Maybe backtrace is unique in being a dependency of both std and the compiler (for example?). I'm not sure what could be causing this odd behavior :/

[petrochenkov]

I'm pretty sure the rebuilds appeared before 87540bd (and I hoped that PR would fix them, but it didn't).

[petrochenkov]

I suspect #67077 to be the cause, at least based on timing.

[mati865]

I've been hitting similar (this?) issue with x86_64-pc-windows-gnu when doing dist builds. Somehow it disappeared whenever I ran build before dist so I kept doing that as the workaround.

[Mark-Simulacrum]

Hm, so #67077 in theory at least wouldn't have affected backtrace specifically, I think, though there may have been knock on effects.

One thing that might be helpful is getting a log with CARGO_LOG=cargo::core::compiler::fingerprint=info on the spurious rebuild (to know why Cargo thinks it needs to rebuild it).

[petrochenkov]

Update: this reproduces on ./x.py build --stage 0 src/libstd too (which runs much quicker and involves less moving parts), just got a backtrace rebuild after a clean build this way.

I'll try CARGO_LOG=cargo::core::compiler::fingerprint=info.