CI - Parallelize smoketests #3702
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…are-python-deps' into bfops/fix-list-tests
…are-python-deps' into bfops/fix-list-tests
…are-python-deps' into bfops/fix-list-tests
bfops
force-pushed
the
bfops/fix-list-tests
branch
from
a8fee32 to
3f5e32a
Compare
…ter' into bfops/parallel-smoketests
…/cargo-ci' into bfops/parallel-smoketests
… of github.com:clockworklabs/SpacetimeDB into bfops/parallel-smoketests
bfops
commented
Nov 21, 2025
tools/ci/src/main.rs
Outdated
| } | ||
| let test_result = bash!(&format!("{python} -m smoketests {}", smoketests_args.join(" "))); | ||
|
|
||
| // TODO: Make an effort to run the wind-down behavior if we ctrl-C this process |
bfops
commented
Nov 21, 2025
tools/ci/src/main.rs
Outdated
| (false, None) => {} | ||
| } | ||
|
|
||
| // TODO: does this work on windows? |
bfops
commented
Nov 21, 2025
tools/ci/src/main.rs
Outdated
| } | ||
| }; | ||
|
|
||
| // TODO: does this work on windows? |
bfops
commented
Nov 21, 2025
| println!("Running smoketests.."); | ||
| let test_result = bash!(&format!("{python} -m smoketests {}", args.join(" "))); | ||
|
|
||
| // TODO: Make an effort to run the wind-down behavior if we ctrl-C this process |
bfops
commented
Nov 21, 2025
| // before the server is up. | ||
| bash!("cargo build -p spacetimedb-cli -p spacetimedb-standalone")?; | ||
|
|
||
| // TODO: Make sure that this isn't brittle / multiple parallel batches don't grab the same port |
…lready building it
bfops
commented
Nov 25, 2025
| batch.clone(), | ||
| std::thread::spawn(move || { | ||
| println!("Running smoketests batch {batch}.."); | ||
| // TODO: capture output and print it only in contiguous blocks |
bfops
commented
Nov 25, 2025
| let start_server_clone = start_server.clone(); | ||
| let python_clone = python.clone(); | ||
| let mut batch_args: Vec<String> = Vec::new(); | ||
| // TODO: this doesn't work properly if the user passed multiple batches as input. |
bfops
commented
Nov 25, 2025
| list_args.push("--list=json".to_string()); | ||
| let list_cmdline = format!("{python} -m smoketests {}", list_args.join(" ")); | ||
|
|
||
| // TODO: do actually check the return code here. and make --list=json not return non-zero if there are errors. |
bfops
commented
Nov 25, 2025
|
|
||
| // Pre-build so that `cargo run -p spacetimedb-cli` will immediately start. Otherwise we risk starting the tests | ||
| // before the server is up. | ||
| // TODO: The `cargo run` invocation still seems to rebuild a bunch? investigate.. maybe we infer the binary path from cargo metadata. |
Collaborator
Author
There was a problem hiding this comment.
fix this TODO (I think it has been resolved)
bfops
commented
Nov 25, 2025
| let python = if let Some(p) = python { | ||
| p | ||
| } else { | ||
| // TODO: does this work on windows? |
bfops
commented
Nov 25, 2025
| pid | ||
| )); | ||
| } else { | ||
| // TODO: I keep getting errors about the pid not existing.. but the servers seem to shut down? |
Collaborator
Author
There was a problem hiding this comment.
resolve this TODO
I think it's because we're not specifying new --data-dirs so they conflict with one another
bfops
commented
Nov 25, 2025
| }; | ||
|
|
||
| println!("Running smoketests.."); | ||
| // TODO: Don't we need to _use_ the port here?! |
Collaborator
Author
There was a problem hiding this comment.
resolve this TODO (this is done)
bfops
commented
Nov 25, 2025
| .read() | ||
| .unwrap_or_default(); | ||
| } else { | ||
| // TODO: Maybe we do this in a thread instead? Then it's easier to kill |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Changes
API and ABI breaking changes
Expected complexity level and risk
Testing