Skip to content

Flaky tests #826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 22, 2025
Merged

Flaky tests #826

merged 4 commits into from
Oct 22, 2025

Conversation

@pblazej
Copy link
Contributor

@pblazej pblazej commented Oct 22, 2025 •
edited
Loading

  • Disable parallel testing (premature optimization?)
  • Increase timeouts accordingly

@github-actions
Copy link

github-actions bot commented Oct 22, 2025 •
edited
Loading

⚠️ This PR does not contain any files in the .changes directory.

@pblazej
Copy link
Contributor Author

pblazej commented Oct 22, 2025

@hiroshihorie IMO the biggest source of flakiness is parallel testing vs cli.

I don't think it's worth the effort to have multiple server clones vs multiple (deterministic) sims.

Remaining errors (during retry):

testRecord, failed: caught error: "Error Domain=NSOSStatusErrorDomain Code=2003334207 "(null)""
testResourcesCleanUp, XCTAssertNil failed: "<LiveKit.WebSocket: 0x110581720>" - Leaked object: Optional(<LiveKit.WebSocket: 0x110581720>)
testReliableRetry, Asynchronous wait failed: Exceeded timeout of 10 seconds, with unfulfilled expectations: "Data received".
testWaitUntilActiveTimeout, failed: caught error: "Error Domain=io.livekit.swift-sdk Code=202 "Network error(Validation response: "success")"

This should help for the last one: #782

@pblazej pblazej marked this pull request as ready for review October 22, 2025 08:45
@pblazej pblazej requested a review from hiroshihorie October 22, 2025 08:46
@pblazej
Copy link
Contributor Author

pblazej commented Oct 22, 2025

Audio error - totally unspecific https://www.osstatus.com/search/results?platform=all&framework=all&search=2003334207

@pblazej pblazej merged commit 263048b into main Oct 22, 2025
27 of 28 checks passed
@pblazej pblazej deleted the blaze/flaky-tests branch October 22, 2025 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hiroshihorie hiroshihorie hiroshihorie approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pblazej @hiroshihorie