Skip to content

Commit 65a0e35

Browse files
committed
Add an alternatives considered section on having passesOnce continue to evaluate the expression after it passes
1 parent 59e71ff commit 65a0e35

File tree

1 file changed

+35
-2
lines changed

1 file changed

+35
-2
lines changed

proposals/testing/NNNN-polling-expectations.md

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ Task {
6262
await subject.raiseDolphins()
6363
}
6464
await #expect(until: .passesOnce) {
65-
subject.dolphins.count() == 1
65+
await subject.dolphins.count == 1
6666
}
6767
```
6868

@@ -212,7 +212,7 @@ These macros can be used with an async test function:
212212
await subject.raiseDolphins()
213213
}
214214
await #expect(until: .passesOnce) {
215-
subject.dolphins.count() == 1
215+
await subject.dolphins.count == 1
216216
}
217217
}
218218
```
@@ -266,11 +266,44 @@ it could be configured in some future global configuration tool.
266266

267267
## Alternatives considered
268268

269+
### Remove `PollingBehavior` in favor of more macros
270+
269271
Instead of creating the `PollingBehavior` type, we could have introduced more
270272
macros to cover that situation: `#expect(until:)` and `#expect(always:)`.
271273
However, this would have resulted in confusion for the compiler and test authors
272274
when trailing closure syntax is used.
273275

276+
### `PollingBehavior.passesOnce` continues to evaluate expression after passing
277+
278+
Under `PollingBehavior.passesOnce`, we thought about requiring the expression
279+
to continue to pass after it starts passing. The idea is to prevent test
280+
flakiness caused by an expectation that initially passes, but stops passing as
281+
a result of (intended) background activity. For example:
282+
283+
```swift
284+
@Test func `The aquarium's dolphin nursery works`() async {
285+
let subject = Aquarium()
286+
await subject.raiseDolphins()
287+
Task {
288+
await subject.raiseDolphins()
289+
}
290+
await #expect(until: .passesOnce) {
291+
await subject.dolphins.count == 1
292+
}
293+
}
294+
```
295+
296+
This test is flaky, but will pass more often than not. However, it is still
297+
incorrect. If we were to change `PollingBehavior.passesOnce` to instead check
298+
that the expression continues to pass after the first time it succeeds until the
299+
timeout is reached, then this test would correctly be flagged as failing each
300+
time it's ran.
301+
302+
We chose to address this by using the name `passesOnce` instead of changing the
303+
behavior. `passesOnce` makes it clear the exact behavior that will happen: the
304+
expression will be evaluated until the first time it passes, and no more. We
305+
hope that this will help test authors to better recognize these situations.
306+
274307
## Acknowledgments
275308

276309
This proposal is heavily inspired by Nimble's [Polling Expectations](https://quick.github.io/Nimble/documentation/nimble/pollingexpectations/).

0 commit comments

Comments
 (0)