Skip to content

Commit d629a00

Browse files
committed
[utils] add update-verify-tests.py from clang
This script is used to automatically update test cases using -verify in clang's test suite. Swift also has a similar -verify test functionality, and while it can automatically fix some of them using -verify-apply-all, this functionality only updates existing checks - it doesn't add or remove any, and it doesn't handle newer and more complex things like expected-expansion. Handling that type of complexity feels out of scope to embed in the compiler, so let's copy clang's approach. This commit adds this script as is from clang. It doesn't work at all for Swift in its current form, as the output from Swift's -verify is formatted differently than in clang. This will be fixed in subsequent commits. This could have been done by adapting the script as-is in the llvm-project repository (since it only exists in the Swift fork, not upstream), but tests using swift-frontend would have to reside in the swift repo, and modifying a script in one repo with tests in a different repo sounds like a recipe for endless CI issues.
1 parent b1c2bc1 commit d629a00

File tree

2 files changed

+492
-0
lines changed

2 files changed

+492
-0
lines changed

utils/update-verify-tests.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
import sys
2+
import argparse
3+
from UpdateVerifyTests.core import check_expectations
4+
5+
"""
6+
Pipe output from clang's -verify into this script to have the test case updated to expect the actual diagnostic output.
7+
When inserting new expected-* checks it will place them on the line before the location of the diagnostic, with an @+1,
8+
or @+N for some N if there are multiple diagnostics emitted on the same line. If the current checks are using @-N for
9+
this line, the new check will follow that convention also.
10+
Existing checks will be left untouched as much as possible, including their location and whitespace content, to minimize
11+
diffs. If inaccurate their count will be updated, or the check removed entirely.
12+
13+
Missing features:
14+
- multiple prefixes on the same line (-verify=my-prefix,my-other-prefix)
15+
- multiple prefixes on separate RUN lines (RUN: -verify=my-prefix\nRUN: -verify my-other-prefix)
16+
- regexes with expected-*-re: existing ones will be left untouched if accurate, but the script will abort if there are any
17+
diagnostic mismatches on the same line.
18+
- multiple checks targeting the same line are supported, but a line may only contain one check
19+
- if multiple checks targeting the same line are failing the script is not guaranteed to produce a minimal diff
20+
21+
Example usage:
22+
clang -verify [file] | python3 update-verify-tests.py
23+
clang -verify=check [file] | python3 update-verify-tests.py --prefix check
24+
"""
25+
26+
27+
def main():
28+
parser = argparse.ArgumentParser(description=__doc__)
29+
parser.add_argument(
30+
"--prefix", default="expected", help="The prefix passed to -verify"
31+
)
32+
args = parser.parse_args()
33+
output = check_expectations(sys.stdin.readlines(), args.prefix)
34+
print(output)
35+
36+
37+
if __name__ == "__main__":
38+
main()
39+

0 commit comments

Comments
 (0)