Skip to content

Conversation

@jack-morrison
Copy link
Member

@jack-morrison jack-morrison commented Jul 14, 2025

Prior to change

A test without reference values prints (r:0, l:None, u:None) for each performance variable. This is #3175.

user@myhost:~/reframe-examples/tutorial$ reframe -c stream/stream_runonly.py -r
<snip>

[----------] start processing checks
[ RUN      ] stream_test /2e15a047 @generic:default+builtin
[       OK ] (1/1) stream_test /2e15a047 @generic:default+builtin
P: copy_bw: 136250.6 MB/s (r:0, l:None, u:None)
P: triad_bw: 127052.0 MB/s (r:0, l:None, u:None)
[----------] all spawned checks have finished

[  PASSED  ] Ran 1/1 test case(s) from 1 check(s) (0 failure(s), 0 expected failure(s), 0 skipped, 0 aborted)
[==========] Finished on Mon Jul 14 19:19:13 2025+0000
Log file(s) saved in '/tmp/rfm-wkcpor0o.log'

After change

A test without reference values does not print the reference tuple. Only the captured performance metric is printed.

user@myhost:~/reframe-examples/tutorial$ reframe -c stream/stream_runonly.py -r
<snip>

[----------] start processing checks
[ RUN      ] stream_test /2e15a047 @generic:default+builtin
[       OK ] (1/1) stream_test /2e15a047 @generic:default+builtin
P: copy_bw: 124658.9 MB/s
P: triad_bw: 99002.0 MB/s
[----------] all spawned checks have finished

[  PASSED  ] Ran 1/1 test case(s) from 1 check(s) (0 failure(s), 0 expected failure(s), 0 skipped, 0 aborted)
[==========] Finished on Mon Jul 14 19:13:53 2025+0000
Log file(s) saved in '/tmp/rfm-_jygclgm.log'

A test with reference values behaves as before. The captured performance metric and its references are both printed.

user@myhost:~/reframe-examples/tutorial$ reframe -c stream/stream_runonly.py -r
<snip>

[----------] start processing checks
[ RUN      ] stream_test /2e15a047 @generic:default+builtin
[       OK ] (1/1) stream_test /2e15a047 @generic:default+builtin
P: copy_bw: 133880.3 MB/s (r:137659, l:-0.1, u:0.3)
P: triad_bw: 129926.7 MB/s (r:137659, l:-0.55, u:0.5)
[----------] all spawned checks have finished

[  PASSED  ] Ran 1/1 test case(s) from 1 check(s) (0 failure(s), 0 expected failure(s), 0 skipped, 0 aborted)
[==========] Finished on Mon Jul 14 19:14:31 2025+0000
Log file(s) saved in '/tmp/rfm-2md5inu3.log'

Closes #3175

@jack-morrison jack-morrison self-assigned this Jul 14, 2025
@codecov
Copy link

codecov bot commented Jul 14, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.26%. Comparing base (d711e28) to head (b34fab8).
⚠️ Report is 3 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #3517   +/-   ##
========================================
  Coverage    91.26%   91.26%           
========================================
  Files           62       62           
  Lines        13581    13583    +2     
========================================
+ Hits         12395    12397    +2     
  Misses        1186     1186           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jack-morrison jack-morrison changed the title [feat] Only print performance variable reference values if defined Only print performance variable reference values if defined Jul 14, 2025
@jack-morrison jack-morrison marked this pull request as ready for review July 16, 2025 14:35
@jack-morrison jack-morrison requested a review from vkarak July 16, 2025 17:06
@vkarak vkarak changed the title Only print performance variable reference values if defined [enhancement] Only print performance variable reference values if defined Jul 17, 2025
@vkarak vkarak modified the milestones: ReFrame 4.8.3, ReFrame 4.9 Jul 17, 2025
@github-project-automation github-project-automation bot moved this from Todo to In Progress in ReFrame Backlog Jul 18, 2025
jack-morrison and others added 2 commits November 11, 2025 21:43
…re all defined

Signed-off-by: Jack Morrison <jack.morrison@cornelisnetworks.com>
@vkarak vkarak force-pushed the jackm/ref-if-declared branch from ba9a711 to b34fab8 Compare November 11, 2025 21:07
@vkarak vkarak changed the title [enhancement] Only print performance variable reference values if defined [enhancement] Print the performance reference tuple while running only if defined Nov 11, 2025
@vkarak vkarak enabled auto-merge November 11, 2025 21:10
@vkarak vkarak merged commit a54740c into reframe-hpc:develop Nov 11, 2025
67 of 71 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in ReFrame Backlog Nov 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Only show reference values if they're defined in a test

2 participants