Skip to content

Commit 54551f5

Browse files
richardelmsrichardelms
authored
add tests to confirm system metrics are present in app start and scene load spans (#211)
1 parent 6ba47e5 commit 54551f5

File tree

9 files changed

+361
-34
lines changed

9 files changed

+361
-34
lines changed

‎features/app_start_spans.feature‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,3 +73,74 @@ Scenario: App Start Clear Customisation
7373
* the trace "Bugsnag-Span-Sampling" header equals "1:4"
7474
* the trace payload field "resourceSpans.0.scopeSpans.0.spans.3.name" equals "[AppStart/UnityRuntime]"
7575

76+
@android_only
77+
Scenario: App Start with metrics
78+
When I run the game in the "AppStartWithMetrics" state
79+
80+
And I wait to receive a span named "[AppStart/UnityRuntime]"
81+
And I wait to receive a span named "[AppStartPhase/LoadAssemblies]"
82+
And I wait to receive a span named "[AppStartPhase/SplashScreen]"
83+
And I wait to receive a span named "[AppStartPhase/LoadFirstScene]"
84+
85+
# rendering metrics on the app-start span
86+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.total_frames" greater than 0
87+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.fps_average" greater than 0
88+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.fps_maximum" greater than 0
89+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.fps_minimum" greater than 0
90+
91+
# memory metrics on the app-start span (arrayed samples)
92+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.memory.timestamps" with at least 5 elements
93+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0
94+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.system.memory.spaces.device.size" greater than 0
95+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.memory.spaces.device.used" with at least 5 elements
96+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.system.memory.spaces.art.size" greater than 0
97+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.memory.spaces.art.used" with at least 5 elements
98+
99+
# CPU metrics on the app-start span
100+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.cpu_measures_timestamps" with at least 5 elements
101+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.cpu_measures_total" with at least 5 elements
102+
* the span named "[AppStart/UnityRuntime]" has a double array attribute "bugsnag.system.cpu_measures_total" containing valid percentages
103+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.cpu_measures_main_thread" with at least 5 elements
104+
* the span named "[AppStart/UnityRuntime]" has a double array attribute "bugsnag.system.cpu_measures_main_thread" containing valid percentages
105+
* the span named "[AppStart/UnityRuntime]" has a double attribute "bugsnag.system.cpu_mean_total" that is a valid percentage
106+
* the span named "[AppStart/UnityRuntime]" has a double attribute "bugsnag.system.cpu_mean_main_thread" that is a valid percentage
107+
108+
# phase spans should also carry the memory metrics
109+
* the span named "[AppStartPhase/LoadAssemblies]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0
110+
* the span named "[AppStartPhase/SplashScreen]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0
111+
* the span named "[AppStartPhase/LoadFirstScene]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0
112+
113+
@ios_only
114+
Scenario: App Start with metrics
115+
When I run the game in the "AppStartWithMetrics" state
116+
117+
And I wait to receive a span named "[AppStart/UnityRuntime]"
118+
And I wait to receive a span named "[AppStartPhase/LoadAssemblies]"
119+
And I wait to receive a span named "[AppStartPhase/SplashScreen]"
120+
And I wait to receive a span named "[AppStartPhase/LoadFirstScene]"
121+
122+
# rendering metrics on the app-start span
123+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.total_frames" greater than 0
124+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.fps_average" greater than 0
125+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.fps_maximum" greater than 0
126+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.rendering.fps_minimum" greater than 0
127+
128+
# memory metrics on the app-start span (arrayed samples)
129+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.memory.timestamps" with at least 5 elements
130+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0
131+
* the span named "[AppStart/UnityRuntime]" has an integer attribute "bugsnag.system.memory.spaces.device.size" greater than 0
132+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.memory.spaces.device.used" with at least 5 elements
133+
134+
# CPU metrics on the app-start span
135+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.cpu_measures_timestamps" with at least 5 elements
136+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.cpu_measures_total" with at least 5 elements
137+
* the span named "[AppStart/UnityRuntime]" has a double array attribute "bugsnag.system.cpu_measures_total" containing valid percentages
138+
* the span named "[AppStart/UnityRuntime]" has an array attribute "bugsnag.system.cpu_measures_main_thread" with at least 5 elements
139+
* the span named "[AppStart/UnityRuntime]" has a double array attribute "bugsnag.system.cpu_measures_main_thread" containing valid percentages
140+
* the span named "[AppStart/UnityRuntime]" has a double attribute "bugsnag.system.cpu_mean_total" that is a valid percentage
141+
* the span named "[AppStart/UnityRuntime]" has a double attribute "bugsnag.system.cpu_mean_main_thread" that is a valid percentage
142+
143+
# phase spans should also carry the memory metrics
144+
* the span named "[AppStartPhase/LoadAssemblies]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0
145+
* the span named "[AppStartPhase/SplashScreen]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0
146+
* the span named "[AppStartPhase/LoadFirstScene]" has an integer attribute "bugsnag.device.physical_device_memory" greater than 0

0 commit comments

Comments
 (0)