|
| 1 | +# UnitTests Performance Analysis Report |
| 2 | + |
| 3 | +## Executive Summary |
| 4 | + |
| 5 | +This report provides a comprehensive performance analysis of the `UnitTests` project, identifying the highest-impact opportunities for test migration to improve CI/CD performance. |
| 6 | + |
| 7 | +**Key Findings:** |
| 8 | +- **Total tests analyzed:** 3,260 tests across 121 test files |
| 9 | +- **Top bottleneck:** Views folder (962 tests, 59.6s, 50% of total runtime) |
| 10 | +- **Highest average time per test:** Input/ folder (0.515s/test) |
| 11 | +- **Tests with AutoInitShutdown:** 449 tests (35.4%) - these are the slowest |
| 12 | +- **Tests with SetupFakeDriver:** 198 tests (15.6%) |
| 13 | +- **Tests with no attributes:** 622 tests (49.0%) - easiest to migrate |
| 14 | + |
| 15 | +## Performance Analysis by Folder |
| 16 | + |
| 17 | +### Folder-Level Timing Results (Ranked by Total Duration) |
| 18 | + |
| 19 | +| Folder | Tests | Duration | Avg/Test | Impact Score | |
| 20 | +|--------|-------|----------|----------|--------------| |
| 21 | +| **Views/** | 962 | 59.64s | 0.062s | ⭐⭐⭐⭐⭐ CRITICAL | |
| 22 | +| **View/** | 739 | 27.14s | 0.036s | ⭐⭐⭐⭐ HIGH | |
| 23 | +| **Application/** | 187 | 14.82s | 0.079s | ⭐⭐⭐ MEDIUM | |
| 24 | +| **Dialogs/** | 116 | 13.42s | 0.115s | ⭐⭐⭐ MEDIUM | |
| 25 | +| **Text/** | 467 | 10.18s | 0.021s | ⭐⭐ LOW | |
| 26 | +| **ConsoleDrivers/** | 475 | 5.74s | 0.012s | ⭐ VERY LOW | |
| 27 | +| **FileServices/** | 35 | 5.56s | 0.158s | ⭐⭐ LOW | |
| 28 | +| **Drawing/** | 173 | 5.35s | 0.030s | ⭐ VERY LOW | |
| 29 | +| **Configuration/** | 98 | 5.05s | 0.051s | ⭐ VERY LOW | |
| 30 | +| **Input/** | 8 | 4.12s | 0.515s | ⭐⭐ LOW | |
| 31 | + |
| 32 | +**Total:** 3,260 tests, ~150s total runtime |
| 33 | + |
| 34 | +### Folder-Level Static Analysis |
| 35 | + |
| 36 | +| Folder | Files | Tests | AutoInit | SetupDrv | App.Begin | App.Top | |
| 37 | +|--------|-------|-------|----------|----------|-----------|---------| |
| 38 | +| Views | 33 | 612 | 232 (37.9%) | 104 (17.0%) | 139 | 219 | |
| 39 | +| Application | 12 | 120 | 27 (22.5%) | 6 (5.0%) | 20 | 145 | |
| 40 | +| Configuration | 9 | 82 | 0 (0.0%) | 0 (0.0%) | 0 | 0 | |
| 41 | +| ConsoleDrivers | 17 | 75 | 15 (20.0%) | 3 (4.0%) | 8 | 34 | |
| 42 | +| Drawing | 4 | 58 | 21 (36.2%) | 32 (55.2%) | 1 | 0 | |
| 43 | +| Dialogs | 3 | 50 | 40 (80.0%) | 0 (0.0%) | 6 | 7 | |
| 44 | +| View/Draw | 7 | 37 | 15 (40.5%) | 17 (45.9%) | 15 | 0 | |
| 45 | + |
| 46 | +## High-Impact Migration Targets |
| 47 | + |
| 48 | +### 🎯 Priority 1: CRITICAL Impact (50-60s potential savings) |
| 49 | + |
| 50 | +#### Views/ Folder - 59.6s (50% of total runtime) |
| 51 | +**Profile:** |
| 52 | +- 962 tests total |
| 53 | +- 232 with AutoInitShutdown (37.9%) |
| 54 | +- 104 with SetupFakeDriver (17.0%) |
| 55 | +- **~380 tests with no attributes** (potential quick wins) |
| 56 | + |
| 57 | +**Top Individual Files:** |
| 58 | +1. **TextViewTests.cs** - 105 tests, 9.26s, 0.088s/test |
| 59 | + - 41 AutoInitShutdown (39%) |
| 60 | + - 64 tests are potentially migratable |
| 61 | + |
| 62 | +2. **TableViewTests.cs** - 80 tests, 5.38s, 0.055s/test |
| 63 | + - 45 SetupFakeDriver (56%) |
| 64 | + - 8 AutoInitShutdown |
| 65 | + - Many rendering tests that may need refactoring |
| 66 | + |
| 67 | +3. **TileViewTests.cs** - 45 tests, 9.25s, 0.197s/test ⚠️ SLOWEST AVG |
| 68 | + - 42 AutoInitShutdown (93%) |
| 69 | + - High overhead per test - prime candidate for optimization |
| 70 | + |
| 71 | +4. **TextFieldTests.cs** - 43 tests |
| 72 | + - 8 AutoInitShutdown (19%) |
| 73 | + - 3 SetupFakeDriver |
| 74 | + - ~32 tests likely migratable |
| 75 | + |
| 76 | +5. **GraphViewTests.cs** - 42 tests |
| 77 | + - 24 AutoInitShutdown (57%) |
| 78 | + - ~18 tests potentially migratable |
| 79 | + |
| 80 | +**Recommendation:** Focus on Views/ folder first |
| 81 | +- Extract simple property/event tests from TextViewTests |
| 82 | +- Refactor TileViewTests to reduce AutoInitShutdown usage |
| 83 | +- Split TableViewTests into unit vs integration tests |
| 84 | + |
| 85 | +### 🎯 Priority 2: HIGH Impact (20-30s potential savings) |
| 86 | + |
| 87 | +#### View/ Folder - 27.14s |
| 88 | +**Profile:** |
| 89 | +- 739 tests total |
| 90 | +- Wide distribution across subdirectories |
| 91 | +- Mix of layout, drawing, and behavioral tests |
| 92 | + |
| 93 | +**Key subdirectories:** |
| 94 | +- View/Layout - 35 tests (6 AutoInit, 1 SetupDriver) |
| 95 | +- View/Draw - 37 tests (15 AutoInit, 17 SetupDriver) |
| 96 | +- View/Adornment - 25 tests (9 AutoInit, 10 SetupDriver) |
| 97 | + |
| 98 | +**Top Files:** |
| 99 | +1. **GetViewsUnderLocationTests.cs** - 21 tests, NO attributes ✅ |
| 100 | + - Easy migration candidate |
| 101 | + |
| 102 | +2. **DrawTests.cs** - 17 tests |
| 103 | + - 10 AutoInitShutdown |
| 104 | + - 6 SetupFakeDriver |
| 105 | + - Mix that needs analysis |
| 106 | + |
| 107 | +**Recommendation:** |
| 108 | +- Migrate GetViewsUnderLocationTests.cs immediately |
| 109 | +- Analyze layout tests for unnecessary Application dependencies |
| 110 | + |
| 111 | +### 🎯 Priority 3: MEDIUM Impact (10-15s potential savings) |
| 112 | + |
| 113 | +#### Dialogs/ Folder - 13.42s |
| 114 | +**Profile:** |
| 115 | +- 116 tests, 0.115s/test average (SLOW) |
| 116 | +- 40 AutoInitShutdown (80% usage rate!) |
| 117 | +- Heavy Application.Begin usage |
| 118 | + |
| 119 | +**Files:** |
| 120 | +1. **DialogTests.cs** - 23 tests, all with AutoInitShutdown |
| 121 | +2. **MessageBoxTests.cs** - 11 tests, all with AutoInitShutdown |
| 122 | + |
| 123 | +**Recommendation:** |
| 124 | +- These are true integration tests that likely need Application |
| 125 | +- Some could be refactored to test dialog construction separately from display |
| 126 | +- Lower priority for migration |
| 127 | + |
| 128 | +#### Application/ Folder - 14.82s |
| 129 | +**Profile:** |
| 130 | +- 187 tests |
| 131 | +- 27 AutoInitShutdown (22.5%) |
| 132 | +- Heavy Application.Top usage (145 occurrences) |
| 133 | + |
| 134 | +**Easy wins:** |
| 135 | +1. **MainLoopTests.cs** - 23 tests, NO attributes ✅ (already migrated) |
| 136 | +2. **ApplicationImplTests.cs** - 13 tests, NO attributes ✅ |
| 137 | +3. **ApplicationPopoverTests.cs** - 10 tests, NO attributes ✅ |
| 138 | + |
| 139 | +**Recommendation:** |
| 140 | +- Migrate the remaining files with no attributes |
| 141 | +- Many Application tests genuinely need Application static state |
| 142 | + |
| 143 | +## Performance by Test Pattern |
| 144 | + |
| 145 | +### AutoInitShutdown Tests (449 tests, ~35% of total) |
| 146 | + |
| 147 | +**Characteristics:** |
| 148 | +- Average 0.115s per test (vs 0.051s for no-attribute tests) |
| 149 | +- **2.25x slower than tests without attributes** |
| 150 | +- Creates Application singleton, initializes driver, sets up MainLoop |
| 151 | +- Calls Application.Shutdown after each test |
| 152 | + |
| 153 | +**Top Files Using AutoInitShutdown:** |
| 154 | +1. TileViewTests.cs - 42 tests (93% usage) |
| 155 | +2. TextViewTests.cs - 41 tests (39% usage) |
| 156 | +3. MenuBarv1Tests.cs - 40 tests (95% usage) |
| 157 | +4. GraphViewTests.cs - 24 tests (57% usage) |
| 158 | +5. DialogTests.cs - 23 tests (100% usage) |
| 159 | +6. MenuBarTests.cs - 20 tests (111% - multiple per test method) |
| 160 | + |
| 161 | +**Estimated Impact:** If 50% of AutoInitShutdown tests can be refactored: |
| 162 | +- ~225 tests × 0.064s overhead = **~14.4s savings** |
| 163 | + |
| 164 | +### SetupFakeDriver Tests (198 tests, ~16% of total) |
| 165 | + |
| 166 | +**Characteristics:** |
| 167 | +- Average 0.055s per test |
| 168 | +- Sets up Application.Driver globally |
| 169 | +- Many test visual output with DriverAssert |
| 170 | +- Less overhead than AutoInitShutdown but still blocks parallelization |
| 171 | + |
| 172 | +**Top Files Using SetupFakeDriver:** |
| 173 | +1. TableViewTests.cs - 45 tests (56% usage) |
| 174 | +2. LineCanvasTests.cs - 30 tests (86% usage) |
| 175 | +3. TabViewTests.cs - 18 tests (53% usage) |
| 176 | +4. TextFormatterTests.cs - 18 tests (78% usage) |
| 177 | +5. ColorPickerTests.cs - 16 tests (100% usage) |
| 178 | + |
| 179 | +**Estimated Impact:** If 30% can be refactored to remove driver dependency: |
| 180 | +- ~60 tests × 0.025s overhead = **~1.5s savings** |
| 181 | + |
| 182 | +### Tests with No Attributes (622 tests, ~49% of total) |
| 183 | + |
| 184 | +**Characteristics:** |
| 185 | +- Average 0.051s per test (fastest) |
| 186 | +- Should be immediately migratable |
| 187 | +- Many already identified in previous migration |
| 188 | + |
| 189 | +**Top Remaining Files:** |
| 190 | +1. ConfigurationMangerTests.cs - 27 tests ✅ (already migrated) |
| 191 | +2. MainLoopTests.cs - 23 tests ✅ (already migrated) |
| 192 | +3. GetViewsUnderLocationTests.cs - 21 tests ⭐ **HIGH PRIORITY** |
| 193 | +4. ConfigPropertyTests.cs - 18 tests (partial migration done) |
| 194 | +5. SchemeManagerTests.cs - 14 tests (partial migration done) |
| 195 | + |
| 196 | +## Recommendations: Phased Approach |
| 197 | + |
| 198 | +### Phase 1: Quick Wins (Estimated 15-20s savings, 1-2 days) |
| 199 | + |
| 200 | +**Target:** 150-200 tests with no attributes |
| 201 | + |
| 202 | +1. **Immediate migrations** (no refactoring needed): |
| 203 | + - GetViewsUnderLocationTests.cs (21 tests) |
| 204 | + - ApplicationImplTests.cs (13 tests) |
| 205 | + - ApplicationPopoverTests.cs (10 tests) |
| 206 | + - HexViewTests.cs (12 tests) |
| 207 | + - TimeFieldTests.cs (6 tests) |
| 208 | + - Various smaller files with no attributes |
| 209 | + |
| 210 | +2. **Complete partial migrations**: |
| 211 | + - ConfigPropertyTests.cs (add 14 more tests) |
| 212 | + - SchemeManagerTests.cs (add 4 more tests) |
| 213 | + - SettingsScopeTests.cs (add 9 more tests) |
| 214 | + |
| 215 | +**Expected Impact:** ~20s runtime reduction in UnitTests |
| 216 | + |
| 217 | +### Phase 2: TextViewTests Refactoring (Estimated 4-5s savings, 2-3 days) |
| 218 | + |
| 219 | +**Target:** Split 64 tests from TextViewTests.cs |
| 220 | + |
| 221 | +1. Extract simple tests (no AutoInitShutdown needed): |
| 222 | + - Property tests (Text, Enabled, Visible, etc.) |
| 223 | + - Event tests (TextChanged, etc.) |
| 224 | + - Constructor tests |
| 225 | + |
| 226 | +2. Extract tests that can use BeginInit/EndInit instead of Application.Begin: |
| 227 | + - Basic layout tests |
| 228 | + - Focus tests |
| 229 | + - Some selection tests |
| 230 | + |
| 231 | +3. Leave integration tests in UnitTests: |
| 232 | + - Tests that verify rendering output |
| 233 | + - Tests that need actual driver interaction |
| 234 | + - Multi-component interaction tests |
| 235 | + |
| 236 | +**Expected Impact:** ~4-5s runtime reduction |
| 237 | + |
| 238 | +### Phase 3: TileViewTests Optimization (Estimated 4-5s savings, 2-3 days) |
| 239 | + |
| 240 | +**Target:** Reduce TileViewTests from 9.25s to ~4s |
| 241 | + |
| 242 | +TileViewTests has the highest average time per test (0.197s) - nearly 4x the normal rate! |
| 243 | + |
| 244 | +**Analysis needed:** |
| 245 | +1. Why are these tests so slow? |
| 246 | +2. Are they testing multiple things per test? |
| 247 | +3. Can Application.Begin calls be replaced with BeginInit/EndInit? |
| 248 | +4. Are there setup/teardown inefficiencies? |
| 249 | + |
| 250 | +**Approach:** |
| 251 | +1. Profile individual test methods |
| 252 | +2. Look for common patterns causing slowness |
| 253 | +3. Refactor to reduce overhead |
| 254 | +4. Consider splitting into multiple focused test classes |
| 255 | + |
| 256 | +**Expected Impact:** ~5s runtime reduction |
| 257 | + |
| 258 | +### Phase 4: TableViewTests Refactoring (Estimated 2-3s savings, 2-3 days) |
| 259 | + |
| 260 | +**Target:** Extract ~35 tests from TableViewTests.cs |
| 261 | + |
| 262 | +TableViewTests has 45 SetupFakeDriver usages for visual testing. However: |
| 263 | +- Some tests may only need basic View hierarchy (BeginInit/EndInit) |
| 264 | +- Some tests may be testing properties that don't need rendering |
| 265 | +- Some tests may be duplicating coverage |
| 266 | + |
| 267 | +**Approach:** |
| 268 | +1. Categorize tests: pure unit vs rendering verification |
| 269 | +2. Extract pure unit tests to Parallelizable |
| 270 | +3. Keep rendering verification tests in UnitTests |
| 271 | +4. Look for duplicate coverage |
| 272 | + |
| 273 | +**Expected Impact:** ~3s runtime reduction |
| 274 | + |
| 275 | +### Phase 5: Additional View Tests (Estimated 10-15s savings, 1-2 weeks) |
| 276 | + |
| 277 | +**Target:** 200-300 tests across multiple View test files |
| 278 | + |
| 279 | +Focus on files with mix of attribute/no-attribute tests: |
| 280 | +- TextFieldTests.cs (43 tests, only 11 with attributes) |
| 281 | +- GraphViewTests.cs (42 tests, 24 AutoInit - can some be refactored?) |
| 282 | +- ListViewTests.cs (27 tests, 6 AutoInit) |
| 283 | +- LabelTests.cs (24 tests, 16 AutoInit + 3 SetupDriver) |
| 284 | +- TreeViewTests.cs (38 tests, 1 AutoInit + 9 SetupDriver) |
| 285 | + |
| 286 | +**Expected Impact:** ~15s runtime reduction |
| 287 | + |
| 288 | +## Summary of Potential Savings |
| 289 | + |
| 290 | +| Phase | Tests Migrated | Estimated Savings | Effort | Priority | |
| 291 | +|-------|----------------|-------------------|--------|----------| |
| 292 | +| Phase 1: Quick Wins | 150-200 | 15-20s | 1-2 days | ⭐⭐⭐⭐⭐ | |
| 293 | +| Phase 2: TextViewTests | 64 | 4-5s | 2-3 days | ⭐⭐⭐⭐ | |
| 294 | +| Phase 3: TileViewTests | 20-30 | 4-5s | 2-3 days | ⭐⭐⭐⭐ | |
| 295 | +| Phase 4: TableViewTests | 35 | 2-3s | 2-3 days | ⭐⭐⭐ | |
| 296 | +| Phase 5: Additional Views | 200-300 | 10-15s | 1-2 weeks | ⭐⭐⭐ | |
| 297 | +| **TOTAL** | **469-623 tests** | **35-48s** | **3-4 weeks** | | |
| 298 | + |
| 299 | +**Target Runtime:** |
| 300 | +- Current: ~90s (UnitTests) |
| 301 | +- After all phases: **~42-55s (38-47% reduction)** |
| 302 | +- Combined with Parallelizable: **~102-115s total (vs 150s current = 23-32% reduction)** |
| 303 | + |
| 304 | +## Key Insights |
| 305 | + |
| 306 | +### Why Some Tests Are Slow |
| 307 | + |
| 308 | +1. **AutoInitShutdown overhead** (0.064s per test): |
| 309 | + - Creates Application singleton |
| 310 | + - Initializes FakeDriver |
| 311 | + - Sets up MainLoop |
| 312 | + - Teardown and cleanup |
| 313 | + |
| 314 | +2. **Application.Begin overhead** (varies): |
| 315 | + - Initializes view hierarchy |
| 316 | + - Runs layout engine |
| 317 | + - Sets up focus/navigation |
| 318 | + - Creates event loops |
| 319 | + |
| 320 | +3. **Integration test nature**: |
| 321 | + - Dialogs/ tests average 0.115s/test |
| 322 | + - FileServices/ tests average 0.158s/test |
| 323 | + - Input/ tests average 0.515s/test (!) |
| 324 | + - These test full workflows, not units |
| 325 | + |
| 326 | +### Migration Difficulty Assessment |
| 327 | + |
| 328 | +**Easy (No refactoring):** |
| 329 | +- Tests with no attributes: 622 tests |
| 330 | +- Simply copy to Parallelizable and add base class |
| 331 | + |
| 332 | +**Medium (Minor refactoring):** |
| 333 | +- Tests using SetupFakeDriver but not Application statics: ~60 tests |
| 334 | +- Replace SetupFakeDriver with inline driver creation if needed |
| 335 | +- Or remove driver dependency entirely |
| 336 | + |
| 337 | +**Hard (Significant refactoring):** |
| 338 | +- Tests using AutoInitShutdown: 449 tests |
| 339 | +- Must replace Application.Begin with BeginInit/EndInit |
| 340 | +- Or split into unit vs integration tests |
| 341 | +- Or redesign test approach |
| 342 | + |
| 343 | +**Very Hard (May not be migratable):** |
| 344 | +- True integration tests: ~100-150 tests |
| 345 | +- Tests requiring actual rendering verification |
| 346 | +- Tests requiring Application singleton behavior |
| 347 | +- Keep these in UnitTests |
| 348 | + |
| 349 | +## Conclusion |
| 350 | + |
| 351 | +The analysis reveals clear opportunities for significant performance improvements: |
| 352 | + |
| 353 | +1. **Immediate impact:** 150-200 tests with no attributes can be migrated in 1-2 days for ~20s savings |
| 354 | +2. **High value:** TextViewTests and TileViewTests contain ~100 tests that can yield ~10s savings with moderate effort |
| 355 | +3. **Long-term:** Systematic refactoring of 469-623 tests could reduce UnitTests runtime by 38-47% |
| 356 | + |
| 357 | +The Views/ folder is the critical bottleneck, representing 50% of runtime. Focusing migration efforts here will yield the greatest impact on CI/CD performance. |
| 358 | + |
| 359 | +--- |
| 360 | + |
| 361 | +**Report Generated:** 2025-10-20 |
| 362 | +**Analysis Method:** Static analysis + runtime profiling |
| 363 | +**Total Tests Analyzed:** 3,260 tests across 121 files |
0 commit comments