@@ -4,26 +4,69 @@ This document outlines the benchmarking process for comparing the performance of
44
55> About tests performance can be found at: [ ![ CodSpeed Badge] ( https://img.shields.io/endpoint?url=https://codspeed.io/badge.json )] ( https://codspeed.io/cpp-linter/cpp-linter-hooks )
66
7- ## Running the Benchmark
7+ ## Run benchmark separately
88
99``` bash
10- python3 testing/benchmark_hooks.py
10+ rm -rf examples || true
11+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
1112
12- # or
13+ pre-commit clean
14+ hyperfine --warmup 1 -r 10 ' pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files'
1315
14- hyperfine -i --warmup 1 -r 5 ' pre-commit run --config ../testing/benchmark_hook_1.yaml --all-files' ' pre-commit run --config ../testing/benchmark_hook_2.yaml --all-files'
16+ rm -rf examples || true
17+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
18+ pre-commit clean
19+
20+ hyperfine --warmup 1 -r 10 ' pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files'
21+ ```
22+
23+ Results:
24+
25+ ``` bash
26+ # Updated on 2025-09-19
27+
28+ Cleaned /home/sxp/.cache/pre-commit.
29+ Benchmark 1: pre-commit run --config testing/test-cpp-linter-hooks.yaml --all-files
30+ Time (mean ± σ): 150.2 ms ± 1.8 ms [User: 121.7 ms, System: 29.2 ms]
31+ Range (min … max): 148.3 ms … 153.9 ms 10 runs
32+
33+ Cleaned /home/sxp/.cache/pre-commit.
34+ Benchmark 1: pre-commit run --config testing/test-mirrors-clang-format.yaml --all-files
35+ Time (mean ± σ): 122.6 ms ± 1.9 ms [User: 98.0 ms, System: 24.7 ms]
36+ Range (min … max): 120.3 ms … 125.5 ms 10 runs
37+ ```
38+
39+ ### Run benchmark comparison
40+
41+ Compare the results of both commands.
42+
43+ ``` bash
44+ rm -rf examples || true
45+ git clone --depth 1 --quiet https://github.com/gouravthakur39/beginners-C-program-examples.git examples
46+
47+ hyperfine -i --warmup 1 -r 20 ' pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ' pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
1548```
1649
17- ## Results
50+ Results:
1851
1952``` bash
20- # Updated on 2025-09-02
21- Benchmark Results:
53+ # Updated on 2025-09-19
54+ Benchmark 1: pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files
55+ Time (mean ± σ): 84.1 ms ± 3.2 ms [User: 73.5 ms, System: 10.2 ms]
56+ Range (min … max): 79.7 ms … 95.2 ms 20 runs
57+
58+ Warning: Ignoring non-zero exit code.
2259
23- Hook | Avg (s) | Std (s) | Min (s) | Max (s) | Runs
24- ---------------------+------------------+------------------+------------------+------------------+-----------------
25- cpp-linter-hooks | 12.473 | 1.738 | 11.334 | 15.514 | 5
26- mirrors-clang-format | 4.960 | 0.229 | 4.645 | 5.284 | 5
60+ Benchmark 2: pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files
61+ Time (mean ± σ): 85.0 ms ± 3.0 ms [User: 71.8 ms, System: 13.3 ms]
62+ Range (min … max): 81.0 ms … 91.0 ms 20 runs
2763
28- Results saved to testing/benchmark_results.txt
64+ Warning: Ignoring non-zero exit code.
65+
66+ Summary
67+ ' pre-commit run --config ../testing/test-cpp-linter-hooks.yaml --all-files' ran
68+ 1.01 ± 0.05 times faster than ' pre-commit run --config ../testing/test-mirrors-clang-format.yaml --all-files'
2969```
70+
71+ > [ !NOTE]
72+ > The results may vary based on the system and environment where the benchmarks are run.
0 commit comments