Skip to content

Commit 4edafea

Browse files
committed
🤖 ci: add adaptive mode support to terminal-bench workflow
Add workflow_dispatch inputs for adaptive concurrency mode: - adaptive_mode: Enable adaptive concurrency (default: false) - max_concurrent: Max concurrency for adaptive mode (default: 16) - load_threshold: Load threshold for adjustments (default: 1.0) When adaptive_mode=true, runs benchmark-terminal-adaptive instead of benchmark-terminal. _Generated with `cmux`_
1 parent 6a5fd3f commit 4edafea

File tree

1 file changed

+23
-1
lines changed

1 file changed

+23
-1
lines changed

.github/workflows/terminal-bench.yml

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,21 @@ on:
7272
description: 'Additional arguments to pass to terminal-bench'
7373
required: false
7474
type: string
75+
adaptive_mode:
76+
description: 'Use adaptive concurrency (overrides concurrency setting)'
77+
required: false
78+
default: false
79+
type: boolean
80+
max_concurrent:
81+
description: 'Max concurrency for adaptive mode (only used if adaptive_mode=true)'
82+
required: false
83+
default: '16'
84+
type: string
85+
load_threshold:
86+
description: 'Load threshold for adaptive mode (only used if adaptive_mode=true)'
87+
required: false
88+
default: '1.0'
89+
type: string
7590

7691
jobs:
7792
benchmark:
@@ -101,10 +116,17 @@ jobs:
101116
run: make build-main build-preload
102117

103118
- name: Run Terminal-Bench
104-
run: make benchmark-terminal
119+
run: |
120+
if [ "${{ inputs.adaptive_mode }}" = "true" ]; then
121+
make benchmark-terminal-adaptive
122+
else
123+
make benchmark-terminal
124+
fi
105125
env:
106126
TB_DATASET: ${{ inputs.dataset }}
107127
TB_CONCURRENCY: ${{ inputs.concurrency }}
128+
TB_MAX_CONCURRENT: ${{ inputs.max_concurrent }}
129+
TB_LOAD_THRESHOLD: ${{ inputs.load_threshold }}
108130
TB_LIVESTREAM: ${{ inputs.livestream && '1' || '' }}
109131
TB_SAMPLE_SIZE: ${{ inputs.sample_size }}
110132
TB_ARGS: ${{ inputs.model_name && format('--agent-kwarg model_name={0} --agent-kwarg thinking_level={1} {2}', inputs.model_name, inputs.thinking_level, inputs.extra_args) || inputs.extra_args }}

0 commit comments

Comments
 (0)