File tree Expand file tree Collapse file tree 3 files changed +6
-15
lines changed
src/inspect_evals/cyberseceval_2 Expand file tree Collapse file tree 3 files changed +6
-15
lines changed Original file line number Diff line number Diff line change 1010Based on: https://github.com/meta-llama/PurpleLlama/tree/70e7a376f2a310fb89f7ad207685baa69f78107f/CybersecurityBenchmarks
1111
1212# eval for default epochs (4)
13- inspect eval inspect_evals/cyberseceval_2
13+ inspect eval inspect_evals/interpreter_abuse -
1414
1515# eval with 1 epoch
16- inspect eval inspect_evals/cyberseceval_2 --epochs 1
17-
18- # with chain of thought
19- inspect eval inspect_evals/cyberseceval_2 -T cot=true
16+ inspect eval inspect_evals/interpreter_abuse --epochs 1
2017"""
2118
2219from typing import List
Original file line number Diff line number Diff line change 1111Based on: https://github.com/meta-llama/PurpleLlama/tree/70e7a376f2a310fb89f7ad207685baa69f78107f/CybersecurityBenchmarks
1212
1313# eval for default epochs (4)
14- inspect eval inspect_evals/cyberseceval_2
14+ inspect eval inspect_evals/prompt_injection
1515
1616# eval with 1 epoch
17- inspect eval inspect_evals/cyberseceval_2 --epochs 1
18-
19- # with chain of thought
20- inspect eval inspect_evals/cyberseceval_2 -T cot=true
17+ inspect eval inspect_evals/prompt_injection --epochs 1
2118"""
2219
2320from typing import List
Original file line number Diff line number Diff line change 1515Based on: https://github.com/meta-llama/PurpleLlama/tree/70e7a376f2a310fb89f7ad207685baa69f78107f/CybersecurityBenchmarks
1616
1717# eval for default epochs (4)
18- inspect eval inspect_evals/cyberseceval_2
18+ inspect eval inspect_evals/vulnerability_exploit
1919
2020# eval with 1 epoch
21- inspect eval inspect_evals/cyberseceval_2 --epochs 1
22-
23- # with chain of thought
24- inspect eval inspect_evals/cyberseceval_2 -T cot=true
21+ inspect eval inspect_evals/vulnerability_exploit --epochs 1
2522"""
2623
2724from pathlib import Path
You can’t perform that action at this time.
0 commit comments