Skip to content

Commit c0fbc3a

Browse files
committed
fix: set seed for shuffling MMLU samples to allow reproducibility
1 parent ea12179 commit c0fbc3a

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

src/inspect_evals/mmlu/mmlu.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,7 @@ def get_mmlu_dataset(
200200
split=split,
201201
sample_fields=record_to_sample,
202202
shuffle=shuffle,
203+
seed=42,
203204
)
204205

205206
# filter dataset if requested

0 commit comments

Comments
 (0)