File tree Expand file tree Collapse file tree 2 files changed +6
-4
lines changed
src/inspect_evals/cybench Expand file tree Collapse file tree 2 files changed +6
-4
lines changed Original file line number Diff line number Diff line change @@ -34,8 +34,9 @@ def cybench(
3434 max_attempts (int): The maximum number of submission attempts before
3535 terminating. This argument is ignored if `agent` is provided.
3636 max_messages (int): The maximum number of messages in the conversation.
37- sandbox_type (Literal["docker", "k8s"]): The type of sandbox to use. Currently
38- "k8s" is not supported for users external to UK AI Safety Institute.
37+ sandbox_type (Literal["docker", "k8s"]): The type of sandbox to use. "k8s"
38+ requires the inspect-k8s-sandbox package:
39+ https://github.com/UKGovernmentBEIS/inspect_k8s_sandbox.
3940 """
4041
4142 def get_challenge_dir_paths () -> list [Path ]:
Original file line number Diff line number Diff line change @@ -19,8 +19,9 @@ def create_dataset(
1919 challenge_dirs (list[Path]): A list of directories to recursively search for
2020 challenges in. A challenge is considered to be a directory containing a
2121 `challenge.yaml` file.
22- sandbox_type (Literal["docker", "k8s"]): The type of sandbox to use. Currently
23- "k8s" is not supported for users external to UK AI Safety Institute.
22+ sandbox_type (Literal["docker", "k8s"]): The type of sandbox to use. "k8s"
23+ requires the inspect-k8s-sandbox package:
24+ https://github.com/UKGovernmentBEIS/inspect_k8s_sandbox.
2425 """
2526 challenge_dirs = list (_find_challenge_dirs_recursive (challenge_dirs ))
2627 return MemoryDataset (samples = list (_create_samples (challenge_dirs , sandbox_type )))
You can’t perform that action at this time.
0 commit comments