Skip to content

Commit cbb0dce

Browse files
committed
Added the problems!
1 parent c998f95 commit cbb0dce

File tree

14 files changed

+4378
-2
lines changed

14 files changed

+4378
-2
lines changed

README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
1-
## Python Reasoning Challenges
1+
# Python Reasoning Challenges
22

3-
This repo contains a dataset of python reasoning challenges which can be used to teach an AI python and evaluate an AI's ability to understand and write python programs. Each challenge takes the form of a python function that takes an answer as an argument. The goal is to find an answer which makes the function return `True`.
3+
This repo contains a dataset of python reasoning challenges which can be used to teach an AI python and evaluate an AI's ability to understand and write python programs.
4+
5+
# [Click here to browse the challenges](/problems/README.md)
6+
7+
## What is a python reasoning challenge?
8+
9+
Each challenge takes the form of a python function that takes an answer as an argument. The goal is to find an answer which makes the function return `True`.
410

511
```python
612
def sat(s: str):

make_dataset.py

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
import argparse
2+
import fnmatch
3+
import time
4+
5+
import problems
6+
import utils
7+
import templates # This loads all the problem templates
8+
9+
TARGET_NUM_PER_PROBLEM = 1000
10+
11+
12+
parser = argparse.ArgumentParser(
13+
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
14+
15+
parser.add_argument('--target_num_per_problem',
16+
'-n',
17+
type=int,
18+
default=TARGET_NUM_PER_PROBLEM,
19+
help='Target number of variants to generate per problem.')
20+
21+
parser.add_argument('--templates',
22+
'-t',
23+
type=str,
24+
default='*',
25+
help='Glob phrase to select the template files to generate from.')
26+
27+
28+
29+
30+
31+
def main(args):
32+
start_time = time.perf_counter()
33+
templates = fnmatch.filter(problems.problem_registry.keys(), args.templates)
34+
utils.info(f"Generating from templates: {templates}")
35+
problem_sets = []
36+
for name in templates: # order determined by import order in templates/__init__.py
37+
entry = problems.problem_registry[name]
38+
ps = problems.ProblemSet(entry["name"], entry["summary"])
39+
for cls in entry["classes"]:
40+
problem = cls()
41+
problem.build(args.target_num_per_problem)
42+
ps.add(problem)
43+
ps.save()
44+
problem_sets.append(ps)
45+
46+
problems.save_readme(problem_sets)
47+
utils.info(f"Elapsed time: {(time.perf_counter()-start_time)/60:.2f} minutes")
48+
49+
if __name__ == "__main__":
50+
args = parser.parse_args()
51+
main(args)

0 commit comments

Comments
 (0)