File tree Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Expand file tree Collapse file tree 1 file changed +1
-1
lines changed Original file line number Diff line number Diff line change 33We ran OpenAI's recently released [codex](https://arxiv.org/abs/2107.03374) API (davinci-codex model) on the problems
44for 1,024 attempts per problem. Within each module below, we sort the problem by difficulty from hardest (unsolved)
55to easiest for codex. We used a 5-shot prompt based on the Tutorial (see our [paper](https://arxiv.org/abs/2106.05784)).
6- We report the success rate (fraction of the 10,000 generated solutions that satisfy the puzzle) and the shortest
6+ We report the success rate (fraction of the 1,024 generated solutions that satisfy the puzzle) and the shortest
77and longest valid solutions below.
88This document summarizes the dataset stored in the `puzzles.json` file in this directory.
99These files are generated from the `generators/*.py` files.
You can’t perform that action at this time.
0 commit comments