File tree Expand file tree Collapse file tree 1 file changed +8
-1
lines changed Expand file tree Collapse file tree 1 file changed +8
-1
lines changed Original file line number Diff line number Diff line change @@ -127,7 +127,7 @@ bigcodebench.evaluate \
127127 --execution [e2b| gradio| local] \
128128 --split [complete| instruct] \
129129 --subset [full| hard] \
130- --backend [vllm| openai| anthropic| google| mistral| hf]
130+ --backend [vllm| openai| anthropic| google| mistral| hf| hf-inference ]
131131```
132132
133133- All the resulted files will be stored in a folder named ` bcb_results ` .
@@ -177,6 +177,13 @@ Access Gemini APIs from [Google AI Studio](https://aistudio.google.com/)
177177export GOOGLE_API_KEY=< your_google_api_key>
178178```
179179
180+ Access the [ Hugging Face Serverless Inference API] ( https://huggingface.co/docs/api-inference/en/index )
181+ ``` bash
182+ export HF_INFERENCE_API_KEY=< your_hf_api_key>
183+ ```
184+
185+ Please make sure your HF access token has the ` Make calls to inference providers ` permission.
186+
180187## 💻 LLM-generated Code
181188
182189We share pre-generated code samples from LLMs we have [ evaluated] ( https://huggingface.co/spaces/bigcode/bigcodebench-leaderboard ) on the full set:
You can’t perform that action at this time.
0 commit comments