Commit 4b9d826
Fix BadRequestError while using OpenAI O1 series models | Add option to bypass temperature (#2151)
we were about to use OpenAI's O1 model for FactualCorrectness metric
since it is good at reasoning, but faced BadRequestError on ragas
eventhough it worked fine in langchain and llama index, then I came to
know ragas is calculating temperature, but it is not supported in O1
models, so added a way to bypass temperature
Reproducible code
```python
from langchain_openai import ChatOpenAI
from llama_index.llms.openai import OpenAI
from ragas.llms import LlamaIndexLLMWrapper,LangchainLLMWrapper
from langchain_core.prompt_values import StringPromptValue
api_key = "<your_api_key>"
langchain_llm=ChatOpenAI(model="o1", api_key=api_key)
llama_index_llm=OpenAI(model="o1", api_key=api_key)
# Both Will work fine
print(langchain_llm.invoke("hi"))
print(llama_index_llm.complete("hi"))
prompt = StringPromptValue(text="hi")
# Both Will raise error
await LlamaIndexLLMWrapper(llama_index_llm).agenerate_text(prompt)
await LangchainLLMWrapper(langchain_llm).agenerate_text(prompt)
```
Error
```
BadRequestError: Error code: 400 - {'error': {'message': "Unsupported parameter: 'temperature' is not supported with this model.", 'type': 'invalid_request_error', 'param': 'temperature', 'code': 'unsupported_parameter'}}
```
After Fix
```python
# Both Will work as expected
await LlamaIndexLLMWrapper(llama_index_llm,bypass_temperature=True).agenerate_text(prompt)
await LangchainLLMWrapper(langchain_llm,bypass_temperature=True).agenerate_text(prompt)
```
Due to scalability reasons, I am adding a flag instead of checking the
model name and removing the temperature
Co-authored-by: Vignesh Arivazhagan <vignesh.arivazhagan@solitontech.com>1 parent 02e0482 commit 4b9d826
1 file changed
+11
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
142 | 142 | | |
143 | 143 | | |
144 | 144 | | |
| 145 | + | |
145 | 146 | | |
146 | 147 | | |
147 | 148 | | |
148 | 149 | | |
149 | 150 | | |
150 | 151 | | |
151 | 152 | | |
| 153 | + | |
| 154 | + | |
152 | 155 | | |
153 | 156 | | |
154 | 157 | | |
| |||
252 | 255 | | |
253 | 256 | | |
254 | 257 | | |
255 | | - | |
| 258 | + | |
256 | 259 | | |
257 | 260 | | |
258 | 261 | | |
| |||
311 | 314 | | |
312 | 315 | | |
313 | 316 | | |
| 317 | + | |
314 | 318 | | |
315 | 319 | | |
316 | 320 | | |
| 321 | + | |
| 322 | + | |
317 | 323 | | |
318 | 324 | | |
319 | 325 | | |
| |||
378 | 384 | | |
379 | 385 | | |
380 | 386 | | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
381 | 391 | | |
382 | 392 | | |
383 | 393 | | |
| |||
0 commit comments