-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Closed as duplicate of#3007
Closed as duplicate of#3007
Copy link
Labels
Description
Initial Checks
- I confirm that I'm using the latest version of Pydantic AI
- I confirm that I searched for my issue in https://github.com/pydantic/pydantic-ai/issues before opening this issue
Description
pydantic-ai/pydantic_ai_slim/pydantic_ai/_parts_manager.py
Lines 130 to 133 in 86b645f
| if thinking_tags and content == thinking_tags[0]: | |
| # When we see a thinking start tag (which is a single token), we'll build a new thinking part instead | |
| self._vendor_id_to_part_index.pop(vendor_part_id, None) | |
| return self.handle_thinking_delta(vendor_part_id=vendor_part_id, content='') |
I notice that sometimes model may output first content containing thinking tag, but not exactly thinking tag, so the logic fails to parse it as ThinkingPart.
For example, the following code's first content could be <think>\nThe user, and sometimes it could be <think>\nI need. I am not sure that's due to some model provider behavior so that the thinking tag is returned along with some additional token, but it could happen and currently implementation is unable to handle such a situation
Example Code
import asyncio
import os
import dotenv
from pydantic_ai import Agent, ModelProfile
from pydantic_ai.models.openai import OpenAIChatModel, OpenAIChatModelSettings
from pydantic_ai.providers.openai import OpenAIProvider
dotenv.load_dotenv()
agent = Agent(
model=OpenAIChatModel(
"minimax-m2",
provider=OpenAIProvider(
base_url=os.getenv("MINIMAX_BASE_URL"), api_key=os.getenv("MINIMAX_API_KEY")
),
profile=ModelProfile(thinking_tags=("<think>\n", "</think>")),
),
model_settings=OpenAIChatModelSettings(
timeout=15,
),
)
async def main():
prompt = "hello"
async with agent.run_stream(prompt) as response:
async for part, is_last in response.stream_responses():
print(part)
print(response.all_messages())
asyncio.run(main())Python, Pydantic AI & LLM client version
python 3.12
pydantic-ai 1.0.18