Skip to content

Dynamic first content containing think tag cannot be parsed as ThinkingPart #3374

@yf-yang

Description

@yf-yang

Initial Checks

Description

if thinking_tags and content == thinking_tags[0]:
# When we see a thinking start tag (which is a single token), we'll build a new thinking part instead
self._vendor_id_to_part_index.pop(vendor_part_id, None)
return self.handle_thinking_delta(vendor_part_id=vendor_part_id, content='')

I notice that sometimes model may output first content containing thinking tag, but not exactly thinking tag, so the logic fails to parse it as ThinkingPart.

For example, the following code's first content could be <think>\nThe user, and sometimes it could be <think>\nI need. I am not sure that's due to some model provider behavior so that the thinking tag is returned along with some additional token, but it could happen and currently implementation is unable to handle such a situation

Example Code

import asyncio
import os

import dotenv
from pydantic_ai import Agent, ModelProfile
from pydantic_ai.models.openai import OpenAIChatModel, OpenAIChatModelSettings
from pydantic_ai.providers.openai import OpenAIProvider

dotenv.load_dotenv()

agent = Agent(
  model=OpenAIChatModel(
    "minimax-m2",
    provider=OpenAIProvider(
      base_url=os.getenv("MINIMAX_BASE_URL"), api_key=os.getenv("MINIMAX_API_KEY")
    ),
    profile=ModelProfile(thinking_tags=("<think>\n", "</think>")),
  ),
  model_settings=OpenAIChatModelSettings(
    timeout=15,
  ),
)


async def main():
  prompt = "hello"
  async with agent.run_stream(prompt) as response:
    async for part, is_last in response.stream_responses():
        print(part)
  print(response.all_messages())


asyncio.run(main())

Python, Pydantic AI & LLM client version

python 3.12
pydantic-ai 1.0.18

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions