Skip to content

Commit 73a99cc

Browse files
authored
[Model] Fixed stream generator for gpt-oss + spec-decoding (vllm-project#26027)
Signed-off-by: Aleksandr Samarin <astrlrd@nebius.com>
1 parent adae0c1 commit 73a99cc

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

vllm/entrypoints/openai/serving_chat.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -691,11 +691,13 @@ async def chat_completion_stream_generator(
691691
if self.use_harmony:
692692
harmony_parser = harmony_parsers[i]
693693
prev_recipient = harmony_parser.current_recipient
694+
delta_text = ""
694695
for token_id in output.token_ids:
695696
harmony_parser.process(token_id)
697+
delta_text += (harmony_parser.last_content_delta
698+
or "")
696699
cur_channel = harmony_parser.current_channel
697700
cur_recipient = harmony_parser.current_recipient
698-
delta_text = harmony_parser.last_content_delta or ""
699701
else:
700702
delta_text = output.text
701703

0 commit comments

Comments
 (0)