write chunks in TextParserStreamer #3025

pavel-esir · 2025-11-17T11:58:56Z

Description

Return DeltaMessage in incremental TextStreamerParser instead of accumulated msg. But accumulated is still available via get_parsed_message()

CVS-CVS-176146

Fixes #(issue)

Checklist:

Tests have been updated or added to cover the new code.
This patch fully addresses the ticket.
I have made corresponding changes to the documentation. TODO

Copilot

Pull Request Overview

This PR refactors the TextParserStreamer to return incremental delta messages in the write() callback instead of accumulated messages, while still maintaining accumulated message access via get_parsed_message(). The key changes update the reasoning parser implementation to properly handle delta messages and modify the concatenation logic in the streamer.

Modified TextParserStreamer to write delta messages instead of accumulated messages
Updated ReasoningIncrementalParser to populate message fields for incremental output
Added concatenate_json_containers() helper function for message accumulation

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
tests/python_tests/test_parsers.py	Updated tests to use `get_parsed_message()` for assertions and removed manual message accumulation in custom streamers
src/cpp/src/text_streamer.cpp	Added `concatenate_json_containers()` function and modified `write()` to return delta messages while maintaining internal accumulation
src/cpp/src/parsers.cpp	Refactored `ReasoningIncrementalParser` to populate `content` field in delta messages and removed `keep_original_content` logic from helper methods

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cpp/src/text_streamer.cpp

src/cpp/src/parsers.cpp

Copilot

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

tests/python_tests/test_parsers.py:1

Assignment to reason_str loses previously accumulated reasoning content. The variable should be appended to, not replaced, to preserve partial reasoning text from previous chunks.

# Copyright (C) 2023-2025 Intel Corporation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/python_tests/test_parsers.py

src/cpp/src/text_streamer.cpp

src/cpp/src/parsers.cpp

Copilot

Pull Request Overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cpp/src/parsers.cpp

src/cpp/src/text_streamer.cpp

src/cpp/src/parsers.cpp

src/cpp/src/text_streamer.cpp

src/cpp/src/parsers.cpp

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/python_tests/test_parsers.py

tests/cpp/parser.cpp

src/cpp/src/parsers.cpp

src/cpp/src/json_container.cpp

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cpp/src/parsers.cpp

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tests/python_tests/test_parsers.py

src/cpp/src/parsers.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/cpp/src/parsers.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-27T23:46:07Z

tests/python_tests/test_parsers.py


    prompt = "Please say \"hello\""
-    res = pipe.generate([prompt], max_new_tokens=600, parsers=[Phi4ReasoningParser()])
+    res = pipe.generate([prompt], max_new_tokens=600, parsers=[ReasoningParser(keep_original_content=False)])


The parser class 'ReasoningParser' is not imported. The removed import at line 7 shows 'Phi4ReasoningParser' was available, but 'ReasoningParser' is not in the import list. This will cause a NameError at runtime.

Copilot · 2025-11-27T23:46:07Z

src/cpp/src/text_streamer.cpp

    }

+    // Every time we start to cycle through iterative parsers we create a new delta_message.
+    // Parsers should neither delete fields nor rewrite they should only append or add new fields.


Missing punctuation after 'rewrite'. The sentence should read 'Parsers should neither delete fields nor rewrite; they should only append or add new fields.' or split into two sentences.

Suggested change

// Parsers should neither delete fields nor rewrite they should only append or add new fields.

// Parsers should neither delete fields nor rewrite; they should only append or add new fields.

Copilot · 2025-11-27T23:46:08Z

src/cpp/src/text_streamer.cpp


+    // Every time we start to cycle through iterative parsers we create a new delta_message.
+    // Parsers should neither delete fields nor rewrite they should only append or add new fields.
+    // The only field is updated automaticall is "content": delta_text is put there.


Corrected spelling of 'automaticall' to 'automatically'.

Suggested change

// The only field is updated automaticall is "content": delta_text is put there.

// The only field that is updated automatically is "content": delta_text is put there.

Copilot · 2025-11-27T23:46:08Z

src/cpp/src/json_container.cpp

+        OPENVINO_ASSERT(src_val.is_string(), "JsonContainer concatenate supports only string concatenation for object values.");
+        auto& dst_val = (*dst_)[it.key()];


The error message states that only string concatenation is supported, but the assertion only checks if src_val is a string. If dst_val exists and is not a string, the concatenation on line 426 will fail with a different error. Consider checking both src_val and dst_val are strings, or improve the error message to be more specific about which value is not a string.

Suggested change

OPENVINO_ASSERT(src_val.is_string(), "JsonContainer concatenate supports only string concatenation for object values.");

auto& dst_val = (*dst_)[it.key()];

auto& dst_val = (*dst_)[it.key()];

OPENVINO_ASSERT(

src_val.is_string() && dst_val.is_string(),

"JsonContainer concatenate supports only string concatenation for object values. "

"Key: '", it.key(), "', src_val type: '", src_val.type_name(), "', dst_val type: '", dst_val.type_name(), "'."

);

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings November 17, 2025 11:58

github-actions bot added no-match-files category: GGUF GGUF file reader category: text streamer labels Nov 17, 2025

pavel-esir requested a review from apaniukov November 17, 2025 11:59

Copilot AI reviewed Nov 17, 2025

View reviewed changes

src/cpp/src/text_streamer.cpp Outdated Show resolved Hide resolved

src/cpp/src/text_streamer.cpp Outdated Show resolved Hide resolved

src/cpp/src/parsers.cpp Outdated Show resolved Hide resolved

src/cpp/src/parsers.cpp Outdated Show resolved Hide resolved

write chunks in TextParserStreamer

2be242f

pavel-esir force-pushed the write_chunks_during_parse branch from 9b7cd5c to a41f0d0 Compare November 21, 2025 12:08

Copilot AI review requested due to automatic review settings November 21, 2025 12:08

Copilot AI reviewed Nov 21, 2025

View reviewed changes

pavel-esir force-pushed the write_chunks_during_parse branch 2 times, most recently from d71fc3f to 4017fb5 Compare November 21, 2025 12:23

Copilot AI review requested due to automatic review settings November 21, 2025 12:23

Copilot AI reviewed Nov 21, 2025

View reviewed changes

src/cpp/src/parsers.cpp Show resolved Hide resolved

src/cpp/src/parsers.cpp Show resolved Hide resolved

Fix pytests and gtests for parsers

4017fb5

apaniukov reviewed Nov 24, 2025

View reviewed changes

src/cpp/src/text_streamer.cpp Outdated Show resolved Hide resolved

apaniukov reviewed Nov 24, 2025

View reviewed changes

src/cpp/src/parsers.cpp Outdated Show resolved Hide resolved

apaniukov reviewed Nov 25, 2025

View reviewed changes

src/cpp/src/text_streamer.cpp Outdated Show resolved Hide resolved

apaniukov reviewed Nov 25, 2025

View reviewed changes

src/cpp/src/parsers.cpp Outdated Show resolved Hide resolved

get rid of 'ensure_message_fields'; extend texts

6e5dcc2

Copilot AI review requested due to automatic review settings November 27, 2025 23:31

github-actions bot added the category: CPP API Changes in GenAI C++ public headers label Nov 27, 2025