Skip to content

Commit 616175f

Browse files
committed
fix: Adjust sentence splitting logic to handle cases without the specified delimiter, ensuring proper text processing
1 parent 5363406 commit 616175f

File tree

1 file changed

+4
-1
lines changed
  • sources/gc-qa-rag-etl/etlapp/common

1 file changed

+4
-1
lines changed

sources/gc-qa-rag-etl/etlapp/common/chunk.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,10 @@ def split_text_into_sentence_groups(
2222
if not text:
2323
return []
2424

25-
sentences = text.split(sentence_delimiter)
25+
if sentence_delimiter in text:
26+
sentences = text.split(sentence_delimiter)
27+
else:
28+
sentences = text.split('\n')
2629

2730
# Remove empty sentences
2831
sentences = [s.strip() for s in sentences if s.strip()]

0 commit comments

Comments
 (0)