Skip to content

Commit 0efb76e

Browse files
author
Workshop Participant
committed
Reads cache tokens from litellm response (openai, anthropic, bedrock)
1 parent a42d4f7 commit 0efb76e

File tree

1 file changed

+9
-1
lines changed

1 file changed

+9
-1
lines changed

src/strands/models/litellm.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,15 @@ async def stream(self, request: dict[str, Any]) -> AsyncGenerator[dict[str, Any]
158158
for event in response:
159159
_ = event
160160

161-
yield {"chunk_type": "metadata", "data": event.usage}
161+
usage = event.usage
162+
cache_read = max(
163+
getattr(usage, "cache_read_input_tokens", 0),
164+
getattr(getattr(usage, "prompt_tokens_details", {}), "cached_tokens", 0),
165+
)
166+
167+
usage.prompt_tokens_details.cached_tokens = cache_read
168+
169+
yield {"chunk_type": "metadata", "data": usage}
162170

163171
@override
164172
async def structured_output(

0 commit comments

Comments
 (0)