Official implementation of "SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching" (COLM 2025). A novel KV cache compression method that organizes cache at sentence level using semantic similarity.

This app leverages Semantic Caching to minimize inference latency and reduce API costs by reusing semantically similar prompt responses.

This is a hands-on workshop to create the application Redis Movies Searcher

Node SDK for the Semcache API

Python library for the Semcache API