CacheRoute is an innovative LLM scheduling scheme dedicated to enabling flexible KV cache reuse across LLM systems, improving task performance and system efficiency.
AstraNetLab/CacheRoute has 184 GitHub stars.
It currently ranks #37016 daily, #37217 weekly, #38301 monthly by momentum on Trending Repos; its best-ever rank is #3359.