A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.
llm-d/llm-d-inference-sim has 198 GitHub stars.
It currently ranks #34402 daily, #34684 weekly, #35977 monthly by momentum on Trending Repos; its best-ever rank is #3723.