Tiiny-AI/PowerInfer

High-speed Large Language Model Serving for Local Deployment

C++9,482 stars575 forksMIT

large-language-modelsllamallmllm-inferencelocal-inference

Rankings

Daily

#2132

Weekly

#3051

Monthly

#2526

Total stars

Daily new stars

Get badge

This repository is tracked by Trending Repos. The badge upgrades automatically if it ever cracks the top 100.

Markdown

![Trending Repos](https://trending-repos.com/badge/Tiiny-AI/PowerInfer.svg)

HTML

<img src="https://trending-repos.com/badge/Tiiny-AI/PowerInfer.svg" alt="Trending Repos" />

Raw URL

https://trending-repos.com/badge/Tiiny-AI/PowerInfer.svg