xaskasdf/ntransformer

High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.

C++457 stars19 forks

Rankings

Daily

#13341

Weekly

#7397

Monthly

#14592

Total stars

Daily new stars

Get badge

This repository is tracked by Trending Repos. The badge upgrades automatically if it ever cracks the top 100.

Markdown

![Trending Repos](https://trending-repos.com/badge/xaskasdf/ntransformer.svg)

HTML

<img src="https://trending-repos.com/badge/xaskasdf/ntransformer.svg" alt="Trending Repos" />

Raw URL

https://trending-repos.com/badge/xaskasdf/ntransformer.svg