marcelroed/gigatoken

GitHub

Language model tokenization at GB/s

Rust3,922 stars204 forksMIT

llm nlp tokenization tokenizer

Momentum

marcelroed/gigatoken has 3,922 GitHub stars.

It currently ranks #36922 daily, #37127 weekly, #38217 monthly by momentum on Trending Repos; its best-ever rank is #28724.

Rankings

PeriodRankScore

Daily

#369220.0

Weekly

#371270.0

Monthly

#382170.0

How scoring works

Engagement

Total stars

Daily new stars

Embeds

Best rank #28724

Format

Markdown

[![marcelroed/gigatoken badge](https://trending-repos.com/badge/marcelroed/gigatoken.svg?v=3)](https://trending-repos.com/repositories/marcelroed/gigatoken?utm_source=github&utm_medium=badge)

marcelroed/gigatoken

Momentum

Rankings

Engagement

Similar trending repositories

Embeds

marcelroed/gigatoken

Momentum

Rankings

Engagement

Similar trending repositories

Embeds