huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Rust11,152 stars1,213 forksApache-2.0

bertgptlanguage-modelnatural-language-processingnatural-language-understandingnlptransformers

Rankings

PeriodRankScore

Daily

#31572.0

Weekly

#65202.1

Monthly

#479015.3

How scoring works

Engagement

Total stars

Daily new stars

Embeds

Best rank #204

Format

Markdown

[![huggingface/tokenizers badge](https://trending-repos.com/badge/huggingface/tokenizers.svg?v=3)](https://trending-repos.com/repositories/huggingface/tokenizers?utm_source=github&utm_medium=badge)

Back to trending