niieani/gpt-tokenizer

The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features.

TypeScript855 stars63 forksMIT

bpe decoder encoder gpt-2 gpt-3 gpt-4 gpt-4o gpt-5 gpt-o1 machine-learning openai tokenizer

Momentum

niieani/gpt-tokenizer has 855 GitHub stars.

It currently ranks #28238 daily, #28726 weekly, #30576 monthly by momentum on Trending Repos; its best-ever rank is #10273.

Trending Repos first recorded it on Aug 16, 2026.

Rankings

PeriodRankScore

Daily

#282380.0

Weekly

#287260.0

Monthly

#305760.0

How scoring works

Engagement

Total stars

Daily new stars

Embeds

Best rank #10273

Format

Markdown

[![niieani/gpt-tokenizer badge](https://trending-repos.com/badge/niieani/gpt-tokenizer.svg?v=3)](https://trending-repos.com/repositories/niieani/gpt-tokenizer?utm_source=github&utm_medium=badge)

niieani/gpt-tokenizer

Momentum

Rankings

Engagement

Similar trending repositories

Embeds

niieani/gpt-tokenizer

Momentum

Rankings

Engagement

Similar trending repositories

Embeds