tiktoken is a fast BPE tokeniser for use with OpenAI's models.
7 daysStars19.6k24h gain+7Score0.0
Search
Find trending repositories by name or description.
3 results for “tiktoken”
RankRepository7 daysStars24h gainScore
Ready-made tokenizer library for working with GPT and tiktoken
7 daysStars39724h gain-Score0.0
The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (gpt-5, gpt-o*, gpt-4o, etc.). Port of OpenAI's tiktoken with additional features.
7 daysStars85524h gain-Score0.0