Tokenx is a lightweight JavaScript library that provides fast token count estimation for large language models, achieving 95-98% accuracy while maintaining a minimal 2kB bundle size. It's designed for developers who need quick token approximations for text processing, chunking, or validation without the overhead of full tokenizers, supporting multiple languages including CJK and accented characters.
Total donated
Undistributed
Share with your subscribers:
Recipients
How the donated funds are distributed
Support the dependencies of johannschopplich/tokenx
Anthony's ESLint config
TypeScript definitions for node
Bump version, commit changes, tag, and push to Git
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
The Elegant Bundler for Libraries
TypeScript is a language for application scale JavaScript development
Next generation testing framework powered by Vite
Support the repos that depend on this repository
Top contributors
Recent events
Kivach works on the Obyte network, and therefore you can track all donations.
No events yet