belladoreai/llama-tokenizer

llama-tokenizer-js is a JavaScript tokenizer for LLaMA 1 and 2 models that works client-side in browsers and Node.js, designed for accurately calculating token counts without dependencies. It's intended for developers building web applications that need to count LLaMA tokens client-side, offering a 670KiB optimized solution with TypeScript support.