stdlib-js/nlp-tokenize

Tokenize a string.

JavaScript

Apache License 2.0

The `stdlib-js/nlp-tokenize` package provides a function to tokenize strings, breaking them into arrays of tokens such as words and punctuation marks. It's designed for developers working on natural language processing tasks in JavaScript or Node.js who need a reliable and well-tested tokenization utility.

Total donated

Undistributed

Share with your subscribers:

Recipients

How the donated funds are distributed

Support the dependencies of stdlib-js/nlp-tokenize

stdlib-js/assert-has-own-property

Test if an object has a specified property.

stdlib-js/assert-is-boolean

Test if a value is a boolean.

stdlib-js/assert-is-string

Test if a value is a string.

stdlib-js/string-format

Insert supplied variable values into a format string.

stdlib-js/error-tools-fmtprodmsg

Format an error message for production.

stdlib-js/assert-is-array

Test if a value is an array.

stdlib-js/string-from-code-point

Create a string from a sequence of Unicode code points.

tape-testing/tape

tap-producing test harness for node and browsers

gotwarlost/istanbul

Yet another JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests. Supports all JS coverage use cases including unit tests, server side functional tests and browser tests. Built for scale