Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript
0
0
Apache License 2.0
Total donated
Undistributed
Share with your subscribers:
Recipients
How the donated funds are distributed
Support the dependencies of silva95gustavo/crawlee
Shareable commitlint config enforcing conventional commits
TypeScript definitions for content-type
TypeScript definitions for deep-equal
TypeScript definitions for express
TypeScript definitions for fs-extra
TypeScript definitions for inquirer
TypeScript definitions for lodash.isequal
TypeScript definitions for lodash.merge
TypeScript definitions for mime-types
TypeScript definitions for node
TypeScript definitions for proper-lockfile
TypeScript definitions for sax
TypeScript definitions for semver
TypeScript definitions for yargs
TypeScript plugin for ESLint
An ESLint custom parser which leverages TypeScript ESTree
V8 coverage provider for Vitest
Lint your commit messages
Run scripts that set and use environment variables across platforms
node's assert.deepEqual algorithm
fs-extra contains methods that aren't included in the vanilla Node.js fs package. Such as recursive mkdir, copy, and remove.
Generate ESM wrapper files for CommonJS modules
Modern native Git hooks
Detect if the current environment is a CI server
Lerna is a fast, modern build system for managing and publishing multiple JavaScript/TypeScript packages from the same repository
Lint files staged by git
HTTP server mocking and expectations library for Node.js
A high-level API to automate web browsers
TypeScript configuration shared across projects in Apify.
TypeScript definitions for domhandler
TypeScript definitions for htmlparser2
TypeScript definitions for is-ci
TypeScript definitions for stream-json
The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Parse Basic Auth `Authorization` HTTP headers
Pure javascript swiss knife for port management
Tools and constants shared across Apify projects.
Playwright package that automatically installs Firefox
Playwright package that automatically installs WebKit
TypeScript definitions for ps-tree
TypeScript definitions for tough-cookie
Playwright package that automatically installs Chromium
An HTTP proxy written with Node.js (think Squid)
Top contributors
mnmkng
1149 contributions
jancurn
711 contributions
![renovate[bot]'s profile](https://avatars.githubusercontent.com/renovate[bot])
renovate[bot]
516 contributions
B4nan
471 contributions
mtrunkat
444 contributions
petrpatek
242 contributions
vladfrangu
99 contributions
barjin
79 contributions
szmarczak
47 contributions
drobnikj
46 contributions
Recent events
Kivach works on the Obyte network, and therefore you can track all donations.
No events yet