Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously
JavaScript
41
7
MIT License
Total donated
Undistributed
Share with your subscribers:
Recipients
How the donated funds are distributed
Support the dependencies of bda-research/node-webcrawler
The fast, flexible & elegant library for parsing and manipulating HTML and XML.
Convert character encodings in pure javascript.
Simplified HTTP request client.
BDD/TDD assertion library for node.js and the browser. Test framework agnostic.
simple, flexible, fun test framework
JavaScript test spies, stubs and mocks.
A JavaScript implementation of many web standards
Parse charset string from http header and hmtl meta
Text recoding in JavaScript for fun and profit!
A library to test if a url(request) is crawled, usually used in a web crawler. Compatible with `request` and `node-crawler`
Distributed task scheduler and rate limiter
Support the repos that depend on bda-research/node-webcrawler
Collect reviews for Android apps from the Google Play Store
Musixmatch SDK for Node.js
Simple CLI to crawl a website/webapp and perform Lighthouse tests.
Collect reviews for iOS apps from the Apple App Store
Top contributors
paulvalla
135 contributions
sylvinus
63 contributions
mike442144
44 contributions
jhurliman
8 contributions
bkw
5 contributions
darrenqc
5 contributions
tzbee
3 contributions
ConnorWeng
2 contributions
digitalfrost
2 contributions
YusefAmen
2 contributions
Recent events
Kivach works on the Obyte network, and therefore you can track all donations.
No events yet