ruipgil/scraperjs

A complete and versatile web scraper.

JavaScript

3720

185

MIT License

Scraperjs is a Node‑JS library that lets you easily extract data from the web by offering a static scraper (using cheerio) and a dynamic scraper (driven by PhantomJS) with a promise‑based API and router support. It’s ideal for JavaScript developers who need to crawl or scrape both static HTML pages and JavaScript‑rendered content, and who want a flexible, reusable framework for building web‑scraping pipelines.

Total donated

Undistributed

Share with your subscribers:

Recipients

How the donated funds are distributed

Support the dependencies of ruipgil/scraperjs

caolan/async

Higher-order functions and common patterns for asynchronous code

cheeriojs/cheerio

The fast, flexible & elegant library for parsing and manipulating HTML and XML.

jquery/jquery

JavaScript library for DOM operations

amir20/phantomjs-node

PhantomJS integration module for NodeJS

request/request

Simplified HTTP request client.

tj/commander.js

the complete solution for node.js command-line programs

nickmerwin/node-coveralls

takes json-cov output into stdin and POSTs to coveralls.io

gruntjs/grunt

The JavaScript Task Runner

jharding/grunt-exec

Grunt task for executing shell commands.

pghalliday/grunt-mocha-test

A grunt task for running server side mocha tests

gotwarlost/istanbul

Yet another JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests. Supports all JS coverage use cases including unit tests, server side functional tests and browser tests. Built for scale