matthewmueller/x-ray

The next web scraper. See through the <html> noise.

JavaScript

5907

342

MIT License

X-ray is a flexible and composable web scraper for Node.js that allows developers to extract structured data from websites using jQuery-like selectors. It supports features like pagination, concurrency control, streaming results, and pluggable drivers, making it ideal for developers who need to responsibly scrape data from the web while maintaining control over request rates and data structure.

Total donated

Undistributed

Share with your subscribers:

Recipients

How the donated funds are distributed

Support the dependencies of matthewmueller/x-ray

visionmedia/batch

Simple async batch with concurrency control and progress reporting.

petkaantonov/bluebird

Full featured Promises/A+ implementation with exceptionally good performance

cheeriojs/cheerio

The fast, flexible & elegant library for parsing and manipulating HTML and XML.

debug-js/debug

Lightweight debugging utility for Node.js and the browser

juliangruber/enstore

In-memory persistence for streams

segmentio/is-url

Check whether a string is a URL.

jasonpincin/stream-to-string

Pipe a stream into a string, collect value with callback or promise

lapwinglabs/x-ray-crawler

x-ray's crawler

lapwinglabs/x-ray-parse

Selector parser

maxogden/concat-stream

writable stream that concatenates strings or binary data and calls a callback with the result

nickmerwin/node-coveralls

takes json-cov output into stdin and POSTs to coveralls.io

gotwarlost/istanbul

Yet another JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests. Supports all JS coverage use cases including unit tests, server side functional tests and browser tests. Built for scale