node-crawler
https://github.com/bda-research/node-crawler
JavaScript
Web Crawler/Spider for NodeJS + server-side jQuery ;-)
Triage Issues!
When you volunteer to triage issues, you'll receive an email each day with a link to an open issue that needs help in this project. You'll also receive instructions on how to triage issues.
Triage Docs!
Receive a documented method or class from your favorite GitHub repos in your inbox every day. If you're really pro, receive undocumented methods or classes and supercharge your commit history.
JavaScript not yet supported0 Subscribers
Add a CodeTriage badge to node-crawler
Help out
- Issues
- Unknow error when request for a binary stream?
- cannot get section node
- Abort crawling
- Inconsistent variable names across examples
- Some requests (stream based) never ends and block the queue
- Returning data from callback
- Update documentation to reflect that node-crawler is based on request
- Integrate with ProxyCrawl crawler
- pause / resume?
- Documentation unclear; CRAWLER Error Error: incorrect header check when fetching
- Docs
- JavaScript not yet supported