Home › Directory › Scraping Libraries › Crawlab
Crawlab
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
| Category | Scraping Libraries |
|---|---|
| Authentication | No auth |
| Pricing | Free |
| Website | https://github.com/crawlab-team/crawlab |
| Docs | https://github.com/crawlab-team/crawlab/tree/main/docs |
| Last checked | 2026-10-14 |
| As JSON | /api/v1/directory/crawlab.json |
Information is summarised in our own words and may be out of date; check the vendor's site for current terms and prices. Not affiliated with Crawlab. Suggest a correction.
More in Scraping Libraries
Scrapy
Mature Python framework for writing fast, scalable crawlers and extracting structured data.
Apache Nutch
Nutch is a highly extensible, highly scalable, matured, production-ready Web crawler which enables fine grained configuration and accomodates a wide variety of data acquisition tasks.
Colly
Elegant Scraper and Crawler Framework for Golang.
Goutte
Goutte, a simple PHP Web Scraper.