Scraping and Crawling Tools

Features currently being evaluated

Feature Description
Scrapy
Celery
Django versions

Filter results

Package django-dynamic-scraper django-scraper
Package django-dynamic-scraper django-scraper
Description Creating Scrapy scrapers via
the Django admin interface
Django application which
crawls and downloads online
content following instructions
Category App App
# Using This 4 0
Python 3?
Development Status Beta Unknown
Last updated June 25, 2021, 7:30 a.m. Oct. 27, 2015, 7:46 a.m.
Version 0.13.3 0.3.8
Repo GitHub GitHub
Commits
Stars 1070 18
Repo Forks 315 13
Participants holgerd77
kas
ryokamiya
pitervergara
zzpwelkin
DeanSherwin
bashu
joetric
rakanalh
zniper
Documentation
Scrapy  , 0.16 (Scrapy 0.18+ not yet supported)
Celery  , 3.0+ (Celery 3.1+ not yet tested)
Django versions  , 1.4+ (also tested against Django 1.5, Django 1.6+ not yet supported)
Search Weight Package Description Last PyPI release: Repo Forks Stars
{{ item.weight / max_weight * 100 | number:0 }}% {{ item.title }} Grid: {{ item.description }} {{ item.last_released | date: 'mediumDate' }} N/A {{ item.repo_forks }} N/A {{ item.repo_watchers }} N/A