Django Packages : Workers, Queues, and Tasks

Workers, Queues, and Tasks

Apps and services for running your Django apps in various parallel environments to the web server, freeing your project architecture from the HTTP-based service lifecycle:

* Construct daemons to batch-process large datasets
* Call functions on a recurring schedule
* Respond to system events
* Sync with external data sources without beach-balling

Use existing components (models, templates, &c) and your favorite Django design patterns in these new contexts as you need -- while including Pythonic batteries from disciplines beyond web development.

Features currently being evaluated

Feature	Description
Synopsis	Packages' organizational style and method of Django integration
Redis Queue
Disque
Django ORM support
ZeroMQ
uWSGI	Native support for uWSGI builtin spooler

Package	celery	huey	Dramatiq	Django-celery	Trigger Happy	django-rq	Django Q	jobtastic	django-background-tasks	django-celery-results	django-apscheduler	django-ztask	Celery Progress Bars for Django	django_dramatiq	django-celery-monitor	django-rq-dashboard	django-pq	django-kitsune	django-chronograph	django-carrot	django-signalqueue	django-task	django-rq-jobs	django-uwsgi-taskmanager	autotask	django-admin-rq	django-toosimple-q	djcall	Django Task API	modelqueue	django-tasker	django-uwsgi-spooler

Package	celery	huey	Dramatiq	Django-celery	Trigger Happy	django-rq	Django Q	jobtastic	django-background-tasks	django-celery-results	django-apscheduler	django-ztask	Celery Progress Bars for Django	django_dramatiq	django-celery-monitor	django-rq-dashboard	django-pq	django-kitsune	django-chronograph	django-carrot	django-signalqueue	django-task	django-rq-jobs	django-uwsgi-taskmanager	autotask	django-admin-rq	django-toosimple-q	djcall	Django Task API	modelqueue	django-tasker	django-uwsgi-spooler
Description	Distributed Task Queue (development branch)	a little task queue for python	A fast and reliable background task processing library for Python 3.	Old Celery integration project for Django	:snake: Trigger Happy - The bus :bus: for your internet services	A simple app that provides django integration for RQ (Redis Queue)	A multiprocessing distributed task queue for Django	Make your user-responsive long-running Celery jobs totally awesomer.	A database-backed work queue for Django	Celery result back end with django	APScheduler for Django	A simple framework for backgrounding tasks with Django, built on top of ZeroMQ.	Drop in, configurable, dependency-free progress bars for your Django/Celery applications.	A Django app that integrates with Dramatiq.	Celery Monitoring for Django	A dashboard for managing RQ in the Django admin	[UNMAINTAINED] A task queue based on the elegant python RQ but with a django postgresql backend.	Host server monitoring app for Django Admin. Allows to schedule checks on hosts and notify results to administrators by mail.		A lightweight task queue for Django using RabbitMQ	Truly asynchronous Django signals!	A Django app to run new background tasks from either admin or cron, and inspect task history from admin	Provides scheduled jobs management from the Django Admin using Django-RQ	Django application to monitor and manage long and/or recurring tasks through uWSGI.		Django admin rq is a django package that creates a 4 step (form, preview, main, complete) asynchronous workflow from a ...	Simplistic task queue and cron-like scheduler for Django	MOVED to yourlabs.io/oss/djcall !! Spooled and Cron generic task models, like django-q but for uWSGI builtin features	A REST API and JavaScript library for managing background tasks in Django projects	Task queue based on Django models.	Dead simple async task queue. Stores tasks in database.	HALTED IN FAVOR OF yourlabs/django-call Task model to get the most out of uWSGI's spooler with minimal effort with CRUDLFA+ ...
Category	Other	App	App	App	App	App	App	Other	App	App	App	App	Other	App	App	App	App	App	App	App	App	App	App	App	App	App	App	App	App	Other	App	App
# Using This	7	6	0	78	5	20	17	1	1	0	0	3	0	0	1	2	0	1	1	1	1	1	1	1	1	1	0	0	0	0	0	0
Python 3?
Development Status	Production/Stable	Beta	Unknown	Production/Stable	Production/Stable	Beta	Production/Stable	Beta	Production/Stable	Production/Stable	Beta	n/a	Unknown	Unknown	Production/Stable	Alpha	Alpha	Unknown	Unknown	Production/Stable	Beta	Production/Stable	Beta	Beta	Beta	Beta	Alpha	Pre-Alpha	Unknown	Alpha	Beta	Planning
Last updated	Sept. 19, 2020, 2:40 p.m.	Aug. 24, 2020, 8:24 a.m.	Sept. 11, 2020, 3:07 a.m.	Feb. 26, 2020, 11:59 a.m.	May 1, 2020, 8:07 a.m.	July 26, 2020, 8:38 p.m.	Sept. 6, 2020, 5:48 a.m.	May 8, 2019, 9:48 a.m.	Dec. 23, 2019, 9:34 a.m.	June 9, 2020, 1:39 a.m.	Sept. 11, 2020, 6:36 a.m.	June 5, 2012, 7:10 p.m.	Sept. 8, 2020, 3:29 a.m.	Sept. 18, 2020, 3:16 a.m.	Jan. 3, 2018, 11:46 a.m.	June 13, 2017, 11:10 a.m.	Oct. 12, 2014, 10:39 p.m.	Nov. 6, 2014, 9:54 a.m.	March 5, 2013, 11:39 p.m.	April 14, 2019, 3:56 p.m.	Jan. 20, 2014, 9:44 p.m.	Nov. 12, 2019, 12:46 p.m.	July 29, 2015, 8:34 a.m.	Sept. 7, 2020, 9:45 a.m.	Aug. 11, 2016, 9:01 a.m.	Nov. 2, 2017, 9:15 a.m.	Sept. 6, 2020, 6:02 p.m.	Sept. 20, 2018, 11:54 a.m.	July 30, 2020, 9:28 a.m.	April 19, 2018, 7:49 p.m.	April 17, 2019, 9:31 a.m.	Aug. 2, 2018, 9:35 a.m.
Version	4.4.7	2.3.0	1.9.0	3.3.1	1.7.0	2.3.2	1.3.3	2.1.1	1.2.5	1.2.1	0.4.2	n/a	0.0.12	0.9.1	1.1.2	0.3.3	0.3.2	0.2.3	0.3.1	1.5.0	0.5.1	1.5.0	0.1.5.1	2.2.7	0.5.4	0.2.0	0.0.3	0.2.0	1.2.2	1.0.3	0.2.60	0.0.0
Repo	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Github	Bitbucket	Github	Github	Github	Github	Github	Bitbucket	Github	Github	Github	Github	Github	Github	Github
Commits
Stars	15813	3168	2196	1389	1344	1270	1077	630	453	286	276	224	187	153	109	86	69	64	62	58	54	29	27	22	8	6	2	2	1	1	1	n/a
Repo Forks	3817	283	145	450	129	214	159	55	86	119	61	31	50	32	48	32	7	12	35	11	4	4	6	2	5	2	n/a	n/a	2	1	n/a	n/a
Participants	ask thedrow auvipy mher ionelmc piotrmaslanka nvie mchataigner steeve PMickael more...	coleifer jbaiter antwan camilonova blablacio logannc adamchainz gl3nn hgdeoro jedie more...	Bogdanp rpkilby ryansm1 takhs91 davidt99 bersace gdvalle najamansari benekastah synweap15 more...	ask auvipy vytisb ionelmc diegueus9 thedrow jezdez jasonbaker iamjstates piotrbulinski more...	foxmask koalie CrazyLlama Logan1x h-chauhan adrienlachaize philippeowagner adriancarayol pouete pohzipohzi more...	selwin lechup SeiryuZ jeanphix dependabot-preview[bot] Krukov acjay hendi therefromhere lucastamoios more...	Koed00 Eagllus jmcvetta danielwelch janneronkko timomeara yannpom benjaoming valentinogagliardi zws2014 more...	winhamwr jlward deti midnightlynx rhunwicks caffodian kylegibson abbasovalex georgemarshall singingwolfboy more...	philippeowagner lilspikey jvamvas yannik-ammann walterrenner Luthaf Zagrebelin johnsensible december1981 cedricfarinazzo more...	ask auvipy thedrow jaylynch arnau126 maxmalysh liquidpele wardal adamchainz millin more...	jcass77 jarekwg sallyruthstruik soon Jedore bnjmnhndrsn countable D4VIDB2 daniel-butler joaodaher more...	themartorana chemt jallum	czue OmarWKH EJH2 oisinBates ashutoshsingh0223 brunostuyts ivn86 mcrot Madjura eeintech more...	Bogdanp jcass77 rpkilby MattBlack85 rakanalh ramonsaraiva Sovetnikov aericson funkybob LucidDan more...	jezdez pyup-bot janezkranjc mhsekhavat	brutasse olarin ihuro jleclanche spapas wastrachan nsbk sbussetti	bretth saulshanabrook	raulgarreta amartinezuy	wnielson	chris104957 ewsterrenburg	fish2000 bitdeli-chef	morlandi	Koed00 likeon	gabrielelucci guglielmo pauloxnet	kbr	Proper-Job	olivierdalang	jpic	nikmolnar dependabot[bot] brendan-ward	grantjenks	wooyek	jpic
Documentation			N/A			N/A		N/A			N/A	N/A		N/A		N/A	N/A	N/A			N/A		N/A			N/A	N/A	N/A			N/A	N/A
Synopsis				Provides a decorator syntax for defining tasks, which run in worker instances of your app, and a backend-agnostic protocol for serializing data between these workers and the web process.		A simple app that provides django integration for RQ (http://python-rq.org)	Multiprocessing worker pool Django native Encrypted and compressed task packages Scheduled tasks Result hooks Result and Failure database Redis, DIsque, IronMQ , SQS, MongoDB or ORM broker Django Admin Multi Cluster Monitor Worker cpu affinity					A django-friendly interface to ZeroMQ -- http://www.zeromq.org/ Django-ztask offers a syntactically celery-esque decorator API for defining tasks. Function objects decorated for ztask have self-evidently named curried sub-functions as attributes, such that e.g. `yodogg(opts)` can be simply changed to`yodogg.async(opts)` to dispatch and run it asynchronously. ZeroMQ is an amazing socket framework/protocol/convention, but django-ztask is written against it in such a manner such that function arguments must be able to survive pickling. The django-signalqueue API (which does some pickling itself, under-the-hood) exposes serialization methods _qua_ type, allowing such types as model instances to be transparently passed; tasks in ztask don't provide anything like this -- you have to hack your own object reference/serialization scheme or stick to the dict/list/tuple collection primitives; this is the library's biggest pain-point.			Monitor Celery tasks and workers in the admin. Formerly included in django-celery.			A Django Admin app to perform host server monitoring. A control panel will be added to the Admin in order to configure hosts, checks and monitor check results. Notification rules can be defined to notify administrator users by mail.	Provides a master hook for the Cron daemon. Periodic tasks can then be defined within the Django admin panel -- no need to mess around with fragile crontab syntax with superuser privileges. Run any Django ./manage.py commands as needed: every minute, every day, or whatever in between.		Define and dispatch asynchronous Django signals, using the same semantics as their inbuilt synchronous counterparts. Originally developed for ImageKit -- https://github.com/jdriscoll/django-imagekit -- to decouple image pipeline processing from view functions. Offers a easily implemented engine API, an extensible JSON-based interprocess protocol; off-the-shelf backends leverage the characteristics of several Redis and relational datastructures.	Purposes: - create async tasks either programmatically or from admin - monitor async tasks from admin - log all tasks in the database for later inspection - optionally save task-specific logs in a TextField and/or in a FileField Based on: - django-rq	Scheduled jobs management Django Admin plugin using Django-RQ Supports once, daily, weekly, monthly, quarterly and yearly scheduled tasks. Jobs can be limited to a set number of repeats. Both regular tasks and management commands can be scheduled. All tasks are queued to RQ , enabling a large payload schedule without delays.	# Features - Start and stop your tasks via admin - Schedule tasks - Plan tasks as cron items - Check or download the generated reports/logs -Simply write a standard Django Command class (your app doesn't need to interact with Django uWSGI Taskmanager)				Django models to wrap around uWSGI native spooler and cron features.		ModelQueue is an Apache2 licensed task queue based on Django models.
Redis Queue
Disque
Django ORM support
ZeroMQ
uWSGI