Workers, Queues, and Tasks

Apps and services for running your Django apps in various parallel environments to the web server, freeing your project architecture from the HTTP-based service lifecycle:

* Construct daemons to batch-process large datasets
* Call functions on a recurring schedule
* Respond to system events
* Sync with external data sources without beach-balling

Use existing components (models, templates, &c) and your favorite Django design patterns in these new contexts as you need -- while including Pythonic batteries from disciplines beyond web development.

Features currently being evaluated

FeatureDescription
Synopsis Packages' organizational style and method of Django integration
Redis Queue
Disque
Django ORM support
ZeroMQ
uWSGI Native support for uWSGI builtin spooler
Package celery huey Dramatiq Django-celery Trigger Happy django-rq Django Q jobtastic django-background-tasks django-celery-results django-apscheduler django-ztask Celery Progress Bars for Django django_dramatiq django-celery-monitor django-rq-dashboard django-pq django-kitsune django-chronograph django-carrot django-signalqueue django-task django-rq-jobs django-uwsgi-taskmanager autotask django-admin-rq djcall django-toosimple-q Django Task API django-tasker modelqueue django-uwsgi-spooler
Package celery huey Dramatiq Django-celery Trigger Happy django-rq Django Q jobtastic django-background-tasks django-celery-results django-apscheduler django-ztask Celery Progress Bars for Django django_dramatiq django-celery-monitor django-rq-dashboard django-pq django-kitsune django-chronograph django-carrot django-signalqueue django-task django-rq-jobs django-uwsgi-taskmanager autotask django-admin-rq djcall django-toosimple-q Django Task API django-tasker modelqueue django-uwsgi-spooler
Description Distributed Task Queue
(development branch)
a little task queue for python A fast and reliable background
task processing library for
Python 3.
Old Celery integration project
for Django
:snake: Trigger Happy - The
bus :bus: for your internet
services
A simple app that provides
django integration for RQ
(Redis Queue)
A multiprocessing distributed
task queue for Django
Make your user-responsive
long-running Celery jobs
totally awesomer.
A database-backed work queue
for Django
Celery result back end with
django
APScheduler for Django A simple framework for
backgrounding tasks with
Django, built on top of
ZeroMQ.
Drop in, configurable,
dependency-free progress bars
for your Django/Celery
applications.
A Django app that integrates
with Dramatiq.
Celery Monitoring for Django A dashboard for managing RQ in
the Django admin
[UNMAINTAINED] A task queue
based on the elegant python RQ
but with a django postgresql
backend.
Host server monitoring app for
Django Admin. Allows to
schedule checks on hosts and
notify results to
administrators by mail.
A lightweight task queue for
Django using RabbitMQ
Truly asynchronous Django
signals!
A Django app to run new
background tasks from either
admin or cron, and inspect
task history from admin
Provides scheduled jobs
management from the Django
Admin using Django-RQ
Django application to monitor
and manage long and/or
recurring tasks through uWSGI.
Django admin rq is a django
package that creates a 4 step
(form, preview, main,
complete) asynchronous
workflow from a ...
MOVED to
yourlabs.io/oss/djcall !!
Spooled and Cron generic task
models, like django-q but for
uWSGI builtin features
Simplistic task queue and
cron-like scheduler for Django
A REST API and JavaScript
library for managing
background tasks in Django
projects
Dead simple async task queue.
Stores tasks in database.
Task queue based on Django
models.
HALTED IN FAVOR OF
yourlabs/django-call Task
model to get the most out of
uWSGI's spooler with minimal
effort with CRUDLFA+ ...
CategoryOtherAppAppAppAppAppAppOtherAppAppAppAppOtherAppAppAppAppAppAppAppAppAppAppAppAppAppAppAppAppAppOtherApp
# Using This76078520161100300120111111111000000
Python 3?
Development Status Production/Stable Beta Unknown Production/Stable Production/Stable Beta Production/Stable Beta Production/Stable Production/Stable Beta n/a Unknown Unknown Production/Stable Alpha Alpha Unknown Unknown Production/Stable Beta Production/Stable Beta Beta Beta Beta Pre-Alpha Alpha Unknown Beta Alpha Planning
Last updated July 8, 2020, 11:32 p.m. June 30, 2020, 9:08 a.m. July 11, 2020, 1:32 a.m. Feb. 26, 2020, 11:59 a.m. May 1, 2020, 8:07 a.m. July 2, 2020, 7:40 p.m. July 8, 2020, 5:20 p.m. May 8, 2019, 9:48 a.m. Dec. 23, 2019, 9:34 a.m. June 9, 2020, 1:39 a.m. July 5, 2020, 9:23 p.m. June 5, 2012, 7:10 p.m. June 10, 2020, 3:18 a.m. April 12, 2020, 7:24 a.m. Jan. 3, 2018, 11:46 a.m. June 13, 2017, 11:10 a.m. Oct. 12, 2014, 10:39 p.m. Nov. 6, 2014, 9:54 a.m. March 5, 2013, 11:39 p.m. April 14, 2019, 3:56 p.m. Jan. 20, 2014, 9:44 p.m. Nov. 12, 2019, 12:46 p.m. July 29, 2015, 8:34 a.m. July 9, 2020, 10:54 a.m. Aug. 11, 2016, 9:01 a.m. Nov. 2, 2017, 9:15 a.m. Sept. 20, 2018, 11:54 a.m. Feb. 10, 2020, 10:55 a.m. June 30, 2020, 6:14 p.m. April 17, 2019, 9:31 a.m. April 19, 2018, 7:49 p.m. Aug. 2, 2018, 9:35 a.m.
Version4.4.62.2.01.9.03.3.11.7.02.3.21.3.22.1.11.2.51.2.10.3.0n/a0.0.100.9.11.1.20.3.30.3.20.2.30.3.11.5.00.5.11.5.00.1.5.12.2.30.5.40.2.00.2.00.0.31.2.20.2.601.0.30.0.0
RepoGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubGithubBitbucketGithubGithubGithubGithubGithubBitbucketGithubGithubGithubGithubGithubGithubGithub
Commits
Stars153843051211113731337123610226314272702412241661311068669636256542827228621111n/a
Repo Forks373327713944712921215255851125732473246307123511446252n/an/a2n/a1n/a
Participantsask
thedrow
auvipy
mher
ionelmc
piotrmaslanka
nvie
mchataigner
steeve
PMickael
more...
coleifer
jbaiter
antwan
camilonova
blablacio
logannc
adamchainz
gl3nn
hgdeoro
jedie
more...
Bogdanp
rpkilby
ryansm1
takhs91
davidt99
bersace
gdvalle
najamansari
benekastah
synweap15
more...
ask
auvipy
vytisb
ionelmc
diegueus9
thedrow
jezdez
jasonbaker
iamjstates
piotrbulinski
more...
foxmask
koalie
CrazyLlama
Logan1x
h-chauhan
adrienlachaize
philippeowagner
adriancarayol
pouete
pohzipohzi
more...
selwin
lechup
SeiryuZ
jeanphix
dependabot-preview[bot]
Krukov
acjay
hendi
therefromhere
lucastamoios
more...
Koed00
Eagllus
jmcvetta
danielwelch
janneronkko
yannpom
benjaoming
valentinogagliardi
zws2014
Urth
more...
winhamwr
jlward
deti
midnightlynx
rhunwicks
caffodian
kylegibson
abbasovalex
georgemarshall
singingwolfboy
more...
philippeowagner
lilspikey
jvamvas
yannik-ammann
walterrenner
Luthaf
Zagrebelin
johnsensible
december1981
cedricfarinazzo
more...
ask
auvipy
thedrow
jaylynch
arnau126
maxmalysh
liquidpele
wardal
adamchainz
millin
more...
jarekwg
sallyruthstruik
jcass77
soon
Jedore
bnjmnhndrsn
countable
D4VIDB2
daniel-butler
joaodaher
more...
themartorana
chemt
jallum
czue
EJH2
oisinBates
ashutoshsingh0223
brunostuyts
ivn86
mcrot
Madjura
eeintech
robertschulze
Bogdanp
jcass77
rpkilby
MattBlack85
rakanalh
ramonsaraiva
Sovetnikov
aericson
funkybob
LucidDan
more...
jezdez
pyup-bot
janezkranjc
mhsekhavat
brutasse
olarin
ihuro
jleclanche
mmsepyx
spapas
wastrachan
nsbk
sbussetti
bretth
saulshanabrook
raulgarreta
amartinezuy
wnielsonchris104957
ewsterrenburg
fish2000
bitdeli-chef
morlandiKoed00
likeon
gabrielelucci
guglielmo
pauloxnet
kbrProper-Jobjpicolivierdalangnikmolnar
dependabot[bot]
brendan-ward
wooyekgrantjenksjpic
Documentation N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A
SynopsisProvides a decorator syntax for defining tasks, which run in worker instances of your app, and a backend-agnostic protocol for serializing data between these workers and the web process.A simple app that provides django integration for RQ (http://python-rq.org)Multiprocessing worker pool

Django native

Encrypted and compressed task packages

Scheduled tasks

Result hooks

Result and Failure database

Redis, DIsque, IronMQ , SQS, MongoDB or ORM broker

Django Admin

Multi Cluster Monitor

Worker cpu affinity
A django-friendly interface to ZeroMQ -- http://www.zeromq.org/

Django-ztask offers a syntactically celery-esque decorator API for defining tasks. Function objects decorated for ztask have self-evidently named curried sub-functions as attributes, such that e.g. `yodogg(**opts)` can be simply changed to`yodogg.async(**opts)` to dispatch and run it asynchronously.

ZeroMQ is an amazing socket framework/protocol/convention, but django-ztask is written against it in such a manner such that function arguments must be able to survive pickling. The django-signalqueue API (which does some pickling itself, under-the-hood) exposes serialization methods _qua_ type, allowing such types as model instances to be transparently passed; tasks in ztask don't provide anything like this -- you have to hack your own object reference/serialization scheme or stick to the dict/list/tuple collection primitives; this is the library's biggest pain-point.
Monitor Celery tasks and workers in the admin. Formerly included in django-celery.A Django Admin app to perform host server monitoring. A control panel will be added to the Admin in order to configure hosts, checks and monitor check results. Notification rules can be defined to notify administrator users by mail.Provides a master hook for the Cron daemon. Periodic tasks can then be defined within the Django admin panel -- no need to mess around with fragile crontab syntax with superuser privileges. Run any Django ./manage.py commands as needed: every minute, every day, or whatever in between.Define and dispatch asynchronous Django signals, using the same semantics as their inbuilt synchronous counterparts. Originally developed for ImageKit -- https://github.com/jdriscoll/django-imagekit -- to decouple image pipeline processing from view functions. Offers a easily implemented engine API, an extensible JSON-based interprocess protocol; off-the-shelf backends leverage the characteristics of several Redis and relational datastructures.Purposes:

- create async tasks either programmatically or from admin

- monitor async tasks from admin

- log all tasks in the database for later inspection

- optionally save task-specific logs in a TextField and/or in a FileField


Based on:
- django-rq
Scheduled jobs management Django Admin plugin using Django-RQ

Supports once, daily, weekly, monthly, quarterly and yearly scheduled tasks.
Jobs can be limited to a set number of repeats.
Both regular tasks and management commands can be scheduled.
All tasks are queued to RQ , enabling a large payload schedule without delays.
# Features

- Start and stop your tasks via admin

- Schedule tasks

- Plan tasks as cron items

- Check or download the generated reports/logs

-Simply write a standard Django Command class (your app doesn't need to interact with Django uWSGI Taskmanager)
Django models to wrap around uWSGI native spooler and cron features.ModelQueue is an Apache2 licensed task queue based on Django models.
Redis Queue
Disque
Django ORM support
ZeroMQ
uWSGI
Search WeightPackageDescriptionLast PyPI release:Repo ForksStars
{{ item.weight / max_weight * 100 | number:0 }}%{{ item.title }}Grid: {{ item.description }} {{ item.last_released | date: 'mediumDate' }} N/A {{ item.repo_forks }} N/A {{ item.repo_watchers }} N/A