IronMQ broker and IronCache result store for Celery

• Cloud message queue and caching services
• No servers
• No maintenance
• Scales effortlessly

Installation

Note: We recommend using virtualenv to avoid any dependency issues.

For IronMQ support, you'll need the iron_celery library:

$ pip install iron_celery

As well as an Iron.io account. Sign up for free at Iron.io.

Configuration

First, you'll need to import the iron_celery library right after you import Celery, for example:

from celery import Celery
import iron_celery

celery = Celery('mytasks', broker='ironmq://', backend='ironcache://')

To use IronMQ, the broker URL should be in this format:

BROKER_URL = 'ironmq://ABCDEFGHIJKLMNOPQRST:ZYXK7NiynGlTogH8Nj+P9nlE73sq3@'

where the URL format is:

ironmq://project_id:token@

The project_id and token are for your Iron.io account, you can find these in the Iron.io HUD. You must remember to include the "@" at the end.

The login credentials can also be set using the environment variables :envvar:IRON_TOKEN and :envvar:IRON_PROJECT_ID, which are set automatically if you use the IronMQ Heroku add-on. And in this case the broker url may only be:

ironmq://

Clouds

The default cloud/region is AWS us-east-1. You can choose the IronMQ Rackspace cloud by changing the URL to::

ironmq://project_id:[email protected]

Results

You can store results in IronCache with the same Iron.io credentials, just set the results URL with the same syntax as the broker URL, but changing the start to ironcache:

CELERY_RESULT_BACKEND = 'ironcache:://project_id:token@'

This will default to a cache named "Celery", if you want to change that:

ironcache:://project_id:token@/awesomecache

Django - Using iron_celery with Django

Setup celery with Django as you normally would, but add import iron_celery and set the BROKER_URL to the URL's above. For example, at the top of your Django settings.py file:

# NOTE: these must go before djcelery.setup_loader() line
BROKER_URL = 'ironmq://project_id:token@'
CELERY_RESULT_BACKEND = 'ironcache://project_id:token@'

import djcelery
import iron_celery

djcelery.setup_loader()

You can test it by going through the First Steps with Django guide in the Celery documentation.

Troubleshooting

If you are using countdown or eta, make sure to use iron_mq_timeout parameter as well (otherwise message will be returned to the IronMQ queue before Celery will ack it).

mytask.apply_async(args = ["Hello"], countdown = 60, iron_mq_timeout = 90)