We run a Flask site under IIS on Windows, and for out-of-process tasks we use Celery. Celery has given us some problems under Windows, but for now we are satisfied running version 3.1.12, using RabbitMQ/AMQP as a back-end, which works under Windows.
The new version of Celery (4) has dropped support for Windows, so I'm looking for a viable alternative.
RQ seems a very nice task queue, but it also does not support Windows (bottom of the page)
I have seen some more, seemingly less popular task queues like:
But it's unclear if these support Windows and Flask. I'm wondering if anyone has experience running a Python task queue under Windows which works. Maybe one of the ones I mentioned, or an alternative.
It's not an option for us to run a Linux machine, because we have no experience administering Linux, and we have a lot of legacy stuff running that requires Windows.
I run Flask with Huey on Windows without any issues, admittedly only for development and testing. For production I use Flask/Huey on Linux servers. Both with the Redis back-end, Flask 0.12 and Huey 1.2.0 .
I use the factory pattern to create a specialised "cut down" version of a Flask app for specific use by Huey tasks. This version doesn't load blueprints or configure Flask-Admin as these aren't required in the Huey tasks.
Example code of
__init__.py in app folder.
App is a class extending from
def create_app(settings_override=None): app = App('app') if settings_override: app.config.from_object(settings_override) else: app.config.from_object(os.environ['APP_SETTINGS']) from .ext import configure_extensions configure_extensions(app, admin, load_modules=True) # REST import rest.api_v1 app.register_blueprint(api_v1_bp, url_prefix='/api/v1') # ... and more suff def create_huey_app(): app = App('huey app') app.config.from_object(os.environ['APP_SETTINGS']) from .ext import configure_extensions configure_extensions(app, admin=None, load_modules=False) return app
The idea of
configure_extensions is taken from Quokka CMS. Examine its
__init__.py and its extensions module to see how this is implemented. Note also how this project too creates a specific app (
create_celery_app) for use with the Celery task queue.
tasks.py. Note the use of
with app.app_context(): to create the Flask context. Now my functions have access to the extensions such as Flask-Mail, Flask-SqlAlchemy (db, models) etc.
@huey.task() def generate_transaction_documents_and_email(transaction_id): app = create_huey_app() with app.app_context(): reports.generate_transaction_documents_and_email(transaction_id) @huey.task() def send_email(subject, recipients, text_body, html_body, attachments=, cc=): app = create_huey_app() with app.app_context(): emails.send_email(subject, recipients, text_body, html_body, attachments, cc) @huey.periodic_task(crontab(minute='30')) def synchronize_mailing_list(): app = create_huey_app() if app.config['CREATESEND_SYNCHRONIZE']: _list_name = app.config['CREATESEND_LIST'] with app.app_context(): sync_delete_ar_subscribers(_list_name) sync_add_ar_subscribers(_list_name)