Cylc (“silk”) is a workflow engine for cycling systems - it orchestrates complex distributed suites of interdependent cycling tasks that may continue indefinitely (Cylc dynamically generates ongoing workflows).

There are several reasons why tasks might need to be cycled:

  • In real time environmental forecasting systems, new forecasts are initiated at regular intervals as new driving data comes in.

  • You may need to split long model runs into many smaller runs, with associated processing tasks for each smaller run.

Cylc was originally developed for operational environmental forecasting at NIWA by Dr Hilary Oliver, and is now an Open Source collaboration involving NIWA, Met Office, and others. It is available under the GPL v3 license.

Suites are defined in a human-readable config file format - so you can use software development power tools for suite development (see here for why this is a good thing).

Configure scheduling with an efficient graph description notation, and task runtime properties in an efficient inheritance hierarchy (to factor out all commonality).

Cylc dynamically generates new workflow without being constrained by a global cycle loop. Cycles interleave naturally, suites flow around failed or delayed tasks, and they adapt to insertion and removal of tasks.

Cylc has low admin overhead and a small security footprint, because - as a distributed system - there is no central server process to manage workflows for all users.

Plus many other features to support both clock-triggered real time and free-flow metascheduling in research and operational environments.

Please let us know if your organization should be included in the list of Cylc users.

Here’s the DOI to use when citing Cylc: DOI

See also publications and citations

DOWNLOAD cylc-7.4.0
Released

Clone cylc on GitHub

cylc-7.4.0,

Various minor changes and fixes, including: open job logs in your editor; `cylc gscan` improvements;...

cylc-7.3.0,

New Suite Design Guide (separate from the User Guide); stop and hold suites and groups...

cylc-7.2.1,

Fixed two recent GUI bugs (principally broken gcylc graph view in 7.2.0), and some other...

cylc-7.2.0,

New Jinja2 "assert" and "raise" functions for better error handling in suites, plus some minor...