Cylc (“silk”) is a workflow engine for cycling systems - it orchestrates
complex distributed suites of interdependent cycling tasks that may
continue indefinitely (Cylc dynamically generates ongoing workflows).
There are several reasons why tasks might need to be cycled:
In real time environmental forecasting systems, new forecasts are initiated
at regular intervals as new driving data comes in.
You may need to split long model runs into many smaller runs, with
associated processing tasks for each smaller run.
Cylc was originally developed for operational environmental forecasting at
NIWA by Dr Hilary
Oliver, and is now an Open
Source collaboration involving NIWA, Met Office,
and others. It is available under the GPL v3
Suites are defined in a human-readable config
file format - so you can use software development power tools for suite
development (see here
for why this is a
Configure scheduling with an efficient graph
description notation, and task runtime properties in an efficient inheritance
hierarchy (to factor out all commonality).
Cylc dynamically generates new workflow
without being constrained by a global cycle loop. Cycles interleave
naturally, suites flow around failed or delayed tasks, and they adapt to
insertion and removal of tasks.
Cylc has low admin overhead and a small
security footprint, because - as a distributed system - there is no central
server process to manage workflows for all users.
Plus many other
to support both clock-triggered real time and free-flow
metascheduling in research and operational environments.
Please let us know if your organization
should be included in the list of Cylc users.
Here’s the DOI to use when citing Cylc:
See also publications and citations