Summary Of Major Changes
Cylc now uses more widely understood terms for several core concepts.
Cylc 7 Term
Cylc 8 Term
Note the configuration filename is now
Cylc 7 Compatibility Mode
Continuing to use the old
suite.rc filename triggers a backward
compatibility mode in Cylc 8 which supports Cylc 7
workflow configurations out of the box, with
some caveats. However, to future-proof
your workflow and take full advantage of Cylc 8 you should upgrade to Cylc 8 syntax.
Upgrading To Cylc 8
There have been some configuration changes at Cylc 8.
To upgrade your Cylc 7 suite to a Cylc 8 workflow, first make sure it validates
in Cylc 7 without any warnings, then rename the workflow configuration file
flow.cylc, then run
cylc validate in Cylc 8 and
take action on any warnings.
Validation warnings use a shorthand notation
to refer to nested configuration settings on a single line, like this:
New Web and Terminal UIs
Major Changes: Cylc 8 UIs
At Cylc 8, there are two UIs available to monitor and control your workflows:
a terminal UI application
cylc tui <workflow_id>
a web based UI application (requires Cylc UI Server)
cylc run <suite_name> at Cylc 7 has become
cylc play <workflow_id>.
At Cylc 8, use
cylc pause <workflow_id> to pause a workflow, halting all job
submission. To restart the workflow, use
cylc play <workflow_id>.
To start a fresh run, use
cylc install and play it safely in the new run
cylc hold and
cylc release pause and release individual tasks.)
Tasks are nodes in the abstract workflow graph, representing applications to run at the appropriate point in the workflow. A job is the script (and subsequent process) submitted by Cylc to actually run the application. A task can have multiple jobs as the result of automatic retries or manual re-triggering.
The 13 task/job states in Cylc 7 have been simplified to 8. Tasks and jobs have been separated and states of both can be viewed in the GUI.
For more information, see Task/Job States.
Optional and Required Task Outputs
Major Changes:Graph branching, optional outputs and suicide triggers
User Guide:Required Outputs
User Guide:Optional Outputs
By default, all Cylc 8 tasks are required to succeed - i.e., success is a required output. Otherwise they will be marked as incomplete tasks needing user intervention. In a workflow with incomplete tasks, if there is nothing left to do, the scheduler will stall rather than shut down.
Cylc 7 was aware of individual job hosts - one selected a host using:
Cylc 8 is aware of sets of host settings called
[job] platforms. To choose a platform for a task use
Hosts of a platform must share a file system and job runner: If one host is unavailable Cylc 8 can use other hosts on the same platform to interact with jobs.
The same hosts can belong to multiple platforms, for example you might be able to use the same host to launch both background and Slurm jobs.
Cylc 8 will pick a sensible platform for your Cylc 7 settings, These deprecated settings will be removed in a future release.
Cylc 8 supports workflow installation.
For users of Rose, this replaces the functionality of
Major Changes: Moving to Cylc Install
$ pwd ~/cylc-src/demo $ ls flow.cylc $ cylc install INSTALLED demo/run1 from /home/oliverh/cylc-src/demo $ cylc play demo ... demo/run1: oliver.niwa.local PID=6702
By default, run numbers increment with each install.
When the first job runs on a remote platform (after start-up, or after a
cylc reload), a
remote initialization process is triggered to install workflow files there.
Workflows can be deleted with
cylc clean - see Removing Workflows. This
rose suite-clean functionality.
There have been fundamental changes to the architecture of Cylc. You can read about the new system design here Architecture.
The scheduling algorithm has been changed, more information is available: Scheduling Algorithm.
There are an assortment of other features implemented at Cylc 8. Some noteworthy minor changes include:
- Runahead Limit
The default runahead limit has been increased from three cycles to five.
- Time Zones
- Job Scripts
All user-defined task scripting now runs in a subshell, so you can safely switch Python environments inside tasks without affecting Cylc. Further information is available in the User Guide: Job Scripts.
Cylc 8 (and its package dependencies) is now available from Conda Forge and PyPI for installations into a Python 3 virtual environment.
- Remote usernames
If usernames differ on remote job hosts they must now be configured using an SSH config file rather than the via Cylc 7
[remote]ownerconfiguration. See Remote Usernames.