Changes

This page contains a summary of significant changes across all Cylc components for each release.

For more detail see the component changelogs:


Cylc 8.5

More task information in the GUI (and Tui)

Task now icons in the GUI and TUI have badges for tasks waiting for:

  • retries retry icon

  • xtriggers xtriggered icon

  • wallclock wallclock icon

Xtriggers, wallclock triggers, retry triggers and task run mode are now shown in the information view:

../_images/info-view.xtriggers.png

Triggering Multiple Tasks

Cylc now respects the dependencies between tasks when triggering multiple tasks at the same time. This provides an easy way to re-run a group of tasks in order:

../_images/group-trigger.gif

This is generally easier than using a new flow.

Technical details:

  • Prerequisites on any tasks that are outside of the group of tasks being triggered are automatically satisfied.

  • Any tasks which have already run within the group will be automatically removed (i.e. cylc remove) to allow them to be re-run without intervention.

  • Any preparing, submitted or running tasks within the group will also be removed if necessary to allow the tasks to re-run in order.

Warning Triangles

The GUI now has warning triangles, these “light up” whenever warnings occur in the workflow.

  • Cylc logs warnings for various things such as task failures, stalled workflows, and command errors. You can see these in the workflow log files.

  • When warnings occur, the warning triangle will illuminate for the corresponding workflow.

  • Hover over the icon to reveal the warning.

  • Click on the icon to dismiss the warning.

  • If workflows are installed hierarchically, (e.g. the “development” and “live” groups in this example), warnings will trickle up the hierarchy to make it easier to monitor groups of workflows.

  • A log of all warnings can be found on the Dashboard page (last 10 warnings for each workflow).

../_images/warning-triangles.gif

“Ghost” Tasks

The GUI and Tui now present some tasks in grey:

../_images/ghost-tasks.png

These are “ghost” tasks, they indicate something is that isn’t presently there:

  • Waiting tasks that the scheduler hasn’t moved onto yet (i.e. inactive tasks that are waiting).

  • Tasks which have been removed (i.e. cylc remove).

  • Tasks which have been triggered in the none flow, so don’t influence the evolution of the workflow.

They appear in grey, if you click on these tasks, you will see the text “Flows: None”.

This helps to identify waiting active tasks (which aren’t ghosts). These are the tasks which Cylc is currently trying to schedule, but are waiting for something, typically:

  • A task prerequisite to be satisfied.

  • An xtrigger or ext-trigger to be satisfied.

  • Someone to resume them.

Compatibility Mode

Cylc 7 compatibility mode has been deprecated, it will be removed in Cylc 8.7.0 which is expected to be released some time in the first half of 2026.

Any workflows that are still using suite.rc files will need to be upgraded to flow.cylc before they are able to run with 8.7.0.

GUI Layout

The Cylc GUI now preserves tab layout between sessions.

When working on a workflow, we can open multiple tabs (tree, table, graph, etc) and tile them up in a layout. If you switch to another workflow, switch back, your layout will be restored.

However, with Cylc 8.4, if you refreshed the browser or opened the GUI in a new tab, the layout would be lost.

With Cylc 8.5, the layout will always be restored.

GUI Log View

The GUI now picks the default job log file to display based on the task state:

  • failed -> job.err

  • submit-failed -> job-activity.log

  • otherwise -> job.out

This speeds up the loading of the log file because the GUI doesn’t have to wait for a listing of available log files before picking one to view.

Additionally, the log view now has an auto scroll feature which follows the end of the file (useful for viewing the file whilst it is being written), and a scroll-to-top button.

Cylc Tui

cylc tui can now open log files in external tools.

../_images/tui-external-editor.gif

Configure your $EDITOR, $GEDITOR and $PAGER environment variables to change which tool is used.

Note

Make sure your configured command waits for the tool to be closed before exiting, e.g. use GEDITOR=gvim -f rather than EDITOR=gvim.

Additionally, Tui now displays task states and flow numbers in context menus for improved clarity / accessibility.

Cylc Reload

cylc reload now has a --global option to reload the global configuration while the server is running. This can be used to update platform settings for an in-progress workflow.

Cylc Set

The cylc set command can now be used to satisfy xtrigger prerequisites. For example if the task 2026/get_data is clock triggered, you might satisfy this prerequisite like so:

$ cylc set myworkflow//2026/get_data --pre xtrigger/wall_clock

The default behaviour of the cylc set command has also changed for tasks where success is optional, it will now set the succeeded output (and any other outputs that are required in the event of task success) which is more consistent with the behaviour for tasks where success is required.

Cycle Share Directory

A new per-cycle share directory has been added, share/cycle/<cycle>.

This directory is now automatically created and provides a convenient location for tasks to share cycle-specific data. See also Shared Task IO Paths.

This largely replicates the functionality of the Rose ROSE_DATAC environment variable, but does not require the use of rose task-env.

Cylc UI Server

The dependency stack of the Cylc UI Server (used to serve the Cylc GUI) has been overhauled.

This allows the UI Server to be installed with newer versions of Python then the old dependency stack allowed.

  • Previously the UI Server worked with Python 3.8-3.9.

  • It now works with Python 3.9 or higher.

This will likely bring efficiency improvements.

Additionally, the Cylc UI Server has now been configured to send “heartbeat” pings down its open websocket connections. This helps to prevent web proxies from closing Cylc GUI connections when workflows are idle, preventing erroneous GUI disconnects.

For more information see the Cylc configuration and the docs for the websocket_ping_interval and websocket_ping_timeout configurations in tornado.

Cylc 8.4

Cylc Components

cylc-flow:

8.4

cylc-ui:

2.7

cylc-uiserver:

1.6

cylc-rose:

1.5

rose:

2.4

Wrapper script fix for Cylc Hub

Added in version cylc-flow: 8.4.1

Fixed wrapper script $PATH override preventing selection of Cylc version when playing workflows in the GUI on Cylc Hub.

Important

Sites with existing wrapper scripts should update them with this change: https://github.com/cylc/cylc-flow/pull/6616/files.

Cylc Remove

cylc remove now allows removing tasks which are no longer active, making it look like they never ran. Removing a running task will now kill that task.

The cylc remove command now has the --flow option, allowing removal of a task from specific flows.

Tasks removed from all flows are retained in the workflow database with flow=none for provenance.

../_images/remove.gif

See also

See Remove Tasks to see an example of this feature in action.

Skip Mode

Tasks can now be run in “skip” mode where they complete their required outputs instantly. Tasks can be configured to run in skip mode either in the workflow configuration or by broadcasting.

../_images/skip-cycle.gui.gif

cylc set --out skip will set outputs for a task as if the task has run in skip mode. The task outputs delivered by skip mode can be customized using [runtime][<namespace>][skip]outputs or defaults to behaviour described in Task Outputs.

This can be used to skip a cycle or a task (for a full example see I want to Skip a cycle of tasks and allow the workflow to continue).

It may also be useful for Efficient Inter-Family Triggering and for Parameter Exclusion.

See also

Skip Mode

Trigger When Paused

Tasks can now be triggered and will run immediately while the workflow is paused.

../_images/trigger-while-paused.gif

See also

Pause The Workflow And Trigger Tasks One By One for an example of this feature.

EmPy Support Removed

Support for the EmPy template processor (an alternative to Jinja2) has been removed.

Info View

An info view has been added, displaying

../_images/cylc-gui-info-view.gif

Cylc 8.3

Cylc Components

cylc-flow:

8.3

cylc-ui:

2.5

cylc-uiserver:

1.5

cylc-rose:

1.4

rose:

2.3

Manually setting task outputs and prerequisites

At Cylc 8.3.0, the cylc set-outputs command has been replaced by the new cylc set command.

The cylc set-outputs command made it look like an output had been generated to downstream tasks, but did not update the task status to match. As a result, it was often necessary to use cylc remove in combination with cylc set-outputs.

The new cylc set command is able to directly set task outputs as if they had completed naturally, making the command more intuitive and avoiding the need for cylc remove. It can also set prerequisites, as if they were satisfied naturally.

For example, say there’s a failed task holding up your workflow and you want Cylc to continue as if the task had succeeded. Here are the interventions you would need to perform with Cylc 8.2 and 8.3 side-by-side.

Cylc 8.2 (set-outputs)

Cylc 8.3 (set)

# let downstream tasks run:
cylc set-outputs <task>
# remove the failed task:
cylc remove <task>
# tell Cylc that the task succeeded:
cylc set <task>
../_images/cylc-set-outputs.gif ../_images/cylc-set.gif

Tui

The Tui (terminal user interface) is a command line version of the Gui. You can use it to monitor and control your workflows.

There has been a major update to Tui at Cylc 8.3.0:

  • Larger workflows will no longer cause Tui to time out.

  • You can now browse all your workflows including stopped workflows.

  • You can monitor multiple workflows at the same time.

  • The workflow and job logs are now available from within Tui.

../_images/tui-1.gif

N-Window selector in the GUI

The n-window determines how much of a workflow is visible in the GUI / Tui.

You can change the n-window extent in the GUI with a toolbar button, to display more or less of the graph around current active tasks. This affects all GUI views equally, not just the graph view.

The n=0 window contains only the active tasks.

The n=1 window displays tasks out to one graph edge around the active tasks; n=2 out to two graph edges; and so on.

../_images/gui-n-window-selector.gif

Note

This is currently a per-workflow setting so changing the n-window in one browser tab will also change it in other browser tabs and Tui sessions.

Warning

Using high n-window values with complex workflows may have performance impacts.

Group by cycle point in the graph view

The graph view now has an option to group tasks by cycle point.

../_images/cylc-graph-group-by-cycle-point.png

Gantt View

The GUI now has a Gantt view option:

A picture of the Gantt view in operation.

Analysis View

New Analysis added - a layout which plots run times against cycle points.

A picture of the Time Series task analysis in operation.

Completion Expressions

When a task achieves a final status, its outputs are validated against a “completion expression” to ensure that it has produced all of its required outputs. If a task fails this validation check it is said to have “incomplete outputs” and will be retained in the n=0 window pending user intervention.

This completion expression is generated automatically from the graph. By default, tasks are expected to succeed, if you register any additional required output in the graph, then these must also be produced.

At Cylc 8.3.0 it is now possible to manually configure this completion expression for finer control. This is particularly useful for anyone using custom outputs.

For example, mytask must produce one of the outputs x or y to pass the completion expression configured here:

[runtime]
    [[mytask]]
        completion = succeeded and (x or y)
        [[[outputs]]]
            x = output-x
            y = output-y

For more information, see the reference for the [runtime][<namespace>]completion configuration.

Workflow State Triggers & Commands

Workflow state xtriggers and command now take Cylc universal IDs instead of separate arguments:

For example, you can (and should) now write:

# On the command line
- cylc workflow-state my-workflow --point 20240101 --task mytask --message "succeeded"
+ cylc workflow-state my-workflow//20240101/mytask:succeeded --triggers

# In the flow.cylc file
- my_xtrigger = workflow_state(
-     workflow="my-workflow",
-     task="mytask",
-     point="20240101",
-     message="succeeded"
- )
+ my_xtrigger = workflow_state('my-workflow//20240101/mytask:succeeded', is_trigger=True)

Important

The new workflow state trigger syntax can use either the trigger or message from trigger=message in [runtime][<namespace>][outputs].

The trigger and message are the same for the most common use cases (succeeded and started) but may differ for other outputs, namely custom outputs.

Note

The suite-state xtrigger has been reimplemented for compatibility with Cylc 7 workflows.


Cylc 8.2

Cylc Components

cylc-flow:

8.2

cylc-uiserver:

1.4

cylc-rose:

1.3

UI now remembers workspace tab layout

Added in version cylc-uiserver: 1.4.4

The UI now remembers the layout of your workspace tabs when you navigate away from that workflow. Note that this only applies per browser session.

../_images/ui-workspace-tabs.gif

Cylc ignores $PYTHONPATH

Cylc now ignores $PYTHONPATH to make it more robust to task environments which set this value. If you want to add to the Cylc environment itself, e.g. to install a Cylc extension, use a custom xtrigger, or event handler use $CYLC_PYTHONPATH.

Upgrade To The Latest Jupyter Releases

Added in version cylc-uiserver: 1.4.0

The Cylc UI Server has been updated to work with the latest releases of Jupyter Server and Jupyter Hub.

If you are utilising Cylc’s multi-user functionality then your configuration will require some changes to work with these releases.

See Authorizing Others to Access Your Workflows for more details

Added in version cylc-uiserver: 1.3.0

You can now configure the view which is opened by default when you navigate to a new workflow in the GUI. Navigate to the settings page to select your chosen view.

../_images/ui-view-selector.jpg

In the future we plan to support configuring a layout of multiple views and configuring certain options on those views.

Reload

Added in version cylc-flow: 8.2.0

When workflows are reloaded, (e.g. by cylc reload), Cylc will now pause the workflow and wait for any preparing tasks to be submitted before proceeding with the reload. Once the reload has been completed, the workflow will be resumed.

You can now see more information about the status of the reload in the workflow status message which appears at the top of the GUI and Tui interfaces.


Cylc 8.1

Cylc Components

cylc-flow:

8.1

cylc-uiserver:

1.2

cylc-rose:

1.1

Warning

Workflows started with Cylc 8.0 which contain multiple flows cannot be restarted with Cylc 8.1 due to database changes.

Analysis View

Added in version cylc-uiserver: 1.2.2

The web UI also has a new view for displaying task queue & run time statistics.

../_images/analysis_view.gif

Graph View

Added in version cylc-uiserver: 1.2.0

The web UI now has a graph view which displays a visualisation of a workflow’s graph:

../_images/cylc-graph.gif

Family & cycle grouping as well as the ability to view graphs for stopped workflows will be added in later releases.

Log View

Added in version cylc-uiserver: 1.2.0

The web UI now has a log view which displays workflow and job log files:

../_images/log-view-screenshot.png

Support for viewing more log files, syntax highlighting, searching and line numbers are planned for future releases.

Edit Runtime

Added in version cylc-uiserver: 1.2.0

The web UI now has a command for editing the [runtime] section of a task or family.

../_images/edit-runtime-screenshot.png

Any changes made are broadcast to the running workflow.

Combined Commands

Added in version cylc-flow: 8.1.0

Two new commands have been added as short-cuts for common working patterns:

cylc vip

Validate, install and plays a workflow, equivalent to:

cylc validate <path>
cylc install <path>
cylc play <id>
cylc vr

Validate and reinstall a workflow, then either: - reload the workflow if it is running. - restart the workflow if it is stopped.

../_images/vip-vr.gif

For more information see the command line help:

cylc vip --help
cylc vr --help

Bash Completion

Added in version cylc-flow: 8.1.0

Cylc now provides a high performance Bash completion script which saves you typing:

  • Cylc commands & options

  • Workflow IDs

  • Cycle points

  • Task names

  • Job numbers

../_images/cylc-completion.bash.gif

Installation instructions.


Cylc 8.0

Cylc Components

cylc-flow:

8.0

cylc-uiserver:

1.1

cylc-rose:

1.1

The first official release of Cylc 8.

For a summary of changes see the migration guide.