Workflow Configuration

Cylc workflow configurations use the .cylc file format.

flow.cylc

Defines a Cylc workflow configuration.

After processing any embedded templating code (see Jinja2) the resultant raw flow.cylc file must be valid. See also The flow.cylc File for a descriptive overview of flow.cylc files, including syntax (Syntax).

Changed in version 8.0.0: The configuration file was previously named suite.rc, but that name is now deprecated. The suite.rc file name now activates Cylc 7 Compatibility Mode. Rename to flow.cylc to turn off compatibility mode.

[meta]

Path: flow.cylc[meta]

Metadata for this workflow.

Cylc defines and uses the terms “title”, “description” and “URL”. Users can define more terms, and use these in event handlers.

Example:

A user could define "workflow-priority". An event handler
would then respond to failure events in a way set by
"workflow-priority".

description

Path: flow.cylc[meta]description
Type: string

A multi-line description of the workflow.

It can be retrieved at run time with the cylc show command.

title

Path: flow.cylc[meta]title
Type: string

A single line description of the workflow.

It can be retrieved at run time with the cylc show command.

URL

Path: flow.cylc[meta]URL
Type: string

A web URL to workflow documentation.

The URL can be retrieved at run time with the cylc show command.

The template variable %(workflow)s will be replaced with the actual workflow ID.

Deprecated since version 8.0.0: The %(suite_name)s template variable is deprecated, please use %(workflow)s.

Example:

http://my-site.com/workflows/%(workflow)s/index.html

<custom metadata>

Path: flow.cylc[meta]<custom metadata>
Type: string

Any user-defined metadata item.

Like title, description and URL these can be passed to workflow event handlers to be interpreted according to your needs.

For example, a user could define an item called “workflow-priority”. An event handler could then respond to failure events in a way set by “workflow-priority”.

[scheduler]

Path: flow.cylc[scheduler]

Settings for the scheduler.

The default value is set in the global config: global.cylc[scheduler].

Note

Not to be confused with flow.cylc[scheduling].

Changed in version 8.0.0: This item was previously called [cylc]

UTC mode

Path: flow.cylc[scheduler]UTC mode
Type: boolean

If True, UTC will be used as the time zone for timestamps in the logs. If False, the local/system time zone will be used.

The default value is set in the global config: global.cylc[scheduler]UTC mode.

See also

To set a time zone for cycle points, see flow.cylc[scheduler]cycle point time zone.

allow implicit tasks

Path: flow.cylc[scheduler]allow implicit tasks
Type: boolean
Default: False

Allow tasks in the graph that are not defined in flow.cylc[runtime].

Implicit tasks are tasks without explicit definitions in flow.cylc[runtime]. By default, these are not allowed, as they are often typos. However, this setting can be set to True to allow implicit tasks. It is recommended to only set this to True if required during development/prototyping of a workflow graph, but set it to False after finishing the flow.cylc[runtime] section.

Cylc 7 compatibility mode

In Cylc 7 Compatibility Mode, implicit tasks are still allowed unless you explicitly set this to False, or unless a rose-suite.conf file is present (to maintain backward compatibility with Rose 2019).

New in version 8.0.0.

install

Path: flow.cylc[scheduler]install
Type: list

Configure custom directories and files to be installed on remote hosts.

Note

The following directories already get installed by default:

ana/: Rose ana analysis modules
app/: Rose applications
bin/: Cylc bin directory (added to PATH)
etc/: Miscellaneous resources
lib/: Cylc lib directory (lib/python added to PYTHONPATH for workflow config)

These should be located in the top level of your Cylc workflow, i.e. the directory that contains your flow.cylc file.

Directories must have a trailing slash. For example, to add the following items to your file installation:

~/cylc-run/workflow_x
|-- dir1/
|-- dir2/
|-- file1
`-- file2

[scheduler]
    install = dir/, dir2/, file1, file2

New in version 8.0.0.

cycle point format

Path: flow.cylc[scheduler]cycle point format
Type: cycle point format

Set the datetime format and precision that Cylc uses for cycle points in datetime cycling workflows.

See also

To alter the time zone used in the datetime cycle point format, see flow.cylc[scheduler]cycle point time zone.
To alter the number of expanded year digits (for years below 0 or above 9999), see flow.cylc[scheduler]cycle point num expanded year digits.

By default, Cylc uses a CCYYMMDDThhmmZ (Z in the special case of UTC) or CCYYMMDDThhmm±hhmm format for writing datetime cycle points, following the ISO 8601 standard.

You may use the isodatetime library’s syntax to set the cycle point format.

You can also use a subset of the strptime/strftime POSIX standard - supported tokens are %F, %H, %M, %S, %Y, %d, %j, %m, %s, %z.

If specifying a format here, we recommend including a time zone - this will be used for displaying cycle points only. To avoid confusion, we recommend using the same time zone as flow.cylc[scheduler]cycle point time zone.

The ISO 8601 extended datetime format (CCYY-MM-DDThh:mm) cannot be used, as cycle points are used in job-log and work directory paths where the “:” character is invalid.

Warning

The smallest unit included in the format sets the precision of cycle points in the workflow. If the precision is lower than the smallest unit in a graph recurrence, the workflow will fail. For example, if you set a format of CCYY, and have a recurrence R/2000/P8M, then both the first and second cycle points will be 2000, which is invalid.

cycle point num expanded year digits

Path: flow.cylc[scheduler]cycle point num expanded year digits
Type: integer
Default: 0

Enable negative years or years more than four digits long.

For years below 0 or above 9999, the ISO 8601 standard specifies that an extra number of year digits and a sign should be used. This extra number needs to be written down somewhere (here).

For example, if this extra number is set to 2, 00Z on the 1st of January in the year 10040 will be represented as +0100400101T0000Z (2 extra year digits used). With this number set to 3, 06Z on the 4th of May 1985 would be written as +00019850504T0600Z.

This number defaults to 0 (no sign or extra digits used).

cycle point time zone

Path: flow.cylc[scheduler]cycle point time zone
Type: cycle point time zone

Time zone to be used for datetime cycle points if not otherwise specified.

This time zone will be used for datetime cycle point dumping and inferring the time zone of cycle points that are input without time zones.

Time zones should be expressed as ISO8601 time zone offsets from UTC, such as +13, +1300, -0500 or +0645, with Z representing the special case of +0000 (UTC). Cycle points will be converted to the time zone you give and will be represented with this string at the end.

If not set, it will default to UTC (Z).

Cylc 7 compatibility mode

In Cylc 7 Compatibility Mode, it will default to the local/system time zone, rather than UTC.

The time zone will persist over reloads/restarts following any local time zone changes (e.g. if the workflow is run during winter time, then stopped, then restarted after summer time has begun, the cycle points will remain in winter time). Changing this setting after the workflow has first started will have no effect.

If you use a custom flow.cylc[scheduler]cycle point format, it is a good idea to set the same time zone here. If you specify a different one here, it will only be used for inferring timezone-less cycle points; cycle points will be displayed in the time zone from the cycle point format.

Caution

It is not recommended to write the time zone with a “:” (e.g. +05:30), given that the time zone is used as part of task output filenames.

See also

flow.cylc[scheduler]UTC mode

Changed in version 7.8.9/7.9.4: The value set here now persists over reloads/restarts after a system time zone change.

Changed in version 8.0.0: The default time zone is now Z instead of the local time of the first workflow start.

[main loop]

Path: flow.cylc[scheduler][main loop]

Configuration of main loop plugins for the scheduler.

The default value is set in the global config: global.cylc[scheduler][main loop].

For a list of built in plugins see Main Loop Plugins.

New in version 8.0.0.

[<plugin name>]

Path: flow.cylc[scheduler][main loop][<plugin name>]

Configure a main loop plugin.

The default value is set in the global config: global.cylc[scheduler][main loop][<plugin name>].

Note

Only the configured list of global.cylc[scheduler][main loop]plugins is loaded when a scheduler is started.

New in version 8.0.0.

interval

Path: flow.cylc[scheduler][main loop][<plugin name>]interval
Type: time interval

Interval at which the plugin is invoked.

The default value is set in the global config: global.cylc[scheduler][main loop][<plugin name>]interval.

New in version 8.0.0.

[events]

Path: flow.cylc[scheduler][events]

Configure the workflow event handling system.

The default value is set in the global config: global.cylc[scheduler][events].

handlers

Path: flow.cylc[scheduler][events]handlers
Type: list

Configure event handlers that run when certain workflow events occur.

The default value is set in the global config: global.cylc[scheduler][events]handlers.

This section configures workflow event handlers; see flow.cylc[runtime][<namespace>][events] for task event handlers.

Event handlers can be held in the workflow bin/ directory, otherwise it is up to you to ensure their location is in $PATH (in the shell in which the scheduler runs). They should require little resource to run and return quickly.

Template variables can be used to configure handlers. For a full list of supported variables see Workflow Event Template Variables.

handler events

Path: flow.cylc[scheduler][events]handler events
Type: list

Specify the events for which workflow event handlers should be invoked.

The default value is set in the global config: global.cylc[scheduler][events]handler events.

mail events

Path: flow.cylc[scheduler][events]mail events
Type: list

Specify the workflow events for which notification emails should be sent.

The default value is set in the global config: global.cylc[scheduler][events]mail events.

startup handlers

Path: flow.cylc[scheduler][events]startup handlers
Type: list

Handlers to run at scheduler startup.

The default value is set in the global config: global.cylc[scheduler][events]startup handlers.

Changed in version 8.0.0: This item was previously called startup handler.

Examples:

# configure a single event handler
startup handlers = echo foo

# provide context to the handler
startup handlers = echo %(workflow)s

# configure multiple event handlers
startup handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'

shutdown handlers

Path: flow.cylc[scheduler][events]shutdown handlers
Type: list

Handlers to run at scheduler shutdown.

The default value is set in the global config: global.cylc[scheduler][events]shutdown handlers.

Changed in version 8.0.0: This item was previously called shutdown handler.

Examples:

# configure a single event handler
shutdown handlers = echo foo

# provide context to the handler
shutdown handlers = echo %(workflow)s

# configure multiple event handlers
shutdown handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'

abort handlers

Path: flow.cylc[scheduler][events]abort handlers
Type: list

Handlers to run if the scheduler shuts down with error status due to a configured timeout or a fatal error condition.

The default value is set in the global config: global.cylc[scheduler][events]abort handlers.

Changed in version 8.0.0: This item was previously called aborted handler.

Examples:

# configure a single event handler
abort handlers = echo foo

# provide context to the handler
abort handlers = echo %(workflow)s

# configure multiple event handlers
abort handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'

workflow timeout

Path: flow.cylc[scheduler][events]workflow timeout
Type: time interval

Workflow timeout interval. The timer starts counting down at scheduler startup. It resets on workflow restart.

The default value is set in the global config: global.cylc[scheduler][events]workflow timeout.

New in version 8.0.0.

workflow timeout handlers

Path: flow.cylc[scheduler][events]workflow timeout handlers
Type: list

Handlers to run if the workflow timer times out.

The default value is set in the global config: global.cylc[scheduler][events]workflow timeout handlers.

New in version 8.0.0.

Examples:

# configure a single event handler
workflow timeout handlers = echo foo

# provide context to the handler
workflow timeout handlers = echo %(workflow)s

# configure multiple event handlers
workflow timeout handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'

abort on workflow timeout

Path: flow.cylc[scheduler][events]abort on workflow timeout
Type: boolean

Whether the scheduler should shut down immediately with error status if the workflow timer times out.

The default value is set in the global config: global.cylc[scheduler][events]abort on workflow timeout.

New in version 8.0.0.

stall handlers

Path: flow.cylc[scheduler][events]stall handlers
Type: list

Handlers to run if the scheduler stalls.

The default value is set in the global config: global.cylc[scheduler][events]stall handlers.

Changed in version 8.0.0: This item was previously called stalled handler.

Examples:

# configure a single event handler
stall handlers = echo foo

# provide context to the handler
stall handlers = echo %(workflow)s

# configure multiple event handlers
stall handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'

stall timeout

Path: flow.cylc[scheduler][events]stall timeout
Type: time interval

The length of a timer which starts if the scheduler stalls.

The default value is set in the global config: global.cylc[scheduler][events]stall timeout.

Changed in version 8.0.0: This item was previously called timeout.

stall timeout handlers

Path: flow.cylc[scheduler][events]stall timeout handlers
Type: list

Handlers to run if the stall timer times out.

The default value is set in the global config: global.cylc[scheduler][events]stall timeout handlers.

Changed in version 8.0.0: This item was previously called timeout handler.

Examples:

# configure a single event handler
stall timeout handlers = echo foo

# provide context to the handler
stall timeout handlers = echo %(workflow)s

# configure multiple event handlers
stall timeout handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'

abort on stall timeout

Path: flow.cylc[scheduler][events]abort on stall timeout
Type: boolean

Whether the scheduler should shut down immediately with error status if the stall timer times out.

The default value is set in the global config: global.cylc[scheduler][events]abort on stall timeout.

Changed in version 8.0.0: This item was previously called abort on timeout.

inactivity timeout

Path: flow.cylc[scheduler][events]inactivity timeout
Type: time interval

Scheduler inactivity timeout interval. The timer resets when any workflow activity occurs.

The default value is set in the global config: global.cylc[scheduler][events]inactivity timeout.

Changed in version 8.0.0: This item was previously called inactivity.

inactivity timeout handlers

Path: flow.cylc[scheduler][events]inactivity timeout handlers
Type: list

Handlers to run if the inactivity timer times out.

The default value is set in the global config: global.cylc[scheduler][events]inactivity timeout handlers.

Changed in version 8.0.0: This item was previously called inactivity handler.

Examples:

# configure a single event handler
inactivity timeout handlers = echo foo

# provide context to the handler
inactivity timeout handlers = echo %(workflow)s

# configure multiple event handlers
inactivity timeout handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'

abort on inactivity timeout

Path: flow.cylc[scheduler][events]abort on inactivity timeout
Type: boolean

Whether the scheduler should shut down immediately with error status if the inactivity timer times out.

The default value is set in the global config: global.cylc[scheduler][events]abort on inactivity timeout.

Changed in version 8.0.0: This item was previously called abort on inactivity.

expected task failures

Path: flow.cylc[scheduler][events]expected task failures
Type: list

(For Cylc developers writing a functional tests only) List of tasks that are expected to fail in the test.

[mail]

Path: flow.cylc[scheduler][mail]

Settings for the scheduler to send event emails.

The default value is set in the global config: global.cylc[scheduler][mail].

These settings are used for both workflow and task events.

New in version 8.0.0.

footer

Path: flow.cylc[scheduler][mail]footer
Type: string

Specify a string or string template for footers of emails sent for both workflow and task events.

The default value is set in the global config: global.cylc[scheduler][mail]footer.

Template variables may be used in the mail footer. For a list of supported variables see Workflow Event Template Variables.

Example:

footer = see http://ahost/%(owner)s/notes/%(workflow)s``

Changed in version 8.0.0: This item was previously called [cylc][events]mail footer.

to

Path: flow.cylc[scheduler][mail]to
Type: string

A list of email addresses that event notifications should be sent to.

The default value is set in the global config: global.cylc[scheduler][mail]to.

Changed in version 8.0.0: This item was previously called [cylc][events]mail to.

from

Path: flow.cylc[scheduler][mail]from
Type: string

Specify an alternative from email address for workflow event notifications.

The default value is set in the global config: global.cylc[scheduler][mail]from.

Changed in version 8.0.0: This item was previously called [cylc][events]mail from.

task event batch interval

Path: flow.cylc[scheduler][mail]task event batch interval
Type: time interval

Gather all task event notifications in the given interval into a single email.

The default value is set in the global config: global.cylc[scheduler][mail]task event batch interval.

Useful to prevent being overwhelmed by emails.

Changed in version 8.0.0: This item was previously called [cylc]task event mail interval.

[task parameters]

Path: flow.cylc[task parameters]

Set task parameters and parameter templates.

Define parameter values here for use in expanding parameterized tasks.

Changed in version 8.0.0: This item was previously called [cylc][parameters].

<parameter>

Path: flow.cylc[task parameters]<parameter>
Type: parameter list

A custom parameter to use in a workflow.

Examples:

run = control, test1, test2
mem = 1..5 (equivalent to 1, 2, 3, 4, 5).
mem = -11..-7..2 (equivalent to -11, -9, -7).

[templates]

Path: flow.cylc[task parameters][templates]

Cylc will expand each parameterized task name using a string template.

You can set templates for any parameter name here to override the default template.

Changed in version 8.0.0: This item was previously called [cylc][parameter templates].

<parameter>

Path: flow.cylc[task parameters][templates]<parameter>
Type: string

A template for a parameter.

For example:

If you set:

[task parameters]
    myparameter = 1..3
    [[templates]]
        myparameter = _run_%(myparameter)s

task name foo<myparameter> becomes foo_run_3 for run == 3.

Note

The default parameter templates are:

For integer parameters:: _p%(p)0Nd where N is the number of digits of the maximum integer value, i.e. If the largest parameter value is 3142 then N = 4.
Default for non-integer parameters:: _%(p)s e.g. foo<run> becomes foo_top for run value top.

[scheduling]

Path: flow.cylc[scheduling]

This section allows Cylc to determine when tasks are ready to run.

Any cycle points defined here without a time zone will use the time zone from flow.cylc[scheduler]cycle point time zone.

initial cycle point

Path: flow.cylc[scheduling]initial cycle point
Type: cycle point

The earliest cycle point at which any task can run.

In a cold start each cycling task (unless specifically excluded under [special tasks]) will be loaded into the workflow with this cycle point, or with the closest subsequent valid cycle point for the task.

In integer cycling, the default is 1.

The string now converts to the current datetime on the workflow host when first starting the workflow (with precision determined by flow.cylc[scheduler]cycle point format).

For more information on setting the initial cycle point relative to the current time see Setting The Initial Cycle Point Relative To The Current Time.

This item can be overridden on the command line using cylc play --initial-cycle-point or --icp.

final cycle point

Path: flow.cylc[scheduling]final cycle point
Type: string

The (optional) last cycle point at which tasks are run.

Once all tasks have reached this cycle point, the workflow will shut down.

This item can be overridden on the command line using cylc play --final-cycle-point or --fcp.

initial cycle point constraints

Path: flow.cylc[scheduling]initial cycle point constraints
Type: list

Rules to allow only some initial datetime cycle points.

Use Case

Writing a workflow where users may change the initial cycle point, but where only some initial cycle points are reasonable.

Set by defining a list of truncated time points, which the initial cycle point must match.

Examples:

T00, T06, T12, T18 - only at 6 hourly intervals.
T-30 - only at half-past an hour.
01T00 - only at midnight on the first day of a month.

See also

Recurrence tutorial.

Note

This setting does not coerce initial cycle point=now.

final cycle point constraints

Path: flow.cylc[scheduling]final cycle point constraints
Type: list

Rules restricting permitted final cycle points.

In a cycling workflow it is possible to restrict the final cycle point by defining a list of truncated time points under the final cycle point constraints.

See also

Recurrence tutorial.

hold after cycle point

Path: flow.cylc[scheduling]hold after cycle point
Type: cycle point

Hold all tasks that pass this cycle point.

Unlike the final cycle point, the workflow does not shut down once all tasks have passed this point. If this item is set you can override it on the command line using --hold-after.

Changed in version 8.0.0: This item was previously called [scheduling]hold after point.

stop after cycle point

Path: flow.cylc[scheduling]stop after cycle point
Type: cycle point

Shut down the workflow after all tasks pass this cycle point.

The stop cycle point can be overridden on the command line using cylc play --stop-cycle-point=POINT

New in version 8.0.0.

cycling mode

Path: flow.cylc[scheduling]cycling mode
Type: string
Default: gregorian
Options: 360day, 360_day, 365day, 365_day, 366day, 366_day, gregorian, integer

Choice of integer cycling or one of several datetime cycling calendars.

Cylc runs workflows using the proleptic Gregorian calendar by default. This setting allows you to instead choose integer cycling, or one of the other supported non-Gregorian datetime calendars: 360 day (12 months of 30 days in a year), 365 day (never a leap year) and 366 day (always a leap year).

runahead limit

Path: flow.cylc[scheduling]runahead limit
Type: string
Default: P4

The scheduler runahead limit determines how many consecutive cycle points can be active at once. The base point of the runahead calculation is the lowest-valued point with active or incomplete tasks present.

An integer interval value of Pn allows up to n+1 cycle points (including the base point) to be active at once.

The default runahead limit is P4, i.e. 5 active points including the base point.

Datetime cycling workflows can optionally use a datetime interval value instead, in which case the number of active cycle points within the interval depends on the cycling intervals present.

See also

Runahead Limiting

Changed in version 8.0.0: The integer format Pn was introduced to replace the deprecated [scheduling]max active cycle points = m (with n = m-1) and unify it with the existing datetime interval runahead limit setting.

[queues]

Path: flow.cylc[scheduling][queues]

Configuration of internal queues of tasks.

This section will allow you to limit the number of simultaneously active tasks (submitted or running) by assigning tasks to queues.

By default, a single queue called default is defined, with all tasks assigned to it and no limit to the number of those tasks which may be active.

To use a single queue for the whole workflow, but limit the number of active tasks, set [default]limit.

To add additional queues define additional sections:

[[queues]]
    [[[user_defined_queue]]]
        limit = 2
        members = TASK_FAMILY_NAME

See also

Internal Queues.

[<queue name>]

Path: flow.cylc[scheduling][queues][<queue name>]

Section heading for configuration of a single queue.

limit

Path: flow.cylc[scheduling][queues][<queue name>]limit
Type: integer
Default: 0

The maximum number of active tasks allowed at any one time, for this queue.

If set to 0 this queue is not limited.

members

Path: flow.cylc[scheduling][queues][<queue name>]members
Type: list

A list of member tasks, or task family names to assign to this queue.

Assigned tasks will automatically be removed from the default queue.

[default]

Path: flow.cylc[scheduling][queues][default]
Inherits: flow.cylc[scheduling][queues][<queue name>]

The default queue for all tasks not assigned to other queues.

limit

Path: flow.cylc[scheduling][queues][default]limit
Type: integer
Default: 0

Controls the total number of active tasks in the default queue.

See also

flow.cylc[scheduling][queues][<queue name>]limit
Internal Queues

[special tasks]

Path: flow.cylc[scheduling][special tasks]

This section is used to identify tasks with special behaviour.

Family names can be used in special task lists as shorthand for listing all member tasks.

clock-trigger

Path: flow.cylc[scheduling][special tasks]clock-trigger
Type: list

Legacy clock trigger definitions.

Deprecated since version 8.0.0: Please read External Triggers before using the older clock triggers described in this section.

Clock-trigger tasks (see Clock Triggers) wait on a wall clock time specified as an offset from their own cycle point.

Example:

foo(PT1H30M), bar(PT1.5H), baz

external-trigger

Path: flow.cylc[scheduling][special tasks]external-trigger
Type: list

Legacy external trigger definition section.

Deprecated since version 8.0.0: Please read External Triggers before using the older mechanism described in this section.

Externally triggered tasks (see Push External Triggers) wait on external events reported via the cylc ext-trigger command. To constrain triggers to a specific cycle point, include $CYLC_TASK_CYCLE_POINT in the trigger message string and pass the cycle point to the cylc ext-trigger command.

clock-expire

Path: flow.cylc[scheduling][special tasks]clock-expire
Type: list

Don’t submit jobs if they are very late in wall clock time.

Clock-expire tasks enter the expired state and skip job submission if too far behind the wall clock when they become ready to run.

The expiry time is specified as an offset from wall-clock time; typically it should be negative - see Clock-Expire Triggers.

Note

The offset:

May be positive or negative
The offset may be omitted if it is zero.

Example:

PT1H - 1 hour

sequential

Path: flow.cylc[scheduling][special tasks]sequential
Type: list

A list of tasks which automatically depend on their own previous-cycle instance.

Tip

Recommend best practice is now to use explicit inter-cycle triggers rather than sequential tasks.

See also

Special Sequential Tasks.

[xtriggers]

Path: flow.cylc[scheduling][xtriggers]

This section is for External Trigger function declarations - see External Triggers.

<xtrigger name>

Path: flow.cylc[scheduling][xtriggers]<xtrigger name>
Type: xtrigger function signature

Any user-defined event trigger function declarations and corresponding labels for use in the graph.

See External Triggers for details.

Example:

``my_trigger(arg1, arg2, kwarg1, kwarg2):PT10S``

[graph]

Path: flow.cylc[scheduling][graph]

The workflow graph is defined under this section.

You can plot the dependency graph as you work on it, with cylc graph.

See also

Scheduling Configuration.

Changed in version 8.0.0: This item was previously called [runtime][dependencies][graph].

<recurrence>

Path: flow.cylc[scheduling][graph]<recurrence>
Type: string

The recurrence defines the sequence of cycle points for which the dependency graph is valid.

See also

Scheduling Configuration

Cycle points should be specified in our ISO 8601 derived sequence syntax, or as integers, in integer cycling mode:

Example Recurrences:

datetime cycling:

R1 - once at the initial cycle point
T00,T06,T12,T18 - daily at 00:00, 06:00, 12:00 & 18:00
PT6H - every six hours starting at the initial cycle point

integer cycling:

R1 - once at the initial cycle point
P2 - every other cycle
P3,P5 - every third or fifth cycle

Note

Unlike other Cylc configurations duplicate recurrences are additive and do not override.

For example this:

[scheduling]
    [[graph]]
        R1 = a => b
        R1 = c => d

Is equivalent to this:

[scheduling]
    [[graph]]
        R1 = """
            a => b
            c => d
        """

See Graph Types for more on recurrence expressions, and how multiple graphs combine.

The value should be a dependency graph the given recurrence. Syntax examples follow; see also Scheduling Configuration and Task Triggering.

Example Graph Strings:

# baz and waz both trigger off bar
foo => bar => baz & waz

# bar triggers off foo[-P1D-PT6H]
foo[-P1D-PT6H] => bar

# faz triggers off a message output of baz
baz:out1 => faz

# Y triggers if X starts executing
X:start => Y

# Y triggers if X fails
X:fail => Y

# bar triggers if foo[-PT6H] fails
foo[-PT6H]:fail => bar

# Y suicides if X succeeds
X => !Y

# Z triggers if X succeeds or fails
X | X:fail => Z

# Z triggers if X succeeds or fails
X:finish => Z

# general conditional triggers
(A | B & C ) | D => foo

# bar triggers if foo is successfully submitted
foo:submit => bar

# bar triggers if submission of foo fails
foo:submit-fail => bar

Changed in version 8.0.0: This item was previously called [runtime][dependencies][graph][<recurrence>]graph.

[runtime]

Path: flow.cylc[runtime]

This section is used to specify settings for tasks to be run.

You can specify:

What scripts or commands you want to execute.
Which compute resource (platform) you wish to use.
How to run your task.

If multiple tasks need the same settings, they can share settings by inheriting them from one or more other tasks.

Precedence is determined by the same C3 linearization algorithm used to find the method resolution order in Python language class hierarchies.

See also

For details and examples see Task Configuration.

[<namespace>]

Path: flow.cylc[runtime][<namespace>]

The name(s) of one or more tasks or task families.

A namespace (i.e. task or family name) or a comma-separated list of namespace names, and repeat as needed to define all tasks in the workflow. Names may contain letters, digits, underscores, and hyphens. A namespace represents a group or family of tasks if other namespaces inherit from it, or a task if no others inherit from it.

Important

Names may not contain : or ..

See task namespace rules.

Examples of legal values:

[foo]
[foo, bar, baz]

If multiple names are listed the subsequent settings apply to all.

All tasks or task families inherit initially from root, which can be explicitly configured to provide or override default settings for all tasks in the workflow.

platform

Path: flow.cylc[runtime][<namespace>]platform
Type: string

The name of a compute resource defined in global.cylc[platforms] or global.cylc[platform groups].

The platform specifies the host(s) that the tasks’ jobs will run on and where (if necessary) files need to be installed, and what job runner will be used.

New in version 8.0.0.

inherit

Path: flow.cylc[runtime][<namespace>]inherit
Type: list

A list of the immediate parent(s) of this task or task family.

If no parents are listed default is root.

script

Path: flow.cylc[runtime][<namespace>]script
Type: string

The main custom script run from the task job script.

It can be an external command or script, or inlined scripting.

See Job Script Variables for the list of variables available in the task execution environment.

See also Job Scripts.

Other user-defined script items:

init-script
env-script
pre-script
post-script
err-script
exit-script

Example:

my_script.sh

init-script

Path: flow.cylc[runtime][<namespace>]init-script
Type: string

Custom script run by the task job script before the task execution environment is configured.

By running before the task execution environment is configured, this script does not have access to any workflow or task environment variables. It can be an external command or script, or inlined scripting. The original intention for this item was to allow remote tasks to source login scripts to configure their access to cylc, but this should no longer be necessary.

See also Job Scripts.

Other user-defined script items:

env-script
pre-script
script
post-script
err-script
exit-script

Example:

echo "Hello World"

env-script

Path: flow.cylc[runtime][<namespace>]env-script
Type: string

Custom script run by the task job script between the cylc-defined environment (workflow and task identity, etc.) and the user-defined task runtime environment.

The env-script has access to the Cylc environment (and the task environment has access to variables defined by this scripting). It can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

init-script
pre-script
script
post-script
err-script
exit-script

Example:

echo "Hello World"

err-script

Path: flow.cylc[runtime][<namespace>]err-script
Type: string

Script run when a task job error is detected.

Custom script to be run at the end of the error trap, which is triggered due to failure of a command in the task job script or trappable job kill.

The output of this script will always be sent to STDERR and $1 is set to the name of the signal caught by the error trap. The script should be fast and use very little system resource to ensure that the error trap can return quickly. Companion of exit-script, which is executed on job success. It can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

init-script
env-script
pre-script
script
post-script
exit-script

Example:

echo "Uh oh, received ${1}"

exit-script

Path: flow.cylc[runtime][<namespace>]exit-script
Type: string

Custom script run at the very end of successful job execution, just before the job script exits.

The exit-script should execute very quickly. Companion of err-script, which is executed on job failure. It can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

init-script
env-script
pre-script
script
post-script
err-script

Example:

rm -f "$TMP_FILES"

pre-script

Path: flow.cylc[runtime][<namespace>]pre-script
Type: string

Custom script run by the task job script immediately before script.

The pre-script can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

init-script
env-script
script
post-script
err-script
exit-script

Example:

echo "Hello from workflow ${CYLC_WORKFLOW_ID}!"

post-script

Path: flow.cylc[runtime][<namespace>]post-script
Type: string

Custom script run by the task job script immediately after script.

The post-script can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

init-script
env-script
pre-script
script
err-script
exit-script

Example:

echo "Hello World"

work sub-directory

Path: flow.cylc[runtime][<namespace>]work sub-directory
Type: string

The directory from which tasks are executed.

Task job scripts are executed from within work directories created automatically under the workflow run directory. A task can get its own work directory from $CYLC_TASK_WORK_DIR (or $PWD if it does not cd elsewhere at runtime). The default directory path contains task name and cycle point, to provide a unique workspace for every instance of every task. If several tasks need to exchange files and simply read and write from their from current working directory, setting work sub-directory can be used to override the default to make them all use the same workspace.

The top level share and work directory location can be changed (e.g. to a large data area) by a global config setting (see global.cylc[install][symlink dirs]).

Caution

If you omit cycle point from the work sub-directory path successive instances of the task will share the same workspace. Consider the effect on cycle point offset housekeeping of work directories before doing this.

Example:

$CYLC_TASK_CYCLE_POINT/shared/

execution polling intervals

Path: flow.cylc[runtime][<namespace>]execution polling intervals
Type: time interval list

List of intervals at which to poll status of job execution.

The default value is set in the global config: global.cylc[platforms][<platform name>]execution polling intervals.

Cylc can poll running jobs to catch problems that prevent task messages from being sent back to the workflow, such as hard job kills, network outages, or unplanned job host shutdown.

The last interval in the list is used repeatedly until the job completes.

Multipliers can be used as shorthand as in the example below.

Example:

5*PT2M, PT5M

Note that if the polling global.cylc[platforms][<platform name>]communication method is used then Cylc relies on polling to detect all task state changes, so you may want to configure more frequent polling.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]execution polling intervals.

execution retry delays

Path: flow.cylc[runtime][<namespace>]execution retry delays
Type: time interval list

Cylc can automate resubmission of a failed task job.

Execution retry delays are a list of ISO 8601 durations/intervals which tell Cylc how long to wait before resubmitting a failed job.

Each time Cylc resubmits a task job it will increment the variable $CYLC_TASK_TRY_NUMBER in the task execution environment. $CYLC_TASK_TRY_NUMBER allows you to vary task behavior between submission attempts.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]execution retry delays.

execution time limit

Path: flow.cylc[runtime][<namespace>]execution time limit
Type: time interval

Set the execution (wallclock) time limit of a task job.

For background and at job runners Cylc runs the job’s script using the timeout command. For other job runners Cylc will convert execution time limit to a directive.

If a task job exceeds its execution time limit Cylc can poll the job multiple times. You can set polling intervals using global.cylc[platforms][<platform name>]execution time limit polling intervals

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]execution time limit.

submission polling intervals

Path: flow.cylc[runtime][<namespace>]submission polling intervals
Type: time interval list

List of intervals at which to poll status of job submission.

The default value is set in the global config: global.cylc[platforms][<platform name>]submission polling intervals.

Cylc can poll submitted jobs to catch problems that prevent the submitted job from executing at all, such as deletion from an external job runner queue.

The last value is used repeatedly until the task starts running.

Multipliers can be used as shorthand as in the example below.

Example:

5*PT2M, PT5M

Note that if the polling global.cylc[platforms][<platform name>]communication method is used then Cylc relies on polling to detect all task state changes, so you may want to configure more frequent polling.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]submission polling intervals.

submission retry delays

Path: flow.cylc[runtime][<namespace>]submission retry delays
Type: time interval list

Cylc can automatically resubmit jobs after submission failures.

The default value is set in the global config: global.cylc[platforms][<platform name>]submission retry delays.

A list of intervals which define when the scheduler will resubmit jobs if submission fails.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]submission retry delays.

[meta]

Path: flow.cylc[runtime][<namespace>][meta]

Metadata for the task or task family.

The meta section contains metadata items for this task or family namespace. The items title, description and URL are pre-defined and are used by Cylc. Others can be user-defined and passed to task event handlers to be interpreted according to your needs. For example, the value of an “importance” item could determine how an event handler responds to task failure events.

Any workflow meta item can now be passed to task event handlers by prefixing the string template item name with workflow_, for example:

[runtime]
    [[root]]
        [[[events]]]
            failed handlers = """
                send-help.sh \
                    %(workflow_title)s \
                    %(workflow_importance)s \
                    %(title)s
             """

title

Path: flow.cylc[runtime][<namespace>][meta]title
Type: string

A single line description of this task or task family.

It is displayed by the cylc list command and can be retrieved from running tasks with the cylc show command.

description

Path: flow.cylc[runtime][<namespace>][meta]description
Type: string

A multi-line description of this task or task family.

It is retrievable from running tasks with the cylc show command.

URL

Path: flow.cylc[runtime][<namespace>][meta]URL
Type: string

A URL link to task documentation for this task or task family.

The templates %(workflow)s and %(task)s will be replaced with the actual workflow ID and task name.

Deprecated since version 8.0.0: The %(suite_name)s template variable is deprecated, please use %(workflow)s.

The %(task_name)s template variable is deprecated, please use %(task)s.

See also

Task & Job States

Caution

Changes to the scheduling algorithm in Cylc 8 mean this event will not be triggered until the expired task is ready to run. Earlier expired-task detection will be implemented in a future Cylc release.

late offset

Path: flow.cylc[runtime][<namespace>][events]late offset
Type: time interval

Offset from cycle point, in real time, at which this task is considered to be “running late” (i.e. the time by which it would normally have started running).

Caution

Changes to the scheduling algorithm in Cylc 8 mean this event will not be triggered until the late task is ready to run. Earlier late-task detection will be implemented in a future Cylc release.

late handlers

Path: flow.cylc[runtime][<namespace>][events]late handlers
Type: list

Handlers to run if this task is late.

Caution

Due to changes to the Cylc 8 scheduling algorithm this event is unlikely to occur until the task is about to submit anyway.

submitted handlers

Path: flow.cylc[runtime][<namespace>][events]submitted handlers
Type: list

Handlers to run when this task is submitted.

started handlers

Path: flow.cylc[runtime][<namespace>][events]started handlers
Type: list

Handlers to run when this task starts executing.

succeeded handlers

Path: flow.cylc[runtime][<namespace>][events]succeeded handlers
Type: list

Handlers to run if this task succeeds.

failed handlers

Path: flow.cylc[runtime][<namespace>][events]failed handlers
Type: list

Handlers to run if this task fails.

submission failed handlers

Path: flow.cylc[runtime][<namespace>][events]submission failed handlers
Type: list

Handlers to run if submission of this task fails.

warning handlers

Path: flow.cylc[runtime][<namespace>][events]warning handlers
Type: list

Handlers to run if this task runs cylc message with severity level “WARNING”.

critical handlers

Path: flow.cylc[runtime][<namespace>][events]critical handlers
Type: list

Handlers to run if this task runs cylc message with severity level “CRITICAL”.

retry handlers

Path: flow.cylc[runtime][<namespace>][events]retry handlers
Type: list

Handlers to run if this task failed but is retrying.

submission retry handlers

Path: flow.cylc[runtime][<namespace>][events]submission retry handlers
Type: list

Handlers to run if a job failed to submit but is retrying.

See also

Task & Job States

execution timeout handlers

Path: flow.cylc[runtime][<namespace>][events]execution timeout handlers
Type: list

Handlers to run if this task execution exceeds flow.cylc[runtime][<namespace>]execution time limit.

submission timeout handlers

Path: flow.cylc[runtime][<namespace>][events]submission timeout handlers
Type: list

Handlers to run if this task exceeds flow.cylc[runtime][<namespace>][events]submission timeout in the submitted state.

custom handlers

Path: flow.cylc[runtime][<namespace>][events]custom handlers
Type: list

Handlers to run if this task runs cylc message with severity level “CUSTOM”.

[mail]

Path: flow.cylc[runtime][<namespace>][mail]

Email notification settings for task events.

New in version 8.0.0.

from

Path: flow.cylc[runtime][<namespace>][mail]from
Type: string

Specify an alternate from: email address for event notifications.

Changed in version 8.0.0: This item was previously called [runtime][task][events]mail from

to

Path: flow.cylc[runtime][<namespace>][mail]to
Type: string

A list of email addresses to send task event notifications.

The list can be any address accepted by the mail command.

Changed in version 8.0.0: This item was previously called [runtime][task][events]mail to

[workflow state polling]

Path: flow.cylc[runtime][<namespace>][workflow state polling]

Configure automatic workflow polling tasks as described in Triggering Off Of Tasks In Other Workflows.

The items in this section reflect options and defaults of the cylc workflow-state command, except that the target workflow name and the --task, --cycle, and --status options are taken from the graph notation.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>]suite state polling.

user

Path: flow.cylc[runtime][<namespace>][workflow state polling]user
Type: string

Username of your account on the workflow host.

The polling cylc workflow-state command will be run on the remote account.

host

Path: flow.cylc[runtime][<namespace>][workflow state polling]host
Type: string

The hostname of the target workflow.

The polling cylc workflow-state command will be run there.

interval

Path: flow.cylc[runtime][<namespace>][workflow state polling]interval
Type: time interval

Polling interval.

max-polls

Path: flow.cylc[runtime][<namespace>][workflow state polling]max-polls
Type: integer

The maximum number of polls before timing out and entering the “failed” state.

message

Path: flow.cylc[runtime][<namespace>][workflow state polling]message
Type: string

Wait for the task in the target workflow to receive a specified message rather than achieve a state.

run-dir

Path: flow.cylc[runtime][<namespace>][workflow state polling]run-dir
Type: string

Specify the location of the top level cylc-run directory for the other workflow.

For your own workflows, there is no need to set this as it is always ~/cylc-run/. But for other workflows, (e.g those owned by others), or mirrored workflow databases use this item to specify the location of the top level cylc run directory (the database should be in a the same place relative to this location for each workflow).

verbose mode

Path: flow.cylc[runtime][<namespace>][workflow state polling]verbose mode
Type: boolean

Run the polling cylc workflow-state command in verbose output mode.

[environment]

Path: flow.cylc[runtime][<namespace>][environment]

The user defined task execution environment.

Variables defined here can refer to cylc workflow and task identity variables, which are exported earlier in the task job script. Variable assignment expressions can use cylc utility commands because access to cylc is also configured earlier in the script.

See also

Job Environment.

You can also specify job environment templates here for parameterized tasks.

<variable>

Path: flow.cylc[runtime][<namespace>][environment]<variable>
Type: string

A custom user defined variable for a task execution environment.

The order of definition is preserved that each variable can refer to previously defined variables. Values are passed through to the task job script without evaluation or manipulation by Cylc (with the exception of valid Python string templates that match parameterized task names - see below), so any variable assignment expression that is legal in the job submission shell can be used. White space around the = is allowed (as far as cylc’s flow.cylc parser is concerned these are just normal configuration items).

Examples:

FOO = $HOME/bar/baz
BAR = ${FOO}$GLOBALVAR
BAZ = $( echo "hello world" )
WAZ = ${FOO%.jpg}.png
NEXT_CYCLE = $( cylc cycle-point --offset=PT6H )
ZAZ = "${FOO#bar}"
# ^ quoted to escape the flow.cylc comment character
DICE = [$((($RANDOM % 6) + 1)) $((($RANDOM % 6) + 1))]

For parameter environment templates, use Python string templates for parameter substitution. This is only relevant for parameterized tasks. The job script will export the named variables specified here (in addition to the standard CYLC_TASK_PARAM_<key> variables), with the template strings substituted with the parameter values.

Examples:

MYNUM = %(i)d
MYITEM = %(item)s
MYFILE = /path/to/%(i)03d/%(item)s

Changed in version 7.8.7/7.9.2: Parameter environment templates (previously in [runtime][X][parameter environment templates]) have moved here.

[directives]

Path: flow.cylc[runtime][<namespace>][directives]

Job runner (batch scheduler) directives.

The default value is set in the global config: global.cylc[platforms][<platform name>][directives].

Supported for use with job runners:

pbs
slurm
loadleveler
lsf
sge
slurm_packjob
moab

Directives are written to the top of the task job script in the correct format for the job runner.

Specifying directives individually like this allows use of default directives for task families which can be individually overridden at lower levels of the runtime namespace hierarchy.

<directive>

Path: flow.cylc[runtime][<namespace>][directives]<directive>
Type: string

Example directives for the built-in job runner handlers are shown in Supported Job Submission Methods.

[outputs]

Path: flow.cylc[runtime][<namespace>][outputs]

Register custom task outputs for use in message triggering in this section (Message Triggers)

<output>

Path: flow.cylc[runtime][<namespace>][outputs]<output>
Type: string

Task output messages (Message Triggers).

The item name is used to select the custom output message in graph trigger notation.

Examples:

out1 = "sea state products ready"
out2 = "NWP restart files completed"

Task outputs are validated by cylc.flow.unicode_rules.TaskOutputValidator.

class cylc.flow.unicode_rules.TaskOutputValidator

The rules for valid task outputs/message triggers:

cannot contain: :

[parameter environment templates]

Path: flow.cylc[runtime][<namespace>][parameter environment templates]

Deprecated since version 7.8.7/7.9.2: Parameter environment templates have moved to flow.cylc[runtime][<namespace>][environment].

This was done to allow users to control the order of definition of the variables. This section will be removed in a future version of Cylc 8.

For the time being, the contents of this section will be prepended to the [environment] section when running a workflow.

<parameter>

Path: flow.cylc[runtime][<namespace>][parameter environment templates]<parameter>
Type: string

Deprecated since version 7.8.7/7.9.2: Parameter environment templates have moved to flow.cylc[runtime][<namespace>][environment].