Workflow Configuration

Cylc workflow configurations use the .cylc file format.

flow.cylc

Defines a Cylc workflow configuration.

After processing any embedded templating code (see Jinja2) the resultant raw flow.cylc file must be valid. See also The flow.cylc File for a descriptive overview of flow.cylc files, including syntax (Syntax).

Changed in version 8.0.0: The configuration file was previously named suite.rc, but that name is now deprecated. The suite.rc file name now activates Cylc 7 Compatibility Mode. Rename to flow.cylc to turn off compatibility mode.

[meta]
Path

flow.cylc[meta]

Metadata for this workflow.

Cylc defines and uses the terms “title”, “description” and “URL”. Users can define more terms, and use these in event handlers.

Example:

A user could define "workflow-priority". An event handler
would then respond to failure events in a way set by
"workflow-priority".
description
Path

flow.cylc[meta]description

Type

string

A multi-line description of the workflow.

It can be retrieved at run time with the cylc show command.

title
Path

flow.cylc[meta]title

Type

string

A single line description of the workflow.

It can be retrieved at run time with the cylc show command.

URL
Path

flow.cylc[meta]URL

Type

string

A web URL to workflow documentation.

The URL can be retrieved at run time with the cylc show command.

The template variable %(workflow)s will be replaced with the actual workflow ID.

Deprecated since version 8.0.0: The %(suite_name)s template variable is deprecated, please use %(workflow)s.

Example:

http://my-site.com/workflows/%(workflow)s/index.html

<custom metadata>
Path

flow.cylc[meta]<custom metadata>

Type

string

Any user-defined metadata item.

Like title, description and URL these can be passed to workflow event handlers to be interpreted according to your needs.

For example, a user could define an item called “workflow-priority”. An event handler could then respond to failure events in a way set by “workflow-priority”.

[scheduler]
Path

flow.cylc[scheduler]

Settings for the scheduler.

The default value is set in the global config: global.cylc[scheduler].

Note

Not to be confused with flow.cylc[scheduling].

Changed in version 8.0.0: This item was previously called [cylc]

UTC mode
Path

flow.cylc[scheduler]UTC mode

Type

boolean

If True, UTC will be used as the time zone for timestamps in the logs. If False, the local/system time zone will be used.

The default value is set in the global config: global.cylc[scheduler]UTC mode.

See also

To set a time zone for cycle points, see flow.cylc[scheduler]cycle point time zone.

allow implicit tasks
Path

flow.cylc[scheduler]allow implicit tasks

Type

boolean

Default

False

Allow tasks in the graph that are not defined in flow.cylc[runtime].

Implicit tasks are tasks without explicit definitions in flow.cylc[runtime]. By default, these are not allowed, as they are often typos. However, this setting can be set to True to allow implicit tasks. It is recommended to only set this to True if required during development/prototyping of a workflow graph, but set it to False after finishing the flow.cylc[runtime] section.

Cylc 7 compatibility mode

In Cylc 7 Compatibility Mode, implicit tasks are still allowed unless you explicitly set this to False, or unless a rose-suite.conf file is present (to maintain backward compatibility with Rose 2019).

New in version 8.0.0.

install
Path

flow.cylc[scheduler]install

Type

list

Configure custom directories and files to be installed on remote hosts.

Note

The following directories already get installed by default:

ana/

Rose ana analysis modules

app/

Rose applications

bin/

Cylc bin directory (added to PATH)

etc/

Miscellaneous resources

lib/

Cylc lib directory (lib/python added to PYTHONPATH for workflow config)

These should be located in the top level of your Cylc workflow, i.e. the directory that contains your flow.cylc file.

Directories must have a trailing slash. For example, to add the following items to your file installation:

~/cylc-run/workflow_x
|-- dir1/
|-- dir2/
|-- file1
`-- file2
[scheduler]
    install = dir/, dir2/, file1, file2

New in version 8.0.0.

cycle point format
Path

flow.cylc[scheduler]cycle point format

Type

cycle point format

Set the datetime format and precision that Cylc uses for cycle points in datetime cycling workflows.

See also

By default, Cylc uses a CCYYMMDDThhmmZ (Z in the special case of UTC) or CCYYMMDDThhmm±hhmm format for writing datetime cycle points, following the ISO 8601 standard.

You may use the isodatetime library’s syntax to set the cycle point format.

You can also use a subset of the strptime/strftime POSIX standard - supported tokens are %F, %H, %M, %S, %Y, %d, %j, %m, %s, %z.

If specifying a format here, we recommend including a time zone - this will be used for displaying cycle points only. To avoid confusion, we recommend using the same time zone as flow.cylc[scheduler]cycle point time zone.

The ISO 8601 extended datetime format (CCYY-MM-DDThh:mm) cannot be used, as cycle points are used in job-log and work directory paths where the “:” character is invalid.

Warning

The smallest unit included in the format sets the precision of cycle points in the workflow. If the precision is lower than the smallest unit in a graph recurrence, the workflow will fail. For example, if you set a format of CCYY, and have a recurrence R/2000/P8M, then both the first and second cycle points will be 2000, which is invalid.

cycle point num expanded year digits
Path

flow.cylc[scheduler]cycle point num expanded year digits

Type

integer

Default

0

Enable negative years or years more than four digits long.

For years below 0 or above 9999, the ISO 8601 standard specifies that an extra number of year digits and a sign should be used. This extra number needs to be written down somewhere (here).

For example, if this extra number is set to 2, 00Z on the 1st of January in the year 10040 will be represented as +0100400101T0000Z (2 extra year digits used). With this number set to 3, 06Z on the 4th of May 1985 would be written as +00019850504T0600Z.

This number defaults to 0 (no sign or extra digits used).

cycle point time zone
Path

flow.cylc[scheduler]cycle point time zone

Type

cycle point time zone

Time zone to be used for datetime cycle points if not otherwise specified.

This time zone will be used for datetime cycle point dumping and inferring the time zone of cycle points that are input without time zones.

Time zones should be expressed as ISO8601 time zone offsets from UTC, such as +13, +1300, -0500 or +0645, with Z representing the special case of +0000 (UTC). Cycle points will be converted to the time zone you give and will be represented with this string at the end.

If not set, it will default to UTC (Z).

Cylc 7 compatibility mode

In Cylc 7 Compatibility Mode, it will default to the local/system time zone, rather than UTC.

The time zone will persist over reloads/restarts following any local time zone changes (e.g. if the workflow is run during winter time, then stopped, then restarted after summer time has begun, the cycle points will remain in winter time). Changing this setting after the workflow has first started will have no effect.

If you use a custom flow.cylc[scheduler]cycle point format, it is a good idea to set the same time zone here. If you specify a different one here, it will only be used for inferring timezone-less cycle points; cycle points will be displayed in the time zone from the cycle point format.

Caution

It is not recommended to write the time zone with a “:” (e.g. +05:30), given that the time zone is used as part of task output filenames.

Changed in version 7.8.9/7.9.4: The value set here now persists over reloads/restarts after a system time zone change.

Changed in version 8.0.0: The default time zone is now Z instead of the local time of the first workflow start.

[main loop]
Path

flow.cylc[scheduler][main loop]

Configuration of main loop plugins for the scheduler.

The default value is set in the global config: global.cylc[scheduler][main loop].

For a list of built in plugins see Main Loop Plugins.

New in version 8.0.0.

[<plugin name>]
Path

flow.cylc[scheduler][main loop][<plugin name>]

Configure a main loop plugin.

The default value is set in the global config: global.cylc[scheduler][main loop][<plugin name>].

Note

Only the configured list of global.cylc[scheduler][main loop]plugins is loaded when a scheduler is started.

New in version 8.0.0.

interval
Path

flow.cylc[scheduler][main loop][<plugin name>]interval

Type

time interval

Interval at which the plugin is invoked.

The default value is set in the global config: global.cylc[scheduler][main loop][<plugin name>]interval.

New in version 8.0.0.

[events]
Path

flow.cylc[scheduler][events]

Configure the workflow event handling system.

The default value is set in the global config: global.cylc[scheduler][events].

handlers
Path

flow.cylc[scheduler][events]handlers

Type

list

Configure event handlers that run when certain workflow events occur.

The default value is set in the global config: global.cylc[scheduler][events]handlers.

This section configures workflow event handlers; see flow.cylc[runtime][<namespace>][events] for task event handlers.

Event handlers can be held in the workflow bin/ directory, otherwise it is up to you to ensure their location is in $PATH (in the shell in which the scheduler runs). They should require little resource to run and return quickly.

Template variables can be used to configure handlers. For a full list of supported variables see Workflow Event Template Variables.

handler events
Path

flow.cylc[scheduler][events]handler events

Type

list

Specify the events for which workflow event handlers should be invoked.

The default value is set in the global config: global.cylc[scheduler][events]handler events.

mail events
Path

flow.cylc[scheduler][events]mail events

Type

list

Specify the workflow events for which notification emails should be sent.

The default value is set in the global config: global.cylc[scheduler][events]mail events.

startup handlers
Path

flow.cylc[scheduler][events]startup handlers

Type

list

Handlers to run at scheduler startup.

The default value is set in the global config: global.cylc[scheduler][events]startup handlers.

Changed in version 8.0.0: This item was previously called startup handler.

Examples:

# configure a single event handler
startup handlers = echo foo

# provide context to the handler
startup handlers = echo %(workflow)s

# configure multiple event handlers
startup handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'
shutdown handlers
Path

flow.cylc[scheduler][events]shutdown handlers

Type

list

Handlers to run at scheduler shutdown.

The default value is set in the global config: global.cylc[scheduler][events]shutdown handlers.

Changed in version 8.0.0: This item was previously called shutdown handler.

Examples:

# configure a single event handler
shutdown handlers = echo foo

# provide context to the handler
shutdown handlers = echo %(workflow)s

# configure multiple event handlers
shutdown handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'
abort handlers
Path

flow.cylc[scheduler][events]abort handlers

Type

list

Handlers to run if the scheduler shuts down with error status due to a configured timeout or a fatal error condition.

The default value is set in the global config: global.cylc[scheduler][events]abort handlers.

Changed in version 8.0.0: This item was previously called aborted handler.

Examples:

# configure a single event handler
abort handlers = echo foo

# provide context to the handler
abort handlers = echo %(workflow)s

# configure multiple event handlers
abort handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'
workflow timeout
Path

flow.cylc[scheduler][events]workflow timeout

Type

time interval

Workflow timeout interval. The timer starts counting down at scheduler startup. It resets on workflow restart.

The default value is set in the global config: global.cylc[scheduler][events]workflow timeout.

New in version 8.0.0.

workflow timeout handlers
Path

flow.cylc[scheduler][events]workflow timeout handlers

Type

list

Handlers to run if the workflow timer times out.

The default value is set in the global config: global.cylc[scheduler][events]workflow timeout handlers.

New in version 8.0.0.

Examples:

# configure a single event handler
workflow timeout handlers = echo foo

# provide context to the handler
workflow timeout handlers = echo %(workflow)s

# configure multiple event handlers
workflow timeout handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'
abort on workflow timeout
Path

flow.cylc[scheduler][events]abort on workflow timeout

Type

boolean

Whether the scheduler should shut down immediately with error status if the workflow timer times out.

The default value is set in the global config: global.cylc[scheduler][events]abort on workflow timeout.

New in version 8.0.0.

stall handlers
Path

flow.cylc[scheduler][events]stall handlers

Type

list

Handlers to run if the scheduler stalls.

The default value is set in the global config: global.cylc[scheduler][events]stall handlers.

Changed in version 8.0.0: This item was previously called stalled handler.

Examples:

# configure a single event handler
stall handlers = echo foo

# provide context to the handler
stall handlers = echo %(workflow)s

# configure multiple event handlers
stall handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'
stall timeout
Path

flow.cylc[scheduler][events]stall timeout

Type

time interval

The length of a timer which starts if the scheduler stalls.

The default value is set in the global config: global.cylc[scheduler][events]stall timeout.

Changed in version 8.0.0: This item was previously called timeout.

stall timeout handlers
Path

flow.cylc[scheduler][events]stall timeout handlers

Type

list

Handlers to run if the stall timer times out.

The default value is set in the global config: global.cylc[scheduler][events]stall timeout handlers.

Changed in version 8.0.0: This item was previously called timeout handler.

Examples:

# configure a single event handler
stall timeout handlers = echo foo

# provide context to the handler
stall timeout handlers = echo %(workflow)s

# configure multiple event handlers
stall timeout handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'
abort on stall timeout
Path

flow.cylc[scheduler][events]abort on stall timeout

Type

boolean

Whether the scheduler should shut down immediately with error status if the stall timer times out.

The default value is set in the global config: global.cylc[scheduler][events]abort on stall timeout.

Changed in version 8.0.0: This item was previously called abort on timeout.

inactivity timeout
Path

flow.cylc[scheduler][events]inactivity timeout

Type

time interval

Scheduler inactivity timeout interval. The timer resets when any workflow activity occurs.

The default value is set in the global config: global.cylc[scheduler][events]inactivity timeout.

Changed in version 8.0.0: This item was previously called inactivity.

inactivity timeout handlers
Path

flow.cylc[scheduler][events]inactivity timeout handlers

Type

list

Handlers to run if the inactivity timer times out.

The default value is set in the global config: global.cylc[scheduler][events]inactivity timeout handlers.

Changed in version 8.0.0: This item was previously called inactivity handler.

Examples:

# configure a single event handler
inactivity timeout handlers = echo foo

# provide context to the handler
inactivity timeout handlers = echo %(workflow)s

# configure multiple event handlers
inactivity timeout handlers = \
    'echo %(workflow)s, %(event)s', \
    'my_exe %(event)s %(message)s' \
    'curl -X PUT -d event=%(event)s host:port'
abort on inactivity timeout
Path

flow.cylc[scheduler][events]abort on inactivity timeout

Type

boolean

Whether the scheduler should shut down immediately with error status if the inactivity timer times out.

The default value is set in the global config: global.cylc[scheduler][events]abort on inactivity timeout.

Changed in version 8.0.0: This item was previously called abort on inactivity.

expected task failures
Path

flow.cylc[scheduler][events]expected task failures

Type

list

(For Cylc developers writing a functional tests only) List of tasks that are expected to fail in the test.

[mail]
Path

flow.cylc[scheduler][mail]

Settings for the scheduler to send event emails.

The default value is set in the global config: global.cylc[scheduler][mail].

These settings are used for both workflow and task events.

New in version 8.0.0.

footer
Path

flow.cylc[scheduler][mail]footer

Type

string

Specify a string or string template for footers of emails sent for both workflow and task events.

The default value is set in the global config: global.cylc[scheduler][mail]footer.

Template variables may be used in the mail footer. For a list of supported variables see Workflow Event Template Variables.

Example:

footer = see http://ahost/%(owner)s/notes/%(workflow)s``

Changed in version 8.0.0: This item was previously called [cylc][events]mail footer.

to
Path

flow.cylc[scheduler][mail]to

Type

string

A list of email addresses that event notifications should be sent to.

The default value is set in the global config: global.cylc[scheduler][mail]to.

Changed in version 8.0.0: This item was previously called [cylc][events]mail to.

from
Path

flow.cylc[scheduler][mail]from

Type

string

Specify an alternative from email address for workflow event notifications.

The default value is set in the global config: global.cylc[scheduler][mail]from.

Changed in version 8.0.0: This item was previously called [cylc][events]mail from.

task event batch interval
Path

flow.cylc[scheduler][mail]task event batch interval

Type

time interval

Gather all task event notifications in the given interval into a single email.

The default value is set in the global config: global.cylc[scheduler][mail]task event batch interval.

Useful to prevent being overwhelmed by emails.

Changed in version 8.0.0: This item was previously called [cylc]task event mail interval.

[task parameters]
Path

flow.cylc[task parameters]

Set task parameters and parameter templates.

Define parameter values here for use in expanding parameterized tasks.

Changed in version 8.0.0: This item was previously called [cylc][parameters].

<parameter>
Path

flow.cylc[task parameters]<parameter>

Type

parameter list

A custom parameter to use in a workflow.

Examples:

  • run = control, test1, test2

  • mem = 1..5 (equivalent to 1, 2, 3, 4, 5).

  • mem = -11..-7..2 (equivalent to -11, -9, -7).

[templates]
Path

flow.cylc[task parameters][templates]

Cylc will expand each parameterized task name using a string template.

You can set templates for any parameter name here to override the default template.

Changed in version 8.0.0: This item was previously called [cylc][parameter templates].

<parameter>
Path

flow.cylc[task parameters][templates]<parameter>

Type

string

A template for a parameter.

For example:

If you set:

[task parameters]
    myparameter = 1..3
    [[templates]]
        myparameter = _run_%(myparameter)s

task name foo<myparameter> becomes foo_run_3 for run == 3.

Note

The default parameter templates are:

For integer parameters:

_p%(p)0Nd where N is the number of digits of the maximum integer value, i.e. If the largest parameter value is 3142 then N = 4.

Default for non-integer parameters:

_%(p)s e.g. foo<run> becomes foo_top for run value top.

[scheduling]
Path

flow.cylc[scheduling]

This section allows Cylc to determine when tasks are ready to run.

Any cycle points defined here without a time zone will use the time zone from flow.cylc[scheduler]cycle point time zone.

initial cycle point
Path

flow.cylc[scheduling]initial cycle point

Type

cycle point

The earliest cycle point at which any task can run.

In a cold start each cycling task (unless specifically excluded under [special tasks]) will be loaded into the workflow with this cycle point, or with the closest subsequent valid cycle point for the task.

In integer cycling, the default is 1.

The string now converts to the current datetime on the workflow host when first starting the workflow (with precision determined by flow.cylc[scheduler]cycle point format).

For more information on setting the initial cycle point relative to the current time see Setting The Initial Cycle Point Relative To The Current Time.

This item can be overridden on the command line using cylc play --initial-cycle-point or --icp.

final cycle point
Path

flow.cylc[scheduling]final cycle point

Type

string

The (optional) last cycle point at which tasks are run.

Once all tasks have reached this cycle point, the workflow will shut down.

This item can be overridden on the command line using cylc play --final-cycle-point or --fcp.

initial cycle point constraints
Path

flow.cylc[scheduling]initial cycle point constraints

Type

list

Rules to allow only some initial datetime cycle points.

Use Case

Writing a workflow where users may change the initial cycle point, but where only some initial cycle points are reasonable.

Set by defining a list of truncated time points, which the initial cycle point must match.

Examples:

  • T00, T06, T12, T18 - only at 6 hourly intervals.

  • T-30 - only at half-past an hour.

  • 01T00 - only at midnight on the first day of a month.

Note

This setting does not coerce initial cycle point=now.

final cycle point constraints
Path

flow.cylc[scheduling]final cycle point constraints

Type

list

Rules restricting permitted final cycle points.

In a cycling workflow it is possible to restrict the final cycle point by defining a list of truncated time points under the final cycle point constraints.

hold after cycle point
Path

flow.cylc[scheduling]hold after cycle point

Type

cycle point

Hold all tasks that pass this cycle point.

Unlike the final cycle point, the workflow does not shut down once all tasks have passed this point. If this item is set you can override it on the command line using --hold-after.

Changed in version 8.0.0: This item was previously called [scheduling]hold after point.

stop after cycle point
Path

flow.cylc[scheduling]stop after cycle point

Type

cycle point

Shut down the workflow after all tasks pass this cycle point.

The stop cycle point can be overridden on the command line using cylc play --stop-cycle-point=POINT

New in version 8.0.0.

cycling mode
Path

flow.cylc[scheduling]cycling mode

Type

string

Default

gregorian

Options

360day, 360_day, 365day, 365_day, 366day, 366_day, gregorian, integer

Choice of integer cycling or one of several datetime cycling calendars.

Cylc runs workflows using the proleptic Gregorian calendar by default. This setting allows you to instead choose integer cycling, or one of the other supported non-Gregorian datetime calendars: 360 day (12 months of 30 days in a year), 365 day (never a leap year) and 366 day (always a leap year).

runahead limit
Path

flow.cylc[scheduling]runahead limit

Type

string

Default

P4

The scheduler runahead limit determines how many consecutive cycle points can be active at once. The base point of the runahead calculation is the lowest-valued point with active or incomplete tasks present.

An integer interval value of Pn allows up to n+1 cycle points (including the base point) to be active at once.

The default runahead limit is P4, i.e. 5 active points including the base point.

Datetime cycling workflows can optionally use a datetime interval value instead, in which case the number of active cycle points within the interval depends on the cycling intervals present.

Changed in version 8.0.0: The integer format Pn was introduced to replace the deprecated [scheduling]max active cycle points = m (with n = m-1) and unify it with the existing datetime interval runahead limit setting.

[queues]
Path

flow.cylc[scheduling][queues]

Configuration of internal queues of tasks.

This section will allow you to limit the number of simultaneously active tasks (submitted or running) by assigning tasks to queues.

By default, a single queue called default is defined, with all tasks assigned to it and no limit to the number of those tasks which may be active.

To use a single queue for the whole workflow, but limit the number of active tasks, set [default]limit.

To add additional queues define additional sections:

[[queues]]
    [[[user_defined_queue]]]
        limit = 2
        members = TASK_FAMILY_NAME

See also

Internal Queues.

[<queue name>]
Path

flow.cylc[scheduling][queues][<queue name>]

Section heading for configuration of a single queue.

limit
Path

flow.cylc[scheduling][queues][<queue name>]limit

Type

integer

Default

0

The maximum number of active tasks allowed at any one time, for this queue.

If set to 0 this queue is not limited.

members
Path

flow.cylc[scheduling][queues][<queue name>]members

Type

list

A list of member tasks, or task family names to assign to this queue.

Assigned tasks will automatically be removed from the default queue.

[default]
Path

flow.cylc[scheduling][queues][default]

Inherits

flow.cylc[scheduling][queues][<queue name>]

The default queue for all tasks not assigned to other queues.

limit
Path

flow.cylc[scheduling][queues][default]limit

Type

integer

Default

100

Controls the total number of active tasks in the default queue.

[special tasks]
Path

flow.cylc[scheduling][special tasks]

This section is used to identify tasks with special behaviour.

Family names can be used in special task lists as shorthand for listing all member tasks.

clock-trigger
Path

flow.cylc[scheduling][special tasks]clock-trigger

Type

list

Legacy clock trigger definitions.

Deprecated since version 8.0.0: Please read External Triggers before using the older clock triggers described in this section.

Clock-trigger tasks (see Clock Triggers) wait on a wall clock time specified as an offset from their own cycle point.

Example:

foo(PT1H30M), bar(PT1.5H), baz

external-trigger
Path

flow.cylc[scheduling][special tasks]external-trigger

Type

list

Legacy external trigger definition section.

Deprecated since version 8.0.0: Please read External Triggers before using the older mechanism described in this section.

Externally triggered tasks (see Push External Triggers) wait on external events reported via the cylc ext-trigger command. To constrain triggers to a specific cycle point, include $CYLC_TASK_CYCLE_POINT in the trigger message string and pass the cycle point to the cylc ext-trigger command.

clock-expire
Path

flow.cylc[scheduling][special tasks]clock-expire

Type

list

Don’t submit jobs if they are very late in wall clock time.

Clock-expire tasks enter the expired state and skip job submission if too far behind the wall clock when they become ready to run.

The expiry time is specified as an offset from wall-clock time; typically it should be negative - see Clock-Expire Triggers.

Note

The offset:

  • May be positive or negative

  • The offset may be omitted if it is zero.

Example:

PT1H - 1 hour

sequential
Path

flow.cylc[scheduling][special tasks]sequential

Type

list

A list of tasks which automatically depend on their own previous-cycle instance.

Tip

Recommend best practice is now to use explicit inter-cycle triggers rather than sequential tasks.

[xtriggers]
Path

flow.cylc[scheduling][xtriggers]

This section is for External Trigger function declarations - see External Triggers.

<xtrigger name>
Path

flow.cylc[scheduling][xtriggers]<xtrigger name>

Type

xtrigger function signature

Any user-defined event trigger function declarations and corresponding labels for use in the graph.

See External Triggers for details.

Example:

``my_trigger(arg1, arg2, kwarg1, kwarg2):PT10S``
[graph]
Path

flow.cylc[scheduling][graph]

The workflow graph is defined under this section.

You can plot the dependency graph as you work on it, with cylc graph.

Changed in version 8.0.0: This item was previously called [runtime][dependencies][graph].

<recurrence>
Path

flow.cylc[scheduling][graph]<recurrence>

Type

string

The recurrence defines the sequence of cycle points for which the dependency graph is valid.

Cycle points should be specified in our ISO 8601 derived sequence syntax, or as integers, in integer cycling mode:

Example Recurrences:

datetime cycling:
  • R1 - once at the initial cycle point

  • T00,T06,T12,T18 - daily at 00:00, 06:00, 12:00 & 18:00

  • PT6H - every six hours starting at the initial cycle point

integer cycling:
  • R1 - once at the initial cycle point

  • P2 - every other cycle

  • P3,P5 - every third or fifth cycle

Note

Unlike other Cylc configurations duplicate recurrences are additive and do not override.

For example this:

[scheduling]
    [[graph]]
        R1 = a => b
        R1 = c => d

Is equivalent to this:

[scheduling]
    [[graph]]
        R1 = """
            a => b
            c => d
        """

See Graph Types for more on recurrence expressions, and how multiple graphs combine.

The value should be a dependency graph the given recurrence. Syntax examples follow; see also Scheduling Configuration and Task Triggering.

Example Graph Strings:

# baz and waz both trigger off bar
foo => bar => baz & waz

# bar triggers off foo[-P1D-PT6H]
foo[-P1D-PT6H] => bar

# faz triggers off a message output of baz
baz:out1 => faz

# Y triggers if X starts executing
X:start => Y

# Y triggers if X fails
X:fail => Y

# bar triggers if foo[-PT6H] fails
foo[-PT6H]:fail => bar

# Y suicides if X succeeds
X => !Y

# Z triggers if X succeeds or fails
X | X:fail => Z

# Z triggers if X succeeds or fails
X:finish => Z

# general conditional triggers
(A | B & C ) | D => foo

# bar triggers if foo is successfully submitted
foo:submit => bar

# bar triggers if submission of foo fails
foo:submit-fail => bar

Changed in version 8.0.0: This item was previously called [runtime][dependencies][graph][<recurrence>]graph.

[runtime]
Path

flow.cylc[runtime]

This section is used to specify settings for tasks to be run.

You can specify:

  • What scripts or commands you want to execute.

  • Which compute resource (platform) you wish to use.

  • How to run your task.

If multiple tasks need the same settings, they can share settings by inheriting them from one or more other tasks.

Precedence is determined by the same C3 linearization algorithm used to find the method resolution order in Python language class hierarchies.

See also

For details and examples see Task Configuration.

[<namespace>]
Path

flow.cylc[runtime][<namespace>]

The name(s) of one or more tasks or task families.

A namespace (i.e. task or family name) or a comma-separated list of namespace names, and repeat as needed to define all tasks in the workflow. Names may contain letters, digits, underscores, and hyphens. A namespace represents a group or family of tasks if other namespaces inherit from it, or a task if no others inherit from it.

Important

Names may not contain : or ..

See task namespace rules.

Examples of legal values:

  • [foo]

  • [foo, bar, baz]

If multiple names are listed the subsequent settings apply to all.

All tasks or task families inherit initially from root, which can be explicitly configured to provide or override default settings for all tasks in the workflow.

platform
Path

flow.cylc[runtime][<namespace>]platform

Type

string

The name of a compute resource defined in global.cylc[platforms] or global.cylc[platform groups].

The platform specifies the host(s) that the tasks’ jobs will run on and where (if necessary) files need to be installed, and what job runner will be used.

New in version 8.0.0.

inherit
Path

flow.cylc[runtime][<namespace>]inherit

Type

list

A list of the immediate parent(s) of this task or task family.

If no parents are listed default is root.

script
Path

flow.cylc[runtime][<namespace>]script

Type

string

The main custom script run from the job script.

It can be an external command or script, or inlined scripting.

See Job Script Variables for the list of variables available in the task execution environment.

See also Job Scripts.

Other user-defined script items:

Example:

my_script.sh
init-script
Path

flow.cylc[runtime][<namespace>]init-script

Type

string

Custom script run by the job script before the task execution environment is configured.

By running before the task execution environment is configured, this script does not have access to any workflow or task environment variables. It can be an external command or script, or inlined scripting. The original intention for this item was to allow remote tasks to source login scripts to configure their access to cylc, but this should no longer be necessary.

See also Job Scripts.

Other user-defined script items:

Example:

echo "Hello World"
env-script
Path

flow.cylc[runtime][<namespace>]env-script

Type

string

Custom script run by the job script between the cylc-defined environment (workflow and task identity, etc.) and the user-defined task runtime environment.

The env-script has access to the Cylc environment (and the task environment has access to variables defined by this scripting). It can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

Example:

echo "Hello World"
err-script
Path

flow.cylc[runtime][<namespace>]err-script

Type

string

Script run when a job error is detected.

Custom script to be run at the end of the error trap, which is triggered due to failure of a command in the job script or trappable job kill.

The output of this script will always be sent to STDERR and $1 is set to the name of the signal caught by the error trap. The script should be fast and use very little system resource to ensure that the error trap can return quickly. Companion of exit-script, which is executed on job success. It can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

Example:

echo "Uh oh, received ${1}"
exit-script
Path

flow.cylc[runtime][<namespace>]exit-script

Type

string

Custom script run at the very end of successful job execution, just before the job script exits.

The exit-script should execute very quickly. Companion of err-script, which is executed on job failure. It can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

Example:

rm -f "$TMP_FILES"
pre-script
Path

flow.cylc[runtime][<namespace>]pre-script

Type

string

Custom script run by the job script immediately before script.

The pre-script can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

Example:

echo "Hello from workflow ${CYLC_WORKFLOW_ID}!"
post-script
Path

flow.cylc[runtime][<namespace>]post-script

Type

string

Custom script run by the job script immediately after script.

The post-script can be an external command or script, or inlined scripting.

See also Job Scripts.

Other user-defined script items:

Example:

echo "Hello World"
work sub-directory
Path

flow.cylc[runtime][<namespace>]work sub-directory

Type

string

The directory from which tasks are executed.

Job scripts are executed from within work directories created automatically under the workflow run directory. A task can get its own work directory from $CYLC_TASK_WORK_DIR (or $PWD if it does not cd elsewhere at runtime). The default directory path contains task name and cycle point, to provide a unique workspace for every instance of every task. If several tasks need to exchange files and simply read and write from their from current working directory, setting work sub-directory can be used to override the default to make them all use the same workspace.

The top level share and work directory location can be changed (e.g. to a large data area) by a global config setting (see global.cylc[install][symlink dirs]).

Caution

If you omit cycle point from the work sub-directory path successive instances of the task will share the same workspace. Consider the effect on cycle point offset housekeeping of work directories before doing this.

Example:

$CYLC_TASK_CYCLE_POINT/shared/

execution polling intervals
Path

flow.cylc[runtime][<namespace>]execution polling intervals

Type

time interval list

List of intervals at which to poll status of job execution.

The default value is set in the global config: global.cylc[platforms][<platform name>]execution polling intervals.

Cylc can poll running jobs to catch problems that prevent task messages from being sent back to the workflow, such as hard job kills, network outages, or unplanned job host shutdown.

The last interval in the list is used repeatedly until the job completes.

Multipliers can be used as shorthand as in the example below.

Example:

5*PT2M, PT5M

Note that if the polling global.cylc[platforms][<platform name>]communication method is used then Cylc relies on polling to detect all task state changes, so you may want to configure more frequent polling.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]execution polling intervals.

execution retry delays
Path

flow.cylc[runtime][<namespace>]execution retry delays

Type

time interval list

Cylc can automate resubmission of failed jobs.

Execution retry delays is a list of ISO 8601 durations which tell Cylc how long to wait before the next try.

The job environment variable $CYLC_TASK_TRY_NUMBER increments with each automatic retry, allowing you to vary task behaviour between retries.

Tasks only go to the failed state if job execution fails with no retries left.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]execution retry delays.

execution time limit
Path

flow.cylc[runtime][<namespace>]execution time limit

Type

time interval

Set the execution (wallclock) time limit of a job.

For background and at job runners Cylc runs the job’s script using the timeout command. For other job runners Cylc will convert execution time limit to a directive.

If a job exceeds its execution time limit Cylc can poll the job multiple times. You can set polling intervals using global.cylc[platforms][<platform name>]execution time limit polling intervals

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]execution time limit.

submission polling intervals
Path

flow.cylc[runtime][<namespace>]submission polling intervals

Type

time interval list

List of intervals at which to poll status of job submission.

The default value is set in the global config: global.cylc[platforms][<platform name>]submission polling intervals.

Cylc can poll submitted jobs to catch problems that prevent the submitted job from executing at all, such as deletion from an external job runner queue.

The last value is used repeatedly until the task starts running.

Multipliers can be used as shorthand as in the example below.

Example:

5*PT2M, PT5M

Note that if the polling global.cylc[platforms][<platform name>]communication method is used then Cylc relies on polling to detect all task state changes, so you may want to configure more frequent polling.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]submission polling intervals.

submission retry delays
Path

flow.cylc[runtime][<namespace>]submission retry delays

Type

time interval list

Cylc can automatically resubmit jobs after submission failures.

The default value is set in the global config: global.cylc[platforms][<platform name>]submission retry delays.

Submission retry delays is a list of ISO 8601 durations which tell Cylc how long to wait before the next try.

The job environment variable $CYLC_TASK_SUBMIT_NUMBER increments with each job submission attempt.

Tasks only go to the submit-failed state if job submission fails with no retries left.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>][job]submission retry delays.

[meta]
Path

flow.cylc[runtime][<namespace>][meta]

Metadata for the task or task family.

The meta section contains metadata items for this task or family namespace. The items title, description and URL are pre-defined and are used by Cylc. Others can be user-defined and passed to task event handlers to be interpreted according to your needs. For example, the value of an “importance” item could determine how an event handler responds to task failure events.

Any workflow meta item can now be passed to task event handlers by prefixing the string template item name with workflow_, for example:

[runtime]
    [[root]]
        [[[events]]]
            failed handlers = """
                send-help.sh \
                    %(workflow_title)s \
                    %(workflow_importance)s \
                    %(title)s
             """
title
Path

flow.cylc[runtime][<namespace>][meta]title

Type

string

A single line description of this task or task family.

It is displayed by the cylc list command and can be retrieved from running tasks with the cylc show command.

description
Path

flow.cylc[runtime][<namespace>][meta]description

Type

string

A multi-line description of this task or task family.

It is retrievable from running tasks with the cylc show command.

URL
Path

flow.cylc[runtime][<namespace>][meta]URL

Type

string

A URL link to task documentation for this task or task family.

The templates %(workflow)s and %(task)s will be replaced with the actual workflow ID and task name.

Deprecated since version 8.0.0: The %(suite_name)s template variable is deprecated, please use %(workflow)s.

The %(task_name)s template variable is deprecated, please use %(task)s.

See also [meta]URL.

Example:

http://my-site.com/workflows/%(workflow)s/%(task)s.html

<custom metadata>
Path

flow.cylc[runtime][<namespace>][meta]<custom metadata>

Type

string

Any user-defined metadata item.

These, like title, description and URL. can be passed to task event handlers to be interpreted according to your needs. For example, the value of an “importance” item could determine how an event handler responds to task failure events.

[simulation]
Path

flow.cylc[runtime][<namespace>][simulation]

Task configuration for workflow simulation and dummy run modes.

For a full description of simulation and dummy run modes see Simulation Modes.

default run length
Path

flow.cylc[runtime][<namespace>][simulation]default run length

Type

time interval

Default

PT10S

The default simulated job run length.

Used if flow.cylc[runtime][<namespace>]execution time limit and flow.cylc[runtime][<namespace>][simulation]speedup factor are not set.

speedup factor
Path

flow.cylc[runtime][<namespace>][simulation]speedup factor

Type

float

Simulated run length = speedup factor * execution time limit.

If flow.cylc[runtime][<namespace>]execution time limit is set, the task simulated run length is computed by dividing it by this factor.

time limit buffer
Path

flow.cylc[runtime][<namespace>][simulation]time limit buffer

Type

time interval

Default

PT30S

For dummy jobs flow.cylc[runtime][<namespace>]execution time limit is extended by time limit buffer.

The time limit buffer is added to prevent dummy jobs being killed after exceeding the execution time limit.

fail cycle points
Path

flow.cylc[runtime][<namespace>][simulation]fail cycle points

Type

list

Configure simulated or dummy jobs to fail at certain cycle points.

Example:

  • all - all instance of the task will fail

  • 2017-08-12T06, 2017-08-12T18 - these instances of the task will fail

fail try 1 only
Path

flow.cylc[runtime][<namespace>][simulation]fail try 1 only

Type

boolean

Default

True

If True only the first run of the task instance will fail, otherwise retries will fail too.

disable task event handlers
Path

flow.cylc[runtime][<namespace>][simulation]disable task event handlers

Type

boolean

Default

True

If True configured task event handlers will not be called in simulation or dummy modes.

[environment filter]
Path

flow.cylc[runtime][<namespace>][environment filter]

This section contains environment variable inclusion and exclusion lists that can be used to filter the inherited environment.

This is not intended as an alternative to a well-designed inheritance hierarchy that provides each task with just the variables it needs.

Filters can improve workflows with tasks which inherit a lot of environment variables: Filters can make it clear which variables each task uses.

You can use filters as explicit “task environment interfaces”. They make sure that variables filtered out of the inherited environment are not used. However, using filters in this way will make your workflow definition longer.

Note

Environment filtering is done after inheritance is completely worked out, not at each level on the way, so filter lists in higher-level namespaces only have an effect if they are not overridden by descendants.

include
Path

flow.cylc[runtime][<namespace>][environment filter]include

Type

list

If given, only variables named in this list will be included from the inherited environment.

Other variables will be filtered out. Variables may also be explicitly excluded by an exclude list.

exclude
Path

flow.cylc[runtime][<namespace>][environment filter]exclude

Type

list

Variables named in this list will be filtered out of the inherited environment.

Variables may also be implicitly excluded by omission from an include list.

[job]
Path

flow.cylc[runtime][<namespace>][job]

Deprecated since version 8.0.0: This is used to select a matching platform. It will be removed in a future version of Cylc 8. Please set a suitable platform in flow.cylc[runtime][<namespace>]platform instead. See the migration guide.

This section configures the means by which cylc submits task job scripts to run.

batch system
Path

flow.cylc[runtime][<namespace>][job]batch system

Type

string

Deprecated since version 8.0.0: Kept for back compatibility but replaced by global.cylc[platforms][<platform name>]job runner.

Batch/queuing system (aka job runner) to submit task jobs to.

batch submit command template
Path

flow.cylc[runtime][<namespace>][job]batch submit command template

Type

string

Deprecated since version 8.0.0: Kept for back compatibility but replaced by global.cylc[platforms][<platform name>]job runner command template.

Override the default job submission command for the chosen batch system.

[remote]
Path

flow.cylc[runtime][<namespace>][remote]

Job host settings.

Deprecated since version 8.0.0: This is used to select a matching platform. It will be removed in a future version of Cylc 8. Please set a suitable platform in flow.cylc[runtime][<namespace>]platform instead. See the migration guide.

host
Path

flow.cylc[runtime][<namespace>][remote]host

Type

string

Hostname or IP address of the job host.

Deprecated since version 8.0.0: This is used to select a matching platform. It will be removed in a future version of Cylc 8. Please set a suitable platform in flow.cylc[runtime][<namespace>]platform instead. See the migration guide.

owner
Path

flow.cylc[runtime][<namespace>][remote]owner

Type

string

Your username on the job host, if different from that on the scheduler host.

Warning

This setting is obsolete at Cylc 8.

See documentation on changes to remote owner

retrieve job logs
Path

flow.cylc[runtime][<namespace>][remote]retrieve job logs

Type

boolean

Whether to retrieve job logs from the job platform.

Deprecated since version 8.0.0: This is used to select a matching platform. It will be removed in a future version of Cylc 8. Please set a suitable platform in flow.cylc[runtime][<namespace>]platform instead. See the migration guide.

retrieve job logs max size
Path

flow.cylc[runtime][<namespace>][remote]retrieve job logs max size

Type

string

The maximum size of job logs to retrieve.

Can be anything accepted by the --max-size=SIZE option of rsync.

Deprecated since version 8.0.0: This is used to select a matching platform. It will be removed in a future version of Cylc 8. Please set a suitable platform in flow.cylc[runtime][<namespace>]platform instead. See the migration guide.

retrieve job logs retry delays
Path

flow.cylc[runtime][<namespace>][remote]retrieve job logs retry delays

Type

time interval list

Configure retries for unsuccessful job log retrieval.

If there is a significant delay between job completion and logs appearing in their final location (due to the job runner) you can configure time intervals here to delay the first and subsequent retrieval attempts.

Deprecated since version 8.0.0: This is used to select a matching platform. It will be removed in a future version of Cylc 8. Please set a suitable platform in flow.cylc[runtime][<namespace>]platform instead. See the migration guide.

[events]
Path

flow.cylc[runtime][<namespace>][events]

Configure the task event handling system.

The default value is set in the global config: global.cylc[task events].

See also flow.cylc[scheduler][events] for workflow events.

Task event handlers are scripts to run when task events occur.

Event handlers can be stored in the workflow bin/ directory, or anywhere the scheduler environment $PATH. They should return quickly.

Multiple event handlers can be specified as a list of command line templates. For supported template variables see Task Event Template Variables. Python template substitution syntax is used: String Formatting Operations in the Python documentation.

execution timeout
Path

flow.cylc[runtime][<namespace>][events]execution timeout

Type

time interval

If a task has not finished after the specified interval, the execution timeout event handler(s) will be called.

The default value is set in the global config: global.cylc[task events]execution timeout.

handlers
Path

flow.cylc[runtime][<namespace>][events]handlers

Type

list

Specify a list of command lines or command line templates as generic task event handlers for each event set in flow.cylc[runtime][<namespace>][events]handler events.

The default value is set in the global config: global.cylc[task events]handlers.

handler events
Path

flow.cylc[runtime][<namespace>][events]handler events

Type

list

Specify the events for which the general task event handlers flow.cylc[runtime][<namespace>][events]handlers should be invoked.

The default value is set in the global config: global.cylc[task events]handler events.

Example:

submission failed, failed
handler retry delays
Path

flow.cylc[runtime][<namespace>][events]handler retry delays

Type

time interval list

Specify an initial delay before running an event handler command and any retry delays in case the command returns a non-zero code.

The default value is set in the global config: global.cylc[task events]handler retry delays.

The default behaviour is to run an event handler command once without any delay.

Example:

PT10S, PT1M, PT5M
mail events
Path

flow.cylc[runtime][<namespace>][events]mail events

Type

list

Specify the events for which notification emails should be sent.

The default value is set in the global config: global.cylc[task events]mail events.

Example:

submission failed, failed
submission timeout
Path

flow.cylc[runtime][<namespace>][events]submission timeout

Type

time interval

If a task has not started after the specified interval, the submission timeout event handler(s) will be called.

The default value is set in the global config: global.cylc[task events]submission timeout.

expired handlers
Path

flow.cylc[runtime][<namespace>][events]expired handlers

Type

list

Handlers to run if this task has expired.

Caution

Changes to the scheduling algorithm in Cylc 8 mean this event will not be triggered until the expired task is ready to run. Earlier expired-task detection will be implemented in a future Cylc release.

late offset
Path

flow.cylc[runtime][<namespace>][events]late offset

Type

time interval

Offset from cycle point, in real time, at which this task is considered to be “running late” (i.e. the time by which it would normally have started running).

Caution

Changes to the scheduling algorithm in Cylc 8 mean this event will not be triggered until the late task is ready to run. Earlier late-task detection will be implemented in a future Cylc release.

late handlers
Path

flow.cylc[runtime][<namespace>][events]late handlers

Type

list

Handlers to run if this task is late.

Caution

Due to changes to the Cylc 8 scheduling algorithm this event is unlikely to occur until the task is about to submit anyway.

submitted handlers
Path

flow.cylc[runtime][<namespace>][events]submitted handlers

Type

list

Handlers to run when this task is submitted.

started handlers
Path

flow.cylc[runtime][<namespace>][events]started handlers

Type

list

Handlers to run when this task starts executing.

succeeded handlers
Path

flow.cylc[runtime][<namespace>][events]succeeded handlers

Type

list

Handlers to run if this task succeeds.

failed handlers
Path

flow.cylc[runtime][<namespace>][events]failed handlers

Type

list

Handlers to run if this task fails.

submission failed handlers
Path

flow.cylc[runtime][<namespace>][events]submission failed handlers

Type

list

Handlers to run if submission of this task fails.

warning handlers
Path

flow.cylc[runtime][<namespace>][events]warning handlers

Type

list

Handlers to run if this task runs cylc message with severity level “WARNING”.

critical handlers
Path

flow.cylc[runtime][<namespace>][events]critical handlers

Type

list

Handlers to run if this task runs cylc message with severity level “CRITICAL”.

retry handlers
Path

flow.cylc[runtime][<namespace>][events]retry handlers

Type

list

Handlers to run if this task failed but is retrying.

submission retry handlers
Path

flow.cylc[runtime][<namespace>][events]submission retry handlers

Type

list

Handlers to run if a job failed to submit but is retrying.

execution timeout handlers
Path

flow.cylc[runtime][<namespace>][events]execution timeout handlers

Type

list

Handlers to run if this task execution exceeds flow.cylc[runtime][<namespace>]execution time limit.

submission timeout handlers
Path

flow.cylc[runtime][<namespace>][events]submission timeout handlers

Type

list

Handlers to run if this task exceeds flow.cylc[runtime][<namespace>][events]submission timeout in the submitted state.

custom handlers
Path

flow.cylc[runtime][<namespace>][events]custom handlers

Type

list

Handlers to run if this task runs cylc message with severity level “CUSTOM”.

[mail]
Path

flow.cylc[runtime][<namespace>][mail]

Email notification settings for task events.

New in version 8.0.0.

from
Path

flow.cylc[runtime][<namespace>][mail]from

Type

string

Specify an alternate from: email address for event notifications.

Changed in version 8.0.0: This item was previously called [runtime][task][events]mail from

to
Path

flow.cylc[runtime][<namespace>][mail]to

Type

string

A list of email addresses to send task event notifications.

The list can be any address accepted by the mail command.

Changed in version 8.0.0: This item was previously called [runtime][task][events]mail to

[workflow state polling]
Path

flow.cylc[runtime][<namespace>][workflow state polling]

Configure automatic workflow polling tasks as described in Triggering Off Of Tasks In Other Workflows.

The items in this section reflect options and defaults of the cylc workflow-state command, except that the target workflow name and the --task, --cycle, and --status options are taken from the graph notation.

Changed in version 8.0.0: This item was previously called [runtime][<namespace>]suite state polling.

user
Path

flow.cylc[runtime][<namespace>][workflow state polling]user

Type

string

Username of your account on the workflow host.

The polling cylc workflow-state command will be run on the remote account.

host
Path

flow.cylc[runtime][<namespace>][workflow state polling]host

Type

string

The hostname of the target workflow.

The polling cylc workflow-state command will be run there.

interval
Path

flow.cylc[runtime][<namespace>][workflow state polling]interval

Type

time interval

Polling interval.

max-polls
Path

flow.cylc[runtime][<namespace>][workflow state polling]max-polls

Type

integer

The maximum number of polls before timing out and entering the “failed” state.

message
Path

flow.cylc[runtime][<namespace>][workflow state polling]message

Type

string

Wait for the task in the target workflow to receive a specified message rather than achieve a state.

run-dir
Path

flow.cylc[runtime][<namespace>][workflow state polling]run-dir

Type

string

Specify the location of the top level cylc-run directory for the other workflow.

For your own workflows, there is no need to set this as it is always ~/cylc-run/. But for other workflows, (e.g those owned by others), or mirrored workflow databases use this item to specify the location of the top level cylc run directory (the database should be in a the same place relative to this location for each workflow).

verbose mode
Path

flow.cylc[runtime][<namespace>][workflow state polling]verbose mode

Type

boolean

Run the polling cylc workflow-state command in verbose output mode.

[environment]
Path

flow.cylc[runtime][<namespace>][environment]

The user defined task execution environment.

Variables defined here can refer to cylc workflow and task identity variables, which are exported earlier in the job script. Variable assignment expressions can use cylc utility commands because access to cylc is also configured earlier in the script.

See also

Job Environment.

You can also specify job environment templates here for parameterized tasks.

<variable>
Path

flow.cylc[runtime][<namespace>][environment]<variable>

Type

string

A custom user defined variable for a task execution environment.

The order of definition is preserved that each variable can refer to previously defined variables. Values are passed through to the job script without evaluation or manipulation by Cylc (with the exception of valid Python string templates that match parameterized task names - see below), so any variable assignment expression that is legal in the job submission shell can be used. White space around the = is allowed (as far as cylc’s flow.cylc parser is concerned these are just normal configuration items).

Examples:

FOO = $HOME/bar/baz
BAR = ${FOO}$GLOBALVAR
BAZ = $( echo "hello world" )
WAZ = ${FOO%.jpg}.png
NEXT_CYCLE = $( cylc cycle-point --offset=PT6H )
ZAZ = "${FOO#bar}"
# ^ quoted to escape the flow.cylc comment character
DICE = [$((($RANDOM % 6) + 1)) $((($RANDOM % 6) + 1))]

For parameter environment templates, use Python string templates for parameter substitution. This is only relevant for parameterized tasks. The job script will export the named variables specified here (in addition to the standard CYLC_TASK_PARAM_<key> variables), with the template strings substituted with the parameter values.

Examples:

MYNUM = %(i)d
MYITEM = %(item)s
MYFILE = /path/to/%(i)03d/%(item)s

Changed in version 7.8.7/7.9.2: Parameter environment templates (previously in [runtime][X][parameter environment templates]) have moved here.

[directives]
Path

flow.cylc[runtime][<namespace>][directives]

Job runner (batch scheduler) directives.

The default value is set in the global config: global.cylc[platforms][<platform name>][directives].

Supported for use with job runners:

  • pbs

  • slurm

  • loadleveler

  • lsf

  • sge

  • slurm_packjob

  • moab

Directives are written to the top of the job script in the correct format for the job runner.

Specifying directives individually like this allows use of default directives for task families which can be individually overridden at lower levels of the runtime namespace hierarchy.

<directive>
Path

flow.cylc[runtime][<namespace>][directives]<directive>

Type

string

Example directives for the built-in job runner handlers are shown in Supported Job Submission Methods.

[outputs]
Path

flow.cylc[runtime][<namespace>][outputs]

Register custom task outputs for use in message triggering in this section (Message Triggers)

<output>
Path

flow.cylc[runtime][<namespace>][outputs]<output>

Type

string

Task output messages (Message Triggers).

The item name is used to select the custom output message in graph trigger notation.

Examples:

out1 = "sea state products ready"
out2 = "NWP restart files completed"

Task outputs are validated by cylc.flow.unicode_rules.TaskOutputValidator.

class cylc.flow.unicode_rules.TaskOutputValidator

The rules for valid task outputs/message triggers:

  • cannot contain: :

[parameter environment templates]
Path

flow.cylc[runtime][<namespace>][parameter environment templates]

Deprecated since version 7.8.7/7.9.2: Parameter environment templates have moved to flow.cylc[runtime][<namespace>][environment].

This was done to allow users to control the order of definition of the variables. This section will be removed in a future version of Cylc 8.

For the time being, the contents of this section will be prepended to the [environment] section when running a workflow.

<parameter>
Path

flow.cylc[runtime][<namespace>][parameter environment templates]<parameter>

Type

string

Deprecated since version 7.8.7/7.9.2: Parameter environment templates have moved to flow.cylc[runtime][<namespace>][environment].