Global Configuration

Cylc global configurations use the .cylc file format.

global.cylc

The global configuration which defines default Cylc Flow settings for a user or site.

To view your global config, run:

$ cylc config

Cylc will attempt to load the global configuration (global.cylc) from a hierarchy of locations, including the site directory (defaults to /etc/cylc/flow/) and the user directory (~/.cylc/flow/). For example at Cylc version 8.0.1, the hierarchy would be, in order of ascending priority:

<site-conf-path>/flow/global.cylc
<site-conf-path>/flow/8/global.cylc
<site-conf-path>/flow/8.0/global.cylc
<site-conf-path>/flow/8.0.1/global.cylc
~/.cylc/flow/global.cylc
~/.cylc/flow/8/global.cylc
~/.cylc/flow/8.0/global.cylc
~/.cylc/flow/8.0.1/global.cylc

Where <site-conf-path> is /etc/cylc/flow/ by default but can be changed by CYLC_SITE_CONF_PATH.

A setting in a file lower down in the list will override the same setting from those higher up (but if a setting is present in a file higher up and not in any files lower down, it will not be overridden).

The following environment variables can change the files which are loaded:

CYLC_CONF_PATH

If set this bypasses the default site/user configuration hierarchy used to load the Cylc Flow global configuration.

This should be set to a directory containing a global.cylc file.

CYLC_SITE_CONF_PATH

By default the site configuration is located in /etc/cylc/. For installations where this is not convenient, this path can be overridden by setting CYLC_SITE_CONF_PATH to point at another location.

Configuration for different Cylc components should be in sub-directories within this location.

For example to configure Cylc Flow you could do the following:

$CYLC_SITE_CONF_PATH/
`-- flow/
    `-- global.cylc

Note

The global.cylc file can be templated using Jinja2 variables. See Jinja2.

Changed in version 8.0.0: Prior to Cylc 8, global.cylc was named global.rc, but that name is no longer supported.

[hub]

Path:: global.cylc[hub]

Configure the public URL of Jupyter Hub.

If configured, the cylc gui command will open a web browser at this location rather than starting a standalone server when called.

See also

The cylc hub Architecture for fuller details.
UI Server Configuration for practical details.

url

Path:: global.cylc[hub]url
Type:: string

Added in version 8.3.0.

Where Jupyter Hub is used a url can be provided for routing on execution of cylc gui command.

[scheduler]

Path:: global.cylc[scheduler]
Defaults For:: flow.cylc[scheduler].

Settings for the scheduler.

UTC mode

Path:: global.cylc[scheduler]UTC mode
Type:: boolean
Default:: False
Default For:: flow.cylc[scheduler]UTC mode.

If True, UTC will be used as the time zone for timestamps in the logs. If False, the local/system time zone will be used.

process pool size

Path:: global.cylc[scheduler]process pool size
Type:: integer
Default:: 4

Maximum number of concurrent processes used to execute external job submission, event handlers, and job poll and kill commands

See also

External Command Execution.

Changed in version 8.0.0: Moved into the [scheduler] section from the top level.

process pool timeout

Path:: global.cylc[scheduler]process pool timeout
Type:: time interval
Default:: PT10M

After this interval Cylc will kill long running commands in the process pool.

See also

External Command Execution.

Note

The default is set quite high to avoid killing important processes when the system is under load.

Changed in version 8.0.0: Moved into the [scheduler] section from the top level.

auto restart delay

Path:: global.cylc[scheduler]auto restart delay
Type:: time interval

Maximum number of seconds the auto-restart mechanism will delay before restarting workflows.

When a host is set to automatically shutdown/restart it waits a random period of time between zero and auto restart delay seconds before beginning the process. This is to prevent large numbers of workflows from restarting simultaneously.

See also

Workflow Migration

Changed in version 8.0.0: This item was previously called global.rc[suite servers]auto restart delay.

[run hosts]

Path:: global.cylc[scheduler][run hosts]

Configure workflow hosts and ports for starting workflows.

Additionally configure host selection settings specifying how to determine the most suitable run host at any given time from those configured.

Changed in version 8.0.0: This item was previously called [suite servers].

available

Path:: global.cylc[scheduler][run hosts]available
Type:: spaceless list

A list of workflow run hosts.

Cylc will choose one of these hosts for a workflow to start on. (Unless an explicit host is provided as an option to the cylc play --host=<myhost> command.)

Changed in version 8.0.0: This item was previously called [suite servers]run hosts.

ports

Path:: global.cylc[scheduler][run hosts]ports
Type:: integer range
Default:: 43001 .. 43101

The range of ports for Cylc to use to run workflows.

The minimum and maximum port numbers in the format min .. max.

Changed in version 8.0.0: This item was previously called [suite servers]run ports. It can no longer be used to define a non-contiguous port range.

condemned

Path:: global.cylc[scheduler][run hosts]condemned
Type:: absolute host list

These hosts will not be used to run jobs.

If workflows are already running on condemned hosts, Cylc will shut them down and restart them on different hosts.

See also

Workflow Migration

Changed in version 8.0.0: This item was previously called [suite servers]condemned hosts.

ranking

Path:: global.cylc[scheduler][run hosts]ranking
Type:: string

Rank and filter run hosts based on system information.

Ranking can be used to provide load balancing to ensure no single run host is overloaded. It also provides thresholds beyond which Cylc will not attempt to start new schedulers on a host.

This should be a multiline string containing Python expressions to rank and/or filter hosts. All psutil attributes are available for use in these expressions.

Ranking

Rankings are expressions which return numerical values. The host which returns the lowest value is chosen. Examples:

# rank hosts by cpu_percent
cpu_percent()

# rank hosts by 15min average of server load
getloadavg()[2]

# rank hosts by the number of cores
# (multiple by -1 because the lowest value is chosen)
-1 * cpu_count()

Threshold

Thresholds are expressions which return boolean values. If a host returns a False value that host will not be selected. Examples:

# filter out hosts with a CPU utilisation of 70% or above
cpu_percent() < 70

# filter out hosts with less than 1GB of RAM available
virtual_memory().available > 1000000000

# filter out hosts with less than 1GB of disk space
# available on the "/" mount
disk_usage('/').free > 1000000000

Combining

Multiple rankings and thresholds can be combined in this section e.g:

# filter hosts
cpu_percent() < 70
disk_usage('/').free > 1000000000

# rank hosts by CPU count
1 / cpu_count()
# if two hosts have the same CPU count
# then rank them by CPU usage
cpu_percent()

Changed in version 8.0.0: This item was previously called [suite servers][run host select]rank.

[host self-identification]

Path:: global.cylc[scheduler][host self-identification]

How Cylc determines and shares the identity of the workflow host.

The workflow host’s identity must be determined locally by cylc and passed to running tasks (via $CYLC_WORKFLOW_HOST) so that task messages can target the right workflow on the right host.

Changed in version 8.0.0: This item was previously called [suite host self-identification].

method

Path:: global.cylc[scheduler][host self-identification]method
Type:: string
Default:: name
Options:: name, address, hardwired

Determines how cylc finds the identity of the workflow host.

Options:

name: (The default method) Self-identified host name. Cylc asks the workflow host for its host name. This should resolve on task hosts to the IP address of the workflow host; if it doesn’t, adjust network settings or use one of the other methods.
address: Automatically determined IP address (requires target). Cylc attempts to use a special external “target address” to determine the IP address of the workflow host as seen by remote task hosts.
hardwired: (only to be used as a last resort) Manually specified host name or IP address (requires host) of the workflow host.

Changed in version 8.0.0: This item was previously called [suite host self-identification].

target

Path:: global.cylc[scheduler][host self-identification]target
Type:: string
Default:: google.com

Target for use by the address self-identification method.

If your workflow host sees the internet, a common address such as google.com will do; otherwise choose a host visible on your intranet.

Changed in version 8.0.0: This item was previously called [suite host self-identification].

host

Path:: global.cylc[scheduler][host self-identification]host
Type:: string

The name or IP address of the workflow host used by the hardwired self-identification method.

Changed in version 8.0.0: This item was previously called [suite host self-identification].

[events]

Path:: global.cylc[scheduler][events]
Default For:: flow.cylc[scheduler][events].

Configure the workflow event handling system.

handlers

Path:: global.cylc[scheduler][events]handlers
Type:: list
Default For:: flow.cylc[scheduler][events]handlers.

Configure event handlers that run when certain workflow events occur.

handler events

Path:: global.cylc[scheduler][events]handler events
Type:: list
Default For:: flow.cylc[scheduler][events]handler events.

Specify the events for which workflow event handlers should be invoked.

mail events

Path:: global.cylc[scheduler][events]mail events
Type:: list
Default For:: flow.cylc[scheduler][events]mail events.

Specify the workflow events for which notification emails should be sent.

startup handlers

Path:: global.cylc[scheduler][events]startup handlers
Type:: list
Default For:: flow.cylc[scheduler][events]startup handlers.

Handlers to run at scheduler startup.

shutdown handlers

Path:: global.cylc[scheduler][events]shutdown handlers
Type:: list
Default For:: flow.cylc[scheduler][events]shutdown handlers.

Handlers to run at scheduler shutdown.

abort handlers

Path:: global.cylc[scheduler][events]abort handlers
Type:: list
Default For:: flow.cylc[scheduler][events]abort handlers.

Handlers to run if the scheduler shuts down with error status due to a configured timeout or a fatal error condition.

workflow timeout

Path:: global.cylc[scheduler][events]workflow timeout
Type:: time interval
Default For:: flow.cylc[scheduler][events]workflow timeout.

Workflow timeout interval. The timer starts counting down at scheduler startup. It resets on workflow restart.

workflow timeout handlers

Path:: global.cylc[scheduler][events]workflow timeout handlers
Type:: list
Default For:: flow.cylc[scheduler][events]workflow timeout handlers.

Handlers to run if the workflow timer times out.

abort on workflow timeout

Path:: global.cylc[scheduler][events]abort on workflow timeout
Type:: boolean
Default:: False
Default For:: flow.cylc[scheduler][events]abort on workflow timeout.

Whether the scheduler should shut down immediately with error status if the workflow timer times out.

stall handlers

Path:: global.cylc[scheduler][events]stall handlers
Type:: list
Default For:: flow.cylc[scheduler][events]stall handlers.

Handlers to run if the scheduler stalls.

stall timeout

Path:: global.cylc[scheduler][events]stall timeout
Type:: time interval
Default:: PT1H
Default For:: flow.cylc[scheduler][events]stall timeout.

The length of a timer which starts if the scheduler stalls.

stall timeout handlers

Path:: global.cylc[scheduler][events]stall timeout handlers
Type:: list
Default For:: flow.cylc[scheduler][events]stall timeout handlers.

Handlers to run if the stall timer times out.

abort on stall timeout

Path:: global.cylc[scheduler][events]abort on stall timeout
Type:: boolean
Default:: True
Default For:: flow.cylc[scheduler][events]abort on stall timeout.

Whether the scheduler should shut down immediately with error status if the stall timer times out.

inactivity timeout

Path:: global.cylc[scheduler][events]inactivity timeout
Type:: time interval
Default For:: flow.cylc[scheduler][events]inactivity timeout.

Scheduler inactivity timeout interval. The timer resets when any workflow activity occurs.

inactivity timeout handlers

Path:: global.cylc[scheduler][events]inactivity timeout handlers
Type:: list
Default For:: flow.cylc[scheduler][events]inactivity timeout handlers.

Handlers to run if the inactivity timer times out.

abort on inactivity timeout

Path:: global.cylc[scheduler][events]abort on inactivity timeout
Type:: boolean
Default:: False
Default For:: flow.cylc[scheduler][events]abort on inactivity timeout.

Whether the scheduler should shut down immediately with error status if the inactivity timer times out.

restart timeout

Path:: global.cylc[scheduler][events]restart timeout
Type:: time interval
Default:: PT2M
Default For:: flow.cylc[scheduler][events]restart timeout.

How long to wait for intervention on restarting a completed workflow. The timer stops if any task is triggered.

[mail]

Path:: global.cylc[scheduler][mail]
Defaults For:: flow.cylc[scheduler][mail].

Settings for the scheduler to send event emails.

from

Path:: global.cylc[scheduler][mail]from
Type:: string
Default For:: flow.cylc[scheduler][mail]from.

Specify an alternative from email address for workflow event notifications.

smtp

Path:: global.cylc[scheduler][mail]smtp
Type:: string

Specify the SMTP server for sending workflow event email notifications.

This cannot be configured in flow.cylc.

Example:

smtp.yourorg

to

Path:: global.cylc[scheduler][mail]to
Type:: string
Default For:: flow.cylc[scheduler][mail]to.

A list of email addresses that event notifications should be sent to.

footer

Path:: global.cylc[scheduler][mail]footer
Type:: string
Default For:: flow.cylc[scheduler][mail]footer.

Specify a string or string template for footers of emails sent for both workflow and task events.

task event batch interval

Path:: global.cylc[scheduler][mail]task event batch interval
Type:: time interval
Default:: PT5M
Default For:: flow.cylc[scheduler][mail]task event batch interval.

Gather all task event notifications in the given interval into a single email.

[main loop]

Path:: global.cylc[scheduler][main loop]
Defaults For:: flow.cylc[scheduler][main loop].

Configuration of main loop plugins for the scheduler.

plugins

Path:: global.cylc[scheduler][main loop]plugins
Type:: list
Default:: 'health check', 'reset bad hosts'

Configure the default main loop plugins to use when starting new workflows.

Only enabled plugins are loaded. Plugins can be enabled in two ways:

Globally:: To enable a plugin for all workflows add it to this setting.
Per-Run:: To enable a plugin for a one-off run of a workflow, specify it on the command line with cylc play --main-loop.

Hint

This appends to the configured list of plugins rather than overriding it.

Added in version 8.0.0.

[<plugin name>]

Path:: global.cylc[scheduler][main loop][<plugin name>]
Defaults For:: flow.cylc[scheduler][main loop][<plugin name>].

Configure a main loop plugin.

interval

Path:: global.cylc[scheduler][main loop][<plugin name>]interval
Type:: time interval
Default:: PT10M
Default For:: flow.cylc[scheduler][main loop][<plugin name>]interval.

Interval at which the plugin is invoked.

[health check]

Path:: global.cylc[scheduler][main loop][health check]
Inherits:: global.cylc[scheduler][main loop][<plugin name>]

Checks the integrity of the workflow run directory.

Added in version 8.0.0.

interval

Path:: global.cylc[scheduler][main loop][health check]interval
Type:: time interval
Default:: PT10M

Interval at which the plugin is invoked.

Added in version 8.0.0.

[auto restart]

Path:: global.cylc[scheduler][main loop][auto restart]
Inherits:: global.cylc[scheduler][main loop][<plugin name>]

Automatically migrates workflows between servers.

For more information see:

Added in version 8.0.0.

interval

Path:: global.cylc[scheduler][main loop][auto restart]interval
Type:: time interval
Default:: PT10M

Interval at which the plugin is invoked.

Added in version 8.0.0.

[reset bad hosts]

Path:: global.cylc[scheduler][main loop][reset bad hosts]
Inherits:: global.cylc[scheduler][main loop][<plugin name>]

Periodically clear the scheduler list of unreachable (bad) hosts.

Added in version 8.0.0.

interval

Path:: global.cylc[scheduler][main loop][reset bad hosts]interval
Type:: time interval
Default:: PT30M

Interval at which the plugin is invoked.

Added in version 8.0.0.

[logging]

Path:: global.cylc[scheduler][logging]

Settings for the workflow event log.

The workflow event log, held under the workflow run directory, is maintained as a rolling archive. Logs are rolled over (backed up and started anew) when they reach a configurable limit size.

Changed in version 8.0.0: This item was previously called [suite logging].

rolling archive length

Path:: global.cylc[scheduler][logging]rolling archive length
Type:: integer
Default:: 15

How many rolled logs to retain in the archive.

maximum size in bytes

Path:: global.cylc[scheduler][logging]maximum size in bytes
Type:: integer
Default:: 1000000

Workflow event logs are rolled over when they reach this file size.

[install]

Path:: global.cylc[install]

Configure directories and files to be installed on remote hosts.

Added in version 8.0.0.

max depth

Path:: global.cylc[install]max depth
Type:: integer
Default:: 4

How many directory levels deep Cylc should look for installed workflows in the cylc-run directory.

This also sets the limit on how deep a workflow ID can be before cylc install will refuse to install it. For example, if set to 4, cylc install one/two/three/four will fail, because the resultant workflow ID would be one/two/three/four/run1, which is 5 levels deep. (However, cylc install one/two/three/four --no-run-name would work.)

Note

A high value may cause a slowdown of Cylc commands such install, scan and clean if there are many run directories in the cylc-run directory for Cylc to check, or if the filesystem is slow (e.g. NFS).

Added in version 8.0.0.

source dirs

Path:: global.cylc[install]source dirs
Type:: list
Default:: ~/cylc-src

List of paths that Cylc searches for workflows to install.

All workflow source directories in these locations will also show up in the GUI, ready for installation.

Note

If workflow source directories of the same name exist in more than one of these paths, only the first one will be picked up.

Added in version 8.0.0.

[symlink dirs]

Path:: global.cylc[install][symlink dirs]

Configure alternate workflow run directory locations.

Symlinks from the the standard $HOME/cylc-run locations will be created.

Added in version 8.0.0.

[<install target>]

Path:: global.cylc[install][symlink dirs][<install target>]

Host on which to create the symlinks.

Added in version 8.0.0.

Note

Changed in version 8.4.0: This section can be a comma separated list.

Example

For example:

[a, b]
    setting = x
[a]
    another_setting = y

Will become:

[a]
    setting = x
[b]
    setting = x
[a]
    another_setting = y

Which will then be combined according to the rules for Cylc config syntax.

run

Path:: global.cylc[install][symlink dirs][<install target>]run
Type:: string

Alternative location for the run dir.

If specified, the workflow run directory will be created in <this-path>/cylc-run/<workflow-id> and a symbolic link will be created from $HOME/cylc-run/<workflow-id>. If not specified the workflow run directory will be created in $HOME/cylc-run/<workflow-id>. All the workflow files and the .service directory get installed into this directory.

Added in version 8.0.0.

share

Path:: global.cylc[install][symlink dirs][<install target>]share
Type:: string

Alternative location for the share dir.

If specified the workflow share directory will be created in <this-path>/cylc-run/<workflow-id>/share and a symbolic link will be created from $HOME/cylc-run/<workflow-id>/share. If not specified the workflow log directory will be created in $HOME/cylc-run/<workflow-id>/share.

Added in version 8.0.0.

share/cycle

Path:: global.cylc[install][symlink dirs][<install target>]share/cycle
Type:: string

Alternative location for the share/cycle dir.

If specified the workflow share/cycle directory will be created in <this-path>/cylc-run/<workflow-id>/share/cycle and a symbolic link will be created from $HOME/cylc-run/<workflow-id>/share/cycle. If not specified the workflow log directory will be created in $HOME/cylc-run/<workflow-id>/share/cycle.

Added in version 8.0.0.

log

Path:: global.cylc[install][symlink dirs][<install target>]log
Type:: string

Alternative location for the log dir.

If specified the workflow log directory will be created in <this-path>/cylc-run/<workflow-id>/log and a symbolic link will be created from $HOME/cylc-run/<workflow-id>/log. If not specified the workflow log directory will be created in $HOME/cylc-run/<workflow-id>/log.

Added in version 8.0.0.

log/job

Path:: global.cylc[install][symlink dirs][<install target>]log/job
Type:: string

Alternative location for the log/job dir.

If specified the workflow log/job directory will be created in <this-path>/cylc-run/<workflow-id>/log/job and a symbolic link will be created from $HOME/cylc-run/<workflow-id>/log/job. If not specified the workflow log directory will be created in $HOME/cylc-run/<workflow-id>/log/job.

Added in version 8.4.0.

work

Path:: global.cylc[install][symlink dirs][<install target>]work
Type:: string

Alternative location for the work dir.

If specified the workflow work directory will be created in <this-path>/cylc-run/<workflow-id>/work and a symbolic link will be created from $HOME/cylc-run/<workflow-id>/work. If not specified the workflow log directory will be created in $HOME/cylc-run/<workflow-id>/work.

Added in version 8.0.0.

[platforms]

Path:: global.cylc[platforms]

Platforms allow you to define compute resources available at your site.

A platform consists of a group of one or more hosts which share a file system and a job runner (batch system).

A platform must allow interaction with the same job from any of its hosts.

Added in version 8.0.0.

[<platform name>]

Path:: global.cylc[platforms][<platform name>]

Configuration defining a platform.

Many of these settings have replaced those of the same name from the old Cylc 7 suite.rc[runtime][<namespace>][job]/[remote] and global.rc[hosts][<host>] sections.

Platform names can be regular expressions: If you have a set of compute resources such as bigmachine1, bigmachine2 or desktop0000, .., desktop9999 one would define platforms with names [[bigmachine[12]]] and [[desktop[0-9]{4}]].

Cylc searches for a matching platform in the reverse of the definition order to allow user defined platforms to override site defined platforms. This means, for example, that if [[a.*]] were set at the bottom of a configuration any platform name beginning with “a” would return that platform.

Note

Each possible match to the definition regular expression is considered a separate platform.

If you had a supercomputer with multiple login nodes this would be a single platform with multiple hosts.

Warning

[platforms][localhost] may be set, to override default settings, but regular expressions which match “localhost” may not. Use comma separated lists instead:

[platforms]
    [[localhost|cylc-server-..]]  # error
    [[localhost, cylc-server-..]]  # ok

See also

Platforms in the Cylc 8 migration guide.
Platform Configuration, an administrator’s guide to platform configurations.

Note

This section can be a comma separated list.

Example

For example:

[a, b]
    setting = x
[a]
    another_setting = y

Will become:

[a]
    setting = x
[b]
    setting = x
[a]
    another_setting = y

Which will then be combined according to the rules for Cylc config syntax.

Added in version 8.0.0.

[meta]

Path:: global.cylc[platforms][<platform name>][meta]

Metadata for this platform or platform group.

Allows writers of platform configurations to add information about platform usage. There are no-preset items because Cylc does not use any platform (or group) metadata internally.

Users can then see information about defined platforms using:

cylc config -i [platforms]
cylc config -i [platform groups]

See also

Platform Configuration

Added in version 8.0.0.

<custom metadata>

Path:: global.cylc[platforms][<platform name>][meta]<custom metadata>
Type:: string

Any user-defined metadata item.

Added in version 8.0.0.

hosts

Path:: global.cylc[platforms][<platform name>]hosts
Type:: list

A list of hosts from which the job host can be selected using [selection]method.

All hosts should share a file system.

Added in version 8.0.0.

job runner

Path:: global.cylc[platforms][<platform name>]job runner
Type:: string
Default:: background

The system used to run jobs on the platform.

Examples:

background

slurm

pbs

See also

List of built-in Job Runners

Added in version 8.0.0: (Replaces the deprecated setting flow.cylc[runtime][<namespace>][job]batch system.)

job runner command template

Path:: global.cylc[platforms][<platform name>]job runner command template
Type:: string

Set the command used by the chosen job runner.

The template’s %(job)s will be substituted by the job file path.

Added in version 8.0.0: (Replaces the deprecated setting flow.cylc[runtime][<namespace>][job]batch submit command template.)

shell

Path:: global.cylc[platforms][<platform name>]shell
Type:: string
Default:: /bin/bash

Changed in version 8.0.0: Moved from suite.rc[runtime][<namespace>]job.

communication method

Path:: global.cylc[platforms][<platform name>]communication method
Type:: string
Default:: zmq
Options:: poll, ssh, zmq

The means by which task progress messages are reported back to the running workflow.

..rubric:: Options:

zmq: Direct client-server TCP communication via network ports
poll: The workflow polls for task status (no task messaging)
ssh: Use non-interactive ssh for task communications

For more information, see Tracking Job Status.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]task communication method.

submission polling intervals

Path:: global.cylc[platforms][<platform name>]submission polling intervals
Type:: time interval list
Default:: PT15M
Default For:: flow.cylc[runtime][<namespace>]submission polling intervals.

List of intervals at which to poll status of job submission.

submission retry delays

Path:: global.cylc[platforms][<platform name>]submission retry delays
Type:: time interval list
Default For:: flow.cylc[runtime][<namespace>]submission retry delays.

Cylc can automatically resubmit jobs after submission failures.

execution polling intervals

Path:: global.cylc[platforms][<platform name>]execution polling intervals
Type:: time interval list
Default:: PT15M
Default For:: flow.cylc[runtime][<namespace>]execution polling intervals.

List of intervals at which to poll status of job execution.

execution time limit polling intervals

Path:: global.cylc[platforms][<platform name>]execution time limit polling intervals
Type:: time interval list
Default:: PT1M, PT2M, PT7M

List of intervals after execution time limit to poll jobs.

If a job exceeds its execution time limit, Cylc can poll more frequently to detect the expected job completion quickly. The last interval in the list is used repeatedly until the job completes. Multipliers can be used as shorthand as in the example below.

Example:

5*PT2M, PT5M

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>][batch systems] [<system>]execution time limit polling.

ssh command

Path:: global.cylc[platforms][<platform name>]ssh command
Type:: string
Default:: ssh -oBatchMode=yes -oConnectTimeout=10

A communication command used to invoke commands on this platform.

Not used on the workflow host unless you run local tasks under another user account. The value is assumed to be ssh with some initial options or a command that implements a similar interface to ssh.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]ssh command.

rsync command

Path:: global.cylc[platforms][<platform name>]rsync command
Type:: string
Default:: rsync

Command used for file installation.

This supports POSIX compliant rsync implementations e.g. GNU or BSD.

Added in version 8.0.0.

use login shell

Path:: global.cylc[platforms][<platform name>]use login shell
Type:: boolean
Default:: True

Whether to use a login shell or not for remote command invocation.

By default, Cylc runs remote SSH commands using a login shell:

ssh user@host 'bash --login cylc ...'

which will source the following files (in order):

/etc/profile
~/.bash_profile
~/.bash_login
~/.profile

For more information on login shells see the “Invocation” section of the Bash man pages.

For security reasons some institutions do not allow unattended commands to start login shells, so you can turn off this behaviour to get:

ssh user@host 'cylc ...'

which will use the default shell on the remote machine, sourcing ~/.bashrc (or ~/.cshrc) to set up the environment.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]use login shell.

cylc path

Path:: global.cylc[platforms][<platform name>]cylc path
Type:: string

The path containing the cylc executable on a remote platform.

This may be necessary if the cylc executable is not in the $PATH for an ssh call. Test whether this is the case by using ssh <host> command -v cylc.

This path is used for remote invocations of the cylc command and is added to the $PATH in job scripts for the configured platform.

Note

If use login shell=True (the default) then an alternative approach is to add cylc to the $PATH in the system or user Bash profile files (e.g. ~/.bash_profile).

Tip

For multi-version installations this should point to the Cylc wrapper script rather than the cylc executable itself.

See Managing Environments for more information on the wrapper script.

Changed in version 8.0.0: Moved from suite.rc[runtime][<namespace>][job] cylc executable.

global init-script

Path:: global.cylc[platforms][<platform name>]global init-script
Type:: string

A per-platform script which is run before other job scripts.

This should be used sparingly to perform any shell configuration that cannot be performed via other means.

Changed in version 8.0.0: The global init-script now runs before any job scripting which introduces caveats outlined below.

Warning

The global init-script has the following caveats, as compared to the other task script-* items:

The script is not covered by error trapping.
The job environment is not available to this script.
In debug mode this script will not be included in xtrace output.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]global init-script.

copyable environment variables

Path:: global.cylc[platforms][<platform name>]copyable environment variables
Type:: list

A list containing the names of the environment variables to be copied from the scheduler to a job.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]copyable environment variables.

retrieve job logs

Path:: global.cylc[platforms][<platform name>]retrieve job logs
Type:: boolean

Whether to retrieve job logs from the job platform.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]retrieve job logs. (Replaces the deprecated setting flow.cylc[runtime][<namespace>][remote]retrieve job logs.)

retrieve job logs command

Path:: global.cylc[platforms][<platform name>]retrieve job logs command
Type:: string
Default:: rsync -a

The command used to retrieve job logs from the job platform.

Note

The default command (rsync -a) means that the retrieved files (and the directories above including job/log) get the same permissions as on the remote host. This can cause problems if the remote host uses different permissions to the scheduler host (e.g. no world read access). To avoid this problem you can set the command to rsync -a --no-p --no-g --chmod=ugo=rwX which means the retrieved files get the default permissions used on the scheduler host.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]retrieve job logs command.

retrieve job logs max size

Path:: global.cylc[platforms][<platform name>]retrieve job logs max size
Type:: string

The maximum size of job logs to retrieve.

Can be anything accepted by the --max-size=SIZE option of rsync.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]retrieve job logs max size. (Replaces the deprecated setting flow.cylc[runtime][<namespace>][remote]retrieve job logs max size.)

retrieve job logs retry delays

Path:: global.cylc[platforms][<platform name>]retrieve job logs retry delays
Type:: time interval list

Configure retries for unsuccessful job log retrieval.

If there is a significant delay between job completion and logs appearing in their final location (due to the job runner) you can configure time intervals here to delay the first and subsequent retrieval attempts.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]retrieve job logs retry delays. (Replaces the deprecated setting flow.cylc[runtime][<namespace>][remote]retrieve job logs retry delays.)

tail command template

Path:: global.cylc[platforms][<platform name>]tail command template
Type:: string
Default:: tail -n +1 --follow=name %(filename)s

A command template (with %(filename)s substitution) to tail-follow job logs this platform, by cylc cat-log.

Warning

You are are unlikely to need to override this. Doing so may adversely affect the UI log view.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>]tail command template.

err tailer

Path:: global.cylc[platforms][<platform name>]err tailer
Type:: string

A command template (with %(job_id)s substitution) that can be used to tail-follow the stderr stream of a running job if SYSTEM does not use the normal log file location while the job is running. This setting overrides tail command template.

Examples:

# for PBS
qcat -f -e %(job_id)s

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>][batch systems] [<system>]err tailer.

out tailer

Path:: global.cylc[platforms][<platform name>]out tailer
Type:: string

A command template (with %(job_id)s substitution) that can be used to tail-follow the stdout stream of a running job if SYSTEM does not use the normal log file location while the job is running. This setting overrides tail command template.

Examples:

# for PBS
qcat -f -o %(job_id)s

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>][batch systems] [<system>]out tailer.

err viewer

Path:: global.cylc[platforms][<platform name>]err viewer
Type:: string

A command template (with %(job_id)s substitution) that can be used to view the stderr stream of a running job if SYSTEM does not use the normal log file location while the job is running.

Examples:

# for PBS
qcat -e %(job_id)s

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>][batch systems] [<system>]err viewer.

out viewer

Path:: global.cylc[platforms][<platform name>]out viewer
Type:: string

A command template (with %(job_id)s substitution) that can be used to view the stdout stream of a running job if SYSTEM does not use the normal log file location while the job is running.

Examples:

# for PBS
qcat -o %(job_id)s

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>][batch systems] [<system>]out viewer.

job name length maximum

Path:: global.cylc[platforms][<platform name>]job name length maximum
Type:: integer

The maximum length for job name acceptable by a job runner on a given host. Currently, this setting is only meaningful for PBS jobs. For example, PBS 12 or older will fail a job submit if the job name has more than 15 characters; whereas PBS 13 accepts up to 236 characters.

Changed in version 8.0.0: This item was previously called global.rc[hosts][<host>][batch systems] [<system>]job name length maximum.

install target

Path:: global.cylc[platforms][<platform name>]install target
Type:: string

This defaults to the platform name. This will be used as the target for remote file installation. For example, if Platform_A shares a file system with localhost:

[platforms]
    [[Platform_A]]
        install target = localhost

Added in version 8.0.0.

clean job submission environment

Path:: global.cylc[platforms][<platform name>]clean job submission environment
Type:: boolean
Default:: False

Job submission subprocesses inherit their parent environment by default. Remote jobs inherit the default non-interactive shell environment for their platform. Jobs on the scheduler host inherit the scheduler environment (unless their job runner prevents this).

If, for example, the $PYTHON variable is different on the scheduler and the remote host the same program may run in different ways.

We recommend using a clean job submission environment for consistent handling of local and remote jobs. However, this is not the default behaviour because it prevents local jobs from running, unless $PATH contains the cylc wrapper script.

Specific environment variables can be singled out to pass through to the clean environment, if necessary.

A standard set of executable paths is passed through to clean environments, and can be added to if necessary.

Added in version 8.0.0.

job submission environment pass-through

Path:: global.cylc[platforms][<platform name>]job submission environment pass-through
Type:: list

List of environment variable names to pass through to job submission subprocesses.

$HOME is passed automatically.

You are unlikely to need this.

Added in version 8.0.0.

job submission executable paths

Path:: global.cylc[platforms][<platform name>]job submission executable paths
Type:: list

Additional executable locations to pass to the job submission subprocess beyond the standard locations /bin, /usr/bin, /usr/local/bin, /sbin, /usr/sbin, /usr/local/sbin. You are unlikely to need this.

Added in version 8.0.0.

max batch submit size

Path:: global.cylc[platforms][<platform name>]max batch submit size
Type:: integer
Default:: 100

Limits the maximum number of jobs that can be submitted at once.

Where possible Cylc will batch together job submissions to the same platform for efficiency. Submitting very large numbers of jobs can cause problems with some submission systems so for safety there is an upper limit on the number of job submissions which can be batched together.

Added in version 8.0.0.

ssh forward environment variables

Path:: global.cylc[platforms][<platform name>]ssh forward environment variables
Type:: list

A list containing the names of the environment variables to forward with SSH connections to the workflow host from the host running ‘cylc play’

Added in version 8.3.0.

[selection]

Path:: global.cylc[platforms][<platform name>][selection]

How to select a host from the list of platform hosts.

Added in version 8.0.0.

method

Path:: global.cylc[platforms][<platform name>][selection]method
Type:: string
Default:: random
Options:: random, definition order

Host selection method for the platform.

Available options

random: Choose randomly from the list of hosts. This is suitable for a pool of identical hosts.
definition order: Take the first host in the list unless that host was unreachable. In many cases this is likely to cause load imbalances, but might be appropriate if following the pattern hosts = main, backup, failsafe.

Added in version 8.0.0.

[directives]

Path:: global.cylc[platforms][<platform name>][directives]
Defaults For:: flow.cylc[runtime][<namespace>][directives].

Job runner (batch scheduler) directives.

Added in version 8.0.0.

<directive>

Path:: global.cylc[platforms][<platform name>][directives]<directive>
Type:: string

Example directives for the built-in job runner handlers are shown in Supported Job Submission Methods.

[localhost]

Path:: global.cylc[platforms][localhost]
Inherits:: global.cylc[platforms][<platform name>]

A default platform for running jobs on the the scheduler host.

This platform configures the host on which schedulers run. By default this is the host where cylc play is run, however, we often configure Cylc to start schedulers on dedicated hosts by configuring global.cylc[scheduler][run hosts]available.

This platform affects connections made to the scheduler host and any jobs run on it.

Added in version 8.0.0.

hosts

Path:: global.cylc[platforms][localhost]hosts
Type:: list
Default:: localhost

List of hosts for the localhost platform. You are unlikely to need to change this.

The scheduler hosts are configured by global.cylc[scheduler][run hosts]available. See Submitting Workflows To a Pool Of Hosts for more information.

[selection]

Path:: global.cylc[platforms][localhost][selection]
Inherits:: global.cylc[platforms][<platform name>][selection]

How to select a host on the “localhost” platform.You are unlikely to need to change this.:cylc:conf:global.cylc[platforms][<platform name>][selection]

method

Path:: global.cylc[platforms][localhost][selection]method
Type:: string
Default:: definition order

Host selection method for the “localhost” platform.

[platform groups]

Path:: global.cylc[platform groups]

Platform groups allow you to group together platforms which would all be suitable for a given job.

When Cylc submits a job it will pick a platform from a group. Cylc will then use the selected platform for all interactions with that job.

For example, if you have a group of computers without a shared file system, but otherwise identical called bigmachine01..02 you might set up a platform group [[bigmachines]]platforms=bigmachine01, bigmachine02.

See also

Platforms in the Cylc 8 migration guide.
Platform Configuration, an guide to platform configurations.

Added in version 8.0.0.

[<group>]

Path:: global.cylc[platform groups][<group>]

The name of a platform group.

[meta]

Path:: global.cylc[platform groups][<group>][meta]

Metadata for this platform or platform group.

Allows writers of platform configurations to add information about platform usage. There are no-preset items because Cylc does not use any platform (or group) metadata internally.

Users can then see information about defined platforms using:

cylc config -i [platforms]
cylc config -i [platform groups]

See also

Platform Configuration

Added in version 8.0.0.

<custom metadata>

Path:: global.cylc[platform groups][<group>][meta]<custom metadata>
Type:: string

Any user-defined metadata item.

Added in version 8.0.0.

platforms

Path:: global.cylc[platform groups][<group>]platforms
Type:: list

A list of platforms which can be selected if flow.cylc[runtime][<namespace>]platform matches the name of this platform group.

Added in version 8.0.0.

Note

Some job runners (“background”, “at”) require a single-host platform, because the job ID is only valid on the submission host.

[selection]

Path:: global.cylc[platform groups][<group>][selection]

Sets how platforms are selected from platform groups.

method

Path:: global.cylc[platform groups][<group>][selection]method
Type:: string
Default:: random
Options:: random, definition order

Method for selecting platform from group.

options:

random: Suitable for an identical pool of platforms.
definition order: Pick the first available platform from the list.

Added in version 8.0.0.

[task events]

Path:: global.cylc[task events]
Defaults For:: flow.cylc[runtime][<namespace>][events].

Configure the task event handling system.

Added in version 8.0.0.

execution timeout

Path:: global.cylc[task events]execution timeout
Type:: time interval
Default For:: flow.cylc[runtime][<namespace>][events]execution timeout.

If a task has not finished after the specified interval, the execution timeout event handler(s) will be called.

handlers

Path:: global.cylc[task events]handlers
Type:: list
Default For:: flow.cylc[runtime][<namespace>][events]handlers.

Commands to run on task handler events.

handler events

Path:: global.cylc[task events]handler events
Type:: list
Default For:: flow.cylc[runtime][<namespace>][events]handler events.

A list of events for which handlers are run.

handler retry delays

Path:: global.cylc[task events]handler retry delays
Type:: time interval list
Default For:: flow.cylc[runtime][<namespace>][events]handler retry delays.

Specify an initial delay before running an event handler command and any retry delays in case the command returns a non-zero code.

mail events

Path:: global.cylc[task events]mail events
Type:: list
Default For:: flow.cylc[runtime][<namespace>][events]mail events.

Specify the events for which notification emails should be sent.

submission timeout

Path:: global.cylc[task events]submission timeout
Type:: time interval
Default For:: flow.cylc[runtime][<namespace>][events]submission timeout.

If a task has not started after the specified interval, the submission timeout event handler(s) will be called.