Installation

Cylc 8 and its core software dependencies can be installed quickly from Conda Forge, into a conda environment; or from PyPI, into a Python 3 virtual environment.

Quick Installation

Cylc runs on Unix-like systems including Linux and Mac OS.

Via Pip (+npm)

Supported Python versions

  • cylc-flow supports Python 3.7+.

  • cylc-uiserver currently only supports Python 3.8-3.9.

Important

We recommend installing Cylc versions into virtual environments. This avoids software dependency conflicts and allows multiple Cylc versions to be installed on your system.

Without virtual environments, users can inadvertently break Cylc (or other Python programs) by pip-installing conflicting package versions to $HOME/.local, which takes precedence over central library locations.

$ pip install cylc-flow

# Install the browser-GUI (optional)
# (requires nodejs & npm)
$ pip install cylc-uiserver

# Install Rose support (optional)
$ pip install cylc-rose metomi-rose

There are also certain optional extra requirements which you may choose to install:

# Support for running the tutorial workflows
$ pip install 'cylc-flow[tutorial]'

# The GUI with multi-user (hub) support
$ pip install 'cylc-uiserver[hub]'
$ npm install configurable-http-proxy

You might also want to configure:

Non-Python Requirements

These dependencies are not installed by Conda or pip:

  • bash

  • GNU coreutils

  • ssh

  • rsync

  • mail (optional - for automated email functionality)

These dependencies are installed by Conda but not by pip:

  • Graphviz (optional - used by cylc graph for displaying workflow graphs)

  • configurable-http-proxy (optional - for multi-user setups; can also be installed using npm)

See also

Syntax Highlighting For Workflow Configuration

Installing On Mac OS

Cylc requires some extra packages to function on Mac OS. We recommend installing them using the Homebrew package manager:

$ brew install bash coreutils gnu-sed

You will need to prepend the coreutils and gnu-sed installations to your $PATH, follow the instructions in the brew install output.

Note

atrun (the at command) does not run out-of-the-box on Mac OS for security reasons and must be manually enabled.

Note

Newer version of Mac OS set zsh as the default shell (as opposed to bash). You do not need to change this but be aware that Cylc uses bash (for job scripts) which has a subtly different syntax.

Warning

For Mac OS Versions 10.15.0 (Catalina) and higher, SSH is disabled by default. The ability to SSH into your Mac OS box may be required for certain Cylc installations.

See the Apple support page for instructions on enabling SSH.

Advanced Installation

For distributed and multi-user installation we recommend using Conda and installing Cylc components only where required.

Tip

For examples of Conda environments and installation options see Conda Environments for examples and details.

The Cylc Components

Cylc is split into a number of components providing different functionality:

Cylc Flow

Provides the scheduler “kernel” of Cylc along with the command line interface.

Cylc UI Server

Provides the “Cylc Hub” and the browser-based “Cylc GUI”.

Cylc Rose

Provides support for Rose suite configurations in Cylc workflows.

Installation Types

Cylc install locations may fall into the following “roles”:

User Machines

Where users write workflows and interact with the command line.

Cylc Servers

Where Cylc schedulers run to manage workflows.

Job Hosts

Where jobs run, e.g. supercomputers or clusters

Note

These roles may overlap. For example, Cylc servers can also be job hosts.

Managing Environments

For Cylc to run, the correct environment must be activated. Cylc can not do this automatically. You may need to have multiple Cylc versions available too.

We recommend using a wrapper script named cylc to activate the correct environment before calling the environment’s cylc command.

Cylc comes with a wrapper that can be adapted to point at your Cylc environments. Extract it to a directory in your $PATH like this:

cylc get-resources cylc /path/to/cylc  # should be in $PATH
chmod +x /path/to/cylc

You may need to modify this file for your local installation e.g:

- CYLC_HOME_ROOT="${CYLC_HOME_ROOT:-/opt}"
+ CYLC_HOME_ROOT="${CYLC_HOME_ROOT:-/path/to}"

Note

Developers can set $CYLC_HOME_ROOT_ALT to point to their development environments. For example:

CYLC_HOME_ROOT_ALT=${HOME}/.conda/envs

You may wish to use the same approach for the isodatetime command, and, if using Rose for the rose and rosie commands.

To do so create a symbolic link to the wrapper, for each of these commands:

cd /path/to       # Using the path where you installed the wrapper script
ln -s cylc rose
ln -s cylc rosie
ln -s cylc isodatetime

Configuration

Cylc uses sane and safe defaults and is suitable for use “out of the box”, if all you need to do is run jobs locally in the background. However, many things may need to be configured, e.g:

Cylc Flow

Cylc Flow is configured by the global.cylc file which supports configuration of the system on both a site and user basis.

Note

Prior to Cylc 8, global.cylc was named global.rc, but that name is no longer supported.

The global.cylc file should be available on user machines (where users interact with Cylc on the command line) and on cylc servers (where Cylc schedulers run). It is not required to be available on job hosts.

More information about supported configuration items and defaults can be found: Global Configuration.

Cylc UI Server

The Cylc UI Server can be configured on a site and user basis. Guidance for configuration file storage, configuration variables and defaults can be found: UI Server Configuration.

Bash Profile

Cylc job scripts are bash scripts, which is good for manipulating files and processes, They invoke bash -l to allow environment configuration in login scripts.

Warning

Sites and users should ensure their bash login scripts configure the environment correctly for Cylc and do not write anything to stdout.

Shell Auto-Completion

Cylc provides auto-completion for the Bash shell which can save you typing:

  • Cylc commands

  • Workflow IDs

  • Cycle points

  • Task names

To extract the auto-completion file run the following command:

cylc get-resources cylc-completion.bash <path-to-copy-file>

Then follow the comments in the file to install it.

Text Editors

There is support for the .cylc file format in various text editors.

See Syntax Highlighting For Workflow Configuration for more details.