Installation

Warning

Cylc 8.0b1 is an early full-system Cylc 8 preview release

It has a fully functional Python 3 workflow service and CLI that can run existing Cylc workflows.

BUT:

  • It is not production-ready yet

    Use the latest Cylc 7.9 (Python 2.7) or 7.8 (Python 2.6) release for production systems.

  • Do not use it where security is a concern

  • The UI includes a prototype “tree view” with no control capability - we are working on other views, and controls

  • Data update in the UI is via polling at 5 second intervals, and monolithic - future releases will use WebSockets and incremental update

Cylc runs on Unix-like operating systems including Mac OS though at present Cylc only tested on Linux.

Quick Installation

Via Conda (recommended):

# install cylc with the browser-GUI
$ conda install cylc

Via pip:

Note

We recommend using a virtual environment.

Note

Requires Python 3.7

# install the "core" cylc package
$ pip install cylc-flow

# install the browser-GUI
$ pip install cylc-uiserver

Non-Python Requirements

The following dependencies are not installed by Conda or pip:

  • bash

  • GNU coreutils

  • mail (for automated email functionality)

The following dependencies are installed by Conda but not by pip:

Installing On Mac OS

Cylc requires some extra packages to function on Mac OS, we recommend installing them using the Homebrew package manager:

brew install bash coreutils gnu-sed

You will need to prepend the coreutils and gnu-sed installations to your $PATH, follow the instructions in the brew install output.

Note

atrun (the at command) does not run out-of-the-box on Mac OS for security reasons and must be manually enabled.

Note

Newer version of Mac OS set zsh as the default shell (as opposed to bash). You do not need to change this but be aware that Cylc uses bash which has a subtly different syntax.

Warning

Cylc currently has DNS issues with the latest versions of Mac OS, to get around them the following diff must be made to the installed source code:

diff --git a/cylc/flow/hostuserutil.py b/cylc/flow/hostuserutil.py
index 1b0bfc37d..73d5c9f98 100644
--- a/cylc/flow/hostuserutil.py
+++ b/cylc/flow/hostuserutil.py
@@ -113,7 +113,7 @@ class HostUtil(object):
         """Return the extended info of the current host."""
         if target not in self._host_exs:
             if target is None:
-                target = socket.getfqdn()
+                target = socket.gethostname()
             try:
                 self._host_exs[target] = socket.gethostbyname_ex(target)
             except IOError as exc:

Warning

For Mac OS Versions 10.15.0 (Catalina) and higher SSH is disabled by default. The ability to SSH into your Mac OS box may be required for certain Cylc installations.

See the Apple support page for instructions on enabling SSH.

Site Installation

For multi-user installation we recommend using Conda and installing Cylc components only where required.

The Cylc Packages

Cylc is split into a number of packages providing different functionality:

Cylc Flow

Provides the scheduler “kernel” of Cylc along with the command-line.

Cylc UI Server

Provides the “Cylc Hub” and the browser-based “Cylc GUI”.

Installation Types

The places where you would want to install Cylc fall into the following “roles”:

User Machines

The boxes where users write workflows and interact with the command line.

Cylc Servers

The boxes where workflows are run.

Job Hosts

The systems where jobs are run (e.g. supercomputers or clusters)

Note

There may be a mix of purposes between the different “roles”, for example it is possible to use job hosts as cylc servers and vice versa.

Managing Environments

In order for Cylc to run the correct environment must be activated. Cylc can not do this automatically.

We recommend using a wrapper script to activate the correct environment and call the cylc command.

An example can be found in usr/bin/cylc, this should be installed to a location in the system searchable $PATH e.g. /usr/local/bin.

Configuration

Cylc uses “sane and safe” defaults and is suitable for use “out of the box”, however, many things may need to be configured e.g:

  • Job hosts

  • Communication methods

  • User/Site preferences

Cylc Flow

Cylc Flow is configured by the global.cylc file which supports configuration of the system on both a site and user basis.

Note

Prior to Cylc 8, global.cylc was named global.rc, but that name is no longer supported.

Bash Profile

Cylc invokes bash -l to run job scripts so sites and users should ensure that their bash login scripts configure the environment correctly for use with Cylc and don’t source unwanted systems or echo to stdout.