Cylc commands identify workflows via their names, which are relative path names
under the cylc-run directory,
~/cylc-run/ by default.
Workflows can be grouped together under sub-directories. E.g. there are three
workflows in the example below:
$ cylc scan --state=all --name nwp --format=tree nwp ├─oper │ ├─ region1 │ └─ region2 └─test └─ region1
This chapter will demonstrate how to install a workflow from an arbitrary
location, called a source directory.
cylc install will create a new run directory in the cylc-run directory
for each installation of a workflow.
The Cylc Install Command
Workflows can be installed with the
cylc install command, which creates
the run directory structure and some service files underneath it.
It is possible to run a workflow without installation by writing it
directly in the run directory.
However, we recommend that you write your workflow in a source
directory and use
cylc install to create a fresh run directory.
Using Cylc Install
cylc install accepts as its only argument either:
a workflow source name (see Configurable Source Directories below) e.g.
a path to the source directory e.g.
/path/to/foo/bar(note that relative paths must begin with
./to distinguish them from workflow source names)
If no argument is supplied, the current working directory is used as the source directory.
To avoid confusion,
cylc install does not permit any of the following
reserved directory names:
The table below illustrates several command line options that control naming
of run directories for installed workflows (the current working directory in
these examples is
Results in Installation in Directory
Configurable Source Directories
You can configure a list of directories that
cylc install SOURCE_NAME
and the GUI will search for source directories inside, using
global.cylc[install]source dirs. For example, if you have
# global.cylc [install] source dirs = ~/cylc-src, ~/roses
cylc install dogs/fido will search for a workflow source directory
~/cylc-src/dogs/fido, or, failing that,
~/roses/dogs/fido, and install
the first match (into
cylc install creates numbered run directories, i.e.
~/cylc-run/<workflow-name>/run<number>, provided the options
--no-run-name are not used. The run number automatically
increments each time
cylc install is run, and a symlink
created/updated to point to the run.
Example: A typical run directory structure, after three executions of
cylc install will look as follows.
├── _cylc-install │ └── source -> /home/cylc-src/test-flow ├── run1 │ ├── flow.cylc │ └── log │ └── install │ └── <time-stamp>-install.log ├── run2 │ ├── flow.cylc │ └── log │ └── install │ └── <time-stamp>-install.log ├── run3 │ ├── flow.cylc │ └── log │ └── install │ └── <time-stamp>-install.log └── runN -> /home/cylc-run/test-flow/run3
The numbered runs option may be overridden, using either the
As an alternative to numbered runs, it is possible to name the runs, using the
In this case, the
runN symlink will not be created.
This option cannot be used if numbered runs are already present. Likewise,
numbered runs cannot be used if named runs are already present.
The Cylc Install Process
There are two main parts of the
cylc install process.
Symlinking of Directories
1. File Installation
Installation will involve copying the files found in the source directory into
a new run directory. If you wish to install files into an existing run
cylc reinstall, see Reinstalling a Workflow.
Excluding Items From Installation
By default, cylc install will exclude
To configure excluded files and directories from the file installation,
.cylcignore file in your source directory, this supports
The following example will detail how to install a workflow, including configuring files to be excluded from the installation.
We will look at running the cylc install command inside the directory
~/cylc-src/test-flow with the following directory structure:
$ pwd /home/cylc-src/test-flow
$ tree -all ├── .cylcignore ├── dir1 │ ├── another-file │ └── file ├── dir2 │ ├── another-file │ └── file ├── file1 ├── file2 ├── file3 ├── flow.cylc ├── textfile1.txt └── textfile2.txt
We wish to omit any files matching the pattern
*.txt, the file
file1, the contents of
dir1 and the contents of
dir2 including the
$ cat .cylcignore *.txt file1 dir1/* dir2
Now we are ready to install our workflow.
$ cylc install INSTALLED test-flow from home/cylc-src/test-flow -> home/cylc-run/test-flow/run1
Looking at the directory structure that has been created
$ tree -all home/cylc-run/test-flow/run1 ├── dir1 ├── file2 ├── file3 ├── flow.cylc ├── log │ └── install │ └── <time-stamp>-install.log └── .service
Automatically Generated Directories and Files
cylc install will generate some extra files in your workflow run
The service directory will be created in preparation for running the workflow. This is needed to store essential files used by Cylc.
_cylc-installdirectory containing a
sourcesymlink to the source directory. This is needed to enable Cylc to determine the original workflow source for
installdirectory in the workflow’s log directory, with a time-stamped install log file containing information about the installation.
Cylc plugins (such as Cylc Rose) may generate additional files.
Reinstalling a Workflow
To apply changes from your source directory to the installed run directory,
cylc reinstall. Changes made will be recorded in the workflow log
$ cylc reinstall myflow/run1
Cylc will determine the source directory and update your workflow.
Returning to the example from above (see Example Installation).
The source directory,
~/cylc-src/test-flow has been altered as follows:
$ tree -all ~/cylc-src/test-flow ├── .cylcignore ├── dir1 │ ├── another-file │ └── file ├── dir2 │ ├── another-file │ └── file ├── dir3 │ ├── another-file │ └── file ├── file1 ├── file2 ├── file3 ├── flow.cylc ├── textfile1.txt └── textfile2.txt
$ cat .cylcignore *.txt file1 dir2
We wish to update our
~/cylc-run/test-flow/run1 with the directories
$ cylc reinstall test-flow/run1
The run directory now looks as follows:
$ tree -all home/cylc-run/test-flow/run1 ├── dir1 │ ├── another-file │ └── file ├── dir3 │ ├── another-file │ └── file ├── file2 ├── file3 ├── flow.cylc ├── log │ └── install │ └── <time-stamp>-install.log │ └── <time-stamp>-reinstall.log └── .service
If your workflow needs to create or install scripts or executables at runtime
and you don’t want Cylc to delete them on re-installation, you can use
lib/python directories in the
workflow share directory.
There are some occasions when installation is expected to fail:
_cylc-installdirectories exist in the source directory
The workflow name is an absolute path or invalid
Workflow names are validated by
- class cylc.flow.unicode_rules.WorkflowNameValidator
The rules for valid workflow names:
must be between 1 and 254 characters long
cannot start with:
can only contain: alphanumeric,
The workflow name contains a directory name that is any of these reserved filenames:
The install would create nested install directories. Neither a new installation in a subdirectory of an existing one, nor a directory containing an existing installation are permitted. For example, having installed a workflow in
baryou would be unable to install one in
foo └── bar ├── _cylc-install ├── baz ├── run1 └── runN
This means you cannot install using
--no-run-namefor a workflow that has installed numbered/named runs, nor can you install numbered/named runs for a workflow where
Trying to install a workflow into an already existing run directory.
cylc reinstallshould be used for this, see Reinstalling a Workflow.
The source directory path does not match the source directory path of a previous installation. i.e. running
~/cylc-src/my-flow, followed by running