3. Installation¶
Cylc runs on Linux. It is tested quite thoroughly on modern RHEL and Ubuntu distros. Some users have also managed to make it work on other Unix variants including Apple OS X, but they are not officially tested and supported.
3.1. Third-Party Software Packages¶
Python 2 >=
2.7 is required. Python 2 >=
2.7.9 is
recommended for the best security. Python 2 should
already be installed in your Linux system.
For Cylc’s HTTPS communications layer:
- OpenSSL
- pyOpenSSL
- python-requests
- python-urllib3 - should be bundled with python-requests
The following packages are highly recommended, but are technically optional as you can construct and run suites without dependency graph visualisation or the Cylc GUIs:
PyGTK - Python bindings for the GTK+ GUI toolkit.
Note
PyGTK typically comes with your system Python 2 version. It is allegedly quite difficult to install if you need to do so for another Python version. At time of writing, for instance, there are no functional PyGTK conda packages available.
Note that we need to do ``import gtk`` in Python, not ``import pygtk``.
In Centos 7.6, for example, the Cylc GUIs run “out of the box” with the system-installed Python 2.7.5. Under the hood, the Python “gtk” package is provided by the “pygtk2” yum package. (The “pygtk” Python module, which we don’t need, is supplied by the “pygobject2” yum package).
Graphviz - graph layout engine (tested 2.36.0)
Pygraphviz - Python Graphviz interface (tested 1.2). To build this you may need some devel packages too:
- python-devel
- graphviz-devel
Note
The
cylc graph
command for static workflow visualization requires PyGTK, but we provide a separatecylc ref-graph
command to print out a simple text-format “reference graph” without PyGTK.
The Cylc Review service does not need any additional packages.
The following packages are necessary for running all the tests in Cylc:
To generate the HTML User Guide, you will need:
- Sphinx of compatible version,
>=
1.5.3 and<=
1.7.9.
In most modern Linux distributions all of the software above can be installed via the system package manager. Otherwise download packages manually and follow their native installation instructions. To check that all packages are installed properly:
$ cylc check-software
Checking your software...
Individual results:
===============================================================================
Package (version requirements) Outcome (version found)
===============================================================================
*REQUIRED SOFTWARE*
Python (2.7+, <3).....................FOUND & min. version MET (2.7.12.final.0)
*OPTIONAL SOFTWARE for the GUI & dependency graph visualisation*
Python:pygraphviz (any)...........................................NOT FOUND (-)
graphviz (any)...................................................FOUND (2.26.0)
Python:pygtk (2.0+)...............................................NOT FOUND (-)
*OPTIONAL SOFTWARE for the HTTPS communications layer*
Python:requests (2.4.2+)......................FOUND & min. version MET (2.11.1)
Python:urllib3 (any)..............................................NOT FOUND (-)
Python:OpenSSL (any)..............................................NOT FOUND (-)
*OPTIONAL SOFTWARE for the configuration templating*
Python:EmPy (any).................................................NOT FOUND (-)
*OPTIONAL SOFTWARE for the HTML documentation*
Python:sphinx (1.5.3+).........................FOUND & min. version MET (1.7.0)
===============================================================================
Summary:
****************************
Core requirements: ok
Full-functionality: not ok
****************************
If errors are reported then the packages concerned are either not installed or not in your Python search path.
Note
cylc check-software
has become quite trivial as we’ve removed or
bundled some former dependencies, but in future we intend to make it
print a comprehensive list of library versions etc. to include in with
bug reports.
To check for specific packages only, supply these as arguments to the
check-software
command, either in the form used in the output of
the bare command, without any parent package prefix and colon, or
alternatively all in lower-case, should the given form contain capitals. For
example:
$ cylc check-software graphviz Python urllib3
With arguments, check-software provides an exit status indicating a collective pass (zero) or a failure of that number of packages to satisfy the requirements (non-zero integer).
3.2. Software Bundled With Cylc¶
Cylc bundles several third party packages which do not need to be installed separately.
- cherrypy 6.0.2 (slightly modified): a pure
Python HTTP framework that we use as a web server for communication between
server processes (suite server programs) and client programs (running tasks,
GUIs, CLI commands).
- Client communication is via the Python requests library if available (recommended) or else pure Python via urllib2.
- Jinja2 2.10: a full featured template engine for Python, and its dependency MarkupSafe 0.23; both BSD licensed.
- the xdot graph viewer (modified), LGPL licensed.
3.3. Installing Cylc¶
Cylc releases can be downloaded from GitHub.
The wrapper script usr/bin/cylc
should be installed to
the system executable search path (e.g. /usr/local/bin/
) and
modified slightly to point to a location such as /opt
where
successive Cylc releases will be unpacked side by side.
To install Cylc, unpack the release tarball in the right location, e.g.
/opt/cylc-7.8.2
, type make
inside the release
directory, and set site defaults - if necessary - in a site global config file
(below).
Make a symbolic link from cylc
to the latest installed version:
ln -s /opt/cylc-7.8.2 /opt/cylc
. This will be invoked by the
central wrapper if a specific version is not requested. Otherwise, the
wrapper will attempt to invoke the Cylc version specified in
$CYLC_VERSION
, e.g. CYLC_VERSION=7.8.2
. This variable
is automatically set in task job scripts to ensure that jobs use the same Cylc
version as their parent suite server program. It can also be set by users,
manually or in login scripts, to fix the Cylc version in their environment.
Installing subsequent releases is just a matter of unpacking the new tarballs
next to the previous releases, running make
in them, and copying
in (possibly with modifications) the previous site global config file.
3.3.1. Local User Installation¶
It is easy to install Cylc under your own user account if you don’t have
root or sudo access to the system: just put the central Cylc wrapper in
$HOME/bin/
(making sure that is in your $PATH
) and
modify it to point to a directory such as $HOME/cylc/
where you
will unpack and install release tarballs. Local installation of third party
dependencies like Graphviz is also possible, but that depends on the particular
installation methods used and is outside of the scope of this document.
3.3.2. Create A Site Config File¶
Site and user global config files define some important parameters that affect all suites, some of which may need to be customized for your site. See Global (Site, User) Configuration Files for how to generate an initial site file and where to install it. All legal site and user global config items are defined in Global (Site, User) Config File Reference.
3.3.3. Configure Site Environment on Job Hosts¶
If your users submit task jobs to hosts other than the hosts they use to run
their suites, you should ensure that the job hosts have the correct environment
for running cylc. A cylc suite generates task job scripts that normally invoke
bash -l
, i.e. it will invoke bash as a login shell to run the job
script. Users and sites should ensure that their bash login profiles are able
to set up the correct environment for running cylc and their task jobs.
Your site administrator may customise the environment for all task jobs by
adding a <cylc-dir>/etc/job-init-env.sh
file and populate it with the
appropriate contents. If customisation is still required, you can add your own
${HOME}/.cylc/job-init-env.sh
file and populate it with the
appropriate contents.
${HOME}/.cylc/job-init-env.sh
<cylc-dir>/etc/job-init-env.sh
The job will attempt to source the first of these files it finds to set up its environment.
3.3.4. Configuring Cylc Review Under Apache¶
The Cylc Review web service displays suite job logs and other information in
web pages - see Viewing Suite Logs in a Web Browser: Cylc Review and
Fig. 14. It can run under a WSGI server (e.g.
Apache with mod_wsgi
) as a service for all users, or as an ad hoc
service under your own user account.
To run Cylc Review under Apache, install mod_wsgi
and configure it
as follows, with paths modified appropriately:
# Apache mod_wsgi config file, e.g.:
# Red Hat Linux: /etc/httpd/conf.d/cylc-wsgi.conf
# Ubuntu Linux: /etc/apache2/mods-available/wsgi.conf
# E.g. for /opt/cylc-7.8.1/
WSGIPythonPath /opt/cylc-7.8.1/lib
WSGIScriptAlias /cylc-review /opt/cylc-7.8.1/bin/cylc-review
(Note the WSGIScriptAlias
determines the service URL under the
server root).
And allow Apache access to the Cylc library:
# Directory access, in main Apache config file, e.g.:
# Red Hat Linux: /etc/httpd/conf/httpd.conf
# Ubuntu Linux: /etc/apache2/apache2.conf
# E.g. for /opt/cylc-7.8.1/
<Directory /opt/cylc-7.8.1/>
AllowOverride None
Require all granted
</Directory>
The host running the Cylc Review web service, and the service itself (or the
user that it runs as) must be able to view the ~/cylc-run
directory
of all Cylc users.
Use the web server log, e.g. /var/log/httpd/
or /var/log/apache2/
, to
debug problems.
3.4. Automated Tests¶
The cylc test battery is primarily intended for developers to check that changes to the source code don’t break existing functionality.
Note
Some test failures can be expected to result from suites timing out,
even if nothing is wrong, if you run too many tests in parallel. See
cylc test-battery --help
.