cylc-admin

Cylc 8 alpha2 Release Notes

Date: July 2020.

The second “alpha” preview release of Cylc 8 is available for testing by interested users.

NOTE this is not a production-ready release.

Feel free to reach us at Discourse if you have questions, or create issues or pull requests via GitHub, should you find any bugs or would like to suggest new features for Cylc 8.

We encourage users who have successfully tested this release to send a short message on Discourse to let us know how was the installation process, any issues found with this release, or anything that we could use to improve future releases.

Installation instructions

The minimum requirements for Cylc 8 are:

The Cylc 8 complete system includes:

Conda

Use Conda to install the whole Cylc 8 system, including the web UI.

# Make sure conda-forge is in your channels.
conda config --add channels conda-forge

# optional conda environment
conda create -n cylc1
conda activate cylc1

# then install Cylc metapackage
conda install cylc

You can review the Conda cylc metapackage here. It will install the required Conda dependencies for Cylc 8, such as Cylc Flow, Cylc UI Server, and Cylc UI.

pip

You can use pip to install just the Python parts of Cylc 8.

Workflows can be executed using just the scheduler and CLI or terminal UI, but if you want to try to web UI and can’t use conda (above), you can now install the web UI manually:

Testing the installation

Once you have installed Cylc 8, you should be able to run Cylc commands, e.g.:

Usage

Start the Hub (JupyterHub gets installed with the Conda “cylc” package):

mkdir -p "${HOME}/srv/cylc/"  # the hub will store session information here
cd "${HOME}/srv/cylc/"
jupyterhub \
    --JupyterHub.spawner_class="jupyterhub.spawner.LocalProcessSpawner" \
    --JupyterHub.logo_file="${CONDA_PREFIX}/work/cylc-ui/img/logo.svg" \
    --Spawner.args="['-s', '${CONDA_PREFIX}/work/cylc-ui']" \
    --Spawner.cmd="cylc-uiserver"

Voilà 🎉. Go to hhttp://localhost:8000, log in to the Hub with your local user credentials, and enjoy Cylc 8 alpha2!

To deactivate and/or remove the conda environment:

(*) The following workflow generates a series of tasks that initially fail before succeeding after a random number of retries (this shows the new “Cylc 8 task/job separation” nicely):

[cylc]
   cycle point format = %Y
   [[parameters]]
      m = 0..5
      n = 0..2
[scheduling]
   initial cycle point = 3000
   [[graph]]
      P1Y = "foo[-P1Y] => foo => bar<m> => qux<m,n> => waz"
[runtime]
   [[root]]
      script = """
         sleep 20
         # fail 50% of the time if try number is less than 5
         if (( CYLC_TASK_TRY_NUMBER < 5 )); then
           if (( RANDOM % 2 < 1 )); then
              exit 1
           fi
         fi"""
      [[[job]]]
         execution retry delays = 6*PT2S

Current Limitations

Cylc-8.0a2 is a preview release with:

It can run existing Cylc-7 workflows. However:

What’s new since Cylc-8.0a1?

A lot has changed since the previous Cylc 8 alpha release.

The following list highlights some of the major changes:

We have also implemented several fixes related to the new architecture of Cylc 8, as well as other changes related to porting old GUI features to the new Web UI, and many software dependency updates (for latest features but also for security).

Task Job Access to Cylc 8

Cylc task jobs need access to the same Cylc version that their parent scheduler is running, e.g. for job status messaging. Users may even need to run multiple workflows at once on a system with multiple Cylc releases installed, and the task jobs of each need to call the correct Cylc.

Cylc 7 handles this via a wrapper script installed as cylc in the user’s executable search path ($PATH). The wrapper intercepts Cylc commands, including cylc message from task jobs, and uses the environment to determine the right Cylc in each case (the scheduler writes $CYLC_VERSION, for example, to task job scripts).

Cylc 8 will use a similar wrapper (which has to deal with the additional complication of Python and conda virtual environments) but we have not yet documented this for users.

For the moment we suggest you do not use a cylc wrapper for Cylc 8 testing, and run only local background jobs because they inherit the scheduler’s virtual environment and will therefore find the right Cylc directly without the help of a wrapper. However, note the following warning.

WARNING: if your .bashrc (or similar) provides access to a Cylc 7 cylc wrapper by prepending its location to $PATH, that wrapper will override Cylc 8 virtual environments and your Cylc 8 jobs will therefore be unable to communicate status back to the scheduler. If so, disable the Cylc 7 wrapper before testing Cylc 8.

The next Cylc 8 release (alpha-3) will provide a backward-compatible wrapper to handle Cylc 7 and 8 together on the same system.

This site is open source. Improve this page.