cylc-admin

Getting Started with Cylc Development

Development Setup

#. Say Hi On The Cylc Developers Chat!

#. Install System Dependencies

[!NOTE] We recommend developing in a Conda environment, although it is possible, though less flexible, to install the required system dependencies and work in Python virtual environments.

Use whatever Conda client you prefer, micromamba is the most performant.

This command will give you a Conda environment containing all the things you might need for Cylc/Rose development:

   conda create -y -n cylc-dev \
      python \
      pip \
      configurable-http-proxy \
      graphviz \
      nodejs \
      yarn \
      shellcheck \
      pygraphviz \
      pygobject \
      gtk3

#. Configure For Distributed Development

Follow these steps if you’re running on a network with a shared filesystem.

Install the Cylc wrapper script.

[!NOTE] Your site may have already done this for you.

Then configure the path to your Conda environment directory in your shell profile file (e.g, .bash_profile), e.g:

   # ~/.bash_profile
   export CYLC_HOME_ROOT_ALT="/path/to/conda/environment/directory"

To locate your environment directory, run:

   conda activate cylc-dev
   dirname $CONDA_PREFIX

Then get Conda to set the CYLC_ENV_NAME environment variable automatically when you active the environment (this tells the wrapper script which Conda environment to use for remote command invocations):

   $ conda activate cylc-dev
   $ echo "export CYLC_ENV_NAME=$CONDA_DEFAULT_ENV" > $CONDA_PREFIX/etc/conda/activate.d/cylc.sh
   $ echo 'unset CYLC_ENV_NAME' > $CONDA_PREFIX/etc/conda/deactivate.d/cylc.sh

#. Fork & Clone

On GitHub, create a fork of any of the Cylc repositories you want to work on and create a local clone of them (i.e, git clone git@github.com/<yourname>/<repo>.git).

Please star cylc-flow and metomi-rose :)

#. Install Python Projects

Install any Python projects you have just cloned into your Conda environment.

Install cylc/cylc-flow and optionally:

• cylc/cylc-uiserver (for web server work)
• metomi/rose (for Rose work)
• cylc/cylc-rose (for Cylc / Rose integration)
• cylc/cylc-doc (for Cylc docs changes)

Install the [all] optional dependency to pick up test, developer and documentation extras, i.e:

   pip install -e "path/to/repo[all]"

When doing this for the first time it’s better to install them all in one go, e.g:

   pip install -e ./cylc-flow[all] \
               -e ./cylc-uiserver[all] \
               -e ./rose[all] \
               -e ./cylc-rose[all] \
               -e ./cylc-doc[all]

[!NOTE] There are dependencies between projects, so they may need to be installed in order, see project dependencies

[!NOTE] Installing in “editable” mode (the -e above) means you only need to repeat this pip install command when certain project files are modified:

• setup.py
• setup.cfg
• pyproject.toml
• MANIFEST.in

[!NOTE] You can use uv pip as a stand-in for pip.

#. Install and configure cylc/cylc-ui (optional)

   cd path/to/cylc-ui
   conda activate cylc-dev
   yarn install
   yarn run build  # create your first production build

The Cylc UI Server comes with a bundled version of Cylc UI. To use your personal build add the following to your Jupyter configuration:

   # ~/.cylc/hub/jupyter_config.py
   # NOTE: some of us use an environment variable to toggle this on/off
   c.CylcUIServer.ui_build_dir = '~/cylc-ui/dist'  # path to build

• See the cylc-uiserver README for more information on ui_build_dir.
• See the cylc-ui README for more information on building/developing the UI.

Then launch the UI:

   # standalone (single-user mode, good for development)
   cylc gui

   # via Jupyter Hub (multi-user mode)
   cylc hub
   # You will be asked to log in with your desktop credentials if you have not
   # done so before.

#. Contributor License Agreement

Read the CONTRIBUTING.md file and add your name to it with your first commit.

You will have to do this for each Cylc / Rose repository. Note repositories may use different licenses.

Running Your First Workflow

conda acitvate cylc-dev

# clone the workflow into ~/cylc-src
cylc get-resources tutorial/cylc-forecasting-workflow

# install the workflow into ~/cylc-run and start it
cylc vip cylc-forecasting-workflow

# watch it run
cylc tui  # in-terminal
# OR
cylc gui  # in-browser

# stop it
cylc stop cylc-forecasting-workflow

# uninstall it from ~/cylc-run
cylc clean cylc-forecasting-workflow

Versions & Branches

Cylc projects use semantic versioning, e.g: for the version 8.1.2:

• 8 = major release (implies breaking changes)
• 1 = minor release (no breaking changes permitted unless forewarning provided)
• 2 = bugfix release (bugfixes and UI/UX issues only, no features or refactors)

Cylc projects are all pinned to the minor version of cylc-flow, e.g:

• cylc-uiserver 1.6.x goes with cylc-flow 8.4.x
• cylc-uiserver 1.7.x goes with cylc-flow 8.5.x
• cylc-rose 1.5.x goes with cylc-flow 8.4.x
• etc

This tight coupling prevents unintended combinations of Cylc components being installed into the same environment whilst allowing them to be installed in a modular fashion.

Bugfixes

When we make a minor release, we create a branch with a .x suffix for bugfixes.

E.g. 8.1.x was for bugfixes to 8.1.0. We merge these .x branches back into master.

Raise any bugfixes against the .x branch, the “sync” PR will be automatically created after it is merged.

Supported Versions

The currently supported (i.e. actively developed) versions are documented here.

Project Dependencies

There are dependencies between our projects, here’s a summary:

---
config:
    look: handDrawn
---
flowchart LR
    isodatetime --> rose
    isodatetime --> cylc-flow
    rose --> cylc-rose
    cylc-flow --> cylc-rose
    cylc-flow --> cylc-uiserver
    cylc-ui -->|bundled| cylc-uiserver

This site is open source. Improve this page.