UMC Core Partner Logos ESiWACE - Cylc - Altair Logos UMC Assoc Partner Logos

3-7 December 2018 Cylc Development Workshop Report

Hilary Oliver, January 2019

Table of Contents

Executive Summary


In December 2018 the Australian Bureau of Meteorology (BOM) hosted the first-ever Cylc development workshop, with participants from NIWA (3), Met Office (3), ESiWACE (1), BOM (1), and Altair Engineering (1). Travel funding was generously provided by the Unified Model Partnership, ESiWACE, and Altair.

The primary goal of the workshop was to plan how to port Cylc from Python 2 to Python 3, with the complication that the current Cylc (and Rose) GUIs need to be completely replaced because they are based on a now-obsolete framework (PyGTK) that will never be ported to Python 3. This has to be done with some urgency due to the impending demise of Python 2, which will officially be retired at the end of 2019. The week in Melbourne also provided an ideal opportunity to introduce new team members, and to meet the BOM and Altair staff building NWP production infrastructure around PBS Pro, Cylc, and Kafka, and to learn about that project.

The workshop was a great success. We settled (with caveats) on communication and working practices for the widely distributed team, and worked through a porting plan and a radically new Cylc architecture to support a modern web UI, with site identity management integration and a single point of access for users.

See below for more information on background, the workshop agenda, and outcomes including a link to Cylc-8 architecture documentation.

Workshop Goals and Agenda



The official Python 2 end-of-life date is 31 December 2019. After that the language will no longer be developed or maintained, not even for security patches. Even worse, the current Cylc GUIs, which embody several developer-years of effort, are built on the obsolete PyGTK framework which will never be ported to Python 3. Major Linux distributions have already announced they will soon be shipping without Python 2 and PyGTK. Cylc is still Python 2-based as we have not previously had the resources to address these major issues.

So there is an urgent need to migrate Cylc to Python 3 and entirely replace the current GUIs with something more modern, which by general agreement should be a web UI. This is a challenging task that requires new system components and a complete rework of Cylc’s local client-server architecture, which is not web-compatible (current clients make use of the local system in ways that browsers do not allow, for instance). And the first major release of the new system, Cylc-8, must be performant “out of the box” for existing critical production suites.



The primary workshop goal was (therefore) to:

Toward this end, sub-goals were to:

The WORKSHOP AGENDA shows the topics covered each day during the week, to support these goals.

Workshop Outcomes

Communications and Working Practices


Technical working practices centered on git and GitHub were discussed, but already well-established in the project. With new team members and people located all over the world, however, we agreed on the need for regular video conferences (monthly); instant communication via a chat platform of some kind; possibly a replacement for our old Google Groups email forum (mainly used for release announcements and infrequent questions from remote users); and something like Trello boards to help with project management as well. Unfortunately it is not so easy to solve these problems because our respective institutions all use mutually incompatible video conferencing technologies and have wildly different restrictions on what can be used on site. Popular platforms such as Slack also place significant restrictions on free accounts (such as loss of history beyond some point) or charge a lot for full functionality.

We decided to:

(Since the workshop one of our major sites has decided to block for security reasons, forcing some of us to use personal devices to stay in touch … so effective communication remains challenging).

Cylc-8 Architecture


We succeeded in working through the detailed Cylc-8 architecture, including all components, technologies, and communications protocols needed to implement the new system. The most visible change will be a major new system component: a “hub” that acts as a single point of access for users, handles authentication via site identity management plugins, spawns suite UI Servers (which in turn spawn Workflow Services - i.e. suites) into user accounts, and then proxies requests to them. Beyond this, a variety of new technologies and communications protocols will be used to build and connect the various system components. These include WebSocket (instead of HTTPS) between UIs and UI Servers; the Vue.js Javascript framework, a GraphQL (instead of REST) API to serve the suite status data model, and the ZeroMQ network library to connect UI Servers with Workflow Services at the back end. Naturally, many details - such as secure backend communications - remain to be worked out during implementation.

The new architecture is written up in some detail, and illustrated, in a separate report: CYLC-8 ARCHITECTURE.

Cylc-8 Implementation


The implementation plan can be summarized as follows.


  1. finish off critical Cylc-7 developments and release cylc-7.8.0, then make a 7.8.x branch for ongoing Cylc-7 maintenance

Then begin Cylc-8 development:

  1. implement formal test coverage reporting, and improve coverage where possible, to support the Python 3 porting effort
  2. port our ISO 8601 date-time library to Python 3
  3. delete the old GUIs and port the rest of Cylc to Python 3…
  4. …replacing the Cherrypy network library in the suite server program with ZeroMQ
    • result: a working Python 3 Cylc with no GUI and local access only; this will provide a platform to build the rest of the system on.
  5. Python package management (pip install cylc etc.)

Concurrent with Python 3 porting:

  1. determine whether or not JupyterHub be used “out of the box” as the new Cylc Hub, and if not, can it be modified to our purposes, including:
    • user authentication and site plugins
    • spawning user processes
    • access to other user’s suites
  2. implement Cylc Hub, leveraging JupyterHub as far as possible
  3. begin work on the UI Server, with Vue.js and other technologies
    • server-facing comms layer: ZeroMQ
    • UI-facing comms layer: Tornado (asynchronous Python web framework), WebSocket, GraphQL
    • sub-services for suite start-up and discovery etc.
  4. understand secure communication and session management
  5. (and more - see the aforementioned architecture document for details)
  6. (and if time allows, continue the mostly-orthogonal “rose suite-run” migration and cluster-awareness work)



Thanks to BOM for hosting the workshop; and the UM Partnership, ESiWACE, and Altair for providing travel funding.

UMC Assoc Partner Logos ESiWACE - Cylc - Altair Logos UMC Logos