Python 2 => 3

Does This Change Affect Me?

This change will affect you if your workflows extend Cylc or Jinja2 with custom Python scripts.

This does not impact task scripts; Cylc can still run Python 2 tasks if desired.

Overview

Cylc 7 ran under Python 2; Cylc 8 runs under Python 3.

Cylc can be extended with custom Python scripts. These scripts are run under the same version of Python used by Cylc.

As a result, if you are moving from Cylc 7 to Cylc 8, you must upgrade any scripts from Python 2 to Python 3 in the process.

If you want to support both Cylc 7 and 8, you must support both Python 2 and 3. There are tools to help you do this, e.g. six.

Impacted Scripts

The following scripts must be upgraded if used:

Custom Jinja2 Filters, Tests and Globals

These allow you to extend Jinja2 with Python code.

These scripts are located in the following directories within a workflow:

Jinja2Filters
Jinja2Tests
Jinja2Globals

Modules imported by Jinja2

Python modules can be imported from Jinja2 e.g:

{% from "os" import path %}

Custom Trigger Functions

Any custom xtrigger functions.

Package Name Changes

Three Python packages have been renamed between Cylc 7 and Cylc 8:

cylc => cylc.flow
isodatetime => metomi.isodatetime
rose => metomi.rose

If you are importing from these packages you will need to update the package names.

Here are some examples:

Convert Python code from Cylc 7 to Cylc 8:

- from cylc import LOG
+ from cylc.flow import LOG
- from isodatetime.data import Duration
+ from metomi.isodatetime.data import Duration

Python code which supports both Cylc 7 & Cylc 8:

import sys
if sys.version[0] == '3':
    from cylc.flow import LOG
    from metomi.isodatetime.data import Duration
else:
    from cylc import LOG
    from isodatetime.data import Duration

Convert Jinja2 code from Cylc 7 to Cylc 8:

#!Jinja2
- {% from "cylc" import LOG %}
+ {% from "cylc.flow" import LOG %}
  {% do LOG.debug("Hello World!") %}

Jinja2 code which supports both Cylc 7 & Cylc 8:

#!Jinja2
{% from "sys" import version -%}
{% if version[0] == '3' -%}
    {% from "cylc.flow" import LOG -%}
{% else -%}
    {% from "cylc" import LOG -%}
{% endif -%}

{% do LOG.debug("Hello World!") %}

Jinja2 - integers with leading zeros

Integers with leading zeros in Jinja2 expressions are now illegal and will cause an error like Jinja2Error: expected token 'x', got 'integer'. For example:

$ cylc validate my-workflow
Jinja2Error: expected token 'end of statement block', got 'integer'
File ~/cylc-run/my-workflow/flow.cylc
  {% if START_HOUR == 06 or START_HOUR == 12 %}       <-- TemplateSyntaxError

The solution in this case is:

-{% if START_HOUR == 06 or START_HOUR == 12 %}
+{% if START_HOUR == 6 or START_HOUR == 12 %}

Rose

The same changes also impact Rose extensions: