Cylc 7 Compatibility Mode
Does This Change Affect Me?
This will affect you if you want to run Cylc 7 (suite.rc
) workflows
using Cylc 8.
Overview
Cylc 8 can run most Cylc 7 workflows “as is”.
The suite.rc
filename triggers a backward compatibility mode in which:
implicit tasks are allowed by default
(unless a
rose-suite.conf
file is found in the run directory for consistency withrose suite-run
behaviour)(Cylc 8 does not allow implicit tasks by default)
cycle point time zone defaults to the local time zone
(Cylc 8 defaults to UTC)
waiting tasks are pre-spawned to mimic the Cylc 7 scheduling algorithm and stall behaviour, and these require suicide triggers for alternate graph branching
(Cylc 8 spawns tasks on demand, and suicide triggers are not needed for branching)
succeeded
task outputs are required, so in the absence of suicide triggers the scheduler will retain other final status tasks in the n=0 window to stall the workflow.(in Cylc 8, all outputs are required unless marked as *optional* by the new
?
syntax)
Required Changes
Providing your Cylc 7 workflow does not use syntax that was deprecated at Cylc 7, you may be able to run it using Cylc 8 without any modifications while in compatibility mode.
First, run cylc validate
with Cylc 7 on your suite.rc
workflow
to check for deprecation warnings and fix those before validating with Cylc 8.
See below for an example.
Warning
cylc validate
operates on the processed suite.rc
, which
means it will not detect any deprecated syntax that is inside a
currently-unused Jinja2/EmPy if...else
branch.
Some workflows may require modifications to either upgrade to Cylc 8 or make interoperable with Cylc 8 backward compatibility mode. Read on for more details.
Cylc commands in task scripts
Check for any use of Cylc commands in task scripting. Some Cylc 7 commands
have been removed and some others now behave differently.
However, cylc message
and cylc broadcast
have not changed.
See the full list of command line interface changes
and see below for an example.
Python 2 to 3
Whereas Cylc 7 runs using Python 2, Cylc 8 runs using Python 3. This affects: - modules imported in Jinja2 - Jinja2 filters, tests and globals - custom xtrigger functions
Note that task scripts are not affected - they run in an independent environment.
See Python 2 => 3 for more information and examples of how to implement interoperability if your workflows extend Cylc or Jinja2 with custom Python scripts.
Other caveats
Cylc 8 cannot restart a partially completed Cylc 7 workflow in-place. If possible, complete the run with Cylc 7. Otherwise, see Continuing a Cylc 7 Workflow with Cylc 8.
Cylc 8 only transfers certain files and directories by default during remote installation. See Remote Installation for more information.
Cylc 8 does not support excluding/including tasks at start-up. If your workflow used this old functionality, it may have been used in combination with the
cylc insert
command (which has been removed from Cylc 8) andcylc remove
(which still exists but is much less needed).Cylc 8 does not support specifying remote usernames using
flow.cylc[runtime][<namespace>][remote]owner
.
Examples
Validating with Cylc 7
Consider this configuration:
[scheduling]
initial cycle point = 11000101T00
[[dependencies]]
[[[R1]]]
graph = task
[runtime]
[[task]]
pre-command scripting = echo "Hello World"
Running cylc validate
at Cylc 7 we see that the
workflow is valid, but we are warned that pre-command scripting
was replaced by pre-script
at 6.4.0:
$ cylc validate .
WARNING - deprecated items were automatically upgraded in 'suite definition':
WARNING - * (6.4.0) [runtime][task][pre-command scripting] -> [runtime][task][pre-script] - value unchanged
Valid for cylc-7.8.7
Note
Cylc 7 has handled this deprecation for us, but at Cylc 8 this workflow will fail validation.
$ cylc validate .
IllegalItemError: [runtime][task]pre-command scripting
You must change the configuration yourself. In this case:
- pre-command scripting = echo "Hello World"
+ pre-script = echo "Hello World"
Validation will now succeed.
Cylc commands in task scripts
You might have a task script that calls a Cylc command like so:
[runtime]
[[foo]]
script = cylc hold "$CYLC_SUITE_NAME"
The cylc hold
command has changed in Cylc 8. It is now used for holding
tasks only; use cylc pause
for entire workflows.
(Additionally, $CYLC_SUITE_NAME
is deprecated in favour of
$CYLC_WORKFLOW_ID
, though still supported.)
In order to make this interoperable, so that you can run it with both Cylc 7 and Cylc 8 backward compatibility mode, you could do something like this in the bash script:
[runtime]
[[foo]]
script = """
if [[ "${CYLC_VERSION:0:1}" == 7 ]]; then
cylc hold "$CYLC_SUITE_NAME"
else
cylc pause "$CYLC_WORKFLOW_ID"
fi
"""
Note this logic (and the $CYLC_VERSION
environment variable) is executed
at runtime on the job host.
Alternatively, you could use Jinja2 like so:
[runtime]
[[foo]]
{% if CYLC_VERSION is defined and CYLC_VERSION[0] == '8' %}
script = cylc pause "$CYLC_WORKFLOW_ID"
{% else %}
script = cylc hold "$CYLC_SUITE_NAME"
{% endif %}
Note this logic (and the CYLC_VERSION
Jinja2 variable) is executed locally
prior to Cylc parsing the workflow configuration.
Renaming to flow.cylc
When your workflow runs successfully in backward compatibility mode, it is
ready for renaming suite.rc
to flow.cylc
. Doing this will turn off
backward compatibility mode, and validation in Cylc 8 will show
deprecation warnings.
See also
Important
More complex workflows (e.g. those with suicide triggers) may fail validation once backward compatibility is off - see Graph branching, optional outputs and suicide triggers