Inter-Workflow Triggering

Get a copy of this example

$ cylc get-resources examples/inter-workflow-triggers

This example shows how one workflow can “trigger off” of tasks in another workflow.

In this example, there are two workflows:

• “upstream” writes a file.

• “downstream” reads this file.

Run both workflows simultaneously to see this in action:

$ cylc vip inter-workflow-triggers/upstream
$ cylc vip inter-workflow-triggers/downstream

Upstream Workflow

[meta]
    title = Upstream Workflow
    description = """
        This is the workflow which is providing the data that the downstream
        workflow wants to use.
    """

[scheduling]
    # start two hours before the current time on the whole hour
    initial cycle point = previous(T-00) - PT2H
    [[graph]]
        PT1H = """
            # wait for the "real world" time before running "a":
            @wall_clock => a

            # then run task "b"
            a => b
        """

[runtime]
    [[a]]
    [[b]]
        # write a random number to ~/cylc-run/<workflow-id>/share/<cycle>
        # for the downstream workflow to use
        script = echo "$RANDOM" > "$file"
        [[[environment]]]
            file = ${CYLC_WORKFLOW_SHARE_DIR}/${CYLC_TASK_CYCLE_POINT}

Downstream Workflow

[meta]
    title = Downstream Workflow
    description = """
        This workflow uses the data provided by the upstream workflow.
    """

[scheduling]
    # start two hours before the current hour
    initial cycle point = previous(T-00) - PT2H
    [[xtriggers]]
        # this is an "xtrigger" - it will wait for the task "b" in the same
        # cycle from the workflow "upstream"
        upstream = workflow_state(workflow_task_id="inter-workflow-triggers/upstream//%(point)s/b")
    [[graph]]
        PT1H = """
            @upstream => process
        """

[runtime]
    [[process]]
        script = echo "The random number is: $(cat "$file")"
        [[[environment]]]
            # this is where the data should be written to in the upstream workflow
            # Note: "runN" will point to the most recent run of a workflow
            file = $HOME/cylc-run/upstream/runN/share/$CYLC_TASK_CYCLE_POINT

Example - Decoupled workflows

This pattern is useful where you have workflows that you want to keep decoupled from one another, but still need to exchange data. E.G. in operational meteorology we might have a global model (covering the whole Earth) and a regional model (just covering a little bit of it) where the regional model obtains its boundary condition from the global model.