.. _tutorial-cylc-message-triggers:
Message Triggers
================
:term:`Message triggers <message trigger>` allow us to trigger dependent tasks
before the upstream task has completed.
Explanation
-----------
We have seen :ref:`before <tutorial-qualifiers>` that tasks can have
:term:`qualifiers <qualifier>` for different
:term:`task states <task state>`.
:term:`Message triggers <message trigger>` are essentially custom qualifiers.
We can produce a bespoke output while our task is still running.
This output could be, for example, a report or perhaps another task.
Usage
-----
:term:`Message triggers <message trigger>` are particularly useful if we have
a long running task and we want to produce multiple tailored outputs whilst
this task is running, rather than having to wait for the task to
complete.
We could also set up :term:`message triggers <message trigger>` to, for example,
send an email to inform us that a submission has failed, making use of Cylc's
task event handling system. More information is available on these in the
`Cylc User Guide`_.
:term:`Message triggers <message trigger>` provide a superior solution to
the problem of file system polling. We could, for example, design our workflow
such that we check if our task is finished by polling at intervals.
It is inefficient to 'spam' task hosts with polling commands, it is preferable
to set up a message trigger.
How to create a message trigger
-------------------------------
In order to get our workflow to trigger messages, we need to:
* specify our custom message in a section called ``[[outputs]]`` within the
``[runtime]`` section of our workflow,
* add ``cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" "YOUR CHOSEN TRIGGER MESSAGE"``
to the ``script`` section of ``[runtime]``, your chosen trigger message
should be unique and should exactly match the message defined in
``[[outputs]]``.
* Refer to these messages in the ``[dependencies]`` section of our workflow.
.. note::
The message will be recorded in the workflow's scheduler log.
See :ref:`scheduler logs.cylc message` for details of how messages appear.
These outputs are then triggered during the running of the task.
We can use these to manage tasks dependent on partially completed tasks.
So, a basic example, where we have a task foo, that when partially completed
triggers another task bar and when fully completed triggers another task, baz.
.. code-block:: cylc
[scheduling]
[[graph]]
R1 = """
foo:out1 => bar
foo => baz
"""
[runtime]
[[foo]]
script = """
sleep 5
cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" "file 1 done"
sleep 10
"""
[[[outputs]]]
out1 = "file 1 done"
[[bar, baz]]
script = sleep 10
.. _message triggers practical:
.. practical::
.. rubric:: In this practical example, we will create a workflow to demonstrate
:term:`message triggers <message trigger>`. We will use message triggers
to both produce a report and trigger a new task from a partially completed
task.
#. **Create a new directory.**
Within your ``~/cylc-src`` directory create a new directory called
``message-triggers`` and move into it:
.. code-block:: bash
mkdir ~/cylc-src/message-triggers
cd ~/cylc-src/message-triggers
#. **Install the script needed for our workflow**
The workflow we will be designing requires a bash script, ``random.sh``,
to produce our report. It will simply create a text file ``report.txt``
with some random numbers in it. This will be executed when the associated
task is run.
Scripts should be kept in the ``bin`` sub-directory within the
:term:`run directory <run directory>`. If a ``/bin``
exists in the run directory, it will be prepended $PATH at run
time.
Create a ``/bin`` directory.
.. code-block:: bash
mkdir ~/cylc-src/message-triggers/bin
Create a bash script in the bin directory:
.. code-block:: bash
touch bin/random.sh
We will need to make this script executable.
.. code-block:: bash
chmod +x bin/random.sh
Open the file and paste the following basic bash script into it:
.. code-block:: bash
#!/usr/bin/env bash
set -eu # Prevent bash script failing quietly.
counter=1
while [ $counter -le 10 ]; do
newrand=$(( (( RANDOM % 40) + 1 ) ));
echo $newrand >> report.txt;
counter=$((counter + 1));
done
#. **Create a new workflow.**
Create a :cylc:conf:`flow.cylc` file and paste the following basic workflow into it:
.. code-block:: cylc
[meta]
title = "test workflow to demo message triggers"
[scheduler]
UTC mode = True
[scheduling]
initial cycle point = 2019-06-27T00Z
final cycle point = 2019-10-27T00Z
[[graph]]
P2M = """
long_forecasting_task => another_weather_task
long_forecasting_task => different_weather_task
long_forecasting_task[-P2M] => long_forecasting_task
"""
This is a basic workflow, currently it does not have any message triggers
attached to any task.
#. **Define our tasks in the runtime section.**
Next we want to create our ``runtime`` section of our workflow.
First we define what the tasks do. In this example
``long_forecasting_task`` will sleep, create a file containing some
random numbers and produce a message.
(Note that the random number generator bash script has already been
preloaded into your ``bin`` directory.)
``another_weather_task`` and ``different_weather_task`` simply sleep.
Add the following code to the :cylc:conf:`flow.cylc` file.
.. code-block:: cylc
[runtime]
[[long_forecasting_task]]
script = """
sleep 2
random.sh
sleep 2
random.sh
sleep 2
random.sh
"""
[[another_weather_task, different_weather_task]]
script = sleep 1
#. **Create message triggers.**
We now have a workflow with a task, ``long_forecasting_task`` which, after
it has fully completed, triggers two more tasks, ``another_weather_task``
and ``different_weather_task``.
Suppose we want ``another_weather_task`` and ``different_weather_task``
to start before ``long_forecasting_task`` has fully completed, perhaps
after some data has become available.
In this case, we shall trigger ``another_weather_task`` after one set of
random numbers has been created
and ``different_weather_task`` after a second set of random numbers has
been created.
There are three aspects of creating message triggers.
The first is to create the messages. Within ``runtime``, ``TASK`` in our
workflow, we need to create a sub-section called ``outputs``. Here we create
our custom outputs.
.. code-block:: diff
+ [[[outputs]]]
+ update1 = "Task partially complete, report ready to view"
+ update2 = "Task partially complete, report updated"
The second thing we need to do is to create a cylc message in our script.
This should be placed where you want the message to be called. In our
case, this is after each of the first two set of random numbers are
generated.
.. tip::
Remember that the ``cylc message`` should exactly match the outputs
stated in our ``[[[outputs]]]`` section.
Modify the ``[[long_forecasting_task]]`` script in the :cylc:conf:`flow.cylc` file
as follows:
.. code-block:: diff
[runtime]
[[long_forecasting_task]]
script = """
sleep 2
random.sh
+ cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \
+ "Task partially complete, report ready to view"
sleep 2
random.sh
+ cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \
+ "Task partially complete, report updated"
sleep 2
random.sh
"""
Lastly, we need to make reference to the messages in the
graph section.
This will ensure your tasks trigger off of the messages correctly.
Adapt the ``[[dependencies]]`` section in the :cylc:conf:`flow.cylc` file to read as
follows:
.. code-block:: diff
[[[P2M]]]
graph = """
- long_forecasting_task => another_weather_task
- long_forecasting_task => different_weather_task
+ long_forecasting_task:update1 => another_weather_task
+ long_forecasting_task:update2 => different_weather_task
long_forecasting_task[-P2M] => long_forecasting_task
"""
This completes our :cylc:conf:`flow.cylc` file.
Our final workflow should look like this:
.. spoiler:: Solution warning
.. code-block:: cylc
[meta]
title = "test workflow to demo message triggers"
[scheduler]
UTC mode = True
[scheduling]
initial cycle point = 2019-06-27T00Z
final cycle point = 2019-10-27T00Z
[[graph]]
P2M = """
long_forecasting_task:update1 => another_weather_task
long_forecasting_task:update2 => different_weather_task
long_forecasting_task[-P2M] => long_forecasting_task
"""
[runtime]
[[long_forecasting_task]]
script = """
sleep 2
random.sh
cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \
"Task partially complete, report ready to view"
sleep 2
random.sh
cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \
"Task partially complete, report updated"
sleep 2
random.sh
"""
[[[outputs]]]
update1 = "Task partially complete, report ready to view"
update2 = "Task partially complete, report updated"
[[another_weather_task, different_weather_task]]
script = sleep 1
#. **Validate the workflow.**
It is a good idea to check that our :cylc:conf:`flow.cylc` file does not have any
configuration issues.
Run ``cylc validate`` to check for any errors:
.. code-block:: bash
cylc validate .
#. **Install and Play the workflow.**
Now we are ready to run our workflow. Validate, install, then open
the :ref:`GUI <tutorial.gui>` or :ref:`TUI <tutorial.tui>` and play
the workflow.
.. code-block:: bash
cylc validate .
cylc install
cylc play message-triggers
Your workflow should now run, the tasks should succeed.
#. **Inspect the work directory.**
You can now check for your report outputs. These should appear in the
:term:`work directory` of the workflow. All being well, our first cycle
point should produce a test file with some random numbers, and each
subsequent cycle point file should have more random numbers added.
#. **Extension.**
Suppose now we would like to send an email alerting us to the reports
being ready to view.
We will need to add to our :cylc:conf:`flow.cylc` file.
In the ``runtime`` section, add a sub-section called ``[[[events]]]``.
Within this section we will make use of the built-in setting
``mail events``.
Here, we specify a list of events for which notifications should be sent.
The events we are interested in are, in this case, our outputs.
Add the following code to your ``[[[events]]]`` section.
.. code-block:: cylc
[[[events]]]
mail events = update1, update2
Our updated workflow should look like this:
.. spoiler:: Solution warning
.. code-block:: cylc
[scheduler]
UTC mode = True
[meta]
title = "test workflow to demo message triggers"
[scheduling]
initial cycle point = 2019-06-27T00Z
final cycle point = 2019-10-27T00Z
[[graph]]
P2M = """
long_forecasting_task:update1 => another_weather_task
long_forecasting_task:update2 => different_weather_task
long_forecasting_task[-P2M] => long_forecasting_task
"""
[runtime]
[[long_forecasting_task]]
script = """
sleep 2
random.sh
cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \
"Task partially complete, report ready to view"
sleep 2
random.sh
cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \
"Task partially complete, report updated"
sleep 2
random.sh
"""
[[[outputs]]]
update1 = "Task partially complete, report ready to view"
update2 = "Task partially complete, report updated"
[[[events]]]
mail events = update1, update2
[[another_weather_task, different_weather_task]]
script = sleep 1
Save your changes and run your workflow.
Check your emails and you should have, one email for the first update and,
a second email alerting you to the subsequent updated reports being ready.
Note that the second email automatically bundles the messages to prevent
your inbox from being flooded.