.. _tutorial-cylc-message-triggers: Message Triggers ================ :term:`Message triggers ` allow us to trigger dependent tasks before the upstream task has completed. Explanation ----------- We have seen :ref:`before ` that tasks can have :term:`qualifiers ` for different :term:`task states `. :term:`Message triggers ` are essentially custom qualifiers. We can produce a bespoke output while our task is still running. This output could be, for example, a report or perhaps another task. Usage ----- :term:`Message triggers ` are particularly useful if we have a long running task and we want to produce multiple tailored outputs whilst this task is running, rather than having to wait for the task to complete. We could also set up :term:`message triggers ` to, for example, send an email to inform us that a submission has failed, making use of Cylc's task event handling system. More information is available on these in the `Cylc User Guide`_. :term:`Message triggers ` provide a superior solution to the problem of file system polling. We could, for example, design our workflow such that we check if our task is finished by polling at intervals. It is inefficient to 'spam' task hosts with polling commands, it is preferable to set up a message trigger. How to create a message trigger ------------------------------- In order to get our workflow to trigger messages, we need to: * specify our custom message in a section called ``[[outputs]]`` within the ``[runtime]`` section of our workflow, * add ``cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" "YOUR CHOSEN TRIGGER MESSAGE"`` to the ``script`` section of ``[runtime]``, your chosen trigger message should be unique and should exactly match the message defined in ``[[outputs]]``. * Refer to these messages in the ``[dependencies]`` section of our workflow. .. note:: The message will be recorded in the workflow's scheduler log. See :ref:`scheduler logs.cylc message` for details of how messages appear. These outputs are then triggered during the running of the task. We can use these to manage tasks dependent on partially completed tasks. So, a basic example, where we have a task foo, that when partially completed triggers another task bar and when fully completed triggers another task, baz. .. code-block:: cylc [scheduling] [[graph]] R1 = """ foo:out1 => bar foo => baz """ [runtime] [[foo]] script = """ sleep 5 cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" "file 1 done" sleep 10 """ [[[outputs]]] out1 = "file 1 done" [[bar, baz]] script = sleep 10 .. _message triggers practical: .. practical:: .. rubric:: In this practical example, we will create a workflow to demonstrate :term:`message triggers `. We will use message triggers to both produce a report and trigger a new task from a partially completed task. #. **Create a new directory.** Within your ``~/cylc-src`` directory create a new directory called ``message-triggers`` and move into it: .. code-block:: bash mkdir ~/cylc-src/message-triggers cd ~/cylc-src/message-triggers #. **Install the script needed for our workflow** The workflow we will be designing requires a bash script, ``random.sh``, to produce our report. It will simply create a text file ``report.txt`` with some random numbers in it. This will be executed when the associated task is run. Scripts should be kept in the ``bin`` sub-directory within the :term:`run directory `. If a ``/bin`` exists in the run directory, it will be prepended $PATH at run time. Create a ``/bin`` directory. .. code-block:: bash mkdir ~/cylc-src/message-triggers/bin Create a bash script in the bin directory: .. code-block:: bash touch bin/random.sh We will need to make this script executable. .. code-block:: bash chmod +x bin/random.sh Open the file and paste the following basic bash script into it: .. code-block:: bash #!/usr/bin/env bash set -eu # Prevent bash script failing quietly. counter=1 while [ $counter -le 10 ]; do newrand=$(( (( RANDOM % 40) + 1 ) )); echo $newrand >> report.txt; counter=$((counter + 1)); done #. **Create a new workflow.** Create a :cylc:conf:`flow.cylc` file and paste the following basic workflow into it: .. code-block:: cylc [meta] title = "test workflow to demo message triggers" [scheduler] UTC mode = True [scheduling] initial cycle point = 2019-06-27T00Z final cycle point = 2019-10-27T00Z [[graph]] P2M = """ long_forecasting_task => another_weather_task long_forecasting_task => different_weather_task long_forecasting_task[-P2M] => long_forecasting_task """ This is a basic workflow, currently it does not have any message triggers attached to any task. #. **Define our tasks in the runtime section.** Next we want to create our ``runtime`` section of our workflow. First we define what the tasks do. In this example ``long_forecasting_task`` will sleep, create a file containing some random numbers and produce a message. (Note that the random number generator bash script has already been preloaded into your ``bin`` directory.) ``another_weather_task`` and ``different_weather_task`` simply sleep. Add the following code to the :cylc:conf:`flow.cylc` file. .. code-block:: cylc [runtime] [[long_forecasting_task]] script = """ sleep 2 random.sh sleep 2 random.sh sleep 2 random.sh """ [[another_weather_task, different_weather_task]] script = sleep 1 #. **Create message triggers.** We now have a workflow with a task, ``long_forecasting_task`` which, after it has fully completed, triggers two more tasks, ``another_weather_task`` and ``different_weather_task``. Suppose we want ``another_weather_task`` and ``different_weather_task`` to start before ``long_forecasting_task`` has fully completed, perhaps after some data has become available. In this case, we shall trigger ``another_weather_task`` after one set of random numbers has been created and ``different_weather_task`` after a second set of random numbers has been created. There are three aspects of creating message triggers. The first is to create the messages. Within ``runtime``, ``TASK`` in our workflow, we need to create a sub-section called ``outputs``. Here we create our custom outputs. .. code-block:: diff + [[[outputs]]] + update1 = "Task partially complete, report ready to view" + update2 = "Task partially complete, report updated" The second thing we need to do is to create a cylc message in our script. This should be placed where you want the message to be called. In our case, this is after each of the first two set of random numbers are generated. .. tip:: Remember that the ``cylc message`` should exactly match the outputs stated in our ``[[[outputs]]]`` section. Modify the ``[[long_forecasting_task]]`` script in the :cylc:conf:`flow.cylc` file as follows: .. code-block:: diff [runtime] [[long_forecasting_task]] script = """ sleep 2 random.sh + cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \ + "Task partially complete, report ready to view" sleep 2 random.sh + cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \ + "Task partially complete, report updated" sleep 2 random.sh """ Lastly, we need to make reference to the messages in the graph section. This will ensure your tasks trigger off of the messages correctly. Adapt the ``[[dependencies]]`` section in the :cylc:conf:`flow.cylc` file to read as follows: .. code-block:: diff [[[P2M]]] graph = """ - long_forecasting_task => another_weather_task - long_forecasting_task => different_weather_task + long_forecasting_task:update1 => another_weather_task + long_forecasting_task:update2 => different_weather_task long_forecasting_task[-P2M] => long_forecasting_task """ This completes our :cylc:conf:`flow.cylc` file. Our final workflow should look like this: .. spoiler:: Solution warning .. code-block:: cylc [meta] title = "test workflow to demo message triggers" [scheduler] UTC mode = True [scheduling] initial cycle point = 2019-06-27T00Z final cycle point = 2019-10-27T00Z [[graph]] P2M = """ long_forecasting_task:update1 => another_weather_task long_forecasting_task:update2 => different_weather_task long_forecasting_task[-P2M] => long_forecasting_task """ [runtime] [[long_forecasting_task]] script = """ sleep 2 random.sh cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \ "Task partially complete, report ready to view" sleep 2 random.sh cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \ "Task partially complete, report updated" sleep 2 random.sh """ [[[outputs]]] update1 = "Task partially complete, report ready to view" update2 = "Task partially complete, report updated" [[another_weather_task, different_weather_task]] script = sleep 1 #. **Validate the workflow.** It is a good idea to check that our :cylc:conf:`flow.cylc` file does not have any configuration issues. Run ``cylc validate`` to check for any errors: .. code-block:: bash cylc validate . #. **Install and Play the workflow.** Now we are ready to run our workflow. Validate, install, then open the :ref:`GUI ` or :ref:`TUI ` and play the workflow. .. code-block:: bash cylc validate . cylc install cylc play message-triggers Your workflow should now run, the tasks should succeed. #. **Inspect the work directory.** You can now check for your report outputs. These should appear in the :term:`work directory` of the workflow. All being well, our first cycle point should produce a test file with some random numbers, and each subsequent cycle point file should have more random numbers added. #. **Extension.** Suppose now we would like to send an email alerting us to the reports being ready to view. We will need to add to our :cylc:conf:`flow.cylc` file. In the ``runtime`` section, add a sub-section called ``[[[events]]]``. Within this section we will make use of the built-in setting ``mail events``. Here, we specify a list of events for which notifications should be sent. The events we are interested in are, in this case, our outputs. Add the following code to your ``[[[events]]]`` section. .. code-block:: cylc [[[events]]] mail events = update1, update2 Our updated workflow should look like this: .. spoiler:: Solution warning .. code-block:: cylc [scheduler] UTC mode = True [meta] title = "test workflow to demo message triggers" [scheduling] initial cycle point = 2019-06-27T00Z final cycle point = 2019-10-27T00Z [[graph]] P2M = """ long_forecasting_task:update1 => another_weather_task long_forecasting_task:update2 => different_weather_task long_forecasting_task[-P2M] => long_forecasting_task """ [runtime] [[long_forecasting_task]] script = """ sleep 2 random.sh cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \ "Task partially complete, report ready to view" sleep 2 random.sh cylc message -- "${CYLC_WORKFLOW_ID}" "${CYLC_TASK_JOB}" \ "Task partially complete, report updated" sleep 2 random.sh """ [[[outputs]]] update1 = "Task partially complete, report ready to view" update2 = "Task partially complete, report updated" [[[events]]] mail events = update1, update2 [[another_weather_task, different_weather_task]] script = sleep 1 Save your changes and run your workflow. Check your emails and you should have, one email for the first update and, a second email alerting you to the subsequent updated reports being ready. Note that the second email automatically bundles the messages to prevent your inbox from being flooded.