EmPy

Cylc also supports the EmPy template processor in workflow configurations. Like Jinja2, EmPy provides variables, mathematical expressions, loop control structures, conditional logic, etc., that gets expanded to generate the final workflow configuration seen - which must must be valid Cylc syntax. See EmPy documentation for details of its templating features and how to use them.

Note

EmPy is not included in the standard Cylc installation. If installing with pip run pip install cylc-flow[empy] to install it.

To enable EmPy place an empy hash-bang comment on the first line of the flow.cylc file:

#!empy

As an example, here is the “cities” workflow from the previous section, implemented with Empy instead of Jinja2.

#!EmPy
[meta]
    title = "EmPy city workflow example."
    description = """
        Illustrates use of variables and math expressions, and programmatic
        generation of groups of related dependencies and runtime properties.
    """

[scheduler]
    allow implicit tasks = True

@{
HOST = "SuperComputer"
CITIES = 'NewYork', 'Philadelphia', 'Newark', 'Houston', 'SantaFe', 'Chicago'
CITYJOBS = 'one', 'two', 'three', 'four'
LIMIT_MINS = 20
CLEANUP = True
}

[scheduling]
    initial cycle point = 2011-08-08T12
    [[graph]]
@[ if CLEANUP ]
        T23 = clean
@[ end if ]
        T00,T12 = """
            setup => get_lbc & get_ic # foo
@[ for CITY in CITIES ]@# comment
            get_lbc => @(CITY)_one
            get_ic => @(CITY)_two
            @(CITY)_one & @(CITY)_two => @(CITY)_three & @(CITY)_four
    @[ if CLEANUP ]
            @(CITY)_three & @(CITY)_four => cleanup
    @[ end if ]
@[ end for ]
        """
[runtime]
    [[on_@HOST ]]
        [[[remote]]]
            host = @HOST
            # (remote cylc directory is set in site/user config for this host)
        [[[directives]]]
            wall_clock_limit = "00:@(LIMIT_MINS + 2):00,00:@(LIMIT_MINS):00"

@[ for CITY in CITIES ]
    [[ @(CITY) ]]
        inherit = on_@(HOST)
    @[ for JOB in CITYJOBS ]
    [[ @(CITY)_@(JOB) ]]
        inherit = @CITY
    @[ end for ]
@[ end for ]

For basic usage, the difference between Jinja2 and EmPy amounts to little more than a different markup syntax. EmPy might be preferable, however, in cases needing more complicated processing logic.

EmPy is a system for embedding Python expressions and statements in template text. It makes the full power of Python language and its ecosystem easily accessible from within the template. This might be desirable for several reasons:

• No need to learn a different language just for writing template logic

• Availability of lambda functions, lists, and dictionary comprehensions can make template code smaller and more readable compared to Jinja2

• Natural and straightforward integration with the Python package ecosystem

• No two-language barrier between writing template logic and processing extensions makes it easier to refactor and maintain the template code as its complexity grows. Inline Python code can be gathered into subroutines and eventually into separate modules and packages in a seamless manner.

Workflow files and Context Variables

The EmPy processor has the same access as Jinja2 to workflow files and workflow context variables.