cylc.flow.main_loop.log_memory

Log the memory usage of a running scheduler over time.

Note

This plugin is for Cylc developers debugging cylc memory usage.

For general interest memory measurement try /usr/bin/time -v cylc play or cylc play --profile.

Note

Pympler associates memory with the first object which references it.

In Cylc we have some objects (e.g. the configuration) which are references from multiple places.

This can result in a certain amount of “jitter” in the results where pympler has swapper from associating memory with one object to another.

Watch out for matching increase/decrease in reported memory in different objects.

Warning

This plugin can slow down a workflow significantly due to the complexity of memory calculations.

Set a sensible interval before running workflows.

If matplotlib is installed this plugin will plot results as a PDF in the run directory when the workflow is shut down (cleanly).

Python API

Coroutines

init	Take an initial memory snapshot.
report	Take a final memory snapshot and dump the results.
take_snapshot	Take a memory snapshot

async cylc.flow.main_loop.log_memory.init(scheduler, state)

Take an initial memory snapshot.

async cylc.flow.main_loop.log_memory.report(scheduler, state)

Take a final memory snapshot and dump the results.

async cylc.flow.main_loop.log_memory.take_snapshot(scheduler, state)

Take a memory snapshot