Workflow Completion

A workflow is complete, and the scheduler will automatically shut down, if no tasks remain in the n=0.

That is, all active tasks have finished, and no tasks remain waiting on prerequisites or “external” constraints (such as xtriggers or task hold).

If no active tasks remain and all external constraints are satisfied, but the n=0 window contains tasks waiting with partially satisfied prerequisites, or tasks with final status and incomplete outputs, then the workflow is not complete and the scheduler will stall pending manual intervention.

Scheduler Stall

A stalled workflow has not run to completion but cannot continue without manual intervention.

A stalled scheduler stays alive for a configurable timeout period pending manual intervention. If it shuts down (on the stall timeout or otherwise) it will remain in the stalled state on restart.

Stalls are often caused by unexpected task failures, either directly (tasks with final status and incomplete outputs) or indirectly (tasks with partially satisfied prerequisites, downstream of an unexpected failure).

Warning

At present you have to consult the scheduler log to see the reason for a stall.