Skip to content

Publish on a schedule and on demand, without a run racing itself #32

Description

@iderex

Depends on the publication route.

Plugins release when they release. The catalogue has to catch up without somebody
remembering to press a button, so the pipeline runs on a schedule, and it also
runs on demand because the first thing anybody wants after a release is the
catalogue updated now.

Two runs at once over the same destination is the failure this issue is mostly
about. A scheduled run and a manual run overlapping can interleave writes and
leave a file neither of them intended. The fix is that runs over one destination
are serialised and a run holding the write is never cancelled mid-write, which is
the opposite of the usual setting where a newer run supersedes an older one.

There is a second-order trap worth writing at the workflow rather than
discovering: a serialisation group usually holds one queued run, so triggering a
manual run while a scheduled one is queued can evict the queued one. That is
acceptable and it has to be stated where somebody reads it, not learned from a
missing publish.

A run that concludes without writing anything is not a success, and the schedule
is exactly where that goes unnoticed.

Done when

The pipeline runs on a schedule and on manual dispatch, two concurrent runs over
one destination cannot interleave, a run holding the write is not cancellable by a
newer one, and the run output distinguishes wrote-new-bytes from
nothing-to-write. Machine-decidable.

Metadata

Metadata

Assignees

Labels

blocked-on-leadWartet auf eine Entscheidung des Leadsci

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions