A supervisor thread ticks every 15 s and does three things: it WARNs
about a worker that has held one job for two minutes, a control request
that never returned, or a lane that has not completed anything while its
queue is non-empty; it samples resident size and the pending-op count
into the log every five minutes; and it answers the systemd watchdog.
The watchdog ping is deliberately not a heartbeat from a thread that
does nothing else — that would keep the unit "healthy" through exactly
the hang it is meant to catch. It goes out only after a real round trip
over the control socket, so it means "a client can still talk to this
daemon", which is the property that failed on 2026-09-18.
`sd_notify` is spoken directly rather than through libsystemd (READY,
WATCHDOG, STOPPING), including the abstract-namespace address; off
systemd every call is a no-op.
The unit gains `NotifyAccess=all` (rather than `Type=notify`, which
would fail the unit on the start timeout when no login is stored yet),
`WatchdogSec=120`, `Restart=always` — a hang is not a failing exit code
— and `MemoryHigh`/`MemoryMax` as a backstop against the 5.2 G growth
the hung run showed.
nvrl