The confirmation run on 812eb3d (PDFS_SIM_KNOWN=fail, twelve seeds)
passed every profile, so no open bug is tolerated any more: the call,
hang and damage matchers for B125 to B134 go, and any of them now fails
the seed. B126, B128 and B133 were fixed with B138, B162 and B137;
B134 stays open with local_first off only, which the runs do not mount.
The known-bug confirmation run failed one_client_on_a_flaky_link seeds 6
and 11 on three bugs around a create whose answer was lost:
- B182: after a restart, a write over a file whose create landed but was
never read back forked a conflict copy. A revision of a file that has
no revision of Drive's now adopts the landed node first.
- B183: the SHA-1s a create sent were kept in memory, so after a restart
its trash or retry no longer knew the node Drive made. They are now on
the op's row (pending_op.sent_sha1, schema 42).
- B184: a rename rewrote the op's target, and the trash and the retry
looked for the node under the last name only. Each attempt's parent
and name are recorded (pending_op.sent_to); the trash looks under all
of them, and a retry adopts the node an earlier attempt made and moves
it to the current name.
A refused change was only on the Sync page. A folder listing now carries
each node's issue (by local id, so a file not yet on Drive has it too),
and the browser's badge ranks it first, folders included, with the
issue's explanation as its tooltip. The event sampler publishes Files
when the set of issues changes, so the badge comes and goes with it.
A change Drive will not take could be exported or retried, never let go
of. The Sync page's Discard button (with a confirmation) and
`pdfs sync discard <id>` drop it: a node made here goes, as if deleted
before it was sent; a node Drive has goes back to what Drive has, under
the same inode, with every change queued on it, while what is queued
below it stays. A change on the wire or an open file is refused.
The daemon logs it as "<reason>; local changes uploaded as <copy>" or
"...kept at <path>", which the Activity page showed as written, in
English. It now reads both and the three reasons, one sentence each.
A mirror target's root is plain local storage, so the durability case's
wait for the mount passed at once after the restart. The queue wait then
read `pdfs --json status` before the new daemon listened, took its null
"mount" for an empty queue, and the run's cleanup found no control socket.
A file deleted after its create was sent keeps the op as a trash. The op
kept the create's attempts and backoff, and the drain recorded the failure
of a create attempt still on the wire on the row that was by then the trash.
The trash waited out the create's backoff.
The trash starts due, as a create that becomes a revision does, and the
drain records a failed attempt only if the op is still the kind it ran as
(`Db::record_attempt_failure`).
An upload that landed let go of its blob before the tree had its node. A
write open in between took the base it sampled at its start, the revision
the upload replaced, found none of its bytes cached, and queued an
incomplete revision; the next write open failed with EIO.
The drain keeps the landing node aside until the tree has it, and a write
open with no queued revision takes its base from that node, or from the
tree as it is when the handle goes in. The fake Drive can hold the drain
in that window (`hold_next_landing`).
A stop that came while a mirror pass was uploading let the pass go on to
the next level of folders until the stop gave up on it and the runtime's
end cancelled the uploads under it. Once the daemon is stopping, a batch
starts none of its waiting ops, lets the ones sending finish, and ends
the pass; the remote walk and a new pass stop too.
A delete of a file whose create was on the wire withdraws the create into
a trash. When its folder was removed too and that trash landed first,
Drive refused the create, and the drain re-homed the deleted file in the
root; only its missing bytes stopped it. The withdrawn create's trash
then read the trashed folder back into the cache before finding nothing
to trash.
A withdrawn create is no longer re-homed, and its trash is done once the
folder it was sent to is trashed or gone.
The hydrate rebuilt each listed folder's listing with a pass over every
entry. With 4,600 listed folders and 37,000 entries that took 1.7 s at
every start before the mount came up; one pass grouping the entries by
parent takes 3 ms.
The account run's 0-byte read of a new file was B166 too: Drive's
listing had the node from before its create was committed, and the
queue no longer held it when read. The fake Drive can now hold a
folder's listing on its way back with one file listed unrevised.
A listing from Drive was overlaid with the queue as it was after Drive
answered. A create that landed in between was in neither, and a wide
folder on Wi-Fi dropped files that were never unlinked. The queue is now
read before Drive is asked too; a node that left it meanwhile is listed
as the DB has it.
A stop joined every background thread for as long as each took. The sync
engine's mirror pass calls Drive with no deadline, and a stop that waited for
it was killed by systemd at TimeoutStopSec. The stop now waits up to 10 s for
the background threads and FUSE workers together, names any still running, and
the runtime shuts down within 2 s.
A listing from Drive overlays what is still queued for the folder. A create
Drive has made but whose answer is on its way, or which the drain is still
reading back, was in both: Drive's node under its new uid and the queue's
stand-in, under one name. A node queued in the folder now holds its name there.
A folder removed and made again met the old folder while its trash was on
the wire and adopted it, so the new folder's contents went into the old
one and were trashed with it. The twin search now skips hidden nodes, as
listings do.
The fake Drive can hold a trash (hold_next_trash).
adopt_real_uid asked whether a move was queued only after reading the
node back. A move that landed in between left no op, and the answer still
had the old name, so the mount showed the file under it. Ask first.
The fake Drive can hold a read's answer (hold_answer_to_next_read) so the
test can land the move while the answer is on its way.
Queueing a folder's create left the workers asleep until their 30 s idle
poll; every other enqueue woke them. A file's create still waits for the
release that attaches its bytes.
A lookup in the root publishes the "Shared with me" entry only if the root
listing did not change while its plan was read, and answered EAGAIN when it
had. Right after a restart the root is listed again as the first lookups come
in, so a write could fail with "Resource temporarily unavailable".
ensure_children now takes the listing again, up to four times.
B141 drops a rename that fails with a newer one queued behind it. The
worker records the failure and releases its claim in two steps, and a
rename queued between them waited behind the old one, which was kept
with a backoff. Released, the old one was not due and the new one was,
so the new one landed first and the old one then put the first name
back.
Releasing the claim now drops the old rename too when a newer one is
queued, and both checks match the node by local id as well as by uid.
When the read-back after an upload went unanswered, the drain let go of
the staged blob without caching it. A partial write opened next had
nothing to fill its gaps from and queued an incomplete revision, so the
write open after it was refused with EIO.
The bytes are now cached under the size and mtime the tree still has,
which is the base the next write takes. Their SHA-1 stands in for the
revision id the read-back did not bring, so that write chains onto the
upload and does not fork a conflict copy of itself.
nvrl
Nils Pukropp