My files, Shared, Trash and the launcher now ask the daemon for video
thumbnails, not only image ones. Grid tiles with a video thumbnail show
a small play badge.
A `.ts` file is a video from 1 MiB up and a TypeScript source below
that. One rule in pdfs-core (`is_video_file_name`) decides it for the
file icon, the thumbnail gate, the launcher icon and preview, and the
opener's `@media` / `@text` classes. The daemon also checks the MPEG-TS
sync byte before it runs ffmpeg, so a large TypeScript bundle gets no
retries.
A video thumbnail no longer downloads the whole file into memory (B283):
- the server thumbnail is tried first, at no content cost;
- files up to 24 MiB come whole;
- larger ones bring a 16 MiB head, plus the MP4/MOV `moov` box when it
lies past the head, into a sparse staging file that ffmpeg reads.
The 256 MiB cap is gone. Without ffmpeg, a video answers "no thumbnail"
at once. The folder thumbnail build includes videos.
A remote change to a pinned file called `ContentCache::evict`, which does
not check pins, and only the pin request itself ever downloaded anything.
An edit from another device, or even a rename, left the file listed as kept
offline with no copy, so the next read downloaded it or failed without a
network. A file added to a pinned folder elsewhere was never downloaded,
and a folder pin whose downloads failed while offline never retried them.
A remote change no longer evicts a pinned file's copy. The copy is checked
against the revision's (mtime, size), so a newer revision does not read
it, and a rename keeps the mtime (B123), so the copy stays valid. A new
pdfs-pin-keeper thread checks pins again and downloads files that are
missing or stale:
- the pins a batch of remote events changed, after the batch's listings
are dropped, including a pinned folder that gets a new file;
- every pin at startup, when the link returns, and after the event feed
loses continuity;
- a file whose version is restored.
Failed downloads retry with a backoff of 30 s to 30 min. Pinning and the
keeper settle a provisional size first: a copy stored under the
ciphertext size never matched the plaintext it holds.
Fixes B282.
`pdfs pin` and `pdfs unpin` resolved every path against the primary mount
with `rel_to_mount`. An absolute path in an on-demand synced folder such as
~/Pictures was refused as "not under the mountpoint", and a relative path
named a file in My files instead. A synced folder's file is a different
Drive node from a same-named file in My files, so keeping one offline did
nothing for the other, and every program reading the synced copy
downloaded it.
Pin and unpin now route through `route_to_mount`, like ls, refresh, rm and
mv since B86, and run on the mount that owns the path. A path in a mirror
folder gets an error saying its files are already on this computer. A pin
made through an on-demand folder records the absolute local path, so
`pdfs pins`, the prompt and the HUD open the file where it was kept; pins
under the primary mount still record the mount-relative path.
Fixes B281.
Schema 45 separates attachments from device backups so stopping sync or
changing mode cannot trash My files folders or evict their local data.
Exclude attachments from device profiles and validate device restores.
Verify attached roots before each pass and cover both local views,
concurrent edits, removal guards, and root loss across daemon restarts.
Refs #77
FUSE mounts over a non-empty folder without complaint, so files already
in the My files folder were hidden under the mount: not uploaded, and
back only once the service stopped, so they looked deleted (#67, B277).
- pdfs-core: new `mountpoint` module. `not_empty` lists what is in a
folder (hidden entries count); `mounted_fs_type` moves here from the
CLI so the daemon can use it. `MountFailure` gains an optional
`not_empty` field, which older readers ignore.
- pdfs-fuse: `mount_with` checks the folder before the root fetch and
before any worker starts. A dead mount is cleared first; one of our
own live mounts is left to `spawn_session`, as before.
- pdfs-cli: `cmd_daemon` records the folder in the failure record. The
refusal backs off and retries like any mount failure, so emptying the
folder is enough.
- pdfs-gui: the status pages explain the refusal, and a dialog offers
to open the folder or choose another one, once per run of failures.
The folder picker refuses a folder with files in it.
Translated in all catalogs. Docs: BUGS (B277), CHANGELOG, FAQ,
TROUBLESHOOTING, CONFIGURATION.
A write open copies the queued revision's blob into its scratch file.
When the upload landed between the sample and the copy, the drain had
already deleted the blob, and the open failed with EIO (B274, seen once
in CI). The drain drops the pending entry before the blob, so a failed
copy whose entry moved on now goes around again, as the re-check after
the copy already does.
Two simulations raced the daemon rather than it:
- a_file_deleted_after_a_restart_leaves_drive_with_what_its_create_sent
let the create's reply through before the kernel's release attached
the second bytes. That attach resets the attempts, so the retry
landed before the test could see the failure. It now waits for the
bytes to be attached.
- a_file_deleted_offline_while_open_still_reads_and_takes_writes read
the cache as the queue emptied, which is a moment before the
eviction. It now waits for the cache to empty.
A create that carries a file's bytes retired its op, then cached the
bytes and gave the tree the real uid. Nothing counted that window, so
`pdfs status` read empty while it ran, which B119 promises means the
file reads without the network.
A delete in that window found the placeholder uid in the tree and
trashed the node by its real uid. `queue_trash` asked the tree whether
that uid was open, found no entry, and dropped the cached bytes the open
handle reads. If the cache store ran at the same moment, its tag was
written after the eviction and never removed. CI caught that as a
stray .meta file.
The create's landing is now counted like an upload's, from the op's
retirement until the tree has the real uid. The delete and the replace
pass whether they found the file open. Opening the cache removes a tag
whose bytes are gone, which also covers a crash between an eviction's
two removals. The simulations wait for `Daemon::drained`, which counts
the landing, before they go offline.
B255 and B256 move to fixed: the account run checked both.
A switch asked for before any pass saw the folder empty found it still
idle, so the check for what only this computer has ran at once and
refused it for good: every synced path read as "deleted here but not on
Proton Drive yet". The account run hit it in the B255 case.
A local copy gone in full is the wipe guard's case, which the push pass
of a queued switch settles without trashing anything on Drive. The
switch now returns NotNow for it, so the request is queued and that pass
runs first. A partial deletion is still refused as before.
The probe that sees the link return runs 2 s after the drop, then 6 s.
A loaded run could turn the link back on after the first probe, and the
next one came after the test's 3 s were up. The test now checks that the
drain takes the mount offline within 2 s, which is the B189 fix itself,
and counts the 3 s from the mount reporting it is back online.
A mirror folder whose files were all gone from this computer stayed in
error for good. The wipe guard rightly refuses to trash the whole folder
on Drive, but nothing got the folder out of that state: a queued switch
to on-demand waits for a pass that ends idle, and its push pass hit the
same guard. Only removing the folder from sync cleared the baseline that
made the empty folder read as "delete everything".
- The push pass for a switch to on-demand forgets the baseline rows of a
local copy that is gone in full instead of trashing them, so the
switch goes ahead and the mount shows what Drive has.
- New Request::ResetSyncFolder, `pdfs sync reset ID` and "Sync From
Scratch…" in a synced folder's menu clear a mirror folder's baseline
(and a queued switch to on-demand, whose push pass would upload a
duplicate of every file). The next pass compares both sides as on the
first sync: one-sided files are copied across, files that differ keep
both versions, nothing is deleted.
- The wipe guards' messages name the reset; restore logs why it skipped
a folder.
Two simulations cover it. The on-demand one failed in exactly the
reported state (mirror, pending ondemand, error) before the fix.
All 8 catalogs translated.
A copy inside the mount went through the kernel's read/write loop, one
FUSE round trip per page, because copy_file_range answered ENOSYS. The
daemon now copies a range whose source bytes are local (an open write
handle, a queued write, the cache) into the destination's scratch file,
the same way write stages them. A range it would have to download gets
EOPNOTSUPP, which makes the kernel copy that one call itself and keeps
the operation enabled. The copy still uploads: Drive cannot copy into a
file cp has already created.
In the simulation, a warm cp of a 256 MiB file went from about
100 MiB/s to about 450 MiB/s (debug build).
`pdfs copy PATH... NEW_PARENT` sends a new control request, Copy, which
copies a file with its content or a folder with its subtree on Proton
Drive through proton-drive-rs's copy_node. Nothing is downloaded or
uploaded. A name already taken in the destination gets the next free
" (n)" suffix through get_available_name.
A copy cannot wait in the queue, because Drive copies what it holds.
So it is refused offline, and while the mount still owes Drive part of
the node: a local placeholder, a queued op, unsaved writes, or, for a
folder, any queued op at all. Local-first does not change this.
DriveApi gains copy_node and get_available_name, and the fake drive
copies subtrees, so a FUSE sim test covers the copy, the suffix, the
folder subtree and both refusals. The activity feed logs a new Copy
kind, translated in every catalog.
Needs the proton-drive-rs release that adds copy_node; until then, build
with a [patch.crates-io] pointing at proton-sdk-rs.
A mirrored folder assumed it was the only client writing to its folder on
Drive. With a second computer syncing the same folder, edits could be lost:
- The first sync took a same-size file at the same path for the same file.
It now records the baseline only if the local file's SHA-1 matches Drive's
digest, and otherwise keeps a conflict copy (B248).
- An upload replaced a revision another client had committed since the pass
looked. The upload now checks Drive's active revision first, and an edit
that read back with someone else's digest is left for the next pass to keep
as a conflict copy (B249).
- A download recorded a revision committed while it ran as synced, so that
edit never arrived. The baseline now records the revision the pass planned
to download (B250).
- Conflict copies now carry the computer's hostname, so two computers do not
make the same name on Drive (B251).
The baseline gains the revision id it was synced at (schema 44), and a new
revision counts as a remote change even at the same mtime and size.
flush answered ok at once and release synced the scratch file, but close(2)
only returns what flush answers, so a full disk at close was reported to
nobody. flush now syncs a handle with writes on a worker and returns ENOSPC,
EDQUOT or EIO; release then finds the pages on disk instead of writing a
whole file back on the dispatch loop. fsync on a descriptor opened for
reading no longer fails with EBADF (B222).
fallocate passed any mode through and tracked only a punched hole or
growth: a zeroed range came back from the remote base at commit, and a
collapse or insert corrupted the file. Unsupported modes now get EOPNOTSUPP,
zeroed ranges count as written, the real errno is returned, and the state
lock is not held across the syscall (B224).
FUSE delivers `release` after `close()` returns, and the release is what
attaches the written bytes to the queued create. A drain worker that is
free can claim the create first and send the file empty, with the bytes
following as a revision. Tests that hold a create, its reply, or lose
its reply then act on the empty create, and their assertions about the
file's content race the revision. CI saw this as an occasional failure
of a_listing_taken_before_a_create_commits_keeps_the_landed_file.
Add Daemon::write_new, which pauses sync, writes the file, waits until
the pending create carries the bytes, and resumes. Use it for every
write that directly follows a create hold.
A folder pin reads each file's modification time and size before it
downloads the file. If the file's create landed during the download, the
landing set Drive's time, and the pin then stored the blob under the old
time. The file showed as kept offline but read as stale.
After each download, the pin now compares the file's time and size with
the tree's. If they changed, it downloads the file again under the new
values, at most three times.
The simulation gains FakeClient::hold_next_download to force that order.
The account run of 3.3.0 failed "move refuses a file still being
written" on the on-demand to mirror pairing: pdfs move moved the folder
while open.txt in it was open for writing, and the write landed after
the move.
retire_create re-keys a landed create's nodes row to the real uid first
and repoints the tree and the open write handles after. A file opened in
between holds a handle with the placeholder uid. has_pending_work_under
placed each open handle by looking its uid up in the nodes table, which
no longer has the placeholder, so it saw nothing open under the folder,
and with the queue already empty the move went ahead. Open handles are
now placed in the in-memory tree by inode, which a landing create does
not change.
The simulation test holds the drain at that point with the new
FakeClient::hold_next_landing_of_any, and fails without the fix.
A simulated daemon could not add a synced folder, so nothing that moves
between a mount, an on-demand folder and a mirror folder could be tested
without a real account.
Listing and registering devices now go through DriveApi, and the fake
Drive keeps the account's computers, each with a root folder outside My
files. The simulated daemons also read their config.json from a scratch
XDG directory instead of the developer's own: an adopted device of the
real account made every sync folder fail to register against the fake
Drive.
The daemon has kept recursive folder pins for a long time, but the app
offered "Available offline" for files only. The context menu, the
details pane and the selection bar now offer it for folders too, and a
pinned folder carries the offline badge in the browser and the launcher.
A folder pin used to download the whole subtree inside the control
request, so a large folder outlasted the client's 120 s read timeout,
and every pinned file was read into memory before it was cached. The
daemon now records the pin, answers at once and downloads on a
background thread as a job the tray and the app show, stopping when the
folder is unpinned or the daemon stops. Pins of files and folders both
go through fetch_content, which streams to disk and skips fresh blobs.
A folder pin is refused while offline instead of looking kept.
A file kept only because a folder above it is pinned has no pin of its
own, and unpinning it silently did nothing. ListDir now reports
pinned_by_folder; the app shows such an entry as available offline with
the toggle greyed out, and the daemon refuses the unpin with a message
naming the folder. Unpinning a folder no longer evicts files that are
pinned on their own.
Requested on Reddit.
A partial write whose base was not in the content cache queued an
incomplete revision, and every write open over it was refused with EIO
until the drain uploaded it. CI's one-client-lan simulation hit this on
seed 1: a create over a file whose partial edit had not drained yet.
The write open now copies the blob and fills its gaps from the remote
base with Core::fill_gaps, as the drain does, and fails only when that
fill fails (B204).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EuyxByBpK3fSWtyPkugeHz
The known-bug confirmation run failed one_client_on_a_flaky_link seeds 6
and 11 on three bugs around a create whose answer was lost:
- B182: after a restart, a write over a file whose create landed but was
never read back forked a conflict copy. A revision of a file that has
no revision of Drive's now adopts the landed node first.
- B183: the SHA-1s a create sent were kept in memory, so after a restart
its trash or retry no longer knew the node Drive made. They are now on
the op's row (pending_op.sent_sha1, schema 42).
- B184: a rename rewrote the op's target, and the trash and the retry
looked for the node under the last name only. Each attempt's parent
and name are recorded (pending_op.sent_to); the trash looks under all
of them, and a retry adopts the node an earlier attempt made and moves
it to the current name.
A refused change was only on the Sync page. A folder listing now carries
each node's issue (by local id, so a file not yet on Drive has it too),
and the browser's badge ranks it first, folders included, with the
issue's explanation as its tooltip. The event sampler publishes Files
when the set of issues changes, so the badge comes and goes with it.
A change Drive will not take could be exported or retried, never let go
of. The Sync page's Discard button (with a confirmation) and
`pdfs sync discard <id>` drop it: a node made here goes, as if deleted
before it was sent; a node Drive has goes back to what Drive has, under
the same inode, with every change queued on it, while what is queued
below it stays. A change on the wire or an open file is refused.
An upload that landed let go of its blob before the tree had its node. A
write open in between took the base it sampled at its start, the revision
the upload replaced, found none of its bytes cached, and queued an
incomplete revision; the next write open failed with EIO.
The drain keeps the landing node aside until the tree has it, and a write
open with no queued revision takes its base from that node, or from the
tree as it is when the handle goes in. The fake Drive can hold the drain
in that window (`hold_next_landing`).
A stop that came while a mirror pass was uploading let the pass go on to
the next level of folders until the stop gave up on it and the runtime's
end cancelled the uploads under it. Once the daemon is stopping, a batch
starts none of its waiting ops, lets the ones sending finish, and ends
the pass; the remote walk and a new pass stop too.
A delete of a file whose create was on the wire withdraws the create into
a trash. When its folder was removed too and that trash landed first,
Drive refused the create, and the drain re-homed the deleted file in the
root; only its missing bytes stopped it. The withdrawn create's trash
then read the trashed folder back into the cache before finding nothing
to trash.
A withdrawn create is no longer re-homed, and its trash is done once the
folder it was sent to is trashed or gone.
The account run's 0-byte read of a new file was B166 too: Drive's
listing had the node from before its create was committed, and the
queue no longer held it when read. The fake Drive can now hold a
folder's listing on its way back with one file listed unrevised.
A listing from Drive was overlaid with the queue as it was after Drive
answered. A create that landed in between was in neither, and a wide
folder on Wi-Fi dropped files that were never unlinked. The queue is now
read before Drive is asked too; a node that left it meanwhile is listed
as the DB has it.
A listing from Drive overlays what is still queued for the folder. A create
Drive has made but whose answer is on its way, or which the drain is still
reading back, was in both: Drive's node under its new uid and the queue's
stand-in, under one name. A node queued in the folder now holds its name there.
A folder removed and made again met the old folder while its trash was on
the wire and adopted it, so the new folder's contents went into the old
one and were trashed with it. The twin search now skips hidden nodes, as
listings do.
The fake Drive can hold a trash (hold_next_trash).
adopt_real_uid asked whether a move was queued only after reading the
node back. A move that landed in between left no op, and the answer still
had the old name, so the mount showed the file under it. Ask first.
The fake Drive can hold a read's answer (hold_answer_to_next_read) so the
test can land the move while the answer is on its way.
Queueing a folder's create left the workers asleep until their 30 s idle
poll; every other enqueue woke them. A file's create still waits for the
release that attaches its bytes.
When the read-back after an upload went unanswered, the drain let go of
the staged blob without caching it. A partial write opened next had
nothing to fill its gaps from and queued an incomplete revision, so the
write open after it was refused with EIO.
The bytes are now cached under the size and mtime the tree still has,
which is the base the next write takes. Their SHA-1 stands in for the
revision id the read-back did not bring, so that write chains onto the
upload and does not fork a conflict copy of itself.
B154 is a regression from the B153 fix: after a failure on a held name the
op woke the ops waiting for it, but the database cannot see a create on the
wire, so the op was due again at once and spent its waits in milliseconds.
No wake while another create on the wire holds the name.
B155: a revision upload that reached Drive but lost its answer left the
file at a revision this daemon never heard of, and the retry kept the write
as a conflict copy of itself. A complete write whose bytes the remote holds,
by size and SHA-1, is sent again instead.
The fake Drive gains hold_reply_to_next_create and
lose_reply_to_next_revision for the two new sim tests.
nvrl
danielebrox
Claude