A switch asked for before any pass saw the folder empty found it still
idle, so the check for what only this computer has ran at once and
refused it for good: every synced path read as "deleted here but not on
Proton Drive yet". The account run hit it in the B255 case.
A local copy gone in full is the wipe guard's case, which the push pass
of a queued switch settles without trashing anything on Drive. The
switch now returns NotNow for it, so the request is queued and that pass
runs first. A partial deletion is still refused as before.
The worker thread's sleeps could begin before the test read the clock, so the elapsed time came out a few milliseconds under the 90 ms bound on a busy runner. Reading the clock first makes the bound hold.
The probe that sees the link return runs 2 s after the drop, then 6 s.
A loaded run could turn the link back on after the first probe, and the
next one came after the test's 3 s were up. The test now checks that the
drain takes the mount offline within 2 s, which is the B189 fix itself,
and counts the 3 s from the mount reporting it is back online.
A mirror folder whose files were all gone from this computer stayed in
error for good. The wipe guard rightly refuses to trash the whole folder
on Drive, but nothing got the folder out of that state: a queued switch
to on-demand waits for a pass that ends idle, and its push pass hit the
same guard. Only removing the folder from sync cleared the baseline that
made the empty folder read as "delete everything".
- The push pass for a switch to on-demand forgets the baseline rows of a
local copy that is gone in full instead of trashing them, so the
switch goes ahead and the mount shows what Drive has.
- New Request::ResetSyncFolder, `pdfs sync reset ID` and "Sync From
Scratch…" in a synced folder's menu clear a mirror folder's baseline
(and a queued switch to on-demand, whose push pass would upload a
duplicate of every file). The next pass compares both sides as on the
first sync: one-sided files are copied across, files that differ keep
both versions, nothing is deleted.
- The wipe guards' messages name the reset; restore logs why it skipped
a folder.
Two simulations cover it. The on-demand one failed in exactly the
reported state (mirror, pending ondemand, error) before the fix.
All 8 catalogs translated.
close(2) returns before the kernel sends release, and release takes the
write handle out of the open writes a few milliseconds before the
revision is in the queue. A `pdfs copy` checked in that gap found the
edit nowhere and copied the version Drive had. A folder copy looked only
at the queue, so it also missed files below it still open with unsaved
writes.
release now keeps the inode in State::releasing until the revision is
queued. A copy refuses a file or folder with unsaved writes, or a
release still queueing, at it or below it, and a move checks the same
set.
A copy inside the mount went through the kernel's read/write loop, one
FUSE round trip per page, because copy_file_range answered ENOSYS. The
daemon now copies a range whose source bytes are local (an open write
handle, a queued write, the cache) into the destination's scratch file,
the same way write stages them. A range it would have to download gets
EOPNOTSUPP, which makes the kernel copy that one call itself and keeps
the operation enabled. The copy still uploads: Drive cannot copy into a
file cp has already created.
In the simulation, a warm cp of a 256 MiB file went from about
100 MiB/s to about 450 MiB/s (debug build).
The share dialog gets Link settings under an existing link: role, password
and expiry apply at once and keep the URL. The daemon fills in the options a
change keeps from the link as it stands, and refuses to keep a password it
cannot read rather than dropping it. pdfs public-link update does the same
from a terminal, and pdfs members shows the expiry.
Details shows a folder's size and item count as Proton Drive counts it,
trash included, and pdfs du prints it. Both are refused offline and for a
folder that is not on Drive yet.
Needs proton-drive-rs with update_public_link, get_folder_size and
PublicLink::custom_password.
`pdfs copy PATH... NEW_PARENT` sends a new control request, Copy, which
copies a file with its content or a folder with its subtree on Proton
Drive through proton-drive-rs's copy_node. Nothing is downloaded or
uploaded. A name already taken in the destination gets the next free
" (n)" suffix through get_available_name.
A copy cannot wait in the queue, because Drive copies what it holds.
So it is refused offline, and while the mount still owes Drive part of
the node: a local placeholder, a queued op, unsaved writes, or, for a
folder, any queued op at all. Local-first does not change this.
DriveApi gains copy_node and get_available_name, and the fake drive
copies subtrees, so a FUSE sim test covers the copy, the suffix, the
folder subtree and both refusals. The activity feed logs a new Copy
kind, translated in every catalog.
Needs the proton-drive-rs release that adds copy_node; until then, build
with a [patch.crates-io] pointing at proton-sdk-rs.
A mirror upload that Drive did not list yet when it was read back left a
baseline row with no revision id. The next walk reused the recorded size
whenever the modification time matched, and a row without an id never
counts as moved. So a revision another computer saved in the same second
looked unchanged and never arrived (B254). The B250 simulation hit this
about one run in five on CI.
A row without an id now reuses its size only while Drive gives no id
either. Otherwise the walk reads the size, and an unchanged file gets
Drive's revision id written into its row, so each row costs one extra
read. Rows from before schema 44 are read once after upgrading.
Every computer that synced into one device wrote the same profile.json, so
each save replaced the other computers' folders with its own (B252). Each
machine now writes profile-<machine id>.json, where the id is random, made on
first use and kept in machine-id next to config.json. No host identifier goes
to Drive.
Restore reads every profile in .proton-drive-linux, including the shared one
older clients wrote. A folder's proposed path and mode come from this
computer's own profile, otherwise from the newest one that has the folder.
The legacy file is left as it is, so a client before 3.5.0 still finds it.
A mirrored folder assumed it was the only client writing to its folder on
Drive. With a second computer syncing the same folder, edits could be lost:
- The first sync took a same-size file at the same path for the same file.
It now records the baseline only if the local file's SHA-1 matches Drive's
digest, and otherwise keeps a conflict copy (B248).
- An upload replaced a revision another client had committed since the pass
looked. The upload now checks Drive's active revision first, and an edit
that read back with someone else's digest is left for the next pass to keep
as a conflict copy (B249).
- A download recorded a revision committed while it ran as synced, so that
edit never arrived. The baseline now records the revision the pass planned
to download (B250).
- Conflict copies now carry the computer's hostname, so two computers do not
make the same name on Drive (B251).
The baseline gains the revision id it was synced at (schema 44), and a new
revision counts as a remote change even at the same mtime and size.
A Takeout photo with no sidecar and no date in its name was filed under
the day of the import, and the app dated uploads by modification time.
A new `exiftime` module reads DateTimeOriginal (then DateTimeDigitized)
with its offset. Takeout uses it for undated photos it copies out for a
thumbnail, and the daemon uses it for app uploads before falling back to
the modification time (B244).
`import-google-photos --wait` and `redate-photos --wait` exited 0 however
the run ended, and a run that failed as a whole left no summary, so the
CLI said no import had run and the app said it finished. Failed runs now
leave a summary with the reason, `--wait` exits 1 on failure, cancel or
failed photos, and the app shows the reason (B239).
A reconnect made every failed op due at once, quota and access refusals
included, and ops that failed together retried in the same second. The
doubling also stopped at 128 s, short of the 300 s maximum. Reconnects
now move up only unrefused ops, and the backoff reaches its maximum with
a per-op spread of up to a fifth (B237).
Takeout imports, re-dates and the app's photo upload all sent an empty
thumbnail list, so the photos showed blank in Proton's apps. Re-dates and
app uploads now make thumbnails from their file. A Takeout photo that can
have one is copied out of the archive to the private staging folder,
thumbnailed and uploaded from the copy (B236).
Nothing stopped `/`, the home folder, the Proton Drive mount, the app's
own state or cache, or a folder nested in another root from becoming a
sync root, and a restore took a path from another machine's profile as
it was. One validator, `syncroot::problem`, now runs where roots are
added and restored, on paths resolved through symlinks with `..`
refused (B234).
The upload deadline assumed at least 32 KiB/s. Under an upload cap shared
by every drain worker and sync task, a large file ran out of time on every
attempt, took the mount offline and restarted from byte 0. The deadline
now counts the upload's share of the cap as the slowest rate when that
share is lower (B231).
- A gap-filled revision blob is recorded as complete once filled, so a
retry after a lost upload answer recognises its own bytes on Drive
instead of forking a conflict copy (B228).
- Commits that accept a change into the queue run at synchronous=FULL,
so an op `close()` reported as done survives a power cut (B229).
- A queued create's adoption window starts at its first send, not when
it was queued, so a create queued offline no longer takes another
device's empty node of the same name. A send record that does not
parse is kept instead of overwritten (B230).
A read on a write handle was answered on the dispatch loop to keep its
order with the handle's writes, and when the bytes it asked for were
neither written nor cached it downloaded them there, holding every request
of the mount for up to the read deadline. The loop now answers from the
scratch file and the local caches only, and hands a read that needs a
download to a transfer worker with the handle as it was when the read came
in (B227).
Thumbnails were decoded without the EXIF orientation, so a portrait photo
stored as landscape pixels plus a tag got a sideways tile, and the drain
uploaded that thumbnail to Drive for every client to show (B226).
flush answered ok at once and release synced the scratch file, but close(2)
only returns what flush answers, so a full disk at close was reported to
nobody. flush now syncs a handle with writes on a worker and returns ENOSPC,
EDQUOT or EIO; release then finds the pages on disk instead of writing a
whole file back on the dispatch loop. fsync on a descriptor opened for
reading no longer fails with EBADF (B222).
fallocate passed any mode through and tracked only a punched hole or
growth: a zeroed range came back from the remote base at commit, and a
collapse or insert corrupted the file. Unsupported modes now get EOPNOTSUPP,
zeroed ranges count as written, the real errno is returned, and the state
lock is not held across the syscall (B224).
A cold listing allocated the local id of every new child in a write
transaction while it held the global state lock, then marked the folder
listed, still under it. Every inline handler of every mount waited on that
transaction, and the transaction on the write connection the drain shares.
The ids are now allocated before the lock is taken, and the folder is marked
listed through the outbox after its children's rows, so a crash in between
no longer leaves it recorded as complete and empty (B221).
A shared folder's listing is fetched again once it is a minute old, and that
fetch ran on the dispatch loop whenever the old listing was still resident,
so every request on the mount waited for Drive (B219).
Thumbnail downloads, size upgrades and the Shared with me listing had no
deadline of their own. On a stalled link each held a mount thread for the
SDK's retry budget, and a folder of photos filled the three metadata workers
that every cold lookup and rename needs. Offline, an uncached thumbnail is
now answered as absent, and these calls give up after 20 or 60 seconds
(B220).
FTS5 keeps deleted entries until their segments are merged, which with
these indexes is close to never. On one machine local_fts held 317 MiB for
40,000 paths, against 20 MiB rebuilt, and search ran four times slower. The
indexes are now compacted once a day after a local scan. A scan that hit the
entry cap or lost a batch no longer prunes the paths it never reached, which
the next scan used to add back (B217).
Five ways a mirror pass could lose files:
- A folder deleted on Drive was removed here with everything in it, including
new files and edits whose uploads had just failed for want of a parent, and
ignored content such as .git. Such a folder now stays and is created on
Drive again (B211).
- A node the SDK could not decrypt was missing from the remote walk and so
read as deleted on Drive. A short listing or an unreadable size now leaves
the affected paths out of the pass, and a pass in which no synced path is
left on Drive is refused (B212).
- A file Drive deleted was removed here even when it had been edited since
the scan. It is now checked again first, and kept (B213).
- A folder deleted here was trashed on Drive although another device had just
put new files in it. It now stays on Drive (B214).
- A decrypted name holding / or .. became a path outside the sync folder, and
failed the whole listing in the mount. Such names are left out (B215).
FUSE delivers `release` after `close()` returns, and the release is what
attaches the written bytes to the queued create. A drain worker that is
free can claim the create first and send the file empty, with the bytes
following as a revision. Tests that hold a create, its reply, or lose
its reply then act on the empty create, and their assertions about the
file's content race the revision. CI saw this as an occasional failure
of a_listing_taken_before_a_create_commits_keeps_the_landed_file.
Add Daemon::write_new, which pauses sync, writes the file, waits until
the pending create carries the bytes, and resumes. Use it for every
write that directly follows a create hold.
A folder pin reads each file's modification time and size before it
downloads the file. If the file's create landed during the download, the
landing set Drive's time, and the pin then stored the blob under the old
time. The file showed as kept offline but read as stale.
After each download, the pin now compares the file's time and size with
the tree's. If they changed, it downloads the file again under the new
values, at most three times.
The simulation gains FakeClient::hold_next_download to force that order.
The account run of 3.3.0 failed "move refuses a file still being
written" on the on-demand to mirror pairing: pdfs move moved the folder
while open.txt in it was open for writing, and the write landed after
the move.
retire_create re-keys a landed create's nodes row to the real uid first
and repoints the tree and the open write handles after. A file opened in
between holds a handle with the placeholder uid. has_pending_work_under
placed each open handle by looking its uid up in the nodes table, which
no longer has the placeholder, so it saw nothing open under the folder,
and with the queue already empty the move went ahead. Open handles are
now placed in the in-memory tree by inode, which a landing create does
not change.
The simulation test holds the drain at that point with the new
FakeClient::hold_next_landing_of_any, and fails without the fix.
A simulated daemon could not add a synced folder, so nothing that moves
between a mount, an on-demand folder and a mirror folder could be tested
without a real account.
Listing and registering devices now go through DriveApi, and the fake
Drive keeps the account's computers, each with a root folder outside My
files. The simulated daemons also read their config.json from a scratch
XDG directory instead of the developer's own: an adopted device of the
real account made every sync folder fail to register against the fake
Drive.
The daemon has kept recursive folder pins for a long time, but the app
offered "Available offline" for files only. The context menu, the
details pane and the selection bar now offer it for folders too, and a
pinned folder carries the offline badge in the browser and the launcher.
A folder pin used to download the whole subtree inside the control
request, so a large folder outlasted the client's 120 s read timeout,
and every pinned file was read into memory before it was cached. The
daemon now records the pin, answers at once and downloads on a
background thread as a job the tray and the app show, stopping when the
folder is unpinned or the daemon stops. Pins of files and folders both
go through fetch_content, which streams to disk and skips fresh blobs.
A folder pin is refused while offline instead of looking kept.
A file kept only because a folder above it is pinned has no pin of its
own, and unpinning it silently did nothing. ListDir now reports
pinned_by_folder; the app shows such an entry as available offline with
the toggle greyed out, and the daemon refuses the unpin with a message
naming the folder. Unpinning a folder no longer evicts files that are
pinned on their own.
Requested on Reddit.
A package upgrade replaces the binaries but does not restart the user
service, so the daemon keeps the old release and its bugs until the next
login. On 3.2.0 the reporter of #33 still saw "Couldn't sync Documents:
read .../snapshots/37: Permission denied", which only a daemon older than
3.2.0 can produce: since then the scan leaves an unreadable subfolder out
and only an unreadable sync root fails the pass (B206).
The daemon now reports its release in Status; one from before the field
omits it, which reads as older. When the daemon is older than the
front-end, the tray says "Update not finished - restart Proton Drive" and
offers a restart, the sync card shows "Restart to finish updating" with a
Restart button, and pdfs status and pdfs diagnose print the systemctl
command. A newer daemon is not flagged, since then the front-end is the
stale one.
Refs #33
Every SDK error that was not an API reply was classified as Offline.
Only a transport failure is. An upload on an account without a photo
library fails in the SDK before any request, and the app said "You're
offline" while every other call worked (issue #38).
Classify by variant: Transport is Offline, InvalidOperation is Invalid,
Serialization is Remote, Crypto and KeysLocked are Internal. The daemon
logs a failed photo upload, and the Photos page greys out Upload when
the account has no photo library.
docs/BUGS.md B205.
A partial write whose base was not in the content cache queued an
incomplete revision, and every write open over it was refused with EIO
until the drain uploaded it. CI's one-client-lan simulation hit this on
seed 1: a create over a file whose partial edit had not drained yet.
The write open now copies the blob and fills its gaps from the remote
base with Core::fill_gaps, as the drain does, and fails only when that
fill fails (B204).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EuyxByBpK3fSWtyPkugeHz
A rotation the keyring refused was retried and kept in the kernel
keyring, but only the log said so, and a reboot before the keyring took
it still signed the user out.
`Response::Status` carries `tokens_unsaved` while one waits. The tray
says "Sign-in not saved — unlock your keyring" and needs attention, the
app's status says the same, `pdfs status` prints a Keyring line and
`pdfs diagnose` warns. New strings are translated in every catalog
(B61).
`pdfs unlock` resumes the daemon's session and may refresh it. The
daemon then holds a spent refresh token, and since B202 a refusal of it
read as an expired session although the keyring held a live one.
Before calling a refused session expired, compare the keyring with what
this process last stored. When it holds a newer rotation or another
login, end the mount with `MountOutcome::ResumeStored` and mount again
from the keyring. A rotation another process already wrote counts as
this session's, so the daemon keeps writing its later ones (B203).
A refresh token Proton refused, or a deleted or disabled account, was
treated as a lost connection: the mount came up "offline", the probe
retried forever and logged only at debug, and nothing told the user that
only signing in again helps (issue #35).
session_revoked() names those errors. The root fetch, probe, drain,
event poll, event seed and photos root now set session_expired, log one
warning naming `pdfs login`, and stop retrying. Cached files stay
readable and changes stay queued.
Response::Status carries session_expired (serde default false). The
tray shows "Session expired — sign in again" with a Sign In Again…
item that opens the app's sign-in page; the app's status and Sync card
say the same with a button. `pdfs status` and `pdfs diagnose` report it.
New strings are translated in every catalog.
docs/BUGS.md B202.
Offline or paused, a drain worker claims nothing and then slept until the
earliest queued op fell due. Every queued op was already due and nothing
attempted it, so the wait was zero and all 16 workers spun on the queue
for the whole outage (issue #35: 17 min of CPU in 2 min 16 s).
A worker that claims nothing now sleeps the full idle poll; a reconnect
or resume wakes it. earliest_due_at also uses the claim's exclusions, so
an op held behind a claimed op on the same node no longer counts as due.
docs/BUGS.md B200.
Stop Sharing removes the link, invitations and members but cannot delete
the share itself, and Drive keeps listing the empty share until its
backend removes it, minutes later.
The daemon now leaves a share with no member, invitation or link out of
the listing (unless a lookup failed, so a live share is never hidden),
and the app drops stopped items from the page as soon as the requests
succeed instead of waiting for the slow reload.
Every mirror pass, the two-minute poll included, reported a "Checking
{folder}" job, and the tray took any job for syncing, so its icon spun
every two minutes over folders with nothing to do.
JobItem gains a background flag (absent from older daemons, so false)
for housekeeping that changes nothing in the user's files: a pass still
scanning, the home index rebuild and the cache integrity check. The tray
keeps them in its menu line but only turns busy for transfers or a job
that is not background; a pass that queues work becomes a Syncing job
and still spins the icon.
The watcher also reconciles only the folder an event lies in, instead of
walking every mirror folder over one edit.
An unreadable subfolder, such as a container's root-owned cache, failed
every pass of its synced folder, so nothing else in the tree reached
Drive (#33, B196). The scan now leaves such a path out, together with an
entry that vanished mid-scan and a name that is not UTF-8, and the pass
drops everything under it from classification, the wipe guard and the
push pass's deletions. It is never read as deleted and never downloaded
into. An unreadable or absent root still fails the pass. The activity
feed names unreadable paths once each time the set changes.
Two related bugs on the way to online-only:
- The push pass read deletions from the raw baseline, so a synced path
that later became ignored was trashed on Drive (B197).
- The switch evicted everything once the folder was idle, including
ignored paths such as .git/, symlinks, unreadable folders and files
changed after the pass, none of which are on Drive (B198). It now
refuses and names the path.
Closes#33
A user switching from a Windows PC to Linux had two half answers:
adopting the old device makes this machine back up as it, and restoring
its folders keeps them in the old device's tree. Neither puts the data
under this machine's own device.
Device roots are separate shares on the main volume, so each top-level
folder of the other device can move under this device's root in one
server-side move. Nothing is uploaded again, and the other device stays
registered, without those folders.
- core: `Request::MigrateDevice { uid }` and
`Db::sync_folder_set_share`
- fuse: `Core::migrate_device` moves every folder, suffixes a name
already taken here with the other device's name, keeps going past a
failed folder, and re-points sync folders that already tracked a
moved folder at the new share
- cli: `pdfs devices migrate UID [--yes]` confirms, moves, then runs
the ordinary restore picker
- gui: "Move to This Computer…" on another computer's row, followed by
the restore picker
- i18n: translate the new strings in every catalog
- docs: CLI.md, USER_GUIDE.md, RECOVERY.md, ARCHITECTURE.md, changelog
The watchdog is armed when the process starts, but only the mount's supervisor
pinged it. A daemon waiting for a login or backing off after a failed mount
(#26: key unlock failing) was aborted after WatchdogSec and left a coredump
instead of its error. Sleep through idle_sleep in those stretches; the mounted
ping still has to pass the control-socket probe.
66 comments cited `bugs.md B<n>`, the ledger's old name; they now say
`docs/BUGS.md B<n>`, and every cited number has an entry. About 90 more
cited offline.md, devices.md, features.md, mount-architecture.md,
plan.md and improvements.md, planning notes that were never in the
repository. Those citations are removed; the comments keep their text.
ARCHITECTURE.md described the 2.x write path, where namespace changes
waited for Drive and every mount had its own inode space. Rewrite it
around the local tree (lid, inode = lid + 1, local~<lid> stand-ins), the
queue and its 16 drain workers, sync issues, and the remaining split
between the mount and the mirror engine. Drop the pins.json claim; pins
moved into the database at schema v5.
TESTING.md gains the simulation runs and loses the per-case narrative
that --list and BUGS.md already carry. ROADMAP.md takes over the
post-3.0.0 items from the milestone status file.
Remove MILESTONE-3.0.0-STATUS.md (a working log) and UI_UX_PLAN.md (the
2.x design audit). Code comments and CI that cited the untracked
docs/MILESTONE-3.0.0.md now point at TESTING.md, ARCHITECTURE.md and
ROADMAP.md. Threat-model links follow the section to §10.
The confirmation run on 812eb3d (PDFS_SIM_KNOWN=fail, twelve seeds)
passed every profile, so no open bug is tolerated any more: the call,
hang and damage matchers for B125 to B134 go, and any of them now fails
the seed. B126, B128 and B133 were fixed with B138, B162 and B137;
B134 stays open with local_first off only, which the runs do not mount.
nvrl
Claude