Every release sent the fixed `external-drive-linux@0.1.0-alpha`. If Proton
ever retires that version, every release fails sign-in at once and no update
can fix it.
- core: APP_VERSION and USER_AGENT follow CARGO_PKG_VERSION
(`external-drive-linux@<version>-stable`, `proton-drive-linux/<version>`)
- core: `app_version` and `user_agent` in config.json are optional overrides;
the fixed values older releases stored there read as unset and leave the
file on the next save, a hand-set value is kept
- gui: the About dialog shows the release, its debug info the identity
actually sent
- docs: describe the new default and the migration
A new key grants its owner only "view"; read needs possession, which a
process has only when its session keyring links to the user keyring. A
systemd user service or a container does not guarantee that, so the
daemon could store the unlock and then fail to read it back (EACCES).
Set the permissions explicitly after add_key.
A two-password account signed in with its login password as the mailbox
password, so the key chain never unlocked and the daemon retried forever.
Login now asks for the mailbox password and checks it against the account
(proton-sdk 0.8 `unlock`), so a typo is a retry. The password is never
stored: the keyring blob keeps the per-key passphrases instead, and for a
two-password account only when the user opts in (`--remember`, or the
"Keep unlocked on this computer" switch). By default they live in the
kernel keyring for the login session and the account is locked after a
reboot until `pdfs unlock` or the app supplies them again.
- core: kernelkey module, Error::Locked / WrongMailboxPassword, unlock,
lock, key_state, ready; blobs written by 3.0.x are migrated on first
start; token refresh updates only the tokens in the stored blob
- cli: `pdfs unlock [--remember]`, `pdfs lock [--forget]`; status and
diagnose report the key state; the daemon waits while locked and is
marked non-dumpable
- gui: mailbox password dialog at sign-in and on open, locked tray state
- po: all catalogs translated
- bump to 3.1.0, depend on proton-sdk and proton-drive-rs 0.8.0
Refs #26.
The watchdog is armed when the process starts, but only the mount's supervisor
pinged it. A daemon waiting for a login or backing off after a failed mount
(#26: key unlock failing) was aborted after WatchdogSec and left a coredump
instead of its error. Sleep through idle_sleep in those stretches; the mounted
ping still has to pass the control-socket probe.
Manual workflow that builds the RPM and the bare binaries in a Fedora
container, so a user can try a branch against Fedora's own GTK stack.
Uploads them as an artifact and, on request, as one rolling pre-release
per branch that needs no GitHub login to download.
DEVELOPMENT.md lists the daemon's current modules, asks for the
simulation runs on FUSE, queue and sync changes, and adds how to handle
a bug fix and a user-visible change, including the six pages the wiki
publisher copies. The release checklist moves to TESTING.md.
66 comments cited `bugs.md B<n>`, the ledger's old name; they now say
`docs/BUGS.md B<n>`, and every cited number has an entry. About 90 more
cited offline.md, devices.md, features.md, mount-architecture.md,
plan.md and improvements.md, planning notes that were never in the
repository. Those citations are removed; the comments keep their text.
The throughput case said every create and unlink on a Drive mount waits
for Drive. Since 3.0.0 that holds only with local_first off; the floor
of 3 operations a second stays for that case.
The ledger ran B193 down to B78, then B1 up to B75, with T1 to T4 in
the middle and a stray top-level heading inside B14. Status lines used a
dozen wordings for the same few states, so finding what is still open
meant reading 6,600 lines.
Add a header: what the numbers promise (permanent, never reused; B76
and B77 were never assigned), the status vocabulary, and how to add an
entry, with a template. Add an index sorted by status, with what is left
for every open or partly fixed entry. Order all entries newest first
under one Entries section, and move the acceptance-runner findings to
their own section. Titles are in sentence case; anchors are unchanged.
Entry text is kept as written, except B89: the file it described,
proton-dev/CLAUDE.md, no longer exists, so it is closed.
The user guide said only writes were queued. Since 3.0.0 creating,
renaming and deleting through the mount return at once too, and changes
Drive refuses become sync issues. Add a section on changes waiting for
Proton Drive: retry, export, discard, and the two cases that land under
another name or place instead of failing.
Troubleshooting covers changes that have not reached other devices yet,
`pdfs sync issues` and the local_first fallback, and names the real
error for symlinks and hard links (EPERM, not EOPNOTSUPP). CONFIGURATION
lists MemoryHigh and the test variables; CLI notes that `sync export`
takes a folder.
ARCHITECTURE.md described the 2.x write path, where namespace changes
waited for Drive and every mount had its own inode space. Rewrite it
around the local tree (lid, inode = lid + 1, local~<lid> stand-ins), the
queue and its 16 drain workers, sync issues, and the remaining split
between the mount and the mirror engine. Drop the pins.json claim; pins
moved into the database at schema v5.
TESTING.md gains the simulation runs and loses the per-case narrative
that --list and BUGS.md already carry. ROADMAP.md takes over the
post-3.0.0 items from the milestone status file.
Remove MILESTONE-3.0.0-STATUS.md (a working log) and UI_UX_PLAN.md (the
2.x design audit). Code comments and CI that cited the untracked
docs/MILESTONE-3.0.0.md now point at TESTING.md, ARCHITECTURE.md and
ROADMAP.md. Threat-model links follow the section to §10.
The confirmation run on 812eb3d (PDFS_SIM_KNOWN=fail, twelve seeds)
passed every profile, so no open bug is tolerated any more: the call,
hang and damage matchers for B125 to B134 go, and any of them now fails
the seed. B126, B128 and B133 were fixed with B138, B162 and B137;
B134 stays open with local_first off only, which the runs do not mount.
The known-bug confirmation run failed one_client_on_a_flaky_link seeds 6
and 11 on three bugs around a create whose answer was lost:
- B182: after a restart, a write over a file whose create landed but was
never read back forked a conflict copy. A revision of a file that has
no revision of Drive's now adopts the landed node first.
- B183: the SHA-1s a create sent were kept in memory, so after a restart
its trash or retry no longer knew the node Drive made. They are now on
the op's row (pending_op.sent_sha1, schema 42).
- B184: a rename rewrote the op's target, and the trash and the retry
looked for the node under the last name only. Each attempt's parent
and name are recorded (pending_op.sent_to); the trash looks under all
of them, and a retry adopts the node an earlier attempt made and moves
it to the current name.
A refused change was only on the Sync page. A folder listing now carries
each node's issue (by local id, so a file not yet on Drive has it too),
and the browser's badge ranks it first, folders included, with the
issue's explanation as its tooltip. The event sampler publishes Files
when the set of issues changes, so the badge comes and goes with it.
A change Drive will not take could be exported or retried, never let go
of. The Sync page's Discard button (with a confirmation) and
`pdfs sync discard <id>` drop it: a node made here goes, as if deleted
before it was sent; a node Drive has goes back to what Drive has, under
the same inode, with every change queued on it, while what is queued
below it stays. A change on the wire or an open file is refused.
The daemon logs it as "<reason>; local changes uploaded as <copy>" or
"...kept at <path>", which the Activity page showed as written, in
English. It now reads both and the three reasons, one sentence each.
A mirror target's root is plain local storage, so the durability case's
wait for the mount passed at once after the restart. The queue wait then
read `pdfs --json status` before the new daemon listened, took its null
"mount" for an empty queue, and the run's cleanup found no control socket.
A file deleted after its create was sent keeps the op as a trash. The op
kept the create's attempts and backoff, and the drain recorded the failure
of a create attempt still on the wire on the row that was by then the trash.
The trash waited out the create's backoff.
The trash starts due, as a create that becomes a revision does, and the
drain records a failed attempt only if the op is still the kind it ran as
(`Db::record_attempt_failure`).
An upload that landed let go of its blob before the tree had its node. A
write open in between took the base it sampled at its start, the revision
the upload replaced, found none of its bytes cached, and queued an
incomplete revision; the next write open failed with EIO.
The drain keeps the landing node aside until the tree has it, and a write
open with no queued revision takes its base from that node, or from the
tree as it is when the handle goes in. The fake Drive can hold the drain
in that window (`hold_next_landing`).
A stop that came while a mirror pass was uploading let the pass go on to
the next level of folders until the stop gave up on it and the runtime's
end cancelled the uploads under it. Once the daemon is stopping, a batch
starts none of its waiting ops, lets the ones sending finish, and ends
the pass; the remote walk and a new pass stop too.
A delete of a file whose create was on the wire withdraws the create into
a trash. When its folder was removed too and that trash landed first,
Drive refused the create, and the drain re-homed the deleted file in the
root; only its missing bytes stopped it. The withdrawn create's trash
then read the trashed folder back into the cache before finding nothing
to trash.
A withdrawn create is no longer re-homed, and its trash is done once the
folder it was sent to is trashed or gone.
The hydrate rebuilt each listed folder's listing with a pass over every
entry. With 4,600 listed folders and 37,000 entries that took 1.7 s at
every start before the mount came up; one pass grouping the entries by
parent takes 3 ms.
The account run's 0-byte read of a new file was B166 too: Drive's
listing had the node from before its create was committed, and the
queue no longer held it when read. The fake Drive can now hold a
folder's listing on its way back with one file listed unrevised.
A listing from Drive was overlaid with the queue as it was after Drive
answered. A create that landed in between was in neither, and a wide
folder on Wi-Fi dropped files that were never unlinked. The queue is now
read before Drive is asked too; a node that left it meanwhile is listed
as the DB has it.
A stop joined every background thread for as long as each took. The sync
engine's mirror pass calls Drive with no deadline, and a stop that waited for
it was killed by systemd at TimeoutStopSec. The stop now waits up to 10 s for
the background threads and FUSE workers together, names any still running, and
the runtime shuts down within 2 s.
A listing from Drive overlays what is still queued for the folder. A create
Drive has made but whose answer is on its way, or which the drain is still
reading back, was in both: Drive's node under its new uid and the queue's
stand-in, under one name. A node queued in the folder now holds its name there.
A folder removed and made again met the old folder while its trash was on
the wire and adopted it, so the new folder's contents went into the old
one and were trashed with it. The twin search now skips hidden nodes, as
listings do.
The fake Drive can hold a trash (hold_next_trash).
adopt_real_uid asked whether a move was queued only after reading the
node back. A move that landed in between left no op, and the answer still
had the old name, so the mount showed the file under it. Ask first.
The fake Drive can hold a read's answer (hold_answer_to_next_read) so the
test can land the move while the answer is on its way.
Queueing a folder's create left the workers asleep until their 30 s idle
poll; every other enqueue woke them. A file's create still waits for the
release that attaches its bytes.
A lookup in the root publishes the "Shared with me" entry only if the root
listing did not change while its plan was read, and answered EAGAIN when it
had. Right after a restart the root is listed again as the first lookups come
in, so a write could fail with "Resource temporarily unavailable".
ensure_children now takes the listing again, up to four times.
B141 drops a rename that fails with a newer one queued behind it. The
worker records the failure and releases its claim in two steps, and a
rename queued between them waited behind the old one, which was kept
with a backoff. Released, the old one was not due and the new one was,
so the new one landed first and the old one then put the first name
back.
Releasing the claim now drops the old rename too when a newer one is
queued, and both checks match the node by local id as well as by uid.
When the read-back after an upload went unanswered, the drain let go of
the staged blob without caching it. A partial write opened next had
nothing to fill its gaps from and queued an incomplete revision, so the
write open after it was refused with EIO.
The bytes are now cached under the size and mtime the tree still has,
which is the base the next write takes. Their SHA-1 stands in for the
revision id the read-back did not bring, so that write chains onto the
upload and does not fork a conflict copy of itself.
A file or folder made on the mount used to get a minted `local~<ms>-<n>`
placeholder. It now gets `local~<lid>`, a function of the row it is given
first (`Db::add_local_row`), so the stand-in never changes and always leads
back to the row, and through it to the real uid once the node lands.
`Db::lid_of` reads it without a lookup.
Migration V41 rewrites the placeholders still stored to their row's
stand-in: the row and its node_json, its children, the ops on it or into
it (uid, parent, sidecar uid, a rename's original parent), and its pins.
Step 6a of the plan for 3.0.0 phase 3.
B154 is a regression from the B153 fix: after a failure on a held name the
op woke the ops waiting for it, but the database cannot see a create on the
wire, so the op was due again at once and spent its waits in milliseconds.
No wake while another create on the wire holds the name.
B155: a revision upload that reached Drive but lost its answer left the
file at a revision this daemon never heard of, and the retry kept the write
as a conflict copy of itself. A complete write whose bytes the remote holds,
by size and SHA-1, is sent again instead.
The fake Drive gains hold_reply_to_next_create and
lose_reply_to_next_revision for the two new sim tests.
Adoption asks again under the tree lock whether a rename is queued, and
writes the node row through the lock's outbox, so a rename made while
the create landed is no longer undone in the tree and the row.
An op that finds its name held fails as NameHeld: it backs off by the
minimum without doubling, and wakes the waiters for its name again once
its failure is recorded, so a trash that landed in between is not missed.
A retried create now adopts the file its lost answer made when Drive's
SHA-1 matches the staged blob, not only when the file is empty. A delete
after a sent create keeps the op as a trash that finds and trashes the
node that create may have made; a new create of that name waits for it.
The simulation's settle now reads the log, so known damage is explained.
With changes through the mount queued, `pdfs rename` and `pdfs move` went
straight to Drive ahead of a rename the mount still had queued, which then
landed and put its name back: the account run's B121 case on every on-demand
mount. They are now queued whenever the mount would queue, or the node has
anything queued, and rewrite a still-queued create's target.
The B113 acceptance case no longer requires creates to reach Drive at once;
under local-first they are queued and the moves rewrite them.
Schema 40 adds nodes.parent_lid, backfilled from parent_uid. An upsert
and a new row's stub keep it in step, a folder's new row links the rows
stored before it, and a row merged into a landing folder hands its
children over. A restart places each node under its parent's row as it
is now, so a child whose stored node still names a landed folder's
placeholder lands under the folder's real uid.
Schema 39 adds pending_op.lid and parent_lid, backfilled from nodes. Every
op queued or retargeted gets both, and a create gets its row before it
is queued. The claim reads a parent's uid through parent_lid, and a
landing folder rewrites the ops made inside it by local id instead of by
its placeholder string. Ops queued against a row Drive listed follow it
when it is merged into the landing one.
A folder pinned before its mkdir landed kept the placeholder uid in
pins.uid, which named no node once the landing moved the row to the
real uid. The landing now moves the pin with it.
create_folder, trash_nodes, the twin listing and the uploads waited on
Drive without a limit, so one call that never answered held a drain
thread for good. They now give up after INTERACTIVE_CALL_TIMEOUT, and
uploads after that plus their size at 32 KiB/s; a missed deadline reads
as RequestTimeout, so the op backs off like any lost link.
one_client_on_a_slow_link runs the Wi-Fi faults without outages and has
to drain within each settle. a_thousand_files_drain_as_fast_on_wifi_as_on_lan
times 100 folders and 1,000 files on LAN and Wi-Fi; it only prints for
now and runs when PDFS_SIM_MEASURE is set.
ssh-keyscan sometimes returns no key from the runner, which failed the
AUR publish for 2.8.0 and 2.10.0 at the fingerprint check. Retry the
scan up to five times with a longer timeout before checking the pin.
A change Drive refuses because the storage is full, access was taken away,
its folder is gone, a folder limit is reached or the change was rejected
used to look like any other failure until six attempts had passed. The
drain now sorts the refusal into an issue and stores it on the queued op
(schema 38), so it needs attention at once in the app, tray and CLI. The
change stays queued and is retried; nothing is deleted.
The queue listing carries the issue and whether the op has file content.
ExportPendingOp copies that content to a new file: the Sync page offers it
as an Export button next to each issue, and the CLI as
`pdfs sync export <id> <dest>`. `pdfs sync issues` lists what needs
attention, and `pdfs sync queue` shows the issue per change.
docs/MILESTONE-3.0.0-STATUS.md tracks what is left for 3.0.0.
A file renamed away from a.txt while its create was on the wire, and a new
a.txt made after it, could both land on the first file's node: the second
create found a fresh, empty a.txt and took it for its own unanswered create.
The first file's rename then carried the second one's bytes away.
A create now leaves alone a node that another create is making under the
same name, and one the tree already knows. It waits for the name like for
any other queued change (docs/BUGS.md B147).
- B144: when Drive says a name is taken but nothing local holds it any more,
the rename, move or create is sent once more under the wanted name before
falling back to a conflict name. The trash that freed it could land between
Drive's answer and the check.
- B145: the park sweep no longer drops a parked create whose node row is not
written yet. It waits a minute, and keeps open files, before taking a
missing row for a deleted file. The account run's B70 check failed with
EBUSY on this.
- B146: a rename queued while an upload's read-back is on the wire is kept;
the tree no longer takes back the name Drive still had.
A new folder, a new file, a rename or a delete through the mount no longer
waits for Drive: it is recorded locally, the call returns at once, and the
drain sends it as it does offline. `"local_first": false` in config.json
brings back the old path for this release.
Fixes found by the simulation runs on the way:
- B129: a file deleted while its first upload is on the wire is trashed once
the upload lands.
- B132, B140: files made in a new folder survive a restart; a landed folder
hands its queued children over in the stored node too.
- B138: a create waits for the rename or trash that frees its name.
- B139: a file closed while its first upload lands keeps its last write.
- B141: a rename a worker holds stays queued, and the newer one waits behind
it instead of racing it.
- B142: a rename of a file whose create lands meanwhile goes to the uid Drive
gave it instead of answering EBUSY.
- B143: a create whose read-back fails as the link drops is adopted before
the next op for it is sent, so its next write is not a conflict copy.
A plain rowid is the largest one plus one, so a trash queued in the same
transaction that dropped the newest op took its id. The drain retires the
op it sent by id, so a rename landing deleted the trash queued over it and
the file stayed on Drive (B137). Schema 37 rebuilds pending_op with
AUTOINCREMENT, keeping every id and index.
A dump of a database the 2.8.2 build wrote, with queued ops due, parked,
failing, deferred for access and claimed, and creates queued below a
folder made offline. Its migration checks every column of the queue and
the sealed revisions, as the real-database test now does too.
On-demand sync folders kept their own State, which the drain, the
event applier and the sweeps had to visit one by one (B74). Every mount
now serves from the primary's tree. A session keeps only its root, which
the kernel knows as inode 1, and its notification channel; work that
changes the tree tells every mounted session. Size upgrades become
daemon-wide, and moves compare mounts by root instead of by state.
An ignored test takes a copy of the database named by PDFS_MIGRATE_DB
with VACUUM INTO over a read-only connection, migrates it, and checks
that every queued op survives unchanged, every node keeps its row
number as its lid, and no node is lost or doubled.
A node's inode is now derived from its row's local id instead of a
per-run counter, so the same node keeps the same inode across restarts
and across every place that interns it. A lid handed out before the
row is stored reads as no node.
A rename of a node whose create is still queued rewrites the create's
target, but the drain had already copied the op. When the create
landed, the row was deleted unread and the rename was lost (B136).
finish_create now learns where the create was sent and where it landed,
and queues a rename of the real uid when the row has moved on since.
adopt_real_uid keeps the tree's name and folder while that rename is
queued. A landed folder's children also stop naming its placeholder as
their parent in memory, which put the placeholder back on their rows.
First step of phase 2 of docs/MILESTONE-3.0.0.md. Schema 36 rebuilds
`nodes` with `lid INTEGER PRIMARY KEY AUTOINCREMENT`. Each row takes its
old rowid as its lid, so the search index, keyed by rowid, stays valid.
AUTOINCREMENT keeps a deleted row's lid from coming back. `uid` stays
unique and becomes nullable, for nodes that are not on Drive yet.
A landed create's row now takes the real uid in place, keeping its lid.
When a listing had already stored the real node, its content is kept
under the placeholder's lid, which the node was known by first.
Nothing reads the lid yet apart from `Db::lid_of`.
Found by the seeded simulation runs (docs/BUGS.md).
- B122: a rename across folders goes to Drive as one move-multiple
request, so a sibling holding the new name in the old folder no longer
fails it with EIO. The drain uses the same request.
- B123: a second write open on a file just written no longer fails with
EIO. An empty file counts as complete, our own renames, moves and
online creates record their echo, a duplicated event is skipped, a
rename keeps the file's mtime while its revision stays, and a landed
upload is cached before open handles are rebased onto it.
- B124: a node put back after a failed replacing rename leaves `hidden`
and its folder is listed again, so it shows in the mount.
- B131: a landed create is read back with the same short retry as an
online create, and its bytes are cached under the real uid before an
open handle is repointed at it.
- B135: an online rename settles the tree through the state guard, so
the database has both folders unlisted before the reply. A lookup
right after it no longer rebuilds the destination without the file.
The simulation runs no longer tolerate B122 or B131.
Phase 1 of docs/MILESTONE-3.0.0.md, without a change in behaviour.
- Own sealed revisions live only in the own_sealed_rev table. The
in-memory map in front of it is gone, so revision_conflict gives the
same answer before and after a restart.
- A queued rename records the node's original name next to its original
parent, and keeps both across renames that replace the row. The drain
logs when Drive has the node neither where the rename found it nor
where it takes it.
The SDK's NodeUpdated event carries no revision id, so echoes of our own
changes stay recognised by the self-change timer.
- B132: a folder whose mkdir is still queued is gone after a restart.
The root's listing, served offline, was invalidated when the link
came back, and the restart listed it on Drive without the queued
folder.
- B133: a file deleted while its upload is in flight can still come
back as a recovered copy. B105's check goes by op id, and SQLite gives
the next op queued the id the delete freed.
- B134: a file deleted online while the drain checks its write is kept
as a conflict copy. The unlink trashes the file on Drive before it
drops the queued write, so B105's check still finds the op.
nvrl