The mount only ever went from offline to online: nothing marked it
offline again, so after the Wi-Fi dropped every create, mkdir, delete and
rename called Drive, waited out the SDK's retries for up to two minutes
and answered EIO. Background loops kept polling the dead link.
Connectivity now lives in pdfs-fuse's new link module and goes both
ways. A call that gets no answer (a transport error or a request
timeout) marks the mount offline; a probe thread checks every 2 s,
backing off to 30 s, and on recovery re-lists folders served from the
database, retries failed operations and wakes the drain. Interactive
calls get a 20 s deadline, so a dead link costs one slow call instead of
many.
While offline:
- create, mkdir, unlink, rmdir and rename fall back to the offline
queue instead of answering EIO;
- folders list from the database even when not marked listed, and are
re-listed once the link returns;
- reads of uncached data wait up to 30 s for the link, then retry once
within the 120 s read deadline;
- statfs serves the last quota instead of fetching, and quota refreshes
are single-flight;
- the event and photo event loops wait for the link instead of polling.
A create whose request timed out may have landed anyway. When the drain
replays it and finds a same-named twin of the same kind, created within
the last two minutes and still empty, it adopts that node instead of
making a conflict copy. The drain also leaves alone a name still held
by a queued trash.
Fixes from the account acceptance runs:
- B104: a reader open across a rename-over keeps working; the replaced
node is unlinked like any other open file instead of forgotten.
- B105: a drain worker holding a revision withdrawn by an unlink no
longer uploads it as a recovered-… conflict copy (Db::op_exists).
- B106: an open victim's queued ops are withdrawn at trash time, so a
repeated editor save no longer fails with AlreadyExists; the staged
blob goes with the last close.
- B107: a remote read gives up after 120 s with EIO instead of hanging
its caller; the timer is built inside the runtime.
- B108: a mirror upload whose local file was deleted after the scan is
skipped rather than counted as an error.
The acceptance script reports the failing line for non-assert errors,
ignores queue ops that predate the run, purges with delete-forever
first, and diagnoses a malformed SQLite database against a local replay.
The FUSE acceptance suite now runs on the signed-in account by default and
cleans up after itself; its first live runs found three daemon bugs.
- B101: a pwrite past EOF now authors the hole it leaves, so truncated bytes
no longer come back from the base. A write attached to a create while that
create uploads becomes a revision of the real uid instead of being dropped.
- B102: removing a folder drops the queued revisions of files below it, found
through the node tree, with their blobs and in-flight uploads. `pdfs rm`
now drops queued ops too.
- B103: each server call of a trash refresh times out after 120 s, so one hung
call can no longer freeze the trash listing until restart.
Acceptance suite: account mode with a manifest, lock and reaper; cleanup by
uid; queued ops naming a test root count as leftovers; new edge-case,
throughput and B100/B101 cases; the wide-directory case is split from the
deep tree and gets a doubled budget. Adds `pdfs trash --json`.
A move request with two absolute paths is resolved against every
location (My files, on-demand and mirror synced folders) and made as a
server-side move_node. The local side is levelled afterwards: mounts
forget and re-list, a mirror folder renames its copy and baseline along
or drops the copy once Drive holds it.
Anything only this disk holds is refused up front: queued ops, open
writes, unsynced or ignored mirror files and symlinks. The GUI Move
dialog gains a top-level list of locations.
The list and grid view from My files moves into widgets/file_list.rs as
FileList, so Trash, Shared with me and Shared by me share its columns,
selection, context menus, keyboard handling and list/grid switch instead of
each drawing its own rows.
Trash shows where each item came from and when it was deleted. Proton does
not expose the trash time through the SDK yet, so the daemon records when it
first sees a node in the trash (migration V35, trash_seen) and reports that;
the date becomes exact once proton-sdk-rs exposes Node::trash_time. Restoring
offers a Show action, and opening a trashed item explains that it needs to be
restored first.
Shared with me splits files and saved links into two views. Files show who
shared them, with a warning pill for unverified senders, the access they give
and when they were shared. Pending invitations sit in a banner that opens a
dialog to accept or decline each one. Several items can be left at once.
Saved links get their own list with open, copy and remove actions.
Shared by me gains a filter (all, links, people, expired links) and columns
for location, people and link state, with a copy button for live links.
Several items can stop being shared at once, the page refreshes after the
share dialog closes, and a reload rebinds only the rows that changed so the
selection survives.
All catalogs are updated and complete.
A 64-bit difference hash of each thumbnail, stored in the new
photos.similar_hash column (schema 34), and the queries the similar-photo
finder needs: what is left to hash, progress, and the sets of shots that
look alike. Sets made only of byte-identical copies are left to the
duplicate finder. A refresh keeps a hash unless the photo's bytes changed.
The daemon pass, the control request and the GUI are not built yet; see
PLAN.md.
- The Proton accent preference becomes a whole Proton theme that also
holds against a GTK theme imported from gtk.css; folders get Proton's
purple folder icon while it is on.
- Places map markers show one of each town's photos with a count, and
zoomed in on a town every photo sits where it was taken
(Request::PlaceLocations). Photos taken on one spot are listed in a
popover. "Show on map" in the photo viewer goes to the Places map.
- The built-in map draws country borders, sea and land in their own
colours.
- An opt-in street map from OpenFreeMap (vector tiles drawn with cairo,
cached on disk up to 256 MB), with the OpenStreetMap credit on the map
and in About.
- Translations for all catalogs; changelog folded into 2.3.0.
Photos now know where they were taken. proton-drive-rs 0.7.0 reads the
Location from a photo's extended attributes, and the timeline refresh stores
it with the photo (schema v33). The migration marks every photo unresolved
once, so the next refresh backfills locations from node metadata without
downloading anything.
Towns are looked up offline: pdfs-core bundles GeoNames' cities15000 table,
trimmed by scripts/geonames-cities.py, and files each photo under the
nearest town within 50 km. The new PhotoPlaces and PlacePhotos requests list
the towns and page one town's photos.
The Photos page gets a third view, Places, with a card per town (newest
photo, town, country, count). A place opens in the gallery like an album.
Country names come translated from the iso-codes catalogs, now an optional
dependency. The About dialog credits GeoNames (CC BY 4.0).
A "Find duplicates" button in the Photos header opens a review of every
set of files that share a content hash. Each set shows its copies side by
side with a Keep toggle; the copy in the most albums, then in the larger
RAW+JPEG group, then the oldest, is kept by default, and a set can't lose
its last kept copy. The copies not kept move to Trash one file at a time.
- core: Db::photo_duplicates, Request::PhotoDuplicates, and a files_only
flag on TrashNodes (defaults to false, so old clients still trash whole
groups).
- core: deleting a group's representative now clears the group key of its
remaining files instead of leaving them pointing at a missing uid.
- fuse: handlers for both requests.
Add a PhotosOnThisDay control request, served from the stored timeline:
photos whose local capture date matches today's in earlier years, one
row per group. The GUI shows a card per year ("3 years ago") with the
year's newest photo as its cover; clicking a card scrolls the timeline
to that day. The strip is revealed only at the top of the whole,
unfiltered timeline, so it gives the grid its room back once you
scroll or filter.
A new pill next to the favorites filter narrows the timeline to the photos
no album of ours holds yet, so what is left to file is easy to find. A
RAW+JPEG pair counts as filed when any of its files is in an album.
PhotosTimeline gains an optional not_in_album flag; older front-ends omit it
and get the whole timeline. Album contents are fetched lazily, so the first
filtered page fetches the contents of any album not yet listed. Photos added
to an album while the filter is on leave the timeline at once.
The conflict sweep flagged divergent copies anywhere in the tree but kept its
dedupe set in memory, so every daemon start logged them again. The conflict
listing only covered My Files, so copies in on-demand sync folders were never
listed, and the Activity page showed them as resolved while they were still on disk.
List copies in synced folders by their absolute local path, route resolves
to the mount that serves them, flag only what the listing can show, and
consult the activity log before flagging a copy again.
Videos without a server thumbnail fell through to the local fallback,
which downloaded the whole file, failed to decode it as an image and
wrote the video off for good. Grab one frame with ffmpeg instead, a
second in or from the start for shorter clips. Probe for ffmpeg before
downloading, bound every run, and skip videos over 256 MiB.
Migration V32 gives the videos written off so far another try.
Those videos had no server thumbnail because our uploads never sent one.
Every upload path (bulk upload, drained mount writes, conflict copies,
new revisions and folder sync) now attaches a thumbnail rendered from
the local file, kept under the 60 KiB Proton's own clients use. Files
that are not images or videos, or cannot be decoded, upload bare as
before; a .ts file only counts as video with an MPEG-TS sync byte.
ffmpeg is a new optional dependency in the PKGBUILDs.
Request::Search takes an optional folder scope (serde default, so older
front-ends keep searching everywhere). The GUI offers a This folder toggle
beside the search box and shows a Location column for hits; the CLI gains
pdfs search --in. Pins report whether a pinned file is cached, so pin-derived
search hits badge correctly.
User-visible strings, doc comments and docs now say "Favorite" and
"color", matching the `pdfs favorite` command; UI_UX_PLAN.md §8 records
the choice. The changelog's 1.11.x and Unreleased entries said schema 30,
but V31 (`photos.resolved_at`) shipped in 1.11.0. Adds the missing
release links for 1.10.0 to 1.11.1.
Backend (docs/UI_UX_PLAN.md B-1, B-2):
- Request::SetSyncPaused { paused, until } pauses the drain worker and the
mirror reconcile. FUSE reads, write staging, the watcher and the poll keep
running. The pause is persisted in the `state` table (`sync_paused_until`),
so it survives a restart; a timed pause ends by itself through a timer
thread, re-armed on mount. Resuming wakes the drain and runs a full
reconcile. Response::Status gains `paused` and `paused_until`.
- Request::ListPendingOps lists the queue with paths, attempts, last error,
next attempt and parked/failing flags. Request::RetryPendingOp { id } makes
one backed-off op (or every failed one) due now. Parked ops are never
un-parked and nothing is dropped from the queue.
Front ends:
- Sync page: a status card with Pause/Resume (split button with 1 h, 8 h and
24 h pauses) and a "Waiting to Upload" list with Retry now and Retry All.
- The sidebar strip and the tray show the paused state; the tray gains Pause
Sync / Resume Sync and names failing changes.
- CLI: `pdfs sync pause [--for 30m|1h|2d]`, `pdfs sync resume`,
`pdfs sync queue`, `pdfs sync retry [id]`; `pdfs status` reports a pause.
Not in this phase: merging Activity into the Sync page, discarding a queued
op (needs a recovery copy first), and per-folder pause.
Four faults found by reading the journal for issues that were never fixed,
tracked as B94-B97 in docs/BUGS.md.
A create parked behind a temp-file rename (B70) had no exit but that rename,
so 43 uploads sat in the queue for weeks holding their staged bytes. The
primary drain worker now sweeps parked creates while the queue is idle: one
whose node is gone is discarded, and one older than PARK_EXPIRY_MS that nobody
holds open is un-parked rather than dropped, since staged bytes are the only
copy of the write. An open node stays parked however old it is.
.tmp and .temp leave TRANSIENT_SUFFIXES. The list served two mechanisms with
different consequences: ignoring a name during mirror reconcile is reversible,
parking a create is not, and a Takeout index.tmp is a final name that nothing
will ever rename.
Staged blobs that match no queued op and name no node were kept forever and
warned about once per blob per mount (95 blobs, ~445 MiB, oldest two months).
Blobs past STAGING_ORPHAN_RETAIN are retired; the rest are one aggregated
warning.
Refresh control requests now run one per scope. Every scope is individually
cheap, but duplicates contend for the database lock the running refresh holds,
which left five in flight for twenty-three minutes. request_kind names the
scope so the report says which view wedged. The photo metadata pass also stops
at shutdown instead of running into the cancelled runtime and logging one
"skipping undecryptable photo" per node.
A timeline refresh resolved every photo's node to learn its name, media
type, favourite tag, content hash and photo relation — 63 round-trips for
a 12,445-photo library, minutes of work, repeated every time the timeline
went stale sixty seconds later. None of that metadata changes on its own,
and when it does change the event feed names the node it changed.
`photos.resolved_at` (schema v31) records that a photo's node was read.
The refresh resolves the photos without that mark: the ones it has never
read, and the ones a remote event marked stale. An existing timeline is
backfilled as resolved, so upgrading does not re-read the library once.
Remote changes now name what they changed instead of asking for
everything. `classify_photo_events` turns a `NodeUpdated` into
`photos_mark_unresolved(uid)`, so a favourite toggled on a phone costs one
node read rather than the whole library; a local toggle is claimed as our
own echo, because the feed replays it and a `NodeUpdated` carries nothing
that says what changed. A continuity loss and `refresh photos --full` drop
every mark, which is the way back to a whole-library read.
Along the way: `photos_replace` never kept a photo's name, so a photo the
SDK could not decrypt was renamed to nothing on every refresh — and the
tab split and the RAW+JPEG grouping are both derived from the name. It is
kept now, and a row that *was* resolved is authoritative instead, so a
photo unlinked from its main elsewhere finally loses its `main_uid`.
`TIMELINE_ENRICH_CHUNK` is 150, matching the SDK's own `MAX_BATCH_COUNT`:
a chunk of 200 was split there into a full request and a 50-uid tail.
Three things the 12,445-photo library showed that the tests did not.
A Pixel raw capture is two files named after the role each plays —
`PXL_….RAW-01.COVER.jpg` and `PXL_….RAW-02.ORIGINAL.dng` — so their stems differ
and the pairing rule never saw one shot. The marker is now cut off before names
are compared, narrowly: digits, then one role word, so an ordinary name that
happens to contain `.raw-` keeps its stem. Fed the real library's first 40 rows
through the grouping, they collapse to 21 tiles.
A raw that has a JPEG beside it is now painted with that JPEG's thumbnail, which
the server usually already holds. The Raw tab costs no raw download and no raw
decode for those tiles. Raws are found in one batched read, so an ordinary page
pays nothing for the lookup.
Photos could not be moved to Trash at all: they live on the photo volume, which
the mount never holds, so no `nodes` row exists, every authority answered
`Unknown`, and the write check refused the whole library with "trash access: not
allowed". Membership of our own timeline is the authority that does exist for a
photo, and it now answers. An album photo on someone else's volume still fails
closed.
Refreshing the gallery then reported `io: Resource temporarily unavailable (os
error 11)` (B92). The daemon held the control reply until the whole timeline had
been re-read — 63 batches of 200 nodes for this library, minutes of work against
a 120 s read bound. The daemon now answers as soon as the refresh is running,
`PhotosRefreshStatus` reports when it is done, and both front ends follow it: the
gallery reloads the page when it clears, and `pdfs refresh photos` blocks so the
command still means what it says. Each refresh logs its photo count, batch count
and duration.
Released as 1.11.0.
A database written before the grouping column opens with `group_key` NULL on
every row, and NULL matches nothing: the join dropped the whole timeline until
the first refresh ran. `COALESCE(group_key, uid)` is what makes such a row its
own group, which is how the gallery behaved before the column existed.
Covered by a migration test that rewinds a fixture to v29.
A camera that writes both puts two files on the volume, and the timeline showed
two tiles of the same photo. They are one shot to the person who took it.
Schema v30 records what ties them together: the server's own duplicate hash and
main-photo relation, plus a `group_key` every member of a shot carries. The key
is computed when the timeline is replaced, in three steps — the server relation
first, because it knows about live photos and bursts and we do not have to guess
at them; then same capture day, same file-name stem and one member raw while the
other is not, which is the pair a camera writes; otherwise the photo is its own
group. The member the grid shows is the non-raw one, since a JPEG decodes in
milliseconds and is what the person expects to see.
The gallery therefore pages by group and carries `group_size` and `has_raw` on
each entry, and the date scrubber counts what the page shows. The Raw tab is the
exception and still lists every raw file, because that tab exists to answer
"which raws do I have". Album pages are not grouped at all: an album is a list
someone made, and it shows exactly what they put in it.
`TimelineRow` is a struct now. It had grown to five positional fields and was
about to grow to seven, at which point nobody can read a call site.
The gallery could show photos and nothing else: a photo deleted on the phone
stayed on the page for minutes, and a photo you wanted gone had to be deleted
on another device. Both are now handled here.
Deleting needs a uid-addressed request, because `Request::Delete` takes a
mountpoint path and the photos volume is not in the FUSE mount. `TrashNodes`
trashes through the SDK and answers per-uid outcomes, so a partial failure is
reportable; the daemon then drops those rows with `Db::photos_delete`, album
membership included, rather than leaving the gallery to wait for the next
timeline refresh. The GUI removes the tiles immediately with an Undo toast and
puts back anything the server refused — the photos are in Proton's trash and
stay recoverable there either way. Selection is Ctrl/Shift+click, the Select
toggle or Delete in the lightbox.
Remote deletions arrive on their own now. The Drive volume and the photos
volume have separate event streams, so the photos volume gets its own poller
and its own cursor. Trash and delete events remove rows; anything else only
marks the timeline stale, because there is no inode space to converge on this
volume. With the timeline TTL down to 60 s and an explicit Refresh that waits
for the server instead of only clearing the freshness stamp, a photo deleted
elsewhere leaves the grid in about ten seconds.
The trash is a flat list of uids, but a delete has a shape: a folder, and
often items inside it that were trashed separately beforehand. Restoring
one of those in isolation is not what it means — a folder came back
without the contents deleted with it, and a file came back inside a
folder that is still trashed, which is to say nowhere the user can see.
- Schema 29: `trash.parent_uid`, filled by the trash refresh from the
node's parent, plus an index for the walk.
- `expand_restore` takes the connected piece of the trashed tree around
what was asked for (every trashed descendant, every trashed ancestor)
and orders it into waves, shallowest first: the server cannot put a
node back under a parent that is still trashed. A uid the listing does
not know about is restored on its own, because the listing is
materialised in chunks and can be stale.
- `Core::restore` restores wave by wave, still streaming per-node
outcomes, so one node the server refuses leaves the rest restored.
The SDK bump brings the 6cbf2f44 port: permanent API errors are no
longer retried on block transfers, expired block URLs are re-minted on
the whole-file and public-link download paths, and a server-supplied
Retry-After is capped at 120 s.
- A queued rename that also changes folder now lands as one `move-multiple`
request carrying the target name instead of rename-then-move, so there is
no half-applied state and no second call for stale name requirements to
hit (B46 follow-up). A destination name collision moves under a conflict
name in one step; same-folder renames still use the rename endpoint.
- Shared with me shows the inviter and invite date (GUI subtitle, `pdfs
shared-with-me`), with a warning when the invitation signature does not
verify against the claimed inviter's keys. `DirEntry` gains `shared_by`,
`shared_at` and `shared_by_unverified`, all serde-defaulted.
- Bump `proton-sdk` / `proton-drive-rs` to 0.6.3 (anonymous-node moves,
`TrashTime` trashed detection).
B69 — spurious `(sync-conflict)` copies came from `(mtime, size)` being used as a
revision identity. The server re-stamps a sealed revision with its own commit
time, so the drain diverted a write into a conflict copy of its own base; and
`plan_file` classified a no-baseline (local, remote) pair as `Conflict`, forking
every pre-existing file on first reconcile.
- `Baseline` carries `revision_id` (from the SDK's new `active_revision_id`),
captured at write-open and preserved across supersede/rebaseline.
`revision_changed` trusts the id and only falls back to `(mtime, size)` for
old sidecars.
- Planner gains `FilePlan::AdoptBaseline`: equal-size no-baseline pairs are
recorded, not conflicted. Divergent sizes stay a real conflict.
- New `sweep.rs` (`pdfs-conflict-sweep`, 5-min cadence) removes conflict copies
proven identical by size + `content_sha1` and surfaces divergent ones once as
an `ActivityKind::Conflict` entry. Never removes what it cannot prove.
B70 — browsers downloading into an on-demand mount had their in-flight temps
sealed as canonical revisions, exiling the finished file to a conflict copy.
`syncignore::is_transient_name` now recognises download/scratch temps, and the
on-demand `release`/`rename` paths hold them (`set_create_hold`) instead of
sealing until the name is finalized.
Workspace deps bumped to proton-sdk/proton-drive-rs 0.2.2 (registry, no patch).
Live validation against a real account still pending for both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8P42asymU158CPnhjRwri
- Durable scratch/staging publication and preservation on failure.
- Transactional pending-operation supersession.
- Conflict copies cannot overwrite older copies or lose the source.
- Single-entry wipe protection and incomplete-scan aborts.
- Failed local deletion retains its baseline.
- Downloads are fsynced before baseline settlement.
- Future database schemas are refused.
- Event cursors advance only after successful invalidation and persistence.
- IPC frame, timeout, and concurrency limits.
- Atomic config saves; malformed configs are preserved.
- Sensitive directories require correct ownership, real directories, and 0700.
- Debian package now includes service and tray autostart files.
- Release workflow enforces versions, tests, lint, formatting, and offline acceptance.
- CI now runs offline filesystem acceptance.
- Standardize POSIX unlink (EISDIR) and rmdir (ENOTDIR/ENOTEMPTY) checks
- Defer local directory eviction in mode switch until FUSE mount verification succeeds
- Enforce fsync before atomic renames in ContentCache store operations
- Add directory ancestor loop check in rename (EINVAL)
- Implement punch_hole interval tracking in fallocate
- Purge un-created local node write revisions on unlink
- Fix control socket permission race with octal umask 0o77
- Add unit tests for state, DB, and FUSE POSIX logic (208/208 tests passing)
Verified against a live install: vacuum reported "34.3 MiB freed (170.3 MiB
-> 136.0 MiB), 0 WAL frame(s) checkpointed" while the WAL grew from 67 MiB to
143 MiB. Net disk use barely moved, and the frame count was zero because the
checkpoint ran before the VACUUM, when there was nothing to fold.
Reordering is strictly better, but the commit message should not overstate it:
the live inflation happens because the daemon's other connections hold read
transactions and a TRUNCATE checkpoint waits on readers, so under a busy
daemon this can still move little. journal_size_limit stays the backstop --
the live WAL did fall back to its 64 MiB cap on its own afterwards.
The accompanying test deliberately does not assert on WAL size or frame count.
Measured both ways, a single-connection test reports frames=0 and a 0-byte WAL
under either ordering, because SQLite folds the log itself without a
concurrent reader to block it; an assertion there would pass on the bug and be
decoration. It asserts what is reliably true instead -- that a file-backed
vacuum reclaims space and leaves the database sound -- and the ordering
rationale plus the residual uncertainty are documented on vacuum() itself.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFzNQyb4TWrCRWcnegqWxY
nvrl
