Compare commits

...
24 Commits
Author SHA1 Message Date
nvrl 6cc8f72bab fix(core,fuse,gui,cli): read only the photos a refresh has not read
A timeline refresh resolved every photo's node to learn its name, media
type, favourite tag, content hash and photo relation — 63 round-trips for
a 12,445-photo library, minutes of work, repeated every time the timeline
went stale sixty seconds later. None of that metadata changes on its own,
and when it does change the event feed names the node it changed.

`photos.resolved_at` (schema v31) records that a photo's node was read.
The refresh resolves the photos without that mark: the ones it has never
read, and the ones a remote event marked stale. An existing timeline is
backfilled as resolved, so upgrading does not re-read the library once.

Remote changes now name what they changed instead of asking for
everything. `classify_photo_events` turns a `NodeUpdated` into
`photos_mark_unresolved(uid)`, so a favourite toggled on a phone costs one
node read rather than the whole library; a local toggle is claimed as our
own echo, because the feed replays it and a `NodeUpdated` carries nothing
that says what changed. A continuity loss and `refresh photos --full` drop
every mark, which is the way back to a whole-library read.

Along the way: `photos_replace` never kept a photo's name, so a photo the
SDK could not decrypt was renamed to nothing on every refresh — and the
tab split and the RAW+JPEG grouping are both derived from the name. It is
kept now, and a row that *was* resolved is authoritative instead, so a
photo unlinked from its main elsewhere finally loses its `main_uid`.

`TIMELINE_ENRICH_CHUNK` is 150, matching the SDK's own `MAX_BATCH_COUNT`:
a chunk of 200 was split there into a full request and a 50-uid tail.
2026-09-19 17:39:49 +02:00
nvrl 376f3edb28 fix(core,fuse,gui,cli): pair a Pixel raw with its JPEG, and stop refresh timing out
Three things the 12,445-photo library showed that the tests did not.

A Pixel raw capture is two files named after the role each plays —
`PXL_….RAW-01.COVER.jpg` and `PXL_….RAW-02.ORIGINAL.dng` — so their stems differ
and the pairing rule never saw one shot. The marker is now cut off before names
are compared, narrowly: digits, then one role word, so an ordinary name that
happens to contain `.raw-` keeps its stem. Fed the real library's first 40 rows
through the grouping, they collapse to 21 tiles.

A raw that has a JPEG beside it is now painted with that JPEG's thumbnail, which
the server usually already holds. The Raw tab costs no raw download and no raw
decode for those tiles. Raws are found in one batched read, so an ordinary page
pays nothing for the lookup.

Photos could not be moved to Trash at all: they live on the photo volume, which
the mount never holds, so no `nodes` row exists, every authority answered
`Unknown`, and the write check refused the whole library with "trash access: not
allowed". Membership of our own timeline is the authority that does exist for a
photo, and it now answers. An album photo on someone else's volume still fails
closed.

Refreshing the gallery then reported `io: Resource temporarily unavailable (os
error 11)` (B92). The daemon held the control reply until the whole timeline had
been re-read — 63 batches of 200 nodes for this library, minutes of work against
a 120 s read bound. The daemon now answers as soon as the refresh is running,
`PhotosRefreshStatus` reports when it is done, and both front ends follow it: the
gallery reloads the page when it clears, and `pdfs refresh photos` blocks so the
command still means what it says. Each refresh logs its photo count, batch count
and duration.

Released as 1.11.0.
2026-09-19 17:08:33 +02:00
nvrl 09c0684a5e fix(core): keep pre-v30 photos on the page until a refresh groups them
A database written before the grouping column opens with `group_key` NULL on
every row, and NULL matches nothing: the join dropped the whole timeline until
the first refresh ran. `COALESCE(group_key, uid)` is what makes such a row its
own group, which is how the gallery behaved before the column existed.

Covered by a migration test that rewinds a fixture to v29.
2026-09-19 15:41:24 +02:00
nvrl c1df3b6cc2 feat(gui): show a grouped shot as one tile, and reach its other files
The grouping landed in the daemon; this is what it looks like. A tile whose shot
was stored as more than one file badges that in the corner — "RAW" when one of
the members is a raw file, since that is the word the person is looking for, and
a plain count otherwise (a live photo, a burst). The lightbox gains a control
that steps through the group's files, which is the only way to reach the RAW now
that the grid shows the shot once.

Deleting says what it will do: the daemon takes the whole group, so the
confirmation names the file count when a selection stands for more files than
tiles.
2026-09-19 15:39:14 +02:00
nvrl afd71713d5 feat(core,fuse): group the RAW and the JPEG of one shot
A camera that writes both puts two files on the volume, and the timeline showed
two tiles of the same photo. They are one shot to the person who took it.

Schema v30 records what ties them together: the server's own duplicate hash and
main-photo relation, plus a `group_key` every member of a shot carries. The key
is computed when the timeline is replaced, in three steps — the server relation
first, because it knows about live photos and bursts and we do not have to guess
at them; then same capture day, same file-name stem and one member raw while the
other is not, which is the pair a camera writes; otherwise the photo is its own
group. The member the grid shows is the non-raw one, since a JPEG decodes in
milliseconds and is what the person expects to see.

The gallery therefore pages by group and carries `group_size` and `has_raw` on
each entry, and the date scrubber counts what the page shows. The Raw tab is the
exception and still lists every raw file, because that tab exists to answer
"which raws do I have". Album pages are not grouped at all: an album is a list
someone made, and it shows exactly what they put in it.

`TimelineRow` is a struct now. It had grown to five positional fields and was
about to grow to seven, at which point nobody can read a call site.
2026-09-19 15:34:17 +02:00
nvrl c6bdba024f feat(gui,fuse): delete photos from the gallery and notice remote deletions
The gallery could show photos and nothing else: a photo deleted on the phone
stayed on the page for minutes, and a photo you wanted gone had to be deleted
on another device. Both are now handled here.

Deleting needs a uid-addressed request, because `Request::Delete` takes a
mountpoint path and the photos volume is not in the FUSE mount. `TrashNodes`
trashes through the SDK and answers per-uid outcomes, so a partial failure is
reportable; the daemon then drops those rows with `Db::photos_delete`, album
membership included, rather than leaving the gallery to wait for the next
timeline refresh. The GUI removes the tiles immediately with an Undo toast and
puts back anything the server refused — the photos are in Proton's trash and
stay recoverable there either way. Selection is Ctrl/Shift+click, the Select
toggle or Delete in the lightbox.

Remote deletions arrive on their own now. The Drive volume and the photos
volume have separate event streams, so the photos volume gets its own poller
and its own cursor. Trash and delete events remove rows; anything else only
marks the timeline stale, because there is no inode space to converge on this
volume. With the timeline TTL down to 60 s and an explicit Refresh that waits
for the server instead of only clearing the freshness stamp, a photo deleted
elsewhere leaves the grid in about ten seconds.
2026-09-19 15:25:04 +02:00
nvrl 5a8e73657c feat(gui): justify the gallery rows and keep the thumbnails on screen
The Photos grid laid every photo out as a centre-cropped square. That is
why portraits showed black bars and why a panorama was unrecognisable at
tile size: the tile's shape had nothing to do with the photo's.

Rows are now justified. Photos are taken in capture order until they no
longer fit the target height, and the row is then scaled to land exactly
on the right margin, so each tile is its own photo's shape and nothing is
cropped or letterboxed. The last row of a day stays at the target height
rather than being stretched — a day holding two photos should look like
a short day, not like a layout bug. Ratios come from the daemon where it
has one; where it has not, the tile is laid out square and the day
re-flows once a decoded thumbnail proves the real shape, on the existing
debounce so a screenful of decodes costs one re-flow. Extreme ratios are
clamped, so one strip of film cannot decide what a row looks like.

Ctrl+scroll and Ctrl+± now move the target row height, and the grid is
tighter to match: 2 px gaps, smaller heading margins, a 6 px radius and
no card behind each tile, so the page reads as photographs rather than
as a deck of cards.

Thumbnails also stopped disappearing on scroll. The texture cache was
evicting in insertion order — which is precisely what a scroll back up
asks for again — so it is now least-recently-used and holds 1500 rather
than 600, which at the new density is several screens instead of under
two. A batch tops itself up with the photos just past the rows on
screen, in the direction of travel, and once the queue is short the
near-miss tiles are fetched rather than dropped.
2026-09-19 15:05:30 +02:00
nvrl cf0d495f8c fix(fuse): join the worker pool before the runtime goes away, and log deep queues
Teardown flagged the FUSE worker pool closed but never joined it, so the
process could drop the tokio runtime while a worker was still inside a
job — which is where 2026-09-18's "A Tokio 1.x context was found, but it
is being shutdown" panic came from. `Workers::stop_and_join` now waits
for the jobs in flight, bounded at 10 s so a worker wedged on the network
cannot hold the stop the user asked for; what is still stuck is named in
the log and shutdown continues around it.

The queues stay unbounded — the only caller that could feel backpressure
is fuser's dispatch loop, and blocking that is the stall the pool exists
to avoid — but a lane now logs its depth every 256 jobs, so a queue that
keeps climbing is visible before it is reported as a slow mount.

Documents the hang as B91, with the journal evidence, what made it
undiagnosable, and the candidates the evidence does not yet separate.
2026-09-19 14:49:31 +02:00
nvrl bee15ee7cf feat(fuse,packaging): watch the daemon for stalls and answer the systemd watchdog
A supervisor thread ticks every 15 s and does three things: it WARNs
about a worker that has held one job for two minutes, a control request
that never returned, or a lane that has not completed anything while its
queue is non-empty; it samples resident size and the pending-op count
into the log every five minutes; and it answers the systemd watchdog.

The watchdog ping is deliberately not a heartbeat from a thread that
does nothing else — that would keep the unit "healthy" through exactly
the hang it is meant to catch. It goes out only after a real round trip
over the control socket, so it means "a client can still talk to this
daemon", which is the property that failed on 2026-09-18.

`sd_notify` is spoken directly rather than through libsystemd (READY,
WATCHDOG, STOPPING), including the abstract-namespace address; off
systemd every call is a no-op.

The unit gains `NotifyAccess=all` (rather than `Type=notify`, which
would fail the unit on the start timeout when no login is stored yet),
`WatchdogSec=120`, `Restart=always` — a hang is not a failing exit code
— and `MemoryHigh`/`MemoryMax` as a backstop against the 5.2 G growth
the hung run showed.
2026-09-19 14:45:15 +02:00
nvrl f24ac8a429 feat(fuse,core,cli): report what the daemon is doing, for the next hang
On 2026-09-18 the daemon hung: 43 minutes of CPU over 20 hours, 5.2 GiB
resident, silent from 17:10 until a manual restart the next day, and no
panic. Nothing in the journal said what it had been waiting on, and
nothing in the daemon could be asked.

- The worker pool publishes per-thread state: whether the thread is busy,
  how long it has been in that state, and the label of the job in hand.
  Every FUSE handler that hands work to the pool now names its job
  (`read`, `lookup`, `rename`), so a stuck worker is named rather than
  merely counted. Queue depths and per-lane completion counts are
  mirrored outside the queue lock.
- `Request::Diagnostics` / `pdfs diagnostics` reports those, plus
  resident size, the control handler count against its limit, the
  in-flight control requests with their ages, and the pending-op count.
- Built to answer while the daemon is wedged: atomics and `try_lock`
  only, and the one field that needs the database reads it on a throwaway
  thread with a 500 ms deadline.
2026-09-19 14:40:26 +02:00
nvrl f78f663740 feat(core,fuse): restore the whole trashed tree, and bump the SDK to 0.6.5
The trash is a flat list of uids, but a delete has a shape: a folder, and
often items inside it that were trashed separately beforehand. Restoring
one of those in isolation is not what it means — a folder came back
without the contents deleted with it, and a file came back inside a
folder that is still trashed, which is to say nowhere the user can see.

- Schema 29: `trash.parent_uid`, filled by the trash refresh from the
  node's parent, plus an index for the walk.
- `expand_restore` takes the connected piece of the trashed tree around
  what was asked for (every trashed descendant, every trashed ancestor)
  and orders it into waves, shallowest first: the server cannot put a
  node back under a parent that is still trashed. A uid the listing does
  not know about is restored on its own, because the listing is
  materialised in chunks and can be stale.
- `Core::restore` restores wave by wave, still streaming per-node
  outcomes, so one node the server refuses leaves the rest restored.

The SDK bump brings the 6cbf2f44 port: permanent API errors are no
longer retried on block transfers, expired block URLs are re-minted on
the whole-file and public-link download paths, and a server-supplied
Retry-After is capped at 120 s.
2026-09-19 14:31:40 +02:00
nvrl 470948c191 chore(version) bumped to 1.10.2 2026-09-16 23:05:09 +02:00
nvrl b095b9f6b0 fix(core,fuse): never stat a foreign mount from the local-file indexer
The "This computer" indexer walks $HOME, and `default_excludes` only kept it
out of the Drive mountpoint and our own state/cache dirs. A home directory
that also holds third-party FUSE mounts (sshfs, rclone) got walked into.

`scan` does ask for `same_file_system(true)`, which looks sufficient and is
not: `ignore` implements it as `is_same_file_system` -> `device_num(path)` ->
`path.metadata()`, run after the directory is queued. Learning that a mount is
foreign requires stat'ing it first, and against a mount whose server is gone
that stat never returns.

The consequence is worse than a stalled scan. The blocked thread sits in
uninterruptible FUSE wait, so it ignores SIGKILL: the daemon can never exit,
systemd's stop times out, and the unreapable process keeps the cache.db lock
the next start needs, turning one dead mount into a permanent restart loop.
No userspace recovery exists once a thread is in that state, so the fix has to
be to never enter it.

Exclude every mountpoint strictly below the scan root, read by path from
/proc/self/mounts (octal-unescaped, so a path with a space still matches).
`filter_entry` runs before the device stat, so the walker never touches them.
Re-read per scan rather than cached at startup: a mount that appeared since
the daemon started is the likeliest to be half-alive.

This does not change what is indexed — a nested mount is a different device,
so it was already going to be skipped. Only the stat in front of that decision
goes away.

Refs: docs/BUGS.md B90
2026-09-16 23:05:06 +02:00
nvrl f4d6580cd3 chore(version) bumped to 1.10.1 2026-09-15 15:13:22 +02:00
nvrl b2f62f70e1 chore(deps,core,gui): upgrade dependencies to latest
- keyring 3 -> keyring-core 1 + dbus-secret-service-keyring-store 1
  (crypto-rust). Same libdbus Secret Service backend; with no target the
  store matches items keyring 3 wrote, so stored sessions survive.
- ksni 0.2 -> 0.3 via the blocking API, with assume_sni_available so a
  tray that starts before the SNI watcher waits instead of failing. Drops
  the clap 2 / atty chain (Dependabot atty advisory).
- gtk4 0.11, libadwaita 0.9, webkit6 0.6, rusqlite 0.40, fuser 0.18,
  directories 6, sha1/sha2 0.11, base64 0.23.
2026-09-15 15:07:00 +02:00
nvrl dcb2ab7fac chore(version) bumped to 1.10.0 2026-09-15 13:59:24 +02:00
nvrl eb56f4a1e0 feat(fuse,core,gui,cli): one-request move+rename drain, show who shared an item
- A queued rename that also changes folder now lands as one `move-multiple`
  request carrying the target name instead of rename-then-move, so there is
  no half-applied state and no second call for stale name requirements to
  hit (B46 follow-up). A destination name collision moves under a conflict
  name in one step; same-folder renames still use the rename endpoint.
- Shared with me shows the inviter and invite date (GUI subtitle, `pdfs
  shared-with-me`), with a warning when the invitation signature does not
  verify against the claimed inviter's keys. `DirEntry` gains `shared_by`,
  `shared_at` and `shared_by_unverified`, all serde-defaulted.
- Bump `proton-sdk` / `proton-drive-rs` to 0.6.3 (anonymous-node moves,
  `TrashTime` trashed detection).
2026-09-15 13:59:13 +02:00
nvrl c28803b037 chore(changelog) updated changelog for 1.9.2 and 1.9.3 2026-09-08 14:58:48 +02:00
nvrl cd8474954b chore(version) bumped to 1.9.3 2026-09-01 17:00:43 +02:00
nvrl d7e07d042d Merge pull request #22 from kacperpaczos/fix/login-poll-backoff
fix(cli): back off while waiting for a stored session
2026-09-01 16:38:29 +02:00
nvrl 45852082c0 fix(cli,core): wake the parked login-wait daemon on sign-in
The login backoff had no wake-up path. A unit started at boot sits in
wait_for_session and reaches the ceiling within ~90s; the user then signs
in and `enable --now` is a no-op on an already-active unit, so the daemon
kept sleeping and the mount appeared up to a full interval late.

- service::enable_start now runs `enable` + `restart` instead of
  `enable --now`, so the GUI login and the tray's Connect interrupt the
  sleep immediately and still start a unit that is down.
- `pdfs login` restarts the unit when it is already active. It stays out
  of `enable` on purpose, so a headless setup driving `pdfs daemon` by
  hand is left alone.
- Lower the backoff ceiling from 60s to 15s. It is still an 80% cut in
  keyring/D-Bus wakeups (28,800/day -> 5,760/day) and it bounds the
  worst-case latency for paths that never touch systemd.

Also corrects the flat-3s-poll figure in the doc comment: 86400/3 is
28,800 checks a day, not 48,000.
2026-09-01 16:28:25 +02:00
nvrl 99ad3838e1 fix(GALLERY) fixed takeout date conversion + repair command 2026-08-28 13:06:50 +02:00
Kacper Paczos 637b512f6a fix(cli): back off while waiting for a stored session
The daemon's wait-for-login loop slept a flat 3 seconds between checks and
never gave up, so an enabled service with no saved session polled the keyring
28,800 times a day. Each check is more than a sleep: auth::load() builds a
fresh keyring::Entry, so every iteration opens a new D-Bus connection to the
Secret Service and negotiates a session on it.

Back off from 3s to a 60s ceiling and say "not logged in" once at INFO rather
than on every iteration, which also stops the journal filling with tens of
thousands of identical lines.

The wait is extracted into wait_for_session() taking the load and sleep calls
as parameters, so the schedule is testable without a keyring and without real
time passing.

Refs #21
2026-08-25 13:24:46 +02:00
nvrl 16ad6548c8 chore(DOCS) updated changelog 2026-08-21 23:32:06 +02:00
52 changed files with 7763 additions and 1521 deletions

No files matched your search

Generated
+513 -793
View File
File diff suppressed because it is too large. Load diff
+14 -10
View File
@@ -3,7 +3,7 @@ resolver = "2"
members = ["crates/pdfs-core", "crates/pdfs-fuse", "crates/pdfs-cli", "crates/pdfs-gui"] members = ["crates/pdfs-core", "crates/pdfs-fuse", "crates/pdfs-cli", "crates/pdfs-gui"]
[workspace.package] [workspace.package]
version = "1.9.1" version = "1.11.0"
edition = "2024" edition = "2024"
license = "MIT" license = "MIT"
authors = ["Nils Pukropp <contact@narl.io>"] authors = ["Nils Pukropp <contact@narl.io>"]
@@ -15,8 +15,8 @@ keywords = ["proton", "drive", "fuse", "sync", "linux"]
categories = ["filesystem", "command-line-utilities"] categories = ["filesystem", "command-line-utilities"]
[workspace.dependencies] [workspace.dependencies]
proton-sdk = { version = "0.6.1" } proton-sdk = { version = "0.6.5" }
proton-drive-rs = { version = "0.6.1" } proton-drive-rs = { version = "0.6.5" }
# Internal crates. # Internal crates.
pdfs-core = { path = "crates/pdfs-core" } pdfs-core = { path = "crates/pdfs-core" }
@@ -25,7 +25,7 @@ pdfs-fuse = { path = "crates/pdfs-fuse" }
tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync", "fs", "io-util", "time", "signal"] } tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync", "fs", "io-util", "time", "signal"] }
serde = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] }
serde_json = "1" serde_json = "1"
base64 = "0.22" base64 = "0.23"
thiserror = "2" thiserror = "2"
anyhow = "1" anyhow = "1"
# Non-poisoning mutexes. A handler that panics while holding a lock must not # Non-poisoning mutexes. A handler that panics while holding a lock must not
@@ -35,24 +35,28 @@ anyhow = "1"
parking_lot = "0.12" parking_lot = "0.12"
tracing = "0.1" tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] } tracing-subscriber = { version = "0.3", features = ["env-filter"] }
directories = "5" directories = "6"
ignore = "0.4" ignore = "0.4"
sha2 = "0.10" sha2 = "0.11"
# Google Takeout exports are zip sets; the importer reads their central # Google Takeout exports are zip sets; the importer reads their central
# directories and streams entries out without extracting to disk first. # directories and streams entries out without extracting to disk first.
zip = { version = "8", default-features = false, features = ["deflate"] } zip = { version = "8", default-features = false, features = ["deflate"] }
# Proton's photo duplicate detection is keyed on the plaintext SHA-1, so an # Proton's photo duplicate detection is keyed on the plaintext SHA-1, so an
# import has to speak SHA-1 whatever its age. # import has to speak SHA-1 whatever its age.
sha1 = "0.10" sha1 = "0.11"
# Thumbnail generation for photos the server has none for. Decoders only for the # Thumbnail generation for photos the server has none for. Decoders only for the
# formats a camera or phone actually produces — the rest of the crate's codecs # formats a camera or phone actually produces — the rest of the crate's codecs
# would be dead weight in the daemon. # would be dead weight in the daemon.
image = { version = "0.25", default-features = false, features = ["jpeg", "png", "webp", "gif", "tiff", "bmp", "rayon"] } image = { version = "0.25", default-features = false, features = ["jpeg", "png", "webp", "gif", "tiff", "bmp", "rayon"] }
rusqlite = { version = "0.32", features = ["bundled"] } rusqlite = { version = "0.40", features = ["bundled"] }
keyring = { version = "3", features = ["sync-secret-service", "crypto-rust"] } # keyring 4 split into `keyring-core` plus per-platform stores. The libdbus
# Secret Service store is the one keyring 3's `sync-secret-service` wrapped, and
# with no `target` it matches items v3 wrote, so existing logins survive.
keyring-core = "1"
dbus-secret-service-keyring-store = { version = "1", features = ["crypto-rust"] }
clap = { version = "4", features = ["derive"] } clap = { version = "4", features = ["derive"] }
rpassword = "7" rpassword = "7"
async-trait = "0.1" async-trait = "0.1"
futures = "0.3" futures = "0.3"
fuser = "0.17" fuser = "0.18"
libc = "0.2" libc = "0.2"
+46 -1
View File
@@ -69,7 +69,7 @@ pdfs-app # the desktop app
- **Local Backup (Computers)**: Sync and back up local directories (like Downloads, Documents, Pictures, etc.) directly to your Proton Drive account. - **Local Backup (Computers)**: Sync and back up local directories (like Downloads, Documents, Pictures, etc.) directly to your Proton Drive account.
- **Locations**: One page (and `pdfs locations`) listing every place Proton Drive occupies on this machine — the main mount plus each backed-up folder — with its mode, sync state, and whether it is read-only. Switching a folder between a full local copy and on-demand happens here. - **Locations**: One page (and `pdfs locations`) listing every place Proton Drive occupies on this machine — the main mount plus each backed-up folder — with its mode, sync state, and whether it is read-only. Switching a folder between a full local copy and on-demand happens here.
- **System Tray Integration**: Background indicator for status monitoring, quick actions, and fast sync controls. - **System Tray Integration**: Background indicator for status monitoring, quick actions, and fast sync controls.
- **Unified Search Launcher (HUD)**: A resident Google Drive-style launcher (`pdfs-prompt`) that searches Proton Drive and local files together, ranks the best matches, tolerates abbreviations and typos, and is ideal for a system-wide hotkey. - **Unified Search Launcher (HUD)**: A resident Google Drive-style launcher (`pdfs-prompt`) that searches Proton Drive and local files together, ranks the best matches, tolerates abbreviations and typos, and is ideal for a system-wide hotkey. It can also present the same search in your own launcher (`--dmenu`) or, searching live as you type, in `fzf` (`--fzf`).
- **Secure Credential Storage**: Integrates with the system Secret Service (GNOME Keyring, KWallet, etc.) with smart in-memory credential caching to avoid UI thread blockages. - **Secure Credential Storage**: Integrates with the system Secret Service (GNOME Keyring, KWallet, etc.) with smart in-memory credential caching to avoid UI thread blockages.
- **Proton Photos Support**: Access your Proton Photos timeline, view thumbnails, and download backed-up media natively (available in the GUI as a navigation tab and via the CLI). - **Proton Photos Support**: Access your Proton Photos timeline, view thumbnails, and download backed-up media natively (available in the GUI as a navigation tab and via the CLI).
- **File Version History**: Every revision Proton Drive still holds for a file, from the browser's details pane (**Versions**) or `pdfs versions list|restore|save|rm`. Restoring is server-side — no re-upload — and an old version can be written out to a local file without touching the live one. - **File Version History**: Every revision Proton Drive still holds for a file, from the browser's details pane (**Versions**) or `pdfs versions list|restore|save|rm`. Restoring is server-side — no re-upload — and an old version can be written out to a local file without touching the live one.
@@ -124,6 +124,51 @@ launcher command, set them in `config.json`:
anywhere in `menu` is replaced by the prompt text; without one, the launcher's anywhere in `menu` is replaced by the prompt text; without one, the launcher's
own prompt flag is appended. own prompt flag is appended.
### Live launcher search (fzf)
The two-step above exists because no dmenu-style launcher can call back for new
results per keystroke. `fzf` can, so `pdfs-prompt --fzf` searches as you type,
exactly like the built-in window:
```bash
pdfs-prompt --fzf # pinned files, then live results as you type
pdfs-prompt --fzf --query invoice # start on a search
```
Every keystroke re-queries the daemon (debounced, like the GTK prompt) and fzf's
own matcher is switched off, so the ordering you see is the daemon's ranking —
abbreviations and typos included, which a launcher filtering a fixed list cannot
do. Enter opens; Escape closes.
fzf is a terminal program, so when it is started without one — from a WM
keybinding — it spawns a terminal for itself: foot, ghostty, kitty, alacritty,
wezterm or xterm, whichever is installed. Each is launched with the app-id/class
`pdfs-prompt`, so one window rule floats it whichever terminal you have. In
Hyprland:
```
windowrulev2 = float, class:^(pdfs-prompt)$
windowrulev2 = size 900 500, class:^(pdfs-prompt)$
windowrulev2 = center, class:^(pdfs-prompt)$
bind = SUPER, space, exec, pdfs-prompt --fzf
```
Set it as the default front end, and pick the terminal explicitly, in
`config.json`:
```json
{
"prompt": {
"mode": "fzf",
"terminal": ["foot", "--app-id=pdfs-prompt", "--"],
"menu_limit": 50
}
}
```
A `{cmd}` token anywhere in `terminal` is replaced by the command to run;
without one it is appended, which is what every detected terminal expects.
## Choosing How Files Open ## Choosing How Files Open
By default every result is handed to `xdg-open`. An `open_with` block in By default every result is handed to `xdg-open`. An `open_with` block in
+453 -13
View File
@@ -17,6 +17,7 @@ use pdfs_core::control::{
RestoreItem as CtlRestoreItem, ShareEntryKind, SyncPhase, pending_summary, RestoreItem as CtlRestoreItem, ShareEntryKind, SyncPhase, pending_summary,
}; };
use pdfs_core::db::Db; use pdfs_core::db::Db;
use pdfs_core::service;
#[derive(Parser)] #[derive(Parser)]
#[command( #[command(
@@ -176,6 +177,43 @@ enum Command {
#[arg(long)] #[arg(long)]
wait: bool, wait: bool,
}, },
/// Fix photos filed under the day they were imported instead of the day
/// they were taken.
///
/// A Takeout whose metadata sidecars did not match imports with no capture
/// time, and Proton then stamps every photo with the moment it arrived. The
/// date is still in the file name (`IMG-20230219-WA0001.jpg`), so this finds
/// photos whose name disagrees with their capture time by more than a day
/// and rewrites them at the right time.
///
/// Proton cannot edit a sealed capture time, so a rewrite means re-uploading
/// the photo and trashing the original — favourites and albums are carried
/// across, and the original goes to Proton's trash, not away. Run it with
/// `--dry-run` first.
RedatePhotos {
/// Report what would be re-dated without uploading or trashing anything.
#[arg(long)]
dry_run: bool,
/// Stop after this many photos.
#[arg(long)]
limit: Option<usize>,
/// Only touch photos whose current capture time is on or after this
/// date (`YYYY-MM-DD`, UTC) — aim the repair at the days an import
/// landed on.
#[arg(long)]
from: Option<String>,
/// Only touch photos whose current capture time is before this date
/// (`YYYY-MM-DD`, UTC).
#[arg(long)]
to: Option<String>,
/// Stay attached and print the report when the run finishes.
#[arg(long)]
wait: bool,
},
/// Show how the running (or last) photo re-date is doing.
RedateStatus,
/// Stop the running photo re-date.
CancelRedate,
/// Show how the running (or last) Google Photos import is doing. /// Show how the running (or last) Google Photos import is doing.
ImportStatus, ImportStatus,
/// Stop the running Google Photos import. /// Stop the running Google Photos import.
@@ -243,6 +281,10 @@ enum Command {
}, },
/// Show the daemon's in-flight transfers (active uploads/downloads). /// Show the daemon's in-flight transfers (active uploads/downloads).
Transfers, Transfers,
/// Show what the daemon is doing right now: worker threads, queue depths,
/// in-flight control requests and memory. For a daemon that has stopped
/// answering rather than one that is merely busy.
Diagnostics,
/// List what is in the account's trash, with the uids the restore and /// List what is in the account's trash, with the uids the restore and
/// delete-forever commands take. /// delete-forever commands take.
Trash, Trash,
@@ -268,6 +310,11 @@ enum Command {
/// What to refresh: a folder path (inside the mountpoint or relative to /// What to refresh: a folder path (inside the mountpoint or relative to
/// it), `trash`, or `photos`. The mount root if omitted. /// it), `trash`, or `photos`. The mount root if omitted.
target: Option<String>, target: Option<String>,
/// With `photos`: read every photo's metadata again instead of only what
/// changed. Minutes on a large library; for a gallery that disagrees with
/// the account.
#[arg(long)]
full: bool,
}, },
/// Manage the account's registered devices. /// Manage the account's registered devices.
@@ -616,6 +663,15 @@ fn main() -> Result<()> {
dry_run, dry_run,
wait, wait,
} => cmd_import_google_photos(archives, dry_run, wait), } => cmd_import_google_photos(archives, dry_run, wait),
Command::RedatePhotos {
dry_run,
limit,
from,
to,
wait,
} => cmd_redate_photos(dry_run, limit, from, to, wait),
Command::RedateStatus => cmd_redate_status(),
Command::CancelRedate => cmd_cancel_redate(),
Command::ImportStatus => cmd_import_status(), Command::ImportStatus => cmd_import_status(),
Command::CancelImport => cmd_cancel_import(), Command::CancelImport => cmd_cancel_import(),
Command::Album { uid, limit, offset } => cmd_album(uid, limit, offset), Command::Album { uid, limit, offset } => cmd_album(uid, limit, offset),
@@ -627,11 +683,12 @@ fn main() -> Result<()> {
Command::Mkdir { parent, name } => cmd_mkdir(parent, name), Command::Mkdir { parent, name } => cmd_mkdir(parent, name),
Command::Upload { sources, parent } => cmd_upload(sources, parent), Command::Upload { sources, parent } => cmd_upload(sources, parent),
Command::Transfers => cmd_transfers(), Command::Transfers => cmd_transfers(),
Command::Diagnostics => cmd_diagnostics(),
Command::Trash => cmd_trash(), Command::Trash => cmd_trash(),
Command::Restore { uids } => cmd_restore(uids), Command::Restore { uids } => cmd_restore(uids),
Command::DeleteForever { uids } => cmd_delete_forever(uids), Command::DeleteForever { uids } => cmd_delete_forever(uids),
Command::EmptyTrash => cmd_empty_trash(), Command::EmptyTrash => cmd_empty_trash(),
Command::Refresh { target } => cmd_refresh(target), Command::Refresh { target, full } => cmd_refresh(target, full),
Command::Devices { action } => cmd_devices(action), Command::Devices { action } => cmd_devices(action),
Command::Sync { action } => cmd_sync(action), Command::Sync { action } => cmd_sync(action),
Command::Share { Command::Share {
@@ -1082,6 +1139,9 @@ fn cmd_shared_with_me(uid: Option<String>) -> Result<()> {
format!(" ({})", e.role) format!(" ({})", e.role)
}; };
println!("{kind} {:>12} {}{role} [{}]", e.size, e.name, e.uid); println!("{kind} {:>12} {}{role} [{}]", e.size, e.name, e.uid);
if let Some(line) = shared_by_line(&e) {
println!("{:>15}{line}", "");
}
} }
} }
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)), CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
@@ -1090,6 +1150,39 @@ fn cmd_shared_with_me(uid: Option<String>) -> Result<()> {
Ok(()) Ok(())
} }
/// Who shared an entry with me, and when, as a line printed under it — `None`
/// when the invitation said nothing (owned content, or a node inside a shared
/// folder rather than its root). An inviter whose signature did not verify is
/// said to be claimed, not stated: the name may be forged.
fn shared_by_line(entry: &pdfs_core::control::DirEntry) -> Option<String> {
if entry.shared_by.is_empty() {
return None;
}
let mut line = format!("shared by {}", entry.shared_by);
if entry.shared_at > 0 {
line.push_str(&format!(" at {}", format_epoch_utc(entry.shared_at)));
}
if entry.shared_by_unverified {
line.push_str(" WARNING: invitation signature did not verify; the sender may be forged");
}
Some(line)
}
/// `YYYY-MM-DD` (UTC) for epoch seconds, without pulling a date crate into the
/// CLI for one column. Civil-from-days per Howard Hinnant's algorithm.
fn format_epoch_utc(secs: i64) -> String {
let z = secs.div_euclid(86_400) + 719_468;
let era = z.div_euclid(146_097);
let doe = z - era * 146_097;
let yoe = (doe - doe / 1_460 + doe / 36_524 - doe / 146_096) / 365;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100);
let mp = (5 * doy + 2) / 153;
let day = doy - (153 * mp + 2) / 5 + 1;
let month = if mp < 10 { mp + 3 } else { mp - 9 };
let year = yoe + era * 400 + i64::from(month <= 2);
format!("{year:04}-{month:02}-{day:02}")
}
/// Download a file shared with me. The daemon puts the plaintext in its content /// Download a file shared with me. The daemon puts the plaintext in its content
/// cache and replies with that path; with a `dest` we copy it out, so the caller /// cache and replies with that path; with a `dest` we copy it out, so the caller
/// gets a file it owns rather than one the cache may later evict. /// gets a file it owns rather than one the cache may later evict.
@@ -1309,6 +1402,15 @@ fn cmd_login(username: Option<String>) -> Result<()> {
other => anyhow::Error::new(other).context("login failed"), other => anyhow::Error::new(other).context("login failed"),
})?; })?;
// A daemon that is already up is parked in `wait_for_session`, sleeping
// between keyring checks; restart it so the mount appears now instead of
// whenever the next poll fires. Only when it is already active — `pdfs
// login` deliberately does not enable the unit the way the GUI does, so a
// headless user driving `pdfs daemon` by hand keeps that setup.
if service::is_active() {
service::restart();
}
println!("Logged in as {username}. Session stored in the system keyring."); println!("Logged in as {username}. Session stored in the system keyring.");
Ok(()) Ok(())
} }
@@ -1448,6 +1550,59 @@ fn cmd_mount(mountpoint: Option<PathBuf>) -> Result<()> {
Ok(()) Ok(())
} }
/// How long to wait before the next login check, after `attempts` checks have
/// already come back [`pdfs_core::Error::NotLoggedIn`].
///
/// Doubles from 3s to a 15s ceiling. The interval is not just a sleep:
/// [`auth::load`] builds a fresh `keyring::Entry` on every call, so each check
/// opens a new D-Bus connection to the Secret Service and negotiates an
/// encrypted session over it. A flat 3s poll meant 28,800 of those a day from a
/// daemon that had nothing to do; the ceiling cuts that by 80%.
///
/// The ceiling stays low because the sleep is also the worst-case login →
/// mount latency for anything that does *not* poke systemd: front-ends restart
/// the unit on login (`service::enable_start`), which interrupts the sleep
/// immediately, but a hand-run `pdfs daemon` has no such wake-up path.
fn login_poll_delay(attempts: u32) -> std::time::Duration {
const BASE_SECS: u64 = 3;
const MAX_SECS: u64 = 15;
// `checked_shl` rather than `<<`: the daemon can sit unauthenticated for
// weeks, and a shift past the width of the type is a panic, not a big number.
let factor = 1u64.checked_shl(attempts).unwrap_or(u64::MAX);
std::time::Duration::from_secs(BASE_SECS.saturating_mul(factor).min(MAX_SECS))
}
/// Block until a session is stored, backing off between checks.
///
/// `load` and `sleep` are parameters rather than direct calls so the backoff
/// schedule can be tested without a keyring and without real time passing.
fn wait_for_session<L, S>(mut load: L, mut sleep: S) -> Result<()>
where
L: FnMut() -> std::result::Result<(), pdfs_core::Error>,
S: FnMut(std::time::Duration),
{
let mut attempts: u32 = 0;
loop {
match load() {
Ok(()) => return Ok(()),
Err(pdfs_core::Error::NotLoggedIn) => {
let delay = login_poll_delay(attempts);
// Say it once at INFO. Repeating it every interval buried the
// journal in tens of thousands of identical lines a day.
if attempts == 0 {
tracing::info!("not logged in; waiting for a session…");
} else {
tracing::debug!(delay_secs = delay.as_secs(), "still not logged in");
}
sleep(delay);
attempts = attempts.saturating_add(1);
}
Err(e) => return Err(e.into()),
}
}
}
/// Run the auto-mount daemon loop. Waits for a stored login, mounts, and keeps /// Run the auto-mount daemon loop. Waits for a stored login, mounts, and keeps
/// the mount alive: a clean stop (SIGTERM via `systemctl --user stop`) exits 0; /// the mount alive: a clean stop (SIGTERM via `systemctl --user stop`) exits 0;
/// an external unmount triggers a remount; errors back off and retry. This is /// an external unmount triggers a remount; errors back off and retry. This is
@@ -1456,16 +1611,7 @@ fn cmd_daemon(mountpoint: Option<PathBuf>) -> Result<()> {
loop { loop {
// Wait until a session is stored. The GUI enables this service on login, // Wait until a session is stored. The GUI enables this service on login,
// but the service may also start at boot before the user has logged in. // but the service may also start at boot before the user has logged in.
loop { wait_for_session(|| auth::load().map(|_| ()), std::thread::sleep)?;
match auth::load() {
Ok(_) => break,
Err(pdfs_core::Error::NotLoggedIn) => {
tracing::info!("not logged in; waiting…");
std::thread::sleep(std::time::Duration::from_secs(3));
}
Err(e) => return Err(e.into()),
}
}
match mount_once(mountpoint.clone()) { match mount_once(mountpoint.clone()) {
Ok(pdfs_fuse::MountOutcome::Shutdown) => { Ok(pdfs_fuse::MountOutcome::Shutdown) => {
@@ -1787,6 +1933,115 @@ fn cmd_import_google_photos(archives: Vec<PathBuf>, dry_run: bool, wait: bool) -
} }
} }
fn cmd_redate_photos(
dry_run: bool,
limit: Option<usize>,
from: Option<String>,
to: Option<String>,
wait: bool,
) -> Result<()> {
let range = match (from.as_deref(), to.as_deref()) {
(None, None) => None,
(from, to) => Some((
from.map(parse_day).transpose()?.unwrap_or(i64::MIN),
to.map(parse_day).transpose()?.unwrap_or(i64::MAX),
)),
};
match control_request(CtlRequest::RedatePhotos {
dry_run,
limit,
range,
})? {
CtlResponse::Ok { message } => println!("{message}"),
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
if !wait {
println!("Run `pdfs redate-status` to follow it.");
return Ok(());
}
// Polled for the same reason the import is: request/response protocol, and a
// run over a whole library is a download plus an upload per photo.
loop {
std::thread::sleep(std::time::Duration::from_secs(2));
match control_request(CtlRequest::RedateStatus)? {
CtlResponse::RedateStatus { running: true, .. } => continue,
CtlResponse::RedateStatus { summary, .. } => {
print_redate_summary(summary.as_ref());
return Ok(());
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
}
}
/// A `YYYY-MM-DD` day as epoch seconds at UTC midnight. Reuses the Takeout
/// scanner's date reader, which already accepts exactly this spelling.
fn parse_day(day: &str) -> Result<i64> {
pdfs_core::takeout::capture_time_from_name(day)
.ok_or_else(|| anyhow!("not a date (expected YYYY-MM-DD): {day}"))
}
fn cmd_redate_status() -> Result<()> {
match control_request(CtlRequest::RedateStatus)? {
CtlResponse::RedateStatus { running, summary } => {
if running {
println!("Re-date running. `pdfs transfers` shows its progress.");
}
print_redate_summary(summary.as_ref());
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
Ok(())
}
fn cmd_cancel_redate() -> Result<()> {
match control_request(CtlRequest::CancelRedate)? {
CtlResponse::Ok { message } => println!("{message}"),
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
Ok(())
}
fn print_redate_summary(summary: Option<&pdfs_core::control::RedateSummary>) {
let Some(summary) = summary else {
println!("No re-date has run yet.");
return;
};
println!("Photos examined : {}", summary.examined);
println!("Mis-dated : {}", summary.candidates);
if summary.dry_run {
println!("(dry run — nothing was uploaded or trashed)");
} else {
println!("Re-dated : {}", summary.redated);
if summary.album_links > 0 {
println!("Albums refiled : {}", summary.album_links);
}
if summary.failed > 0 {
println!("Failed : {}", summary.failed);
}
if summary.bytes > 0 {
println!("Uploaded bytes : {}", summary.bytes);
}
}
for (name, stored, wanted) in &summary.samples {
// Epoch seconds, as `pdfs photos` prints capture times.
println!(" {name}: {stored} -> {wanted}");
}
if summary.candidates > summary.samples.len() {
println!(
" … and {} more",
summary.candidates - summary.samples.len()
);
}
if summary.cancelled {
println!("Run was cancelled before it finished.");
}
}
fn cmd_import_status() -> Result<()> { fn cmd_import_status() -> Result<()> {
match control_request(CtlRequest::ImportStatus)? { match control_request(CtlRequest::ImportStatus)? {
CtlResponse::ImportStatus { running, summary } => { CtlResponse::ImportStatus { running, summary } => {
@@ -1908,6 +2163,67 @@ fn cmd_rm(path: PathBuf) -> Result<()> {
Ok(()) Ok(())
} }
/// Seconds as something a human reads at a glance: the ages here run from
/// "still starting" to "wedged since yesterday".
fn human_secs(secs: u64) -> String {
match secs {
s if s < 60 => format!("{s}s"),
s if s < 3600 => format!("{}m{}s", s / 60, s % 60),
s if s < 86400 => format!("{}h{}m", s / 3600, (s % 3600) / 60),
s => format!("{}d{}h", s / 86400, (s % 86400) / 3600),
}
}
fn cmd_diagnostics() -> Result<()> {
let response = control_request(CtlRequest::Diagnostics)?;
if emit_json(&response)? {
return Ok(());
}
match response {
CtlResponse::Diagnostics { diagnostics: d } => {
println!(
"uptime {} rss {} pending ops {}",
human_secs(d.uptime_secs),
human_bytes(d.rss_bytes),
d.pending_ops,
);
println!(
"control handlers {}/{}",
d.control_handlers, d.control_handler_limit,
);
println!(
"queued: meta {} (done {}), transfer {} (done {})",
d.meta_queued, d.meta_completed, d.transfer_queued, d.transfer_completed,
);
println!("workers:");
for worker in &d.workers {
let what = if worker.busy {
worker.label.as_str()
} else {
"idle"
};
println!(
" {:<20} {:<12} {}",
worker.name,
what,
human_secs(worker.age_secs)
);
}
if d.inflight.is_empty() {
println!("in-flight requests: none");
} else {
println!("in-flight requests (oldest first):");
for request in &d.inflight {
println!(" {:<20} {}", request.kind, human_secs(request.age_secs));
}
}
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
Ok(())
}
fn cmd_trash() -> Result<()> { fn cmd_trash() -> Result<()> {
match control_request(CtlRequest::ListTrash)? { match control_request(CtlRequest::ListTrash)? {
CtlResponse::Entries { entries } if entries.is_empty() => println!("(trash is empty)"), CtlResponse::Entries { entries } if entries.is_empty() => println!("(trash is empty)"),
@@ -1958,22 +2274,51 @@ fn cmd_empty_trash() -> Result<()> {
/// Drop a cached listing. `trash` and `photos` name those two listings; anything /// Drop a cached listing. `trash` and `photos` name those two listings; anything
/// else is read as a folder path, so a folder actually called "trash" is still /// else is read as a folder path, so a folder actually called "trash" is still
/// reachable as `./trash`. /// reachable as `./trash`.
fn cmd_refresh(target: Option<String>) -> Result<()> { fn cmd_refresh(target: Option<String>, full: bool) -> Result<()> {
let scope = match target.as_deref() { let scope = match target.as_deref() {
Some("trash") => RefreshScope::Trash, Some("trash") => RefreshScope::Trash,
Some("photos") => RefreshScope::Photos, Some("photos") => RefreshScope::Photos { full },
path => RefreshScope::Dir { path => RefreshScope::Dir {
path: path.unwrap_or("").to_string(), path: path.unwrap_or("").to_string(),
}, },
}; };
let photos = matches!(scope, RefreshScope::Photos { .. });
match control_request(CtlRequest::Refresh { scope })? { match control_request(CtlRequest::Refresh { scope })? {
CtlResponse::Ok { message } => println!("{message}"), CtlResponse::Ok { message } => println!("{message}"),
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)), CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"), other => bail!("unexpected response: {other:?}"),
} }
if photos {
wait_for_photos_refresh()?;
}
Ok(()) Ok(())
} }
/// How often [`wait_for_photos_refresh`] asks the daemon whether the timeline
/// refresh it started is done.
const PHOTOS_REFRESH_POLL: std::time::Duration = std::time::Duration::from_secs(2);
/// Block until the daemon's timeline refresh has finished.
///
/// The daemon answers `Refresh { Photos }` as soon as the refresh is running,
/// because re-reading a large library takes minutes and a control reply may not.
/// `pdfs refresh photos` means "the timeline is up to date when I return", so
/// the waiting happens here, where no socket read bound applies.
fn wait_for_photos_refresh() -> Result<()> {
loop {
std::thread::sleep(PHOTOS_REFRESH_POLL);
match control_request(CtlRequest::PhotosRefreshStatus)? {
CtlResponse::PhotosRefresh { running: true } => continue,
CtlResponse::PhotosRefresh { running: false } => {
println!("refreshed");
return Ok(());
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
}
}
/// Ask for a typed `yes` before an irreversible destroy. Errors (aborting the /// Ask for a typed `yes` before an irreversible destroy. Errors (aborting the
/// command) on anything else, including a non-interactive stdin: a piped or /// command) on anything else, including a non-interactive stdin: a piped or
/// scripted invocation must not silently wipe the trash. /// scripted invocation must not silently wipe the trash.
@@ -2537,3 +2882,98 @@ mod diagnose_tests {
std::fs::remove_dir(dir).unwrap(); std::fs::remove_dir(dir).unwrap();
} }
} }
#[cfg(test)]
mod login_wait_tests {
use super::*;
use std::time::Duration;
#[test]
fn poll_delay_doubles_then_holds_at_the_ceiling() {
assert_eq!(login_poll_delay(0), Duration::from_secs(3));
assert_eq!(login_poll_delay(1), Duration::from_secs(6));
assert_eq!(login_poll_delay(2), Duration::from_secs(12));
assert_eq!(login_poll_delay(3), Duration::from_secs(15));
assert_eq!(login_poll_delay(4), Duration::from_secs(15));
// A daemon left unauthenticated for weeks reaches attempt counts well
// past the width of the shift; that must cap, not panic.
assert_eq!(login_poll_delay(64), Duration::from_secs(15));
assert_eq!(login_poll_delay(u32::MAX), Duration::from_secs(15));
}
#[test]
fn waiting_for_a_session_stops_hammering_the_keyring() {
let mut checks_left = 20;
let mut slept: Vec<Duration> = Vec::new();
wait_for_session(
|| {
if checks_left == 0 {
Ok(())
} else {
checks_left -= 1;
Err(pdfs_core::Error::NotLoggedIn)
}
},
|delay| slept.push(delay),
)
.unwrap();
assert_eq!(slept.len(), 20, "one wait per failed check");
assert!(
slept.windows(2).all(|w| w[1] >= w[0]),
"the interval must never shrink: {slept:?}"
);
// The point of the change. Every check is a fresh D-Bus session against
// the Secret Service, so 20 of them must not fit inside a single minute
// the way a flat 3s poll did.
let total: Duration = slept.iter().sum();
assert!(
total >= Duration::from_secs(4 * 60),
"20 checks should span at least 4 minutes, spanned {total:?}"
);
}
#[test]
fn a_stored_session_is_not_waited_for_at_all() {
let mut slept = 0;
wait_for_session(|| Ok(()), |_| slept += 1).unwrap();
assert_eq!(slept, 0);
}
#[test]
fn epoch_seconds_format_as_utc_calendar_dates() {
assert_eq!(format_epoch_utc(0), "1970-01-01");
assert_eq!(format_epoch_utc(951_782_400), "2000-02-29");
assert_eq!(format_epoch_utc(1_700_000_000), "2023-11-14");
assert_eq!(format_epoch_utc(-86_400), "1969-12-31");
}
#[test]
fn a_forged_looking_invitation_is_flagged_not_stated() {
let mut entry = pdfs_core::control::DirEntry {
name: "Budget.ods".into(),
is_dir: false,
size: 1,
modified: 0,
pinned: false,
cached: false,
uid: "vol~link".into(),
path: String::new(),
role: "viewer".into(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
};
assert_eq!(shared_by_line(&entry), None, "no invitation, no line");
entry.shared_by = "alice@proton.me".into();
entry.shared_at = 1_700_000_000;
assert_eq!(
shared_by_line(&entry).as_deref(),
Some("shared by alice@proton.me at 2023-11-14")
);
entry.shared_by_unverified = true;
assert!(shared_by_line(&entry).unwrap().contains("WARNING"));
}
}
+2 -1
View File
@@ -23,7 +23,8 @@ sha2.workspace = true
sha1.workspace = true sha1.workspace = true
zip.workspace = true zip.workspace = true
rusqlite.workspace = true rusqlite.workspace = true
keyring.workspace = true keyring-core.workspace = true
dbus-secret-service-keyring-store.workspace = true
parking_lot.workspace = true parking_lot.workspace = true
async-trait.workspace = true async-trait.workspace = true
libc.workspace = true libc.workspace = true
+8 -3
View File
@@ -6,7 +6,7 @@
//! interactive step; the refresh token auto-renews via the HTTP client's 401 //! interactive step; the refresh token auto-renews via the HTTP client's 401
//! path, so no fresh 2FA is required until the refresh token itself expires. //! path, so no fresh 2FA is required until the refresh token itself expires.
use keyring::Entry; use keyring_core::Entry;
use proton_drive_rs::{KeySalt, ProtonDriveClient}; use proton_drive_rs::{KeySalt, ProtonDriveClient};
use proton_sdk::api::{HumanVerification, HumanVerificationCredential}; use proton_sdk::api::{HumanVerification, HumanVerificationCredential};
use proton_sdk::cache::EncryptedCacheRepository; use proton_sdk::cache::EncryptedCacheRepository;
@@ -81,6 +81,11 @@ fn client_config() -> ProtonClientConfiguration {
} }
fn keyring_entry() -> Result<Entry> { fn keyring_entry() -> Result<Entry> {
// Installed lazily rather than once at startup so a session bus that was not
// up yet on the first attempt is retried on the next one.
if keyring_core::get_default_store().is_none() {
keyring_core::set_default_store(dbus_secret_service_keyring_store::Store::new()?);
}
Ok(Entry::new(KEYRING_SERVICE, KEYRING_USER)?) Ok(Entry::new(KEYRING_SERVICE, KEYRING_USER)?)
} }
@@ -215,7 +220,7 @@ pub fn load() -> Result<StoredSession> {
let entry = keyring_entry()?; let entry = keyring_entry()?;
match entry.get_password() { match entry.get_password() {
Ok(json) => Ok(serde_json::from_str(&json)?), Ok(json) => Ok(serde_json::from_str(&json)?),
Err(keyring::Error::NoEntry) => Err(Error::NotLoggedIn), Err(keyring_core::Error::NoEntry) => Err(Error::NotLoggedIn),
Err(e) => Err(e.into()), Err(e) => Err(e.into()),
} }
} }
@@ -238,7 +243,7 @@ pub fn logout() -> Result<()> {
} }
} }
match keyring_entry()?.delete_credential() { match keyring_entry()?.delete_credential() {
Ok(()) | Err(keyring::Error::NoEntry) => Ok(()), Ok(()) | Err(keyring_core::Error::NoEntry) => Ok(()),
Err(e) => Err(e.into()), Err(e) => Err(e.into()),
} }
} }
+295 -3
View File
@@ -68,6 +68,11 @@ pub enum Request {
#[serde(default)] #[serde(default)]
favorites: bool, favorites: bool,
}, },
/// Every photo of one photo's group — the JPEG and the RAW of one shot, a
/// live photo and its clip. Replies with [`Response::Photos`], holding one
/// item per file in server order; a photo that stands alone comes back as a
/// single item. The lightbox uses it to switch between a shot's files.
PhotoGroup { uid: String },
/// The months the timeline spans (newest first, with per-month counts) so a /// The months the timeline spans (newest first, with per-month counts) so a
/// front-end can build a date scrubber without paging the whole library. /// front-end can build a date scrubber without paging the whole library.
/// `kind` scopes the counts to one tab when set. /// `kind` scopes the counts to one tab when set.
@@ -93,6 +98,17 @@ pub enum Request {
/// already cached (one batched round-trip) and replying with their on-disk /// already cached (one batched round-trip) and replying with their on-disk
/// paths. Keep the batch small — it is served on demand, as tiles scroll in. /// paths. Keep the batch small — it is served on demand, as tiles scroll in.
PhotoThumbs { uids: Vec<String> }, PhotoThumbs { uids: Vec<String> },
/// Trash photos by node uid, answered with [`Response::Trashed`].
///
/// Uid-addressed rather than path-addressed because the photos volume is not
/// part of the FUSE mount: the gallery knows a photo only by its uid, and
/// [`Request::Delete`] has no path to take for it. The nodes land in Proton
/// trash, so this stays recoverable from the Trash page.
///
/// A photo that belongs to a group takes the whole group with it: trashing
/// the JPEG of a shot and leaving its RAW behind is not what anyone means by
/// deleting that photo.
TrashNodes { uids: Vec<String> },
/// Fetch thumbnails for ordinary Drive image files shown outside the Photos /// Fetch thumbnails for ordinary Drive image files shown outside the Photos
/// timeline. The modification time is the cache validity tag, so replacing /// timeline. The modification time is the cache validity tag, so replacing
/// an image can never reuse its previous revision's thumbnail. /// an image can never reuse its previous revision's thumbnail.
@@ -119,6 +135,12 @@ pub enum Request {
CancelThumbnailBuild, CancelThumbnailBuild,
/// Read progress for the recursive ordinary-file thumbnail build. /// Read progress for the recursive ordinary-file thumbnail build.
ThumbnailBuildStatus, ThumbnailBuildStatus,
/// Whether a photos-timeline refresh is running right now. Replies with
/// [`Response::PhotosRefresh`].
///
/// [`RefreshScope::Photos`] starts the refresh and returns at once, so this
/// is how a front-end knows when the refreshed timeline is worth re-reading.
PhotosRefreshStatus,
/// Download a photo's full content into the cache; replies with its path. /// Download a photo's full content into the cache; replies with its path.
OpenPhoto { uid: String }, OpenPhoto { uid: String },
/// Add or remove Proton's `Favorite` tag on a photo. Replies with /// Add or remove Proton's `Favorite` tag on a photo. Replies with
@@ -149,6 +171,43 @@ pub enum Request {
#[serde(default)] #[serde(default)]
dry_run: bool, dry_run: bool,
}, },
/// Re-date photos whose capture time is when they were *imported* rather
/// than when they were taken.
///
/// Proton has no API for editing a photo's capture time — it is sealed into
/// the revision — so the only repair is to upload the photo again with the
/// right time and trash the original. That is what this does, for photos
/// whose file name carries a date that disagrees with their stored capture
/// time by more than a day: `IMG-20230219-WA0001.jpg` filed under 2026 is
/// the signature of an import whose metadata sidecars did not match.
///
/// Favourites and album memberships are carried across; the original is
/// **trashed**, not deleted, so a bad run is recoverable from Proton's
/// trash. With `dry_run` nothing is uploaded or trashed and the report says
/// what would have been done.
///
/// Acks immediately with [`Response::Ok`] and works in the background — each
/// photo is a download plus an upload. Progress shows up in
/// [`Request::GetQueueStatus`] as a job, and the counts in
/// [`Request::RedateStatus`].
RedatePhotos {
#[serde(default)]
dry_run: bool,
/// Stop after this many photos. `None` is the whole library.
#[serde(default)]
limit: Option<usize>,
/// Only consider photos whose *stored* capture time falls in this
/// `[from, to)` window — the way to aim the repair at the days an
/// import landed on and leave the rest of the library untouched.
#[serde(default)]
range: Option<(i64, i64)>,
},
/// How the running (or last finished) re-date is doing. Replies with
/// [`Response::RedateStatus`].
RedateStatus,
/// Ask the running re-date to stop. It finishes the photo on the wire, then
/// reports `cancelled`. Replies with [`Response::Ok`].
CancelRedate,
/// How the running (or last finished) Takeout import is doing. Replies with /// How the running (or last finished) Takeout import is doing. Replies with
/// [`Response::ImportStatus`]. /// [`Response::ImportStatus`].
ImportStatus, ImportStatus,
@@ -262,6 +321,13 @@ pub enum Request {
/// [`Response::Transfers`]. Cheap to poll: the daemon keeps the registry in /// [`Response::Transfers`]. Cheap to poll: the daemon keeps the registry in
/// memory, so a front-end can render a live progress widget. /// memory, so a front-end can render a live progress widget.
GetQueueStatus, GetQueueStatus,
/// Report what the daemon is doing *right now*, for debugging a daemon that
/// stopped answering rather than one that is merely busy. Replies
/// [`Response::Diagnostics`].
///
/// Answered from lock-free state on purpose: it has to come back while the
/// daemon is wedged, which is the only moment anyone asks for it.
Diagnostics,
/// List what is in the account's trash. Replies with [`Response::Entries`]; /// List what is in the account's trash. Replies with [`Response::Entries`];
/// a trashed node has no path inside the mount, so each entry carries only /// a trashed node has no path inside the mount, so each entry carries only
/// its `uid` — the handle for [`Request::Restore`] and /// its `uid` — the handle for [`Request::Restore`] and
@@ -522,8 +588,19 @@ pub enum RefreshScope {
Dir { path: String }, Dir { path: String },
/// The trash listing. /// The trash listing.
Trash, Trash,
/// The photos timeline. /// The photos timeline. The reply comes back as soon as the refresh has
Photos, /// *started*, not when it has finished: re-reading a large library takes
/// minutes, which is far longer than a control reply may take. Poll
/// [`Request::PhotosRefreshStatus`] to know when the new timeline is there.
///
/// A refresh normally only reads the photos the daemon has never read and
/// the ones a remote event marked changed. `full` drops that knowledge and
/// reads every photo's node again — the repair for a timeline that somehow
/// disagrees with the account, and minutes of work on a large library.
Photos {
#[serde(default)]
full: bool,
},
} }
/// A registered device in a [`Response::Devices`] listing. /// A registered device in a [`Response::Devices`] listing.
@@ -869,6 +946,22 @@ pub struct DirEntry {
/// report. Defaulted for wire-compat with clients/daemons predating it. /// report. Defaulted for wire-compat with clients/daemons predating it.
#[serde(default)] #[serde(default)]
pub role: String, pub role: String,
/// Who shared this node with me, as the invitation claims it — empty for
/// content I own, for a node reached *inside* a shared folder (only the share
/// root carries the invitation), and when the API did not say. Trust it only
/// as far as [`shared_by_unverified`](Self::shared_by_unverified) allows.
/// Defaulted for wire-compat with clients/daemons predating it.
#[serde(default)]
pub shared_by: String,
/// When it was shared with me, epoch seconds; 0 when unknown.
#[serde(default)]
pub shared_at: i64,
/// The invitation's signature did not check out against `shared_by`'s keys,
/// so the name may be forged. False when it verified *and* when it could not
/// be checked at all (no signature, no keys): only an actual mismatch is a
/// warning worth showing.
#[serde(default)]
pub shared_by_unverified: bool,
} }
/// One hit in a [`Request::Search`] result. Like [`DirEntry`] but carries the /// One hit in a [`Request::Search`] result. Like [`DirEntry`] but carries the
@@ -1109,6 +1202,35 @@ pub struct PhotoMonth {
pub count: usize, pub count: usize,
} }
/// How a photo re-date is going, or how the last one ended
/// ([`Response::RedateStatus`]). Cumulative for the run, so the same shape
/// serves progress and the final report.
#[derive(Serialize, Deserialize, Debug, Clone, Default, PartialEq, Eq)]
pub struct RedateSummary {
/// Photos examined.
pub examined: usize,
/// Photos whose name implies a capture time more than a day from the one
/// they carry — what the run would (or did) repair.
pub candidates: usize,
/// Photos re-uploaded with the right capture time, their original trashed.
pub redated: usize,
/// Photos that could not be downloaded, re-uploaded, or trashed. The
/// original is left alone in every failure case.
pub failed: usize,
/// Album memberships re-created on the new copies.
pub album_links: usize,
/// Bytes uploaded.
pub bytes: u64,
/// True when the run stopped early because it was cancelled.
pub cancelled: bool,
/// True when nothing was actually uploaded or trashed.
pub dry_run: bool,
/// Up to a handful of examples, as `(name, stored capture time, the time the
/// name implies)` — what a dry run is read for.
#[serde(default)]
pub samples: Vec<(String, i64, i64)>,
}
/// How a Google Photos Takeout import is going, or how the last one ended /// How a Google Photos Takeout import is going, or how the last one ended
/// ([`Response::ImportStatus`]). /// ([`Response::ImportStatus`]).
/// ///
@@ -1191,6 +1313,15 @@ pub struct PhotoItem {
/// tags it has not learned yet. /// tags it has not learned yet.
#[serde(default)] #[serde(default)]
pub favorite: bool, pub favorite: bool,
/// How many files this entry stands for: a shot stored as a RAW and a JPEG
/// is one entry and two files. Older daemons omit it, and `0` reads the same
/// as `1` — one file, nothing to indicate.
#[serde(default)]
pub group_size: u32,
/// Whether one of those files is a camera raw. The tile badges it, because
/// the JPEG it shows is not the whole of what is stored.
#[serde(default)]
pub has_raw: bool,
} }
/// A daemon too old to classify a timeline entry is assumed to have served a /// A daemon too old to classify a timeline entry is assumed to have served a
@@ -1199,6 +1330,13 @@ fn default_photo_kind() -> PhotoKind {
PhotoKind::Photo PhotoKind::Photo
} }
/// One node a [`Request::TrashNodes`] batch could not trash, and why.
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct TrashFailure {
pub uid: String,
pub message: String,
}
/// One thumbnail in a [`Response::Thumbs`] batch. /// One thumbnail in a [`Response::Thumbs`] batch.
#[derive(Serialize, Deserialize, Debug, Clone)] #[derive(Serialize, Deserialize, Debug, Clone)]
pub struct PhotoThumb { pub struct PhotoThumb {
@@ -1318,6 +1456,55 @@ pub fn pending_summary(uploads: u64, changes: u64) -> Option<String> {
} }
} }
/// One worker thread in a [`Diagnostics`] report.
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct WorkerState {
/// Thread name (`pdfs-fuse-meta-0`, `pdfs-fuse-3`).
pub name: String,
pub busy: bool,
/// The job in hand (`read`, `lookup`), empty when idle.
pub label: String,
/// How long it has been in that state. A busy worker with a large age is
/// the shape of a hang.
pub age_secs: u64,
}
/// One control request the daemon is still serving, in a [`Diagnostics`] report.
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct InflightRequest {
/// The request's variant name (`OpenFile`, `PhotosTimeline`).
pub kind: String,
pub age_secs: u64,
}
/// What the daemon is doing right now (reply to [`Request::Diagnostics`]).
#[derive(Serialize, Deserialize, Debug, Clone, Default)]
pub struct Diagnostics {
/// How long this daemon process has been running.
pub uptime_secs: u64,
/// Resident set size, or `0` where it could not be read. The 2026-09-18 hang
/// peaked at 5.2 GiB, so this is the number that says whether the daemon is
/// growing again.
pub rss_bytes: u64,
pub workers: Vec<WorkerState>,
/// Jobs waiting for a worker, per lane.
pub meta_queued: usize,
pub transfer_queued: usize,
/// Jobs finished per lane since start. Queue depth that does not move while
/// these do not either is a stalled lane, not a loaded one.
pub meta_completed: u64,
pub transfer_completed: u64,
/// Control requests still being served, oldest first.
pub inflight: Vec<InflightRequest>,
/// Control handler threads in use, and the ceiling. At the ceiling the
/// daemon refuses new connections, which looks exactly like a dead daemon
/// from the CLI.
pub control_handlers: usize,
pub control_handler_limit: usize,
/// Queued mutations not yet drained.
pub pending_ops: u64,
}
/// The daemon's reply to a [`Request`]. /// The daemon's reply to a [`Request`].
#[derive(Serialize, Deserialize, Debug)] #[derive(Serialize, Deserialize, Debug)]
pub enum Response { pub enum Response {
@@ -1426,6 +1613,8 @@ pub enum Response {
available: bool, available: bool,
items: Vec<AlbumInfo>, items: Vec<AlbumInfo>,
}, },
/// What the daemon is doing right now (reply to [`Request::Diagnostics`]).
Diagnostics { diagnostics: Diagnostics },
/// How the Takeout import is doing (reply to [`Request::ImportStatus`]). /// How the Takeout import is doing (reply to [`Request::ImportStatus`]).
/// `running` is false once it has finished, when `summary` is the final /// `running` is false once it has finished, when `summary` is the final
/// report; `summary` is `None` when no import has run this session. /// report; `summary` is `None` when no import has run this session.
@@ -1434,11 +1623,27 @@ pub enum Response {
#[serde(default)] #[serde(default)]
summary: Option<ImportSummary>, summary: Option<ImportSummary>,
}, },
/// How the photo re-date is doing (reply to [`Request::RedateStatus`]).
/// `running` is false once it has finished, when `summary` is the final
/// report; `summary` is `None` when no re-date has run this session.
RedateStatus {
running: bool,
#[serde(default)]
summary: Option<RedateSummary>,
},
/// The months the timeline spans (reply to [`Request::PhotoMonths`]), /// The months the timeline spans (reply to [`Request::PhotoMonths`]),
/// newest first. /// newest first.
PhotoMonths { months: Vec<PhotoMonth> }, PhotoMonths { months: Vec<PhotoMonth> },
/// Thumbnails for a [`Request::PhotoThumbs`] batch. /// Thumbnails for a [`Request::PhotoThumbs`] batch.
Thumbs { items: Vec<PhotoThumb> }, Thumbs { items: Vec<PhotoThumb> },
/// Per-node outcome of a [`Request::TrashNodes`] batch. Reported per uid
/// rather than as one verdict: the server answers each node separately, and
/// a front-end that removed all of them on a partial success would show a
/// photo as gone that is still there.
Trashed {
trashed: Vec<String>,
failed: Vec<TrashFailure>,
},
/// A [`Request::FileThumbs`] generation is no longer current. This is /// A [`Request::FileThumbs`] generation is no longer current. This is
/// retryable after reserving a fresh generation and must never be interpreted /// retryable after reserving a fresh generation and must never be interpreted
/// as a permanent "no thumbnail" verdict. /// as a permanent "no thumbnail" verdict.
@@ -1448,6 +1653,9 @@ pub enum Response {
FileThumbGeneration { generation: u64 }, FileThumbGeneration { generation: u64 },
/// Current recursive thumbnail-build progress. /// Current recursive thumbnail-build progress.
ThumbnailBuild { status: ThumbnailBuildStatus }, ThumbnailBuild { status: ThumbnailBuildStatus },
/// Whether a photos-timeline refresh is still running (reply to
/// [`Request::PhotosRefreshStatus`]).
PhotosRefresh { running: bool },
/// An on-disk path the front-end can open (e.g. a downloaded photo). /// An on-disk path the front-end can open (e.g. a downloaded photo).
FilePath { path: String }, FilePath { path: String },
/// Full-text search results (reply to [`Request::Search`]). /// Full-text search results (reply to [`Request::Search`]).
@@ -1777,6 +1985,62 @@ mod tests {
assert!(items[0].name.is_empty()); assert!(items[0].name.is_empty());
} }
/// The lightbox asks for one shot's files by the uid of the tile that was
/// clicked, and gets them back in the shape the gallery already paints.
#[test]
fn photo_group_requests_roundtrip() {
let request = Request::PhotoGroup {
uid: "volume~first".into(),
};
let line = serde_json::to_string(&request).unwrap();
assert_eq!(line, r#"{"PhotoGroup":{"uid":"volume~first"}}"#);
let decoded: Request = serde_json::from_str(&line).unwrap();
assert_eq!(line, serde_json::to_string(&decoded).unwrap());
}
/// The group fields are new, so a page from a daemon that predates them has
/// to keep painting: one file, nothing to badge.
#[test]
fn an_older_daemons_photo_has_no_group() {
let item: PhotoItem = serde_json::from_str(
r#"{"uid":"volume~first","capture_time":1700000000,"thumb_path":null,
"name":"a.jpg","ratio":1.5,"no_thumb":false}"#,
)
.unwrap();
assert_eq!(item.group_size, 0, "absent reads the same as one file");
assert!(!item.has_raw);
}
/// The gallery trashes by uid, and reports per-uid outcomes back: a partial
/// failure has to survive the wire so the GUI can put those tiles back.
#[test]
fn trash_requests_and_outcomes_roundtrip() {
let request = Request::TrashNodes {
uids: vec!["volume~first".into(), "volume~second".into()],
};
let line = serde_json::to_string(&request).unwrap();
assert!(!line.contains('\n'), "wire form must be a single line");
let decoded: Request = serde_json::from_str(&line).unwrap();
assert_eq!(line, serde_json::to_string(&decoded).unwrap());
let response = Response::Trashed {
trashed: vec!["volume~first".into()],
failed: vec![TrashFailure {
uid: "volume~second".into(),
message: "permission denied".into(),
}],
};
let line = serde_json::to_string(&response).unwrap();
let decoded: Response = serde_json::from_str(&line).unwrap();
let Response::Trashed { trashed, failed } = decoded else {
panic!("decoded the wrong response variant");
};
assert_eq!(trashed, ["volume~first"]);
assert_eq!(failed.len(), 1);
assert_eq!(failed[0].uid, "volume~second");
assert_eq!(failed[0].message, "permission denied");
}
#[test] #[test]
fn thumbnail_build_requests_and_progress_roundtrip() { fn thumbnail_build_requests_and_progress_roundtrip() {
let requests = [ let requests = [
@@ -1821,6 +2085,31 @@ mod tests {
assert_eq!(line, serde_json::to_string(&decoded).unwrap()); assert_eq!(line, serde_json::to_string(&decoded).unwrap());
} }
/// `full` defaults, so a front-end built against the older wire shape still
/// asks for the ordinary refresh rather than failing to parse.
#[test]
fn a_photos_refresh_scope_defaults_to_not_full() {
let scope: RefreshScope = serde_json::from_str(r#"{"Photos":{}}"#).unwrap();
assert_eq!(scope, RefreshScope::Photos { full: false });
let line = serde_json::to_string(&RefreshScope::Photos { full: true }).unwrap();
assert_eq!(line, r#"{"Photos":{"full":true}}"#);
}
/// A photos refresh is followed, not awaited, so the request that reports
/// its progress is part of the wire contract.
#[test]
fn photos_refresh_status_roundtrips() {
let line = serde_json::to_string(&Request::PhotosRefreshStatus).unwrap();
assert_eq!(line, r#""PhotosRefreshStatus""#);
let decoded: Request = serde_json::from_str(&line).unwrap();
assert_eq!(line, serde_json::to_string(&decoded).unwrap());
let line = serde_json::to_string(&Response::PhotosRefresh { running: true }).unwrap();
assert_eq!(line, r#"{"PhotosRefresh":{"running":true}}"#);
let decoded: Response = serde_json::from_str(&line).unwrap();
assert_eq!(line, serde_json::to_string(&decoded).unwrap());
}
#[test] #[test]
fn unified_search_request_and_response_roundtrip() { fn unified_search_request_and_response_roundtrip() {
let request = Request::SearchV2 { let request = Request::SearchV2 {
@@ -1991,7 +2280,10 @@ mod tests {
scope: RefreshScope::Trash, scope: RefreshScope::Trash,
}, },
Request::Refresh { Request::Refresh {
scope: RefreshScope::Photos, scope: RefreshScope::Photos { full: false },
},
Request::Refresh {
scope: RefreshScope::Photos { full: true },
}, },
]; ];
for req in reqs { for req in reqs {
+21
View File
@@ -182,12 +182,29 @@ impl Db {
// photo on someone else's volume is not in our timeline, and // photo on someone else's volume is not in our timeline, and
// favouriting one is not supported yet anyway. // favouriting one is not supported yet anyway.
favorite: false, favorite: false,
// Nor are album rows grouped: an album is a list the user
// made, and it shows exactly what they put in it.
group_size: 1,
has_raw: false,
}) })
})? })?
.collect::<rusqlite::Result<_>>()?; .collect::<rusqlite::Result<_>>()?;
Ok(rows) Ok(rows)
} }
/// Which albums hold `uid`, by album node uid. Read by the photo re-date,
/// which has to put the re-uploaded copy back into the albums the original
/// was in.
pub fn albums_of_photo(&self, uid: &str) -> Result<Vec<String>> {
let conn = self.read();
let mut stmt =
conn.prepare("SELECT DISTINCT album_uid FROM album_photos WHERE uid = ?1")?;
let rows = stmt
.query_map([uid], |r| r.get(0))?
.collect::<rusqlite::Result<_>>()?;
Ok(rows)
}
/// How many photos of `album_uid` are persisted. /// How many photos of `album_uid` are persisted.
pub fn album_photos_count(&self, album_uid: &str) -> Result<usize> { pub fn album_photos_count(&self, album_uid: &str) -> Result<usize> {
let conn = self.read(); let conn = self.read();
@@ -226,6 +243,10 @@ impl Db {
// photo on someone else's volume is not in our timeline, and // photo on someone else's volume is not in our timeline, and
// favouriting one is not supported yet anyway. // favouriting one is not supported yet anyway.
favorite: false, favorite: false,
// Nor are album rows grouped: an album is a list the user
// made, and it shows exactly what they put in it.
group_size: 1,
has_raw: false,
}) })
})? })?
.collect::<rusqlite::Result<_>>()?; .collect::<rusqlite::Result<_>>()?;
+107 -1
View File
@@ -9,7 +9,7 @@ use super::Db;
use crate::Result; use crate::Result;
/// Current schema version. Bump on every forward migration added below. /// Current schema version. Bump on every forward migration added below.
pub(super) const SCHEMA_VERSION: i64 = 28; pub(super) const SCHEMA_VERSION: i64 = 31;
impl Db { impl Db {
pub(super) fn migrate(&self) -> Result<()> { pub(super) fn migrate(&self) -> Result<()> {
@@ -218,6 +218,58 @@ impl Db {
tx.execute_batch(MIGRATION_V28)?; tx.execute_batch(MIGRATION_V28)?;
} }
} }
if current < 29 {
// Same guards as V26/V27/V28: an old fixture may not have `trash`,
// and a fixture rewound from the current schema already has the
// column.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('trash') WHERE name = 'parent_uid'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_trash: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'trash'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_trash && !has_column {
tx.execute_batch(MIGRATION_V29)?;
}
}
if current < 30 {
// Same guards as V26-V29: the column may already be there on a
// fixture rewound from the current schema, and a very old fixture
// may not have `photos` at all.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('photos') WHERE name = 'group_key'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_photos: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'photos'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_photos && !has_column {
tx.execute_batch(MIGRATION_V30)?;
}
}
if current < 31 {
// Same guards as V26-V30.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('photos') WHERE name = 'resolved_at'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_photos: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'photos'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_photos && !has_column {
tx.execute_batch(MIGRATION_V31)?;
}
}
tx.execute( tx.execute(
"INSERT INTO sync_state (key, value) VALUES ('schema_version', ?1) "INSERT INTO sync_state (key, value) VALUES ('schema_version', ?1)
ON CONFLICT(key) DO UPDATE SET value = excluded.value", ON CONFLICT(key) DO UPDATE SET value = excluded.value",
@@ -846,3 +898,57 @@ ALTER TABLE pending_op ADD COLUMN access_deferred_since INTEGER NOT NULL DEFAULT
const MIGRATION_V28: &str = " const MIGRATION_V28: &str = "
ALTER TABLE sync_entry ADD COLUMN local_mtime_ns INTEGER; ALTER TABLE sync_entry ADD COLUMN local_mtime_ns INTEGER;
"; ";
/// Schema v29: the trash listing remembers each trashed node's parent, so a
/// restore can reason about the shape of what it is putting back.
///
/// Trash is a flat list of uids on the wire, but the user deletes a *tree*: a
/// folder and, often, items inside it that were trashed separately. Restoring
/// one of those in isolation is what the parent link fixes — a child restored
/// under a still-trashed parent lands somewhere invisible, and a folder restored
/// without its separately-trashed contents comes back empty.
///
/// `NULL` means "this row predates the column" (or the node is a volume root);
/// the next trash refresh fills it in, and until then the restore behaves as it
/// did before.
const MIGRATION_V29: &str = "
ALTER TABLE trash ADD COLUMN parent_uid TEXT;
CREATE INDEX IF NOT EXISTS idx_trash_parent ON trash(parent_uid);
";
/// Schema v30: a RAW and the JPEG of the same shot are one photo to the person
/// who took it, so the timeline records what ties them together.
///
/// `content_hash` and `main_uid` come from the server's own photo properties —
/// the duplicate-detection hash, and the main photo a related one belongs to.
/// `group_key` is what the gallery pages by: every member of one shot carries
/// the same key, and the member the grid shows is the one whose `main_uid` is
/// its own uid.
///
/// All three are `NULL` until the next timeline refresh resolves the photo's
/// node, and a row without a `group_key` is its own group — which is exactly how
/// the gallery behaved before this column existed.
const MIGRATION_V30: &str = "
ALTER TABLE photos ADD COLUMN content_hash TEXT;
ALTER TABLE photos ADD COLUMN main_uid TEXT;
ALTER TABLE photos ADD COLUMN group_key TEXT;
CREATE INDEX IF NOT EXISTS idx_photos_group ON photos(group_key);
";
/// Schema v31: which photos a refresh still has to read from the server.
///
/// A refresh used to resolve every photo's node to learn its name, media type,
/// favourite tag, content hash and photo relation — minutes of round-trips for a
/// library of any size, repeated every time the timeline went stale. `resolved_at`
/// records that the work was already done, so a refresh only reads the photos it
/// has never read and the ones a remote event marked stale (`resolved_at = NULL`).
///
/// The backfill treats an existing timeline as resolved, so an upgrade does not
/// re-read the whole library once. `WHERE name IS NOT NULL` keeps the photos
/// whose earlier resolve had failed eligible: a nameless row is one nothing was
/// ever learned about.
const MIGRATION_V31: &str = "
ALTER TABLE photos ADD COLUMN resolved_at INTEGER;
CREATE INDEX IF NOT EXISTS idx_photos_unresolved ON photos(resolved_at);
UPDATE photos SET resolved_at = strftime('%s','now') WHERE name IS NOT NULL;
";
+448 -75
View File
@@ -34,67 +34,347 @@ pub struct StoredPhoto {
pub kind: crate::control::PhotoKind, pub kind: crate::control::PhotoKind,
/// Whether the photo carries Proton's `Favorite` tag. /// Whether the photo carries Proton's `Favorite` tag.
pub favorite: bool, pub favorite: bool,
/// How many photos this photo's group holds — one shot taken as RAW+JPEG is
/// two files and one group. `1` for a photo that stands on its own.
pub group_size: usize,
/// Whether any member of the group is a camera raw file. The grid badges
/// that, because the JPEG it shows is not the whole of what is stored.
pub has_raw: bool,
} }
/// One row of a timeline replacement: `(uid, capture_time, name, media_type, /// One row of a timeline replacement. Everything but the uid and the capture
/// favorite)`. The two trailing options are "what this refresh learned" — a /// time is "what this refresh learned": a `None` keeps whatever is already
/// `None` keeps whatever is already stored rather than clearing it. /// stored rather than clearing it, because the timeline DTO carries only the uid
pub type TimelineRow = (String, i64, Option<String>, Option<String>, Option<bool>); /// and the capture time and resolving the node can fail.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct TimelineRow {
pub uid: String,
pub capture_time: i64,
pub name: Option<String>,
pub media_type: Option<String>,
pub favorite: Option<bool>,
/// The server's duplicate-detection hash for the plaintext.
pub content_hash: Option<String>,
/// The main photo this one is *related* to (a live-photo video, a burst
/// sibling), when the server says it belongs to one.
pub main_uid: Option<String>,
/// When this refresh resolved the photo's node, if it did at all. `Some`
/// makes every other field authoritative — including a `None` that means
/// "the server no longer reports one" — while `None` means the refresh
/// skipped this photo and whatever is stored must be kept.
pub resolved_at: Option<i64>,
}
impl TimelineRow {
/// A row carrying only what the timeline listing itself gives.
pub fn new(uid: impl Into<String>, capture_time: i64) -> Self {
Self {
uid: uid.into(),
capture_time,
..Self::default()
}
}
}
/// What a previous refresh learned about a photo and a new one must not lose.
#[derive(Clone, Debug, Default)]
struct Learned {
name: Option<String>,
ratio: Option<f64>,
thumb_state: i64,
media_type: Option<String>,
favorite: bool,
content_hash: Option<String>,
main_uid: Option<String>,
/// When this photo's node was last resolved, kept so a refresh that skips it
/// does not make it look unresolved again.
resolved_at: Option<i64>,
}
/// The local calendar day a capture time falls on, as a `YYYY-MM-DD` string.
/// Grouping by day rather than by exact second is deliberate: a camera writes
/// the RAW and the JPEG of one shot a moment apart, and two shots of the same
/// subject on different days are not one photo.
fn capture_day(conn: &rusqlite::Connection, capture_time: i64) -> Option<String> {
conn.query_row(
"SELECT strftime('%Y-%m-%d', ?1, 'unixepoch', 'localtime')",
params![capture_time],
|r| r.get::<_, Option<String>>(0),
)
.ok()
.flatten()
}
/// The file name without its extension, lowercased — `IMG_1234.CR2` and
/// `IMG_1234.JPG` share `img_1234` — with the camera's own part-of-a-shot
/// marker removed, so the two files a raw capture writes share a stem too.
fn name_stem(name: Option<&str>) -> Option<String> {
let name = name?;
let stem = match name.rsplit_once('.') {
Some((stem, _)) if !stem.is_empty() => stem,
_ => name,
};
let stem = stem.to_ascii_lowercase();
Some(strip_shot_marker(&stem).to_string())
}
/// Drop a `.RAW-NN.ROLE` marker from a stem.
///
/// Pixel stores one raw capture as `PXL_20260919_000625670.RAW-01.COVER.jpg`
/// and `PXL_20260919_000625670.RAW-02.ORIGINAL.dng`: one shot, two files, two
/// stems that differ in exactly the part naming which file of the shot this is.
/// Cutting the marker off is what lets rule 2 see the pair. The check is narrow
/// on purpose — digits, then a single role word — so an ordinary name that
/// happens to contain `.raw-` keeps its stem.
fn strip_shot_marker(stem: &str) -> &str {
let Some(at) = stem.rfind(".raw-") else {
return stem;
};
let rest = &stem[at + ".raw-".len()..];
let Some((index, role)) = rest.split_once('.') else {
return stem;
};
let marked = !index.is_empty()
&& index.bytes().all(|b| b.is_ascii_digit())
&& !role.is_empty()
&& role.bytes().all(|b| b.is_ascii_alphabetic());
if marked { &stem[..at] } else { stem }
}
/// One entry of the grouping pass: what rule 2 needs, plus what choosing a
/// representative needs.
struct Grouped {
uid: String,
kind: crate::control::PhotoKind,
main_uid: Option<String>,
day: Option<String>,
stem: Option<String>,
}
/// Which group each photo belongs to, and which member of it the gallery shows.
///
/// Precedence, as agreed:
///
/// 1. The server's own relation — a photo with a `main_photo_uid` joins that
/// photo's group. The server knows about live photos and bursts; we do not
/// have to guess at them.
/// 2. Otherwise, same capture day **and** same file-name stem **and** one of the
/// two is a camera raw while the other is not. That is the RAW+JPEG pair a
/// camera writes, and the differing-kind clause keeps two JPEGs that happen
/// to share a name apart.
/// 3. Otherwise the photo is its own group.
///
/// The representative is the non-raw member when the group has one — a JPEG
/// decodes in milliseconds and is what the person expects to see — and the
/// earliest member in server order otherwise.
fn group_photos(entries: &[Grouped]) -> Vec<String> {
let mut parent: Vec<usize> = (0..entries.len()).collect();
fn find(parent: &mut [usize], mut i: usize) -> usize {
while parent[i] != i {
parent[i] = parent[parent[i]];
i = parent[i];
}
i
}
let union = |parent: &mut [usize], a: usize, b: usize| {
let (a, b) = (find(parent, a), find(parent, b));
if a != b {
parent[b] = a;
}
};
let index: HashMap<&str, usize> = entries
.iter()
.enumerate()
.map(|(i, e)| (e.uid.as_str(), i))
.collect();
for (i, entry) in entries.iter().enumerate() {
if let Some(main) = entry.main_uid.as_deref()
&& let Some(&j) = index.get(main)
{
union(&mut parent, j, i);
}
}
// Rule 2 needs only one pass: everything sharing a day and a stem lands in
// the same bucket, and the raw / non-raw split is checked pairwise there.
let mut by_name: HashMap<(&str, &str), Vec<usize>> = HashMap::new();
for (i, entry) in entries.iter().enumerate() {
if let (Some(day), Some(stem)) = (entry.day.as_deref(), entry.stem.as_deref()) {
by_name.entry((day, stem)).or_default().push(i);
}
}
let is_raw = |i: usize| entries[i].kind == crate::control::PhotoKind::Raw;
for bucket in by_name.values() {
for (n, &i) in bucket.iter().enumerate() {
for &j in &bucket[n + 1..] {
if is_raw(i) != is_raw(j) {
union(&mut parent, i, j);
}
}
}
}
// Representative per root, then the key every member carries.
let mut representative: HashMap<usize, usize> = HashMap::new();
for i in 0..entries.len() {
let root = find(&mut parent, i);
match representative.get(&root).copied() {
Some(current) if !is_raw(current) || is_raw(i) => {}
_ => {
representative.insert(root, i);
}
}
}
(0..entries.len())
.map(|i| {
let root = find(&mut parent, i);
entries[representative[&root]].uid.clone()
})
.collect()
}
/// The columns every photo read selects, with the group aggregates joined in.
/// `g.n` is how many photos the group holds and `g.raw` whether any of them is a
/// camera raw file — one query rather than a second lookup per tile.
///
/// `COALESCE(group_key, uid)` is what makes a row written before schema v30 its
/// own group: the column is `NULL` until the next refresh computes it, and a
/// `NULL` would otherwise match nothing and drop the photo off the page.
const PHOTO_SELECT: &str = "SELECT p.uid, p.capture_time, p.name, p.ratio, p.thumb_state, \
p.kind, p.favorite, g.n, g.raw \
FROM photos p JOIN (SELECT COALESCE(group_key, uid) AS k, COUNT(*) AS n, \
MAX(kind = 2) AS raw FROM photos GROUP BY k) g ON g.k = COALESCE(p.group_key, p.uid)";
/// Read one row of [`PHOTO_SELECT`].
fn stored_photo(r: &rusqlite::Row<'_>) -> rusqlite::Result<StoredPhoto> {
Ok(StoredPhoto {
uid: r.get(0)?,
capture_time: r.get(1)?,
name: r.get(2)?,
ratio: r.get(3)?,
thumb_state: r.get(4)?,
kind: crate::control::PhotoKind::from_i64(r.get(5)?),
favorite: r.get::<_, i64>(6)? != 0,
group_size: r.get::<_, i64>(7)?.max(1) as usize,
has_raw: r.get::<_, i64>(8)? != 0,
})
}
impl Db { impl Db {
/// Replace the timeline wholesale. Each item is /// Replace the timeline wholesale. A row whose `resolved_at` is `Some` was
/// `(uid, capture_time, name, media_type, favorite)`; `favorite` is `None` /// read from the server in this pass, so its fields are taken as they are —
/// when the refresh could not resolve that photo's node, in which case the /// a `None` there means the server no longer reports that value. A row whose
/// flag already stored is kept rather than silently cleared. /// `resolved_at` is `None` was skipped or failed to resolve, and keeps
/// whatever is already stored rather than having it silently cleared.
pub fn photos_replace(&self, items: &[TimelineRow]) -> Result<()> { pub fn photos_replace(&self, items: &[TimelineRow]) -> Result<()> {
let mut conn = self.conn.lock(); let mut conn = self.conn.lock();
let tx = conn.transaction()?; let tx = conn.transaction()?;
// `media_type` is learned-and-kept like the ratio and thumb verdict: the // `name` and `media_type` are learned-and-kept like the ratio and thumb
// timeline DTO carries only the uid and capture time, so the daemon may // verdict: the timeline DTO carries only the uid and capture time, so the
// not know a photo's media type yet when it replaces the timeline. Keep // daemon may not know them when it replaces the timeline — because the
// any previously learned value so the Photos/Videos/Raw split survives a // refresh skipped an already-resolved photo, or because resolving it
// refresh instead of collapsing back to name-extension guesses. // failed. Keep any previously learned value so the tab split and the
let learned: HashMap<String, (Option<f64>, i64, Option<String>, bool)> = { // RAW+JPEG grouping survive a refresh instead of collapsing to nothing:
let mut stmt = // both are derived from the name. The photo relation is kept for the same
tx.prepare("SELECT uid, ratio, thumb_state, media_type, favorite FROM photos")?; // reason — losing it would break a group up until the next resolve.
let learned: HashMap<String, Learned> = {
let mut stmt = tx.prepare(
"SELECT uid, name, ratio, thumb_state, media_type, favorite, content_hash, \
main_uid, resolved_at FROM photos",
)?;
let rows = stmt.query_map([], |r| { let rows = stmt.query_map([], |r| {
Ok(( Ok((
r.get::<_, String>(0)?, r.get::<_, String>(0)?,
(r.get(1)?, r.get(2)?, r.get(3)?, r.get::<_, i64>(4)? != 0), Learned {
name: r.get(1)?,
ratio: r.get(2)?,
thumb_state: r.get(3)?,
media_type: r.get(4)?,
favorite: r.get::<_, i64>(5)? != 0,
content_hash: r.get(6)?,
main_uid: r.get(7)?,
resolved_at: r.get(8)?,
},
)) ))
})?; })?;
rows.collect::<rusqlite::Result<_>>()? rows.collect::<rusqlite::Result<_>>()?
}; };
// Everything the insert and the grouping need, resolved once per photo.
let resolved: Vec<(&TimelineRow, Learned, crate::control::PhotoKind)> = items
.iter()
.map(|row| {
let mut carried = learned.get(&row.uid).cloned().unwrap_or(Learned {
thumb_state: THUMB_UNKNOWN,
..Learned::default()
});
if row.resolved_at.is_some() {
// This pass read the node itself, so what it says is the
// truth — including what it no longer says. A photo unlinked
// from its main on another device loses its `main_uid` here,
// which a carry could never express.
carried.name = row.name.clone();
carried.media_type = row.media_type.clone();
carried.favorite = row.favorite.unwrap_or(false);
carried.content_hash = row.content_hash.clone();
carried.main_uid = row.main_uid.clone();
} else {
carried.name = row.name.clone().or(carried.name);
carried.media_type = row.media_type.clone().or(carried.media_type);
carried.favorite = row.favorite.unwrap_or(carried.favorite);
carried.content_hash = row.content_hash.clone().or(carried.content_hash);
carried.main_uid = row.main_uid.clone().or(carried.main_uid);
}
carried.resolved_at = row.resolved_at.or(carried.resolved_at);
// The tab this photo lands in is derived here, once, so a page or
// count query is a plain indexed `WHERE kind = ?` rather than a
// reclassification of every row.
let kind = crate::control::PhotoKind::classify(
carried.name.as_deref(),
carried.media_type.as_deref(),
);
(row, carried, kind)
})
.collect();
let entries: Vec<Grouped> = resolved
.iter()
.map(|(row, carried, kind)| Grouped {
uid: row.uid.clone(),
kind: *kind,
main_uid: carried.main_uid.clone(),
day: capture_day(&tx, row.capture_time),
stem: name_stem(carried.name.as_deref()),
})
.collect();
let group_keys = group_photos(&entries);
tx.execute("DELETE FROM photos", [])?; tx.execute("DELETE FROM photos", [])?;
{ {
let mut stmt = tx.prepare( let mut stmt = tx.prepare(
"INSERT INTO photos "INSERT INTO photos
(uid, capture_time, name, ratio, thumb_state, seq, media_type, kind, favorite) (uid, capture_time, name, ratio, thumb_state, seq, media_type, kind, favorite,
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)", content_hash, main_uid, group_key, resolved_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13)",
)?; )?;
for (seq, (uid, capture_time, name, media_type, favorite)) in items.iter().enumerate() { for (seq, ((row, carried, kind), group_key)) in
let (ratio, thumb_state, learned_media, learned_favorite) = learned resolved.iter().zip(group_keys.iter()).enumerate()
.get(uid) {
.cloned()
.unwrap_or((None, THUMB_UNKNOWN, None, false));
let media_type = media_type.clone().or(learned_media);
let favorite = favorite.unwrap_or(learned_favorite);
// The tab this photo lands in is derived here, once, so a page or
// count query is a plain indexed `WHERE kind = ?` rather than a
// reclassification of every row.
let kind =
crate::control::PhotoKind::classify(name.as_deref(), media_type.as_deref());
stmt.execute(params![ stmt.execute(params![
uid, row.uid,
capture_time, row.capture_time,
name, carried.name,
ratio, carried.ratio,
thumb_state, carried.thumb_state,
seq as i64, seq as i64,
media_type, carried.media_type,
kind.as_i64(), kind.as_i64(),
favorite as i64, carried.favorite as i64,
carried.content_hash,
carried.main_uid,
group_key,
carried.resolved_at,
])?; ])?;
} }
} }
@@ -102,11 +382,84 @@ impl Db {
Ok(()) Ok(())
} }
/// The uids whose node metadata is already resolved, so a refresh can skip
/// reading their nodes again. Everything else — a photo never seen before, or
/// one a remote event marked stale — is resolved. That is what keeps a
/// 12k-photo library off 63 round-trips of `enumerate_nodes` every refresh.
pub fn photos_resolved(&self) -> Result<Vec<String>> {
let conn = self.read();
let mut stmt = conn.prepare("SELECT uid FROM photos WHERE resolved_at IS NOT NULL")?;
let rows = stmt.query_map([], |r| r.get::<_, String>(0))?;
let mut uids = Vec::new();
for row in rows {
uids.push(row?);
}
Ok(uids)
}
/// Mark photos as needing a fresh metadata resolve, and report how many rows
/// that matched. This is how a remote change is recorded: the event feed says
/// *which* node changed but never *what* changed about it, so the uid is
/// written down and the next refresh reads the node.
pub fn photos_mark_unresolved(&self, uids: &[String]) -> Result<usize> {
if uids.is_empty() {
return Ok(0);
}
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let mut marked = 0;
{
let mut stmt = tx.prepare("UPDATE photos SET resolved_at = NULL WHERE uid = ?1")?;
for uid in uids {
marked += stmt.execute(params![uid])?;
}
}
tx.commit()?;
Ok(marked)
}
/// The same for the whole timeline: every photo is resolved again on the next
/// refresh. For a continuity loss, where the daemon cannot know what it
/// missed, and for an explicit `refresh photos --full`.
pub fn photos_mark_all_unresolved(&self) -> Result<usize> {
let conn = self.conn.lock();
Ok(conn.execute("UPDATE photos SET resolved_at = NULL", [])?)
}
/// Forget photos that have been trashed, so the gallery does not show them
/// until the next timeline refresh would have caught up.
///
/// Album membership goes with them: a trashed photo is gone from every album
/// that held it, and leaving the rows behind would keep it on the album
/// pages and in the album covers.
pub fn photos_delete(&self, uids: &[String]) -> Result<usize> {
if uids.is_empty() {
return Ok(0);
}
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let mut removed = 0;
{
let mut photos = tx.prepare("DELETE FROM photos WHERE uid = ?1")?;
let mut albums = tx.prepare("DELETE FROM album_photos WHERE uid = ?1")?;
for uid in uids {
removed += photos.execute([uid])?;
albums.execute([uid])?;
}
}
tx.commit()?;
Ok(removed)
}
/// One page of the persisted timeline, newest first. `kind`, when set, /// One page of the persisted timeline, newest first. `kind`, when set,
/// restricts the page to one tab (Photos / Videos / Raw); `range`, when set, /// restricts the page to one tab (Photos / Videos / Raw); `range`, when set,
/// restricts it to a `[from, to)` capture-time window (epoch seconds) — the /// restricts it to a `[from, to)` capture-time window (epoch seconds) — the
/// date scrubber's jump. `offset` is relative to whatever the filters leave. /// date scrubber's jump. `offset` is relative to whatever the filters leave.
/// `favorites` restricts the page to favourited photos. /// `favorites` restricts the page to favourited photos.
///
/// The page holds one row per *group*: the RAW and the JPEG of one shot are
/// one tile. The Raw tab is the exception and lists every raw file, because
/// that tab exists to answer "which raws do I have".
pub fn photos_page( pub fn photos_page(
&self, &self,
offset: usize, offset: usize,
@@ -118,23 +471,24 @@ impl Db {
let conn = self.read(); let conn = self.read();
// Built up so any combination of the optional filters is one indexed // Built up so any combination of the optional filters is one indexed
// query rather than a statement per case. // query rather than a statement per case.
let mut sql = String::from( let mut sql = String::from(PHOTO_SELECT);
"SELECT uid, capture_time, name, ratio, thumb_state, kind, favorite FROM photos",
);
let mut binds: Vec<i64> = Vec::new(); let mut binds: Vec<i64> = Vec::new();
let mut conds: Vec<String> = Vec::new(); let mut conds: Vec<String> = Vec::new();
if kind != Some(crate::control::PhotoKind::Raw) {
conds.push("p.uid = COALESCE(p.group_key, p.uid)".to_string());
}
if let Some(k) = kind { if let Some(k) = kind {
binds.push(k.as_i64()); binds.push(k.as_i64());
conds.push(format!("kind = ?{}", binds.len())); conds.push(format!("p.kind = ?{}", binds.len()));
} }
if let Some((from, to)) = range { if let Some((from, to)) = range {
binds.push(from); binds.push(from);
conds.push(format!("capture_time >= ?{}", binds.len())); conds.push(format!("p.capture_time >= ?{}", binds.len()));
binds.push(to); binds.push(to);
conds.push(format!("capture_time < ?{}", binds.len())); conds.push(format!("p.capture_time < ?{}", binds.len()));
} }
if favorites { if favorites {
conds.push("favorite = 1".to_string()); conds.push("p.favorite = 1".to_string());
} }
if !conds.is_empty() { if !conds.is_empty() {
sql.push_str(" WHERE "); sql.push_str(" WHERE ");
@@ -145,22 +499,41 @@ impl Db {
binds.push(offset as i64); binds.push(offset as i64);
let offset_pos = binds.len(); let offset_pos = binds.len();
sql.push_str(&format!( sql.push_str(&format!(
" ORDER BY seq LIMIT ?{limit_pos} OFFSET ?{offset_pos}" " ORDER BY p.seq LIMIT ?{limit_pos} OFFSET ?{offset_pos}"
)); ));
let mut stmt = conn.prepare(&sql)?; let mut stmt = conn.prepare(&sql)?;
let rows = stmt let rows = stmt
.query_map(rusqlite::params_from_iter(binds), |r| { .query_map(rusqlite::params_from_iter(binds), stored_photo)?
Ok(StoredPhoto { .collect::<rusqlite::Result<_>>()?;
uid: r.get(0)?, Ok(rows)
capture_time: r.get(1)?, }
name: r.get(2)?,
ratio: r.get(3)?, /// Every photo of one photo's group, in server order — the RAW and the JPEG
thumb_state: r.get(4)?, /// of a shot, or just the photo itself when it stands alone. The lightbox
kind: crate::control::PhotoKind::from_i64(r.get(5)?), /// switches between these, and deleting a tile trashes all of them.
favorite: r.get::<_, i64>(6)? != 0, pub fn photos_group(&self, uid: &str) -> Result<Vec<StoredPhoto>> {
}) let conn = self.read();
})? let sql = format!(
"{PHOTO_SELECT} WHERE COALESCE(p.group_key, p.uid) = \
(SELECT COALESCE(group_key, uid) FROM photos WHERE uid = ?1) ORDER BY p.seq"
);
let mut stmt = conn.prepare(&sql)?;
let rows = stmt
.query_map(params![uid], stored_photo)?
.collect::<rusqlite::Result<_>>()?;
Ok(rows)
}
/// The whole persisted timeline, in stored order — every file, not one row
/// per group. Only for whole-library passes (the photo re-date); the gallery
/// pages with [`Db::photos_page`].
pub fn photos_all(&self) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
let sql = format!("{PHOTO_SELECT} ORDER BY p.seq");
let mut stmt = conn.prepare(&sql)?;
let rows = stmt
.query_map([], stored_photo)?
.collect::<rusqlite::Result<_>>()?; .collect::<rusqlite::Result<_>>()?;
Ok(rows) Ok(rows)
} }
@@ -175,9 +548,15 @@ impl Db {
kind: Option<crate::control::PhotoKind>, kind: Option<crate::control::PhotoKind>,
) -> Result<Vec<(i32, i32, usize)>> { ) -> Result<Vec<(i32, i32, usize)>> {
let conn = self.read(); let conn = self.read();
// The scrubber has to agree with the page it scrubs, so it counts what
// the page shows: groups everywhere except on the Raw tab.
let (filter, binds): (&str, Vec<i64>) = match kind { let (filter, binds): (&str, Vec<i64>) = match kind {
Some(k) => (" WHERE kind = ?1", vec![k.as_i64()]), Some(crate::control::PhotoKind::Raw) => (" WHERE kind = 2", Vec::new()),
None => ("", Vec::new()), Some(k) => (
" WHERE uid = COALESCE(group_key, uid) AND kind = ?1",
vec![k.as_i64()],
),
None => (" WHERE uid = COALESCE(group_key, uid)", Vec::new()),
}; };
let sql = format!( let sql = format!(
"SELECT CAST(strftime('%Y', capture_time, 'unixepoch', 'localtime') AS INTEGER) AS y, \ "SELECT CAST(strftime('%Y', capture_time, 'unixepoch', 'localtime') AS INTEGER) AS y, \
@@ -202,7 +581,13 @@ impl Db {
pub fn photos_counts(&self) -> Result<(usize, usize, usize)> { pub fn photos_counts(&self) -> Result<(usize, usize, usize)> {
use crate::control::PhotoKind; use crate::control::PhotoKind;
let conn = self.read(); let conn = self.read();
let mut stmt = conn.prepare("SELECT kind, COUNT(*) FROM photos GROUP BY kind")?; // Photos and Videos count groups — one shot is one entry, however many
// files it was stored as. Raw counts files, because the Raw tab lists
// them individually.
let mut stmt = conn.prepare(
"SELECT kind, COUNT(*) FROM photos \
WHERE uid = COALESCE(group_key, uid) OR kind = 2 GROUP BY kind",
)?;
let rows = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?)))?; let rows = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?)))?;
let (mut photos, mut videos, mut raw) = (0usize, 0usize, 0usize); let (mut photos, mut videos, mut raw) = (0usize, 0usize, 0usize);
for row in rows { for row in rows {
@@ -225,21 +610,9 @@ impl Db {
} }
let conn = self.read(); let conn = self.read();
let placeholders = vec!["?"; uids.len()].join(","); let placeholders = vec!["?"; uids.len()].join(",");
let mut stmt = conn.prepare(&format!( let sql = format!("{PHOTO_SELECT} WHERE p.uid IN ({placeholders})");
"SELECT uid, capture_time, name, ratio, thumb_state, kind, favorite FROM photos let mut stmt = conn.prepare(&sql)?;
WHERE uid IN ({placeholders})" let rows = stmt.query_map(rusqlite::params_from_iter(uids), stored_photo)?;
))?;
let rows = stmt.query_map(rusqlite::params_from_iter(uids), |r| {
Ok(StoredPhoto {
uid: r.get(0)?,
capture_time: r.get(1)?,
name: r.get(2)?,
ratio: r.get(3)?,
thumb_state: r.get(4)?,
kind: crate::control::PhotoKind::from_i64(r.get(5)?),
favorite: r.get::<_, i64>(6)? != 0,
})
})?;
let mut photos = Vec::new(); let mut photos = Vec::new();
for row in rows { for row in rows {
photos.push(row?); photos.push(row?);
+539 -11
View File
@@ -113,9 +113,13 @@ fn node_from(parent: serde_json::Value, link: &str, name: &str, kind: serde_json
fn photos_replace_keeps_what_was_learned_and_drops_what_left() { fn photos_replace_keeps_what_was_learned_and_drops_what_left() {
let db = Db::open_in_memory().unwrap(); let db = Db::open_in_memory().unwrap();
db.photos_replace(&[ db.photos_replace(&[
("p1".into(), 300, None, Some("video/mp4".into()), Some(true)), TimelineRow {
("p2".into(), 200, None, None, None), media_type: Some("video/mp4".into()),
("p3".into(), 100, None, None, None), favorite: Some(true),
..TimelineRow::new("p1", 300)
},
TimelineRow::new("p2", 200),
TimelineRow::new("p3", 100),
]) ])
.unwrap(); .unwrap();
@@ -126,9 +130,13 @@ fn photos_replace_keeps_what_was_learned_and_drops_what_left() {
// The next refresh brings a new photo, keeps p1 and p2, and loses p3. // The next refresh brings a new photo, keeps p1 and p2, and loses p3.
// p1's media type arrives as `None` this time and must not be forgotten. // p1's media type arrives as `None` this time and must not be forgotten.
db.photos_replace(&[ db.photos_replace(&[
("p0".into(), 400, Some("new.jpg".into()), None, Some(false)), TimelineRow {
("p1".into(), 300, None, None, None), name: Some("new.jpg".into()),
("p2".into(), 200, None, None, None), favorite: Some(false),
..TimelineRow::new("p0", 400)
},
TimelineRow::new("p1", 300),
TimelineRow::new("p2", 200),
]) ])
.unwrap(); .unwrap();
@@ -200,8 +208,14 @@ fn photos_replace_keeps_what_was_learned_and_drops_what_left() {
fn favorites_are_remembered_across_refreshes_and_filter_a_page() { fn favorites_are_remembered_across_refreshes_and_filter_a_page() {
let db = Db::open_in_memory().unwrap(); let db = Db::open_in_memory().unwrap();
db.photos_replace(&[ db.photos_replace(&[
("p1".into(), 300, None, None, Some(true)), TimelineRow {
("p2".into(), 200, None, None, Some(false)), favorite: Some(true),
..TimelineRow::new("p1", 300)
},
TimelineRow {
favorite: Some(false),
..TimelineRow::new("p2", 200)
},
]) ])
.unwrap(); .unwrap();
let favorites = db.photos_page(0, 10, None, None, true).unwrap(); let favorites = db.photos_page(0, 10, None, None, true).unwrap();
@@ -215,8 +229,11 @@ fn favorites_are_remembered_across_refreshes_and_filter_a_page() {
// (`None`), and loses to one that could. // (`None`), and loses to one that could.
db.photos_set_favorite("p2", true).unwrap(); db.photos_set_favorite("p2", true).unwrap();
db.photos_replace(&[ db.photos_replace(&[
("p1".into(), 300, None, None, Some(false)), TimelineRow {
("p2".into(), 200, None, None, None), favorite: Some(false),
..TimelineRow::new("p1", 300)
},
TimelineRow::new("p2", 200),
]) ])
.unwrap(); .unwrap();
let favorites = db.photos_page(0, 10, None, None, true).unwrap(); let favorites = db.photos_page(0, 10, None, None, true).unwrap();
@@ -232,7 +249,7 @@ fn favorites_are_remembered_across_refreshes_and_filter_a_page() {
fn photos_page_slices_the_timeline_in_order() { fn photos_page_slices_the_timeline_in_order() {
let db = Db::open_in_memory().unwrap(); let db = Db::open_in_memory().unwrap();
let items: Vec<_> = (0..5) let items: Vec<_> = (0..5)
.map(|i| (format!("p{i}"), 500 - i as i64, None, None::<String>, None)) .map(|i| TimelineRow::new(format!("p{i}"), 500 - i as i64))
.collect(); .collect();
db.photos_replace(&items).unwrap(); db.photos_replace(&items).unwrap();
@@ -254,6 +271,7 @@ fn trash_replace_lists_folders_first() {
is_dir: false, is_dir: false,
size: 10, size: 10,
mtime: 1, mtime: 1,
parent_uid: Some("t2".into()),
}, },
StoredTrash { StoredTrash {
uid: "t2".into(), uid: "t2".into(),
@@ -261,6 +279,7 @@ fn trash_replace_lists_folders_first() {
is_dir: true, is_dir: true,
size: 0, size: 0,
mtime: 2, mtime: 2,
parent_uid: None,
}, },
]) ])
.unwrap(); .unwrap();
@@ -271,6 +290,18 @@ fn trash_replace_lists_folders_first() {
["Alpha", "zeta.txt"] ["Alpha", "zeta.txt"]
); );
// The parent links come back with the rows: a restore reads the tree shape
// from them.
let mut parents = db.trash_parents().unwrap();
parents.sort();
assert_eq!(
parents,
[
("t1".to_string(), Some("t2".to_string())),
("t2".to_string(), None),
]
);
// A replace is a replace: emptying the trash on the server empties it here. // A replace is a replace: emptying the trash on the server empties it here.
db.trash_replace(&[]).unwrap(); db.trash_replace(&[]).unwrap();
assert!(db.trash_list().unwrap().is_empty()); assert!(db.trash_list().unwrap().is_empty());
@@ -3201,6 +3232,8 @@ fn test_db_has_children_and_trashed_filtering() {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
}; };
db.upsert_node(&parent_node).unwrap(); db.upsert_node(&parent_node).unwrap();
@@ -3227,6 +3260,8 @@ fn test_db_has_children_and_trashed_filtering() {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
}; };
db.upsert_node(&child_node).unwrap(); db.upsert_node(&child_node).unwrap();
@@ -3901,3 +3936,496 @@ fn an_idle_worker_does_not_wait_on_an_op_another_worker_has() {
"an op somebody else is draining is not work this worker is waiting for" "an op somebody else is draining is not work this worker is waiting for"
); );
} }
/// Trashing a photo from the gallery must not leave it behind on an album page:
/// the row goes, its album membership goes with it, and nothing else moves.
#[test]
fn photos_delete_removes_the_photo_and_its_album_membership() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow::new("p1", 300),
TimelineRow::new("p2", 200),
TimelineRow::new("p3", 100),
])
.unwrap();
db.albums_replace(&[StoredAlbum {
uid: "a1".into(),
name: "Trip".into(),
photo_count: 3,
cover_uid: Some("p1".into()),
last_activity: Some(500),
shared: false,
}])
.unwrap();
db.album_photos_replace(
"a1",
&[
("p1".into(), 300, None, None),
("p2".into(), 200, None, None),
("p3".into(), 100, None, None),
],
)
.unwrap();
assert_eq!(db.photos_delete(&["p2".into()]).unwrap(), 1);
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(
page.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["p1", "p3"],
"only the trashed photo leaves the timeline"
);
assert_eq!(db.album_photos_count("a1").unwrap(), 2);
let album = db.album_photos_page("a1", 0, 10).unwrap();
assert_eq!(
album.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["p1", "p3"]
);
// A uid we no longer hold is not an error: the event poller and the trash
// reply can both report the same deletion.
assert_eq!(db.photos_delete(&["p2".into()]).unwrap(), 0);
assert_eq!(db.photos_delete(&[]).unwrap(), 0);
}
/// A shot taken as RAW+JPEG is one photo to the person who took it: the grid
/// shows the JPEG once, and the Raw tab still lists the raw file.
#[test]
fn a_raw_and_its_jpeg_are_one_entry_in_the_grid() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_1234.JPG".into()),
..TimelineRow::new("jpeg", 300)
},
TimelineRow {
name: Some("img_1234.cr2".into()),
..TimelineRow::new("raw", 300)
},
TimelineRow {
name: Some("IMG_9999.JPG".into()),
..TimelineRow::new("alone", 200)
},
])
.unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(
page.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["jpeg", "alone"],
"the pair shows once, as the member that can be displayed"
);
assert_eq!(page[0].group_size, 2);
assert!(page[0].has_raw, "the tile can say the raw is there");
assert_eq!(page[1].group_size, 1);
assert!(!page[1].has_raw);
// The Raw tab answers "which raws do I have", so it lists files.
let raws = db
.photos_page(0, 10, Some(crate::control::PhotoKind::Raw), None, false)
.unwrap();
assert_eq!(
raws.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["raw"]
);
assert_eq!(db.photos_counts().unwrap(), (2, 0, 1));
// Both members are reachable from either uid: the lightbox switches between
// them, and deleting the tile has to trash both.
let group = db.photos_group("raw").unwrap();
assert_eq!(
group.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["jpeg", "raw"]
);
assert_eq!(db.photos_group("alone").unwrap().len(), 1);
}
/// Pixel names the two files of one raw capture after the role each plays, so
/// their stems differ. The shot is still one shot, and the grid shows the cover
/// JPEG.
#[test]
fn a_pixel_raw_capture_groups_with_its_cover_jpeg() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("PXL_20260919_000625670.RAW-01.COVER.jpg".into()),
..TimelineRow::new("cover", 1_789_776_385)
},
TimelineRow {
name: Some("PXL_20260919_000625670.RAW-02.ORIGINAL.dng".into()),
..TimelineRow::new("original", 1_789_776_385)
},
// A name that merely contains the marker's opening is not a marker.
TimelineRow {
name: Some("scan.raw-notes.jpg".into()),
..TimelineRow::new("notes", 1_789_776_385)
},
TimelineRow {
name: Some("scan.raw-other.dng".into()),
..TimelineRow::new("other", 1_789_776_385)
},
])
.unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(
page.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["cover", "notes", "other"],
"the capture shows once, as the cover JPEG"
);
assert_eq!(page[0].group_size, 2);
assert!(page[0].has_raw);
assert_eq!(
db.photos_group("original")
.unwrap()
.iter()
.map(|p| p.uid.as_str())
.collect::<Vec<_>>(),
["cover", "original"]
);
}
/// Two files that merely share a name are not one shot: the same stem on
/// different days, or two files of the same kind, stay apart.
#[test]
fn a_shared_name_alone_does_not_group_photos() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_1234.JPG".into()),
..TimelineRow::new("today", 1_700_000_000)
},
TimelineRow {
name: Some("IMG_1234.CR2".into()),
..TimelineRow::new("last-year", 1_600_000_000)
},
TimelineRow {
name: Some("IMG_1234.PNG".into()),
..TimelineRow::new("same-kind", 1_700_000_000)
},
])
.unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(page.len(), 3, "three shots, three tiles");
assert!(page.iter().all(|p| p.group_size == 1));
}
/// The server knows about live photos and bursts, so its own relation groups
/// files this client would never pair by name.
#[test]
fn the_servers_photo_relation_groups_a_live_photo() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_0001.HEIC".into()),
..TimelineRow::new("main", 300)
},
TimelineRow {
name: Some("clip.mov".into()),
main_uid: Some("main".into()),
..TimelineRow::new("related", 299)
},
])
.unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(
page.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["main"]
);
assert_eq!(page[0].group_size, 2);
assert!(!page[0].has_raw, "a live photo's video is not a raw file");
// The relation is learned-and-kept: a refresh that could not resolve the
// nodes must not break the group up.
db.photos_replace(&[
TimelineRow::new("main", 300),
TimelineRow::new("related", 299),
])
.unwrap();
assert_eq!(db.photos_group("related").unwrap().len(), 2);
}
/// A database written before grouping existed opens, keeps its timeline, and
/// shows every photo — a row with no `group_key` is its own group, which is how
/// the gallery behaved before the column.
#[test]
fn migration_v30_leaves_a_v29_timeline_intact() {
let path = std::env::temp_dir().join(format!(
"pdfs-db-v29-fixture-{}-{}.db",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
{
let db = Db::open(&path).unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_1234.JPG".into()),
..TimelineRow::new("jpeg", 300)
},
TimelineRow {
name: Some("IMG_1234.CR2".into()),
..TimelineRow::new("raw", 300)
},
])
.unwrap();
db.photo_set_thumb("jpeg", THUMB_HAVE, Some(1.5)).unwrap();
}
{
// Put the file back in the state a released V29 database was in.
let conn = rusqlite::Connection::open(&path).unwrap();
conn.execute_batch(
"DROP INDEX idx_photos_group;
ALTER TABLE photos DROP COLUMN group_key;
ALTER TABLE photos DROP COLUMN main_uid;
ALTER TABLE photos DROP COLUMN content_hash;
UPDATE sync_state SET value = '29' WHERE key = 'schema_version';",
)
.unwrap();
}
let db = Db::open(&path).unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(
page.iter().map(|p| p.uid.as_str()).collect::<Vec<_>>(),
["jpeg", "raw"],
"no photo goes missing before the next refresh groups them"
);
assert!(page.iter().all(|p| p.group_size == 1));
assert_eq!(page[0].ratio, Some(1.5), "what was learned is still there");
// The next refresh fills the column in and the pair collapses.
db.photos_replace(&[
TimelineRow {
name: Some("IMG_1234.JPG".into()),
..TimelineRow::new("jpeg", 300)
},
TimelineRow {
name: Some("IMG_1234.CR2".into()),
..TimelineRow::new("raw", 300)
},
])
.unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(page.len(), 1);
assert_eq!(page[0].uid, "jpeg");
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_file(format!("{}.lock", path.display()));
}
/// A refresh that skips an already-resolved photo sends a row carrying only the
/// uid and capture time. The name must survive that, because the tab split and
/// the grouping are both derived from it.
#[test]
fn photos_replace_keeps_a_skipped_photos_name() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_1234.JPG".into()),
media_type: Some("image/jpeg".into()),
resolved_at: Some(1),
..TimelineRow::new("jpeg", 300)
},
TimelineRow {
name: Some("IMG_1234.CR2".into()),
media_type: Some("application/octet-stream".into()),
resolved_at: Some(1),
..TimelineRow::new("raw", 300)
},
])
.unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(page.len(), 1, "one shot, two files");
assert!(page[0].has_raw);
// The next refresh resolves neither, because both are already resolved.
db.photos_replace(&[TimelineRow::new("jpeg", 300), TimelineRow::new("raw", 300)])
.unwrap();
let page = db.photos_page(0, 10, None, None, false).unwrap();
assert_eq!(page.len(), 1, "the group survives a skipped refresh");
assert_eq!(page[0].name.as_deref(), Some("IMG_1234.JPG"));
assert!(page[0].has_raw, "the raw file is still a raw file");
assert_eq!(page[0].group_size, 2);
}
/// Only the photos a refresh actually read are marked resolved; the rest stay
/// eligible, so the next refresh tries them again.
#[test]
fn photos_replace_marks_only_the_rows_it_resolved() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("ok.jpg".into()),
resolved_at: Some(1),
..TimelineRow::new("ok", 300)
},
// The chunk holding this one failed.
TimelineRow::new("failed", 200),
])
.unwrap();
assert_eq!(db.photos_resolved().unwrap(), ["ok"]);
// A resolved photo stays resolved across a refresh that skips it.
db.photos_replace(&[
TimelineRow::new("ok", 300),
TimelineRow {
name: Some("late.jpg".into()),
resolved_at: Some(2),
..TimelineRow::new("failed", 200)
},
])
.unwrap();
let mut resolved = db.photos_resolved().unwrap();
resolved.sort();
assert_eq!(resolved, ["failed", "ok"]);
}
/// What a refresh read is the truth, including what it no longer reports: a
/// photo unlinked from its main on another device leaves the group.
#[test]
fn a_resolved_replace_clears_a_relation_the_server_dropped() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_0001.HEIC".into()),
resolved_at: Some(1),
..TimelineRow::new("main", 300)
},
TimelineRow {
name: Some("clip.mov".into()),
main_uid: Some("main".into()),
resolved_at: Some(1),
..TimelineRow::new("related", 299)
},
])
.unwrap();
assert_eq!(db.photos_group("related").unwrap().len(), 2);
// Both are read again and the server no longer reports the relation.
db.photos_replace(&[
TimelineRow {
name: Some("IMG_0001.HEIC".into()),
resolved_at: Some(2),
..TimelineRow::new("main", 300)
},
TimelineRow {
name: Some("clip.mov".into()),
resolved_at: Some(2),
..TimelineRow::new("related", 299)
},
])
.unwrap();
assert_eq!(
db.photos_group("related").unwrap().len(),
1,
"an unlinked photo stands on its own"
);
}
/// A refresh that did *not* read the nodes must not break the group up, which is
/// the older behaviour and still the one a failed resolve gets.
#[test]
fn an_unresolved_replace_keeps_a_relation() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_0001.HEIC".into()),
resolved_at: Some(1),
..TimelineRow::new("main", 300)
},
TimelineRow {
name: Some("clip.mov".into()),
main_uid: Some("main".into()),
resolved_at: Some(1),
..TimelineRow::new("related", 299)
},
])
.unwrap();
db.photos_replace(&[
TimelineRow::new("main", 300),
TimelineRow::new("related", 299),
])
.unwrap();
assert_eq!(db.photos_group("related").unwrap().len(), 2);
}
/// A remote event names the photo it changed, and that is all it takes to have
/// the next refresh read that one node again.
#[test]
fn marking_a_photo_unresolved_makes_it_eligible_again() {
let db = Db::open_in_memory().unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("a.jpg".into()),
resolved_at: Some(1),
..TimelineRow::new("a", 300)
},
TimelineRow {
name: Some("b.jpg".into()),
resolved_at: Some(1),
..TimelineRow::new("b", 200)
},
])
.unwrap();
assert_eq!(db.photos_mark_unresolved(&["a".to_string()]).unwrap(), 1);
assert_eq!(db.photos_resolved().unwrap(), ["b"]);
// A uid the timeline does not hold changes nothing.
assert_eq!(db.photos_mark_unresolved(&["gone".to_string()]).unwrap(), 0);
assert_eq!(db.photos_mark_all_unresolved().unwrap(), 2);
assert!(db.photos_resolved().unwrap().is_empty());
}
/// An existing timeline opens as resolved, so upgrading does not re-read the
/// whole library once — except for the photos nothing was ever learned about.
#[test]
fn migration_v31_treats_an_existing_timeline_as_resolved() {
let path = std::env::temp_dir().join(format!(
"pdfs-db-v30-fixture-{}-{}.db",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
{
let db = Db::open(&path).unwrap();
db.photos_replace(&[
TimelineRow {
name: Some("IMG_1234.JPG".into()),
resolved_at: Some(1),
..TimelineRow::new("named", 300)
},
// A photo whose resolve had failed: nothing was ever learned.
TimelineRow::new("nameless", 200),
])
.unwrap();
}
{
// Put the file back in the state a released V30 database was in.
let conn = rusqlite::Connection::open(&path).unwrap();
conn.execute_batch(
"DROP INDEX idx_photos_unresolved;
ALTER TABLE photos DROP COLUMN resolved_at;
UPDATE sync_state SET value = '30' WHERE key = 'schema_version';",
)
.unwrap();
}
let db = Db::open(&path).unwrap();
assert_eq!(
db.photos_resolved().unwrap(),
["named"],
"the stored timeline is trusted, a nameless row is not"
);
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_file(format!("{}.lock", path.display()));
}
+24 -3
View File
@@ -11,6 +11,10 @@ pub struct StoredTrash {
pub is_dir: bool, pub is_dir: bool,
pub size: i64, pub size: i64,
pub mtime: i64, pub mtime: i64,
/// The folder this node was trashed from, when it is known. `None` for a
/// row written before schema v29, and for a node whose parent the server
/// did not report.
pub parent_uid: Option<String>,
} }
impl Db { impl Db {
@@ -20,7 +24,8 @@ impl Db {
tx.execute("DELETE FROM trash", [])?; tx.execute("DELETE FROM trash", [])?;
{ {
let mut stmt = tx.prepare( let mut stmt = tx.prepare(
"INSERT INTO trash (uid, name, is_dir, size, mtime) VALUES (?1, ?2, ?3, ?4, ?5)", "INSERT INTO trash (uid, name, is_dir, size, mtime, parent_uid) \
VALUES (?1, ?2, ?3, ?4, ?5, ?6)",
)?; )?;
for item in items { for item in items {
stmt.execute(params![ stmt.execute(params![
@@ -28,7 +33,8 @@ impl Db {
item.name, item.name,
item.is_dir as i64, item.is_dir as i64,
item.size, item.size,
item.mtime item.mtime,
item.parent_uid
])?; ])?;
} }
} }
@@ -41,7 +47,7 @@ impl Db {
pub fn trash_list(&self) -> Result<Vec<StoredTrash>> { pub fn trash_list(&self) -> Result<Vec<StoredTrash>> {
let conn = self.read(); let conn = self.read();
let mut stmt = conn.prepare( let mut stmt = conn.prepare(
"SELECT uid, name, is_dir, size, mtime FROM trash "SELECT uid, name, is_dir, size, mtime, parent_uid FROM trash
ORDER BY is_dir DESC, name COLLATE NOCASE", ORDER BY is_dir DESC, name COLLATE NOCASE",
)?; )?;
let rows = stmt.query_map([], |r| { let rows = stmt.query_map([], |r| {
@@ -51,6 +57,7 @@ impl Db {
is_dir: r.get::<_, i64>(2)? != 0, is_dir: r.get::<_, i64>(2)? != 0,
size: r.get(3)?, size: r.get(3)?,
mtime: r.get(4)?, mtime: r.get(4)?,
parent_uid: r.get(5)?,
}) })
})?; })?;
let mut items = Vec::new(); let mut items = Vec::new();
@@ -60,5 +67,19 @@ impl Db {
Ok(items) Ok(items)
} }
/// Every trashed node paired with the folder it was trashed from, for the
/// restore's tree walk. Kept separate from [`Db::trash_list`] because the
/// restore wants only the links, not the names, sizes or ordering.
pub fn trash_parents(&self) -> Result<Vec<(String, Option<String>)>> {
let conn = self.read();
let mut stmt = conn.prepare("SELECT uid, parent_uid FROM trash")?;
let rows = stmt.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?;
let mut pairs = Vec::new();
for row in rows {
pairs.push(row?);
}
Ok(pairs)
}
// ---- device sync (devices.md) ----------------------------------------- // ---- device sync (devices.md) -----------------------------------------
} }
+1 -1
View File
@@ -11,7 +11,7 @@ pub enum Error {
Proton(#[from] proton_sdk::ProtonError), Proton(#[from] proton_sdk::ProtonError),
#[error("keyring: {0}")] #[error("keyring: {0}")]
Keyring(#[from] keyring::Error), Keyring(#[from] keyring_core::Error),
#[error("serialization: {0}")] #[error("serialization: {0}")]
Serde(#[from] serde_json::Error), Serde(#[from] serde_json::Error),
+101
View File
@@ -199,6 +199,9 @@ fn to_entry(entry: &ignore::DirEntry) -> Option<LocalEntry> {
/// The paths a scan should never descend into: the Drive mountpoint (walking it /// The paths a scan should never descend into: the Drive mountpoint (walking it
/// would fault every remote node in through FUSE) plus our own state and cache /// would fault every remote node in through FUSE) plus our own state and cache
/// dirs, which hold blobs no user searches for by name. /// dirs, which hold blobs no user searches for by name.
///
/// Callers should extend this with [`nested_mount_points`] for each scan root —
/// see that function for why `same_file_system(true)` is not enough on its own.
pub fn default_excludes(mountpoint: &Path, state_dir: &Path, cache_dir: &Path) -> Vec<PathBuf> { pub fn default_excludes(mountpoint: &Path, state_dir: &Path, cache_dir: &Path) -> Vec<PathBuf> {
vec![ vec![
mountpoint.to_path_buf(), mountpoint.to_path_buf(),
@@ -207,6 +210,66 @@ pub fn default_excludes(mountpoint: &Path, state_dir: &Path, cache_dir: &Path) -
] ]
} }
/// Every mountpoint strictly below `root`, read from `/proc/self/mounts`.
///
/// [`scan`] already asks the walker to stay on one filesystem, but that check is
/// a *stat of the mountpoint* (`ignore`'s `is_same_file_system`), performed after
/// the directory is queued — so the walker has to touch a foreign mount to learn
/// it should skip it. On a dead network FUSE mount (an sshfs whose server is
/// gone, an unresponsive rclone) that stat blocks in `request_wait_answer` with
/// nobody left to answer it, and blocks *uninterruptibly*: the walker thread then
/// ignores SIGKILL, so the daemon can never exit, systemd's stop times out, and
/// the half-dead process keeps the cache.db lock that the next start needs
/// (docs/BUGS.md B90).
///
/// Excluding the mountpoints by path avoids that stat entirely. It does not
/// change what ends up indexed: a nested mount is a different device, so
/// `same_file_system(true)` was already going to skip it.
pub fn nested_mount_points(root: &Path) -> Vec<PathBuf> {
let Ok(mounts) = std::fs::read_to_string("/proc/self/mounts") else {
return Vec::new();
};
parse_nested_mount_points(&mounts, root)
}
/// The mountpoint-field parse behind [`nested_mount_points`], split out so it can
/// be tested without a real `/proc`.
fn parse_nested_mount_points(mounts: &str, root: &Path) -> Vec<PathBuf> {
mounts
.lines()
.filter_map(|line| line.split_whitespace().nth(1))
.map(|field| PathBuf::from(unescape_mount_field(field)))
// Strictly below: `root` itself being a mountpoint must not exclude the
// whole scan.
.filter(|path| path.starts_with(root) && path != root)
.collect()
}
/// Undo the octal escaping `/proc/self/mounts` applies to space, tab, newline and
/// backslash in path fields. An un-unescaped `\040` would simply fail to match a
/// real path, silently putting a mount with a space in its name back in the walk.
fn unescape_mount_field(field: &str) -> String {
let mut out = String::with_capacity(field.len());
let mut rest = field;
while let Some(at) = rest.find('\\') {
out.push_str(&rest[..at]);
let escape = rest[at + 1..].get(..3);
match escape.and_then(|digits| u8::from_str_radix(digits, 8).ok()) {
Some(byte) => {
out.push(byte as char);
rest = &rest[at + 4..];
}
// Not an octal escape after all — keep the backslash verbatim.
None => {
out.push('\\');
rest = &rest[at + 1..];
}
}
}
out.push_str(rest);
out
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -268,4 +331,42 @@ mod tests {
std::fs::remove_dir_all(&tmp).unwrap(); std::fs::remove_dir_all(&tmp).unwrap();
} }
/// A dead network FUSE mount under the scan root must be skipped by *path*,
/// before anything stats it — see `nested_mount_points`. The scan root itself
/// being a mountpoint must not exclude the entire walk.
#[test]
fn nested_mount_points_are_excluded_but_the_root_mount_is_not() {
let mounts = "\
/dev/sda2 /home ext4 rw 0 0
narl@narl.io:/opt /home/narl/remote/narl.io fuse.sshfs rw 0 0
google_drive: /home/narl/remote/gdrive fuse.rclone rw 0 0
tmpfs /run/user/1000 tmpfs rw 0 0
";
let nested = parse_nested_mount_points(mounts, Path::new("/home/narl"));
assert_eq!(
nested,
vec![
PathBuf::from("/home/narl/remote/narl.io"),
PathBuf::from("/home/narl/remote/gdrive"),
]
);
// `/home` is an ancestor, not a descendant: excluding it would empty the
// scan. `/run/user/1000` is unrelated.
assert!(parse_nested_mount_points(mounts, Path::new("/home")).len() == 2);
}
/// `/proc/self/mounts` octal-escapes a space; leaving it escaped would put a
/// mount whose path contains one back into the walk.
#[test]
fn mount_paths_are_unescaped() {
let mounts = "srv:/x /home/narl/My\\040Cloud fuse.sshfs rw 0 0\n";
assert_eq!(
parse_nested_mount_points(mounts, Path::new("/home/narl")),
vec![PathBuf::from("/home/narl/My Cloud")]
);
assert_eq!(unescape_mount_field("/a\\134b"), "/a\\b");
assert_eq!(unescape_mount_field("/plain/path"), "/plain/path");
}
} }
+108 -1
View File
@@ -72,6 +72,25 @@ const KNOWN_MENUS: [MenuFlavor; 6] = [
}, },
]; ];
/// Terminals that can host `pdfs-prompt --fzf`, most-preferred first.
///
/// The fzf front end is a terminal program, so a keybinding that has no tty
/// needs one spawned for it. Each entry sets a stable app-id/class
/// ([`TERMINAL_APP_ID`]) so a tiling WM can be told to float exactly this
/// window, and ends with the flag after which the command to run follows.
const KNOWN_TERMINALS: [&[&str]; 6] = [
&["foot", "--app-id=pdfs-prompt", "--"],
&["ghostty", "--class=pdfs-prompt", "-e"],
&["kitty", "--class", "pdfs-prompt"],
&["alacritty", "--class", "pdfs-prompt", "-e"],
&["wezterm", "start", "--class", "pdfs-prompt", "--"],
&["xterm", "-class", "pdfs-prompt", "-e"],
];
/// The app-id/class every [`KNOWN_TERMINALS`] entry sets, so the window rule a
/// user writes for one terminal keeps working if they switch.
pub const TERMINAL_APP_ID: &str = "pdfs-prompt";
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum PromptFlag { enum PromptFlag {
/// `--prompt=TEXT` /// `--prompt=TEXT`
@@ -105,6 +124,13 @@ pub struct PromptConfig {
/// the prompt text; without one the launcher's own prompt flag is appended. /// the prompt text; without one the launcher's own prompt flag is appended.
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub menu: Option<Vec<String>>, pub menu: Option<Vec<String>>,
/// Terminal argv for `pdfs-prompt --fzf` when it is started without a tty.
/// Unset means "find one" — see [`resolve_terminal`]. A `{cmd}` token
/// anywhere in the argv is replaced by the command to run; without one the
/// command is appended, which is what every [`KNOWN_TERMINALS`] entry
/// expects.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub terminal: Option<Vec<String>>,
/// How many results to feed the launcher. Unset means /// How many results to feed the launcher. Unset means
/// [`DEFAULT_MENU_LIMIT`]. /// [`DEFAULT_MENU_LIMIT`].
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
@@ -124,6 +150,8 @@ pub enum PromptMode {
Gtk, Gtk,
/// The user's own launcher, driven over stdin/stdout. /// The user's own launcher, driven over stdin/stdout.
Dmenu, Dmenu,
/// `fzf` in a terminal, re-querying the daemon on every keystroke.
Fzf,
} }
/// Rows an external launcher can show without becoming a scroll exercise. Larger /// Rows an external launcher can show without becoming a scroll exercise. Larger
@@ -186,6 +214,43 @@ pub fn resolve_menu(configured: Option<&Vec<String>>) -> Option<Vec<String>> {
Some(argv) Some(argv)
} }
/// The terminal to host `pdfs-prompt --fzf` in: the configured one, or the
/// first [`KNOWN_TERMINALS`] entry on `PATH`. Returns the argv up to and
/// including the "run this command" flag — the caller appends the command, or
/// substitutes it for a `{cmd}` token, via [`with_command`].
pub fn resolve_terminal(configured: Option<&Vec<String>>) -> Option<Vec<String>> {
if let Some(argv) = configured.filter(|argv| !argv.is_empty()) {
return Some(argv.clone());
}
let flavor = KNOWN_TERMINALS
.iter()
.find(|argv| argv.first().is_some_and(|program| on_path(program)))?;
Some(flavor.iter().map(|arg| (*arg).to_string()).collect())
}
/// Place `command` into a terminal argv: substituted for a `{cmd}` token if the
/// user wrote one, otherwise appended.
///
/// The token exists for terminals that want the command in the middle of their
/// own arguments; appending is right for every [`KNOWN_TERMINALS`] entry, whose
/// last element is the flag the command follows.
pub fn with_command(argv: &[String], command: &[String]) -> Vec<String> {
if !argv.iter().any(|arg| arg == "{cmd}") {
let mut argv = argv.to_vec();
argv.extend_from_slice(command);
return argv;
}
let mut out = Vec::with_capacity(argv.len() + command.len());
for arg in argv {
if arg == "{cmd}" {
out.extend_from_slice(command);
} else {
out.push(arg.clone());
}
}
out
}
/// Show `items` in the launcher named by `argv` and wait for a choice. /// Show `items` in the launcher named by `argv` and wait for a choice.
/// ///
/// Blocking by design: the caller is a one-shot CLI invocation whose entire job /// Blocking by design: the caller is a one-shot CLI invocation whose entire job
@@ -305,7 +370,8 @@ fn flavor_of(program: &str) -> Option<&'static MenuFlavor> {
KNOWN_MENUS.iter().find(|flavor| flavor.program == base) KNOWN_MENUS.iter().find(|flavor| flavor.program == base)
} }
fn on_path(program: &str) -> bool { /// Whether `program` is an executable file on `PATH`.
pub fn on_path(program: &str) -> bool {
let Some(path) = std::env::var_os("PATH") else { let Some(path) = std::env::var_os("PATH") else {
return false; return false;
}; };
@@ -408,6 +474,47 @@ mod tests {
); );
} }
#[test]
fn a_configured_terminal_is_used_verbatim() {
let configured = vec!["my-term".to_string(), "-e".to_string()];
assert_eq!(
resolve_terminal(Some(&configured)),
Some(configured.clone())
);
// An empty list is "not configured", not "run nothing".
assert_eq!(
resolve_terminal(Some(&Vec::new())),
resolve_terminal(None),
"an empty terminal list must fall back to detection"
);
}
#[test]
fn the_command_is_appended_unless_the_user_placed_it() {
let command = vec!["pdfs-prompt".to_string(), "--fzf".to_string()];
assert_eq!(
with_command(&["foot".into(), "--".into()], &command),
vec!["foot", "--", "pdfs-prompt", "--fzf"]
);
assert_eq!(
with_command(
&["term".into(), "-e".into(), "{cmd}".into(), "--hold".into()],
&command
),
vec!["term", "-e", "pdfs-prompt", "--fzf", "--hold"]
);
}
#[test]
fn every_known_terminal_sets_the_same_app_id() {
for argv in KNOWN_TERMINALS {
assert!(
argv.iter().any(|arg| arg.contains(TERMINAL_APP_ID)),
"{argv:?} must be floatable by one window rule"
);
}
}
#[test] #[test]
fn a_configured_launcher_is_used_verbatim() { fn a_configured_launcher_is_used_verbatim() {
let configured = vec!["my-menu".to_string(), "--pick".to_string()]; let configured = vec!["my-menu".to_string(), "--pick".to_string()];
+21 -2
View File
@@ -23,8 +23,9 @@
//! ``` //! ```
use std::ffi::OsStr; use std::ffi::OsStr;
use std::os::unix::process::CommandExt;
use std::path::Path; use std::path::Path;
use std::process::Command; use std::process::{Command, Stdio};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -165,12 +166,30 @@ pub fn open_named(policy: &OpenWith, path: &Path, name: &str, is_dir: bool) {
spawn(&argv, path); spawn(&argv, path);
} }
/// Launch `argv`, detached from whatever is opening it.
///
/// The child gets its own process group and no inherited stdio, because the
/// caller may be about to exit and take its terminal with it: `pdfs-prompt
/// --fzf` runs inside a terminal it spawned for itself, and that terminal
/// closes the moment the selection is made. A child left in the terminal's
/// foreground process group is sent `SIGHUP` when the pty hangs up, so
/// "open this folder in a terminal" would open a window that dies immediately
/// — which is exactly what it did. Nothing here needs the caller's stdio in the
/// first place: every `argv` is another application, and a terminal one is
/// wrapped in its own emulator by the `terminal` rule flag.
fn spawn(argv: &[String], path: &Path) { fn spawn(argv: &[String], path: &Path) {
let Some((program, args)) = argv.split_first() else { let Some((program, args)) = argv.split_first() else {
return; return;
}; };
tracing::info!(path = %path.display(), command = ?argv, "opening"); tracing::info!(path = %path.display(), command = ?argv, "opening");
if let Err(e) = Command::new(program).args(args).spawn() { let spawned = Command::new(program)
.args(args)
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.process_group(0)
.spawn();
if let Err(e) = spawned {
tracing::error!(command = ?argv, "open failed: {e}"); tracing::error!(command = ?argv, "open failed: {e}");
} }
} }
+11 -3
View File
@@ -47,10 +47,18 @@ fn systemctl_query(args: &[&str]) -> bool {
.unwrap_or(false) .unwrap_or(false)
} }
/// Enable the service so it survives reboots/logins and start it now. Called on /// Enable the service so it survives reboots/logins, and bring it up now.
/// successful login. /// Called on successful login.
///
/// `restart` rather than `enable --now`: the unit may already be active but
/// parked in the daemon's login-wait backoff (`pdfs daemon` sleeps between
/// keyring checks). `--now` is a no-op on an active unit, so the daemon would
/// keep sleeping and the mount would not appear until the next poll fired.
/// Restarting makes the new session take effect immediately, and starts the
/// unit when it is not running at all.
pub fn enable_start() -> bool { pub fn enable_start() -> bool {
systemctl(&["enable", "--now"]) let enabled = systemctl(&["enable"]);
systemctl(&["restart"]) && enabled
} }
/// Stop the running mount without disabling it. Called by the tray's /// Stop the running mount without disabling it. Called by the tray's
+318 -29
View File
@@ -393,22 +393,27 @@ fn assemble(entries: Vec<RawEntry>, sidecars: &HashMap<String, Vec<u8>>) -> Take
.unwrap_or(&file_name) .unwrap_or(&file_name)
.to_string(); .to_string();
// Sidecar first, then the date stamped into the name.
let capture_time = sidecar
.photo_taken_time
.as_ref()
.and_then(SidecarTime::epoch_seconds)
.or_else(|| {
sidecar
.creation_time
.as_ref()
.and_then(SidecarTime::epoch_seconds)
})
.or_else(|| capture_time_from_name(&name))
.or_else(|| capture_time_from_name(&file_name));
scan.photos.push(TakeoutPhoto { scan.photos.push(TakeoutPhoto {
archive: raw.archive, archive: raw.archive,
entry: raw.entry.clone(), entry: raw.entry.clone(),
name, name,
size: raw.size, size: raw.size,
media_type: media_type.to_string(), media_type: media_type.to_string(),
capture_time: sidecar capture_time,
.photo_taken_time
.as_ref()
.and_then(SidecarTime::epoch_seconds)
.or_else(|| {
sidecar
.creation_time
.as_ref()
.and_then(SidecarTime::epoch_seconds)
}),
favorite: sidecar.favorited, favorite: sidecar.favorited,
bucket, bucket,
}); });
@@ -507,34 +512,218 @@ fn unswap_duplicate_marker(stem: &str) -> Option<String> {
Some(format!("{}{}{}", &head[..dot], marker, &head[dot..])) Some(format!("{}{}{}", &head[..dot], marker, &head[dot..]))
} }
/// The capture time a *file name* implies, in epoch seconds, or `None` when it
/// carries no plausible date.
///
/// The last resort when a photo has no sidecar. Camera and messenger apps stamp
/// the date into the name — `PXL_20260818_171030868.jpg`,
/// `IMG_20230219_171030.jpg`, `Screenshot_20230101-102950.png`,
/// `IMG-20230219-WA0001.jpg`, `Screenshot from 2023-01-01 10-29-50.png` — and a
/// date that is merely *approximately* right still files the photo in the right
/// year, which "the moment it was imported" does not.
///
/// The wall clock in a name has no zone attached, so it is read as UTC. That can
/// be a few hours out; it is never the years out that the import-time fallback
/// is. A sidecar always wins over this.
pub fn capture_time_from_name(name: &str) -> Option<i64> {
let bytes = name.as_bytes();
let digit = |i: usize| bytes.get(i).is_some_and(u8::is_ascii_digit);
let num =
|start: usize, len: usize| -> Option<i64> { name.get(start..start + len)?.parse().ok() };
for start in 0..bytes.len() {
if !digit(start) || (start > 0 && digit(start - 1)) {
continue;
}
let run = |from: usize, len: usize| (0..len).all(|k| digit(from + k));
// `YYYYMMDD` as one run of exactly eight digits, or `YYYY-MM-DD` with
// any single separator repeated between the parts.
let (year, month, day, after) = if run(start, 8) && !digit(start + 8) {
(
num(start, 4)?,
num(start + 4, 2)?,
num(start + 6, 2)?,
start + 8,
)
} else if run(start, 4)
&& !digit(start + 4)
&& bytes.get(start + 4) == bytes.get(start + 7)
&& run(start + 5, 2)
&& run(start + 8, 2)
&& !digit(start + 10)
{
(
num(start, 4)?,
num(start + 5, 2)?,
num(start + 8, 2)?,
start + 10,
)
} else {
continue;
};
if !(1990..=2100).contains(&year) || !(1..=12).contains(&month) || !(1..=31).contains(&day)
{
continue;
}
// An optional time right behind the date, past one separator character.
let after = match bytes.get(after) {
Some(b'_' | b'-' | b' ' | b'.' | b',' | b'T' | b't' | b'@') => after + 1,
_ => after,
};
let time = if run(after, 6) {
// Six digits, possibly with the milliseconds tail Pixel appends.
Some((num(after, 2)?, num(after + 2, 2)?, num(after + 4, 2)?))
} else if run(after, 2)
&& !digit(after + 2)
&& bytes.get(after + 2) == bytes.get(after + 5)
&& run(after + 3, 2)
&& run(after + 6, 2)
{
Some((num(after, 2)?, num(after + 3, 2)?, num(after + 6, 2)?))
} else {
None
};
let (hour, minute, second) = match time {
Some((h, m, sec)) if h < 24 && m < 60 && sec < 60 => (h, m, sec),
_ => (0, 0, 0),
};
return Some(
days_from_civil(year, month, day) * 86_400 + hour * 3_600 + minute * 60 + second,
);
}
None
}
/// Days between 1970-01-01 and a proleptic-Gregorian date (Howard Hinnant's
/// `days_from_civil`). Avoids pulling a date crate in for one conversion.
fn days_from_civil(year: i64, month: i64, day: i64) -> i64 {
let year = year - i64::from(month <= 2);
let era = if year >= 0 { year } else { year - 399 } / 400;
let year_of_era = year - era * 400;
let day_of_year = (153 * (month + if month > 2 { -3 } else { 9 }) + 2) / 5 + day - 1;
let day_of_era = year_of_era * 365 + year_of_era / 4 - year_of_era / 100 + day_of_year;
era * 146_097 + day_of_era - 719_468
}
/// The inverse of [`unswap_duplicate_marker`]: `IMG_1234(1).jpg` as Google
/// writes its sidecar, `IMG_1234.jpg(1)`. `None` when the name carries no
/// duplicate marker.
fn swap_duplicate_marker(file_name: &str) -> Option<String> {
let marker_start = file_name.rfind('(')?;
let marker_end = marker_start + file_name[marker_start..].find(')')? + 1;
let marker = &file_name[marker_start..marker_end];
if !marker[1..marker.len() - 1]
.chars()
.all(|c| c.is_ascii_digit())
{
return None;
}
let tail = &file_name[marker_end..];
if !tail.starts_with('.') {
return None;
}
Some(format!("{}{}{}", &file_name[..marker_start], tail, marker))
}
/// Find the sidecar belonging to `file_name` among the JSON files of its folder. /// Find the sidecar belonging to `file_name` among the JSON files of its folder.
/// ///
/// Tried in order of confidence: the exact name, the `(N)`-swapped name, then a /// Tried in order of confidence: the exact name, the `(N)`-swapped name, an
/// prefix match for the case where truncation cut into the photo name itself. /// unknown trailing suffix, then a prefix match for the case where truncation
/// cut into the photo name itself. The best-ranked candidate in the folder wins.
fn find_sidecar<'a>( fn find_sidecar<'a>(
file_name: &str, file_name: &str,
in_folder: &BTreeMap<String, &'a Vec<u8>>, in_folder: &BTreeMap<String, &'a Vec<u8>>,
) -> Option<&'a Vec<u8>> { ) -> Option<&'a Vec<u8>> {
let mut prefix_match: Option<&'a Vec<u8>> = None; let mut best: Option<((u8, usize), &'a Vec<u8>)> = None;
let mut prefix_len = 0usize;
for (sidecar_name, body) in in_folder { for (sidecar_name, body) in in_folder {
let stem = strip_sidecar_suffix(sidecar_name); let Some(rank) = sidecar_rank(strip_json_suffix(sidecar_name), file_name) else {
if stem == file_name { continue;
return Some(body); };
} if best.as_ref().is_none_or(|(current, _)| rank < *current) {
if unswap_duplicate_marker(&stem).as_deref() == Some(file_name) { best = Some((rank, body));
return Some(body);
}
// Truncated: the stem is a prefix of the photo's name. Keep the longest
// such prefix — a folder holding `IMG_1.jpg` and `IMG_12.jpg` must not
// let the shorter stem claim the longer photo.
if !stem.is_empty() && file_name.starts_with(&stem) && stem.len() > prefix_len {
prefix_len = stem.len();
prefix_match = Some(body);
} }
} }
prefix_match best.map(|(_, body)| body)
}
/// How well a sidecar's `.json`-stripped `stem` matches `file_name`, as a sort
/// key where **lower is better**: the leading number is the rule that matched,
/// the second breaks ties within it.
fn sidecar_rank(stem: &str, file_name: &str) -> Option<(u8, usize)> {
// 0 — the stem is the photo name once the (English) suffix is off.
let plain = strip_sidecar_suffix(stem);
if plain == file_name {
return Some((0, 0));
}
// 1 — the same, for `IMG_1234.jpg(1).json` against `IMG_1234(1).jpg`.
if unswap_duplicate_marker(&plain).as_deref() == Some(file_name) {
return Some((1, 0));
}
// 2 — the photo name is intact but a suffix we do not recognise follows it.
// Google *localizes* `.supplemental-metadata` (a German export writes
// `.ergänzende-Metadaten`) and has shipped misspelt variants of it, so the
// suffix cannot be enumerated — matching on the intact name in front of it
// is what stays locale-agnostic. The shortest leftover wins, so a folder
// holding both `IMG_1.jpg.<suffix>` and `IMG_1.jpg(1).<suffix>` gives each
// photo its own.
for candidate in [
Some(file_name.to_string()),
swap_duplicate_marker(file_name),
]
.into_iter()
.flatten()
{
if let Some(rest) = stem.strip_prefix(&candidate)
&& looks_like_sidecar_suffix(rest)
{
return Some((2, rest.len()));
}
}
// 3 — truncation ate into the photo name itself, leaving only a prefix of
// it. Every suffix boundary in the stem is tried, so this holds for a
// localized suffix too; the longest surviving prefix wins, because a folder
// with `IMG_1.jpg` and `IMG_12.jpg` must not let the shorter stem claim the
// longer photo.
let mut longest: Option<usize> = None;
for head in suffix_boundaries(stem) {
if !head.is_empty()
&& file_name.starts_with(head)
&& longest.is_none_or(|len| head.len() > len)
{
longest = Some(head.len());
}
}
longest.map(|len| (3, usize::MAX - len))
}
/// The stem plus each of its prefixes ending before a `.`, longest first — the
/// possible boundaries between a photo name and a sidecar suffix.
fn suffix_boundaries(stem: &str) -> impl Iterator<Item = &str> {
std::iter::once(stem).chain(
stem.char_indices()
.filter(|(_, c)| *c == '.')
.map(|(index, _)| &stem[..index])
.collect::<Vec<_>>()
.into_iter()
.rev(),
)
}
/// Whether `rest` — what follows an intact photo name in a sidecar's stem —
/// looks like a metadata suffix rather than the tail of a different file's name.
/// Google's suffixes all start at a `.`, or at the `(N)` duplicate marker.
fn looks_like_sidecar_suffix(rest: &str) -> bool {
rest.starts_with('.') || rest.starts_with('(')
}
/// Drop a trailing `.json`, case-insensitively.
fn strip_json_suffix(sidecar: &str) -> &str {
match sidecar.len().checked_sub(5) {
Some(cut) if sidecar[cut..].eq_ignore_ascii_case(".json") => &sidecar[..cut],
_ => sidecar,
}
} }
#[cfg(test)] #[cfg(test)]
@@ -663,6 +852,106 @@ mod tests {
assert_eq!(scan.photos[0].capture_time, Some(1_500_000_000)); assert_eq!(scan.photos[0].capture_time, Some(1_500_000_000));
} }
#[test]
fn a_localized_supplemental_suffix_still_matches() {
// A German export writes `.ergänzende-Metadaten` where an English one
// writes `.supplemental-metadata`; the photo name in front of it is
// what the match hangs on.
let mut sidecars = HashMap::new();
sidecar(
&mut sidecars,
"Takeout/Google Fotos/Fotos von 2023/IMG-20230219-WA0001.jpg.ergänzende-Metadaten.json",
r#"{"photoTakenTime":{"timestamp":"1676800000"}}"#,
);
let scan = assemble(
vec![entry(
0,
"Takeout/Google Fotos/Fotos von 2023/IMG-20230219-WA0001.jpg",
10,
)],
&sidecars,
);
assert_eq!(scan.photos[0].capture_time, Some(1_676_800_000));
}
#[test]
fn a_localized_suffix_does_not_let_one_sidecar_claim_a_sibling() {
let mut sidecars = HashMap::new();
for name in ["IMG_1.jpg", "IMG_1(1).jpg"] {
sidecar(
&mut sidecars,
&format!("Takeout/Google Fotos/Album/{name}.ergänzende-Metadaten.json"),
r#"{"photoTakenTime":{"timestamp":"1"}}"#,
);
}
// Google writes the duplicate's sidecar with the marker after the
// extension, so this is the spelling that actually ships.
sidecars.remove("Album/IMG_1(1).jpg.ergänzende-Metadaten.json");
sidecar(
&mut sidecars,
"Takeout/Google Fotos/Album/IMG_1.jpg(1).ergänzende-Metadaten.json",
r#"{"photoTakenTime":{"timestamp":"2"}}"#,
);
let scan = assemble(
vec![
entry(0, "Takeout/Google Fotos/Album/IMG_1.jpg", 10),
entry(0, "Takeout/Google Fotos/Album/IMG_1(1).jpg", 10),
],
&sidecars,
);
assert_eq!(scan.photos[0].capture_time, Some(1));
assert_eq!(scan.photos[1].capture_time, Some(2));
}
#[test]
fn the_file_name_date_is_the_fallback_when_no_sidecar_matches() {
let scan = assemble(
vec![entry(
0,
"Takeout/Google Fotos/Fotos von 2023/IMG-20230219-WA0001.jpg",
10,
)],
&HashMap::new(),
);
// 2023-02-19 00:00:00 UTC.
assert_eq!(scan.photos[0].capture_time, Some(1_676_764_800));
}
#[test]
fn file_name_dates_cover_the_common_camera_and_messenger_spellings() {
// 2026-08-18 17:10:30 UTC, with Pixel's milliseconds tail.
assert_eq!(
capture_time_from_name("PXL_20260818_171030868.RAW-01.COVER.jpg"),
Some(1_787_073_030)
);
assert_eq!(
capture_time_from_name("IMG_20260818_171030.jpg"),
Some(1_787_073_030)
);
assert_eq!(
capture_time_from_name("Screenshot_20260818-171030.png"),
Some(1_787_073_030)
);
assert_eq!(
capture_time_from_name("Screenshot from 2026-08-18 17-10-30.png"),
Some(1_787_073_030)
);
// A date with no usable time lands at midnight, not at import time.
assert_eq!(
capture_time_from_name("IMG-20260818-WA0001.jpg"),
Some(1_787_011_200)
);
}
#[test]
fn a_name_without_a_plausible_date_yields_nothing() {
assert_eq!(capture_time_from_name("100_1234.JPG"), None);
assert_eq!(capture_time_from_name("DSC00042.jpg"), None);
// A nine-digit run is not a date; neither is an out-of-range month.
assert_eq!(capture_time_from_name("IMG_202608181.jpg"), None);
assert_eq!(capture_time_from_name("IMG_20261818.jpg"), None);
}
#[test] #[test]
fn truncated_supplemental_suffixes_still_match() { fn truncated_supplemental_suffixes_still_match() {
// Google caps the sidecar name at 51 chars, cutting into the suffix. // Google caps the sidecar name at 51 chars, cutting into the suffix.
+259 -1
View File
@@ -425,6 +425,165 @@ pub(super) async fn run_event_sync(
} }
} }
/// The state key holding the photos volume's event cursor. Separate from the
/// Drive cursor because the two volumes have their own event streams, and one
/// cursor cannot address both.
const PHOTOS_EVENT_CURSOR: &str = "photos_event_cursor";
/// Poll the photos volume's event stream, so a photo deleted or added on another
/// device shows up in the gallery within one poll instead of on the timeline's
/// freshness window.
///
/// Deliberately much smaller than [`run_event_sync`]: the photos volume is not
/// part of the FUSE mount, so there is no inode space to converge and nothing to
/// invalidate in the kernel. A trash or a delete removes the rows behind the
/// gallery; anything else only drops the timeline's freshness stamp, and the
/// next read re-enumerates. Getting that wrong costs a stale tile, not a wrong
/// file.
pub(super) async fn run_photos_event_sync(core: Core) {
let photos = core.photos();
// The photos volume only exists once the user has used Photos at all, and
// the daemon may well start before the network does.
let scope = loop {
match photos.get_photos_root().await {
Ok(Some(root)) => break root.tree_event_scope_id(),
Ok(None) => {
debug!("no photos volume; photo event sync not started");
return;
}
Err(error) => {
warn!(%error, "reading the photos root failed; retrying");
tokio::time::sleep(ONLINE_PROBE_MAX).await;
}
}
};
let mut cursor: Option<DriveEventId> = core
.db
.state_str(PHOTOS_EVENT_CURSOR)
.ok()
.flatten()
.map(DriveEventId::from);
info!(?cursor, "photo event sync started");
loop {
tokio::time::sleep(POLL_INTERVAL).await;
let events = match photos.enumerate_events(&scope, cursor.as_ref()).await {
Ok(events) => events,
Err(error) => {
warn!(%error, "photo event poll failed; retrying after interval");
continue;
}
};
let Some(head) = events.last().map(|event| event.id().clone()) else {
continue;
};
apply_photo_events(&core, &events);
// One cursor write per batch, after the batch is applied: a crash in
// between replays events whose effects are idempotent.
if let Err(error) = core.db.set_state_str(PHOTOS_EVENT_CURSOR, head.as_str()) {
warn!(%error, "persisting the photo event cursor failed");
continue;
}
cursor = Some(head);
}
}
/// What one batch of photos-volume events asks the gallery's tables to do.
#[derive(Debug, Default, PartialEq)]
pub(super) struct PhotoEventPlan {
/// Photos that left the volume, by delete or by trash.
pub(super) gone: Vec<String>,
/// Photos whose node changed, so their stored metadata is no longer trusted.
pub(super) touched: Vec<String>,
/// Whether the whole timeline has to be read again.
pub(super) full: bool,
}
impl PhotoEventPlan {
fn is_empty(&self) -> bool {
self.gone.is_empty() && self.touched.is_empty() && !self.full
}
}
/// Read one batch of photos-volume events. Pure, so the classification is
/// testable without a volume behind it.
///
/// `is_echo` claims an event as this daemon's own write. The feed replays our
/// writes back at us and a `NodeUpdated` carries no revision id, so without this
/// every favourite the user toggles here would mark its own photo stale and buy
/// a resolve of a node we just wrote.
fn classify_photo_events(
events: &[DriveEvent],
mut is_echo: impl FnMut(&NodeUid) -> bool,
) -> PhotoEventPlan {
let mut plan = PhotoEventPlan::default();
for event in events {
match event {
DriveEvent::NodeDeleted { node_uid, .. } => plan.gone.push(node_uid.to_string()),
DriveEvent::NodeUpdated {
node_uid,
is_trashed,
..
} => {
if *is_trashed {
plan.gone.push(node_uid.to_string());
} else if !is_echo(node_uid) {
// An untrash, a rename, a favourite toggled on the phone: the
// event says which node changed but never what about it, so
// the uid is marked stale and the next refresh reads it.
plan.touched.push(node_uid.to_string());
}
}
// The daemon cannot know what it missed here, so everything it
// thinks it knows about every photo is dropped.
DriveEvent::ContinuityLost { .. } | DriveEvent::ScopeAccessLost { .. } => {
plan.full = true;
}
DriveEvent::CursorAdvanced { .. } | DriveEvent::SharedWithMeUpdated { .. } => {}
}
}
plan
}
/// Apply one batch of photos-volume events to the gallery's tables.
fn apply_photo_events(core: &Core, events: &[DriveEvent]) {
let plan = classify_photo_events(events, |uid| core.take_self_change(uid));
let PhotoEventPlan {
gone,
touched,
full,
} = &plan;
if !gone.is_empty() {
match core.db.photos_delete(gone) {
Ok(removed) if removed > 0 => {
debug!(removed, "photos removed by a remote event");
}
Ok(_) => {}
Err(error) => warn!(%error, "removing remotely deleted photos failed"),
}
}
// A stale mark is what makes the next refresh cheap: it reads these photos'
// nodes and carries every other photo's metadata as it stands.
if !touched.is_empty() {
match core.db.photos_mark_unresolved(touched) {
Ok(marked) if marked > 0 => debug!(marked, "photos changed by a remote event"),
Ok(_) => {}
Err(error) => warn!(%error, "marking remotely changed photos failed"),
}
}
if *full {
match core.db.photos_mark_all_unresolved() {
Ok(marked) => debug!(marked, "the photo event stream lost continuity"),
Err(error) => warn!(%error, "marking the whole timeline unresolved failed"),
}
}
if !plan.is_empty() {
core.invalidate_photos();
core.invalidate_albums();
}
}
/// Keep the local-file index fresh for the launcher prompt's "This computer" /// Keep the local-file index fresh for the launcher prompt's "This computer"
/// results. Rebuilds the index whenever it is older than [`LOCAL_INDEX_TTL`], /// results. Rebuilds the index whenever it is older than [`LOCAL_INDEX_TTL`],
/// then sleeps; runs on its own thread for the life of the daemon. /// then sleeps; runs on its own thread for the life of the daemon.
@@ -487,7 +646,12 @@ fn scan_local_once(
} }
}; };
let excludes = localindex::default_excludes(mountpoint, &dirs.state_dir(), &dirs.cache_dir()); let mut excludes =
localindex::default_excludes(mountpoint, &dirs.state_dir(), &dirs.cache_dir());
// Resolved per scan rather than once at startup: mounts come and go, and a
// mount that appeared since the daemon started is exactly the one most
// likely to be a half-alive network filesystem (see `nested_mount_points`).
excludes.extend(localindex::nested_mount_points(&home));
indexing.store(true, Ordering::Relaxed); indexing.store(true, Ordering::Relaxed);
let started = Instant::now(); let started = Instant::now();
@@ -541,6 +705,8 @@ mod tests {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
@@ -787,4 +953,96 @@ mod tests {
assert!(acknowledge_applied_event(&db, &event, applied).is_err()); assert!(acknowledge_applied_event(&db, &event, applied).is_err());
assert!(db.get_event_cursor().unwrap().is_none()); assert!(db.get_event_cursor().unwrap().is_none());
} }
/// A photo trashed or deleted elsewhere leaves the gallery; one that only
/// changed is marked stale, so the next refresh reads that node and no other.
#[test]
fn a_photo_event_names_what_it_changed() {
let plan = classify_photo_events(
&[
DriveEvent::NodeDeleted {
id: DriveEventId::from("e1"),
node_uid: uid("deleted"),
parent_node_uid: None,
},
DriveEvent::NodeUpdated {
id: DriveEventId::from("e2"),
node_uid: uid("trashed"),
parent_node_uid: None,
is_trashed: true,
is_shared: false,
},
DriveEvent::NodeUpdated {
id: DriveEventId::from("e3"),
node_uid: uid("favourited"),
parent_node_uid: None,
is_trashed: false,
is_shared: false,
},
],
|_| false,
);
assert_eq!(
plan,
PhotoEventPlan {
gone: vec![uid("deleted").to_string(), uid("trashed").to_string()],
touched: vec![uid("favourited").to_string()],
full: false,
}
);
}
/// The feed replays this daemon's own writes. Applying one would cost a
/// resolve of a node whose changed field the daemon just wrote itself.
#[test]
fn our_own_photo_write_is_not_a_remote_change() {
let plan = classify_photo_events(
&[DriveEvent::NodeUpdated {
id: DriveEventId::from("e1"),
node_uid: uid("ours"),
parent_node_uid: None,
is_trashed: false,
is_shared: false,
}],
|claimed| *claimed == uid("ours"),
);
assert_eq!(plan, PhotoEventPlan::default());
}
/// A gap in the stream means the daemon cannot know what it missed, so it
/// keeps nothing.
#[test]
fn a_lost_photo_event_stream_asks_for_everything() {
for event in [
DriveEvent::ContinuityLost {
id: DriveEventId::from("e1"),
},
DriveEvent::ScopeAccessLost {
id: DriveEventId::from("e1"),
},
] {
let plan = classify_photo_events(std::slice::from_ref(&event), |_| false);
assert!(plan.full, "{event:?}");
assert!(plan.gone.is_empty());
assert!(plan.touched.is_empty());
}
}
/// Bookkeeping events change nothing about the gallery, and must not buy a
/// refresh of it.
#[test]
fn photo_bookkeeping_events_ask_for_nothing() {
let plan = classify_photo_events(
&[
DriveEvent::CursorAdvanced {
id: DriveEventId::from("e1"),
},
DriveEvent::SharedWithMeUpdated {
id: DriveEventId::from("e2"),
},
],
|_| false,
);
assert_eq!(plan, PhotoEventPlan::default());
}
} }
+89 -3
View File
@@ -18,7 +18,7 @@ use std::time::{Duration, Instant};
use pdfs_core::config::AppDirs; use pdfs_core::config::AppDirs;
use pdfs_core::control::{ use pdfs_core::control::{
ActivityKind, PhotoMonth, RefreshScope, Request as CtlRequest, Response as CtlResponse, ActivityKind, PhotoMonth, RefreshScope, Request as CtlRequest, Response as CtlResponse,
TransferDirection, TransferDirection, TrashFailure,
}; };
use pdfs_core::{CoreError, CoreResult}; use pdfs_core::{CoreError, CoreResult};
use proton_drive_rs::proton_sdk::ids::NodeUid; use proton_drive_rs::proton_sdk::ids::NodeUid;
@@ -67,6 +67,16 @@ impl Drop for ControlHandlerPermit {
} }
} }
/// Handler threads in use, for the diagnostics. At the limit the daemon refuses
/// connections, which a front-end cannot tell apart from a dead daemon.
pub(crate) fn active_handlers() -> usize {
ACTIVE_CONTROL_HANDLERS.load(Ordering::Acquire)
}
pub(crate) fn handler_limit() -> usize {
MAX_CONTROL_HANDLERS
}
fn read_request_line(reader: &mut impl BufRead) -> std::io::Result<Option<String>> { fn read_request_line(reader: &mut impl BufRead) -> std::io::Result<Option<String>> {
let mut bytes = Vec::new(); let mut bytes = Vec::new();
loop { loop {
@@ -178,6 +188,9 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
return; return;
} }
}; };
// Registered before the request is even parsed, so a handler that wedges
// inside the daemon shows up by name in `pdfs diagnostics`.
let _inflight = super::diagnostics::InflightGuard::new(super::diagnostics::request_kind(&line));
let response = match serde_json::from_str::<CtlRequest>(line.trim()) { let response = match serde_json::from_str::<CtlRequest>(line.trim()) {
Ok(CtlRequest::Status) => { Ok(CtlRequest::Status) => {
let pins = core.cache.list_pins(); let pins = core.cache.list_pins();
@@ -242,17 +255,33 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
core.invalidate_trash(); core.invalidate_trash();
Ok(()) Ok(())
} }
RefreshScope::Photos => { RefreshScope::Photos { full } => {
core.invalidate_photos(); core.invalidate_photos();
// A full refresh is "forget what you know", not a second code
// path: the ordinary refresh then finds every photo stale.
if let Some(Err(e)) = full.then(|| core.db.photos_mark_all_unresolved()) {
warn!(error = %e, "marking the whole timeline unresolved failed");
}
// Whatever changed the timeline can just as easily have // Whatever changed the timeline can just as easily have
// changed an album; one scope covers the whole photos view. // changed an album; one scope covers the whole photos view.
core.invalidate_albums(); core.invalidate_albums();
// The refresh is started here, not awaited. A library of
// twelve thousand photos is re-read in chunks of
// `TIMELINE_ENRICH_CHUNK` nodes, which takes minutes —
// longer than the client's read bound, so awaiting it
// returned `Resource temporarily unavailable` to a caller
// whose refresh was in fact running fine (B92). The front
// end follows it with `PhotosRefreshStatus`.
core.spawn_timeline_refresh();
Ok(()) Ok(())
} }
}; };
match result { match result {
Ok(()) => CtlResponse::Ok { Ok(()) => CtlResponse::Ok {
message: "refreshed".to_string(), message: match scope {
RefreshScope::Photos { .. } => "refreshing".to_string(),
_ => "refreshed".to_string(),
},
}, },
Err(e) => CtlResponse::error(e), Err(e) => CtlResponse::error(e),
} }
@@ -276,6 +305,14 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
}, },
Err(e) => CtlResponse::error(e), Err(e) => CtlResponse::error(e),
}, },
Ok(CtlRequest::PhotoGroup { uid }) => match core.photo_group(&uid) {
Ok(items) => CtlResponse::Photos {
available: true,
items,
counts: None,
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::SetPhotoFavorite { uid, favorite }) => match parse_uid(&uid) { Ok(CtlRequest::SetPhotoFavorite { uid, favorite }) => match parse_uid(&uid) {
Some(node_uid) => match core.set_photo_favorite(&node_uid, favorite) { Some(node_uid) => match core.set_photo_favorite(&node_uid, favorite) {
Ok(()) => CtlResponse::Ok { Ok(()) => CtlResponse::Ok {
@@ -329,6 +366,19 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
items: core.photo_thumbs(&parsed), items: core.photo_thumbs(&parsed),
} }
} }
Ok(CtlRequest::TrashNodes { uids }) => match Core::parse_uids(&uids) {
Ok(parsed) => match core.trash_photos(&parsed) {
Ok((trashed, failed)) => CtlResponse::Trashed {
trashed,
failed: failed
.into_iter()
.map(|(uid, message)| TrashFailure { uid, message })
.collect(),
},
Err(e) => CtlResponse::error(e),
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::FileThumbs { items, generation }) => { Ok(CtlRequest::FileThumbs { items, generation }) => {
match core.file_thumbs(&items, generation) { match core.file_thumbs(&items, generation) {
Some(items) => CtlResponse::Thumbs { items }, Some(items) => CtlResponse::Thumbs { items },
@@ -357,6 +407,9 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Ok(CtlRequest::ThumbnailBuildStatus) => CtlResponse::ThumbnailBuild { Ok(CtlRequest::ThumbnailBuildStatus) => CtlResponse::ThumbnailBuild {
status: core.thumbnail_build_status(), status: core.thumbnail_build_status(),
}, },
Ok(CtlRequest::PhotosRefreshStatus) => CtlResponse::PhotosRefresh {
running: core.timeline_refreshing(),
},
Ok(CtlRequest::OpenPhoto { uid }) => match parse_uid(&uid) { Ok(CtlRequest::OpenPhoto { uid }) => match parse_uid(&uid) {
Some(u) => match core.open_photo(&u) { Some(u) => match core.open_photo(&u) {
Ok(p) => CtlResponse::FilePath { Ok(p) => CtlResponse::FilePath {
@@ -411,6 +464,36 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
CtlResponse::error(CoreError::invalid("no import is running")) CtlResponse::error(CoreError::invalid("no import is running"))
} }
} }
// The scan reads the whole persisted timeline and every candidate is a
// download plus an upload, so this goes to a background thread and is
// read back through `RedateStatus`.
Ok(CtlRequest::RedatePhotos {
dry_run,
limit,
range,
}) => match core.spawn_redate(dry_run, limit, range) {
Ok(()) => CtlResponse::Ok {
message: if dry_run {
"checking photo dates".to_string()
} else {
"fixing photo dates".to_string()
},
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::RedateStatus) => {
let (running, summary) = crate::redate::redate_status();
CtlResponse::RedateStatus { running, summary }
}
Ok(CtlRequest::CancelRedate) => {
if crate::redate::cancel_redate() {
CtlResponse::Ok {
message: "stopping the re-date".to_string(),
}
} else {
CtlResponse::error(CoreError::invalid("no photo re-date is running"))
}
}
Ok(CtlRequest::UploadPhoto { Ok(CtlRequest::UploadPhoto {
name, name,
media_type, media_type,
@@ -627,6 +710,9 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Err(e) => CtlResponse::error(e), Err(e) => CtlResponse::error(e),
} }
} }
Ok(CtlRequest::Diagnostics) => CtlResponse::Diagnostics {
diagnostics: core.diagnostics(),
},
Ok(CtlRequest::ListTrash) => match core.list_trash() { Ok(CtlRequest::ListTrash) => match core.list_trash() {
Ok(entries) => CtlResponse::Entries { entries }, Ok(entries) => CtlResponse::Entries { entries },
Err(e) => CtlResponse::error(e), Err(e) => CtlResponse::error(e),
+212
View File
@@ -0,0 +1,212 @@
//! What the daemon is doing right now, for the case where it has stopped doing
//! anything at all.
//!
//! On 2026-09-18 the daemon hung: it consumed 43 minutes of CPU over 20 hours,
//! peaked at 5.2 GiB resident, went silent at 17:10, and only came back after a
//! manual `systemctl --user restart`. There was no panic, so nothing in the
//! journal said what it had been waiting on — and nothing in the daemon could be
//! asked, either. This module is the answer to "what was it stuck on?".
//!
//! Everything here is built to work *while* the daemon is wedged, which rules
//! out the obvious implementations: no lock that a stuck job might hold, no
//! database query on the calling thread, no waiting on the worker pool. A
//! diagnostic that blocks on the hang it is diagnosing reports nothing at
//! exactly the moment it matters.
use std::sync::LazyLock;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::Instant;
use parking_lot::Mutex;
use pdfs_core::control::{Diagnostics, InflightRequest, WorkerState};
use super::Core;
/// When this process started. Read at the first touch, which
/// [`super::mount::mount`] makes happen during startup so the uptime is the
/// daemon's, not the first diagnostics request's.
static STARTED: LazyLock<Instant> = LazyLock::new(Instant::now);
/// Control requests currently being served, keyed by a monotonic ticket.
///
/// A `Vec` rather than a map because it holds at most `MAX_CONTROL_HANDLERS`
/// entries, and the only operations are push, remove-by-ticket and read-all.
static INFLIGHT: Mutex<Vec<(u64, String, Instant)>> = Mutex::new(Vec::new());
static NEXT_TICKET: AtomicU64 = AtomicU64::new(0);
/// Start the daemon's uptime clock. Called once, from the mount path.
pub(crate) fn start_clock() {
LazyLock::force(&STARTED);
}
/// Registers one in-flight control request for as long as it is held.
pub(crate) struct InflightGuard(u64);
impl InflightGuard {
pub(crate) fn new(kind: String) -> Self {
let ticket = NEXT_TICKET.fetch_add(1, Ordering::Relaxed);
INFLIGHT.lock().push((ticket, kind, Instant::now()));
Self(ticket)
}
}
impl Drop for InflightGuard {
fn drop(&mut self) {
INFLIGHT.lock().retain(|(ticket, _, _)| *ticket != self.0);
}
}
/// The variant name of a serialised [`pdfs_core::control::Request`], without
/// parsing it into one.
///
/// serde's external tagging makes this a property of the text: a unit variant is
/// the bare string `"Status"`, anything else is an object whose single key is
/// the variant name. Reading the name off the line rather than matching on 80
/// variants keeps this cheap, and keeps it correct when a variant is added.
pub(crate) fn request_kind(line: &str) -> String {
let line = line.trim();
let name = if let Some(rest) = line.strip_prefix('{') {
rest.trim_start()
.strip_prefix('"')
.and_then(|rest| rest.split('"').next())
} else {
line.strip_prefix('"')
.and_then(|rest| rest.split('"').next())
};
name.unwrap_or("unknown").to_string()
}
/// Resident set size in bytes, or `0` when `/proc` did not answer.
///
/// From `VmRSS` in `/proc/self/status`, which is already in kB — `statm` would
/// need the page size, and this needs no dependency at all.
pub(crate) fn rss_bytes() -> u64 {
let Ok(status) = std::fs::read_to_string("/proc/self/status") else {
return 0;
};
parse_vm_rss(&status)
}
/// The `VmRSS` line of a `/proc/<pid>/status`, in bytes.
fn parse_vm_rss(status: &str) -> u64 {
status
.lines()
.find_map(|line| line.strip_prefix("VmRSS:"))
.and_then(|rest| rest.split_whitespace().next())
.and_then(|kb| kb.parse::<u64>().ok())
.map_or(0, |kb| kb.saturating_mul(1024))
}
impl Core {
/// Assemble a [`Diagnostics`] report.
///
/// The pending-op count is the one field that needs the database, so it is
/// read on a throwaway thread with a short deadline: a daemon wedged *on*
/// the database must still be able to report everything else about itself.
pub(crate) fn diagnostics(&self) -> Diagnostics {
let pool = self.workers.snapshot();
let now = Instant::now();
let mut inflight: Vec<InflightRequest> = INFLIGHT
.try_lock()
.map(|entries| {
entries
.iter()
.map(|(_, kind, started)| InflightRequest {
kind: kind.clone(),
age_secs: now.duration_since(*started).as_secs(),
})
.collect()
})
.unwrap_or_default();
inflight.sort_by_key(|request| std::cmp::Reverse(request.age_secs));
Diagnostics {
uptime_secs: STARTED.elapsed().as_secs(),
rss_bytes: rss_bytes(),
workers: pool
.workers
.into_iter()
.map(|worker| WorkerState {
name: worker.name,
busy: worker.busy,
label: worker.label,
age_secs: worker.age_ms / 1000,
})
.collect(),
meta_queued: pool.meta_queued,
transfer_queued: pool.transfer_queued,
meta_completed: pool.meta_completed,
transfer_completed: pool.transfer_completed,
inflight,
control_handlers: super::control::active_handlers(),
control_handler_limit: super::control::handler_limit(),
pending_ops: self.pending_ops_quickly(),
}
}
/// Queued mutations, or `0` if the database did not answer within
/// [`PENDING_COUNT_DEADLINE`].
fn pending_ops_quickly(&self) -> u64 {
let db = self.db.clone();
let (tx, rx) = std::sync::mpsc::channel();
if std::thread::Builder::new()
.name("pdfs-diagnostics".into())
.spawn(move || {
let counts = db.pending_op_counts().unwrap_or_default();
let total = counts.uploads.max(0) as u64 + counts.changes.max(0) as u64;
let _ = tx.send(total);
})
.is_err()
{
return 0;
}
rx.recv_timeout(PENDING_COUNT_DEADLINE).unwrap_or(0)
}
}
/// How long the report waits for the database before reporting without it.
const PENDING_COUNT_DEADLINE: std::time::Duration = std::time::Duration::from_millis(500);
#[cfg(test)]
mod tests {
use super::*;
/// The request name is read off the wire format, so it must match both
/// shapes serde produces.
#[test]
fn the_request_kind_comes_off_both_wire_shapes() {
assert_eq!(request_kind("\"Status\""), "Status");
assert_eq!(request_kind("{\"Pin\":{\"path\":\"a\"}}"), "Pin");
assert_eq!(request_kind(" { \"Restore\" : {} } "), "Restore");
assert_eq!(request_kind("garbage"), "unknown");
}
/// A guard registers for its lifetime and nothing longer: a handler that
/// returns must not leave a permanent "in flight" entry behind.
#[test]
fn a_guard_registers_only_while_it_lives() {
let before = INFLIGHT.lock().len();
{
let _guard = InflightGuard::new("OpenFile".into());
assert_eq!(INFLIGHT.lock().len(), before + 1);
}
assert_eq!(INFLIGHT.lock().len(), before);
}
/// Whatever else it reports, the resident size has to be a real reading.
#[test]
fn rss_is_readable() {
assert!(rss_bytes() > 0);
}
/// `VmRSS` is in kB, and a status without one must read as unknown rather
/// than as a daemon using no memory.
#[test]
fn vm_rss_is_kilobytes_and_absence_is_zero() {
assert_eq!(
parse_vm_rss("VmPeak:\t 100 kB\nVmRSS:\t 2048 kB\n"),
2048 * 1024
);
assert_eq!(parse_vm_rss("VmPeak:\t 100 kB\n"), 0);
}
}
+90 -42
View File
@@ -29,12 +29,14 @@ use std::time::{Duration, Instant};
use parking_lot::Mutex; use parking_lot::Mutex;
use futures::StreamExt;
use pdfs_core::batch; use pdfs_core::batch;
use pdfs_core::cache::{Baseline, StagedWrite}; use pdfs_core::cache::{Baseline, StagedWrite};
use pdfs_core::control::{ActivityKind, TransferDirection}; use pdfs_core::control::{ActivityKind, TransferDirection};
use pdfs_core::db::{OP_CREATE, OP_MKDIR, OP_RENAME, OP_REVISION, OP_TRASH, PendingOp, RenameMeta}; use pdfs_core::db::{OP_CREATE, OP_MKDIR, OP_RENAME, OP_REVISION, OP_TRASH, PendingOp, RenameMeta};
use proton_drive_rs::Node; use proton_drive_rs::proton_sdk::error::ProtonError;
use proton_drive_rs::proton_sdk::ids::NodeUid; use proton_drive_rs::proton_sdk::ids::NodeUid;
use proton_drive_rs::{Node, NodeMoveItem};
use tracing::{debug, error, info, warn}; use tracing::{debug, error, info, warn};
use super::state::Intervals; use super::state::Intervals;
@@ -642,61 +644,46 @@ impl Core {
return Ok(()); return Ok(());
} }
}; };
// The name half goes first, so that a collision on the move half below is let landed = if node.parent_uid.as_ref() == Some(&parent) {
// about the name the node will actually land under rather than the one it self.drain_rename_in_place(&uid, &node.name, &name)?
// is about to lose. } else {
let mut landed = node.name.clone(); // Move and rename land as one request (`move-multiple` takes a
if landed != name { // target name), so there is no half-applied state between them and
match self.rt.block_on(self.client.rename_node(&uid, &name, None)) { // no window for the requirements to go stale between two calls —
Ok(()) => landed = name.clone(), // the race B46 queued this op to get away from.
// Someone took the name while we were offline. Renaming to a let target = (node.name != name).then_some(name.as_str());
match self.move_renaming(&uid, &parent, target) {
Ok(()) => name.clone(),
// The destination holds that name already. Landing under a
// *different* name is the non-destructive resolution: it neither // *different* name is the non-destructive resolution: it neither
// clobbers their file nor drops ours, and it is visible. // clobbers their file nor drops ours, and it is visible.
Err(e) if is_already_exists(&e) => { Err(e) if is_already_exists(&e) => {
let alt = conflict_name(&name, now_secs()); let alt = conflict_name(&name, now_secs());
warn!(%uid, name, alt, "rename target name is taken; using a conflict name"); warn!(%uid, name, alt, "destination already holds that name; using a conflict name");
self.rt self.move_renaming(&uid, &parent, Some(&alt))?;
.block_on(self.client.rename_node(&uid, &alt, None))?;
landed = alt.clone();
self.adopt_drained_name(&uid, &alt); self.adopt_drained_name(&uid, &alt);
self.log_activity( self.log_activity(
ActivityKind::Rename, ActivityKind::Rename,
&name, &name,
format!("name was taken remotely; renamed to {alt}"),
false,
);
}
Err(e) => return Err(e.into()),
}
}
if node.parent_uid.as_ref() != Some(&parent) {
match self.rt.block_on(self.client.move_node(&uid, &parent)) {
Ok(()) => {}
// The destination holds a `landed` of its own. Same resolution as
// a name collision, and it has to happen before the move: the API
// has no move-and-rename, so the node is renamed out of the way
// here and moved second.
Err(e) if is_already_exists(&e) => {
let alt = conflict_name(&landed, now_secs());
warn!(%uid, name = %landed, alt, "destination already holds that name; using a conflict name");
self.rt
.block_on(self.client.rename_node(&uid, &alt, None))?;
self.rt.block_on(self.client.move_node(&uid, &parent))?;
self.adopt_drained_name(&uid, &alt);
self.log_activity(
ActivityKind::Rename,
&landed,
format!("destination already had that name; moved as {alt}"), format!("destination already had that name; moved as {alt}"),
false, false,
); );
alt
} }
// The destination folder is gone. Leaving the node in its current // The destination folder is gone. Leaving the node in its current
// parent is the honest outcome: it is not where the user asked for // parent is the honest outcome: it is not where the user asked for
// it, but it exists, it is where it has always been, and the // it, but it exists and it is where it has always been. The name
// rename half above still applied. Retrying could only fail again // the user chose still applies there. Retrying the move could
// — the folder is not coming back — and would wedge the queue. // only fail again — the folder is not coming back — and would
// wedge the queue.
Err(e) if is_gone(&e) => { Err(e) if is_gone(&e) => {
warn!(%uid, name = %landed, %parent, "move destination is gone; leaving the node where it is"); warn!(%uid, name, %parent, "move destination is gone; leaving the node where it is");
let landed = match self.drain_rename_in_place(&uid, &node.name, &name) {
Ok(landed) => landed,
// The node went with it; nothing is left to rename.
Err(e) if is_gone(&e) => node.name.clone(),
Err(e) => return Err(e.into()),
};
self.log_activity( self.log_activity(
ActivityKind::Rename, ActivityKind::Rename,
&landed, &landed,
@@ -704,16 +691,73 @@ impl Core {
.to_string(), .to_string(),
false, false,
); );
landed
} }
Err(e) => return Err(e.into()), Err(e) => return Err(e.into()),
} }
} };
self.db.delete_op(op.id)?; self.db.delete_op(op.id)?;
self.note_self_change(&uid); self.note_self_change(&uid);
info!(%uid, name = %landed, "pending rename landed"); info!(%uid, name = %landed, "pending rename landed");
Ok(()) Ok(())
} }
/// Rename `uid` without moving it, from `current` to `name`, and return the
/// name it landed under. Someone may have taken the name while we were
/// offline; the node then lands under a conflict name instead.
fn drain_rename_in_place(
&self,
uid: &NodeUid,
current: &str,
name: &str,
) -> Result<String, ProtonError> {
if current == name {
return Ok(current.to_string());
}
match self.rt.block_on(self.client.rename_node(uid, name, None)) {
Ok(()) => Ok(name.to_string()),
Err(e) if is_already_exists(&e) => {
let alt = conflict_name(name, now_secs());
warn!(%uid, name, alt, "rename target name is taken; using a conflict name");
self.rt.block_on(self.client.rename_node(uid, &alt, None))?;
self.adopt_drained_name(uid, &alt);
self.log_activity(
ActivityKind::Rename,
name,
format!("name was taken remotely; renamed to {alt}"),
false,
);
Ok(alt)
}
Err(e) => Err(e),
}
}
/// Move `uid` under `parent` in one request, renaming it to `target_name` on
/// the way when one is given.
fn move_renaming(
&self,
uid: &NodeUid,
parent: &NodeUid,
target_name: Option<&str>,
) -> Result<(), ProtonError> {
let item = NodeMoveItem {
uid: uid.clone(),
target_name: target_name.map(str::to_string),
};
self.rt.block_on(async {
let mut outcomes =
std::pin::pin!(self.client.move_nodes_streaming(vec![item], parent.clone()));
match outcomes.next().await {
Some(Ok((_, outcome))) => outcome,
Some(Err(e)) => Err(e),
None => Err(ProtonError::invalid_operation(format!(
"move of {uid} reported no outcome"
))),
}
})
}
/// Apply a queued trash to the remote. /// Apply a queued trash to the remote.
/// ///
/// A node that is already gone is a success, not a failure: the outcome the /// A node that is already gone is a success, not a failure: the outcome the
@@ -1587,6 +1631,8 @@ mod tests {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
@@ -1879,6 +1925,8 @@ mod tests {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
+25 -25
View File
@@ -49,7 +49,7 @@ impl Filesystem for ProtonFs {
/// ([`Core::account_quota_cached`] serves a warm one from memory). /// ([`Core::account_quota_cached`] serves a warm one from memory).
fn statfs(&self, _req: &Request, _ino: INodeNo, reply: ReplyStatfs) { fn statfs(&self, _req: &Request, _ino: INodeNo, reply: ReplyStatfs) {
let core = self.core.clone(); let core = self.core.clone();
self.core.workers.run(Lane::Meta, move || { self.core.workers.run(Lane::Meta, "statfs", move || {
let (max_space, used_space) = core.account_quota_cached().unwrap_or((0, 0)); let (max_space, used_space) = core.account_quota_cached().unwrap_or((0, 0));
let bsize = u64::from(REPORTED_BLKSIZE); let bsize = u64::from(REPORTED_BLKSIZE);
let blocks = (max_space.max(0) as u64).div_ceil(bsize); let blocks = (max_space.max(0) as u64).div_ceil(bsize);
@@ -89,7 +89,7 @@ impl Filesystem for ProtonFs {
return; return;
} }
let fs = self.clone(); let fs = self.clone();
self.core.workers.run(Lane::Meta, move || { self.core.workers.run(Lane::Meta, "lookup", move || {
fs.serve_lookup(parent, &name, reply, true) fs.serve_lookup(parent, &name, reply, true)
}); });
} }
@@ -183,7 +183,7 @@ impl Filesystem for ProtonFs {
let fs = self.clone(); let fs = self.clone();
self.core self.core
.workers .workers
.run(Lane::Meta, move || fs.serve_opendir(ino, reply)); .run(Lane::Meta, "opendir", move || fs.serve_opendir(ino, reply));
} }
fn readdir( fn readdir(
@@ -210,9 +210,9 @@ impl Filesystem for ProtonFs {
return; return;
} }
let fs = self.clone(); let fs = self.clone();
self.core self.core.workers.run(Lane::Meta, "readdir", move || {
.workers fs.serve_readdir(ino, offset, reply)
.run(Lane::Meta, move || fs.serve_readdir(ino, offset, reply)); });
} }
fn releasedir( fn releasedir(
@@ -237,9 +237,9 @@ impl Filesystem for ProtonFs {
return; return;
} }
let fs = self.clone(); let fs = self.clone();
self.core self.core.workers.run(Lane::Transfer, "open", move || {
.workers fs.serve_open(ino, flags, reply)
.run(Lane::Transfer, move || fs.serve_open(ino, flags, reply)); });
} }
fn read( fn read(
@@ -329,7 +329,7 @@ impl Filesystem for ProtonFs {
// even read off the FUSE device. It only reads `state`, so moving it // even read off the FUSE device. It only reads `state`, so moving it
// races with nothing. FUSE does not require replies in request order. // races with nothing. FUSE does not require replies in request order.
let core = self.core.clone(); let core = self.core.clone();
self.core.workers.run(Lane::Transfer, move || { self.core.workers.run(Lane::Transfer, "read", move || {
match core.read_range(&uid, mtime, fsize, offset, size as u64, cache_blocks) { match core.read_range(&uid, mtime, fsize, offset, size as u64, cache_blocks) {
Ok(bytes) => reply.data(&bytes), Ok(bytes) => reply.data(&bytes),
Err(e) => reply.error(e), Err(e) => reply.error(e),
@@ -366,9 +366,9 @@ impl Filesystem for ProtonFs {
// one create at a time, with `ls` on the same mount timing out for as // one create at a time, with `ls` on the same mount timing out for as
// long as the burst lasted. See [`Workers`]. // long as the burst lasted. See [`Workers`].
let fs = self.clone(); let fs = self.clone();
self.core self.core.workers.run(Lane::Meta, "create", move || {
.workers fs.serve_create(parent, &name, reply)
.run(Lane::Meta, move || fs.serve_create(parent, &name, reply)); });
} }
fn write( fn write(
@@ -506,7 +506,7 @@ impl Filesystem for ProtonFs {
// the scratch file). // the scratch file).
let ino = ino.0; let ino = ino.0;
let fs = self.clone(); let fs = self.clone();
self.core.workers.run(Lane::Transfer, move || { self.core.workers.run(Lane::Transfer, "truncate", move || {
if let Err(e) = fs.core.queue_truncate(ino, size) { if let Err(e) = fs.core.queue_truncate(ino, size) {
reply.error(e); reply.error(e);
return; return;
@@ -620,7 +620,7 @@ impl Filesystem for ProtonFs {
let fs = self.clone(); let fs = self.clone();
self.core self.core
.workers .workers
.run(Lane::Transfer, move || fs.serve_fsync(fh, reply)); .run(Lane::Transfer, "fsync", move || fs.serve_fsync(fh, reply));
} }
fn release( fn release(
@@ -680,7 +680,7 @@ impl Filesystem for ProtonFs {
}; };
// `trash_child` is a remote call; see `serve_create`. // `trash_child` is a remote call; see `serve_create`.
let fs = self.clone(); let fs = self.clone();
self.core.workers.run(Lane::Meta, move || { self.core.workers.run(Lane::Meta, "unlink", move || {
fs.serve_unlink(parent, &name_str, reply) fs.serve_unlink(parent, &name_str, reply)
}); });
} }
@@ -701,9 +701,9 @@ impl Filesystem for ProtonFs {
// Enumerates the folder to apply rmdir's emptiness rule, then trashes // Enumerates the folder to apply rmdir's emptiness rule, then trashes
// it: two remote calls. See `serve_create`. // it: two remote calls. See `serve_create`.
let fs = self.clone(); let fs = self.clone();
self.core self.core.workers.run(Lane::Meta, "rmdir", move || {
.workers fs.serve_rmdir(parent, &name_str, reply)
.run(Lane::Meta, move || fs.serve_rmdir(parent, &name_str, reply)); });
} }
fn mkdir( fn mkdir(
@@ -731,9 +731,9 @@ impl Filesystem for ProtonFs {
// remote round trip, and inline it blocks every other request on the // remote round trip, and inline it blocks every other request on the
// mount until it answers. // mount until it answers.
let fs = self.clone(); let fs = self.clone();
self.core self.core.workers.run(Lane::Meta, "mkdir", move || {
.workers fs.serve_mkdir(parent, &name, reply)
.run(Lane::Meta, move || fs.serve_mkdir(parent, &name, reply)); });
} }
fn rename( fn rename(
@@ -774,7 +774,7 @@ impl Filesystem for ProtonFs {
// move), and the replaced-destination path adds a trash on top. See // move), and the replaced-destination path adds a trash on top. See
// `serve_create` for why none of that may run on the dispatch loop. // `serve_create` for why none of that may run on the dispatch loop.
let fs = self.clone(); let fs = self.clone();
self.core.workers.run(Lane::Meta, move || { self.core.workers.run(Lane::Meta, "rename", move || {
fs.serve_rename(parent, &name, newparent, &newname, flags, reply) fs.serve_rename(parent, &name, newparent, &newname, flags, reply)
}); });
} }
@@ -799,7 +799,7 @@ impl Filesystem for ProtonFs {
// network per call (B5). // network per call (B5).
let fs = self.clone(); let fs = self.clone();
let name = name.to_os_string(); let name = name.to_os_string();
self.core.workers.run(Lane::Meta, move || { self.core.workers.run(Lane::Meta, "getxattr", move || {
fs.serve_getxattr(ino, &name, size, reply) fs.serve_getxattr(ino, &name, size, reply)
}); });
} }
@@ -1099,7 +1099,7 @@ impl ProtonFs {
if !off_loop { if !off_loop {
let fs = self.clone(); let fs = self.clone();
let name = name.to_string(); let name = name.to_string();
self.core.workers.run(Lane::Meta, move || { self.core.workers.run(Lane::Meta, "lookup", move || {
fs.serve_lookup(parent, &name, reply, true) fs.serve_lookup(parent, &name, reply, true)
}); });
return; return;
+279 -31
View File
@@ -83,6 +83,7 @@ mod background;
mod control; mod control;
use control::run_control_socket; use control::run_control_socket;
mod devices; mod devices;
mod diagnostics;
mod drain; mod drain;
mod filesystem; mod filesystem;
pub use filesystem::ProtonFs; pub use filesystem::ProtonFs;
@@ -90,18 +91,21 @@ mod mount;
mod photos; mod photos;
mod profile; mod profile;
mod reads; mod reads;
mod redate;
mod revisions; mod revisions;
mod sharing; mod sharing;
mod shutdown; mod shutdown;
mod state; mod state;
mod supervisor;
mod sweep; mod sweep;
mod sync; mod sync;
mod systemd;
mod takeout; mod takeout;
mod transfers; mod transfers;
mod upload; mod upload;
mod r#virtual; mod r#virtual;
mod workers; mod workers;
use background::{run_event_sync, run_local_index}; use background::{run_event_sync, run_local_index, run_photos_event_sync};
pub(crate) use mount::is_stale_mount; pub(crate) use mount::is_stale_mount;
pub use mount::{MountOptions, MountOutcome, mount}; pub use mount::{MountOptions, MountOutcome, mount};
use mount::{ use mount::{
@@ -199,13 +203,20 @@ const ONLINE_PROBE_MAX: Duration = Duration::from_secs(300);
/// revalidates it. The SDK hands back the whole timeline at once, so it is stored /// revalidates it. The SDK hands back the whole timeline at once, so it is stored
/// in the DB and every page is sliced from there; a stale one is still served /// in the DB and every page is sliced from there; a stale one is still served
/// immediately and refreshed in the background. /// immediately and refreshed in the background.
const TIMELINE_TTL: Duration = Duration::from_secs(5 * 60); ///
/// Short, because the cost of a stale timeline is a photo the user deleted on
/// their phone still sitting in the gallery, and the cost of the check is one
/// listing call that only runs when somebody is actually looking at the page.
const TIMELINE_TTL: Duration = Duration::from_secs(60);
/// How many photo nodes are resolved per [`ProtonPhotosClient::enumerate_nodes`] /// How many photo nodes are resolved per [`ProtonPhotosClient::enumerate_nodes`]
/// call when enriching a refreshed timeline with names and media types (for the /// call when enriching a refreshed timeline with names and media types (for the
/// Photos / Videos / Raw split). Batched so a large library is a handful of /// Photos / Videos / Raw split). Batched so a large library is a handful of
/// round-trips rather than one request per photo, and bounded so a single call /// round-trips rather than one request per photo, and bounded so a single call
/// never asks the server to decrypt the whole library at once. /// never asks the server to decrypt the whole library at once.
const TIMELINE_ENRICH_CHUNK: usize = 200; ///
/// 150 because that is the SDK's own `MAX_BATCH_COUNT`: a larger chunk is split
/// there anyway, into one full request and a ragged tail.
const TIMELINE_ENRICH_CHUNK: usize = 150;
/// The same, for the persisted trash listing. Shorter, because the trash is the /// The same, for the persisted trash listing. Shorter, because the trash is the
/// one listing a user changes and then immediately looks at — though our own /// one listing a user changes and then immediately looks at — though our own
/// mutations also invalidate it outright, so this only covers other clients. /// mutations also invalidate it outright, so this only covers other clients.
@@ -3737,6 +3748,9 @@ impl Core {
// My own tree: a role only means something under a share, and // My own tree: a role only means something under a share, and
// this listing never crosses into one. // this listing never crosses into one.
role: String::new(), role: String::new(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
}) })
.collect()) .collect())
} }
@@ -4318,6 +4332,9 @@ impl Core {
uid: item.uid, uid: item.uid,
path: String::new(), path: String::new(),
role: String::new(), role: String::new(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
}) })
.collect()) .collect())
} }
@@ -4359,6 +4376,9 @@ impl Core {
is_dir: node.is_folder(), is_dir: node.is_folder(),
size: node_size(&node) as i64, size: node_size(&node) as i64,
mtime: node.modification_time, mtime: node.modification_time,
// The folder the node will go back to, which is also what tells
// a restore whether that folder is itself in the trash.
parent_uid: node.parent_uid.as_ref().map(|uid| uid.to_string()),
})); }));
// Cumulative, so the table is always a prefix of the real trash // Cumulative, so the table is always a prefix of the real trash
// rather than a mix of this refresh and the last one. // rather than a mix of this refresh and the last one.
@@ -4464,15 +4484,48 @@ impl Core {
.collect() .collect()
} }
/// Restore trashed nodes to the folders they were trashed from. The parents /// Restore trashed nodes to the folders they were trashed from, together
/// are read *before* the restore — a restored node reappears in a listing the /// with the rest of the trashed tree they belong to.
/// daemon may already have cached, so each destination folder is invalidated ///
/// and re-enumerated on next access. /// The trash is a flat list, but a user deletes a shape: a folder, and often
/// items inside it that were trashed separately beforehand. Restoring one of
/// those in isolation is not what anyone means by it — a folder would come
/// back without the contents that were deleted with it, and a file would come
/// back inside a folder that is still in the trash, which is to say nowhere
/// the user can see. So a restore takes the whole connected piece:
/// every trashed descendant of what was asked for, and every trashed ancestor
/// above it. See [`expand_restore`] for the exact rule.
///
/// The waves are restored shallowest first, because a node cannot be put back
/// under a parent that is still trashed.
///
/// The destination parents are read *before* the restore — a restored node
/// reappears in a listing the daemon may already have cached, so each one is
/// invalidated and re-enumerated on next access.
fn restore(&self, uids: &[String]) -> CoreResult<usize> { fn restore(&self, uids: &[String]) -> CoreResult<usize> {
let parsed = Self::parse_uids(uids)?; let requested = Self::parse_uids(uids)?;
// A listing we could not read is not a reason to refuse the restore: it
// only costs the expansion, and the requested nodes still go back.
let relations = self.db.trash_parents().unwrap_or_default();
let waves = expand_restore(uids, &relations);
let waves: Vec<Vec<NodeUid>> = if waves.is_empty() {
vec![requested.clone()]
} else {
waves
.iter()
.map(|wave| {
wave.iter()
.filter_map(|uid| parse_uid(uid))
.collect::<Vec<_>>()
})
.filter(|wave: &Vec<NodeUid>| !wave.is_empty())
.collect()
};
let all: Vec<NodeUid> = waves.iter().flatten().cloned().collect();
let parents: Vec<NodeUid> = self let parents: Vec<NodeUid> = self
.rt .rt
.block_on(self.client.enumerate_nodes(&parsed)) .block_on(self.client.enumerate_nodes(&all))
.map_err(|e| CoreError::from_api(&e, "enumerate nodes"))? .map_err(|e| CoreError::from_api(&e, "enumerate nodes"))?
.into_iter() .into_iter()
.filter_map(|n| n.parent_uid) .filter_map(|n| n.parent_uid)
@@ -4483,27 +4536,29 @@ impl Core {
// half way through a large restore leaves local state describing what the // half way through a large restore leaves local state describing what the
// server actually did. Only a batch that restored nothing is an error the // server actually did. Only a batch that restored nothing is an error the
// caller can act on. // caller can act on.
let mut restored: Vec<NodeUid> = Vec::with_capacity(parsed.len()); let mut restored: Vec<NodeUid> = Vec::with_capacity(all.len());
let mut first_error: Option<ProtonError> = None; let mut first_error: Option<ProtonError> = None;
self.rt for wave in &waves {
.block_on(async { self.rt
let mut outcomes = std::pin::pin!(self.client.restore_nodes_streaming(&parsed)); .block_on(async {
while let Some(item) = outcomes.next().await { let mut outcomes = std::pin::pin!(self.client.restore_nodes_streaming(wave));
let (uid, outcome) = item?; while let Some(item) = outcomes.next().await {
match outcome { let (uid, outcome) = item?;
Ok(()) => { match outcome {
self.hidden.lock().remove(&uid); Ok(()) => {
restored.push(uid); self.hidden.lock().remove(&uid);
} restored.push(uid);
Err(e) => { }
warn!(%uid, error = %e, "restore failed for a node"); Err(e) => {
first_error.get_or_insert(e); warn!(%uid, error = %e, "restore failed for a node");
first_error.get_or_insert(e);
}
} }
} }
} Ok::<(), ProtonError>(())
Ok::<(), ProtonError>(()) })
}) .map_err(|e| CoreError::from_api(&e, "restore"))?;
.map_err(|e| CoreError::from_api(&e, "restore"))?; }
if restored.is_empty() if restored.is_empty()
&& let Some(error) = first_error && let Some(error) = first_error
{ {
@@ -4856,7 +4911,7 @@ fn public_link_info(link: proton_drive_rs::PublicLink) -> PublicLinkInfo {
/// Parse a `volume~link` uid display string back into a [`NodeUid`]. Front-ends /// Parse a `volume~link` uid display string back into a [`NodeUid`]. Front-ends
/// receive uids as strings over the control socket and pass them back verbatim. /// receive uids as strings over the control socket and pass them back verbatim.
fn parse_uid(s: &str) -> Option<NodeUid> { pub(crate) fn parse_uid(s: &str) -> Option<NodeUid> {
let (vol, link) = s.split_once('~')?; let (vol, link) = s.split_once('~')?;
if vol.is_empty() || link.is_empty() || link.contains('~') { if vol.is_empty() || link.is_empty() || link.contains('~') {
return None; return None;
@@ -4993,6 +5048,8 @@ fn local_node(uid: NodeUid, parent_uid: NodeUid, name: String, is_dir: bool) ->
album: None, album: None,
// Nothing signed it: it has never been near the crypto layer. // Nothing signed it: it has never been near the crypto layer.
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
@@ -5276,6 +5333,96 @@ fn node_size(node: &Node) -> u64 {
} }
} }
/// Expand a restore request across the trashed tree, and order it so a parent is
/// always restored before its children.
///
/// `relations` is the persisted trash listing as `(uid, parent_uid)` pairs; a
/// parent that is not itself in the listing is a live folder, which ends the
/// walk upward. The rule is:
///
/// - every trashed descendant of a requested node joins the restore, so a folder
/// comes back with the items that were trashed inside it;
/// - every trashed ancestor of a requested node joins it too, so a file never
/// lands in a folder that is still in the trash;
/// - a requested uid the listing does not know about is restored on its own,
/// because a stale or half-materialised listing must not silently drop it.
///
/// The result is one wave per depth, shallowest first. Waves are disjoint and
/// together contain each uid exactly once.
fn expand_restore(
requested: &[String],
relations: &[(String, Option<String>)],
) -> Vec<Vec<String>> {
let parents: HashMap<&str, Option<&str>> = relations
.iter()
.map(|(uid, parent)| (uid.as_str(), parent.as_deref()))
.collect();
let mut children: HashMap<&str, Vec<&str>> = HashMap::new();
for (uid, parent) in relations {
// Only a trashed parent: a live one keeps its child as a root here.
if let Some(parent) = parent.as_deref()
&& parents.contains_key(parent)
{
children.entry(parent).or_default().push(uid.as_str());
}
}
let mut selected: HashSet<&str> = HashSet::new();
for uid in requested {
let uid = uid.as_str();
if !parents.contains_key(uid) {
selected.insert(uid);
continue;
}
// Upward: every trashed ancestor, so the destination folder exists again.
let mut cursor = Some(uid);
while let Some(current) = cursor {
// A parent outside the listing is live: the walk stops under it.
if !parents.contains_key(current) || !selected.insert(current) {
break;
}
cursor = parents.get(current).copied().flatten();
}
// Downward: the whole trashed subtree, so the folder comes back full.
let mut stack = vec![uid];
while let Some(current) = stack.pop() {
for child in children.get(current).into_iter().flatten() {
if selected.insert(child) {
stack.push(child);
}
}
}
}
// Depth within the *trash*, not within the drive: a node whose parent is live
// (or unknown) is at depth 0 and can be restored straight away.
let mut waves: Vec<Vec<String>> = Vec::new();
for uid in &selected {
let mut depth = 0usize;
let mut cursor = parents.get(uid).copied().flatten();
// Only trashed ancestors count, and the walk is bounded by the listing's
// size: server data that somehow describes a cycle must not spin here.
while let Some(current) = cursor
&& parents.contains_key(current)
&& depth < parents.len()
{
depth += 1;
cursor = parents.get(current).copied().flatten();
}
if waves.len() <= depth {
waves.resize(depth + 1, Vec::new());
}
waves[depth].push((*uid).to_string());
}
// Stable output, so a restore of the same trash is the same sequence of calls
// every time — the batches are what shows up in the log and in the tests.
for wave in &mut waves {
wave.sort();
}
waves.retain(|wave| !wave.is_empty());
waves
}
/// The server revision id of a node's active revision, if it is a file that has /// The server revision id of a node's active revision, if it is a file that has
/// one. The stable identity the drain conflict-checks against (see [`Baseline`]). /// one. The stable identity the drain conflict-checks against (see [`Baseline`]).
fn node_revision_id(node: &Node) -> Option<String> { fn node_revision_id(node: &Node) -> Option<String> {
@@ -5917,6 +6064,8 @@ mod pending_size_tests {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
@@ -5974,9 +6123,9 @@ mod tests {
use super::{ use super::{
Access, AccessFlags, Errno, HashMap, Intervals, PendingRevision, RootListingSnapshot, Access, AccessFlags, Errno, HashMap, Intervals, PendingRevision, RootListingSnapshot,
SELF_CHANGE_TTL_MS, ShareId, SharedWithMeItem, StateRegistry, VirtualRootPlan, SELF_CHANGE_TTL_MS, ShareId, SharedWithMeItem, StateRegistry, VirtualRootPlan,
accepted_share_provenance, conflict_name, copy_pending_for_truncate, fuse_name, accepted_share_provenance, conflict_name, copy_pending_for_truncate, expand_restore,
is_stale_mount, node_visible, note_self_change, parse_node_uid, prepare_shared_roots, fuse_name, is_stale_mount, node_visible, note_self_change, parse_node_uid,
preserve_on_access_denied, publish_virtual_root_in_listing, prepare_shared_roots, preserve_on_access_denied, publish_virtual_root_in_listing,
reconcile_virtual_root_in_listing, release_can_discard_unlinked, reconcile_virtual_root_in_listing, release_can_discard_unlinked,
release_must_retain_queued_trash, release_unlinked_entry, rename_needs_queue, release_must_retain_queued_trash, release_unlinked_entry, rename_needs_queue,
require_node_parent_access, require_rename_access, resolve_anywhere_with, require_node_parent_access, require_rename_access, resolve_anywhere_with,
@@ -5984,6 +6133,100 @@ mod tests {
}; };
use super::{Db, WriteAuthority}; use super::{Db, WriteAuthority};
/// `(uid, parent)` pairs in the shape [`Db::trash_parents`] returns.
fn relations(pairs: &[(&str, Option<&str>)]) -> Vec<(String, Option<String>)> {
pairs
.iter()
.map(|(uid, parent)| (uid.to_string(), parent.map(str::to_string)))
.collect()
}
fn uids(values: &[&str]) -> Vec<String> {
values.iter().map(|v| v.to_string()).collect()
}
/// Restoring a folder brings back what was trashed inside it, and the folder
/// goes first: the server has nowhere to put a child whose parent is trashed.
#[test]
fn restoring_a_folder_restores_its_trashed_contents_parents_first() {
let trash = relations(&[
("dir", Some("live")),
("file", Some("dir")),
("deep", Some("sub")),
("sub", Some("dir")),
]);
assert_eq!(
expand_restore(&uids(&["dir"]), &trash),
vec![
vec!["dir".to_string()],
vec!["file".to_string(), "sub".to_string()],
vec!["deep".to_string()],
]
);
}
/// Restoring a file inside a trashed folder restores the folder too —
/// otherwise the file comes back somewhere the user cannot reach.
#[test]
fn restoring_content_restores_the_trashed_parents_above_it() {
let trash = relations(&[
("dir", Some("live")),
("sub", Some("dir")),
("file", Some("sub")),
("other", Some("dir")),
]);
// `other` is a sibling of `sub`, not an ancestor of `file`: it stays put.
assert_eq!(
expand_restore(&uids(&["file"]), &trash),
vec![
vec!["dir".to_string()],
vec!["sub".to_string()],
vec!["file".to_string()],
]
);
}
/// A node whose parent is not in the trash is already at the top: one wave,
/// no ancestors invented.
#[test]
fn a_node_under_a_live_parent_restores_on_its_own() {
let trash = relations(&[("file", Some("live")), ("elsewhere", None)]);
assert_eq!(
expand_restore(&uids(&["file"]), &trash),
vec![vec!["file".to_string()]]
);
}
/// The listing is materialised in chunks and can be stale, so a uid it does
/// not know about must still be restored rather than dropped.
#[test]
fn an_unlisted_uid_is_still_restored() {
let trash = relations(&[("dir", None)]);
assert_eq!(
expand_restore(&uids(&["missing"]), &trash),
vec![vec!["missing".to_string()]]
);
}
/// Asking for both a folder and something inside it restores each node once.
#[test]
fn overlapping_requests_do_not_duplicate_nodes() {
let trash = relations(&[("dir", None), ("file", Some("dir"))]);
assert_eq!(
expand_restore(&uids(&["dir", "file"]), &trash),
vec![vec!["dir".to_string()], vec!["file".to_string()]]
);
}
/// A parent chain that loops back on itself must terminate.
#[test]
fn a_cyclic_listing_terminates() {
let trash = relations(&[("a", Some("b")), ("b", Some("a"))]);
let waves = expand_restore(&uids(&["a"]), &trash);
let total: usize = waves.iter().map(Vec::len).sum();
assert_eq!(total, 2);
}
/// The collapse [`Core::require_uid_writable`] performs, without a `Core`: /// The collapse [`Core::require_uid_writable`] performs, without a `Core`:
/// every non-writable authority is one `EACCES` on the syscall path. /// every non-writable authority is one `EACCES` on the syscall path.
fn require_uid_access(db: &Db, uid: &NodeUid, live: &[Access]) -> Result<(), Errno> { fn require_uid_access(db: &Db, uid: &NodeUid, live: &[Access]) -> Result<(), Errno> {
@@ -7127,6 +7370,9 @@ mod tests {
share_id, share_id,
membership_id: ShareMembershipId::from("membership"), membership_id: ShareMembershipId::from("membership"),
permissions, permissions,
invite_time: None,
inviter_email: None,
inviter_verification: None,
}); });
node node
}; };
@@ -7587,6 +7833,8 @@ mod tests {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
} }
+40
View File
@@ -341,6 +341,9 @@ pub fn mount(
username, username,
sweep_mode, sweep_mode,
} = options; } = options;
// Start the uptime clock here rather than at the first `pdfs diagnostics`,
// so the age it reports is the daemon's own.
super::diagnostics::start_clock();
let (root, online) = fetch_or_recall_root(&client, &rt, &db)?; let (root, online) = fetch_or_recall_root(&client, &rt, &db)?;
let scope = root.tree_event_scope_id(); let scope = root.tree_event_scope_id();
db.mount_upsert_my_files(&mountpoint.to_string_lossy(), &root.uid.to_string(), None) db.mount_upsert_my_files(&mountpoint.to_string_lossy(), &root.uid.to_string(), None)
@@ -551,6 +554,26 @@ pub fn mount(
} }
} }
} }
// The supervisor watches the pool and answers the systemd watchdog. Started
// after the listener, because its liveness probe is a round trip over that
// socket.
{
let supervisor_core = core.clone();
let socket = control_socket.to_path_buf();
match std::thread::Builder::new()
.name("pdfs-supervisor".into())
.spawn(move || supervisor::run(supervisor_core, socket))
{
Ok(handle) => workers.push(handle),
// Not fatal: the mount works without a supervisor, it just goes back
// to being undiagnosable if it wedges.
Err(error) => warn!(error = %error, "could not start the daemon supervisor"),
}
}
// The mount is live and the control socket answers: this is the moment the
// unit is actually usable, which is what `Type=notify` reports.
systemd::ready();
// Re-establish on-demand mounts only after the primary session is live. // Re-establish on-demand mounts only after the primary session is live.
// Keep the worker so shutdown can close publication, wait for any in-flight // Keep the worker so shutdown can close publication, wait for any in-flight
// fetch to observe that closure, and then drain a stable registry. // fetch to observe that closure, and then drain a stable registry.
@@ -568,6 +591,9 @@ pub fn mount(
} }
}; };
rt.spawn(run_event_sync(client, scope, core.clone())); rt.spawn(run_event_sync(client, scope, core.clone()));
// The photos volume has its own event stream; without it the gallery only
// learns about a phone-side deletion when the timeline goes stale.
rt.spawn(run_photos_event_sync(core.clone()));
if online { if online {
let repair_core = core.clone(); let repair_core = core.clone();
let root_uid = core.primary_root_uid.clone(); let root_uid = core.primary_root_uid.clone();
@@ -663,6 +689,11 @@ pub fn mount(
Ok(outcome) Ok(outcome)
} }
/// How long teardown waits for the FUSE worker pool before proceeding without
/// it. Long enough for an in-flight block to finish, short enough that
/// `systemctl --user stop` still returns inside `TimeoutStopSec`.
const WORKER_JOIN_DEADLINE: Duration = Duration::from_secs(10);
/// Signal every background loop, wake the ones that are blocked, and join them. /// Signal every background loop, wake the ones that are blocked, and join them.
/// ///
/// The mount used to return with its drain workers, sync engine, sweep, online /// The mount used to return with its drain workers, sync engine, sweep, online
@@ -677,6 +708,9 @@ pub fn mount(
/// expected and correct: a worker part-way through an upload finishes it rather /// expected and correct: a worker part-way through an upload finishes it rather
/// than abandoning the user's bytes mid-flight. /// than abandoning the user's bytes mid-flight.
fn stop_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::JoinHandle<()>>) { fn stop_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::JoinHandle<()>>) {
// Say so before the joins: a teardown that takes a while must not be read as
// a missed watchdog ping and turned into a kill.
systemd::stopping();
core.shutdown.stop(); core.shutdown.stop();
core.wake_drain(); core.wake_drain();
let _ = core.sync_tx.send(sync::SyncMsg::Stop); let _ = core.sync_tx.send(sync::SyncMsg::Stop);
@@ -691,6 +725,12 @@ fn stop_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::Jo
warn!("a background worker panicked before shutdown"); warn!("a background worker panicked before shutdown");
} }
} }
// The FUSE worker pool last, and *before* the caller drops the tokio
// runtime: a worker still inside a job when the runtime goes away panics on
// its next timer with "A Tokio 1.x context was found, but it is being
// shutdown" (seen 2026-09-18 09:58:35). Bounded, so a worker wedged on the
// network cannot hold the stop the user asked for.
core.workers.stop_and_join(WORKER_JOIN_DEADLINE);
debug!("background workers stopped"); debug!("background workers stopped");
} }
+281 -23
View File
@@ -29,8 +29,8 @@ use proton_drive_rs::{NodeKind, PhotoTag, PhotoTagsUpdate, ThumbnailType};
use tracing::{info, warn}; use tracing::{info, warn};
use super::{ use super::{
Core, PHOTOS_AVAILABLE, PHOTOS_SYNCED_MS, TIMELINE_ENRICH_CHUNK, TIMELINE_TTL, node_size, Core, PHOTOS_AVAILABLE, PHOTOS_SYNCED_MS, TIMELINE_ENRICH_CHUNK, TIMELINE_TTL, WriteAuthority,
now_ms, parse_uid, node_size, now_ms, parse_uid,
}; };
/// Longest edge, in px, of a thumbnail generated locally for a photo the server /// Longest edge, in px, of a thumbnail generated locally for a photo the server
@@ -402,7 +402,21 @@ fn extract_raw_preview(
.map(|preview| (preview, orientation))) .map(|preview| (preview, orientation)))
} }
/// What a [`Core::trash_photos`] batch did: the uids that left the server, and
/// the ones that did not with the reason each was refused.
pub(crate) type TrashOutcome = (Vec<String>, Vec<(String, String)>);
/// Formats supported by the local thumbnail decoder and exposed by the GUI. /// Formats supported by the local thumbnail decoder and exposed by the GUI.
/// What a timeline enrichment pass resolves for one photo, beyond what the
/// timeline listing already carries.
#[derive(Clone, Debug)]
struct PhotoMeta {
name: Option<String>,
media_type: Option<String>,
favorite: bool,
content_hash: Option<String>,
}
impl Core { impl Core {
pub(crate) fn photos_timeline( pub(crate) fn photos_timeline(
&self, &self,
@@ -454,6 +468,8 @@ impl Core {
no_thumb: photo.thumb_state == db::THUMB_NONE, no_thumb: photo.thumb_state == db::THUMB_NONE,
kind: photo.kind, kind: photo.kind,
favorite: photo.favorite, favorite: photo.favorite,
group_size: photo.group_size as u32,
has_raw: photo.has_raw,
} }
} }
@@ -483,12 +499,135 @@ impl Core {
.map_err(|e| CoreError::from_api(&e, "update photo tags"))?; .map_err(|e| CoreError::from_api(&e, "update photo tags"))?;
pdfs_core::batch::into_unit(outcomes) pdfs_core::batch::into_unit(outcomes)
.map_err(|e| CoreError::from_api(&e, "update photo tags"))?; .map_err(|e| CoreError::from_api(&e, "update photo tags"))?;
// The photo event feed replays this write back at us as a bare
// `NodeUpdated`, which is indistinguishable from the same photo being
// favourited on a phone. Claim the echo, or the gallery pays for a
// resolve of a node whose one changed field is already stored below.
self.note_self_change(uid);
self.db self.db
.photos_set_favorite(&uid.to_string(), favorite) .photos_set_favorite(&uid.to_string(), favorite)
.map_err(CoreError::from)?; .map_err(CoreError::from)?;
Ok(()) Ok(())
} }
/// Every file of one photo's group, in server order, as wire items. A photo
/// the timeline does not hold comes back empty, which the lightbox reads as
/// "nothing to switch between".
pub(crate) fn photo_group(&self, uid: &str) -> CoreResult<Vec<PhotoItem>> {
let members = self.db.photos_group(uid).map_err(CoreError::from)?;
Ok(members
.into_iter()
.map(|photo| self.photo_item(photo))
.collect())
}
/// Every uid asked for, plus the other files of any group they belong to,
/// each once. A uid the timeline does not hold (an album photo on someone
/// else's volume) is passed through unchanged.
fn expand_photo_groups(&self, uids: &[NodeUid]) -> Vec<NodeUid> {
let mut out: Vec<NodeUid> = Vec::with_capacity(uids.len());
let mut seen: HashSet<String> = HashSet::new();
for uid in uids {
let members = self.db.photos_group(&uid.to_string()).unwrap_or_default();
let mut group: Vec<NodeUid> = members
.iter()
.filter_map(|photo| parse_uid(&photo.uid))
.collect();
if group.is_empty() {
group.push(uid.clone());
}
for member in group {
if seen.insert(member.to_string()) {
out.push(member);
}
}
}
out
}
/// Mutation admission for a photo, which [`Core::require_uid_writable`]
/// cannot answer on its own.
///
/// Photos live on the photo volume, and that volume is not part of the FUSE
/// tree: a photo has no `nodes` row, so every authority comes back
/// [`WriteAuthority::Unknown`] and the plain check refuses the whole
/// library. Membership of our own timeline is the authority that does exist
/// here — the timeline is enumerated from our photo share, so a uid in it is
/// ours to trash. Anything else (an album photo on someone else's volume)
/// still fails closed.
fn require_photo_writable(&self, uid: &NodeUid) -> Result<(), fuser::Errno> {
match self.uid_write_authority(uid) {
WriteAuthority::Writable => Ok(()),
WriteAuthority::Denied => Err(fuser::Errno::EACCES),
WriteAuthority::Unknown => {
let key = uid.to_string();
let known = self
.db
.photos_by_uid(std::slice::from_ref(&key))
.map(|rows| !rows.is_empty())
.unwrap_or(false);
if known {
Ok(())
} else {
Err(fuser::Errno::EACCES)
}
}
}
}
/// Trash photos by uid, and forget them locally.
///
/// The gallery is not part of the FUSE mount, so this is the photos-side
/// counterpart of [`Core::delete`]: same destination (Proton trash, so the
/// Trash page can restore them), addressed by uid because that is all a tile
/// has.
///
/// The server answers per node, and so does this: a batch where one photo
/// fails still removes the ones that went, and the caller is told which is
/// which. Only what actually left the server is forgotten locally.
///
/// A photo that is one file of a group takes its whole group with it. The
/// gallery shows a RAW+JPEG shot as one tile, so deleting that tile has to
/// delete the shot — leaving the RAW behind would put the photo back on the
/// Raw tab and nowhere else.
pub(crate) fn trash_photos(&self, uids: &[NodeUid]) -> CoreResult<TrashOutcome> {
if uids.is_empty() {
return Ok((Vec::new(), Vec::new()));
}
let uids = &self.expand_photo_groups(uids);
for uid in uids {
self.require_photo_writable(uid)
.map_err(|errno| self.errno_error(errno, "trash access"))?;
}
let outcomes = self
.rt
.block_on(self.client.trash_nodes(uids))
.map_err(|e| CoreError::from_api(&e, "trash"))?;
let (trashed, failed) = pdfs_core::batch::split(outcomes);
let trashed: Vec<String> = trashed.iter().map(|uid| uid.to_string()).collect();
for uid in &trashed {
if let Some(parsed) = parse_uid(uid) {
self.cache.evict(&parsed);
self.evict_reader(&parsed);
}
}
// The timeline is served from the database, so removing the rows is what
// makes the photos disappear now rather than at the next refresh.
if let Err(error) = self.db.photos_delete(&trashed) {
warn!(%error, "trashed photos could not be removed from the timeline");
}
self.invalidate_photos();
self.invalidate_albums();
self.invalidate_trash();
let failed = failed
.into_iter()
.map(|(uid, error)| (uid.to_string(), error.to_string()))
.collect();
Ok((trashed, failed))
}
/// Thumbnails for `uids`, served from the cache, fetched from the server for /// Thumbnails for `uids`, served from the cache, fetched from the server for
/// whatever is missing, and — for the photos the server has no thumbnail for /// whatever is missing, and — for the photos the server has no thumbnail for
/// at all — *generated locally* from the full file (see /// at all — *generated locally* from the full file (see
@@ -498,8 +637,36 @@ impl Core {
/// ///
/// A photo absent from the persisted timeline is skipped: its capture time is /// A photo absent from the persisted timeline is skipped: its capture time is
/// the cache's validity tag, and guessing that would poison the cache. /// the cache's validity tag, and guessing that would poison the cache.
pub(crate) fn photo_thumbs(&self, uids: &[NodeUid]) -> Vec<PhotoThumb> { ///
/// A camera raw is served by the JPEG of the same shot when there is one:
/// the two files show one picture, and the JPEG is already decodable in
/// milliseconds where the raw costs a full download and a raw decode. The
/// tile still answers under the uid that was asked for.
pub(crate) fn photo_thumbs(&self, requested: &[NodeUid]) -> Vec<PhotoThumb> {
let ttype = ThumbnailType::Thumbnail.as_i32(); let ttype = ThumbnailType::Thumbnail.as_i32();
// Only a raw can be stood in for, and that is one batched read; the
// per-shot group lookup is then paid for raws alone, not for a page of
// ordinary photos.
let asked: Vec<String> = requested.iter().map(|u| u.to_string()).collect();
let raws: HashSet<String> = self
.db
.photos_by_uid(&asked)
.unwrap_or_default()
.into_iter()
.filter(|photo| photo.kind == PhotoKind::Raw)
.map(|photo| photo.uid)
.collect();
let sources: Vec<NodeUid> = requested
.iter()
.map(|uid| {
if raws.contains(&uid.to_string()) {
self.thumb_source(uid)
} else {
uid.clone()
}
})
.collect();
let uids = &sources;
let keys: Vec<String> = uids.iter().map(|u| u.to_string()).collect(); let keys: Vec<String> = uids.iter().map(|u| u.to_string()).collect();
let mut stored = self.db.photos_by_uid(&keys).unwrap_or_default(); let mut stored = self.db.photos_by_uid(&keys).unwrap_or_default();
// Album covers and the contents of an album shared with us are not in our // Album covers and the contents of an album shared with us are not in our
@@ -582,19 +749,47 @@ impl Core {
} }
let pending = self.thumb_gen.lock(); let pending = self.thumb_gen.lock();
uids.iter() requested
.map(|uid| PhotoThumb { .iter()
uid: uid.to_string(), .zip(uids)
path: tags.get(&uid.to_string()).and_then(|&tag| { .map(|(asked, source)| PhotoThumb {
uid: asked.to_string(),
path: tags.get(&source.to_string()).and_then(|&tag| {
self.cache self.cache
.cached_thumbnail_path(uid, ttype, tag) .cached_thumbnail_path(source, ttype, tag)
.map(|p| p.display().to_string()) .map(|p| p.display().to_string())
}), }),
pending: pending.contains(uid), pending: pending.contains(source),
}) })
.collect() .collect()
} }
/// Which file's thumbnail stands for `uid` on a tile.
///
/// Itself, for all but a camera raw that was written next to a JPEG of the
/// same shot. For that one, the JPEG: same picture, already compressed,
/// often carrying a server thumbnail the raw does not have — so the Raw tab
/// costs no raw download and no raw decode. A raw that stands alone keeps
/// its own local generation.
fn thumb_source(&self, uid: &NodeUid) -> NodeUid {
let members = match self.db.photos_group(&uid.to_string()) {
Ok(members) if members.len() > 1 => members,
_ => return uid.clone(),
};
let key = uid.to_string();
if !members
.iter()
.any(|photo| photo.uid == key && photo.kind == PhotoKind::Raw)
{
return uid.clone();
}
members
.iter()
.find(|photo| photo.kind != PhotoKind::Raw)
.and_then(|photo| parse_uid(&photo.uid))
.unwrap_or_else(|| uid.clone())
}
/// Thumbnails for image files shown by the regular file browser, Shared and /// Thumbnails for image files shown by the regular file browser, Shared and
/// Trash pages. Unlike [`Core::photo_thumbs`], these nodes need not be in the /// Trash pages. Unlike [`Core::photo_thumbs`], these nodes need not be in the
/// Photos timeline: the request supplies each file's modification time as /// Photos timeline: the request supplies each file's modification time as
@@ -1209,6 +1404,7 @@ impl Core {
/// Whether the listing stamped under `key` is older than `ttl` (or was never /// Whether the listing stamped under `key` is older than `ttl` (or was never
/// fetched). /// fetched).
pub(crate) async fn refresh_timeline(&self) -> CoreResult<bool> { pub(crate) async fn refresh_timeline(&self) -> CoreResult<bool> {
let started = Instant::now();
let photos = self.photos(); let photos = self.photos();
if photos if photos
.get_photos_root() .get_photos_root()
@@ -1225,14 +1421,35 @@ impl Core {
.await .await
.map_err(|e| CoreError::from_api(&e, "timeline"))?; .map_err(|e| CoreError::from_api(&e, "timeline"))?;
// The timeline DTO carries only a uid and capture time, but the Photos // The timeline DTO carries only a uid and capture time, but the gallery
// page has to split into Photos / Videos / Raw — which needs each photo's // has to split into its tabs — which needs each photo's name and media
// name and media type. Resolve those in batches off the request path. // type. Resolve those in batches off the request path. Best-effort: a
// Best-effort: a photo whose node we fail to resolve keeps whatever was // photo whose node we fail to resolve keeps whatever was learned before
// learned before (or classifies from nothing, i.e. a still photo), so a // (or classifies from nothing, i.e. a still photo), so a partial resolve
// partial resolve never blanks the timeline. // never blanks the timeline.
let uids: Vec<NodeUid> = items.iter().map(|it| it.uid.clone()).collect(); //
let mut meta: HashMap<String, (Option<String>, Option<String>, bool)> = HashMap::new(); // Only photos the daemon has never read, and ones a remote event marked
// stale, are read. Resolving all of them every time is what made a
// 12k-photo refresh take minutes (`docs/BUGS.md` B92): this metadata does
// not change on its own, and when it does change the event feed names the
// uid that changed (`Db::photos_mark_unresolved`).
let known: HashSet<String> = self
.db
.photos_resolved()
.map_err(CoreError::from)?
.into_iter()
.collect();
let uids: Vec<NodeUid> = items
.iter()
.filter(|it| !known.contains(&it.uid.to_string()))
.map(|it| it.uid.clone())
.collect();
let resolved_at = now_ms();
let mut meta: HashMap<String, PhotoMeta> = HashMap::new();
// A main photo names its related photos rather than the other way round,
// so the relation is recorded from whichever end resolves — the two ends
// can land in different chunks, and one of them may not resolve at all.
let mut main_of: HashMap<String, String> = HashMap::new();
for chunk in uids.chunks(TIMELINE_ENRICH_CHUNK) { for chunk in uids.chunks(TIMELINE_ENRICH_CHUNK) {
match photos.enumerate_nodes(chunk).await { match photos.enumerate_nodes(chunk).await {
Ok(nodes) => { Ok(nodes) => {
@@ -1248,9 +1465,24 @@ impl Core {
.photo .photo
.as_ref() .as_ref()
.is_some_and(|p| p.tags.contains(&PhotoTag::Favorite)); .is_some_and(|p| p.tags.contains(&PhotoTag::Favorite));
let key = node.uid.to_string();
let content_hash = node.photo.as_ref().and_then(|p| p.content_hash.clone());
if let Some(photo) = node.photo.as_ref() {
if let Some(main) = photo.main_photo_uid.as_ref() {
main_of.insert(key.clone(), main.to_string());
}
for related in &photo.related_photo_uids {
main_of.insert(related.to_string(), key.clone());
}
}
meta.insert( meta.insert(
node.uid.to_string(), key,
(Some(node.name), media_type, favorite), PhotoMeta {
name: Some(node.name),
media_type,
favorite,
content_hash,
},
); );
} }
} }
@@ -1262,19 +1494,39 @@ impl Core {
.iter() .iter()
.map(|it| { .map(|it| {
let key = it.uid.to_string(); let key = it.uid.to_string();
let main_uid = main_of.get(&key).cloned();
// An unresolved photo keeps whatever was learned before, for the // An unresolved photo keeps whatever was learned before, for the
// favourite flag as much as for the name and media type. // favourite flag as much as for the name and media type.
match meta.get(&key).cloned() { match meta.get(&key).cloned() {
Some((name, media_type, favorite)) => { Some(m) => db::TimelineRow {
(key, it.capture_time, name, media_type, Some(favorite)) name: m.name,
} media_type: m.media_type,
None => (key, it.capture_time, None, None, None), favorite: Some(m.favorite),
content_hash: m.content_hash,
main_uid,
resolved_at: Some(resolved_at),
..db::TimelineRow::new(key, it.capture_time)
},
None => db::TimelineRow {
main_uid,
..db::TimelineRow::new(key, it.capture_time)
},
} }
}) })
.collect(); .collect();
self.db.photos_replace(&rows).map_err(CoreError::from)?; self.db.photos_replace(&rows).map_err(CoreError::from)?;
let _ = self.db.set_state_i64(PHOTOS_AVAILABLE, 1); let _ = self.db.set_state_i64(PHOTOS_AVAILABLE, 1);
let _ = self.db.set_state_i64(PHOTOS_SYNCED_MS, now_ms()); let _ = self.db.set_state_i64(PHOTOS_SYNCED_MS, now_ms());
// How long this took is the first thing to know when a refresh looks
// like it never happened, and the chunk count is what it scales with.
info!(
photos = rows.len(),
resolved = uids.len(),
skipped = rows.len().saturating_sub(uids.len()),
chunks = uids.len().div_ceil(TIMELINE_ENRICH_CHUNK),
elapsed_ms = started.elapsed().as_millis() as u64,
"photos timeline refreshed"
);
Ok(true) Ok(true)
} }
@@ -1293,6 +1545,12 @@ impl Core {
}); });
} }
/// Whether a timeline refresh is running. A front-end that asked for one
/// watches this to know when the new timeline is worth re-reading.
pub(crate) fn timeline_refreshing(&self) -> bool {
self.timeline_refreshing.load(Ordering::SeqCst)
}
/// Download a photo's full content into the content cache, returning its /// Download a photo's full content into the content cache, returning its
/// on-disk path (served from cache when a fresh blob already exists). /// on-disk path (served from cache when a fresh blob already exists).
pub(crate) fn open_photo(&self, uid: &NodeUid) -> CoreResult<PathBuf> { pub(crate) fn open_photo(&self, uid: &NodeUid) -> CoreResult<PathBuf> {
+332
View File
@@ -0,0 +1,332 @@
//! Repairing photos whose capture time is when they were *imported* rather than
//! when they were taken.
//!
//! A Google Photos Takeout whose metadata sidecars did not match — a localized
//! `.supplemental-metadata` spelling, say — imports with no capture time at all,
//! and the SDK then seals "now" into every revision. A decade of photos lands on
//! one day.
//!
//! Proton has no API for editing a sealed capture time, so the repair is a
//! rewrite: download the photo, upload it again with the capture time its file
//! name implies, put the new copy back into the albums and favourites the old
//! one was in, then **trash** (not delete) the original. Trash rather than
//! delete so a bad run is recoverable from Proton's own trash for 30 days.
//!
//! A photo is a candidate only when its name carries a date that disagrees with
//! its stored capture time by more than [`DRIFT`]. That is deliberately narrow:
//! a photo whose name says nothing about its date is left alone, because there
//! is nothing better to set it to.
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use pdfs_core::control::{RedateSummary, TransferDirection};
use pdfs_core::takeout::{capture_time_from_name, media_type_of};
use pdfs_core::{CoreError, CoreResult};
use proton_drive_rs::proton_sdk::ids::NodeUid;
use proton_drive_rs::{PhotoTag, PhotoTagsUpdate, PhotoUploadMetadata};
use tracing::{info, warn};
use super::Core;
use crate::parse_uid as parse_node_uid;
use crate::transfers::CountingReader;
/// How far a stored capture time may sit from the one the file name implies
/// before the photo counts as mis-dated. A day, because a name usually carries
/// only the date and the stored time carries a clock — and because a timezone
/// can legitimately move a photo by hours.
const DRIFT: i64 = 86_400;
/// How many examples a report carries.
const SAMPLES: usize = 10;
/// The one re-date this daemon may be running, and how the last one ended.
/// Process-global for the same reasons as the Takeout import's state: one run
/// per daemon however many `Core` clones exist, and a front-end asking after it
/// finished must still find the report.
static REDATE: std::sync::LazyLock<parking_lot::Mutex<RedateState>> =
std::sync::LazyLock::new(|| parking_lot::Mutex::new(RedateState::default()));
#[derive(Default)]
struct RedateState {
running: bool,
cancel: Option<Arc<AtomicBool>>,
last: Option<RedateSummary>,
}
/// Whether a re-date is running, and the last one's report.
pub(crate) fn redate_status() -> (bool, Option<RedateSummary>) {
let state = REDATE.lock();
(state.running, state.last.clone())
}
/// Ask the running re-date to stop. False when none is running.
pub(crate) fn cancel_redate() -> bool {
match &REDATE.lock().cancel {
Some(flag) => {
flag.store(true, Ordering::Relaxed);
true
}
None => false,
}
}
/// One photo to repair.
struct Candidate {
uid: NodeUid,
name: String,
/// The capture time it carries now.
stored: i64,
/// The capture time its name implies.
wanted: i64,
favorite: bool,
/// Album node uids the original is filed under.
albums: Vec<String>,
}
impl Core {
/// Re-date mis-dated photos. Blocking; run on its own thread.
///
/// `cancel` stops the run at the next photo boundary. A photo is only ever
/// trashed after its replacement has uploaded, so a cancel — or a crash —
/// can leave a duplicate, never a hole.
pub(crate) fn redate_photos(
&self,
dry_run: bool,
limit: Option<usize>,
range: Option<(i64, i64)>,
cancel: Arc<AtomicBool>,
) -> CoreResult<RedateSummary> {
let job = self.transfers.begin_job("Fixing photo dates");
job.detail("looking for mis-dated photos");
let stored = self
.db
.photos_all()
.map_err(|e| CoreError::internal(format!("read timeline: {e}")))?;
let mut summary = RedateSummary {
examined: stored.len(),
dry_run,
..Default::default()
};
let mut candidates = Vec::new();
for photo in stored {
if let Some((from, to)) = range
&& !(from..to).contains(&photo.capture_time)
{
continue;
}
let Some(name) = photo.name.filter(|n| !n.is_empty()) else {
continue;
};
let Some(wanted) = capture_time_from_name(&name) else {
continue;
};
if (wanted - photo.capture_time).abs() <= DRIFT {
continue;
}
let Some(uid) = parse_node_uid(&photo.uid) else {
continue;
};
candidates.push(Candidate {
uid,
name,
stored: photo.capture_time,
wanted,
favorite: photo.favorite,
albums: self.db.albums_of_photo(&photo.uid).unwrap_or_default(),
});
}
if let Some(limit) = limit {
candidates.truncate(limit);
}
summary.candidates = candidates.len();
// Sampled after the limit, so the examples are of photos the run would
// actually touch.
summary.samples = candidates
.iter()
.take(SAMPLES)
.map(|c| (c.name.clone(), c.stored, c.wanted))
.collect();
job.set_total(candidates.len() as u64);
info!(
examined = summary.examined,
candidates = summary.candidates,
dry_run,
"photo re-date scan complete"
);
if dry_run || candidates.is_empty() {
return Ok(summary);
}
for candidate in candidates {
if cancel.load(Ordering::Relaxed) || self.shutdown.is_stopping() {
break;
}
job.detail(candidate.name.clone());
if let Err(e) = self.redate_one(&candidate, &mut summary) {
summary.failed += 1;
warn!(name = candidate.name, error = %e, "cannot re-date photo");
}
job.step();
}
self.spawn_timeline_refresh();
summary.cancelled = cancel.load(Ordering::Relaxed);
Ok(summary)
}
/// Rewrite one photo at the capture time its name implies.
///
/// Order is load-bearing: upload first, file the new copy, and only then
/// trash the original. Every earlier step failing leaves the account exactly
/// as it was.
fn redate_one(&self, candidate: &Candidate, summary: &mut RedateSummary) -> CoreResult<()> {
let photos = self.photos();
let path = self.open_photo(&candidate.uid)?;
let size = std::fs::metadata(&path)
.map_err(|e| CoreError::internal(format!("stat {}: {e}", path.display())))?
.len();
let media_type = media_type_of(&candidate.name).unwrap_or("application/octet-stream");
let guard =
self.transfers
.begin(candidate.name.clone(), "", TransferDirection::Upload, size);
let file = std::fs::File::open(&path)
.map_err(|e| CoreError::internal(format!("open {}: {e}", path.display())))?;
let reader = CountingReader::new(std::io::BufReader::new(file), &guard);
let metadata = PhotoUploadMetadata {
capture_time: Some(candidate.wanted),
..Default::default()
};
let uploaded = self.rt.block_on(photos.upload_photo_from(
&candidate.name,
media_type,
reader,
size as i64,
Vec::new(),
metadata,
false,
));
drop(guard);
let new_uid = uploaded.map_err(|e| CoreError::from_api(&e, "re-upload photo"))?;
// Favourite and albums before the trash: a failure here leaves both
// copies on the account, which is recoverable by hand. Losing the
// original first would not be.
if candidate.favorite {
let update = PhotoTagsUpdate {
node_uid: new_uid.clone(),
tags_to_add: vec![PhotoTag::Favorite],
tags_to_remove: Vec::new(),
};
match self.rt.block_on(photos.update_photos(&[update])) {
// As in `set_photo_favorite`: claim our own echo, so the tag we
// just set does not come back as a remote change.
Ok(_) => self.note_self_change(&new_uid),
Err(e) => {
warn!(name = candidate.name, error = %e, "cannot re-favourite re-dated photo")
}
}
}
for album in &candidate.albums {
let Some(album_uid) = parse_node_uid(album) else {
continue;
};
match self
.rt
.block_on(photos.add_photos_to_album(&album_uid, std::slice::from_ref(&new_uid)))
{
Ok(_) => summary.album_links += 1,
Err(e) => {
warn!(name = candidate.name, error = %e, "cannot re-file re-dated photo")
}
}
}
let outcomes = self
.rt
.block_on(
self.client
.trash_nodes(std::slice::from_ref(&candidate.uid)),
)
.map_err(|e| CoreError::from_api(&e, "trash the mis-dated original"))?;
if let Some((_, Err(e))) = outcomes.into_iter().next() {
return Err(CoreError::from_api(&e, "trash the mis-dated original"));
}
summary.redated += 1;
summary.bytes += size;
info!(
name = candidate.name,
from = candidate.stored,
to = candidate.wanted,
"photo re-dated"
);
Ok(())
}
/// Start a re-date on its own thread. Errors when one is already running.
pub(crate) fn spawn_redate(
&self,
dry_run: bool,
limit: Option<usize>,
range: Option<(i64, i64)>,
) -> CoreResult<()> {
let cancel = Arc::new(AtomicBool::new(false));
{
let mut state = REDATE.lock();
if state.running {
return Err(CoreError::conflict(
"a photo re-date is already running; wait for it to finish or cancel it",
));
}
state.running = true;
state.cancel = Some(cancel.clone());
state.last = None;
}
let core = self.clone();
std::thread::spawn(move || {
let outcome = core.redate_photos(dry_run, limit, range, cancel);
let mut state = REDATE.lock();
state.running = false;
state.cancel = None;
state.last = outcome.as_ref().ok().cloned();
match &outcome {
Ok(summary) => info!(
candidates = summary.candidates,
redated = summary.redated,
failed = summary.failed,
"photo re-date finished"
),
Err(e) => warn!(error = %e, "photo re-date failed"),
}
});
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_uid_without_both_halves_is_not_a_node() {
assert!(parse_node_uid("vol~link").is_some());
assert!(parse_node_uid("vol~").is_none());
assert!(parse_node_uid("~link").is_none());
assert!(parse_node_uid("nothing").is_none());
}
#[test]
fn only_a_disagreement_larger_than_a_day_counts() {
// The name says 2023-02-19; a stored time inside that day is fine.
let wanted = capture_time_from_name("IMG-20230219-WA0001.jpg").unwrap();
assert!((wanted - (wanted + 3_600)).abs() <= DRIFT);
// An import three years later is not.
assert!((wanted - (wanted + 3 * 365 * 86_400)).abs() > DRIFT);
}
}
+16
View File
@@ -15,6 +15,7 @@ use pdfs_core::{CoreError, CoreResult};
use tracing::debug; use tracing::debug;
use crate::QUOTA_TTL; use crate::QUOTA_TTL;
use proton_drive_rs::proton_sdk::crypto::VerificationStatus;
use proton_drive_rs::proton_sdk::ids::NodeUid; use proton_drive_rs::proton_sdk::ids::NodeUid;
use proton_drive_rs::{MemberRole, NodeKind}; use proton_drive_rs::{MemberRole, NodeKind};
@@ -47,6 +48,18 @@ fn shared_entry_of(core: &Core, n: Node) -> DirEntry {
.map(role_to_str) .map(role_to_str)
.unwrap_or_default() .unwrap_or_default()
.to_string(); .to_string();
let invitation = n.membership.as_ref();
let shared_by = invitation
.and_then(|m| m.inviter_email.clone())
.unwrap_or_default();
let shared_at = invitation.and_then(|m| m.invite_time).unwrap_or(0);
// Only a signature that is present and wrong is a warning. An invitation
// that could not be checked — unsigned, or no keys for the claimed inviter —
// is not evidence of forgery, and flagging every such share would teach the
// user to ignore the flag.
let shared_by_unverified = invitation
.and_then(|m| m.inviter_verification)
.is_some_and(|status| status == VerificationStatus::Failed);
DirEntry { DirEntry {
name: n.name, name: n.name,
is_dir, is_dir,
@@ -57,6 +70,9 @@ fn shared_entry_of(core: &Core, n: Node) -> DirEntry {
path: core.rel_path_for_uid(&n.uid).unwrap_or_default(), path: core.rel_path_for_uid(&n.uid).unwrap_or_default(),
uid: n.uid.to_string(), uid: n.uid.to_string(),
role, role,
shared_by,
shared_at,
shared_by_unverified,
} }
} }
+5
View File
@@ -1120,6 +1120,8 @@ mod tests {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
@@ -1128,6 +1130,9 @@ mod tests {
share_id: ShareId::from("share"), share_id: ShareId::from("share"),
membership_id: ShareMembershipId::from("membership"), membership_id: ShareMembershipId::from("membership"),
permissions, permissions,
invite_time: None,
inviter_email: None,
inviter_verification: None,
} }
} }
+229
View File
@@ -0,0 +1,229 @@
//! The thread that watches the daemon for the thing the daemon cannot report
//! about itself: that it has stopped making progress.
//!
//! Three jobs, all cheap, all on one timer:
//!
//! 1. **Say so in the journal.** A worker that has held one job for minutes, a
//! queue that is not draining, a control request that never returned — each
//! gets a WARN naming it. The 2026-09-18 hang left nothing in the journal at
//! all; this is what turns the next one into a one-line diagnosis.
//! 2. **Ping the systemd watchdog**, but only after a real round trip over the
//! control socket. Pinging unconditionally from a thread that does nothing
//! else would keep the unit "healthy" through exactly the hang it is meant to
//! catch: the probe is what makes the ping mean "a client can still talk to
//! this daemon".
//! 3. **Sample memory.** That hang peaked at 5.2 GiB resident, and nobody knew
//! until it was over.
use std::io::{BufRead, BufReader, Write as _};
use std::os::unix::net::UnixStream;
use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};
use pdfs_core::control::Request as CtlRequest;
use tracing::{info, warn};
use super::Core;
use super::systemd;
/// How often the supervisor looks. Fast enough that a stall is named while the
/// user is still watching it, slow enough to cost nothing.
const TICK: Duration = Duration::from_secs(15);
/// How long one job may hold a worker, or one control request may run, before it
/// is called out. Generous: a large upload chunk or a cold folder listing on a
/// slow link is minutes of legitimate work, and a false WARN every few minutes
/// would train everyone to ignore the real one.
const STALL_AFTER: Duration = Duration::from_secs(120);
/// How often resident size goes into the log.
const RSS_EVERY: Duration = Duration::from_secs(300);
/// How long the liveness probe waits for the daemon to answer its own control
/// socket. Well under the watchdog period, so a probe that times out still
/// leaves time for the manager to hear nothing and act.
const PROBE_TIMEOUT: Duration = Duration::from_secs(10);
/// Run until shutdown. Spawned once per mount, from [`super::mount::mount`].
pub(crate) fn run(core: Core, control_socket: PathBuf) {
let watchdog = systemd::watchdog_period();
if let Some(period) = watchdog {
info!(period_secs = period.as_secs(), "systemd watchdog enabled");
}
let mut last_rss = Instant::now();
let mut last_ping = Instant::now();
// Per-lane completion counts from the previous tick, to tell a queue that is
// being worked through from one that is not moving at all.
let mut last_completed = (0_u64, 0_u64);
while core.shutdown.sleep(TICK) {
let report = core.diagnostics();
let stalled = report_stalls(&report, last_completed);
last_completed = (report.meta_completed, report.transfer_completed);
if last_rss.elapsed() >= RSS_EVERY {
info!(
rss_bytes = report.rss_bytes,
pending_ops = report.pending_ops,
meta_queued = report.meta_queued,
transfer_queued = report.transfer_queued,
stalled,
"daemon resource sample"
);
last_rss = Instant::now();
}
// A stalled lane is deliberately *not* a reason to withhold the ping:
// the mount may still be answering everything else, and a restart would
// throw away in-flight uploads. The probe decides that; a stall only
// gets said out loud.
if let Some(period) = watchdog
&& last_ping.elapsed() >= period
{
if probe(&control_socket) {
systemd::watchdog_ping();
} else {
warn!(
"daemon did not answer its own control socket; withholding the watchdog ping"
);
}
last_ping = Instant::now();
}
}
}
/// WARN about anything that has not moved for [`STALL_AFTER`]. Returns whether
/// anything was reported, so the caller can weigh it.
fn report_stalls(report: &pdfs_core::control::Diagnostics, last_completed: (u64, u64)) -> bool {
let threshold = STALL_AFTER.as_secs();
let mut stalled = false;
for worker in &report.workers {
if worker.busy && worker.age_secs >= threshold {
stalled = true;
warn!(
worker = %worker.name,
job = %worker.label,
age_secs = worker.age_secs,
"a fuse worker has held the same job for a long time"
);
}
}
if let Some(oldest) = report.inflight.first()
&& oldest.age_secs >= threshold
{
stalled = true;
warn!(
request = %oldest.kind,
age_secs = oldest.age_secs,
in_flight = report.inflight.len(),
"a control request has been running for a long time"
);
}
// A backed-up lane is normal; a backed-up lane that finished nothing since
// the last tick is not.
if report.meta_queued > 0 && report.meta_completed == last_completed.0 {
stalled = true;
warn!(
queued = report.meta_queued,
"the metadata lane has not completed a job since the last check"
);
}
if report.transfer_queued > 0 && report.transfer_completed == last_completed.1 {
stalled = true;
warn!(
queued = report.transfer_queued,
"the transfer lane has not completed a job since the last check"
);
}
stalled
}
/// One round trip over the control socket: connect, ask for the cheapest
/// request there is, read the answer.
///
/// This is the liveness the watchdog ping stands for. It covers the accept loop,
/// the handler pool and the reply path — everything a `pdfs` command or the GUI
/// depends on — without touching the network or the database.
fn probe(control_socket: &Path) -> bool {
let Ok(stream) = UnixStream::connect(control_socket) else {
return false;
};
if stream.set_read_timeout(Some(PROBE_TIMEOUT)).is_err()
|| stream.set_write_timeout(Some(PROBE_TIMEOUT)).is_err()
{
return false;
}
let Ok(request) = serde_json::to_string(&CtlRequest::Diagnostics) else {
return false;
};
let mut stream = stream;
if stream.write_all(format!("{request}\n").as_bytes()).is_err() {
return false;
}
let mut line = String::new();
BufReader::new(stream).read_line(&mut line).is_ok() && !line.trim().is_empty()
}
#[cfg(test)]
mod tests {
use super::*;
use pdfs_core::control::{Diagnostics, InflightRequest, WorkerState};
fn busy_worker(age_secs: u64) -> WorkerState {
WorkerState {
name: "pdfs-fuse-3".into(),
busy: true,
label: "read".into(),
age_secs,
}
}
/// A worker doing normal work must not be reported: a WARN every tick is a
/// WARN nobody reads.
#[test]
fn ordinary_work_is_not_a_stall() {
let report = Diagnostics {
workers: vec![busy_worker(5)],
..Default::default()
};
assert!(!report_stalls(&report, (0, 0)));
}
/// A worker on the same job past the threshold is the shape of the hang.
#[test]
fn a_long_held_job_is_a_stall() {
let report = Diagnostics {
workers: vec![busy_worker(STALL_AFTER.as_secs() + 1)],
..Default::default()
};
assert!(report_stalls(&report, (0, 0)));
}
/// So is a control request that never returned.
#[test]
fn a_long_running_request_is_a_stall() {
let report = Diagnostics {
inflight: vec![InflightRequest {
kind: "OpenFile".into(),
age_secs: STALL_AFTER.as_secs() + 1,
}],
..Default::default()
};
assert!(report_stalls(&report, (0, 0)));
}
/// A queue that is draining is busy, not stalled — that is what comparing
/// the completion counts is for.
#[test]
fn a_draining_queue_is_not_a_stall() {
let report = Diagnostics {
transfer_queued: 12,
transfer_completed: 40,
..Default::default()
};
assert!(!report_stalls(&report, (0, 30)));
assert!(report_stalls(&report, (0, 40)));
}
}
+2
View File
@@ -368,6 +368,8 @@ mod tests {
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
+99
View File
@@ -0,0 +1,99 @@
//! The `sd_notify` half of the service contract, without linking libsystemd.
//!
//! The protocol is one datagram of `KEY=VALUE` lines to the socket in
//! `$NOTIFY_SOCKET`. That is little enough to write directly, and writing it
//! directly keeps the daemon buildable on a machine without systemd headers —
//! everything here degrades to "no socket, do nothing" off systemd.
use std::os::unix::net::UnixDatagram;
use std::time::Duration;
use tracing::debug;
/// Send one notification. `false` when there is no manager to send it to, or the
/// send failed — a caller has nothing useful to do about either.
pub(crate) fn notify(message: &str) -> bool {
let Ok(socket_path) = std::env::var("NOTIFY_SOCKET") else {
return false;
};
if socket_path.is_empty() {
return false;
}
let Ok(socket) = UnixDatagram::unbound() else {
return false;
};
if let Some(name) = socket_path.strip_prefix('@') {
// Abstract namespace: the address is a NUL byte followed by the name,
// which `std`'s path-based API cannot express.
send_abstract(&socket, name, message)
} else {
socket.send_to(message.as_bytes(), &socket_path).is_ok()
}
}
/// `sendto` on an abstract-namespace address, which needs a hand-built
/// `sockaddr_un` because the first byte of the name is NUL.
fn send_abstract(socket: &UnixDatagram, name: &str, message: &str) -> bool {
use std::os::fd::AsRawFd;
let mut addr: libc::sockaddr_un = unsafe { std::mem::zeroed() };
addr.sun_family = libc::AF_UNIX as libc::sa_family_t;
let bytes = name.as_bytes();
// One byte for the leading NUL, and the path must still be addressable.
if bytes.len() + 1 > addr.sun_path.len() {
return false;
}
for (slot, byte) in addr.sun_path.iter_mut().skip(1).zip(bytes) {
*slot = *byte as libc::c_char;
}
let len = std::mem::size_of::<libc::sa_family_t>() + 1 + bytes.len();
let sent = unsafe {
libc::sendto(
socket.as_raw_fd(),
message.as_ptr().cast(),
message.len(),
0,
std::ptr::from_ref(&addr).cast(),
len as libc::socklen_t,
)
};
sent >= 0
}
/// Tell the manager the daemon is up and serving. Called once the mount is live,
/// so `systemctl --user start` returns when the mount actually works rather than
/// when the process exists.
pub(crate) fn ready() {
if notify("READY=1\n") {
debug!("notified systemd: ready");
}
}
/// Tell the manager the daemon is still healthy.
pub(crate) fn watchdog_ping() {
let _ = notify("WATCHDOG=1\n");
}
/// Tell the manager the daemon is stopping, so a slow teardown is not mistaken
/// for a hang by the watchdog it is racing.
pub(crate) fn stopping() {
let _ = notify("STOPPING=1\n");
}
/// How often the manager expects a ping, from `$WATCHDOG_USEC`, or `None` when
/// the watchdog is off or belongs to another process.
///
/// Reports *half* the configured interval, the interval systemd documents for
/// pinging: a ping period equal to the deadline would restart the daemon on the
/// first late scheduling.
pub(crate) fn watchdog_period() -> Option<Duration> {
// `WATCHDOG_PID` is set when the manager wants a specific process to answer;
// in that case a ping from anyone else is not the one it is waiting for.
if let Ok(pid) = std::env::var("WATCHDOG_PID")
&& pid.parse::<u32>().ok() != Some(std::process::id())
{
return None;
}
let usec: u64 = std::env::var("WATCHDOG_USEC").ok()?.parse().ok()?;
(usec > 0).then(|| Duration::from_micros(usec / 2))
}
+2
View File
@@ -51,6 +51,8 @@ pub(crate) fn virtual_node(parent_uid: NodeUid, name: String, timestamp: i64) ->
photo: None, photo: None,
album: None, album: None,
verification: Default::default(), verification: Default::default(),
direct_role: None,
share_id: None,
} }
} }
+333 -33
View File
@@ -12,6 +12,8 @@
use std::collections::VecDeque; use std::collections::VecDeque;
use std::sync::Arc; use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::time::Instant;
use parking_lot::{Condvar, Mutex}; use parking_lot::{Condvar, Mutex};
use tracing::warn; use tracing::warn;
@@ -51,8 +53,25 @@ pub(crate) const FUSE_WORKERS: usize = 11;
/// exactly the blocking this split exists to prevent. /// exactly the blocking this split exists to prevent.
const META_WORKERS: usize = 3; const META_WORKERS: usize = 3;
/// Queue depth that gets a line in the log, and the step at which it repeats.
/// Every job holds a `Reply` and a few fields, so depth is cheap — but a lane
/// this far behind is worth knowing about before the user reports a slow mount.
const QUEUE_WARN_STEP: usize = 256;
/// Whether a queue reaching `depth` is one of the depths worth logging.
fn crosses_warn_step(depth: usize) -> bool {
depth >= QUEUE_WARN_STEP && depth.is_multiple_of(QUEUE_WARN_STEP)
}
type Job = Box<dyn FnOnce() + Send + 'static>; type Job = Box<dyn FnOnce() + Send + 'static>;
/// A queued job and the name its caller gave it, so a job that never finishes
/// can be named in the diagnostics instead of showing up as "a worker is busy".
struct Queued {
label: &'static str,
job: Job,
}
/// Which lane a job belongs in. /// Which lane a job belongs in.
#[derive(Clone, Copy, PartialEq, Eq)] #[derive(Clone, Copy, PartialEq, Eq)]
pub(crate) enum Lane { pub(crate) enum Lane {
@@ -65,12 +84,63 @@ pub(crate) enum Lane {
#[derive(Default)] #[derive(Default)]
struct Queues { struct Queues {
meta: VecDeque<Job>, meta: VecDeque<Queued>,
transfer: VecDeque<Job>, transfer: VecDeque<Queued>,
/// Set when the pool is dropped, to wake and retire every worker. /// Set when the pool is dropped, to wake and retire every worker.
closed: bool, closed: bool,
} }
/// What one worker thread is doing, published for [`Workers::snapshot`].
///
/// Every field is an atomic (or a lock the worker only ever holds for the length
/// of a pointer write) because the whole point of the snapshot is to answer
/// *while the daemon is wedged*: a diagnostic that waits on the same lock the
/// stuck job holds reports nothing at exactly the moment it is needed.
struct Slot {
name: String,
/// True between picking a job up and finishing it.
busy: AtomicBool,
/// When the current state (busy or idle) began, as milliseconds since the
/// pool started. Monotonic, so an age is always a real duration.
since_ms: AtomicU64,
/// The label of the job in hand. Behind a mutex because a `&'static str` is
/// two words and there is no atomic for that; the worker holds it only to
/// swap the pointer, and the snapshot never blocks on it (see
/// [`Workers::snapshot`]).
label: Mutex<&'static str>,
}
impl Slot {
fn new(name: String, now_ms: u64) -> Self {
Self {
name,
busy: AtomicBool::new(false),
since_ms: AtomicU64::new(now_ms),
label: Mutex::new(""),
}
}
}
/// One worker's state at the moment [`Workers::snapshot`] read it.
pub(crate) struct WorkerSnapshot {
pub(crate) name: String,
pub(crate) busy: bool,
/// How long the worker has been in that state.
pub(crate) age_ms: u64,
/// The job in hand, or `""` when idle. `"?"` when the label could not be
/// read without blocking.
pub(crate) label: String,
}
/// The pool's state at the moment [`Workers::snapshot`] read it.
pub(crate) struct PoolSnapshot {
pub(crate) workers: Vec<WorkerSnapshot>,
pub(crate) meta_queued: usize,
pub(crate) transfer_queued: usize,
pub(crate) meta_completed: u64,
pub(crate) transfer_completed: u64,
}
/// The queues and the two wait sets over them. /// The queues and the two wait sets over them.
/// ///
/// Two condvars, not one, because the workers are not interchangeable: a /// Two condvars, not one, because the workers are not interchangeable: a
@@ -86,6 +156,22 @@ struct Pool {
meta_cv: Condvar, meta_cv: Condvar,
/// Waited on by the general workers. /// Waited on by the general workers.
general_cv: Condvar, general_cv: Condvar,
/// Queue depths, mirrored outside the lock so a snapshot can read them
/// without waiting on whatever is holding it.
meta_queued: AtomicUsize,
transfer_queued: AtomicUsize,
/// Jobs finished per lane. A depth that stays high while this stays still is
/// a stalled lane rather than a busy one.
meta_completed: AtomicU64,
transfer_completed: AtomicU64,
slots: Vec<Slot>,
started: Instant,
}
impl Pool {
fn now_ms(&self) -> u64 {
self.started.elapsed().as_millis() as u64
}
} }
/// Bounded thread pool behind the network-touching FUSE handlers. /// Bounded thread pool behind the network-touching FUSE handlers.
@@ -94,25 +180,46 @@ struct Pool {
/// on-demand sync folder), so the bound is per daemon rather than per mount. /// on-demand sync folder), so the bound is per daemon rather than per mount.
pub(crate) struct Workers { pub(crate) struct Workers {
inner: Arc<Pool>, inner: Arc<Pool>,
/// Kept so teardown can *join* the workers, not merely flag them. A worker
/// still inside a job when the process drops its tokio runtime panics with
/// "A Tokio 1.x context was found, but it is being shutdown" (seen
/// 2026-09-18 09:58:35), which is a teardown ordering bug rather than a real
/// failure.
handles: Mutex<Vec<std::thread::JoinHandle<()>>>,
} }
impl Workers { impl Workers {
pub(crate) fn new(n: usize) -> std::io::Result<Self> { pub(crate) fn new(n: usize) -> std::io::Result<Self> {
// Never leave the pool without a general worker, however small `n` is.
let meta_workers = META_WORKERS.min(n.saturating_sub(1));
let names: Vec<String> = (0..n)
.map(|i| {
format!(
"pdfs-fuse-{}{i}",
if i < meta_workers { "meta-" } else { "" }
)
})
.collect();
let inner = Arc::new(Pool { let inner = Arc::new(Pool {
queues: Mutex::new(Queues::default()), queues: Mutex::new(Queues::default()),
meta_cv: Condvar::new(), meta_cv: Condvar::new(),
general_cv: Condvar::new(), general_cv: Condvar::new(),
meta_queued: AtomicUsize::new(0),
transfer_queued: AtomicUsize::new(0),
meta_completed: AtomicU64::new(0),
transfer_completed: AtomicU64::new(0),
slots: names
.iter()
.map(|name| Slot::new(name.clone(), 0))
.collect(),
started: Instant::now(),
}); });
// Never leave the pool without a general worker, however small `n` is. let mut handles = Vec::with_capacity(n);
let meta_workers = META_WORKERS.min(n.saturating_sub(1)); for (i, name) in names.iter().enumerate() {
for i in 0..n {
let inner = inner.clone(); let inner = inner.clone();
let meta_only = i < meta_workers; let meta_only = i < meta_workers;
std::thread::Builder::new() let handle = std::thread::Builder::new()
.name(format!( .name(name.clone())
"pdfs-fuse-{}{i}",
if meta_only { "meta-" } else { "" }
))
.spawn(move || { .spawn(move || {
let cv = if meta_only { let cv = if meta_only {
&inner.meta_cv &inner.meta_cv
@@ -121,17 +228,20 @@ impl Workers {
}; };
loop { loop {
let mut q = inner.queues.lock(); let mut q = inner.queues.lock();
let job = loop { let picked = loop {
// A general worker takes transfers first: metadata // A general worker takes transfers first: metadata
// already has threads of its own, so draining the // already has threads of its own, so draining the
// bulk queue is the useful thing for it to do. // bulk queue is the useful thing for it to do.
let picked = if meta_only { let taken = if meta_only {
q.meta.pop_front() q.meta.pop_front().map(|job| (Lane::Meta, job))
} else { } else {
q.transfer.pop_front().or_else(|| q.meta.pop_front()) q.transfer
.pop_front()
.map(|job| (Lane::Transfer, job))
.or_else(|| q.meta.pop_front().map(|job| (Lane::Meta, job)))
}; };
if let Some(job) = picked { if let Some(taken) = taken {
break Some(job); break Some(taken);
} }
if q.closed { if q.closed {
break None; break None;
@@ -139,7 +249,15 @@ impl Workers {
cv.wait(&mut q); cv.wait(&mut q);
}; };
drop(q); drop(q);
let Some(job) = job else { break }; let Some((lane, queued)) = picked else { break };
match lane {
Lane::Meta => inner.meta_queued.fetch_sub(1, Ordering::Relaxed),
Lane::Transfer => inner.transfer_queued.fetch_sub(1, Ordering::Relaxed),
};
let slot = &inner.slots[i];
*slot.label.lock() = queued.label;
slot.since_ms.store(inner.now_ms(), Ordering::Relaxed);
slot.busy.store(true, Ordering::Relaxed);
// A panicking handler must not cost the pool a worker // A panicking handler must not cost the pool a worker
// for the rest of the run. The dropped `Reply` answers // for the rest of the run. The dropped `Reply` answers
// EIO on its own, so the caller of the failed op is // EIO on its own, so the caller of the failed op is
@@ -164,11 +282,70 @@ impl Workers {
// poisoning at all. Anything new reached from a job owes // poisoning at all. Anything new reached from a job owes
// the same check — an SDK-side `.lock().unwrap()` // the same check — an SDK-side `.lock().unwrap()`
// silently invalidates this comment. // silently invalidates this comment.
let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(job)); let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(queued.job));
slot.busy.store(false, Ordering::Relaxed);
slot.since_ms.store(inner.now_ms(), Ordering::Relaxed);
*slot.label.lock() = "";
match lane {
Lane::Meta => inner.meta_completed.fetch_add(1, Ordering::Relaxed),
Lane::Transfer => {
inner.transfer_completed.fetch_add(1, Ordering::Relaxed)
}
};
} }
})?; })?;
handles.push(handle);
} }
Ok(Self { inner }) Ok(Self {
inner,
handles: Mutex::new(handles),
})
}
/// Stop accepting work, wake every worker, and wait up to `deadline` for the
/// jobs in flight to finish.
///
/// Bounded because a join is only worth what it costs: a worker wedged on a
/// network call would otherwise hold the whole teardown, and the user asked
/// for the daemon to stop. What is stuck gets named in the log, and the
/// process continues shutting down around it.
pub(crate) fn stop_and_join(&self, deadline: std::time::Duration) {
self.close();
let handles: Vec<_> = std::mem::take(&mut *self.handles.lock());
if handles.is_empty() {
return;
}
let (tx, rx) = std::sync::mpsc::channel();
let joiner = std::thread::Builder::new()
.name("pdfs-fuse-join".into())
.spawn(move || {
for handle in handles {
let _ = handle.join();
}
let _ = tx.send(());
});
if joiner.is_err() {
return;
}
if rx.recv_timeout(deadline).is_err() {
let busy: Vec<String> = self
.snapshot()
.workers
.into_iter()
.filter(|worker| worker.busy)
.map(|worker| format!("{}={}", worker.name, worker.label))
.collect();
warn!(
stuck = busy.join(","),
"fuse workers did not finish in time; shutting down around them"
);
}
}
fn close(&self) {
self.inner.queues.lock().closed = true;
self.inner.meta_cv.notify_all();
self.inner.general_cv.notify_all();
} }
/// Queue `job` in `lane`. /// Queue `job` in `lane`.
@@ -179,7 +356,15 @@ impl Workers {
/// a pending job holds a `Reply` and a few fields, while the 4 MiB block /// a pending job holds a `Reply` and a few fields, while the 4 MiB block
/// buffer is allocated inside the job once it runs, and the SDK's in-flight /// buffer is allocated inside the job once it runs, and the SDK's in-flight
/// semaphore is what bounds how many of those exist at once. /// semaphore is what bounds how many of those exist at once.
pub(crate) fn run(&self, lane: Lane, job: impl FnOnce() + Send + 'static) { /// The queue is *not* bounded, and the depth warning above is deliberately
/// all this does about it: the only caller able to feel backpressure here is
/// fuser's dispatch loop, and blocking that is the stall the whole module
/// exists to avoid. A depth that keeps climbing is a symptom to read in the
/// log, not something to fix by stopping the mount.
///
/// `label` names the work for the diagnostics (`pdfs diagnostics`) and for
/// the stall warnings: it is what a hung daemon reports instead of "busy".
pub(crate) fn run(&self, lane: Lane, label: &'static str, job: impl FnOnce() + Send + 'static) {
let mut q = self.inner.queues.lock(); let mut q = self.inner.queues.lock();
if q.closed { if q.closed {
// Pre-pool behaviour: a shut-down pool degrades to a slow mount // Pre-pool behaviour: a shut-down pool degrades to a slow mount
@@ -189,11 +374,31 @@ impl Workers {
job(); job();
return; return;
} }
match lane { let queued = Queued {
Lane::Meta => q.meta.push_back(Box::new(job)), label,
Lane::Transfer => q.transfer.push_back(Box::new(job)), job: Box::new(job),
} };
let depth = match lane {
Lane::Meta => {
q.meta.push_back(queued);
self.inner.meta_queued.fetch_add(1, Ordering::Relaxed) + 1
}
Lane::Transfer => {
q.transfer.push_back(queued);
self.inner.transfer_queued.fetch_add(1, Ordering::Relaxed) + 1
}
};
drop(q); drop(q);
if crosses_warn_step(depth) {
warn!(
lane = if matches!(lane, Lane::Meta) {
"meta"
} else {
"transfer"
},
depth, label, "fuse worker queue is deep"
);
}
match lane { match lane {
// Either class can serve metadata, and only one of them needs to: // Either class can serve metadata, and only one of them needs to:
// whichever wakes first takes it, and the other finds the queue // whichever wakes first takes it, and the other finds the queue
@@ -209,13 +414,50 @@ impl Workers {
} }
} }
} }
/// What every worker is doing right now, without taking the queue lock.
///
/// A diagnostic is worth having exactly when the daemon is stuck, so this
/// reads atomics only, and `try_lock`s each label rather than waiting for
/// it: a label is held for one pointer write, so failing to get it means a
/// racing worker, not a wedged one, and `"?"` is a better answer than a
/// diagnostics request that hangs too.
pub(crate) fn snapshot(&self) -> PoolSnapshot {
let now = self.inner.now_ms();
let workers = self
.inner
.slots
.iter()
.map(|slot| {
let label = slot
.label
.try_lock()
.map_or_else(|| "?".to_string(), |label| (*label).to_string());
WorkerSnapshot {
name: slot.name.clone(),
busy: slot.busy.load(Ordering::Relaxed),
age_ms: now.saturating_sub(slot.since_ms.load(Ordering::Relaxed)),
label,
}
})
.collect();
PoolSnapshot {
workers,
meta_queued: self.inner.meta_queued.load(Ordering::Relaxed),
transfer_queued: self.inner.transfer_queued.load(Ordering::Relaxed),
meta_completed: self.inner.meta_completed.load(Ordering::Relaxed),
transfer_completed: self.inner.transfer_completed.load(Ordering::Relaxed),
}
}
} }
impl Drop for Workers { impl Drop for Workers {
/// A backstop for the paths that never call
/// [`stop_and_join`](Workers::stop_and_join) — tests, and a mount that fails
/// before teardown is wired up. Flag and wake only: a `Drop` that blocks on
/// a network call is worse than a worker outliving its pool by a moment.
fn drop(&mut self) { fn drop(&mut self) {
self.inner.queues.lock().closed = true; self.close();
self.inner.meta_cv.notify_all();
self.inner.general_cv.notify_all();
} }
} }
@@ -225,6 +467,64 @@ mod tests {
use std::sync::mpsc; use std::sync::mpsc;
use std::time::Duration; use std::time::Duration;
/// The depth warning fires on the step, and only on the step: a lane that
/// oscillates around the threshold must not log on every single job.
#[test]
fn queue_depth_warns_on_each_step() {
assert!(!crosses_warn_step(1));
assert!(!crosses_warn_step(QUEUE_WARN_STEP - 1));
assert!(crosses_warn_step(QUEUE_WARN_STEP));
assert!(!crosses_warn_step(QUEUE_WARN_STEP + 1));
assert!(crosses_warn_step(QUEUE_WARN_STEP * 3));
}
/// Teardown must return even while a job is running, and must not leave the
/// pool accepting work.
#[test]
fn stopping_is_bounded_and_closes_the_pool() {
let pool = Workers::new(2).unwrap();
let (tx, rx) = mpsc::channel();
pool.run(Lane::Meta, "test", move || tx.send(()).unwrap());
rx.recv_timeout(Duration::from_secs(5)).expect("it runs");
pool.stop_and_join(Duration::from_secs(5));
// A job queued after the stop runs inline rather than disappearing.
let (tx, rx) = mpsc::channel();
pool.run(Lane::Meta, "test", move || tx.send(()).unwrap());
rx.recv_timeout(Duration::from_secs(5))
.expect("a closed pool serves inline");
}
/// A diagnostic is only useful if it names the work. A running job must show
/// up under its label, and the queue behind it must show up as depth.
#[test]
fn a_snapshot_names_the_running_job_and_the_queue_behind_it() {
let pool = Workers::new(2).unwrap();
let (release_tx, release_rx) = mpsc::channel::<()>();
let release_rx = Arc::new(Mutex::new(release_rx));
let (started_tx, started_rx) = mpsc::channel();
{
let release_rx = release_rx.clone();
pool.run(Lane::Transfer, "read", move || {
started_tx.send(()).unwrap();
let _ = release_rx.lock().recv();
});
}
started_rx
.recv_timeout(Duration::from_secs(5))
.expect("the job starts");
let snapshot = pool.snapshot();
assert!(
snapshot.workers.iter().any(|w| w.busy && w.label == "read"),
"the running job is named"
);
assert_eq!(snapshot.transfer_completed, 0);
let _ = release_tx.send(());
}
/// Audit A6. Transfers filling every general worker must not delay a /// Audit A6. Transfers filling every general worker must not delay a
/// metadata job: that is what the reserved threads are for. /// metadata job: that is what the reserved threads are for.
#[test] #[test]
@@ -240,7 +540,7 @@ mod tests {
for _ in 0..general { for _ in 0..general {
let release_rx = release_rx.clone(); let release_rx = release_rx.clone();
let started_tx = started_tx.clone(); let started_tx = started_tx.clone();
pool.run(Lane::Transfer, move || { pool.run(Lane::Transfer, "test", move || {
started_tx.send(()).unwrap(); started_tx.send(()).unwrap();
let _ = release_rx.lock().recv(); let _ = release_rx.lock().recv();
}); });
@@ -255,13 +555,13 @@ mod tests {
// in the way that used to starve metadata. // in the way that used to starve metadata.
for _ in 0..16 { for _ in 0..16 {
let release_rx = release_rx.clone(); let release_rx = release_rx.clone();
pool.run(Lane::Transfer, move || { pool.run(Lane::Transfer, "test", move || {
let _ = release_rx.lock().recv(); let _ = release_rx.lock().recv();
}); });
} }
let (meta_tx, meta_rx) = mpsc::channel(); let (meta_tx, meta_rx) = mpsc::channel();
pool.run(Lane::Meta, move || meta_tx.send(()).unwrap()); pool.run(Lane::Meta, "test", move || meta_tx.send(()).unwrap());
meta_rx meta_rx
.recv_timeout(Duration::from_secs(5)) .recv_timeout(Duration::from_secs(5))
.expect("a metadata job runs while every transfer thread is stuck"); .expect("a metadata job runs while every transfer thread is stuck");
@@ -279,7 +579,7 @@ mod tests {
// must have been served by the general worker. // must have been served by the general worker.
let pool = Workers::new(1).unwrap(); let pool = Workers::new(1).unwrap();
let (tx, rx) = mpsc::channel(); let (tx, rx) = mpsc::channel();
pool.run(Lane::Meta, move || tx.send(()).unwrap()); pool.run(Lane::Meta, "test", move || tx.send(()).unwrap());
rx.recv_timeout(Duration::from_secs(5)) rx.recv_timeout(Duration::from_secs(5))
.expect("a general worker takes metadata when no transfer is waiting"); .expect("a general worker takes metadata when no transfer is waiting");
} }
@@ -288,9 +588,9 @@ mod tests {
#[test] #[test]
fn a_panicking_job_does_not_retire_its_worker() { fn a_panicking_job_does_not_retire_its_worker() {
let pool = Workers::new(1).unwrap(); let pool = Workers::new(1).unwrap();
pool.run(Lane::Meta, || panic!("boom")); pool.run(Lane::Meta, "test", || panic!("boom"));
let (tx, rx) = mpsc::channel(); let (tx, rx) = mpsc::channel();
pool.run(Lane::Meta, move || tx.send(()).unwrap()); pool.run(Lane::Meta, "test", move || tx.send(()).unwrap());
rx.recv_timeout(Duration::from_secs(5)) rx.recv_timeout(Duration::from_secs(5))
.expect("the pool still serves work after a panic"); .expect("the pool still serves work after a panic");
} }
+5 -4
View File
@@ -27,10 +27,11 @@ tracing.workspace = true
tracing-subscriber.workspace = true tracing-subscriber.workspace = true
anyhow.workspace = true anyhow.workspace = true
tokio.workspace = true tokio.workspace = true
serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
ksni = "0.2" ksni = { version = "0.3", features = ["blocking"] }
gtk4 = { version = "0.9", features = ["v4_12"] } gtk4 = { version = "0.11", features = ["v4_12"] }
adw = { package = "libadwaita", version = "0.7", features = ["v1_5"] } adw = { package = "libadwaita", version = "0.9", features = ["v1_5"] }
async-channel = "2" async-channel = "2"
exif = { version = "0.6", package = "kamadak-exif" } exif = { version = "0.6", package = "kamadak-exif" }
webkit6 = "0.4" webkit6 = "0.6"
+226 -43
View File
@@ -24,7 +24,7 @@ use widgets::versions_dialog::*;
use std::cell::{Cell, RefCell}; use std::cell::{Cell, RefCell};
use std::collections::{HashMap, HashSet, VecDeque}; use std::collections::{BTreeMap, HashMap, HashSet, VecDeque};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
@@ -78,6 +78,15 @@ const CONNECT_RETRY_INTERVAL: Duration = Duration::from_secs(2);
/// reload by clearing the page's timestamp. /// reload by clearing the page's timestamp.
const PAGE_TTL: Duration = Duration::from_secs(30); const PAGE_TTL: Duration = Duration::from_secs(30);
/// How often the gallery asks whether the timeline refresh it started has
/// finished.
const PHOTOS_REFRESH_POLL: Duration = Duration::from_secs(2);
/// How long the gallery keeps following one timeline refresh. A library of tens
/// of thousands of photos is re-read in batches over several minutes; past this
/// the daemon carries on and the page catches up on its next load.
const PHOTOS_REFRESH_FOLLOW_LIMIT: Duration = Duration::from_secs(15 * 60);
/// All widgets the periodic refresh and the action handlers mutate, plus the /// All widgets the periodic refresh and the action handlers mutate, plus the
/// resolved paths they act on. Wrapped in an [`Rc`] so handlers and the timeout /// resolved paths they act on. Wrapped in an [`Rc`] so handlers and the timeout
/// closure share one instance. /// closure share one instance.
@@ -162,6 +171,22 @@ impl Ui {
} }
} }
} }
/// Mark a cached thumbnail as freshly used, so eviction takes the tiles
/// nobody has looked at rather than the oldest ones.
///
/// Insertion order alone evicts the top of the timeline first — exactly what
/// a scroll back up then asks for again.
fn touch_texture(&self, uid: &str) {
let mut order = self.gallery.photo_tex_order.borrow_mut();
if order.back().is_some_and(|back| back == uid) {
return;
}
if let Some(at) = order.iter().position(|held| held == uid) {
order.remove(at);
order.push_back(uid.to_string());
}
}
} }
fn main() -> glib::ExitCode { fn main() -> glib::ExitCode {
@@ -237,15 +262,19 @@ fn load_proton_theme() {
.viewer-status {{ color: white; font-size: 1rem; background-color: rgba(0, 0, 0, 0.75); padding: 12px 24px; border-radius: 12px; }}\n\ .viewer-status {{ color: white; font-size: 1rem; background-color: rgba(0, 0, 0, 0.75); padding: 12px 24px; border-radius: 12px; }}\n\
.viewer-info-panel {{ background-color: @window_bg_color; border-left: 1px solid alpha(currentColor, 0.12); }}\n\ .viewer-info-panel {{ background-color: @window_bg_color; border-left: 1px solid alpha(currentColor, 0.12); }}\n\
.gallery-day {{ font-size: 0.82rem; font-weight: 700; letter-spacing: 0.04em; text-transform: uppercase; color: alpha(currentColor, 0.65); padding: 4px 2px 8px 2px; }}\n\ .gallery-day {{ font-size: 0.82rem; font-weight: 700; letter-spacing: 0.04em; text-transform: uppercase; color: alpha(currentColor, 0.65); padding: 4px 2px 8px 2px; }}\n\
.photo-tile {{ padding: 0; margin: 0; min-height: 0; min-width: 0; border-radius: 14px; background: alpha(currentColor, 0.06); box-shadow: none; transition: box-shadow 180ms ease, background 180ms ease; }}\n\ .photo-tile {{ padding: 0; margin: 0; min-height: 0; min-width: 0; border-radius: 6px; background: none; box-shadow: none; transition: box-shadow 180ms ease; }}\n\
.photo-tile:hover {{ box-shadow: 0 8px 22px rgba(0, 0, 0, 0.45); }}\n\ .photo-tile:hover {{ box-shadow: 0 8px 22px rgba(0, 0, 0, 0.45); }}\n\
.photo-tile:focus {{ outline: 2px solid {PROTON_PURPLE}; outline-offset: -2px; }}\n\ .photo-tile:focus {{ outline: 2px solid {PROTON_PURPLE}; outline-offset: -2px; }}\n\
.photo-thumb {{ transition: transform 220ms ease; }}\n\ .photo-thumb {{ transition: transform 220ms ease; }}\n\
.photo-tile:hover .photo-thumb {{ transform: scale(1.06); }}\n\ .photo-tile:hover .photo-thumb {{ transform: scale(1.06); }}\n\
.photo-check {{ color: white; background: rgba(0, 0, 0, 0.5); border-radius: 999px; padding: 3px; }}\n\
.photo-check-on {{ background: {PROTON_PURPLE}; }}\n\
.photo-tile-selected {{ outline: 3px solid {PROTON_PURPLE}; outline-offset: -3px; }}\n\
.photo-placeholder {{ color: alpha(currentColor, 0.35); background: alpha(currentColor, 0.07); }}\n\ .photo-placeholder {{ color: alpha(currentColor, 0.35); background: alpha(currentColor, 0.07); }}\n\
.photo-caption {{ font-size: 0.78rem; color: white; text-shadow: 0 1px 3px rgba(0, 0, 0, 0.9); padding: 22px 10px 6px 10px; opacity: 0; transition: opacity 180ms ease; }}\n\ .photo-caption {{ font-size: 0.78rem; color: white; text-shadow: 0 1px 3px rgba(0, 0, 0, 0.9); padding: 22px 10px 6px 10px; opacity: 0; transition: opacity 180ms ease; }}\n\
.photo-video-badge {{ color: white; background: rgba(0, 0, 0, 0.45); border-radius: 999px; padding: 8px; min-width: 20px; min-height: 20px; box-shadow: 0 2px 8px rgba(0, 0, 0, 0.5); transition: background 160ms ease; }}\n\ .photo-video-badge {{ color: white; background: rgba(0, 0, 0, 0.45); border-radius: 999px; padding: 8px; min-width: 20px; min-height: 20px; box-shadow: 0 2px 8px rgba(0, 0, 0, 0.5); transition: background 160ms ease; }}\n\
.photo-tile:hover .photo-video-badge {{ background: alpha({PROTON_PURPLE}, 0.85); }}\n\ .photo-tile:hover .photo-video-badge {{ background: alpha({PROTON_PURPLE}, 0.85); }}\n\
.photo-group-badge {{ font-size: 0.68rem; font-weight: 700; letter-spacing: 0.04em; color: white; background: rgba(0, 0, 0, 0.55); border-radius: 6px; padding: 1px 5px; margin: 6px; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.9); }}\n\
.photo-tile:hover .photo-caption {{ opacity: 1; background: linear-gradient(to top, rgba(0, 0, 0, 0.55), rgba(0, 0, 0, 0)); }}\n\ .photo-tile:hover .photo-caption {{ opacity: 1; background: linear-gradient(to top, rgba(0, 0, 0, 0.55), rgba(0, 0, 0, 0)); }}\n\
.album-card {{ padding: 0; border-radius: 16px; transition: background 180ms ease; }}\n\ .album-card {{ padding: 0; border-radius: 16px; transition: background 180ms ease; }}\n\
.album-card:hover {{ background: alpha(currentColor, 0.07); }}\n\ .album-card:hover {{ background: alpha(currentColor, 0.07); }}\n\
@@ -443,7 +472,11 @@ fn build_window(app: &adw::Application) {
gallery: GalleryState { gallery: GalleryState {
model: gallery_widgets.model.clone(), model: gallery_widgets.model.clone(),
groups: gallery_widgets.groups.clone(), groups: gallery_widgets.groups.clone(),
tile: Cell::new(TILE_DEFAULT), row_height: Cell::new(ROW_DEFAULT),
learned_ratios: RefCell::new(HashMap::new()),
scrolling_down: Cell::new(true),
scroll_offset: Cell::new(0.0),
assumed_ratios: RefCell::new(HashSet::new()),
content: gallery_widgets.content.clone(), content: gallery_widgets.content.clone(),
status: gallery_widgets.status.clone(), status: gallery_widgets.status.clone(),
retry: gallery_widgets.retry.clone(), retry: gallery_widgets.retry.clone(),
@@ -484,7 +517,14 @@ fn build_window(app: &adw::Application) {
decode_idle: Cell::new(false), decode_idle: Cell::new(false),
thumb_source: RefCell::new(None), thumb_source: RefCell::new(None),
relayout_source: RefCell::new(None), relayout_source: RefCell::new(None),
bound: RefCell::new(HashMap::new()), bound: RefCell::new(BTreeMap::new()),
list: gallery_widgets.list.clone(),
selecting: Cell::new(false),
selected: RefCell::new(HashSet::new()),
select_btn: gallery_widgets.select_btn.clone(),
select_bar: gallery_widgets.select_bar.clone(),
select_label: gallery_widgets.select_label.clone(),
select_trash: gallery_widgets.select_trash.clone(),
}, },
shared: SharedState { shared: SharedState {
content: shared_widgets.content.clone(), content: shared_widgets.content.clone(),
@@ -582,7 +622,12 @@ fn build_window(app: &adw::Application) {
); );
wire_details(&ui); wire_details(&ui);
wire_search(&ui); wire_search(&ui);
wire_gallery(&ui, &gallery_widgets.list, &gallery_widgets.scroll); wire_gallery(
&ui,
&gallery_widgets.list,
&gallery_widgets.scroll,
&gallery_widgets.select_done,
);
wire_gallery_empty( wire_gallery_empty(
&ui, &ui,
&gallery_widgets.empty_upload, &gallery_widgets.empty_upload,
@@ -803,7 +848,7 @@ fn reload_current_page(ui: &Rc<Ui>) {
} }
// One scope covers the whole photos view, so Refresh reloads whichever of // One scope covers the whole photos view, so Refresh reloads whichever of
// the two — album grid or timeline/album — is actually on screen. // the two — album grid or timeline/album — is actually on screen.
Some("gallery") => refresh_then(ui, RefreshScope::Photos, |ui| { Some("gallery") => refresh_then(ui, RefreshScope::Photos { full: false }, |ui| {
if ui.gallery.content.visible_child_name().as_deref() == Some("albums") { if ui.gallery.content.visible_child_name().as_deref() == Some("albums") {
load_albums(ui); load_albums(ui);
} else { } else {
@@ -838,11 +883,54 @@ fn reload_current_page(ui: &Rc<Ui>) {
/// how to report an unreachable daemon on its own page, and a refresh that fails /// how to report an unreachable daemon on its own page, and a refresh that fails
/// silently would read as a dead button. /// silently would read as a dead button.
fn refresh_then(ui: &Rc<Ui>, scope: RefreshScope, load: fn(&Rc<Ui>)) { fn refresh_then(ui: &Rc<Ui>, scope: RefreshScope, load: fn(&Rc<Ui>)) {
// A photos refresh answers as soon as it has started, because re-reading a
// large library takes minutes. The page is therefore loaded twice: once from
// what the daemon already holds, and again when the refresh has landed.
let follow = matches!(scope, RefreshScope::Photos { .. });
let rx = spawn_request(ui.dirs.control_socket(), Request::Refresh { scope }); let rx = spawn_request(ui.dirs.control_socket(), Request::Refresh { scope });
let ui = ui.clone(); let ui = ui.clone();
glib::spawn_future_local(async move { glib::spawn_future_local(async move {
let _ = rx.recv().await; let _ = rx.recv().await;
load(&ui); load(&ui);
if follow {
follow_photos_refresh(&ui, load);
}
});
}
/// Watch the timeline refresh the daemon just started, and load the page again
/// when it finishes.
///
/// Giving up after [`PHOTOS_REFRESH_FOLLOW_LIMIT`] is not a failure: the refresh
/// keeps running in the daemon and the next visit to the page shows its result.
/// It only stops this front-end from polling a daemon forever.
fn follow_photos_refresh(ui: &Rc<Ui>, load: fn(&Rc<Ui>)) {
let ui = ui.clone();
let give_up_at = Instant::now() + PHOTOS_REFRESH_FOLLOW_LIMIT;
glib::spawn_future_local(async move {
loop {
glib::timeout_future(PHOTOS_REFRESH_POLL).await;
// The person may have left the gallery in the meantime; reloading a
// page nobody is looking at would only cost the daemon work.
if ui.stack.visible_child_name().as_deref() != Some("gallery") {
return;
}
let rx = spawn_request(ui.dirs.control_socket(), Request::PhotosRefreshStatus);
match rx.recv().await {
Ok(Ok(Response::PhotosRefresh { running: true })) => {
if Instant::now() >= give_up_at {
return;
}
}
Ok(Ok(Response::PhotosRefresh { running: false })) => {
load(&ui);
return;
}
// An older daemon does not know the request, and anything else
// means we cannot tell — either way, stop asking.
_ => return,
}
}
}); });
} }
@@ -1169,54 +1257,149 @@ fn human_bytes(bytes: u64) -> String {
mod tests { mod tests {
use super::*; use super::*;
/// The full width one row of `columns` tiles occupies, gaps included. /// The full width a row occupies, gaps included.
fn row_width(columns: usize, edge: i32) -> i32 { fn row_width(widths: &[i32]) -> i32 {
edge * columns as i32 + TILE_GAP * (columns as i32 - 1) widths.iter().sum::<i32>() + TILE_GAP * (widths.len() as i32 - 1)
}
/// Ratios of a typical mixed day: landscape phone shots, a portrait, a
/// square crop.
fn mixed() -> Vec<f64> {
vec![
1.5, 1.5, 0.75, 1.0, 1.33, 1.5, 0.75, 1.78, 1.0, 1.5, 1.5, 1.33,
]
} }
#[test] #[test]
fn the_grid_spans_the_content_width() { fn every_full_row_spans_the_content_width() {
// Whatever the width, the columns plus their gaps land on it (bar the // The point of justifying: no ragged right margin at any width, at any
// few px integer division cannot divide), so no ragged right margin. // zoom, for any mix of shapes.
for width in [640, 900, 1000, 1440, 1920, 2560] { for width in [640, 900, 1000, 1440, 1920, 2560] {
let (columns, edge) = plan_grid(TILE_DEFAULT, width); let rows = plan_rows(&mixed(), width, ROW_DEFAULT);
let used = row_width(columns, edge); // The last row is deliberately short, so it is not part of this.
assert!( for (_, widths) in rows.iter().take(rows.len() - 1) {
used <= width && width - used < columns as i32, assert_eq!(row_width(widths), width, "row does not span {width}px");
"{columns} x {edge}px = {used}px does not span {width}px" }
);
} }
} }
#[test] #[test]
fn tiles_stay_near_the_target_size() { fn a_tile_keeps_its_photos_shape() {
// The column count rounds and the edge absorbs the remainder, so a tile // Each tile is its photo's ratio at the row's height, which is what
// is never more than one gap-and-a-bit away from what was asked for. // makes cropping unnecessary.
let (columns, edge) = plan_grid(TILE_DEFAULT, 1920); let ratios = mixed();
assert!(columns >= 7, "1920px should hold several 220px tiles"); let rows = plan_rows(&ratios, 1440, ROW_DEFAULT);
assert!( let (height, widths) = &rows[0];
(edge - TILE_DEFAULT).abs() < TILE_DEFAULT / 3, for (width, ratio) in widths.iter().zip(&ratios).take(widths.len() - 1) {
"{edge}px is not near the {TILE_DEFAULT}px target" let laid_out = *width as f64 / *height as f64;
assert!((laid_out - ratio).abs() < 0.02, "{laid_out} is not {ratio}");
}
}
#[test]
fn rows_land_near_the_target_height() {
// A row is closed as soon as it no longer fits at the target, so it is
// never taller than the target and never far below it.
for target in [ROW_MIN, ROW_DEFAULT, ROW_MAX] {
let rows = plan_rows(&mixed(), 1600, target);
for (height, _) in rows.iter().take(rows.len() - 1) {
assert!(
*height <= target,
"{height}px exceeds the {target}px target"
);
assert!(*height > target / 2, "{height}px is far under {target}px");
}
}
}
#[test]
fn a_short_day_is_not_stretched_across_the_window() {
// Two photos are two photos, not two half-window tiles.
let rows = plan_rows(&[1.5, 1.5], 1920, ROW_DEFAULT);
assert_eq!(rows.len(), 1);
let (height, widths) = &rows[0];
assert_eq!(*height, ROW_DEFAULT);
assert!(row_width(widths) < 1920 / 2);
}
#[test]
fn zooming_in_puts_fewer_photos_in_a_row() {
let small = plan_rows(&mixed(), 1200, ROW_MIN);
let big = plan_rows(&mixed(), 1200, ROW_MAX);
assert!(small[0].1.len() > big[0].1.len());
}
#[test]
fn every_photo_is_laid_out_exactly_once() {
let ratios = mixed();
let placed: usize = plan_rows(&ratios, 1000, ROW_DEFAULT)
.iter()
.map(|(_, widths)| widths.len())
.sum();
assert_eq!(placed, ratios.len());
assert!(plan_rows(&[], 1000, ROW_DEFAULT).is_empty());
}
/// One file of a shot, for the group-switch tests.
fn member(uid: &str, name: Option<&str>, kind: PhotoKind) -> PhotoItem {
PhotoItem {
uid: uid.into(),
capture_time: 0,
thumb_path: None,
name: name.map(str::to_string),
ratio: None,
no_thumb: false,
kind,
favorite: false,
group_size: 2,
has_raw: kind == PhotoKind::Raw,
}
}
/// The lightbox steps through a shot's files and comes back round, so a
/// burst of three is as reachable as a RAW+JPEG pair.
#[test]
fn the_group_switch_cycles_through_every_file() {
let members = [
member("jpeg", Some("IMG_1.JPG"), PhotoKind::Photo),
member("raw", Some("IMG_1.CR2"), PhotoKind::Raw),
member("clip", None, PhotoKind::Video),
];
let next = |uid| next_group_member(&members, uid).map(|item| item.uid.as_str());
assert_eq!(next("jpeg"), Some("raw"));
assert_eq!(next("raw"), Some("clip"));
assert_eq!(next("clip"), Some("jpeg"));
// A photo that is not in the group leaves the button alone rather than
// jumping somewhere arbitrary.
assert_eq!(next("elsewhere"), None);
assert!(next_group_member(&[], "jpeg").is_none());
}
/// The switch names what it will show; a file whose name the daemon has not
/// resolved still gets a sentence rather than a blank.
#[test]
fn a_group_member_without_a_name_is_still_described() {
assert_eq!(
member_label(&member("raw", Some("IMG_1.CR2"), PhotoKind::Raw)),
"IMG_1.CR2"
);
assert_eq!(
member_label(&member("raw", None, PhotoKind::Raw)),
"the raw file"
);
assert_eq!(
member_label(&member("clip", Some(""), PhotoKind::Video)),
"the video"
); );
} }
#[test] #[test]
fn zooming_in_widens_the_tiles_and_drops_columns() { fn a_window_narrower_than_one_photo_still_lays_it_out() {
let (wide_columns, small_edge) = plan_grid(TILE_MIN, 1200); // A tile of at least one px, rather than a zero-width widget or a
let (few_columns, big_edge) = plan_grid(TILE_MAX, 1200); // division by zero.
assert!(wide_columns > few_columns); for width in [0, 40] {
assert!(big_edge > small_edge); let rows = plan_rows(&[1.5, 1.5, 1.5], width, ROW_DEFAULT);
} assert!(rows.iter().all(|(h, w)| *h > 0 && w.iter().all(|w| *w > 0)));
}
#[test]
fn a_window_narrower_than_one_tile_still_gets_a_column() {
// A single column at whatever fits, rather than zero columns (which would
// divide by zero) or a zero-px tile.
let (columns, edge) = plan_grid(TILE_DEFAULT, 40);
assert_eq!(columns, 1);
assert!(edge > 0);
let (columns, edge) = plan_grid(TILE_DEFAULT, 0);
assert_eq!(columns, 1);
assert!(edge > 0);
} }
} }
+3
View File
@@ -2345,6 +2345,9 @@ pub(crate) fn repaint_search(ui: &Rc<Ui>, hits: &[SearchHit]) {
uid: h.uid.clone(), uid: h.uid.clone(),
path: h.path.clone(), path: h.path.clone(),
role: String::new(), role: String::new(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
}) })
.collect(); .collect();
entries.sort_by(|a, b| { entries.sort_by(|a, b| {
@@ -24,6 +24,11 @@ pub(crate) struct Viewer {
/// uses for its pin switch, so painting a photo doesn't fire a round-trip. /// uses for its pin switch, so painting a photo doesn't fire a round-trip.
pub(crate) favorite: gtk4::ToggleButton, pub(crate) favorite: gtk4::ToggleButton,
pub(crate) favorite_suppress: Cell<bool>, pub(crate) favorite_suppress: Cell<bool>,
/// The button that switches between the files of one shot, and the files it
/// switches between (the JPEG and the RAW of a photo, a live photo and its
/// clip). Empty, and the button hidden, for a photo stored as one file.
pub(crate) group_btn: gtk4::Button,
pub(crate) group: RefCell<Vec<PhotoItem>>,
/// uid of the photo currently on screen. /// uid of the photo currently on screen.
pub(crate) uid: RefCell<String>, pub(crate) uid: RefCell<String>,
/// On-disk path of the full-size photo, once it has been downloaded. /// On-disk path of the full-size photo, once it has been downloaded.
@@ -81,6 +86,59 @@ fn set_gallery_favorite(ui: &Rc<Ui>, uid: &str, favorite: bool) {
/// Show the photo behind `uid`: paint its (already cached) thumbnail immediately /// Show the photo behind `uid`: paint its (already cached) thumbnail immediately
/// so the lightbox never opens on a blank screen, ask the daemon for the /// so the lightbox never opens on a blank screen, ask the daemon for the
/// full-size file, and swap it in — plus its EXIF — when it lands. /// full-size file, and swap it in — plus its EXIF — when it lands.
/// Ask the daemon which files this shot was stored as, and offer a switch when
/// there is more than one. The grid shows such a shot as a single tile, so the
/// lightbox is the only place the RAW is reachable.
fn load_photo_group(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: &str) {
viewer.group.borrow_mut().clear();
viewer.group_btn.set_visible(false);
let rx = spawn_request(
ui.dirs.control_socket(),
Request::PhotoGroup {
uid: uid.to_string(),
},
);
let viewer = viewer.clone();
let uid = uid.to_string();
glib::spawn_future_local(async move {
let Ok(Ok(Response::Photos { items, .. })) = rx.recv().await else {
return;
};
// A late reply for a photo the user has already left must not relabel
// the button under the one now on screen.
if *viewer.uid.borrow() != uid || items.len() < 2 {
return;
}
if let Some(next) = next_group_member(&items, &uid) {
viewer
.group_btn
.set_tooltip_text(Some(&format!("Show {}", member_label(next))));
}
*viewer.group.borrow_mut() = items;
viewer.group_btn.set_visible(true);
});
}
/// The member after `uid`, wrapping round — the button steps through a shot's
/// files rather than toggling two of them, so a burst of three works too.
pub(crate) fn next_group_member<'a>(members: &'a [PhotoItem], uid: &str) -> Option<&'a PhotoItem> {
let at = members.iter().position(|item| item.uid == uid)?;
members.get((at + 1) % members.len())
}
/// What to call one file of a shot: its name, or its kind when the daemon has
/// not resolved a name for it.
pub(crate) fn member_label(item: &PhotoItem) -> String {
match item.name.as_deref() {
Some(name) if !name.is_empty() => name.to_string(),
_ => match item.kind {
PhotoKind::Raw => "the raw file".to_string(),
PhotoKind::Video => "the video".to_string(),
PhotoKind::Photo => "the photo".to_string(),
},
}
}
pub(crate) fn load_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: String) { pub(crate) fn load_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: String) {
viewer.spinner.set_visible(true); viewer.spinner.set_visible(true);
viewer.spinner.start(); viewer.spinner.start();
@@ -96,6 +154,7 @@ pub(crate) fn load_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: String) {
.and_downcast::<BoxedAnyObject>() .and_downcast::<BoxedAnyObject>()
.is_some_and(|boxed| boxed.borrow::<PhotoItem>().favorite); .is_some_and(|boxed| boxed.borrow::<PhotoItem>().favorite);
show_favorite(viewer, favorite); show_favorite(viewer, favorite);
load_photo_group(ui, viewer, &uid);
// The thumbnail the gallery already decoded stands in for the full photo // The thumbnail the gallery already decoded stands in for the full photo
// while it downloads: blurry for a moment beats black for a second. // while it downloads: blurry for a moment beats black for a second.
@@ -503,7 +562,13 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) {
favorite_btn.add_css_class("flat"); favorite_btn.add_css_class("flat");
favorite_btn.add_css_class("viewer-action-btn"); favorite_btn.add_css_class("viewer-action-btn");
// Only shown for a shot stored as more than one file; `load_photo_group`
// decides that per photo.
let group_btn = action("view-paged-symbolic", "Other files of this photo");
group_btn.set_visible(false);
let download_btn = action("document-save-symbolic", "Save a copy…"); let download_btn = action("document-save-symbolic", "Save a copy…");
let delete_btn = action("user-trash-symbolic", "Move to Trash (Delete)");
let open_ext_btn = action("document-open-symbolic", "Open with another app"); let open_ext_btn = action("document-open-symbolic", "Open with another app");
let close_btn = action("window-close-symbolic", "Close (Esc)"); let close_btn = action("window-close-symbolic", "Close (Esc)");
close_btn.add_css_class("viewer-close-btn"); close_btn.add_css_class("viewer-close-btn");
@@ -513,9 +578,11 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) {
top_bar.set_valign(gtk4::Align::Start); top_bar.set_valign(gtk4::Align::Start);
top_bar.set_hexpand(true); top_bar.set_hexpand(true);
top_bar.append(&titles); top_bar.append(&titles);
top_bar.append(&group_btn);
top_bar.append(&favorite_btn); top_bar.append(&favorite_btn);
top_bar.append(&info_toggle); top_bar.append(&info_toggle);
top_bar.append(&download_btn); top_bar.append(&download_btn);
top_bar.append(&delete_btn);
top_bar.append(&open_ext_btn); top_bar.append(&open_ext_btn);
top_bar.append(&close_btn); top_bar.append(&close_btn);
overlay.add_overlay(&top_bar); overlay.add_overlay(&top_bar);
@@ -614,6 +681,8 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) {
info_map: info_map.clone(), info_map: info_map.clone(),
favorite: favorite_btn.clone(), favorite: favorite_btn.clone(),
favorite_suppress: Cell::new(false), favorite_suppress: Cell::new(false),
group_btn: group_btn.clone(),
group: RefCell::new(Vec::new()),
coords: RefCell::new(None), coords: RefCell::new(None),
uid: RefCell::new(initial_uid.clone()), uid: RefCell::new(initial_uid.clone()),
path: RefCell::new(None), path: RefCell::new(None),
@@ -695,6 +764,30 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) {
}); });
}); });
// Deleting the photo on screen closes the lightbox: what it was showing is
// gone, and leaving it open on a trashed photo invites a second Delete on
// something the user cannot see.
let ui_delete = ui.clone();
let viewer_delete = viewer.clone();
let w_delete = window.clone();
delete_btn.connect_clicked(move |_| {
let uid = viewer_delete.uid.borrow().clone();
w_delete.close();
trash_photos(&ui_delete, vec![uid]);
});
let ui_group = ui.clone();
let viewer_group = viewer.clone();
group_btn.connect_clicked(move |_| {
let uid = viewer_group.uid.borrow().clone();
let next =
next_group_member(&viewer_group.group.borrow(), &uid).map(|item| item.uid.clone());
if let Some(next) = next {
*viewer_group.uid.borrow_mut() = next.clone();
load_photo(&ui_group, &viewer_group, next);
}
});
let w_download = window.clone(); let w_download = window.clone();
let viewer_download = viewer.clone(); let viewer_download = viewer.clone();
download_btn.connect_clicked(move |_| { download_btn.connect_clicked(move |_| {
@@ -757,6 +850,11 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) {
w_key.fullscreen(); w_key.fullscreen();
} }
} }
Some("Delete" | "KP_Delete") => {
let uid = viewer_key.uid.borrow().clone();
w_key.close();
trash_photos(&ui_key, vec![uid]);
}
Some("Escape" | "q") => w_key.close(), Some("Escape" | "q") => w_key.close(),
Some("w") if ctrl => w_key.close(), Some("w") if ctrl => w_key.close(),
_ => return glib::Propagation::Proceed, _ => return glib::Propagation::Proceed,
File diff suppressed because it is too large. Load diff
+48 -4
View File
@@ -309,6 +309,17 @@ fn shared_entry_row(ui: &Rc<Ui>, entry: &DirEntry) -> adw::ActionRow {
if let Some(badge) = role_badge(&entry.role) { if let Some(badge) = role_badge(&entry.role) {
row.add_suffix(&badge); row.add_suffix(&badge);
} }
// The "shared by" in the subtitle is what the invitation claims. When its
// signature does not check out against that person's keys, the claim must
// not read as fact — someone may be impersonating a contact to get a file
// opened.
if entry.shared_by_unverified {
let warning = gtk4::Image::from_icon_name("dialog-warning-symbolic");
warning.add_css_class("warning");
warning.set_valign(gtk4::Align::Center);
warning.set_tooltip_text(Some(UNVERIFIED_INVITER_TOOLTIP));
row.add_suffix(&warning);
}
row.add_prefix(&file_thumbnail(ui, entry, 40, 24, true)); row.add_prefix(&file_thumbnail(ui, entry, 40, 24, true));
let ui_act = ui.clone(); let ui_act = ui.clone();
let uid = entry.uid.clone(); let uid = entry.uid.clone();
@@ -329,12 +340,29 @@ fn shared_entry_row(ui: &Rc<Ui>, entry: &DirEntry) -> adw::ActionRow {
row row
} }
/// A shared row's subtitle: its size (files only), and where it can be reached /// Tooltip on the warning shown next to a share whose invitation did not verify.
/// in the local mount once the synthetic `Shared with me/` directory has interned const UNVERIFIED_INVITER_TOOLTIP: &str = "The invitation's signature doesn't match the sender's keys. \
/// it. An unlisted share has no local path yet, and says nothing rather than It may not really be from them.";
/// pointing at a directory that is not there.
/// A shared row's subtitle: who shared it and when (share roots only — that is
/// where the invitation lives), its size (files only), and where it can be
/// reached in the local mount once the synthetic `Shared with me/` directory has
/// interned it. An unlisted share has no local path yet, and says nothing rather
/// than pointing at a directory that is not there.
pub(crate) fn shared_row_subtitle(entry: &DirEntry) -> String { pub(crate) fn shared_row_subtitle(entry: &DirEntry) -> String {
let mut parts: Vec<String> = Vec::new(); let mut parts: Vec<String> = Vec::new();
if !entry.shared_by.is_empty() {
let date = if entry.shared_at > 0 {
format_modified(entry.shared_at)
} else {
String::new()
};
parts.push(if date.is_empty() {
format!("Shared by {}", entry.shared_by)
} else {
format!("Shared by {} on {date}", entry.shared_by)
});
}
if !entry.is_dir { if !entry.is_dir {
parts.push(human_bytes(entry.size)); parts.push(human_bytes(entry.size));
} }
@@ -723,9 +751,25 @@ mod tests {
uid: "vol~link".into(), uid: "vol~link".into(),
path: path.into(), path: path.into(),
role: role.into(), role: role.into(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
} }
} }
#[test]
fn a_share_root_says_who_shared_it() {
let mut root = entry(true, "", "viewer");
root.shared_by = "alice@proton.me".into();
assert_eq!(shared_row_subtitle(&root), "Shared by alice@proton.me");
root.shared_at = 1_700_000_000;
let subtitle = shared_row_subtitle(&root);
assert!(
subtitle.starts_with("Shared by alice@proton.me on "),
"{subtitle}"
);
}
#[test] #[test]
fn a_share_with_no_local_path_says_nothing_about_one() { fn a_share_with_no_local_path_says_nothing_about_one() {
// Before the synthetic directory has been listed, the daemon cannot // Before the synthetic directory has been listed, the daemon cannot
@@ -293,5 +293,8 @@ pub(crate) fn shared_item_as_entry(item: &SharedItem) -> DirEntry {
path: item.path.clone(), path: item.path.clone(),
// Shared *by* me: I own it, so there is no role of mine to report. // Shared *by* me: I own it, so there is no role of mine to report.
role: String::new(), role: String::new(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
} }
} }
+15 -172
View File
@@ -1,26 +1,23 @@
//! `pdfs-prompt --dmenu`: the same search, presented in the user's own launcher. //! `pdfs-prompt --dmenu`: the same search, presented in the user's own launcher.
//! //!
//! Everything here is blocking and GTK-free — the process exists for one //! Everything here is blocking and GTK-free — the process exists for one
//! round-trip and then exits, so there is no main loop to keep off. It reuses //! round-trip and then exits, so there is no main loop to keep off. Talking to
//! the GTK prompt's [`Hit`](crate::Hit) model and ranking, and its Drive //! the daemon and opening a chosen file are [`crate::query`]'s, shared with the
//! activation policy, so the two front ends cannot drift on what "open" means. //! fzf front end and built on the GTK prompt's own [`Hit`](crate::Hit) model
//! and activation policy; this module is only presentation.
//! //!
//! Launchers filter a static list; they cannot call back for a new one per //! Launchers filter a static list; they cannot call back for a new one per
//! keystroke. So the interaction is a loop: the first menu shows pinned files //! keystroke. So the interaction is a loop: the first menu shows pinned files
//! and accepts free text, and any text that matches no entry becomes the next //! and accepts free text, and any text that matches no entry becomes the next
//! search. Escape (an empty selection) ends it. //! search. Escape (an empty selection) ends it. `--fzf` ([`crate::fzf`]) is the
//! front end for users who want results to appear while they type instead.
use std::path::{Path, PathBuf};
use pdfs_core::config::AppDirs; use pdfs_core::config::AppDirs;
use pdfs_core::control::{
Request, Response, SearchFilters, SearchHit, SearchKind, SearchSource, send,
};
use pdfs_core::menu::{self, MenuChoice, MenuItem, PromptConfig}; use pdfs_core::menu::{self, MenuChoice, MenuItem, PromptConfig};
use pdfs_core::opener::{self, OpenWith}; use pdfs_core::opener::OpenWith;
use crate::activation::{DriveActivation, drive_activation, mounted_or_relative, mounted_target}; use crate::Hit;
use crate::{Hit, file_name, is_document, is_image, is_media, rank_hits}; use crate::query;
/// Guard against a launcher that keeps handing back text we cannot satisfy — /// Guard against a launcher that keeps handing back text we cannot satisfy —
/// without it a scripted (non-interactive) menu could spin forever. /// without it a scripted (non-interactive) menu could spin forever.
@@ -48,10 +45,7 @@ pub(crate) fn run(options: Options) -> Result<(), String> {
.to_string() .to_string()
})?; })?;
let mountpoint = match request(&socket, Request::Status)? { let mountpoint = query::mountpoint(&socket, &dirs);
Response::Status { mountpoint, .. } => PathBuf::from(mountpoint),
_ => dirs.default_mountpoint(),
};
let mut query = options let mut query = options
.query .query
@@ -59,8 +53,8 @@ pub(crate) fn run(options: Options) -> Result<(), String> {
.filter(|q| !q.is_empty()); .filter(|q| !q.is_empty());
for _ in 0..MAX_ROUNDS { for _ in 0..MAX_ROUNDS {
let hits = match &query { let hits = match &query {
Some(text) => search(&socket, text, prompt.resolved_menu_limit())?, Some(text) => query::search(&socket, text, prompt.resolved_menu_limit())?,
None => pins(&socket)?, None => query::pins(&socket)?,
}; };
let items: Vec<MenuItem> = if hits.is_empty() { let items: Vec<MenuItem> = if hits.is_empty() {
vec![MenuItem::new(placeholder(query.as_deref()), None)] vec![MenuItem::new(placeholder(query.as_deref()), None)]
@@ -77,7 +71,7 @@ pub(crate) fn run(options: Options) -> Result<(), String> {
query = None; query = None;
continue; continue;
}; };
return open(&socket, &mountpoint, &policy, hit, query.is_none()); return query::open(&socket, &mountpoint, &policy, hit, query.is_none());
} }
// No entry matched what was typed: treat it as the next query. // No entry matched what was typed: treat it as the next query.
MenuChoice::Custom(text) => query = Some(text), MenuChoice::Custom(text) => query = Some(text),
@@ -118,57 +112,6 @@ fn placeholder(query: Option<&str>) -> String {
} }
} }
/// Pinned files — the same "you asked to keep these" set the GTK prompt opens
/// with, and the only listing the daemon offers without a query.
fn pins(socket: &Path) -> Result<Vec<Hit>, String> {
let Response::Pins { pins } = request(socket, Request::ListPins)? else {
return Ok(Vec::new());
};
Ok(pins
.into_iter()
.map(|pin| {
Hit::Drive(SearchHit {
name: file_name(&pin.path),
path: pin.path,
is_dir: pin.is_dir.unwrap_or(pin.recursive),
size: 0,
modified: 0,
pinned: true,
uid: pin.uid,
mounted_path: None,
score: 0,
})
})
.collect())
}
fn search(socket: &Path, query: &str, limit: usize) -> Result<Vec<Hit>, String> {
let reply = request(
socket,
Request::SearchV2 {
query: query.to_string(),
limit,
filters: SearchFilters {
sources: vec![SearchSource::Drive, SearchSource::Local],
kind: SearchKind::All,
},
},
)?;
let Response::SearchResultsV2 {
drive_hits,
local_hits,
..
} = reply
else {
return Ok(Vec::new());
};
let mut hits: Vec<Hit> = drive_hits.into_iter().map(Hit::Drive).collect();
hits.extend(local_hits.into_iter().map(Hit::Local));
rank_hits(&mut hits);
hits.truncate(limit);
Ok(hits)
}
/// One launcher line per hit. Labels must be unique: they are how the selection /// One launcher line per hit. Labels must be unique: they are how the selection
/// is matched back, and two files of the same name in the same folder listing /// is matched back, and two files of the same name in the same folder listing
/// would otherwise be indistinguishable. /// would otherwise be indistinguishable.
@@ -181,101 +124,15 @@ fn label_all(hits: &[Hit]) -> Vec<MenuItem> {
label = format!("{} · {} ({suffix})", hit.name(), hit.location()); label = format!("{} · {} ({suffix})", hit.name(), hit.location());
suffix += 1; suffix += 1;
} }
items.push(MenuItem::new(label, Some(icon_name(hit)))); items.push(MenuItem::new(label, Some(query::icon_name(hit))));
} }
items items
} }
/// Icon theme name for a hit.
///
/// Coarse on purpose: `content_type_guess` would give an exact MIME name, but
/// only with file *contents* to sniff — a Drive hit is metadata, and passing it
/// an empty buffer answers `application-x-zerosize` for everything. The generic
/// names below exist in every icon theme.
fn icon_name(hit: &Hit) -> String {
if hit.is_dir() {
return "folder".to_string();
}
let name = hit.name();
if is_image(name) {
"image-x-generic"
} else if is_media(name) {
"video-x-generic"
} else if is_document(name) {
"text-x-generic"
} else {
"application-x-generic"
}
.to_string()
}
/// Open a chosen hit, mirroring the GTK prompt: local files are already on
/// disk, a Drive folder or streamable file goes through the mount, and anything
/// else is materialised by the daemon first.
fn open(
socket: &Path,
mountpoint: &Path,
policy: &OpenWith,
hit: &Hit,
from_pins: bool,
) -> Result<(), String> {
match hit {
Hit::Local(local) => {
opener::open(policy, Path::new(&local.path), local.is_dir);
Ok(())
}
Hit::Drive(drive) => {
// A folder has no materialised form, and a pin row's `is_dir` is pin
// policy rather than node kind, so both go to the mount unchecked.
let unconditional = from_pins
|| matches!(
drive_activation(&drive.name, drive.is_dir),
DriveActivation::Folder | DriveActivation::MountedMedia
);
if unconditional {
opener::open(
policy,
&mounted_or_relative(mountpoint, drive),
drive.is_dir,
);
return Ok(());
}
// Anything else the mount exposes opens there too, so an editor
// saves back into Drive instead of into a cache blob.
if let Some(path) = mounted_target(mountpoint, drive) {
opener::open(policy, &path, false);
return Ok(());
}
match request(
socket,
Request::OpenFile {
path: drive.path.clone(),
uid: Some(drive.uid.clone()),
},
)? {
Response::FilePath { path } => {
// The reply is a content-hash blob; the rules key off the
// Drive name, which is what the user actually opened.
opener::open_named(policy, Path::new(&path), &drive.name, false);
Ok(())
}
Response::Error { message, .. } => Err(format!("could not open: {message}")),
_ => Err("unexpected reply from the daemon".to_string()),
}
}
}
}
/// One control-socket round-trip, with the daemon-is-down case phrased for a
/// terminal rather than a status page.
fn request(socket: &Path, request: Request) -> Result<Response, String> {
send(socket, &request).map_err(|e| format!("cannot reach the Proton Drive daemon: {e}"))
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use pdfs_core::control::LocalHit; use pdfs_core::control::{LocalHit, SearchHit};
fn drive(name: &str, path: &str, is_dir: bool) -> Hit { fn drive(name: &str, path: &str, is_dir: bool) -> Hit {
Hit::Drive(SearchHit { Hit::Drive(SearchHit {
@@ -331,18 +188,4 @@ mod tests {
assert!(placeholder(None).contains("type to search")); assert!(placeholder(None).contains("type to search"));
assert!(placeholder(Some("md")).contains("md")); assert!(placeholder(Some("md")).contains("md"));
} }
#[test]
fn icons_are_theme_names_the_launcher_can_look_up() {
assert_eq!(icon_name(&drive("Photos", "Photos", true)), "folder");
assert_eq!(
icon_name(&drive("a.png", "a.png", false)),
"image-x-generic"
);
assert_eq!(icon_name(&drive("a.md", "a.md", false)), "text-x-generic");
assert_eq!(
icon_name(&drive("a.bin", "a.bin", false)),
"application-x-generic"
);
}
} }
+372
View File
@@ -0,0 +1,372 @@
//! `pdfs-prompt --fzf`: launcher search that queries the daemon per keystroke.
//!
//! A dmenu-protocol launcher (fuzzel, rofi, wofi, …) is handed one static list
//! and filters it itself; it cannot ask for a new one as the user types, which
//! is why [`crate::dmenu`] has to make the search a two-step. `fzf` can:
//! `--bind change:reload(…)` re-runs a command on every keystroke. So this
//! front end re-executes *this binary* in `--feed` mode, and the daemon's own
//! ranking ([`pdfs_core::control::Request::SearchV2`]) decides the order —
//! `--disabled` turns fzf's matcher off so it does not re-sort what came back.
//!
//! Three roles, all the same binary:
//!
//! * **outer** — `--fzf` with no tty (a WM keybinding): spawn a terminal that
//! runs the inner role, and exit.
//! * **inner** — `--fzf` with a tty: run fzf, open what was chosen.
//! * **feed** — `--feed <TEXT>`: print one line per hit and exit. fzf spawns
//! this per keystroke and kills the one it superseded.
//!
//! A line is `display\tpayload`, shown through `--with-nth=1` so only the left
//! half is visible. The payload is the JSON [`Hit`], which is what makes the
//! selection unambiguous: unlike the dmenu path, identity does not have to be
//! recovered from the label, so [`crate::dmenu::label_all`]'s uniquifying
//! suffix has no counterpart here. JSON escapes tabs and newlines, so no file
//! name can split a line.
use std::io::{BufRead, BufReader, IsTerminal, Write};
use std::path::PathBuf;
use std::process::{Command, Stdio};
use pdfs_core::config::AppDirs;
use pdfs_core::menu::{self, PromptConfig};
use pdfs_core::opener::OpenWith;
use crate::query;
use crate::{Hit, SEARCH_DEBOUNCE};
/// Separates the visible half of a feed line from its payload.
const FIELD: char = '\t';
/// What the launcher prompt reads, matching the dmenu front end's wording.
const PROMPT: &str = "Search Drive › ";
pub(crate) struct Options {
/// Search for this immediately instead of opening on the pinned files.
pub query: Option<String>,
/// Set by the terminal we spawned: run fzf here even without a tty rather
/// than spawning a second terminal, which would recurse forever if the
/// terminal does not give its child one.
pub inner: bool,
}
/// Run the fzf flow. Returns an error only for conditions the user must act on
/// (no fzf, no terminal, no daemon); a plain cancel is `Ok`.
pub(crate) fn run(options: Options) -> Result<(), String> {
if options.inner || std::io::stdin().is_terminal() {
return interactive(options.query);
}
spawn_terminal(options.query)
}
/// Re-launch this binary's inner role inside a terminal.
///
/// Not waited on: a keybinding should return immediately, and the terminal
/// owns the rest of the interaction — including opening the chosen file, which
/// `opener` detaches anyway.
fn spawn_terminal(query: Option<String>) -> Result<(), String> {
let dirs = AppDirs::new().map_err(|e| format!("cannot resolve app dirs: {e}"))?;
let prompt: PromptConfig = dirs.load_config().resolved_prompt();
let terminal = menu::resolve_terminal(prompt.terminal.as_ref()).ok_or_else(|| {
"no terminal found — install foot/kitty/alacritty, or set prompt.terminal in config.json"
.to_string()
})?;
let argv = menu::with_command(&terminal, &inner_command(self_exe()?, query.as_deref()));
let (program, args) = argv
.split_first()
.ok_or_else(|| "no terminal command configured".to_string())?;
Command::new(program)
.args(args)
.spawn()
.map_err(|e| format!("cannot run terminal {program}: {e}"))?;
Ok(())
}
/// The argv the spawned terminal runs.
fn inner_command(exe: PathBuf, query: Option<&str>) -> Vec<String> {
let mut argv = vec![
exe.display().to_string(),
"--fzf".to_string(),
"--inner".to_string(),
];
if let Some(text) = query.filter(|text| !text.trim().is_empty()) {
argv.push("--query".to_string());
argv.push(text.to_string());
}
argv
}
/// Run fzf here and act on the choice.
fn interactive(query: Option<String>) -> Result<(), String> {
if !menu::on_path("fzf") {
return Err("fzf is not installed — install it, or use --dmenu".to_string());
}
let dirs = AppDirs::new().map_err(|e| format!("cannot resolve app dirs: {e}"))?;
let config = dirs.load_config();
let prompt: PromptConfig = config.resolved_prompt();
let policy: OpenWith = config.resolved_open_with();
let socket = dirs.control_socket();
let query = query
.map(|q| q.trim().to_string())
.filter(|q| !q.is_empty());
// fzf only fires `change` for a *keystroke*, so the first screen — pins, or
// the results for an initial --query — has to arrive on stdin.
let initial = hits(&socket, query.as_deref(), prompt.resolved_menu_limit());
let argv = fzf_argv(&self_exe()?, query.as_deref());
let (program, args) = argv.split_first().expect("fzf_argv is never empty");
let mut child = Command::new(program)
.args(args)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.map_err(|e| format!("cannot run fzf: {e}"))?;
// Written on a worker for the same reason the dmenu path does it: fzf reads
// stdin only once its UI is up, and a full pipe buffer would deadlock us.
let payload: String = initial.iter().map(encode_line).collect();
let mut stdin = child
.stdin
.take()
.ok_or_else(|| "fzf stdin unavailable".to_string())?;
let writer = std::thread::spawn(move || {
let _ = stdin.write_all(payload.as_bytes());
let _ = stdin.flush();
});
let mut selected = String::new();
if let Some(stdout) = child.stdout.take() {
let mut reader = BufReader::new(stdout);
let mut line = String::new();
while reader.read_line(&mut line).unwrap_or(0) > 0 {
if selected.is_empty() {
selected = line.trim_end_matches('\n').to_string();
}
line.clear();
}
}
let _ = child.wait();
let _ = writer.join();
// Escape, or Enter on the "cannot reach the daemon" row, which carries no
// payload — neither is an error.
let Some(hit) = decode_line(&selected) else {
return Ok(());
};
let mountpoint = query::mountpoint(&socket, &dirs);
// Only the no-query screen is the pin list, and only there does a row's
// `is_dir` mean pin policy rather than node kind.
query::open(&socket, &mountpoint, &policy, &hit, query.is_none())
}
/// `--feed`: one round-trip, printed for fzf to swallow.
///
/// Always succeeds. A reload command that exits non-zero leaves fzf showing an
/// empty list with no explanation, so a failure becomes a visible row instead.
pub(crate) fn feed(query: Option<&str>) {
let Ok(dirs) = AppDirs::new() else {
println!("(cannot resolve app dirs)");
return;
};
let limit = dirs.load_config().resolved_prompt().resolved_menu_limit();
let socket = dirs.control_socket();
let query = query.map(str::trim).filter(|text| !text.is_empty());
let stdout = std::io::stdout();
let mut out = stdout.lock();
for line in hits(&socket, query, limit).iter().map(encode_line) {
let _ = out.write_all(line.as_bytes());
}
}
/// The rows for one screen: pins when there is nothing typed, search results
/// otherwise — the same split the GTK prompt makes on an empty entry.
fn hits(socket: &std::path::Path, query: Option<&str>, limit: usize) -> Vec<Hit> {
let result = match query {
Some(text) => query::search(socket, text, limit),
None => query::pins(socket),
};
match result {
Ok(hits) => hits,
Err(message) => {
// Printed as a row rather than to stderr: fzf's alternate screen
// hides stderr, so this is the only place the user would see it.
println!("({message})");
Vec::new()
}
}
}
fn fzf_argv(exe: &std::path::Path, query: Option<&str>) -> Vec<String> {
let feed = format!("{} --feed {{q}}", shell_quote(&exe.display().to_string()));
// The sleep is the debounce: fzf kills a reload child that a later keystroke
// superseded, so a burst of typing costs one daemon round-trip, exactly as
// the GTK prompt's timeout does.
let debounce = SEARCH_DEBOUNCE.as_secs_f32();
let mut argv: Vec<String> = [
"fzf",
// fzf's own fuzzy matcher would re-order and re-filter what the daemon
// already ranked, and would hide rows it does not think match.
"--disabled",
"--no-sort",
"--no-multi",
"--layout=reverse",
"--info=inline",
"--delimiter=\t",
"--with-nth=1",
]
.iter()
.map(|arg| (*arg).to_string())
.collect();
argv.push(format!("--prompt={PROMPT}"));
// `reload:` rather than `reload(…)`: the unparenthesised form runs to the
// end of the argument, so an install path containing a bracket cannot
// truncate the command fzf parses out of it.
argv.push(format!("--bind=change:reload:sleep {debounce}; {feed}"));
if let Some(text) = query {
argv.push(format!("--query={text}"));
}
argv
}
/// `display\tpayload\n`. The display half is scrubbed of the delimiter and of
/// newlines; the payload is JSON, which escapes both.
fn encode_line(hit: &Hit) -> String {
let display =
format!("{} · {}", hit.name(), hit.location()).replace(['\n', '\r', FIELD], " ");
let payload = serde_json::to_string(hit).unwrap_or_default();
format!("{display}{FIELD}{payload}\n")
}
/// The hit a selected line stands for, or `None` for a line that carries no
/// payload — the error/empty row, or an empty selection (Escape).
fn decode_line(line: &str) -> Option<Hit> {
let (_, payload) = line.trim_end_matches('\n').rsplit_once(FIELD)?;
serde_json::from_str(payload).ok()
}
fn self_exe() -> Result<PathBuf, String> {
std::env::current_exe().map_err(|e| format!("cannot locate pdfs-prompt: {e}"))
}
/// POSIX single-quoting, for the one string that goes through fzf's `sh -c`.
fn shell_quote(text: &str) -> String {
format!("'{}'", text.replace('\'', r"'\''"))
}
#[cfg(test)]
mod tests {
use super::*;
use pdfs_core::control::{LocalHit, SearchHit};
fn drive(name: &str, path: &str) -> Hit {
Hit::Drive(SearchHit {
name: name.into(),
path: path.into(),
is_dir: false,
size: 7,
modified: 9,
pinned: false,
uid: "uid~1".into(),
mounted_path: None,
score: 42,
})
}
#[test]
fn a_line_round_trips_through_the_payload() {
let hit = drive("notes.md", "Docs/notes.md");
let line = encode_line(&hit);
assert!(line.ends_with('\n'));
let back = decode_line(&line).expect("payload decodes");
assert_eq!(back.name(), "notes.md");
assert_eq!(back.key(), hit.key());
assert_eq!(back.score(), 42);
}
#[test]
fn a_local_hit_round_trips_as_itself() {
let hit = Hit::Local(LocalHit {
name: "notes.md".into(),
path: "/home/me/Docs/notes.md".into(),
is_dir: false,
size: 1,
modified: 2,
score: 3,
});
let back = decode_line(&encode_line(&hit)).expect("payload decodes");
assert!(matches!(back, Hit::Local(_)));
assert_eq!(back.key(), hit.key());
}
#[test]
fn a_name_with_a_tab_or_newline_stays_one_line() {
let hit = drive("two\tparts\nfile.md", "Docs/two\tparts\nfile.md");
let line = encode_line(&hit);
assert_eq!(line.matches('\n').count(), 1, "{line:?}");
assert_eq!(line.matches(FIELD).count(), 1, "{line:?}");
assert_eq!(
decode_line(&line).expect("payload decodes").name(),
"two\tparts\nfile.md"
);
}
#[test]
fn a_row_without_a_payload_is_not_a_hit() {
assert!(decode_line("").is_none());
assert!(decode_line("(cannot reach the Proton Drive daemon)").is_none());
assert!(decode_line("bogus\tnot json").is_none());
}
#[test]
fn fzf_is_told_not_to_do_its_own_matching() {
let argv = fzf_argv(std::path::Path::new("/usr/bin/pdfs-prompt"), None);
assert!(argv.contains(&"--disabled".to_string()), "{argv:?}");
assert!(argv.contains(&"--no-sort".to_string()), "{argv:?}");
assert!(
argv.iter().any(|arg| arg.starts_with("--with-nth=")),
"the payload field must stay hidden: {argv:?}"
);
}
#[test]
fn every_keystroke_re_runs_the_feed_after_the_debounce() {
let argv = fzf_argv(std::path::Path::new("/usr/bin/pdfs-prompt"), Some("md"));
let bind = argv
.iter()
.find(|arg| arg.starts_with("--bind=change:"))
.expect("a change binding");
assert!(bind.contains("reload"), "{bind}");
assert!(bind.contains("--feed {q}"), "{bind}");
assert!(bind.contains(&format!("sleep {}", SEARCH_DEBOUNCE.as_secs_f32())));
assert!(argv.contains(&"--query=md".to_string()), "{argv:?}");
}
#[test]
fn a_path_with_a_quote_cannot_break_out_of_the_reload_command() {
let argv = fzf_argv(std::path::Path::new("/home/o'brien/my prompt"), None);
let bind = argv
.iter()
.find(|arg| arg.starts_with("--bind=change:"))
.expect("a change binding");
assert!(bind.contains(r"'/home/o'\''brien/my prompt'"), "{bind}");
// The unparenthesised reload form: a bracket in the path must not end
// the command as far as fzf's own parser is concerned.
assert!(bind.starts_with("--bind=change:reload:"), "{bind}");
}
#[test]
fn the_terminal_runs_the_inner_role_so_it_cannot_recurse() {
let argv = inner_command(PathBuf::from("/usr/bin/pdfs-prompt"), Some("md"));
assert_eq!(
argv,
vec!["/usr/bin/pdfs-prompt", "--fzf", "--inner", "--query", "md"]
);
assert_eq!(
inner_command(PathBuf::from("/usr/bin/pdfs-prompt"), Some(" ")),
vec!["/usr/bin/pdfs-prompt", "--fzf", "--inner"],
"a blank query is no query"
);
}
}
+9 -8
View File
@@ -14,8 +14,9 @@ use std::process::Command;
use std::time::Duration; use std::time::Duration;
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use ksni::blocking::TrayMethods;
use ksni::menu::StandardItem; use ksni::menu::StandardItem;
use ksni::{MenuItem, Tray, TrayService}; use ksni::{MenuItem, Tray};
use pdfs_core::auth; use pdfs_core::auth;
use pdfs_core::config::AppDirs; use pdfs_core::config::AppDirs;
use pdfs_core::control::{ use pdfs_core::control::{
@@ -282,13 +283,13 @@ fn main() -> Result<()> {
state: poll_state(&socket, &default_mountpoint), state: poll_state(&socket, &default_mountpoint),
}; };
let service = TrayService::new(tray); // The tray autostarts with the session and can beat the desktop's SNI
let handle = service.handle(); // watcher onto the bus. Without `assume_sni_available` that race is a hard
std::thread::spawn(move || { // spawn error; with it the item waits and registers once the watcher shows.
if let Err(e) = service.run() { let handle = tray
tracing::error!("tray service stopped: {e:?}"); .assume_sni_available(true)
} .spawn()
}); .context("start tray service")?;
// Poll the daemon forever, pushing each fresh snapshot into the tray so the // Poll the daemon forever, pushing each fresh snapshot into the tray so the
// menu reflects mount/login changes made elsewhere (e.g. via the CLI). // menu reflects mount/login changes made elsewhere (e.g. via the CLI).
+44 -6
View File
@@ -24,6 +24,8 @@ use pdfs_core::opener::{self, OpenWith};
mod activation; mod activation;
mod dmenu; mod dmenu;
mod fzf;
mod query;
use activation::{DriveActivation, drive_activation, mounted_or_relative, mounted_target}; use activation::{DriveActivation, drive_activation, mounted_or_relative, mounted_target};
const APP_ID: &str = "io.narl.proton-drive-linux-prompt"; const APP_ID: &str = "io.narl.proton-drive-linux-prompt";
@@ -125,7 +127,7 @@ fn is_media(name: &str) -> bool {
/// One row in the unified result list. The two sections hold different payloads /// One row in the unified result list. The two sections hold different payloads
/// — a Drive hit must be hydrated through the daemon before it can be opened, a /// — a Drive hit must be hydrated through the daemon before it can be opened, a
/// local file is already on disk — but they share one keyboard cursor. /// local file is already on disk — but they share one keyboard cursor.
#[derive(Clone)] #[derive(Clone, serde::Serialize, serde::Deserialize)]
enum Hit { enum Hit {
Drive(SearchHit), Drive(SearchHit),
Local(LocalHit), Local(LocalHit),
@@ -341,12 +343,20 @@ Options:
rofi, …) instead of the built-in GTK window. Also rofi, …) instead of the built-in GTK window. Also
settable permanently as \"prompt\": { \"mode\": \"dmenu\" } settable permanently as \"prompt\": { \"mode\": \"dmenu\" }
in config.json. in config.json.
--fzf Search in fzf, in a terminal, re-querying the daemon on
every keystroke — results appear as you type, unlike
--dmenu. Also settable as \"prompt\": { \"mode\": \"fzf\" }.
--gtk Force the built-in window, overriding that setting. --gtk Force the built-in window, overriding that setting.
--menu <COMMAND> Launcher command line for --dmenu, e.g. --menu <COMMAND> Launcher command line for --dmenu, e.g.
--menu 'fuzzel --dmenu --width 60'. Overrides --menu 'fuzzel --dmenu --width 60'. Overrides
\"prompt\": { \"menu\": [...] }. \"prompt\": { \"menu\": [...] }.
--query <TEXT> Search for TEXT immediately instead of opening on the --query <TEXT> Search for TEXT immediately instead of opening on the
pinned-files list (--dmenu only). pinned-files list (--dmenu and --fzf only).
--feed <TEXT> Print the hits for TEXT, one per line, and exit. This is
what --fzf re-runs on each keystroke; it is not meant to
be typed.
--inner Run fzf here rather than spawning a terminal. Set by the
terminal --fzf spawns; not meant to be typed.
-h, --help Show this help. -h, --help Show this help.
"; ";
@@ -356,6 +366,12 @@ struct Args {
mode: Option<PromptMode>, mode: Option<PromptMode>,
menu: Option<Vec<String>>, menu: Option<Vec<String>>,
query: Option<String>, query: Option<String>,
/// `--feed`: print hits for this text and exit. `Some(None)` is `--feed`
/// with an empty argument, which fzf sends for an empty input — a real
/// request for the pinned list, not an absent flag.
feed: Option<Option<String>>,
/// `--inner`: we are the process a spawned terminal is running.
inner: bool,
} }
fn parse_args(argv: impl Iterator<Item = String>) -> Result<Args, String> { fn parse_args(argv: impl Iterator<Item = String>) -> Result<Args, String> {
@@ -363,12 +379,19 @@ fn parse_args(argv: impl Iterator<Item = String>) -> Result<Args, String> {
mode: None, mode: None,
menu: None, menu: None,
query: None, query: None,
feed: None,
inner: false,
}; };
let mut argv = argv.peekable(); let mut argv = argv.peekable();
while let Some(arg) = argv.next() { while let Some(arg) = argv.next() {
match arg.as_str() { match arg.as_str() {
"--dmenu" => args.mode = Some(PromptMode::Dmenu), "--dmenu" => args.mode = Some(PromptMode::Dmenu),
"--fzf" => args.mode = Some(PromptMode::Fzf),
"--gtk" => args.mode = Some(PromptMode::Gtk), "--gtk" => args.mode = Some(PromptMode::Gtk),
"--inner" => args.inner = true,
// fzf passes an empty argument for an empty input, so a missing
// value here is that case rather than a user error.
"--feed" => args.feed = Some(argv.next().filter(|text| !text.trim().is_empty())),
// A launcher command is one shell-ish string so it can live in a // A launcher command is one shell-ish string so it can live in a
// keybinding; splitting on whitespace is enough for flags, and a // keybinding; splitting on whitespace is enough for flags, and a
// launcher argument needing spaces belongs in config.json. // launcher argument needing spaces belongs in config.json.
@@ -416,16 +439,31 @@ fn main() -> glib::ExitCode {
} }
}; };
// A feed is fzf's own reload child, not a front end: it must print hits and
// nothing else, so it is answered before the stored mode is even read.
if let Some(query) = args.feed {
fzf::feed(query.as_deref());
return glib::ExitCode::SUCCESS;
}
// The stored mode makes an existing keybinding switch front ends without // The stored mode makes an existing keybinding switch front ends without
// being re-bound; an explicit flag still wins. // being re-bound; an explicit flag still wins.
let stored = AppDirs::new() let stored = AppDirs::new()
.map(|dirs| dirs.load_config().resolved_prompt().resolved_mode()) .map(|dirs| dirs.load_config().resolved_prompt().resolved_mode())
.unwrap_or_default(); .unwrap_or_default();
if args.mode.unwrap_or(stored) == PromptMode::Dmenu { let launcher = match args.mode.unwrap_or(stored) {
return match dmenu::run(dmenu::Options { PromptMode::Dmenu => Some(dmenu::run(dmenu::Options {
menu: args.menu, menu: args.menu,
query: args.query, query: args.query,
}) { })),
PromptMode::Fzf => Some(fzf::run(fzf::Options {
query: args.query,
inner: args.inner,
})),
PromptMode::Gtk => None,
};
if let Some(result) = launcher {
return match result {
Ok(()) => glib::ExitCode::SUCCESS, Ok(()) => glib::ExitCode::SUCCESS,
Err(message) => { Err(message) => {
eprintln!("pdfs-prompt: {message}"); eprintln!("pdfs-prompt: {message}");
@@ -1326,7 +1364,7 @@ fn icon_for(name: &str, is_dir: bool) -> gio::Icon {
if is_dir { if is_dir {
return gio::ThemedIcon::new("folder").upcast(); return gio::ThemedIcon::new("folder").upcast();
} }
let (content_type, _uncertain) = gio::functions::content_type_guess(Some(name), &[]); let (content_type, _uncertain) = gio::functions::content_type_guess(Some(name), None);
gio::functions::content_type_get_icon(&content_type) gio::functions::content_type_get_icon(&content_type)
} }
+194
View File
@@ -0,0 +1,194 @@
//! The daemon side of both launcher front ends: ask for hits, open a chosen one.
//!
//! `--dmenu` and `--fzf` differ only in how they present a list and collect a
//! choice. Everything between the control socket and the chosen file lives here
//! so the two cannot drift on what a result set is or what "open" means — the
//! same reason both already share [`Hit`] and [`rank_hits`] with the GTK prompt.
//!
//! Everything here is blocking and GTK-free: the callers are one-shot processes
//! with no main loop to keep free.
use std::path::{Path, PathBuf};
use pdfs_core::config::AppDirs;
use pdfs_core::control::{
Request, Response, SearchFilters, SearchHit, SearchKind, SearchSource, send,
};
use pdfs_core::opener::{self, OpenWith};
use crate::activation::{DriveActivation, drive_activation, mounted_or_relative, mounted_target};
use crate::{Hit, file_name, is_document, is_image, is_media, rank_hits};
/// Where the mount lives, for turning a Drive path into something an
/// application can open. Falls back to the configured default so a momentarily
/// unreachable daemon still produces a plausible path rather than an error.
pub(crate) fn mountpoint(socket: &Path, dirs: &AppDirs) -> PathBuf {
match request(socket, Request::Status) {
Ok(Response::Status { mountpoint, .. }) => PathBuf::from(mountpoint),
_ => dirs.default_mountpoint(),
}
}
/// Pinned files — the same "you asked to keep these" set the GTK prompt opens
/// with, and the only listing the daemon offers without a query.
pub(crate) fn pins(socket: &Path) -> Result<Vec<Hit>, String> {
let Response::Pins { pins } = request(socket, Request::ListPins)? else {
return Ok(Vec::new());
};
Ok(pins
.into_iter()
.map(|pin| {
Hit::Drive(SearchHit {
name: file_name(&pin.path),
path: pin.path,
is_dir: pin.is_dir.unwrap_or(pin.recursive),
size: 0,
modified: 0,
pinned: true,
uid: pin.uid,
mounted_path: None,
score: 0,
})
})
.collect())
}
pub(crate) fn search(socket: &Path, query: &str, limit: usize) -> Result<Vec<Hit>, String> {
let reply = request(
socket,
Request::SearchV2 {
query: query.to_string(),
limit,
filters: SearchFilters {
sources: vec![SearchSource::Drive, SearchSource::Local],
kind: SearchKind::All,
},
},
)?;
let Response::SearchResultsV2 {
drive_hits,
local_hits,
..
} = reply
else {
return Ok(Vec::new());
};
let mut hits: Vec<Hit> = drive_hits.into_iter().map(Hit::Drive).collect();
hits.extend(local_hits.into_iter().map(Hit::Local));
rank_hits(&mut hits);
hits.truncate(limit);
Ok(hits)
}
/// Icon theme name for a hit.
///
/// Coarse on purpose: `content_type_guess` would give an exact MIME name, but
/// only with file *contents* to sniff — a Drive hit is metadata, and passing it
/// an empty buffer answers `application-x-zerosize` for everything. The generic
/// names below exist in every icon theme.
pub(crate) fn icon_name(hit: &Hit) -> String {
if hit.is_dir() {
return "folder".to_string();
}
let name = hit.name();
if is_image(name) {
"image-x-generic"
} else if is_media(name) {
"video-x-generic"
} else if is_document(name) {
"text-x-generic"
} else {
"application-x-generic"
}
.to_string()
}
/// Open a chosen hit, mirroring the GTK prompt: local files are already on
/// disk, a Drive folder or streamable file goes through the mount, and anything
/// else is materialised by the daemon first.
pub(crate) fn open(
socket: &Path,
mountpoint: &Path,
policy: &OpenWith,
hit: &Hit,
from_pins: bool,
) -> Result<(), String> {
match hit {
Hit::Local(local) => {
opener::open(policy, Path::new(&local.path), local.is_dir);
Ok(())
}
Hit::Drive(drive) => {
// A folder has no materialised form, and a pin row's `is_dir` is pin
// policy rather than node kind, so both go to the mount unchecked.
let unconditional = from_pins
|| matches!(
drive_activation(&drive.name, drive.is_dir),
DriveActivation::Folder | DriveActivation::MountedMedia
);
if unconditional {
opener::open(
policy,
&mounted_or_relative(mountpoint, drive),
drive.is_dir,
);
return Ok(());
}
// Anything else the mount exposes opens there too, so an editor
// saves back into Drive instead of into a cache blob.
if let Some(path) = mounted_target(mountpoint, drive) {
opener::open(policy, &path, false);
return Ok(());
}
match request(
socket,
Request::OpenFile {
path: drive.path.clone(),
uid: Some(drive.uid.clone()),
},
)? {
Response::FilePath { path } => {
// The reply is a content-hash blob; the rules key off the
// Drive name, which is what the user actually opened.
opener::open_named(policy, Path::new(&path), &drive.name, false);
Ok(())
}
Response::Error { message, .. } => Err(format!("could not open: {message}")),
_ => Err("unexpected reply from the daemon".to_string()),
}
}
}
}
/// One control-socket round-trip, with the daemon-is-down case phrased for a
/// terminal rather than a status page.
pub(crate) fn request(socket: &Path, request: Request) -> Result<Response, String> {
send(socket, &request).map_err(|e| format!("cannot reach the Proton Drive daemon: {e}"))
}
#[cfg(test)]
mod tests {
use super::*;
fn drive(name: &str, is_dir: bool) -> Hit {
Hit::Drive(SearchHit {
name: name.into(),
path: name.into(),
is_dir,
size: 0,
modified: 0,
pinned: false,
uid: "uid".into(),
mounted_path: None,
score: 0,
})
}
#[test]
fn icons_are_theme_names_the_launcher_can_look_up() {
assert_eq!(icon_name(&drive("Photos", true)), "folder");
assert_eq!(icon_name(&drive("a.png", false)), "image-x-generic");
assert_eq!(icon_name(&drive("a.md", false)), "text-x-generic");
assert_eq!(icon_name(&drive("a.bin", false)), "application-x-generic");
}
}
+231
View File
@@ -12,6 +12,227 @@ Conventions:
--- ---
## B93 — A photo whose node would not resolve lost its name, and with it its tab and its group
**Status:** Fixed (unverified) — found and fixed while making the timeline refresh incremental
(B92); no library with an undecryptable photo in it has been driven against the fix.
**Found:** 2026-09-19, by reading `photos_replace` before making a refresh skip photos it has
already resolved. Skipping one means writing a `TimelineRow` that carries only the uid and the
capture time, which is exactly the row a failed resolve already produced.
**Where:** `crates/pdfs-core/src/db/photos.rs` (`Learned`, `photos_replace`).
**Cause.** `photos_replace` keeps what a refresh could not learn — ratio, thumbnail verdict, media
type, favourite flag, content hash, photo relation all survive a `None`. The `name` did not:
`Learned` never held it, the SELECT never read it, and the INSERT bound `row.name` straight
through. So a photo the SDK could not decrypt (it logs and skips the node) had its stored name
overwritten with `NULL` on every refresh. That is not a cosmetic loss — both the tab split and the
grouping are derived from the name. `PhotoKind::classify` is extension-first, so a nameless RAW
file classifies as a still photo and leaves the Raw tab; `name_stem` returns `None`, so the RAW and
the JPEG of one shot stop being one shot; and `redate` skips nameless photos outright.
**What changed.** `Learned` carries the name like everything else, and the kind and the group stem
are computed from the carried name rather than the incoming row. A row that *was* resolved is now
authoritative instead: its fields are taken as they are, so a photo unlinked from its main on
another device finally loses its `main_uid` — a carry could never express "the server says none".
---
## B92 — Refreshing the gallery reported "Resource temporarily unavailable"
**Status:** Fixed (unverified) — the daemon carrying the change has not been driven against a real
library yet.
**Found:** 2026-09-19, while checking why a 12,445-photo timeline still showed the pre-grouping
layout after pressing Refresh. `pdfs refresh photos` failed after about two minutes with:
```
Caused by:
0: io: Resource temporarily unavailable (os error 11)
1: Resource temporarily unavailable (os error 11)
```
**Where:** `crates/pdfs-fuse/src/control.rs` (`RefreshScope::Photos`),
`crates/pdfs-core/src/control.rs` (`READ_TIMEOUT`, the new `PhotosRefreshStatus`),
`crates/pdfs-gui/src/app/main.rs` (`refresh_then`), `crates/pdfs-cli/src/main.rs` (`cmd_refresh`).
**Cause.** `Refresh { Photos }` awaited `refresh_timeline()` before replying, on the reasoning that
somebody who presses Refresh wants the truth rather than a stale page. But the refresh resolves
every photo's node in batches of `TIMELINE_ENRICH_CHUNK` (200), so a 12,445-photo library is 63
server round-trips — minutes of work. The client's read bound is `READ_TIMEOUT` = 120 s, so the
socket timed out with `EAGAIN` while the refresh was running perfectly. In the GUI the timeout was
invisible: `refresh_then` ignores the reply and loads the page anyway, which re-read the *old*
timeline and made Refresh look like a dead button.
**What changed.** The daemon starts the refresh (`spawn_timeline_refresh`, which already dedupes)
and replies `refreshing` at once. `Request::PhotosRefreshStatus` reports whether one is running;
the gallery polls it every 2 s and reloads the page when it clears, and `pdfs refresh photos`
polls in the same way so the command still means "the timeline is up to date when I return".
`refresh_timeline` now logs `photos`, `chunks` and `elapsed_ms`.
**What changed, second pass (2026-09-19).** The refresh is incremental. `photos.resolved_at`
(schema v31) records that a photo's node was read, and `refresh_timeline` resolves only the photos
that have no such mark — the ones it has never read, and the ones a remote event marked stale.
A library that has already been read costs one `enumerate_timeline` and no `enumerate_nodes` call
at all.
The favourite question resolved by moving it off the refresh and onto the event feed:
`classify_photo_events` turns a `NodeUpdated` into `photos_mark_unresolved(uid)` instead of a
blanket "re-read everything", so a favourite toggled on a phone costs one node read rather than
63. A local toggle is claimed as our own echo (`note_self_change`), because the feed replays it and
a `NodeUpdated` carries nothing that says what changed. `ContinuityLost` / `ScopeAccessLost` and
`pdfs refresh photos --full` drop every mark, which is the one way back to a whole-library read.
`TIMELINE_ENRICH_CHUNK` is 150, matching the SDK's own `MAX_BATCH_COUNT` — a chunk of 200 was split
there into a full request and a 50-uid tail.
**Still open.** `invalidate_albums()` drives the same pattern per album
(`crates/pdfs-fuse/src/albums.rs:96` and `:191`). It is bounded by album size rather than library
size, so it is nowhere near as expensive, but the technique that fixed this would apply there too.
---
## B91 — The daemon hung overnight, and nothing could say what it was waiting on
**Status:** Open — the hang itself is undiagnosed. What is fixed (unverified) is the
*undiagnosability*: the daemon now reports what it is doing, warns when it stops making progress,
and is restarted by systemd when it stops answering.
**Found:** 2026-09-18, from a user report — the mount stopped responding and only came back after
`systemctl --user restart`.
**Where:** `crates/pdfs-fuse/src/{workers.rs,diagnostics.rs,supervisor.rs,systemd.rs,mount.rs}`,
`packaging/proton-drive.service`.
**Symptom.** The mount stops answering. The process is alive and is not spinning; there is no
panic, no error, and no log line after the moment it stopped. The accounting for the hung run:
```
Consumed 43min 39.045s CPU time over 20h 8min 15.733s wall clock time,
5.2G memory peak, 65.7M memory swap peak
```
It went silent at 17:10 and stayed that way until the manual stop at 13:18:54 the next day. A
restart picked everything back up, which rules out corrupt local state.
**What made it undiagnosable.** Nothing in the daemon could be asked what it was doing:
- the worker pool published no state at all — not which threads were busy, not with what, not for
how long, not how deep the queues were;
- control requests were not tracked, so a handler that never returned looked exactly like a
handler that was never asked for;
- there was no watchdog, so a daemon that stopped answering stayed "active (running)" for twenty
hours;
- resident size was never sampled, so the 5.2 G peak was only visible in systemd's post-mortem.
**Related, and a real bug on its own.** At 2026-09-18 09:58:35 a meta worker panicked during
shutdown:
```
thread 'pdfs-fuse-meta-0' (3044) panicked at ...:
A Tokio 1.x context was found, but it is being shutdown.
```
The pool was flagged closed at teardown but never joined, so the process could drop the tokio
runtime while a worker was still inside a job. `catch_unwind` caught it, which is why it was only
ever a log line — but it is teardown ordering, not an error.
**What changed.**
- `Workers` publishes per-thread state (busy, job label, age) plus queue depths and per-lane
completion counts, all outside the queue lock, and every FUSE handler labels the job it hands
over.
- `Request::Diagnostics` / `pdfs diagnostics` reports that, the in-flight control requests with
their ages, the handler count against its limit, resident size and the pending-op count. It is
built to answer while the daemon is wedged: atomics and `try_lock` only, with the one database
read on a throwaway thread behind a 500 ms deadline.
- A supervisor thread WARNs when a worker has held one job for 120 s, when a control request has
run that long, or when a lane has a queue but completed nothing since the last tick; it samples
RSS every 5 minutes.
- The supervisor answers the systemd watchdog, but only after a real round trip over the control
socket — so the ping means "a client can still talk to this daemon" rather than "a thread is
still scheduled". `WatchdogSec=120` and `Restart=always` turn the next hang into a restart
instead of a morning of downtime, and `MemoryHigh=2G`/`MemoryMax=6G` bound the growth.
- Teardown joins the worker pool, with a 10 s deadline, before the caller drops the runtime.
**Still open.** None of this says *why* it hung. The candidates the evidence does not yet separate:
the 5.2 G working set (a leak, or a legitimately huge listing retained), a lock held across an
await somewhere in the control plane, and the ~150 `rt.block_on` sites on the control path that
have no op-level deadline of their own — each individual HTTP call is bounded by the SDK's 30 s
API / 300 s storage timeouts, but a sequence of them is not. The next occurrence should be a
one-line answer from `pdfs diagnostics` or from the stall WARN.
---
## B90 — The local-file indexer walks into foreign FUSE mounts, and a dead one makes the daemon unkillable
**Status:** Fixed (unverified) — the walker no longer stats a nested mountpoint. The stuck-daemon
half cannot be fixed in userspace at all; see "Why there is no second line of defence".
**Found:** 2026-09-16, from a user report of `systemctl --user restart` failing in a loop.
**Where:** `crates/pdfs-core/src/localindex.rs` (`default_excludes`, `scan`),
`crates/pdfs-fuse/src/background.rs` (`scan_local_once`).
**Symptom.** Stopping the service times out, SIGKILL does not help, and every subsequent start
fails with `another Proton Drive daemon is already using .../cache.db`:
```
systemd[2196]: proton-drive.service: State 'final-sigterm' timed out. Killing.
systemd[2196]: proton-drive.service: Killing process 2221 (pdfs) with signal SIGKILL.
systemd[2196]: proton-drive.service: Processes still around after final SIGKILL. Entering failed mode.
systemd[2196]: proton-drive.service: Unit process 2221 (pdfs) remains running after unit stopped.
pdfs[73921]: ERROR mount failed; retrying in 5s error="open cache db: another Proton Drive daemon
is already using /home/narl/.local/state/proton-drive-linux/cache.db"
```
**Diagnosis.** The surviving process is a zombie that still has a live thread:
```
$ cat /proc/2221/status | grep -E 'State|Threads'
State: Z (zombie)
Threads: 2
$ cat /proc/2221/task/73231/comm; cat /proc/2221/task/73231/stack
pdfs-localindex
request_wait_answer
```
`pdfs-localindex` is parked in a FUSE request nobody will answer, in uninterruptible `D` state, so
it ignores SIGKILL. The thread group leader exits, the thread does not, the process cannot be
reaped — and it keeps the `cache.db` lock, which is what turns one stuck scan into a permanent
restart loop. The same dead connection also blocked an unrelated `duf` in `fuse_statfs`
(`/sys/fs/fuse/connections/69/waiting` was 2).
**Cause.** `scan_local_once` walks `$HOME`, and `default_excludes` only excluded the Drive
mountpoint plus our own state and cache dirs. The reporter's home also held
`~/remote/narl.io` (sshfs) and `~/remote/gdrive` (rclone) — third-party FUSE mounts the walk
descends into.
`scan` does set `same_file_system(true)`, which looks like it should already cover this. It does
not: in `ignore` 0.4.33 the check is `is_same_file_system` → `device_num(path)` →
`path.metadata()` (`walk.rs:2164`), i.e. **a stat of the mountpoint**, performed in `run_one`
*after* the directory has been queued. To discover that a mount is foreign, the walker first has
to touch it. Against a network FUSE mount whose server is gone, that stat never returns.
**Fix.** `localindex::nested_mount_points` reads `/proc/self/mounts` and returns every mountpoint
strictly below the scan root; `scan_local_once` appends those to the excludes. `filter_entry` runs
before the device stat (`walk.rs:1926`, before `send`), so a path-only rejection keeps the walker
from ever touching the mount. Mountpoint fields are octal-unescaped, because a `\040` left in
place would silently fail to match and put the mount back in the walk. Re-read per scan, not
cached at startup: a mount that appeared since the daemon started is the likeliest to be a
half-alive network filesystem.
This changes nothing about what gets indexed. A nested mount is a different device, so
`same_file_system(true)` was always going to skip it — the exclude only removes the stat that had
to happen first.
**Why there is no second line of defence.** Once a thread is in uninterruptible FUSE wait, no
userspace change rescues it: not a join timeout, not `std::process::exit`, not SIGKILL. The kernel
will not reap the task until the syscall returns, which needs someone to answer the request or
abort the connection (`echo 1 > /sys/fs/fuse/connections/<id>/abort`, root-only). Avoiding entry is
the entire fix. That makes any *new* code path that stats a user-supplied or mount-crossing path
from a daemon thread a repeat of this bug — mirror-folder scanning is the obvious next candidate.
**Recovery for an already-wedged daemon.** As root, abort the connection with waiters
(`grep . /sys/fs/fuse/connections/*/waiting` finds it), which releases the thread, lets the zombie
be reaped and frees the `cache.db` lock. A reboot also clears it.
---
## B89 — The recorded SDK dependency is three minor versions out of date in the working memory ## B89 — The recorded SDK dependency is three minor versions out of date in the working memory
**Status:** Open — documentation only, but it misdirects every reader of the SDK code. **Status:** Open — documentation only, but it misdirects every reader of the SDK code.
@@ -2650,6 +2871,16 @@ collisions without losing the source. Simple rename-only and move-only calls
remain synchronous. The managed live suite subsequently passed the replacement remain synchronous. The managed live suite subsequently passed the replacement
and combined move/rename cases. and combined move/rename cases.
**Follow-up (2026-09-15, unverified):** With SDK 0.6.3 the drain sends the
move and the rename as one `move-multiple` request carrying the target name,
instead of renaming in the source folder and then moving. That removes the
second call the stale requirements could hit, and the SDK retries a remaining
`InvalidRequirements` once with the server's current name hash. A destination
name collision now moves under a conflict name in one step. Same-folder renames
are unchanged. The synchronous FUSE `rename` path (`serve_rename`) still renames
then moves with its bounded retry and has not been converted. Needs a managed
live run of the combined move/rename cases.
--- ---
## B47 — FUSE Accepts Path Components Longer Than NAME_MAX ## B47 — FUSE Accepts Path Components Longer Than NAME_MAX
+300
View File
@@ -9,6 +9,302 @@ release ships. Migrations are forward-only — a database written by a newer cli
refuse-to-open, not a downgrade, so rolling back a release means restoring the cache from refuse-to-open, not a downgrade, so rolling back a release means restoring the cache from
scratch (user data in `staging/` and `recovery/` is never touched by this). scratch (user data in `staging/` and `recovery/` is never touched by this).
## [1.11.0] — 2026-09-19
Schema: **30** (`photos.content_hash` / `main_uid` / `group_key`); SDK bumped to `proton-sdk` /
`proton-drive-rs` **0.6.5**.
### Changed
- **The gallery lays photos out in justified rows.** Every tile is now its own photo's shape:
photos are taken in capture order until they no longer fit the target height, and the row is
then scaled to land exactly on the right margin. Nothing is cropped and nothing is
letterboxed, which is what the square centre-cropped tiles used to do to portraits and
panoramas. The last row of a day stays at the target height rather than being stretched, and
Ctrl+scroll / Ctrl+± now changes that target height. Gaps and tile corners are smaller, so the
page reads as photos rather than as cards.
- **Thumbnails stay on screen while you scroll.** A decoded thumbnail now lands in the texture
cache even when the tile that asked for it has already been recycled, the cache is a real LRU
holding 1500 textures instead of 600, and the rows just past the edge of the window are
fetched ahead of the scroll direction. Scrolling fast through a large timeline no longer
leaves a screen of blank tiles behind.
- **Refresh on the Photos page really refreshes.** Instead of only clearing the freshness stamp
and letting the next read serve the stale page, it starts the timeline re-read and follows it:
the page paints from what the daemon holds, then paints again when the new timeline lands. The
timeline is also considered stale after 60 s rather than 5 minutes. `pdfs refresh photos`
blocks until the re-read is done, as it reads.
- **Restore puts back the whole tree.** The trash is a flat list, but you delete a shape: a
folder, and often things you had already deleted inside it. Restoring a folder now also
restores the items that were trashed inside it, and restoring one of those items first
restores the trashed folders above it — otherwise the file came back into a folder that is
still in the trash, which is to say nowhere you can see it. Parents are restored before their
children, and each node is still restored individually, so one node the server refuses does
not cancel the rest.
### Added
- **A RAW and its JPEG are one photo.** A camera that writes both put two tiles of the same shot
in the timeline; now it puts one, badged `RAW`, and the lightbox has a control that switches
between the files. The Raw tab still lists every raw file individually, because that is what
the tab is for. Grouping follows the server's own photo relation where it has one — which
also collapses live photos and bursts — and otherwise pairs files with the same name taken on
the same day where one is raw and the other is not. Deleting the tile deletes the whole shot,
and the confirmation says how many files that is.
- **Delete from the gallery.** Select photos with the Select button, with Ctrl+click or with
Shift+click, then Move to Trash — or press Delete in the lightbox. The tiles leave the grid
immediately with an Undo toast, and the photos land in Proton's trash, where they stay
recoverable. A photo the server refuses to trash comes back into the grid.
- **A photo deleted on your phone disappears here.** The daemon now follows the photos volume's
own event stream, so a photo trashed on another device leaves the gallery within about ten
seconds instead of waiting for the timeline to go stale.
- **`pdfs diagnostics`.** Reports what the daemon is doing right now: every worker thread with the job
it holds and for how long, queue depths per lane, the control requests still in flight, resident
size and the pending-op count. Answers even while the daemon is wedged, which is the only time
anyone asks (B91).
- **A stall watchdog.** The daemon warns in the journal when a worker has held one job for two
minutes, when a control request has run that long, or when a queue is not draining — and it now
answers the systemd watchdog, but only after a real round trip over its own control socket. A
daemon that stops answering is restarted instead of sitting there looking healthy (B91).
### Fixed
- **A Pixel raw capture groups with its JPEG.** Pixel names the two files of one shot
`PXL_….RAW-01.COVER.jpg` and `PXL_….RAW-02.ORIGINAL.dng`, so their stems differ and the pairing
rule never saw them as one shot. The part-of-a-shot marker is now cut off before the names are
compared, and the grid shows the cover JPEG once. The grouping is computed by a timeline
refresh, so an existing library pairs up after the next one.
- **The Raw tab no longer decodes a raw it does not have to.** A raw file that was written next
to a JPEG of the same shot is now given that JPEG's thumbnail, which the server usually already
has. No raw download and no raw decode for those tiles.
- **Photos can be moved to Trash from the gallery again.** Photos live on the photo volume, which
the mount never holds, so the write check found no node to consult and refused every photo with
"trash access: not allowed". Membership of our own timeline now answers for them; an album photo
on someone else's volume is still refused.
- **A photos refresh no longer fails with "Resource temporarily unavailable" (B92).** Re-reading a
twelve-thousand-photo library resolves the nodes in batches of 200 and takes minutes, but the
daemon held the control reply until it had finished — well past the client's 120 s read bound.
The caller therefore saw `io: Resource temporarily unavailable (os error 11)` for a refresh that
was running perfectly, and the gallery reloaded from the old timeline. The daemon now answers as
soon as the refresh is running, and the new `PhotosRefreshStatus` request reports when it is
done. Each refresh also logs its photo count, batch count and duration.
- **A worker no longer panics during shutdown (B91).** The FUSE worker pool is joined before the
tokio runtime is dropped, so a worker still inside a job does not find a runtime that is going
away underneath it.
- **A permanently rejected block upload is no longer retried (SDK 0.6.5).** The SDK now knows
which API errors are worth replaying: 4xx is permanent, except 404, 408 and 429.
- **Long downloads no longer fail on expired block URLs (SDK 0.6.5).** A download whose block
URLs lapse mid-transfer re-mints them once instead of failing, on the whole-file path and on
public links, as reads already did.
- **A long rate-limit answer can no longer look like a hang (SDK 0.6.5).** A server-supplied
`Retry-After` is now capped at 120 s.
## [1.10.2] — 2026-09-16
Fixes a daemon that could not be stopped or restarted. No schema change.
Schema: **28**; no SDK change.
### Fixed
- **A dead network mount in your home directory no longer wedges the daemon (B90).** The
"This computer" file indexer walked `$HOME` and descended into other FUSE mounts — an sshfs
whose server had gone away, an unresponsive rclone. Stat'ing one of those blocks
uninterruptibly, so the indexer thread stopped responding to SIGKILL, the daemon could never
exit, `systemctl --user stop` timed out, and the half-dead process kept the cache database lock
that the next start needed: every restart then failed with "another Proton Drive daemon is
already using cache.db". The indexer now skips mountpoints below your home directory by path,
without touching them. Nothing changes about which files are indexed — those mounts were always
excluded from the results, just not before they had been stat'ed.
## [1.10.1] — 2026-09-15
Dependency upgrade release, clearing both open Dependabot advisories. No schema change.
Schema: **28**; SDK bumped to `proton-sdk` / `proton-drive-rs` **0.6.4**.
### Security
- **`lru` IterMut Stacked Borrows advisory.** The SDK now depends on `lru` 0.18.
- **`atty` unaligned read advisory.** The tray moved to `ksni` 0.3, which no longer pulls
`dbus-codegen` → `clap` 2 → `atty`.
### Changed
- **Keyring backend is `keyring-core` + `dbus-secret-service-keyring-store`** (keyring 4's split).
Same libdbus Secret Service as before; stored sessions written by earlier releases are still
found, so no re-login is needed.
- **The tray waits for a late StatusNotifierWatcher** instead of failing to start when it beats
the desktop's tray host onto the session bus.
- Upgraded `gtk4` 0.11, `libadwaita` 0.9, `webkit6` 0.6, `rusqlite` 0.40 (bundled SQLite),
`fuser` 0.18, `directories` 6, `sha1`/`sha2` 0.11, `base64` 0.23.
## [1.10.0] — 2026-09-15
One-request move-and-rename in the drain, and who shared an item with you. No schema change.
Schema: **28**; SDK bumped to `proton-sdk` / `proton-drive-rs` **0.6.3**.
### Added
- **Shared with me says who shared an item, and when.** The GUI row subtitle and `pdfs
shared-with-me` show the inviter and invite date for a share root. The invitation's signature is
checked against the claimed inviter's keys; when it does not verify, the GUI shows a warning
and the CLI prints one, since the sender may be forged. `DirEntry` gains `shared_by`,
`shared_at` and `shared_by_unverified` (defaulted, so older front-ends and daemons interoperate).
### Changed
- **A queued move-and-rename lands as one request.** `drain_rename` used to rename in the source
folder and then move, with a rename-out-of-the-way step when the destination held the name. It
now calls `move-multiple` with a target name, so there is no half-applied state between the two
halves and no second call for the encrypted-name requirements to go stale against (B46). A
rename that stays in the same folder still uses the rename endpoint.
### Fixed
- **Moving a node with no signature email no longer fails.** Such nodes (created anonymously, e.g.
through a public link) need the passphrase signed by the mover; SDK 0.6.3 sends that signature.
- **A node trashed through the newer API field is recognised as trashed.** The SDK now reads
`TrashTime` as well as the legacy `Trashed` field, so the drain no longer tries to rename a node
that is already in the trash.
## [1.9.3] — 2026-09-01
The daemon's wait-for-login loop, and the latency that fixing it introduced. No schema change.
Schema: **28**; SDK unchanged at `proton-sdk` / `proton-drive-rs` **0.6.2**.
### Fixed
- **An enabled service with no saved session no longer polls the keyring 28,800 times a day.**
`wait_for_session` slept a flat 3 s between checks, and a check is not just a sleep — `auth::load`
builds a fresh `keyring::Entry`, so every iteration opened a new D-Bus connection to the Secret
Service and negotiated a session on it. The wait now backs off, and "not logged in" is said once
at `info` instead of on every pass, which also stops the journal filling with tens of thousands
of identical lines (`#21`).
- **Signing in wakes the parked daemon instead of leaving the mount up to an interval late.** The
backoff had no wake-up path: a unit started at boot reaches the ceiling within ~90 s, and
`enable --now` is a no-op on an already-active unit, so the GUI login and the tray's Connect left
it sleeping. `service::enable_start` now runs `enable` + `restart`, which interrupts the sleep and
still starts a unit that is down, and `pdfs login` restarts the unit when it is already active —
deliberately without touching `enable`, so a headless setup driving `pdfs daemon` by hand is left
alone. The ceiling also drops from 60 s to 15 s, still an 80% cut in keyring/D-Bus wakeups
(28,800/day → 5,760/day), bounding the worst case for paths that never reach systemd.
### Changed
- The wait is extracted into `wait_for_session()`, taking the load and sleep calls as parameters,
so the schedule is testable without a keyring and without real time passing.
## [1.9.2] — 2026-08-28
Repair for photos a Takeout import filed under the day they were imported, an `fzf` launcher front
end, and a gallery that survives 1,600 photos on one date. No schema change. Schema: **28**; SDK
`proton-sdk` / `proton-drive-rs` **0.6.2**.
### Added
- **`pdfs redate-photos`: repair photos filed under the day they were imported.** A Takeout whose
metadata sidecars did not match imports with no capture time at all, and the SDK then seals
"now" into every revision — a decade of photos lands on one date. The date is still in the file
name, so this finds photos whose name disagrees with their stored capture time by more than a
day and rewrites them: download, re-upload at the right time, carry favourites and album
memberships across, then **trash** (never delete) the original — Proton has no API for editing a
sealed capture time. `--dry-run` reports without touching anything, `--from`/`--to` aim the run
at the days an import landed on, and `--limit` caps it. Followed with `pdfs redate-status`,
stopped with `pdfs cancel-redate`.
- **`pdfs-prompt --fzf`: launcher search that updates as you type.** A dmenu-style launcher
(fuzzel, rofi, wofi, …) is handed one static list and filters it itself, which is why `--dmenu`
has to make searching two steps — type, Enter, *then* results. `fzf` can re-run a command on
every keystroke, so the new mode re-queries the daemon per keystroke (debounced to match the GTK
prompt) and keeps `--disabled` on so the daemon's ranking, not fzf's matcher, decides the order.
Abbreviation and typo tolerance therefore work here exactly as they do in the built-in window.
- Started without a tty — from a WM keybinding — `--fzf` spawns a terminal for itself (foot,
ghostty, kitty, alacritty, wezterm or xterm, whichever is installed), each with the app-id/class
`pdfs-prompt` so a single window rule floats it. Override with `"prompt": { "terminal": [...] }`,
where a `{cmd}` token places the command and its absence appends it. `"prompt": { "mode": "fzf" }`
makes it the default front end for an existing keybinding.
### Fixed
- **Google Photos Takeout: sidecars with a localized suffix are matched again.** Google localizes
the `.supplemental-metadata` it appends to a photo's JSON sidecar — a German export writes
`.ergänzende-Metadaten` — and the scanner only knew the English spelling. No sidecar matched, so
no photo had a capture time, so every photo in the export was stamped with the moment it was
uploaded. Matching now hangs on the photo name in front of the suffix rather than on the suffix
itself, which is locale-agnostic, and covers the misspelt variants Google has also shipped.
- **A photo with no usable sidecar now takes its date from its file name.** `PXL_20260818_171030868.jpg`,
`IMG_20230219_171030.jpg`, `Screenshot_20230101-102950.png`, `IMG-20230219-WA0001.jpg` and
`Screenshot from 2023-01-01 10-29-50.png` all carry the date the photo was taken. Read as UTC, so
it can be hours out — but never the years out that "the moment it was imported" is. A sidecar
still wins.
- **An opened file or folder no longer dies with the window that opened it.** Everything launched
through `open_with` was started in the caller's process group with the caller's stdio, which was
invisible from a resident GUI but fatal from `--fzf`: the prompt exits the moment you choose
something, its terminal closes, and the pty hangup killed the application it had just started —
so an `@dir` rule like `alacritty --working-directory {}` opened nothing at all. Opened children
now get their own process group and no inherited stdio. Affects every front end; only `--fzf`
could observe the bug.
### Changed
- **The gallery virtualises at the row, not the day.** The Photos ListView had one item per day,
and a bind built every one of that day's tiles — fine for a normal day, ruinous for the ~1,600
photos a Takeout import drops onto a single date, which was 1,600 widgets constructed on the main
thread before the section could be shown. Days are now flattened into heading rows and grid rows,
so a bind is a handful of widgets whatever the day holds. A resize or a zoom step rebuilds the row
model (the column count decides which photos share a row) and scrolls back to the photo that was
at the top.
- Updated to `proton-sdk` / `proton-drive-rs` 0.6.2 (photos save-to-timeline, `enumerate_events`).
- The daemon side of both launcher front ends — search, pins, mountpoint lookup and the
open-a-result policy — moved out of `pdfs-gui/src/dmenu.rs` into a shared `pdfs-gui/src/query.rs`,
so `--dmenu` and `--fzf` cannot drift on what a result set is or what "open" means. No behaviour
change to `--dmenu`.
## [1.9.1] — 2026-08-21
Thumbnails everywhere the GUI lists files, a status bar under the Files browser, and the
repository's contribution/branding files. No schema change. Schema: **28**; SDK unchanged at
`proton-sdk` / `proton-drive-rs` **0.6.1**.
`exiftool` (`libimage-exiftool-perl` / `perl-image-exiftool` / `perl-Image-ExifTool`) is a new
runtime dependency, declared in the `.deb`, `.rpm` and `PKGBUILD` packaging. It is only used to
pull the embedded preview out of a camera RAW; without it every other image format still gets a
thumbnail.
### Added
- **Thumbnails for ordinary Drive files**, not just the Photos timeline — painted in Files (both
the grid and the Name column of the list), Shared, Shared by me, and Trash. Standard raster
formats (`bmp`, `gif`, `jpeg`, `png`, `tiff`, `webp`) are decoded in the daemon; camera RAWs
(`arw`, `cr2`, `cr3`, `dng`, `nef`, `orf`, `raf`, `rw2`, and the rest of the concrete list) are
served from their embedded preview via `exiftool`. Previews are cached beside the existing photo
thumbnails, outside the cache-budget scan, and tagged with the node's modification time — so a
new revision of an image can never be shown with its predecessor's thumbnail.
- Thumbnails for visible rows are requested opportunistically as tiles scroll in, and the daemon
drops work for listings the user has already navigated away from. The listing generation is
issued by the daemon (`ReserveFileThumbGeneration`), not seeded from the front-end's clock.
- **Build a whole folder tree ahead of time** from the Files toolbar: one recursive local-thumbnail
job per daemon, with live scan/found/completed/failed progress and a Cancel that appears only
once the daemon has acknowledged the start. The job outlives navigation — it is deliberate work,
separate from the opportunistic per-tile requests.
- **A Dolphin-style status bar under the Files browser**: the current listing's counts, a Zoom
slider for the icon grid, and account storage usage. Zoom resizes the realised grid cells rather
than rebuilding the model, so dragging the slider does not tear down the selection; the default
size reproduces the previous fixed 72 px tiles. Capacity stays out of the bar until a real
reading lands, and the quota fetch (60 s TTL) now also runs while Files is on screen instead of
only in Settings.
- Repository housekeeping: `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `SECURITY.md`, issue and pull
request templates, a project logo, and workspace crate metadata (`repository`, `homepage`,
`readme`, `keywords`, `categories`).
### Fixed
- A remote thumbnail miss no longer suppresses local generation. The two negative results are now
held in separate bounded caches: "Proton has no thumbnail for this node" and "this exact revision
was decoded locally and could not be read" mean different things, and only the second is a
permanent verdict for that revision.
- A RAW whose preview could not be extracted because of the *environment* — `exiftool` missing, a
failed download — is retried instead of being remembered as undecodable. Only bytes conclusively
outside the supported formats are cached as a negative.
- A recursive build that finds a uid an opportunistic tile job already owns waits briefly for it
rather than either duplicating the work or blocking on it forever, and a cancellation racing a
waiter's registration is caught by a backstop instead of leaving the waiter parked.
- Starting a build for a root that is already building attaches to that job; starting one for a
*different* root is refused explicitly rather than silently reporting the running job's progress
as the caller's.
- Decrypted camera RAWs are staged for `exiftool` in an owner-only (`0700`) directory of their own,
removed as soon as the preview is out and swept at cache open after an unclean shutdown.
- The RAW extension list had `rw` (not a format) and was missing `raw`; both classification helpers
now read from one shared table instead of two hand-maintained `matches!` arms.
- The tray and desktop-entry icons were wrong/missing; `.desktop` files and the packaged SVG were
corrected.
## [1.9.0] — 2026-08-18 ## [1.9.0] — 2026-08-18
Google Photos import, and a pass over the desktop app's interaction gaps. No schema change. Google Photos import, and a pass over the desktop app's interaction gaps. No schema change.
@@ -501,6 +797,10 @@ First stable release: FUSE files-on-demand mount, sync daemon under `proton-driv
- The outstanding FUSE defects tracked in `docs/BUGS.md`, plus a truncate defect, validated - The outstanding FUSE defects tracked in `docs/BUGS.md`, plus a truncate defect, validated
by a new POSIX compliance suite for the filesystem. by a new POSIX compliance suite for the filesystem.
[1.9.3]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.9.3
[1.9.2]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.9.2
[1.9.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.9.1
[1.9.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.9.0
[1.8.2]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.8.2 [1.8.2]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.8.2
[1.8.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.8.1 [1.8.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.8.1
[1.8.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.8.0 [1.8.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.8.0
+23
View File
@@ -142,6 +142,27 @@ The gallery can mark and filter favourites, through the SDK's photo-tag API (`up
Favouriting a photo that is not on this account's own photos volume (shared with us, or album-only) needs it re-encrypted for our timeline root, which the SDK does not implement; the daemon surfaces that as an error rather than silently doing nothing. Favouriting a photo that is not on this account's own photos volume (shared with us, or album-only) needs it re-encrypted for our timeline root, which the SDK does not implement; the daemon surfaces that as an error rather than silently doing nothing.
### Justified Gallery, Gallery Delete (Implemented)
**Files**: [`photos.rs`](../crates/pdfs-gui/src/app/pages/photos.rs), [`photo_viewer.rs`](../crates/pdfs-gui/src/app/pages/photo_viewer.rs), [`photos.rs`](../crates/pdfs-fuse/src/photos.rs), [`background.rs`](../crates/pdfs-fuse/src/background.rs)
The gallery lays each day out in justified rows (`justify_rows` / `plan_rows` / `fit_row`): photos are taken in capture order until their summed aspect ratio no longer fits the target row height, and the row is then scaled so it ends exactly on the content width. A tile is therefore its own photo's shape, and `ContentFit::Contain` has nothing left to crop. The last row of a day is left at the target height rather than stretched, because a day holding two photos is a short day, not a layout fault.
- **Ratios that are not known yet.** `PhotoItem::ratio` is persisted, but a photo that has never been decoded has none. Those are laid out square and remembered in `assumed_ratios`; when a decode proves the real shape the day re-flows on the existing debounce timer, so a screenful of decodes costs one re-flow rather than one per photo. Ratios are clamped to 0.4–3.0 so a single panorama cannot decide what a row looks like.
- **Thumbnails survive recycling.** A reply for a tile that has already been re-bound still lands in the texture cache, which is an LRU of 1500 textures keyed by uid, and the rows just outside the realised range are prefetched in the scroll direction.
- **Delete is uid-addressed.** The photos volume is not in the FUSE mount, so the path-based `Request::Delete` cannot reach it. `Request::TrashNodes { uids }` trashes through the SDK and answers `Response::Trashed { trashed, failed }`; the daemon then drops those rows with `Db::photos_delete` (album membership included) so the gallery does not wait for a timeline refresh. The GUI removes the tiles optimistically with an Undo toast and puts back anything the server refused.
- **Remote deletions arrive on their own.** The Drive volume and the photos volume have separate event streams, so `run_photos_event_sync` polls the photos volume with its own cursor (`photos_event_cursor` in the state table). Trash and delete events remove rows; anything else only invalidates freshness, because there is no inode space to converge on this volume. Together with a 60 s `TIMELINE_TTL` and a `RefreshScope::Photos` that now awaits the refresh, a photo deleted on a phone leaves the grid in about ten seconds.
### RAW + JPEG Grouping (Implemented)
**Files**: [`photos.rs`](../crates/pdfs-core/src/db/photos.rs), [`migrations.rs`](../crates/pdfs-core/src/db/migrations.rs), [`photos.rs`](../crates/pdfs-fuse/src/photos.rs), [`photo_viewer.rs`](../crates/pdfs-gui/src/app/pages/photo_viewer.rs)
Schema **v30** adds `content_hash`, `main_uid` and an indexed `group_key` to `photos`. The first two come from the server's `PhotoProperties` and are filled by the enrichment pass in `refresh_timeline` that already resolves each photo's name, media type and favourite tag; `group_key` is computed in `photos_replace`, where the whole timeline is in hand.
- **Precedence.** The server relation first (`main_photo_uid` and `related_photo_uids`, read from whichever end resolves — the two can land in different enrichment chunks); then same capture day, same case-insensitive name stem, and one member raw while the other is not; otherwise the photo is its own group. The union-find pass is over the timeline in memory, so it costs one pass per refresh rather than a query per photo.
- **Representative.** The non-raw member when there is one — a JPEG decodes in milliseconds and is what the person expects to see. `group_key` holds that member's uid, so "is this the tile" is `uid = group_key`, an index scan.
- **What is grouped and what is not.** `photos_page`, `photos_months` and `photos_counts` count groups, except on the Raw tab, which lists files. Album pages are never grouped: an album is a list someone made. The relation is learned-and-kept like `media_type`, so a refresh that could not resolve a node does not break a group up.
- **Deleting.** `Core::trash_photos` expands each uid to its group before trashing, because the grid shows the shot as one tile — leaving the RAW behind would put the photo back on the Raw tab and nowhere else. The GUI's confirmation names the file count.
- **The other files.** `Request::PhotoGroup { uid }` answers with the group's members as ordinary `PhotoItem`s, which is what the lightbox's switch steps through.
### Persistent SDK Entity Cache (Implemented) ### Persistent SDK Entity Cache (Implemented)
**Files**: [`sdkcache.rs`](../crates/pdfs-core/src/sdkcache.rs), [`auth.rs`](../crates/pdfs-core/src/auth.rs), [`background.rs`](../crates/pdfs-fuse/src/background.rs) **Files**: [`sdkcache.rs`](../crates/pdfs-core/src/sdkcache.rs), [`auth.rs`](../crates/pdfs-core/src/auth.rs), [`background.rs`](../crates/pdfs-fuse/src/background.rs)
@@ -158,6 +179,8 @@ Needed a small additive SDK change (`ProtonDriveClient::with_entity_repository`,
SDK 0.5.0 made `trash_nodes` / `restore_nodes` / `delete_nodes` report one outcome per node instead of failing the whole call (mirroring upstream's streamed `NodeActionResult`). `pdfs_core::batch::into_unit` collapses the single-node calls back to a `Result`; `batch::split` handles a collected batch. SDK 0.5.0 made `trash_nodes` / `restore_nodes` / `delete_nodes` report one outcome per node instead of failing the whole call (mirroring upstream's streamed `NodeActionResult`). `pdfs_core::batch::into_unit` collapses the single-node calls back to a `Result`; `batch::split` handles a collected batch.
A restore is expanded before it is sent (`expand_restore`, `pdfs-fuse/src/lib.rs`): the persisted trash listing carries each row's `parent_uid` (schema 29), so a restore takes the connected piece of the trashed tree — every trashed descendant of what was asked for, and every trashed ancestor above it — and sends it shallowest wave first, because the server cannot put a node back under a parent that is still trashed. A uid the listing does not know about (it is materialised in chunks, and can be stale) is restored on its own rather than dropped.
The trash view's restore and permanent-delete use the SDK's **streaming** variants (`restore_nodes_streaming` / `delete_nodes_streaming`) and apply local state per node as each batch lands rather than after the last one. For a permanent delete — which is irreversible — that means a daemon interrupted mid-batch has forgotten exactly the nodes the server destroyed, no more and no fewer. Both report per-node failures and count only what actually succeeded; only a batch where nothing succeeded is an error. The trash view's restore and permanent-delete use the SDK's **streaming** variants (`restore_nodes_streaming` / `delete_nodes_streaming`) and apply local state per node as each batch lands rather than after the last one. For a permanent delete — which is irreversible — that means a daemon interrupted mid-batch has forgotten exactly the nodes the server destroyed, no more and no fewer. Both report per-node failures and count only what actually succeeded; only a batch where nothing succeeded is an error.
+4 -2
View File
@@ -9,7 +9,7 @@
# Build + install: cd packaging && makepkg -fi # Build + install: cd packaging && makepkg -fi
pkgname=proton-drive-linux pkgname=proton-drive-linux
pkgver=1.9.1 pkgver=1.11.0
pkgrel=1 pkgrel=1
pkgdesc="Unofficial Proton Drive client for Linux: files-on-demand FUSE mount, CLI, GTK4 app + tray" pkgdesc="Unofficial Proton Drive client for Linux: files-on-demand FUSE mount, CLI, GTK4 app + tray"
arch=('x86_64') arch=('x86_64')
@@ -20,7 +20,9 @@ makedepends=('cargo' 'git')
provides=('pdfs') provides=('pdfs')
# Files-on-demand mount + keyring need these at runtime. # Files-on-demand mount + keyring need these at runtime.
optdepends=('gnome-keyring: credential storage via Secret Service' optdepends=('gnome-keyring: credential storage via Secret Service'
'xdg-utils: "Open folder" action from the tray') 'xdg-utils: "Open folder" action from the tray'
'fzf: live search-as-you-type launcher (pdfs-prompt --fzf)'
'foot: terminal for pdfs-prompt --fzf started from a keybinding')
options=('!lto' '!debug') options=('!lto' '!debug')
# Built in place from the checkout; nothing to fetch or checksum. # Built in place from the checkout; nothing to fetch or checksum.
+4
View File
@@ -41,6 +41,10 @@ Recommends: gnome-keyring
Recommends: gnome-shell-extension-appindicator Recommends: gnome-shell-extension-appindicator
Recommends: kwallet Recommends: kwallet
# Only for `pdfs-prompt --fzf`; every other front end works without them.
Suggests: fzf
Suggests: foot
Provides: pdfs = %{version}-%{release} Provides: pdfs = %{version}-%{release}
%description %description
+17 -1
View File
@@ -6,12 +6,28 @@ After=graphical-session.target
PartOf=graphical-session.target PartOf=graphical-session.target
[Service] [Service]
# Deliberately not Type=notify: the daemon waits for a stored login before it
# mounts anything, and a machine that boots before the user logs in would then
# fail the unit on the start timeout. NotifyAccess=all gets the daemon a
# NOTIFY_SOCKET all the same, which is all the watchdog below needs.
Type=simple Type=simple
NotifyAccess=all
# The daemon waits for a stored login, mounts ~/ProtonDrive, and keeps it # The daemon waits for a stored login, mounts ~/ProtonDrive, and keeps it
# mounted. SIGTERM (from `systemctl --user stop`) triggers a clean lazy unmount. # mounted. SIGTERM (from `systemctl --user stop`) triggers a clean lazy unmount.
ExecStart=/usr/bin/pdfs daemon ExecStart=/usr/bin/pdfs daemon
Restart=on-failure # Always, not on-failure: a daemon that hangs is not a failure by exit code, and
# on 2026-09-18 one had to be restarted by hand after wedging overnight.
Restart=always
RestartSec=3 RestartSec=3
# The supervisor thread pings this, but only after a round trip over the control
# socket — so a daemon that stops answering its own clients is restarted instead
# of sitting there looking alive. It pings at half this interval.
WatchdogSec=120
# Backstop against unbounded growth: the hung run peaked at 5.2G resident.
# MemoryHigh throttles and reclaims first; MemoryMax is the hard stop, above any
# legitimate working set, so it only fires on a leak.
MemoryHigh=2G
MemoryMax=6G
# Give the lazy unmount time to finish before SIGKILL. # Give the lazy unmount time to finish before SIGKILL.
TimeoutStopSec=15 TimeoutStopSec=15