Compare commits

..
Author SHA1 Message Date
nvrl d0f9681593 fix(trash): list a large trash, empty it, and browse it by folder
A large trash never finished listing, so the Trash page kept the rows of
the last refresh that got anywhere, and Empty Trash listed the trash before
sending the delete and hung there (B293). The paging fix is in
proton-drive-rs 0.12.5.

- The refresh reads only nodes it has not stored yet, without unlocking
  file keys, keeps the time the server says each was trashed, and drops
  rows only once the whole listing is in.
- Empty Trash sends the delete at once under the trash call time limit
  and clears the stored listing. Restore and Delete Permanently drop their
  rows right away.
- Schema v47 indexes trash.parent_uid and drops the stored rows once, so
  they are read again with real deletion times.
- The Trash page groups items by the folder they were deleted from, opens
  deleted folders through the new ListTrashedFolder request, and has a
  search box and a switch back to one list. Restore and Delete Permanently
  on a folder row act on everything deleted from it.
- pdfs trash takes --by-folder and --folder UID.
2026-11-22 02:38:48 +01:00
nvrl f141d88d97 Merge pull request #123 from narrrl/feat/offline-basemap
feat(gui): offline basemap under the Places map
2026-11-22 01:06:31 +01:00
nvrl 795dd7edb0 feat(gui): offline basemap under the Places map
Places now draws the proton-map-rs basemap that Proton Gallery uses: seas by
depth, lakes, rivers, built-up areas, borders and town names, in light and
dark. Tiles render on two worker threads into the existing tile layer, so
caching, ancestor stand-ins and density restarts work as before. Opening
Places sends no request.

The OpenFreeMap street map stays opt-in and replaces the basemap while it
is turned on. Without the basemap file the map falls back to the coarse
Natural Earth outlines.

scripts/fetch-basemap.sh downloads basemap-v1.pmtiles and checks it against
a pinned SHA-256. The deb, rpm, tarball, local and AUR PKGBUILDs, COPR spec
and Nix package install it to /usr/share/proton-drive-linux. PDFS_BASEMAP
points the app at another copy.

The new credit and preference text is translated in every catalog.
2026-11-22 00:44:14 +01:00
nvrl 7be4c7db1e Merge pull request #122 from narrrl/feat/protect-queued-changes
feat: protect queued changes at shutdown and sign-out (B66)
2026-11-21 23:46:37 +01:00
nvrl 2b5d6f2436 feat: protect queued changes at shutdown and sign-out (B66)
While the queue holds changes the drain can send, the daemon holds a logind
shutdown inhibitor through systemd-inhibit, so desktops show why a power-off
is held up. Parked transient files and changes that need the user never hold
it. pdfs sync flush retries failed changes and waits for the queue, exiting 3
when what is left needs the user and 4 on timeout. Signing out, in the app or
with pdfs logout, says how many changes are not on Proton Drive yet.
2026-11-21 23:42:54 +01:00
nvrl ff8ee9d997 Merge pull request #121 from narrrl/perf/takeout-parallel
perf(fuse): upload two Takeout photos at once
2026-11-21 23:07:35 +01:00
nvrl aa26a8e1aa Merge branch 'main' into perf/takeout-parallel 2026-11-21 23:07:08 +01:00
nvrl 9027a1465f perf(fuse): upload two Takeout photos at once
The import uploaded one photo at a time, leaving the link idle while each
upload waited on Proton between blocks. Staged photos now go to two upload
workers while the import thread reads the next one out of its archive. A
round ends only when its uploads have landed, so a live pair's video still
finds its still's uid. Photos without a thumbnail still stream straight from
the archive on the import thread. Warnings no longer name the photo.
2026-11-21 23:04:28 +01:00
nvrl 66b634e4a1 Merge pull request #120 from narrrl/chore/sdk-0.12.4
chore(deps): proton-sdk 0.12.4
2026-11-21 22:51:29 +01:00
nvrl ae46e1e544 chore(deps): proton-sdk 0.12.4 2026-11-21 22:49:27 +01:00
nvrl 2c4c95812f Merge pull request #119 from narrrl/chore/sdk-0.12.3
chore(deps): proton-sdk 0.12.3 and proton-gallery-rs 0.1.5
2026-11-21 17:12:11 +01:00
nvrl 2bf1622793 chore(deps): proton-sdk 0.12.3 and proton-gallery-rs 0.1.5
SDK 0.12.3 fetches a thumbnail with fewer round trips and caches the
thumbnail target per revision. proton-gallery-rs 0.1.5 advances the
Photos revision only when an album changed.
2026-11-21 17:08:41 +01:00
nvrl 2ea189f1d9 Merge pull request #118 from narrrl/chore/gallery-rs-0.1.3
perf(gui): cluster Places markers with proton-gallery-rs 0.1.3
2026-11-20 22:12:51 +01:00
nvrl 8af671d8ed perf(gui): cluster Places markers with proton-gallery-rs 0.1.3
The map kept its own copy of the marker clustering, identical to the one
in proton-gallery-rs. 0.1.3 makes that one fast (a grid of cells instead
of a scan of every bubble so far, same results), so the map now uses it.
2026-11-20 22:09:04 +01:00
nvrl 9047bdd741 Merge pull request #117 from narrrl/perf/photos-main-thread
perf(gui): keep Photos decoding and file access off the main thread
2026-11-20 21:48:52 +01:00
nvrl ca20e5ed22 perf(gui): keep Takeout staging file access off the main thread
The Import page read and wrote its remembered archive list, checked each
dropped path and stat'ed every staged archive on the GTK thread. Do all
of it on gio's blocking pool. Nothing is saved until the remembered list
has been read back, so it can't be overwritten by the empty list shown
meanwhile, and a late save never overwrites a newer one.
2026-11-20 20:47:47 +01:00
nvrl 10bc9a6a37 perf(gui): cap the Places map's photo marker covers
Every photo cover the map was shown while zooming around stayed in
memory for as long as the map existed. Keep at most 1024 and drop the
ones drawn longest ago, never a town's cover or one drawn this frame,
and ask for a dropped cover again when its marker returns.
2026-11-20 20:43:44 +01:00
nvrl 890adbc189 perf(gui): debounce the Drive gallery re-flow on resize
The Drive gallery rebuilt its whole row model on every frame the width
changed, so dragging the window edge rebuilt it dozens of times a
second. Lay out the first width at once as before, and wait for a resize
to settle (80 ms, like the timeline) before rebuilding.
2026-11-20 20:41:59 +01:00
nvrl 960c6afe2d perf(gui): decode Drive gallery thumbnails off the main thread
Drive gallery tiles and the hidden-album preview decoded the base64
reply and the image itself on the GTK thread. Do both on gio's blocking
pool, then re-check the request generation and lock state before
painting, as before.
2026-11-20 20:37:22 +01:00
nvrl 1b545ba369 perf(gui): look Drive gallery thumbnails up by key
The Drive gallery's thumbnail cache was a queue searched linearly on
every tile bind. Keep it in the shared LRU cache instead, still capped at
256 thumbnails, so a lookup and its touch no longer scan the queue.
2026-11-20 20:36:12 +01:00
nvrl b854c1e819 perf(gui): decode timeline thumbnails off the main thread
The Photos timeline decoded up to four thumbnails per idle pass on the
GTK thread, which stuttered a fast scroll. Decode each batch on gio's
blocking pool instead and only paint the results on the main thread.
Results from before a timeline reset are dropped, and tiles that were
unbound or rebound meanwhile are no longer waiting, so they are not
painted.
2026-11-20 20:34:41 +01:00
nvrl 3fbe651f94 perf(gui): budget the Photos texture cache by bytes
The timeline kept up to 1500 decoded thumbnails regardless of their size,
and marking one as used scanned the whole eviction order on every tile
bind. Replace it with an LRU cache keyed on decoded pixel bytes (512 MiB,
about 900 ordinary 512 px thumbnails) whose touch is logarithmic.
2026-11-20 20:32:59 +01:00
nvrl 61b992cb48 Merge pull request #115 from narrrl/fix/drive-gallery-header
fix(gui): give Drive galleries room under their header
2026-11-20 20:04:38 +01:00
nvrl 08e0e80f34 fix(gui): give Drive galleries room under their header
An open Drive gallery kept Add This Folder, Unlock and its options button
in a row of their own above the grid. The row took height from the photos
and left the options button touching the first row of tiles. They now sit
in the page's header bar, where the timeline keeps its own menu.

Rows in a Drive gallery also lose their extra side inset and use the
timeline's tile gap vertically too, and the options menu no longer lists
Refresh twice while hidden albums are locked.
2026-11-20 19:27:57 +01:00
nvrl 0488ccd5f6 Merge pull request #114 from narrrl/refactor/sdk-login
refactor(core): sign in through proton_sdk::login
2026-11-20 17:53:42 +01:00
nvrl 62d16e03cf refactor(core): sign in through proton_sdk::login
The sign-in sequence moved from proton_auth_rs::session into proton-sdk
0.12.2. Same stored session, same messages; pdfs-core keeps only the
keyring feature of proton-auth-rs.
2026-11-20 17:37:54 +01:00
nvrl c92f78b5c8 Merge pull request #113 from narrrl/refactor/shared-sign-in
refactor(core): sign in through proton_auth_rs::session
2026-11-20 15:52:58 +01:00
nvrl b034e1d4e3 Merge pull request #112 from narrrl/fix/po-catalogs
fix(i18n): bring every catalog up to date with the sources
2026-11-20 15:52:49 +01:00
nvrl e15e1dcc76 refactor(core): sign in through proton_auth_rs::session
The sign-in steps are now the sequence Proton Gallery and Proton Stream
share. The stored session, the lock/unlock choice for two-password
accounts, the three mailbox-password tries and the errors front-ends
word themselves are unchanged. An account whose only second factor is
a security key now gets a clear message instead of a code prompt that
could only fail.
2026-11-20 15:43:31 +01:00
nvrl 43b76d4791 fix(i18n): bring every catalog up to date with the sources
The Drive galleries, hidden albums and move-recovery strings had been
added without regenerating po/, so 11 were untranslated and 17 fell back
to unrelated fuzzy matches ("Hidden albums" showed as "No albums").
Regenerated with po/update.sh and translated in all eight languages.
2026-11-20 15:37:46 +01:00
nvrl 734e9040fb Merge pull request #111 from narrrl/refactor/proton-media-rs
refactor(fuse): block fetch single flight from proton-media-rs
2026-11-20 13:28:42 +01:00
nvrl 01e3860741 refactor(fuse): block fetch single flight from proton-media-rs
BlockFlight is now proton_media_rs::SingleFlight joined with the typed
block result. Same semantics: one fetch per block, the key retires
before the result is published, and a waiter whose leader is cancelled
fetches the block itself.
2026-11-20 13:27:59 +01:00
nvrl 7a65e7011f Merge pull request #110 from narrrl/refactor/proton-base-rs
refactor(core): atomic file writes from proton-base-rs
2026-11-20 12:51:09 +01:00
nvrl 233d9e9e51 refactor(core): atomic file writes from proton-base-rs
Config, machine id and mount-failure files go through
proton_base_rs::atomic. Same modes as before (config 0600, the others
0644); the mount-failure file is now synced before its rename.
2026-11-20 12:16:32 +01:00
nvrl 73c4e1f6bf Merge pull request #107 from narrrl/refactor/proton-rs
refactor(photos): take the Gallery engine from crates.io
2026-11-20 12:10:34 +01:00
nvrl 26272cda6c refactor(core): take the keyring store from proton-auth-rs
The Secret Service connection that reconnects after the provider
restarts (B285) and the item replacement that oo7 before 0.7.0 needs
(B291) moved to proton-auth-rs, shared with Proton Gallery and Proton
Stream. pdfs-core keeps one KeyringStore per process; the session format,
service and user are unchanged. The two secretservice tests moved with
the code.
2026-11-20 12:01:50 +01:00
nvrl 370abd58dc refactor(photos): use the shared photo logic from proton-gallery-rs
RAW+JPEG grouping, Takeout reading, EXIF capture times, look-alike
hashes and the offline town lookup now come from proton-gallery-rs
0.1.1, shared with Proton Gallery. PhotoKind::classify delegates to the
shared MediaClass; the extension lists live there. No behaviour change.
2026-11-20 04:16:52 +01:00
nvrl 24594b24ae refactor(photos): take the Gallery engine from crates.io
Replace the git revision pin on Proton Gallery with proton-gallery-rs and
proton-gallery-app-rs 0.1.0 from the proton-rs workspace. This also moves
the engine from 858d244 to b171232. The Nix package no longer needs to
fetch from Git.
2026-11-20 02:44:22 +01:00
nvrl 04f0713012 Merge pull request #106 from narrrl/feat/shared-gallery
feat(photos): release Proton Drive 3.9.0
2026-11-19 21:31:16 +01:00
nvrl cc020bfb54 feat(photos): unlock hidden albums with system login 2026-11-19 20:55:26 +01:00
nvrl 1e6043d12a fix(photos): keep hidden albums open after unlock 2026-11-19 20:44:44 +01:00
nvrl 834684f6d7 chore(release): prepare Proton Drive 3.9.0 2026-11-19 18:15:54 +01:00
nvrl 1c2a66169f feat(photos): embed galleries and speed up zoom
Keep album browsing and private galleries inside Photos. Build GTK
tiles near the viewport and anchor relayouts on the visible photo.

Migrate the Photos cache to persist pending event work with its cursor.
Pin the published Gallery engine for scoped private album reads.
2026-11-19 17:56:32 +01:00
nvrl cb5ff8b8da feat(photos): integrate the shared Gallery engine
Share gallery settings and layout with Proton Gallery. Preserve Git
source mappings so COPR can resolve the dependency offline.
2026-11-19 11:42:47 +01:00
nvrl c57d98799a fix(sync): explain moves blocked by Trash
Keep blocked moves queued so they can land after a restore. Retain
simulation logs for intermittent conflicts and pick up the SDK fix for
uploads from machines with fast clocks.
2026-11-19 05:01:52 +01:00
nvrl 7d9d5850a4 Merge pull request #105 from narrrl/feat/branding
Feat/branding
2026-11-19 01:13:39 +01:00
nvrl a69ebbdd10 chore(version) 2026-11-19 00:57:31 +01:00
nvrl ddf7335894 fix(cache eviction) fixed a bug which didn't scan past pinned files and now reapplies at startup 2026-11-19 00:54:47 +01:00
nvrl 140bd2eee3 Merge pull request #104 from narrrl/feat/branding
feat(branding) added new icons and branding
2026-11-18 11:16:58 +01:00
nvrl 2462cb24bd fix(removed old logo) 2026-11-18 01:42:13 +01:00
nvrl a1dd2b5a9b feat(branding) added new icons and branding 2026-11-18 01:35:08 +01:00
Nils Pukropp e9d6d4a907 chore(aur): update packages to 3.8.0 2026-11-16 00:32:51 +00:00
nvrl 6fa91e83f1 Merge pull request #81 from narrrl/chore/release-3.8.0
chore(release): 3.8.0
2026-11-16 01:23:44 +01:00
nvrl 7d445bb02c Merge pull request #84 from narrrl/feat/video-thumbnails
feat(gui,fuse,core): video thumbnails in file views
2026-11-14 03:43:36 +01:00
124 changed files with 16541 additions and 42333 deletions

No files matched your search

+8 -5
View File
@@ -105,14 +105,17 @@ jobs:
- name: Simulation runs
run: 'cargo test -p pdfs-fuse --lib --locked sim:: -- --ignored --test-threads=1'
# A hung step writes every daemon thread's stack next to the run's
# state; without them a hang that does not replay cannot be told apart.
- name: Keep the stacks of a hung run
# Retain the operation trace and daemon logs for the failing seed, plus
# every daemon thread's stack if the watchdog caught a hang.
- name: Keep simulation failure diagnostics
if: failure()
uses: actions/upload-artifact@v7
with:
name: simulation-stacks
path: /tmp/pdfs-sim-*/stacks.txt
name: simulation-diagnostics
path: |
/tmp/pdfs-sim-*/steps.log
/tmp/pdfs-sim-*/daemon.log
/tmp/pdfs-sim-*/stacks.txt
if-no-files-found: ignore
# Keeps the release-profile dependencies cached on main, where the release
+13 -3
View File
@@ -129,8 +129,12 @@ jobs:
# Every crate the build needs, so COPR can build the spec without network.
- name: Package vendored crates (.tar.xz)
run: |
cargo vendor --locked dist/vendor > /dev/null
tar -cJf proton-drive-linux-vendor-${{ env.VERSION }}.tar.xz -C dist vendor
mkdir -p dist/.cargo
(
cd dist
cargo vendor --locked --manifest-path ../Cargo.toml vendor > .cargo/config.toml
)
tar -cJf proton-drive-linux-vendor-${{ env.VERSION }}.tar.xz -C dist vendor .cargo/config.toml
- name: Package Debian package (.deb)
run: |
@@ -152,8 +156,12 @@ jobs:
cp packaging/io.narl.proton-drive-linux.desktop deb/usr/share/applications/
cp packaging/io.narl.proton-drive-linux-tray.desktop deb/etc/xdg/autostart/
cp packaging/io.narl.proton-drive-linux.svg deb/usr/share/icons/hicolor/scalable/apps/
install -Dm644 packaging/io.narl.proton-drive-linux.policy deb/usr/share/polkit-1/actions/io.narl.proton-drive-linux.policy
cp packaging/proton-drive.service deb/usr/lib/systemd/user/
cp LICENSE deb/usr/share/licenses/proton-drive-linux/
# The offline Places basemap, already fetched for the tarball.
scripts/fetch-basemap.sh
install -Dm644 data/basemap-v1.pmtiles deb/usr/share/proton-drive-linux/basemap-v1.pmtiles
po/build.sh deb/usr/share/locale
packaging/completions.sh target/release/pdfs deb/usr/share
@@ -171,7 +179,7 @@ jobs:
Priority: optional
Architecture: amd64
Maintainer: Nils Pukropp <contact@narl.io>
Depends: fuse3, libgtk-4-1 (>= 4.12.0), libadwaita-1-0 (>= 1.5.0), libsecret-1-0, libwebkitgtk-6.0-4, libimage-exiftool-perl
Depends: polkitd, fuse3, libgtk-4-1 (>= 4.12.0), libadwaita-1-0 (>= 1.5.0), libsecret-1-0, libwebkitgtk-6.0-4, libimage-exiftool-perl
Installed-Size: ${INSTALLED_SIZE}
Homepage: https://proton-drive.narl.io
Description: Unofficial Proton Drive client for Linux
@@ -186,6 +194,8 @@ jobs:
- name: Package Fedora package (.rpm)
run: |
# The in-tree spec build installs the basemap from data/.
scripts/fetch-basemap.sh
# Sanitize version (replace hyphens with dots for RPM compatibility)
RPM_VERSION=$(echo "${{ env.VERSION }}" | tr '-' '.')
+1
View File
@@ -65,6 +65,7 @@ jobs:
git config --global --add safe.directory "$GITHUB_WORKSPACE"
VERSION=$(grep -m1 '^version' Cargo.toml | cut -d'"' -f2)
SHORT=${GITHUB_SHA::7}
scripts/fetch-basemap.sh
mkdir -p rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS,BUILDROOT,db}
rpmbuild -bb \
--nodeps \
+1
View File
@@ -5,3 +5,4 @@
/rpmbuild
/result
/result-*
/data/*.pmtiles
Generated
+247 -13
View File
@@ -2,6 +2,22 @@
# It is not intended for manual editing.
version = 4
[[package]]
name = "ab_glyph"
version = "0.2.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01c0457472c38ea5bd1c3b5ada5e368271cb550be7a4ca4a0b4634e9913f6cc2"
dependencies = [
"ab_glyph_rasterizer",
"owned_ttf_parser",
]
[[package]]
name = "ab_glyph_rasterizer"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "366ffbaa4442f4684d91e2cd7c5ea7c4ed8add41959a31447066e279e432b618"
[[package]]
name = "adler2"
version = "2.0.1"
@@ -49,6 +65,7 @@ dependencies = [
"cipher 0.5.2",
"cpubits",
"cpufeatures 0.3.1",
"zeroize",
]
[[package]]
@@ -77,6 +94,7 @@ dependencies = [
"ctr 0.10.1",
"ctutils",
"ghash 0.6.0",
"zeroize",
]
[[package]]
@@ -180,6 +198,17 @@ version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470"
[[package]]
name = "apple-native-keyring-store"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b350bfd03649e07aa05c0a81b3e15934374e585c98204a57e20b9d49f49bb9a"
dependencies = [
"keyring-core",
"log",
"security-framework",
]
[[package]]
name = "arbitrary"
version = "1.5.0"
@@ -219,6 +248,12 @@ dependencies = [
"zeroize",
]
[[package]]
name = "arrayref"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76a2e8124351fda1ef8aaaa3bbd7ebbcb486bbcd4225aca0aa0d84bb2db8fecb"
[[package]]
name = "arrayvec"
version = "0.7.8"
@@ -1878,6 +1913,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2eecf2d5dc9b66b732b97707a0210906b1d30523eb773193ab777c0c84b3e8d5"
dependencies = [
"polyval 0.7.3",
"zeroize",
]
[[package]]
@@ -2237,6 +2273,12 @@ version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87"
[[package]]
name = "httpdate"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
[[package]]
name = "hybrid-array"
version = "0.4.15"
@@ -2260,6 +2302,7 @@ dependencies = [
"http",
"http-body",
"httparse",
"httpdate",
"itoa",
"pin-project-lite",
"smallvec",
@@ -3404,6 +3447,15 @@ dependencies = [
"pin-project-lite",
]
[[package]]
name = "owned_ttf_parser"
version = "0.25.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36820e9051aca1014ddc75770aab4d68bc1e9e632f0f5627c4086bc216fb583b"
dependencies = [
"ttf-parser",
]
[[package]]
name = "p256"
version = "0.13.2"
@@ -3582,7 +3634,7 @@ checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4"
[[package]]
name = "pdfs-cli"
version = "3.8.0"
version = "3.9.0"
dependencies = [
"anyhow",
"clap",
@@ -3599,7 +3651,7 @@ dependencies = [
[[package]]
name = "pdfs-core"
version = "3.8.0"
version = "3.9.0"
dependencies = [
"async-trait",
"base64 0.23.1",
@@ -3608,33 +3660,37 @@ dependencies = [
"dbus-secret-service-keyring-store",
"directories",
"ignore",
"kamadak-exif",
"keyring-core",
"libc",
"parking_lot 0.13.0",
"proton-auth-rs",
"proton-base-rs",
"proton-drive-rs",
"proton-gallery-rs",
"proton-sdk",
"rusqlite",
"serde",
"serde_json",
"sha1 0.11.0",
"sha2 0.11.1",
"shell-words",
"thiserror 2.0.21",
"tokio",
"tracing",
"zeroize",
"zip",
]
[[package]]
name = "pdfs-fuse"
version = "3.8.0"
version = "3.9.0"
dependencies = [
"async-trait",
"base64 0.23.1",
"fuser",
"futures",
"getrandom 0.4.3",
"http-body-util",
"hyper",
"hyper-util",
"image",
"libc",
"lofty",
@@ -3642,21 +3698,27 @@ dependencies = [
"parking_lot 0.13.0",
"pdfs-core",
"proton-drive-rs",
"proton-gallery-app-rs",
"proton-gallery-rs",
"proton-media-rs",
"rustix",
"serde",
"serde_json",
"sha1 0.11.0",
"tokio",
"tracing",
"tracing-subscriber",
"zeroize",
"zip",
]
[[package]]
name = "pdfs-gui"
version = "3.8.0"
version = "3.9.0"
dependencies = [
"anyhow",
"async-channel",
"base64 0.23.1",
"gettext-rs",
"gtk4",
"kamadak-exif",
@@ -3664,6 +3726,8 @@ dependencies = [
"libadwaita",
"pangocairo",
"pdfs-core",
"proton-gallery-rs",
"proton-map-rs",
"reqwest",
"serde",
"serde_json",
@@ -3825,6 +3889,7 @@ dependencies = [
"cpubits",
"cpufeatures 0.3.1",
"universal-hash 0.6.1",
"zeroize",
]
[[package]]
@@ -3892,10 +3957,36 @@ dependencies = [
]
[[package]]
name = "proton-drive-rs"
version = "0.12.0"
name = "proton-auth-rs"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4265c2cf3bdc59ee273d7363c8b6a7cb4e98be6102ffa18144c4d8ebda00737b"
checksum = "383119b502a1741f1ec4a248cb1e8d2f4835d5ea415803b041f76b6f89cf1ffc"
dependencies = [
"apple-native-keyring-store",
"dbus-secret-service",
"dbus-secret-service-keyring-store",
"keyring-core",
"thiserror 2.0.21",
"tracing",
"windows-native-keyring-store",
"zeroize",
]
[[package]]
name = "proton-base-rs"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "451151bb1025ea5cb362f69e2bfb2d45afed603e5c6daa3dc752333a7c29c878"
dependencies = [
"rusqlite",
"thiserror 2.0.21",
]
[[package]]
name = "proton-drive-rs"
version = "0.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "670bfcf0eeb15886c962c18c76dbac976236e4377bfe073eb2360dc1ce448da2"
dependencies = [
"async-trait",
"base64 0.23.1",
@@ -3920,10 +4011,102 @@ dependencies = [
]
[[package]]
name = "proton-sdk"
version = "0.12.0"
name = "proton-gallery-app-rs"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb98ab3e336ed95f82368a50348dad00e5fc919e982d748e83e844afcadbd5d8"
checksum = "d6d18d11266ab2be5ab2f4b4040941caf2736c257e13909af7be8ea331892d59"
dependencies = [
"async-trait",
"futures-util",
"hex",
"proton-drive-rs",
"proton-gallery-drive-rs",
"proton-gallery-rs",
"proton-sdk",
"serde_json",
"sha2 0.11.1",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tracing",
"zeroize",
]
[[package]]
name = "proton-gallery-drive-rs"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "535a14f63fd542701d2606acb971d4a7e8f235ca7b9439fea8c80a3f26110186"
dependencies = [
"async-trait",
"hex",
"proton-drive-rs",
"proton-gallery-rs",
"proton-sdk",
"serde",
"serde_json",
"sha2 0.11.1",
"thiserror 2.0.21",
"tokio",
"tracing",
"zeroize",
]
[[package]]
name = "proton-gallery-rs"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e374d2053de5df0e6b0797df65d0d024dbce35f7aedfefd4a656896a0495dc9b"
dependencies = [
"aes-gcm 0.11.1",
"async-trait",
"getrandom 0.4.3",
"hex",
"kamadak-exif",
"proton-base-rs",
"rusqlite",
"serde",
"serde_json",
"sha2 0.11.1",
"tempfile",
"thiserror 2.0.21",
"tokio",
"tokio-util",
"tracing",
"zeroize",
"zip",
]
[[package]]
name = "proton-map-rs"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a8440e8a25b89e1c3d3c0747285ebc558e84a019b237307111f21306f183e78d"
dependencies = [
"ab_glyph",
"flate2",
"thiserror 2.0.21",
"tiny-skia",
]
[[package]]
name = "proton-media-rs"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f03aa6c403e4f2f7a8f9a444dff1e268d8bc9bcaf1d0d953cc3a1312d798f1ef"
dependencies = [
"lru",
"sha2 0.11.1",
"thiserror 2.0.21",
"tokio",
"tracing",
]
[[package]]
name = "proton-sdk"
version = "0.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f578a9d920cef3f6c4da611793001fe6d2ed81face13c45f5e721274b1945f7d"
dependencies = [
"aes-gcm 0.11.1",
"async-trait",
@@ -4954,6 +5137,12 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "strict-num"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6637bab7722d379c8b41ba849228d680cc12d0a45ba1fa2b48f2a30577a06731"
[[package]]
name = "strsim"
version = "0.11.1"
@@ -5128,6 +5317,31 @@ dependencies = [
"zune-jpeg",
]
[[package]]
name = "tiny-skia"
version = "0.11.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83d13394d44dae3207b52a326c0c85a8bf87f1541f23b0d143811088497b09ab"
dependencies = [
"arrayref",
"arrayvec",
"bytemuck",
"cfg-if",
"log",
"tiny-skia-path",
]
[[package]]
name = "tiny-skia-path"
version = "0.11.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c9e7fc0c2e86a30b117d0462aa261b72b7a99b7ebd7deb3a14ceda95c5bdc93"
dependencies = [
"arrayref",
"bytemuck",
"strict-num",
]
[[package]]
name = "tinystr"
version = "0.8.5"
@@ -5191,6 +5405,7 @@ dependencies = [
"bytes",
"futures-core",
"futures-sink",
"futures-util",
"libc",
"pin-project-lite",
"tokio",
@@ -5364,6 +5579,12 @@ version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
[[package]]
name = "ttf-parser"
version = "0.25.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2df906b07856748fa3f6e0ad0cbaa047052d4a7dd609e231c4f72cee8c36f31"
[[package]]
name = "twofish"
version = "0.7.1"
@@ -5776,6 +5997,19 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-native-keyring-store"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "063426e76fdec7438d56bb777f67e318a84a25c707b07e575cb8b78e10c028f8"
dependencies = [
"byteorder",
"keyring-core",
"regex",
"windows-sys 0.61.2",
"zeroize",
]
[[package]]
name = "windows-result"
version = "0.4.1"
+11 -3
View File
@@ -3,7 +3,7 @@ resolver = "2"
members = ["crates/pdfs-core", "crates/pdfs-fuse", "crates/pdfs-cli", "crates/pdfs-gui"]
[workspace.package]
version = "3.8.0"
version = "3.9.0"
edition = "2024"
license = "MIT"
authors = ["Nils Pukropp <contact@narl.io>"]
@@ -15,8 +15,11 @@ keywords = ["proton", "drive", "fuse", "sync", "linux"]
categories = ["filesystem", "command-line-utilities"]
[workspace.dependencies]
proton-sdk = { version = "0.12.0" }
proton-drive-rs = { version = "0.12.0" }
proton-sdk = { version = "0.12.4" }
proton-drive-rs = { version = "0.12.4" }
proton-gallery-app-rs = "0.1.0"
proton-gallery-rs = "0.1.5"
proton-map-rs = "0.1.0"
chrono = { version = "0.4", default-features = false, features = ["std"] }
# Internal crates.
@@ -61,6 +64,11 @@ rusqlite = { version = "0.41", features = ["bundled"] }
# Secret Service store is the one keyring 3's `sync-secret-service` wrapped, and
# with no `target` it matches items v3 wrote, so existing logins survive.
keyring-core = "1"
# The keyring store all three apps share: reconnects after a Secret Service
# restart (B285) and replaces items instead of editing them (B291).
proton-auth-rs = "0.1.1"
proton-base-rs = "0.1"
proton-media-rs = "0.1.1"
dbus-secret-service-keyring-store = { version = "1", features = ["crypto-rust"] }
# The same Secret Service client the store wraps, used directly to replace an
# item instead of editing it, which oo7 before 0.7.0 kept only in memory (B291).
+3 -2
View File
@@ -1,6 +1,6 @@
<div align="center">
<img src="images/logo.svg" alt="" width="112" height="112">
<a href="images/logo.svg"><img src="images/logo.svg" alt="Proton Drive for Linux icon" width="128" height="128"></a>
# Proton Drive for Linux
@@ -193,7 +193,8 @@ Proton Drive Linux is free and built in spare time. If it saves you a few headac
Released under the [MIT License](LICENSE).
The Places map uses [GeoNames](https://www.geonames.org/) data (CC BY 4.0) and
[Natural Earth](https://www.naturalearthdata.com/) outlines (public domain). The optional street
[Natural Earth](https://www.naturalearthdata.com/) land, water and borders (public domain), drawn
through [proton-map-rs](https://github.com/narrrl/proton-rs). The optional street
map comes from [OpenFreeMap](https://openfreemap.org/), © [OpenStreetMap](https://www.openstreetmap.org/copyright)
contributors.
+146 -14
View File
@@ -1,6 +1,6 @@
//! `pdfs` — command-line front-end for the Proton Drive Linux client.
use std::collections::HashMap;
use std::collections::{BTreeMap, HashMap};
use std::fs::OpenOptions;
use std::io::Write;
use std::os::unix::fs::FileTypeExt;
@@ -158,8 +158,12 @@ enum Command {
#[arg(long)]
forget: bool,
},
/// Forget the stored session.
Logout,
/// Forget the stored session. Asks first while changes are still queued.
Logout {
/// Do not ask first.
#[arg(long)]
yes: bool,
},
/// Show account login state, and mount status if a daemon is running.
Status,
/// List every local Proton Drive location.
@@ -432,7 +436,14 @@ enum Command {
Diagnostics,
/// List what is in the account's trash, with the uids the restore and
/// delete-forever commands take.
Trash,
Trash {
/// Group the trash by the folder each item was deleted from.
#[arg(long, conflicts_with = "folder")]
by_folder: bool,
/// List what a trashed folder holds, by its uid.
#[arg(long, value_name = "UID")]
folder: Option<String>,
},
/// Restore trashed items, by uid, to the folders they were trashed from.
Restore {
/// Node uids (`volume~link`), as printed by `pdfs trash`.
@@ -857,6 +868,13 @@ enum SyncCmd {
},
/// List uploads and changes not yet on Proton Drive.
Queue,
/// Wait until every queued change is on Proton Drive. Exits 0 once nothing
/// is left, 3 when what is left needs you (see `sync issues`), 4 on timeout.
Flush {
/// Give up after this many seconds.
#[arg(long)]
timeout: Option<u64>,
},
/// List queued changes Proton Drive refused or that keep failing, and what to do.
Issues,
/// Save the content of a queued upload (by id from `sync queue`) to a file.
@@ -1047,7 +1065,7 @@ fn main() -> Result<()> {
cli.command,
Command::Login { .. }
| Command::Unlock { .. }
| Command::Logout
| Command::Logout { .. }
| Command::Mount { .. }
| Command::Daemon { .. }
) {
@@ -1057,7 +1075,7 @@ fn main() -> Result<()> {
Command::Login { username, remember } => cmd_login(username, remember),
Command::Unlock { remember } => cmd_unlock(remember),
Command::Lock { forget } => cmd_lock(forget),
Command::Logout => cmd_logout(),
Command::Logout { yes } => cmd_logout(yes),
Command::Status => cmd_status(),
Command::Locations => cmd_locations(),
Command::Mount { mountpoint } => cmd_mount(mountpoint),
@@ -1128,7 +1146,7 @@ fn main() -> Result<()> {
Command::Upload { sources, parent } => cmd_upload(sources, parent),
Command::Transfers => cmd_transfers(),
Command::Diagnostics => cmd_diagnostics(),
Command::Trash => cmd_trash(),
Command::Trash { by_folder, folder } => cmd_trash(by_folder, folder),
Command::Restore { uids } => cmd_restore(uids),
Command::DeleteForever { uids } => cmd_delete_forever(uids),
Command::EmptyTrash => cmd_empty_trash(),
@@ -1362,6 +1380,7 @@ fn cmd_sync(action: SyncCmd) -> Result<()> {
})?)?
}
SyncCmd::Queue => return cmd_sync_queue(),
SyncCmd::Flush { timeout } => return cmd_sync_flush(timeout),
SyncCmd::Issues => return cmd_sync_issues(),
SyncCmd::Export { id, dest } => return cmd_sync_export(id, &dest),
SyncCmd::Limit { up, down } => return cmd_sync_limit(up, down),
@@ -1465,6 +1484,71 @@ fn cmd_sync_queue() -> Result<()> {
Ok(())
}
/// `pdfs sync flush`: wait for the queue to drain.
///
/// Failed changes are retried at once rather than after their backoff. The
/// wait ends early, with exit code 3, when everything left is waiting on the
/// user: refused by Drive, failing again and again, or a temporary file
/// waiting for its final name. Paused or offline, it keeps waiting and says
/// why.
fn cmd_sync_flush(timeout: Option<u64>) -> Result<()> {
const POLL: std::time::Duration = std::time::Duration::from_secs(1);
let deadline =
timeout.map(|secs| std::time::Instant::now() + std::time::Duration::from_secs(secs));
ok_or_bail(control_request(CtlRequest::RetryPendingOp { id: None })?)?;
let mut last_line = String::new();
loop {
let items = match control_request(CtlRequest::ListPendingOps)? {
CtlResponse::PendingOps { items } => items,
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
};
let waiting = items
.iter()
.filter(|op| !op.parked && !op.failing && op.issue.is_none())
.count();
if items.is_empty() {
finish_line(&last_line);
println!("Everything is on Proton Drive.");
return Ok(());
}
if waiting == 0 {
finish_line(&last_line);
eprintln!(
"{} queued change(s) need you before they can be sent. See `pdfs sync issues`.",
items.len()
);
std::process::exit(3);
}
let reason = match control_request(CtlRequest::Status)? {
CtlResponse::Status { paused: true, .. } => " (sync is paused; `pdfs sync resume`)",
CtlResponse::Status { online: false, .. } => " (offline; waiting for Proton Drive)",
_ => "",
};
let line = format!("{waiting} change(s) left to send{reason}");
if line != last_line {
if !last_line.is_empty() {
print!("\r\x1b[2K");
}
print!("{line}");
last_line = line;
}
if deadline.is_some_and(|deadline| std::time::Instant::now() >= deadline) {
finish_line(&last_line);
eprintln!("Timed out with {waiting} change(s) still queued.");
std::process::exit(4);
}
std::thread::sleep(POLL);
}
}
/// End a progress line so the next message starts on its own.
fn finish_line(last_line: &str) {
if !last_line.is_empty() {
println!();
}
}
/// `pdfs sync issues`: the queued changes that need the user, and what to do.
fn cmd_sync_issues() -> Result<()> {
let response = control_request(CtlRequest::ListPendingOps)?;
@@ -1511,6 +1595,7 @@ fn issue_text(issue: SyncIssue) -> &'static str {
"you can no longer change this on Proton Drive, for example because a share was \
revoked. Save a copy to keep the content."
}
SyncIssue::Trashed => "restore the item or folder from Trash, then retry the move.",
SyncIssue::Missing => {
"the file or its folder is gone from Proton Drive. Save a copy to keep the content."
}
@@ -2297,7 +2382,27 @@ fn key_state_name() -> &'static str {
}
}
fn cmd_logout() -> Result<()> {
fn cmd_logout(yes: bool) -> Result<()> {
// Queued changes stay staged on this computer and are sent after the next
// sign-in, but until then they exist nowhere else. Without a running
// daemon there is no count to show.
if !yes
&& let Ok(CtlResponse::Status {
pending_uploads,
pending_changes,
parked_uploads,
..
}) = control_request(CtlRequest::Status)
{
let queued = pending_uploads + pending_changes + parked_uploads;
if queued > 0 {
println!(
"{queued} change(s) are not on Proton Drive yet. They stay on this computer and are \
sent after you sign in again. Run `pdfs sync flush` first to send them now."
);
confirm("Sign out anyway?")?;
}
}
if auth::logout()? {
println!("Signed out: the session is ended on Proton and cleared here.");
} else {
@@ -2695,6 +2800,7 @@ fn mount_once(mountpoint: Option<PathBuf>) -> Result<pdfs_fuse::MountOutcome> {
db,
pdfs_fuse::MountOptions {
username,
gallery_session: Some(session.clone()),
sweep_mode: config.resolved_conflict_sweep(),
upload_limit: config.upload_limit.unwrap_or(0),
download_limit: config.download_limit.unwrap_or(0),
@@ -3518,17 +3624,37 @@ fn cmd_diagnostics() -> Result<()> {
Ok(())
}
fn cmd_trash() -> Result<()> {
let response = control_request(CtlRequest::ListTrash)?;
fn cmd_trash(by_folder: bool, folder: Option<String>) -> Result<()> {
let request = match folder {
Some(uid) => CtlRequest::ListTrashedFolder { uid },
None => CtlRequest::ListTrash,
};
let response = control_request(request)?;
if emit_json(&response)? {
return Ok(());
}
match response {
CtlResponse::Entries { entries } if entries.is_empty() => println!("(trash is empty)"),
CtlResponse::Entries { entries } if entries.is_empty() => println!("(nothing here)"),
CtlResponse::Entries { entries } if by_folder => {
let mut groups: BTreeMap<String, Vec<&pdfs_core::control::DirEntry>> = BTreeMap::new();
for e in entries.iter().filter(|e| e.trashed_parent.is_none()) {
let folder = match e.trashed_from.as_deref() {
None => "(unknown folder)".to_string(),
Some("") => "/".to_string(),
Some(path) => format!("/{path}"),
};
groups.entry(folder).or_default().push(e);
}
for (folder, items) in groups {
println!("{folder} ({} item(s))", items.len());
for e in items {
print_trashed(e, " ");
}
}
}
CtlResponse::Entries { entries } => {
for e in entries {
let kind = if e.is_dir { "d" } else { "-" };
println!("{kind} {:>12} {} [{}]", e.size, e.name, e.uid);
for e in &entries {
print_trashed(e, "");
}
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
@@ -3537,6 +3663,11 @@ fn cmd_trash() -> Result<()> {
Ok(())
}
fn print_trashed(e: &pdfs_core::control::DirEntry, indent: &str) {
let kind = if e.is_dir { "d" } else { "-" };
println!("{indent}{kind} {:>12} {} [{}]", e.size, e.name, e.uid);
}
fn cmd_restore(uids: Vec<String>) -> Result<()> {
match control_request(CtlRequest::Restore { uids })? {
CtlResponse::Ok { message } => println!("{message}"),
@@ -4992,6 +5123,7 @@ mod login_wait_tests {
shared_by_unverified: false,
trashed_at: 0,
trashed_from: None,
trashed_parent: None,
issue: None,
};
assert_eq!(shared_by_line(&entry), None, "no invitation, no line");
+3 -3
View File
@@ -12,6 +12,7 @@ description = "Core auth/session/config for the Proton Drive Linux client"
[dependencies]
proton-sdk.workspace = true
proton-drive-rs.workspace = true
proton-gallery-rs.workspace = true
chrono.workspace = true
tokio.workspace = true
serde.workspace = true
@@ -22,10 +23,10 @@ directories.workspace = true
ignore.workspace = true
shell-words.workspace = true
sha2.workspace = true
sha1.workspace = true
zip.workspace = true
rusqlite.workspace = true
keyring-core.workspace = true
proton-auth-rs = { workspace = true, features = ["keyring"] }
proton-base-rs.workspace = true
dbus-secret-service-keyring-store.workspace = true
dbus-secret-service.workspace = true
parking_lot.workspace = true
@@ -33,4 +34,3 @@ async-trait.workspace = true
libc.workspace = true
zeroize.workspace = true
base64.workspace = true
exif = { version = "0.6", package = "kamadak-exif" }
File diff suppressed because it is too large. Load diff
-250
View File
@@ -1,250 +0,0 @@
AD Andorra
AE United Arab Emirates
AF Afghanistan
AG Antigua and Barbuda
AI Anguilla
AL Albania
AM Armenia
AO Angola
AQ Antarctica
AR Argentina
AS American Samoa
AT Austria
AU Australia
AW Aruba
AX Åland Islands
AZ Azerbaijan
BA Bosnia and Herzegovina
BB Barbados
BD Bangladesh
BE Belgium
BF Burkina Faso
BG Bulgaria
BH Bahrain
BI Burundi
BJ Benin
BL Saint Barthélemy
BM Bermuda
BN Brunei Darussalam
BO Bolivia, Plurinational State of
BQ Bonaire, Sint Eustatius and Saba
BR Brazil
BS Bahamas
BT Bhutan
BV Bouvet Island
BW Botswana
BY Belarus
BZ Belize
CA Canada
CC Cocos (Keeling) Islands
CD Congo, The Democratic Republic of the
CF Central African Republic
CG Congo
CH Switzerland
CI Côte d'Ivoire
CK Cook Islands
CL Chile
CM Cameroon
CN China
CO Colombia
CR Costa Rica
CU Cuba
CV Cabo Verde
CW Curaçao
CX Christmas Island
CY Cyprus
CZ Czechia
DE Germany
DJ Djibouti
DK Denmark
DM Dominica
DO Dominican Republic
DZ Algeria
EC Ecuador
EE Estonia
EG Egypt
EH Western Sahara
ER Eritrea
ES Spain
ET Ethiopia
FI Finland
FJ Fiji
FK Falkland Islands (Malvinas)
FM Micronesia, Federated States of
FO Faroe Islands
FR France
GA Gabon
GB United Kingdom
GD Grenada
GE Georgia
GF French Guiana
GG Guernsey
GH Ghana
GI Gibraltar
GL Greenland
GM Gambia
GN Guinea
GP Guadeloupe
GQ Equatorial Guinea
GR Greece
GS South Georgia and the South Sandwich Islands
GT Guatemala
GU Guam
GW Guinea-Bissau
GY Guyana
HK Hong Kong
HM Heard Island and McDonald Islands
HN Honduras
HR Croatia
HT Haiti
HU Hungary
ID Indonesia
IE Ireland
IL Israel
IM Isle of Man
IN India
IO British Indian Ocean Territory
IQ Iraq
IR Iran, Islamic Republic of
IS Iceland
IT Italy
JE Jersey
JM Jamaica
JO Jordan
JP Japan
KE Kenya
KG Kyrgyzstan
KH Cambodia
KI Kiribati
KM Comoros
KN Saint Kitts and Nevis
KP Korea, Democratic People's Republic of
KR Korea, Republic of
KW Kuwait
KY Cayman Islands
KZ Kazakhstan
LA Lao People's Democratic Republic
LB Lebanon
LC Saint Lucia
LI Liechtenstein
LK Sri Lanka
LR Liberia
LS Lesotho
LT Lithuania
LU Luxembourg
LV Latvia
LY Libya
MA Morocco
MC Monaco
MD Moldova, Republic of
ME Montenegro
MF Saint Martin (French part)
MG Madagascar
MH Marshall Islands
MK North Macedonia
ML Mali
MM Myanmar
MN Mongolia
MO Macao
MP Northern Mariana Islands
MQ Martinique
MR Mauritania
MS Montserrat
MT Malta
MU Mauritius
MV Maldives
MW Malawi
MX Mexico
MY Malaysia
MZ Mozambique
NA Namibia
NC New Caledonia
NE Niger
NF Norfolk Island
NG Nigeria
NI Nicaragua
NL Netherlands
NO Norway
NP Nepal
NR Nauru
NU Niue
NZ New Zealand
OM Oman
PA Panama
PE Peru
PF French Polynesia
PG Papua New Guinea
PH Philippines
PK Pakistan
PL Poland
PM Saint Pierre and Miquelon
PN Pitcairn
PR Puerto Rico
PS Palestine, State of
PT Portugal
PW Palau
PY Paraguay
QA Qatar
RE Réunion
RO Romania
RS Serbia
RU Russian Federation
RW Rwanda
SA Saudi Arabia
SB Solomon Islands
SC Seychelles
SD Sudan
SE Sweden
SG Singapore
SH Saint Helena, Ascension and Tristan da Cunha
SI Slovenia
SJ Svalbard and Jan Mayen
SK Slovakia
SL Sierra Leone
SM San Marino
SN Senegal
SO Somalia
SR Suriname
SS South Sudan
ST Sao Tome and Principe
SV El Salvador
SX Sint Maarten (Dutch part)
SY Syrian Arab Republic
SZ Eswatini
TC Turks and Caicos Islands
TD Chad
TF French Southern Territories
TG Togo
TH Thailand
TJ Tajikistan
TK Tokelau
TL Timor-Leste
TM Turkmenistan
TN Tunisia
TO Tonga
TR Türkiye
TT Trinidad and Tobago
TV Tuvalu
TW Taiwan, Province of China
TZ Tanzania, United Republic of
UA Ukraine
UG Uganda
UM United States Minor Outlying Islands
US United States
UY Uruguay
UZ Uzbekistan
VA Holy See (Vatican City State)
VC Saint Vincent and the Grenadines
VE Venezuela, Bolivarian Republic of
VG Virgin Islands, British
VI Virgin Islands, U.S.
VN Viet Nam
VU Vanuatu
WF Wallis and Futuna
WS Samoa
XK Kosovo
YE Yemen
YT Mayotte
ZA South Africa
ZM Zambia
ZW Zimbabwe
1 AD Andorra
2 AE United Arab Emirates
3 AF Afghanistan
4 AG Antigua and Barbuda
5 AI Anguilla
6 AL Albania
7 AM Armenia
8 AO Angola
9 AQ Antarctica
10 AR Argentina
11 AS American Samoa
12 AT Austria
13 AU Australia
14 AW Aruba
15 AX Åland Islands
16 AZ Azerbaijan
17 BA Bosnia and Herzegovina
18 BB Barbados
19 BD Bangladesh
20 BE Belgium
21 BF Burkina Faso
22 BG Bulgaria
23 BH Bahrain
24 BI Burundi
25 BJ Benin
26 BL Saint Barthélemy
27 BM Bermuda
28 BN Brunei Darussalam
29 BO Bolivia, Plurinational State of
30 BQ Bonaire, Sint Eustatius and Saba
31 BR Brazil
32 BS Bahamas
33 BT Bhutan
34 BV Bouvet Island
35 BW Botswana
36 BY Belarus
37 BZ Belize
38 CA Canada
39 CC Cocos (Keeling) Islands
40 CD Congo, The Democratic Republic of the
41 CF Central African Republic
42 CG Congo
43 CH Switzerland
44 CI Côte d'Ivoire
45 CK Cook Islands
46 CL Chile
47 CM Cameroon
48 CN China
49 CO Colombia
50 CR Costa Rica
51 CU Cuba
52 CV Cabo Verde
53 CW Curaçao
54 CX Christmas Island
55 CY Cyprus
56 CZ Czechia
57 DE Germany
58 DJ Djibouti
59 DK Denmark
60 DM Dominica
61 DO Dominican Republic
62 DZ Algeria
63 EC Ecuador
64 EE Estonia
65 EG Egypt
66 EH Western Sahara
67 ER Eritrea
68 ES Spain
69 ET Ethiopia
70 FI Finland
71 FJ Fiji
72 FK Falkland Islands (Malvinas)
73 FM Micronesia, Federated States of
74 FO Faroe Islands
75 FR France
76 GA Gabon
77 GB United Kingdom
78 GD Grenada
79 GE Georgia
80 GF French Guiana
81 GG Guernsey
82 GH Ghana
83 GI Gibraltar
84 GL Greenland
85 GM Gambia
86 GN Guinea
87 GP Guadeloupe
88 GQ Equatorial Guinea
89 GR Greece
90 GS South Georgia and the South Sandwich Islands
91 GT Guatemala
92 GU Guam
93 GW Guinea-Bissau
94 GY Guyana
95 HK Hong Kong
96 HM Heard Island and McDonald Islands
97 HN Honduras
98 HR Croatia
99 HT Haiti
100 HU Hungary
101 ID Indonesia
102 IE Ireland
103 IL Israel
104 IM Isle of Man
105 IN India
106 IO British Indian Ocean Territory
107 IQ Iraq
108 IR Iran, Islamic Republic of
109 IS Iceland
110 IT Italy
111 JE Jersey
112 JM Jamaica
113 JO Jordan
114 JP Japan
115 KE Kenya
116 KG Kyrgyzstan
117 KH Cambodia
118 KI Kiribati
119 KM Comoros
120 KN Saint Kitts and Nevis
121 KP Korea, Democratic People's Republic of
122 KR Korea, Republic of
123 KW Kuwait
124 KY Cayman Islands
125 KZ Kazakhstan
126 LA Lao People's Democratic Republic
127 LB Lebanon
128 LC Saint Lucia
129 LI Liechtenstein
130 LK Sri Lanka
131 LR Liberia
132 LS Lesotho
133 LT Lithuania
134 LU Luxembourg
135 LV Latvia
136 LY Libya
137 MA Morocco
138 MC Monaco
139 MD Moldova, Republic of
140 ME Montenegro
141 MF Saint Martin (French part)
142 MG Madagascar
143 MH Marshall Islands
144 MK North Macedonia
145 ML Mali
146 MM Myanmar
147 MN Mongolia
148 MO Macao
149 MP Northern Mariana Islands
150 MQ Martinique
151 MR Mauritania
152 MS Montserrat
153 MT Malta
154 MU Mauritius
155 MV Maldives
156 MW Malawi
157 MX Mexico
158 MY Malaysia
159 MZ Mozambique
160 NA Namibia
161 NC New Caledonia
162 NE Niger
163 NF Norfolk Island
164 NG Nigeria
165 NI Nicaragua
166 NL Netherlands
167 NO Norway
168 NP Nepal
169 NR Nauru
170 NU Niue
171 NZ New Zealand
172 OM Oman
173 PA Panama
174 PE Peru
175 PF French Polynesia
176 PG Papua New Guinea
177 PH Philippines
178 PK Pakistan
179 PL Poland
180 PM Saint Pierre and Miquelon
181 PN Pitcairn
182 PR Puerto Rico
183 PS Palestine, State of
184 PT Portugal
185 PW Palau
186 PY Paraguay
187 QA Qatar
188 RE Réunion
189 RO Romania
190 RS Serbia
191 RU Russian Federation
192 RW Rwanda
193 SA Saudi Arabia
194 SB Solomon Islands
195 SC Seychelles
196 SD Sudan
197 SE Sweden
198 SG Singapore
199 SH Saint Helena, Ascension and Tristan da Cunha
200 SI Slovenia
201 SJ Svalbard and Jan Mayen
202 SK Slovakia
203 SL Sierra Leone
204 SM San Marino
205 SN Senegal
206 SO Somalia
207 SR Suriname
208 SS South Sudan
209 ST Sao Tome and Principe
210 SV El Salvador
211 SX Sint Maarten (Dutch part)
212 SY Syrian Arab Republic
213 SZ Eswatini
214 TC Turks and Caicos Islands
215 TD Chad
216 TF French Southern Territories
217 TG Togo
218 TH Thailand
219 TJ Tajikistan
220 TK Tokelau
221 TL Timor-Leste
222 TM Turkmenistan
223 TN Tunisia
224 TO Tonga
225 TR Türkiye
226 TT Trinidad and Tobago
227 TV Tuvalu
228 TW Taiwan, Province of China
229 TZ Tanzania, United Republic of
230 UA Ukraine
231 UG Uganda
232 UM United States Minor Outlying Islands
233 US United States
234 UY Uruguay
235 UZ Uzbekistan
236 VA Holy See (Vatican City State)
237 VC Saint Vincent and the Grenadines
238 VE Venezuela, Bolivarian Republic of
239 VG Virgin Islands, British
240 VI Virgin Islands, U.S.
241 VN Viet Nam
242 VU Vanuatu
243 WF Wallis and Futuna
244 WS Samoa
245 XK Kosovo
246 YE Yemen
247 YT Mayotte
248 ZA South Africa
249 ZM Zambia
250 ZW Zimbabwe
+59
View File
@@ -337,6 +337,12 @@ impl Demo {
Request::ListDir { path } => Response::Entries {
entries: self.dir(path),
},
Request::ListTrash => Response::Entries {
entries: self.trash(),
},
Request::ListTrashedFolder { uid } => Response::Entries {
entries: self.trashed_folder(uid),
},
Request::ReserveFileThumbGeneration => Response::FileThumbGeneration { generation: 1 },
Request::FileThumbs { items, .. } => Response::Thumbs {
items: items
@@ -591,6 +597,59 @@ impl Demo {
.collect()
}
/// The trash: things deleted from a few folders, and a deleted folder with
/// a file deleted from it before it went.
fn trash(&self) -> Vec<DirEntry> {
let item = |name: &str, size: u64, hours: i64, from: &str| DirEntry {
name: name.into(),
is_dir: size == 0,
size,
modified: self.now - 40 * DAY,
uid: format!("demo-trash~{from}/{name}"),
trashed_at: self.now - hours * 3600,
trashed_from: Some(from.into()),
..Default::default()
};
let mut inside = item("Draft 3.odt", 48_120, 30, "");
inside.uid = "demo-trash~Old drafts/Draft 3.odt".into();
inside.trashed_from = None;
inside.trashed_parent = Some("demo-trash~Work/Old drafts".into());
vec![
item("Budget 2025.ods", 51_200, 2, "Documents"),
item("Scan 0042.pdf", 1_204_331, 5, "Documents"),
item("Scan 0043.pdf", 1_198_002, 5, "Documents"),
item("IMG_2041.jpg", 3_402_112, 20, "Pictures/Lisbon"),
item("IMG_2042.jpg", 3_311_870, 20, "Pictures/Lisbon"),
item("IMG_2043.jpg", 3_587_004, 20, "Pictures/Lisbon"),
item("Old drafts", 0, 26, "Work"),
item("Meeting notes (copy).md", 6_120, 50, ""),
item("Invoice 118.pdf", 88_341, 80, "Taxes/2025"),
inside,
]
}
/// What the deleted folder in [`Self::trash`] holds.
fn trashed_folder(&self, uid: &str) -> Vec<DirEntry> {
if uid != "demo-trash~Work/Old drafts" {
return Vec::new();
}
let file = |name: &str, size: u64| DirEntry {
name: name.into(),
size,
modified: self.now - 50 * DAY,
uid: format!("demo-trash~Old drafts/{name}"),
trashed_parent: Some(uid.into()),
..Default::default()
};
let mut gone = file("Draft 3.odt", 48_120);
gone.trashed_at = self.now - 30 * 3600;
vec![
file("Draft 1.odt", 41_002),
file("Draft 2.odt", 44_870),
gone,
]
}
/// My files: folders first, then a few files, some of them pictures with
/// thumbnails. Any folder below the root holds the same few documents.
fn dir(&self, path: &str) -> Vec<DirEntry> {
+105 -170
View File
@@ -28,6 +28,7 @@ use std::time::Duration;
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD as BASE64;
use keyring_core::Entry;
use proton_auth_rs::{KeyringStore, SecretStore};
use proton_drive_rs::{KeySalt, ProtonDriveClient};
use proton_sdk::account::KeyPassphrases;
use proton_sdk::api::{HumanVerification, HumanVerificationCredential};
@@ -35,6 +36,7 @@ use proton_sdk::cache::EncryptedCacheRepository;
use proton_sdk::config::ProtonClientConfiguration;
use proton_sdk::error::ProtonError;
use proton_sdk::http::Tokens;
use proton_sdk::login::{self, LoginError, Step};
use proton_sdk::session::{PasswordMode, ProtonApiSession, ResumeParameters};
use proton_sdk::telemetry::TracingTelemetry;
use serde::{Deserialize, Serialize};
@@ -125,82 +127,28 @@ fn client_config() -> ProtonClientConfiguration {
/// Run `op` on the session's keyring entry.
fn with_keyring<T>(op: impl Fn(&Entry) -> keyring_core::Result<T>) -> Result<T> {
with_keyring_entry(KEYRING_SERVICE, KEYRING_USER, op)
Ok(keyring().with_entry(KEYRING_USER, op)?)
}
/// Run `op` on the entry `service`/`user`, reconnecting once if the Secret
/// Service fails the call.
/// The process's keyring, connected on first use.
///
/// The store holds one D-Bus connection and one Secret Service session for the
/// whole process. The session lives in the provider's memory: once the provider
/// restarts (oo7's unit restarts it on failure, and both oo7 and gnome-keyring
/// can be `--replace`d), every read and write on the old session fails with
/// `NoSession`, for as long as this process runs. A daemon then keeps refreshed
/// tokens only in memory, and a cold boot loses them (`docs/BUGS.md` B285). A
/// platform failure therefore drops the store, and the call is made once more
/// on a new one. A locked keyring, a dismissed prompt or a missing entry is
/// not the connection's fault and comes back as it is.
fn with_keyring_entry<T>(
service: &str,
user: &str,
op: impl Fn(&Entry) -> keyring_core::Result<T>,
) -> Result<T> {
let store = keyring_store()?;
match op(&Entry::new(service, user)?) {
Err(keyring_core::Error::PlatformFailure(e)) => {
tracing::warn!(
error = %e,
"the Secret Service failed this process's connection; reconnecting"
);
drop_keyring_store(&store);
keyring_store()?;
Ok(op(&Entry::new(service, user)?)?)
/// One connection for the whole process; when the Secret Service fails it
/// (a restarted provider forgets the session, `docs/BUGS.md` B285), the call
/// reconnects and is made once more. A changed session replaces its item
/// instead of editing it (B291). Both live in `proton-auth-rs`.
fn keyring() -> &'static KeyringStore {
static STORE: std::sync::OnceLock<KeyringStore> = std::sync::OnceLock::new();
STORE.get_or_init(|| {
#[cfg(not(test))]
{
KeyringStore::new(KEYRING_SERVICE)
}
result => Ok(result?),
}
}
/// Held while the default store is looked at and replaced, so that threads
/// failing at the same time drop the broken store once and share its
/// replacement.
static KEYRING_STORE: parking_lot::Mutex<()> = parking_lot::Mutex::new(());
/// The process's Secret Service store, connected now if there is none.
///
/// Connected lazily rather than once at startup so a session bus that was not
/// up yet on the first attempt is tried again on the next one.
fn keyring_store() -> Result<Arc<keyring_core::CredentialStore>> {
let _swap = KEYRING_STORE.lock();
if let Some(store) = keyring_core::get_default_store() {
return Ok(store);
}
let store = connect_keyring_store()?;
keyring_core::set_default_store(Arc::clone(&store));
Ok(store)
}
/// Forget `failed`, unless another thread has already replaced it.
fn drop_keyring_store(failed: &Arc<keyring_core::CredentialStore>) {
let _swap = KEYRING_STORE.lock();
if keyring_core::get_default_store().is_some_and(|store| Arc::ptr_eq(&store, failed)) {
keyring_core::unset_default_store();
}
}
#[cfg(not(test))]
fn connect_keyring_store() -> Result<Arc<keyring_core::CredentialStore>> {
Ok(dbus_secret_service_keyring_store::Store::new()?)
}
/// The tests' in-memory store when they set one, so a reconnect finds the
/// same entries; the real Secret Service otherwise.
#[cfg(test)]
fn connect_keyring_store() -> Result<Arc<keyring_core::CredentialStore>> {
tests::KEYRING_CONNECTS.fetch_add(1, Ordering::SeqCst);
match tests::MOCK_STORE.lock().clone() {
Some(store) => Ok(store),
None => Ok(dbus_secret_service_keyring_store::Store::new()?),
}
#[cfg(test)]
{
KeyringStore::with_connector(KEYRING_SERVICE, Arc::new(tests::connect_keyring_store))
.replace_items(|| tests::MOCK_STORE.lock().is_none())
}
})
}
/// What the user answered when asked for the mailbox password.
@@ -332,74 +280,56 @@ pub async fn login_verified(
get_totp: impl Fn() -> Result<String>,
get_mailbox: impl Fn(u32) -> Result<Mailbox>,
) -> Result<()> {
let password = Zeroizing::new(password.to_owned());
let session = ProtonApiSession::begin_verified(
client_config(),
username,
password.as_bytes(),
verification,
)
.await
.map_err(classify_verification_gate);
let mut session = session?;
if session.is_waiting_for_second_factor() {
let code = get_totp()?;
session.apply_second_factor_code(code.trim()).await?;
}
// Grab the key salts while this access token still has the `locked` scope:
// after its first refresh it never will again, and without them no later
// resume can unlock the key chain. See `StoredSession::key_salts`.
let key_salts = ProtonDriveClient::new(&session, Vec::new())
.account()
.key_salts()
.await?;
// A single-password account's mailbox password is the login password. A
// two-password one has a second secret the user must type, and it is checked
// against the account now — a typo found at login is a retry, found later it
// is a daemon that never mounts.
let (passphrases, remember) = match session.password_mode() {
PasswordMode::Single => (
derive_verified(&session, password.as_bytes(), key_salts.clone()).await?,
true,
),
PasswordMode::Dual => {
let mut attempt = 0;
loop {
let mailbox = get_mailbox(attempt)?;
match derive_verified(&session, mailbox.password.as_bytes(), key_salts.clone())
.await
{
Ok(passphrases) => break (passphrases, mailbox.remember),
Err(Error::WrongMailboxPassword) if attempt + 1 < MAILBOX_ATTEMPTS => {
attempt += 1;
}
Err(e) => return Err(e),
}
// The sequence is `proton_sdk::login`, shared with the other Proton
// apps. It reads the key salts before unlocking, while this access token
// still has the `locked` scope: after its first refresh it never will
// again, and without them no later resume can unlock the key chain. See
// `StoredSession::key_salts`.
let mut step = login::begin(client_config(), username, password, verification)
.await
.map_err(sign_in_error)?;
let (unlocked, remember) = loop {
step = match step {
Step::SecondFactor(pending) => {
let code = get_totp()?;
pending.second_factor(&code).await.map_err(sign_in_error)?
}
}
// A single-password account's mailbox password is the login password.
Step::Unlock(pending) => break (pending.unlock().await.map_err(sign_in_error)?, true),
// A two-password one has a second secret the user must type, and it is
// checked against the account now — a typo found at login is a retry,
// found later it is a daemon that never mounts.
Step::MailboxPassword(pending) => {
let mut attempt = 0;
break loop {
let mailbox = get_mailbox(attempt)?;
match pending.mailbox_password(&mailbox.password).await {
Ok(unlocked) => break (unlocked, mailbox.remember),
Err(LoginError::WrongMailboxPassword) if attempt + 1 < MAILBOX_ATTEMPTS => {
attempt += 1;
}
Err(e) => return Err(sign_in_error(e)),
}
};
}
};
};
let tokens = session.current_tokens().await;
register_refresh_handler(&session, tokens.refresh_token.clone());
let tokens = unlocked.resume_parameters().await;
register_refresh_handler(&unlocked.session, tokens.refresh_token.clone());
let stored = StoredSession {
session_id: session.session_id().as_str().to_owned(),
username: session.username().to_owned(),
user_id: session.user_id().as_str().to_owned(),
session_id: tokens.session_id.as_str().to_owned(),
username: tokens.username,
user_id: tokens.user_id.as_str().to_owned(),
access_token: tokens.access_token,
refresh_token: tokens.refresh_token,
scopes: session.scopes().to_vec(),
password_mode: match session.password_mode() {
PasswordMode::Single => 1,
PasswordMode::Dual => 2,
},
scopes: tokens.scopes,
password_mode: tokens.password_mode.to_wire() as u8,
mailbox_password: String::new(),
key_salts,
key_salts: unlocked.key_salts,
key_passphrases: None,
};
keep_passphrases(stored, &passphrases, remember)
keep_passphrases(stored, &unlocked.key_passphrases, remember)
}
/// Derive the key passphrases from `mailbox` and prove they unlock the account.
@@ -473,52 +403,34 @@ fn decode_secret(secret: &[u8]) -> Option<KeyPassphrases> {
decode(&serde_json::from_slice(secret).ok()?)
}
/// Lift a human-verification gate out of the generic API error into the typed
/// variant a front-end can act on.
/// A failed sign-in step as this crate's error.
///
/// Only a gate that names a solvable method is converted. A `9001` with no
/// `Details`, or one offering only `email`/`sms`, stays a plain API error:
/// promoting it would send the UI to open a verification page it cannot
/// complete, which reads to the user as a hang rather than a refusal.
fn classify_verification_gate(e: ProtonError) -> Error {
let ProtonError::Api(api) = &e else {
return e.into();
};
match api.human_verification() {
Some(hv) if hv.supports_captcha() => Error::HumanVerificationRequired(Box::new(hv)),
_ => e.into(),
/// A blocked, refused or wrong-code attempt goes back to the Proton error it
/// came from, which is what the front-ends word themselves. Only a gate that
/// names a solvable method is a [`Error::HumanVerificationRequired`]: one that
/// offers only `email`/`sms` stays a plain API error, because sending the UI to
/// open a page it cannot complete reads as a hang rather than a refusal.
fn sign_in_error(e: LoginError) -> Error {
match e {
LoginError::HumanVerification(hv) => Error::HumanVerificationRequired(hv),
LoginError::SecurityKeyOnly => Error::SecurityKeyOnly,
LoginError::Blocked(api) | LoginError::Rejected(api) | LoginError::WrongCode(api) => {
Error::Proton(ProtonError::Api(api))
}
LoginError::WrongMailboxPassword => Error::WrongMailboxPassword,
LoginError::Proton(e) => e.into(),
other => Error::Other(other.to_string()),
}
}
/// Save `stored` to the keyring.
///
/// An existing entry is replaced by a new item rather than edited, because
/// oo7 before 0.7.0 keeps an edit only in memory and a reboot brings the
/// spent refresh token back (`docs/BUGS.md` B291). The usual write is the
/// fallback: for the first save, and whenever replacing is not possible.
/// Save the session. An existing entry is replaced by a new item rather than
/// edited, because oo7 before 0.7.0 keeps an edit only in memory and a reboot
/// brings the spent refresh token back (`docs/BUGS.md` B291).
fn write_stored(stored: &StoredSession) -> Result<()> {
let json = Zeroizing::new(serde_json::to_string(stored)?);
if replace_items() {
match crate::secretservice::replace(KEYRING_SERVICE, KEYRING_USER, json.as_bytes()) {
crate::secretservice::Replaced::Saved => return Ok(()),
crate::secretservice::Replaced::Skipped(why) => {
tracing::debug!(%why, "not replacing the keyring item; writing it in place");
}
}
}
with_keyring(|entry| entry.set_password(&json))
}
/// Whether [`write_stored`] talks to the Secret Service directly: always,
/// except in tests that use the in-memory store.
#[cfg(not(test))]
fn replace_items() -> bool {
true
}
#[cfg(test)]
fn replace_items() -> bool {
tests::MOCK_STORE.lock().is_none()
Ok(keyring().set(KEYRING_USER, &json)?)
}
/// A short, one-way tag for a refresh token, for the log: enough to tell
@@ -1324,9 +1236,30 @@ mod tests {
let store: Arc<keyring_core::CredentialStore> = keyring_core::mock::Store::new().unwrap();
*MOCK_STORE.lock() = Some(Arc::clone(&store));
keyring_core::set_default_store(store);
keyring().disconnect();
turn
}
/// The tests' in-memory store when they set one, so a reconnect finds the
/// same entries; the real Secret Service otherwise.
pub(super) fn connect_keyring_store() -> keyring_core::Result<Arc<keyring_core::CredentialStore>>
{
KEYRING_CONNECTS.fetch_add(1, Ordering::SeqCst);
match MOCK_STORE.lock().clone() {
Some(store) => Ok(store),
None => Ok(dbus_secret_service_keyring_store::Store::new()?),
}
}
/// Run `op` on the entry `service`/`user` of the real Secret Service.
fn with_keyring_entry<T>(
store: &KeyringStore,
user: &str,
op: impl Fn(&Entry) -> keyring_core::Result<T>,
) -> Result<T> {
Ok(store.with_entry(user, op)?)
}
static KEYRING: parking_lot::Mutex<()> = parking_lot::Mutex::new(());
/// The in-memory store a reconnect hands out, so it finds the same
@@ -1427,7 +1360,8 @@ mod tests {
let _turn = KEYRING.lock();
*MOCK_STORE.lock() = None;
keyring_core::unset_default_store();
let (service, user) = ("pdfs-test", "restart");
let user = "restart";
let service = &KeyringStore::with_connector("pdfs-test", Arc::new(connect_keyring_store));
with_keyring_entry(service, user, |e| e.set_password("before")).unwrap();
let status = std::process::Command::new("sh")
@@ -1468,6 +1402,7 @@ mod tests {
let _turn = KEYRING.lock();
*MOCK_STORE.lock() = None;
keyring_core::unset_default_store();
keyring().disconnect();
for refresh in ["r1", "r2", "r3"] {
write_stored(&stored_with(refresh)).unwrap();
+71 -34
View File
@@ -579,6 +579,7 @@ impl ContentCache {
// the disk.
cache.reseed_totals();
cache.import_legacy_pins()?;
cache.set_budget(max_bytes);
Ok(cache)
}
@@ -1153,16 +1154,18 @@ impl ContentCache {
};
let mut skipped = 0usize;
while running > cap {
// Fetch a batch past whatever we have already skipped as exempt.
let want = skipped + EVICT_BATCH;
let Ok(batch) = self.db.cache_eviction_candidates(kind, want) else {
// A whole batch can be pinned. Keep looking until the index has
// no entries past the ones we have skipped.
let Ok(batch) = self
.db
.cache_eviction_candidates(kind, EVICT_BATCH, skipped)
else {
break;
};
if batch.len() <= skipped {
break; // nothing new to consider
if batch.is_empty() {
break;
}
let mut evicted_any = false;
for (key, size) in batch.into_iter().skip(skipped) {
for (key, size) in batch {
if running <= cap {
break;
}
@@ -1170,14 +1173,18 @@ impl ContentCache {
skipped += 1; // counts toward the total, never a victim
continue;
}
let _ = std::fs::remove_file(dir.join(&key));
if let Err(error) = std::fs::remove_file(dir.join(&key))
&& error.kind() != std::io::ErrorKind::NotFound
{
skipped += 1;
continue;
}
let _ = std::fs::remove_file(dir.join(format!("{key}.meta")));
let _ = self.db.cache_remove(&key);
if self.db.cache_remove(&key).is_err() {
skipped += 1;
continue;
}
running = running.saturating_sub(size);
evicted_any = true;
}
if !evicted_any {
break; // every remaining candidate is exempt
}
}
// The index is the truth; realign the fast path with it.
@@ -1405,7 +1412,7 @@ impl ContentCache {
.collect();
let content_dir = self.content_dir.clone();
// Force pool enforcement ignoring current cap check to free disk space immediately
let Ok(list) = self.db.cache_eviction_candidates(KIND_BLOB, 10) else {
let Ok(list) = self.db.cache_eviction_candidates(KIND_BLOB, 10, 0) else {
return;
};
for (key, bytes) in list {
@@ -1883,13 +1890,13 @@ impl ContentCache {
// Pinned blobs (by cache key) are exempt from eviction. Resolves direct
// pins plus every descendant of a recursively-pinned folder, hashed into
// cache keys to match on-disk filenames.
let pinned: HashSet<String> = self
.db
.pinned_uids()
.unwrap_or_default()
.iter()
.map(|uid| Self::key_str(uid))
.collect();
let pinned = match self.db.pinned_uids() {
Ok(uids) => uids.iter().map(|uid| Self::key_str(uid)).collect(),
Err(error) => {
tracing::warn!(%error, "reading pins for cache eviction failed");
return;
}
};
let content_dir = self.content_dir.clone();
self.enforce_pool(KIND_BLOB, &content_dir, Some(&pinned));
}
@@ -2010,8 +2017,7 @@ impl ContentCache {
}
/// Retune the soft byte cap (`0` = unlimited) at runtime and immediately
/// enforce it: blobs and blocks are LRU-evicted back under the new cap (a
/// lower cap frees space now; a higher one is a no-op until the next store).
/// enforce it across all three pools.
/// Called by the daemon when the Settings page changes the cache budget.
pub fn set_budget(&self, bytes: u64) {
self.max_bytes.store(bytes, Ordering::Relaxed);
@@ -2245,19 +2251,50 @@ mod tests {
// crash) but the same on-disk cache: reconcile repopulates from disk.
let db2 = Arc::new(Db::open_in_memory().unwrap());
assert!(db2.cache_entries_by_kind("blob").unwrap().is_empty());
let c = ContentCache::open(content, pins, 8, db2.clone()).unwrap();
let c = ContentCache::open(content, pins, 0, db2.clone()).unwrap();
assert_eq!(db2.cache_entries_by_kind("blob").unwrap().len(), 3);
// With the index rebuilt, a further store enforces the budget against the
// reconciled entries instead of leaking them.
c.store(&uid("d"), 1, 4, b"dddd").unwrap();
let total: u64 = db2
.cache_entries_by_kind("blob")
.unwrap()
.iter()
.map(|(_, s)| s)
.sum();
assert!(total <= 8, "budget enforced after reconcile, total={total}");
assert_eq!(c.usage(), 12);
}
#[test]
fn reopening_trims_all_cache_pools_even_when_the_oldest_batch_is_pinned() {
let (c, _dir) = cache();
for i in 0..EVICT_BATCH {
let u = uid(&format!("pinned-{i}"));
c.store(&u, 1, 1, b"p").unwrap();
c.add_pin(&u, Path::new("pinned"), false).unwrap();
std::fs::File::open(c.blob_path(&u))
.unwrap()
.set_modified(UNIX_EPOCH)
.unwrap();
}
let unpinned = uid("unpinned");
let payload = [0u8; 64];
let size = payload.len() as u64;
c.store(&unpinned, 1, size, &payload).unwrap();
c.store_block(&unpinned, 1, size, blk(size, 0), &payload)
.unwrap();
c.store_thumbnail(&unpinned, 1, 1, &payload).unwrap();
let recovery = c.recovery_dir.join("recovered.bin");
std::fs::write(&recovery, b"recovered write").unwrap();
let budget = (EVICT_BATCH as u64 * 100).div_ceil(SPLIT_BLOB);
let reopened = ContentCache::open(
c.content_dir.clone(),
c.pins_path.clone(),
budget,
c.db.clone(),
)
.unwrap();
assert_eq!(reopened.usage(), EVICT_BATCH as u64);
assert!(!reopened.blob_path(&unpinned).exists());
for i in 0..EVICT_BATCH {
assert!(reopened.is_cached(&uid(&format!("pinned-{i}")), 1, 1));
}
assert_eq!(std::fs::read(recovery).unwrap(), b"recovered write");
}
/// An eviction removes the blob, then its tag. A crash in between, or a
+3 -29
View File
@@ -457,26 +457,7 @@ fn load_config_at(path: &Path) -> AppConfig {
/// [`AppDirs::save_config`] for the file at `path`.
fn save_config_at(path: &Path, config: &AppConfig) -> Result<()> {
refuse_to_replace_unreadable(path)?;
let parent = path
.parent()
.ok_or_else(|| Error::Other("configuration path has no parent".into()))?;
std::fs::create_dir_all(parent)?;
let nonce = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|duration| duration.as_nanos())
.unwrap_or(0);
let tmp = path.with_extension(format!("json.tmp-{}-{nonce}", std::process::id()));
use std::os::unix::fs::OpenOptionsExt as _;
let mut file = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.mode(0o600)
.open(&tmp)?;
use std::io::Write as _;
file.write_all(&serde_json::to_vec_pretty(config)?)?;
file.sync_all()?;
std::fs::rename(&tmp, path)?;
std::fs::File::open(parent)?.sync_all()?;
proton_base_rs::atomic::write(path, &serde_json::to_vec_pretty(config)?)?;
Ok(())
}
@@ -621,15 +602,8 @@ impl AppDirs {
use std::io::Read as _;
std::fs::File::open("/dev/urandom")?.read_exact(&mut bytes)?;
let id: String = bytes.iter().map(|b| format!("{b:02x}")).collect();
let parent = path
.parent()
.ok_or_else(|| Error::Other("machine id path has no parent".into()))?;
std::fs::create_dir_all(parent)?;
let tmp = path.with_extension(format!("tmp-{}", std::process::id()));
std::fs::write(&tmp, format!("{id}\n"))?;
std::fs::File::open(&tmp)?.sync_all()?;
std::fs::rename(&tmp, &path)?;
std::fs::File::open(parent)?.sync_all()?;
// Not secret, and other tools may read it; keep it world-readable.
proton_base_rs::atomic::write_with_mode(&path, format!("{id}\n").as_bytes(), 0o644)?;
Ok(id)
}
File diff suppressed because it is too large. Load diff
+129 -34
View File
@@ -31,8 +31,30 @@ pub struct StoredAlbum {
impl Db {
/// Replace the album listing wholesale, in the given order.
pub fn albums_replace(&self, albums: &[StoredAlbum]) -> Result<()> {
self.albums_replace_impl(albums, None).map(|_| ())
}
pub fn albums_replace_at(
&self,
albums: &[StoredAlbum],
generation: i64,
now: i64,
) -> Result<bool> {
self.albums_replace_impl(albums, Some((generation, now)))
}
fn albums_replace_impl(
&self,
albums: &[StoredAlbum],
stamp: Option<(i64, i64)>,
) -> Result<bool> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
if let Some((generation, _)) = stamp
&& super::photos::photos_generation(&tx)? != generation
{
return Ok(false);
}
// Contents of an album that no longer exists (unshared, deleted) go with
// it — otherwise they would linger forever, keyed by an album uid no
// listing mentions again.
@@ -56,7 +78,11 @@ impl Db {
let mut stmt = tx.prepare(
"INSERT INTO albums
(uid, name, photo_count, cover_uid, last_activity, shared, seq)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)",
SELECT ?1,?2,?3,
CASE WHEN EXISTS(SELECT 1 FROM photo_tombstones WHERE uid=?4)
THEN (SELECT uid FROM album_photos WHERE album_uid=?1 ORDER BY seq LIMIT 1) ELSE ?4 END,
?5,?6,?7
WHERE NOT EXISTS(SELECT 1 FROM photo_tombstones WHERE uid=?1)",
)?;
for (seq, album) in albums.iter().enumerate() {
stmt.execute(params![
@@ -70,8 +96,11 @@ impl Db {
])?;
}
}
if let Some((_, now)) = stamp {
super::photos::gallery_state(&tx, "albums_synced_ms", now)?;
}
tx.commit()?;
Ok(())
Ok(true)
}
/// The persisted album listing, in stored order.
@@ -111,15 +140,44 @@ impl Db {
album_uid: &str,
items: &[(String, i64, Option<String>, Option<String>)],
) -> Result<()> {
self.album_photos_replace_impl(album_uid, items, None)
.map(|_| ())
}
pub fn album_photos_replace_at(
&self,
album_uid: &str,
items: &[(String, i64, Option<String>, Option<String>)],
generation: i64,
now: i64,
) -> Result<bool> {
self.album_photos_replace_impl(album_uid, items, Some((generation, now)))
}
fn album_photos_replace_impl(
&self,
album_uid: &str,
items: &[(String, i64, Option<String>, Option<String>)],
stamp: Option<(i64, i64)>,
) -> Result<bool> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let learned: HashMap<String, (Option<f64>, i64, Option<String>)> = {
if let Some((generation, _)) = stamp
&& super::photos::photos_generation(&tx)? != generation
{
return Ok(false);
}
type StoredPhotoMetadata = (Option<f64>, i64, Option<String>, Option<String>);
let learned: HashMap<String, StoredPhotoMetadata> = {
let mut stmt = tx.prepare(
"SELECT uid, ratio, thumb_state, media_type FROM album_photos
"SELECT uid, ratio, thumb_state, media_type, name FROM album_photos
WHERE album_uid = ?1",
)?;
let rows = stmt.query_map([album_uid], |r| {
Ok((r.get::<_, String>(0)?, (r.get(1)?, r.get(2)?, r.get(3)?)))
Ok((
r.get::<_, String>(0)?,
(r.get(1)?, r.get(2)?, r.get(3)?, r.get(4)?),
))
})?;
rows.collect::<rusqlite::Result<_>>()?
};
@@ -128,15 +186,17 @@ impl Db {
{
let mut stmt = tx.prepare(
"INSERT INTO album_photos
(album_uid, uid, capture_time, name, media_type, kind, ratio, thumb_state, seq)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)",
(album_uid, uid, capture_time, name, media_type, kind, ratio, thumb_state, seq, resolved)
SELECT ?1,?2,?3,?4,?5,?6,?7,?8,?9,?10
WHERE NOT EXISTS(SELECT 1 FROM photo_tombstones WHERE uid IN (?1,?2))",
)?;
for (seq, (uid, capture_time, name, media_type)) in items.iter().enumerate() {
let (ratio, thumb_state, learned_media) =
learned
.get(uid)
.cloned()
.unwrap_or((None, THUMB_UNKNOWN, None));
let (ratio, thumb_state, learned_media, learned_name) = learned
.get(uid)
.cloned()
.unwrap_or((None, THUMB_UNKNOWN, None, None));
let resolved = name.is_some();
let name = name.clone().or(learned_name);
let media_type = media_type.clone().or(learned_media);
let kind =
crate::control::PhotoKind::classify(name.as_deref(), media_type.as_deref());
@@ -150,11 +210,32 @@ impl Db {
ratio,
thumb_state,
seq as i64,
resolved,
])?;
}
}
if let Some((_, now)) = stamp {
super::photos::gallery_state(&tx, &format!("album_synced_ms:{album_uid}"), now)?;
}
tx.commit()?;
Ok(())
Ok(true)
}
/// Reuse current timeline metadata and resolved album-only metadata.
pub fn album_photo_metadata(
&self,
album: &str,
) -> Result<HashMap<String, (String, Option<String>)>> {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT uid,name,media_type FROM album_photos
WHERE album_uid=?1 AND resolved=1 AND name IS NOT NULL
UNION ALL SELECT uid,name,media_type FROM photos
WHERE resolved_at IS NOT NULL AND name IS NOT NULL",
)?;
Ok(stmt
.query_map([album], |r| Ok((r.get(0)?, (r.get(1)?, r.get(2)?))))?
.collect::<rusqlite::Result<_>>()?)
}
/// One page of an album's photos, in stored order.
@@ -163,33 +244,47 @@ impl Db {
album_uid: &str,
offset: usize,
limit: usize,
) -> Result<Vec<StoredPhoto>> {
self.album_photos_page_visible(album_uid, offset, limit, "[]")
}
pub fn album_photos_page_visible(
&self,
album_uid: &str,
offset: usize,
limit: usize,
excluded: &str,
) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT uid, capture_time, name, ratio, thumb_state, kind FROM album_photos
WHERE album_uid = ?1 ORDER BY seq LIMIT ?2 OFFSET ?3",
WHERE album_uid = ?1 AND uid NOT IN (SELECT value FROM json_each(?4))
ORDER BY seq LIMIT ?2 OFFSET ?3",
)?;
let rows = stmt
.query_map(params![album_uid, limit as i64, offset as i64], |r| {
Ok(StoredPhoto {
uid: r.get(0)?,
capture_time: r.get(1)?,
name: r.get(2)?,
ratio: r.get(3)?,
thumb_state: r.get(4)?,
kind: crate::control::PhotoKind::from_i64(r.get(5)?),
// Album rows have no favorite state of their own: an album
// photo on someone else's volume is not in our timeline, and
// favouriting one is not supported yet anyway.
favorite: false,
// Nor are album rows grouped: an album is a list the user
// made, and it shows exactly what they put in it.
group_size: 1,
has_raw: false,
// Album rows carry no location.
place_id: None,
})
})?
.query_map(
params![album_uid, limit as i64, offset as i64, excluded],
|r| {
Ok(StoredPhoto {
uid: r.get(0)?,
capture_time: r.get(1)?,
name: r.get(2)?,
ratio: r.get(3)?,
thumb_state: r.get(4)?,
kind: crate::control::PhotoKind::from_i64(r.get(5)?),
// Album rows have no favorite state of their own: an album
// photo on someone else's volume is not in our timeline, and
// favouriting one is not supported yet anyway.
favorite: false,
// Nor are album rows grouped: an album is a list the user
// made, and it shows exactly what they put in it.
group_size: 1,
has_raw: false,
// Album rows carry no location.
place_id: None,
})
},
)?
.collect::<rusqlite::Result<_>>()?;
Ok(rows)
}
+5 -3
View File
@@ -108,7 +108,8 @@ impl Db {
Ok(total.max(0) as u64)
}
/// The `limit` least-recently-accessed entries of `kind`, oldest first.
/// The `limit` least-recently-accessed entries of `kind` after `offset`,
/// oldest first.
///
/// Eviction wants a handful of victims, not the whole table. Bounded so a
/// pass over a large cache does not read every row to drop a few, and so the
@@ -118,13 +119,14 @@ impl Db {
&self,
kind: &str,
limit: usize,
offset: usize,
) -> Result<Vec<(String, u64)>> {
let conn = self.read();
let mut stmt = conn.prepare_cached(
"SELECT cache_key, size_bytes FROM cache_entries
WHERE kind = ?1 ORDER BY last_accessed ASC LIMIT ?2",
WHERE kind = ?1 ORDER BY last_accessed ASC LIMIT ?2 OFFSET ?3",
)?;
let rows = stmt.query_map(params![kind, limit as i64], |r| {
let rows = stmt.query_map(params![kind, limit as i64, offset as i64], |r| {
Ok((r.get::<_, String>(0)?, r.get::<_, i64>(1)?.max(0) as u64))
})?;
let mut out = Vec::new();
+41 -1
View File
@@ -11,7 +11,7 @@ use super::Db;
use crate::Result;
/// Current schema version. Bump on every forward migration added below.
pub(super) const SCHEMA_VERSION: i64 = 45;
pub(super) const SCHEMA_VERSION: i64 = 47;
impl Db {
pub(super) fn migrate(&self) -> Result<()> {
@@ -490,6 +490,27 @@ impl Db {
}
}
}
if current < 46 {
tx.execute_batch(MIGRATION_V46)?;
let has_resolved: bool = tx.query_row(
"SELECT EXISTS(SELECT 1 FROM pragma_table_info('album_photos') WHERE name='resolved')",
[],
|row| row.get(0),
)?;
if !has_resolved {
tx.execute_batch(MIGRATION_V46_ALBUM_PHOTOS)?;
}
}
if current < 47 {
let has_trash: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'trash'",
[],
|row| row.get::<_, i64>(0).map(|n| n > 0),
)?;
if has_trash {
tx.execute_batch(MIGRATION_V47)?;
}
}
tx.execute(
"INSERT INTO sync_state (key, value) VALUES ('schema_version', ?1)
ON CONFLICT(key) DO UPDATE SET value = excluded.value",
@@ -1459,3 +1480,22 @@ BEGIN
UPDATE mount SET kind = NEW.kind WHERE sync_folder_id = NEW.id;
END;
";
/// Keep deletion evidence across listing races and remember unresolved album metadata.
const MIGRATION_V46: &str = "
CREATE TABLE IF NOT EXISTS photo_tombstones(uid TEXT PRIMARY KEY);
";
const MIGRATION_V46_ALBUM_PHOTOS: &str = "
ALTER TABLE album_photos ADD COLUMN resolved INTEGER NOT NULL DEFAULT 0;
UPDATE album_photos SET resolved=1 WHERE name IS NOT NULL;
";
/// Index the trash by parent, for browsing it by folder. The rows are dropped
/// so the next refresh reads them again with the time each was trashed, which
/// earlier builds did not get from the server and made up instead.
const MIGRATION_V47: &str = "
CREATE INDEX IF NOT EXISTS trash_parent ON trash(parent_uid);
DELETE FROM trash;
DELETE FROM sync_state WHERE key IN ('trash_synced_ms', 'trash_complete_ms');
";
+58 -10
View File
@@ -252,9 +252,32 @@ const UID_NOW: &str = "COALESCE((SELECT uid FROM nodes WHERE lid = pending_op.li
const PARENT_NOW: &str =
"COALESCE((SELECT uid FROM nodes WHERE lid = pending_op.parent_lid), parent_uid)";
/// Whether any queued move takes a node out of the folder `source.uid`.
/// A move updates the local node tree immediately, so its original parent
/// comes from the queue, including while a worker is sending it to Drive.
const MOVES_OUT_OF_FOLDER: &str = "
COALESCE((SELECT is_dir FROM nodes WHERE uid = source.uid), 1) = 1
AND EXISTS (
WITH RECURSIVE folders(uid, lid) AS (
SELECT source.uid, (SELECT lid FROM nodes WHERE uid = source.uid)
UNION
SELECT n.uid, n.lid FROM nodes n JOIN folders f
ON n.parent_lid = f.lid OR (n.parent_lid IS NULL AND n.parent_uid = f.uid)
WHERE n.is_dir = 1
)
SELECT 1 FROM pending_op AS move
WHERE move.kind = 'rename'
AND CASE WHEN json_valid(move.meta_json)
THEN json_extract(move.meta_json, '$.original_parent_uid') END
IN (SELECT uid FROM folders)
AND move.parent_uid IS NOT NULL
AND NOT EXISTS (SELECT 1 FROM folders f
WHERE f.lid = move.parent_lid OR f.uid = move.parent_uid)
)";
/// The `WHERE` terms for a row a drain worker may claim once it falls due: not
/// claimed already, no other claimed op on the same node, and not under a
/// folder that has yet to land.
/// claimed already, no other claimed op on the same node, no pending creation
/// of its parent, and no moves that still need a folder we are about to trash.
///
/// One fragment for [`Db::claim_next_due_op`] and [`Db::earliest_due_at`]. A
/// row the claim skips but the wait counts is due "now" for as long as it is
@@ -265,7 +288,9 @@ fn claimable() -> String {
AND (parent_uid IS NULL OR substr({PARENT_NOW}, 1, {n}) <> '{v}~')
AND uid NOT IN (SELECT uid FROM pending_op WHERE claimed_at <> 0)
AND (lid IS NULL OR lid NOT IN (
SELECT lid FROM pending_op WHERE claimed_at <> 0 AND lid IS NOT NULL))",
SELECT lid FROM pending_op WHERE claimed_at <> 0 AND lid IS NOT NULL))
AND (kind <> 'trash' OR NOT EXISTS (
SELECT 1 FROM (SELECT {UID_NOW} AS uid) AS source WHERE {MOVES_OUT_OF_FOLDER}))",
v = LOCAL_VOLUME,
n = LOCAL_VOLUME.len() + 1,
)
@@ -475,19 +500,40 @@ pub struct AttachedBlob {
}
impl Db {
/// Whether a queued trash still holds `name` somewhere.
/// A folder can look empty locally while Drive still holds the queued
/// moves' source files. A trash has to go through the drain in that case.
pub fn has_pending_moves_out_of(&self, uid: &str) -> Result<bool> {
let conn = self.read();
Ok(conn
.prepare_cached(&format!(
"SELECT EXISTS (SELECT 1 FROM (SELECT ?1 AS uid) AS source
WHERE {MOVES_OUT_OF_FOLDER})"
))?
.query_row(params![uid], |row| row.get(0))?)
}
/// Whether a queued trash still holds `name` in `parent_uid`.
///
/// A replacing rename or a delete-then-recreate queues the trash of the old
/// node ahead of the op that wants its name. With several drain workers the
/// later op can run first and find the name still taken; this tells it to
/// wait for the trash rather than land under a conflict name.
pub fn has_pending_trash_named(&self, name: &str) -> Result<bool> {
/// wait for the trash rather than land under a conflict name. Older rows
/// whose node and parent are unknown are matched by name alone.
pub fn has_pending_trash_named(&self, name: &str, parent_uid: Option<&str>) -> Result<bool> {
let conn = self.read();
conn.query_row(
"SELECT EXISTS(
SELECT 1 FROM pending_op WHERE kind = ?1 AND name = ?2
)",
params![OP_TRASH, name],
SELECT 1 FROM pending_op AS p
WHERE p.kind = ?1 AND p.name = ?2
AND (?3 IS NULL OR ?3 = COALESCE(
(SELECT uid FROM nodes WHERE lid = p.parent_lid),
p.parent_uid,
(SELECT COALESCE(parent.uid, n.parent_uid)
FROM nodes n LEFT JOIN nodes parent ON parent.lid = n.parent_lid
WHERE n.uid = p.uid),
?3))
)",
params![OP_TRASH, name, parent_uid],
|row| row.get(0),
)
.map_err(Into::into)
@@ -1219,7 +1265,7 @@ impl Db {
/// the claim is a column: a row is offered to exactly one worker, and the
/// worker that took it is the only one that may retire, defer or fail it.
///
/// Two exclusions on top of [`next_due_op`](Self::next_due_op)'s:
/// The selected row must also satisfy [`claimable`]:
///
/// * `claimed_at = 0` — not already in flight somewhere else.
/// * no *other* claimed op shares this `uid`. Ordering only has to hold per
@@ -1229,6 +1275,8 @@ impl Db {
/// pending entry. `ORDER BY id` plus this exclusion gives per-uid serial
/// order for free — the older op is claimed first and the newer one is
/// invisible until it retires.
/// * its destination folder already exists remotely.
/// * a folder trash waits for pending moves out of that folder's subtree.
///
/// Select and mark are one transaction, so two workers cannot both see the
/// row as unclaimed.
+400 -173
View File
@@ -1,10 +1,12 @@
//! The photos timeline: a flat, date-ordered projection of the photo share,
//! persisted so the gallery opens instantly instead of re-enumerating on launch.
use rusqlite::params;
use rusqlite::{Connection, OptionalExtension, Transaction, params};
use std::collections::{HashMap, HashSet};
use proton_gallery_rs::grouping::{Grouped, group_photos, name_stem};
use super::Db;
use crate::Result;
@@ -121,135 +123,6 @@ fn capture_day(conn: &rusqlite::Connection, capture_time: i64) -> Option<String>
.flatten()
}
/// The file name without its extension, lowercased — `IMG_1234.CR2` and
/// `IMG_1234.JPG` share `img_1234` — with the camera's own part-of-a-shot
/// marker removed, so the two files a raw capture writes share a stem too.
fn name_stem(name: Option<&str>) -> Option<String> {
let name = name?;
let stem = match name.rsplit_once('.') {
Some((stem, _)) if !stem.is_empty() => stem,
_ => name,
};
let stem = stem.to_ascii_lowercase();
Some(strip_shot_marker(&stem).to_string())
}
/// Drop a `.RAW-NN.ROLE` marker from a stem.
///
/// Pixel stores one raw capture as `PXL_20260919_000625670.RAW-01.COVER.jpg`
/// and `PXL_20260919_000625670.RAW-02.ORIGINAL.dng`: one shot, two files, two
/// stems that differ in exactly the part naming which file of the shot this is.
/// Cutting the marker off is what lets rule 2 see the pair. The check is narrow
/// on purpose — digits, then a single role word — so an ordinary name that
/// happens to contain `.raw-` keeps its stem.
fn strip_shot_marker(stem: &str) -> &str {
let Some(at) = stem.rfind(".raw-") else {
return stem;
};
let rest = &stem[at + ".raw-".len()..];
let Some((index, role)) = rest.split_once('.') else {
return stem;
};
let marked = !index.is_empty()
&& index.bytes().all(|b| b.is_ascii_digit())
&& !role.is_empty()
&& role.bytes().all(|b| b.is_ascii_alphabetic());
if marked { &stem[..at] } else { stem }
}
/// One entry of the grouping pass: what rule 2 needs, plus what choosing a
/// representative needs.
struct Grouped {
uid: String,
kind: crate::control::PhotoKind,
main_uid: Option<String>,
day: Option<String>,
stem: Option<String>,
}
/// Which group each photo belongs to, and which member of it the gallery shows.
///
/// Precedence, as agreed:
///
/// 1. The server's own relation — a photo with a `main_photo_uid` joins that
/// photo's group. The server knows about live photos and bursts; we do not
/// have to guess at them.
/// 2. Otherwise, same capture day **and** same file-name stem **and** one of the
/// two is a camera raw while the other is not. That is the RAW+JPEG pair a
/// camera writes, and the differing-kind clause keeps two JPEGs that happen
/// to share a name apart.
/// 3. Otherwise the photo is its own group.
///
/// The representative is the non-raw member when the group has one — a JPEG
/// decodes in milliseconds and is what the person expects to see — and the
/// earliest member in server order otherwise.
fn group_photos(entries: &[Grouped]) -> Vec<String> {
let mut parent: Vec<usize> = (0..entries.len()).collect();
fn find(parent: &mut [usize], mut i: usize) -> usize {
while parent[i] != i {
parent[i] = parent[parent[i]];
i = parent[i];
}
i
}
let union = |parent: &mut [usize], a: usize, b: usize| {
let (a, b) = (find(parent, a), find(parent, b));
if a != b {
parent[b] = a;
}
};
let index: HashMap<&str, usize> = entries
.iter()
.enumerate()
.map(|(i, e)| (e.uid.as_str(), i))
.collect();
for (i, entry) in entries.iter().enumerate() {
if let Some(main) = entry.main_uid.as_deref()
&& let Some(&j) = index.get(main)
{
union(&mut parent, j, i);
}
}
// Rule 2 needs only one pass: everything sharing a day and a stem lands in
// the same bucket, and the raw / non-raw split is checked pairwise there.
let mut by_name: HashMap<(&str, &str), Vec<usize>> = HashMap::new();
for (i, entry) in entries.iter().enumerate() {
if let (Some(day), Some(stem)) = (entry.day.as_deref(), entry.stem.as_deref()) {
by_name.entry((day, stem)).or_default().push(i);
}
}
let is_raw = |i: usize| entries[i].kind == crate::control::PhotoKind::Raw;
for bucket in by_name.values() {
for (n, &i) in bucket.iter().enumerate() {
for &j in &bucket[n + 1..] {
if is_raw(i) != is_raw(j) {
union(&mut parent, i, j);
}
}
}
}
// Representative per root, then the key every member carries.
let mut representative: HashMap<usize, usize> = HashMap::new();
for i in 0..entries.len() {
let root = find(&mut parent, i);
match representative.get(&root).copied() {
Some(current) if !is_raw(current) || is_raw(i) => {}
_ => {
representative.insert(root, i);
}
}
}
(0..entries.len())
.map(|i| {
let root = find(&mut parent, i);
entries[representative[&root]].uid.clone()
})
.collect()
}
/// The columns every photo read selects, with the group aggregates joined in.
/// `g.n` is how many photos the group holds and `g.raw` whether any of them is a
/// camera raw file — one query rather than a second lookup per tile.
@@ -278,15 +151,286 @@ fn stored_photo(r: &rusqlite::Row<'_>) -> rusqlite::Result<StoredPhoto> {
})
}
const PHOTOS_GENERATION: &str = "photos_projection_generation";
const PHOTOS_SYNCED: &str = "photos_synced_ms";
const PHOTOS_AVAILABLE: &str = "photos_available";
const PHOTOS_RECONCILE: &str = "photos_reconcile";
pub(super) fn photos_generation(conn: &Connection) -> rusqlite::Result<i64> {
conn.query_row(
"SELECT coalesce((SELECT CAST(value AS INTEGER) FROM sync_state WHERE key=?1),0)",
[PHOTOS_GENERATION],
|row| row.get(0),
)
}
pub(super) fn gallery_state(
conn: &Connection,
key: &str,
value: impl rusqlite::ToSql,
) -> rusqlite::Result<()> {
conn.execute(
"INSERT INTO sync_state(key,value) VALUES (?1,?2)
ON CONFLICT(key) DO UPDATE SET value=excluded.value",
params![key, value],
)?;
Ok(())
}
fn photos_delete_tx(tx: &Transaction<'_>, uids: &[String]) -> Result<usize> {
let mut removed = 0;
for uid in uids {
tx.execute(
"INSERT OR IGNORE INTO photo_tombstones(uid) VALUES (?1)",
[uid],
)?;
removed += tx.execute("DELETE FROM photos WHERE uid=?1", [uid])?;
tx.execute("DELETE FROM albums WHERE uid=?1", [uid])?;
tx.execute("DELETE FROM album_photos WHERE album_uid=?1", [uid])?;
tx.execute(
"UPDATE albums SET photo_count=max(photo_count-1,0)
WHERE cover_uid=?1 OR uid IN (SELECT album_uid FROM album_photos WHERE uid=?1)",
[uid],
)?;
tx.execute("DELETE FROM album_photos WHERE uid=?1", [uid])?;
tx.execute(
"UPDATE albums SET cover_uid=(SELECT uid FROM album_photos
WHERE album_uid=albums.uid ORDER BY seq LIMIT 1) WHERE cover_uid=?1",
[uid],
)?;
}
tx.execute(
"UPDATE photos SET group_key=NULL WHERE group_key IS NOT NULL
AND group_key NOT IN (SELECT uid FROM photos)",
[],
)?;
Ok(removed)
}
impl Db {
pub fn photo_projection_generation(&self) -> Result<i64> {
Ok(photos_generation(&self.read())?)
}
/// One transaction records the deletion, stale metadata, listing work and feed position.
pub fn photos_apply_events(
&self,
cursor_key: &str,
cursor: &str,
gone: &[String],
touched: &[String],
gap: bool,
seed: bool,
) -> Result<()> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
if !gone.is_empty() {
photos_delete_tx(&tx, gone)?;
}
for uid in touched {
tx.execute("DELETE FROM photo_tombstones WHERE uid=?1", [uid])?;
tx.execute("UPDATE photos SET resolved_at=NULL WHERE uid=?1", [uid])?;
tx.execute("UPDATE album_photos SET resolved=0 WHERE uid=?1", [uid])?;
}
if gap {
tx.execute("UPDATE photos SET resolved_at=NULL", [])?;
tx.execute("UPDATE album_photos SET resolved=0", [])?;
gallery_state(&tx, PHOTOS_RECONCILE, 1)?;
}
if seed || gap || !gone.is_empty() || !touched.is_empty() {
let generation = photos_generation(&tx)?;
gallery_state(&tx, PHOTOS_GENERATION, generation + 1)?;
tx.execute(
"DELETE FROM sync_state WHERE key='albums_synced_ms'
OR key GLOB 'album_synced_ms:*'",
[],
)?;
}
if seed || gap || !touched.is_empty() {
tx.execute("DELETE FROM sync_state WHERE key=?1", [PHOTOS_SYNCED])?;
}
gallery_state(&tx, cursor_key, cursor)?;
tx.commit()?;
Ok(())
}
pub fn photos_invalidate(&self) -> Result<()> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let generation = photos_generation(&tx)?;
gallery_state(&tx, PHOTOS_GENERATION, generation + 1)?;
tx.execute("DELETE FROM sync_state WHERE key=?1", [PHOTOS_SYNCED])?;
tx.commit()?;
Ok(())
}
pub fn photos_need_work(&self) -> Result<bool> {
let conn = self.read();
Ok(conn.query_row(
"SELECT NOT EXISTS(SELECT 1 FROM sync_state WHERE key=?1)
OR EXISTS(SELECT 1 FROM photos WHERE resolved_at IS NULL)",
[PHOTOS_SYNCED],
|r| r.get(0),
)?)
}
pub fn photos_pending(&self) -> Result<Vec<String>> {
let conn = self.read();
let mut stmt =
conn.prepare("SELECT uid FROM photos WHERE resolved_at IS NULL ORDER BY seq")?;
Ok(stmt
.query_map([], |r| r.get(0))?
.collect::<rusqlite::Result<_>>()?)
}
pub fn photo_removed(&self, uid: &str) -> Result<bool> {
Ok(self.read().query_row(
"SELECT EXISTS(SELECT 1 FROM photo_tombstones WHERE uid=?1)",
[uid],
|r| r.get(0),
)?)
}
pub fn photos_contains(&self, uid: &str, album: Option<&str>) -> Result<bool> {
let conn = self.read();
Ok(match album {
Some(album) => conn.query_row(
"SELECT EXISTS(SELECT 1 FROM album_photos WHERE album_uid=?1 AND uid=?2)",
params![album, uid],
|r| r.get(0),
)?,
None => conn.query_row(
"SELECT EXISTS(SELECT 1 FROM photos WHERE uid=?1)
OR EXISTS(SELECT 1 FROM album_photos WHERE uid=?1)",
[uid],
|r| r.get(0),
)?,
})
}
/// Persist one enrichment batch. The generation check and UPDATE share the
/// writer transaction with deletion, so a stale batch cannot restore a row.
pub fn photos_resolve_at(&self, generation: i64, rows: &[TimelineRow]) -> Result<bool> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
if photos_generation(&tx)? != generation {
return Ok(false);
}
{
let mut update = tx.prepare(
"UPDATE photos SET name=?2, media_type=?3, kind=?4, favorite=?5,
similar_hash=CASE WHEN content_hash IS ?6 THEN similar_hash ELSE NULL END,
content_hash=?6, main_uid=?7, latitude=?8, longitude=?9, place_id=?10,
resolved_at=?11 WHERE uid=?1",
)?;
for row in rows.iter().filter(|row| row.resolved_at.is_some()) {
let kind = crate::control::PhotoKind::classify(
row.name.as_deref(),
row.media_type.as_deref(),
);
update.execute(params![
row.uid,
row.name,
row.media_type,
kind.as_i64(),
row.favorite.unwrap_or(false),
row.content_hash,
row.main_uid,
row.location.map(|p| p.0),
row.location.map(|p| p.1),
row.location
.and_then(|(lat, lon)| crate::places::nearest(lat, lon))
.map(|city| city.id),
row.resolved_at
])?;
}
}
if !rows.is_empty() {
let (entries, existing): (Vec<_>, Vec<Option<String>>) = {
let mut stmt = tx.prepare(
"SELECT uid,kind,main_uid,strftime('%Y-%m-%d',capture_time,'unixepoch','localtime'),
name,group_key FROM photos ORDER BY seq",
)?;
stmt.query_map([], |r| {
let name: Option<String> = r.get(4)?;
Ok((
Grouped {
uid: r.get(0)?,
raw: crate::control::PhotoKind::from_i64(r.get(1)?)
== crate::control::PhotoKind::Raw,
main_uid: r.get(2)?,
day: r.get(3)?,
stem: name_stem(name.as_deref()),
},
r.get(5)?,
))
})?
.collect::<rusqlite::Result<Vec<_>>>()?
.into_iter()
.unzip()
};
let mut update = tx.prepare("UPDATE photos SET group_key=?2 WHERE uid=?1")?;
for ((entry, old), group) in entries.iter().zip(existing).zip(group_photos(&entries)) {
if old.as_deref() != Some(group.as_str()) {
update.execute(params![entry.uid, group])?;
}
}
}
tx.commit()?;
Ok(true)
}
/// Replace the timeline wholesale. A row whose `resolved_at` is `Some` was
/// read from the server in this pass, so its fields are taken as they are —
/// a `None` there means the server no longer reports that value. A row whose
/// `resolved_at` is `None` was skipped or failed to resolve, and keeps
/// whatever is already stored rather than having it silently cleared.
pub fn photos_replace(&self, items: &[TimelineRow]) -> Result<()> {
self.photos_replace_impl(items, None).map(|_| ())
}
pub fn photos_replace_at(
&self,
items: &[TimelineRow],
generation: i64,
available: bool,
now: i64,
) -> Result<bool> {
self.photos_replace_impl(items, Some((generation, available, now)))
}
fn photos_replace_impl(
&self,
items: &[TimelineRow],
stamp: Option<(i64, bool, i64)>,
) -> Result<bool> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let mut tombstones = HashSet::new();
if let Some((generation, _, _)) = stamp {
if photos_generation(&tx)? != generation {
return Ok(false);
}
let repairing = tx
.query_row(
"SELECT 1 FROM sync_state WHERE key=?1",
[PHOTOS_RECONCILE],
|_| Ok(()),
)
.optional()?
.is_some();
if repairing {
let mut remove = tx.prepare("DELETE FROM photo_tombstones WHERE uid=?1")?;
for row in items {
remove.execute([&row.uid])?;
}
tx.execute("DELETE FROM sync_state WHERE key=?1", [PHOTOS_RECONCILE])?;
}
let mut stmt = tx.prepare("SELECT uid FROM photo_tombstones")?;
tombstones = stmt
.query_map([], |r| r.get::<_, String>(0))?
.collect::<rusqlite::Result<_>>()?;
}
// `name` and `media_type` are learned-and-kept like the ratio and thumb
// verdict: the timeline DTO carries only the uid and capture time, so the
// daemon may not know them when it replaces the timeline — because the
@@ -325,6 +469,7 @@ impl Db {
// Everything the insert and the grouping need, resolved once per photo.
let resolved: Vec<(&TimelineRow, Learned, crate::control::PhotoKind)> = items
.iter()
.filter(|row| !tombstones.contains(&row.uid))
.map(|row| {
let mut carried = learned.get(&row.uid).cloned().unwrap_or(Learned {
thumb_state: THUMB_UNKNOWN,
@@ -369,7 +514,7 @@ impl Db {
.iter()
.map(|(row, carried, kind)| Grouped {
uid: row.uid.clone(),
kind: *kind,
raw: *kind == crate::control::PhotoKind::Raw,
main_uid: carried.main_uid.clone(),
day: capture_day(&tx, row.capture_time),
stem: name_stem(carried.name.as_deref()),
@@ -414,8 +559,12 @@ impl Db {
])?;
}
}
if let Some((_, available, now)) = stamp {
gallery_state(&tx, PHOTOS_AVAILABLE, available as i64)?;
gallery_state(&tx, PHOTOS_SYNCED, now)?;
}
tx.commit()?;
Ok(())
Ok(true)
}
/// The uids whose node metadata is already resolved, so a refresh can skip
@@ -458,8 +607,16 @@ impl Db {
/// refresh. For a continuity loss, where the daemon cannot know what it
/// missed, and for an explicit `refresh photos --full`.
pub fn photos_mark_all_unresolved(&self) -> Result<usize> {
let conn = self.conn.lock();
Ok(conn.execute("UPDATE photos SET resolved_at = NULL", [])?)
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let marked = tx.execute("UPDATE photos SET resolved_at=NULL", [])?;
tx.execute("UPDATE album_photos SET resolved=0", [])?;
let generation = photos_generation(&tx)?;
gallery_state(&tx, PHOTOS_GENERATION, generation + 1)?;
gallery_state(&tx, PHOTOS_RECONCILE, 1)?;
tx.execute("DELETE FROM sync_state WHERE key=?1", [PHOTOS_SYNCED])?;
tx.commit()?;
Ok(marked)
}
/// Forget photos that have been trashed, so the gallery does not show them
@@ -474,23 +631,9 @@ impl Db {
}
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let mut removed = 0;
{
let mut photos = tx.prepare("DELETE FROM photos WHERE uid = ?1")?;
let mut albums = tx.prepare("DELETE FROM album_photos WHERE uid = ?1")?;
for uid in uids {
removed += photos.execute([uid])?;
albums.execute([uid])?;
}
}
// A group whose representative went is left pointing at nothing, which
// drops its other files off every page. Make them stand alone until the
// next refresh groups them again.
tx.execute(
"UPDATE photos SET group_key = NULL WHERE group_key IS NOT NULL \
AND group_key NOT IN (SELECT uid FROM photos)",
[],
)?;
let removed = photos_delete_tx(&tx, uids)?;
let generation = photos_generation(&tx)?;
gallery_state(&tx, PHOTOS_GENERATION, generation + 1)?;
tx.commit()?;
Ok(removed)
}
@@ -513,24 +656,42 @@ impl Db {
range: Option<(i64, i64)>,
favorites: bool,
not_in_album: bool,
) -> Result<Vec<StoredPhoto>> {
self.photos_page_visible(offset, limit, kind, range, favorites, not_in_album, "[]")
}
#[allow(clippy::too_many_arguments)]
pub fn photos_page_visible(
&self,
offset: usize,
limit: usize,
kind: Option<crate::control::PhotoKind>,
range: Option<(i64, i64)>,
favorites: bool,
not_in_album: bool,
excluded: &str,
) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
// Built up so any combination of the optional filters is one indexed
// query rather than a statement per case.
let mut sql = String::from(PHOTO_SELECT);
let mut binds: Vec<i64> = Vec::new();
let mut binds: Vec<rusqlite::types::Value> = Vec::new();
let mut conds: Vec<String> = Vec::new();
if excluded != "[]" {
binds.push(excluded.to_owned().into());
conds.push("p.uid NOT IN (SELECT value FROM json_each(?1))".to_owned());
}
if kind != Some(crate::control::PhotoKind::Raw) {
conds.push("p.uid = COALESCE(p.group_key, p.uid)".to_string());
}
if let Some(k) = kind {
binds.push(k.as_i64());
binds.push((k.as_i64()).into());
conds.push(format!("p.kind = ?{}", binds.len()));
}
if let Some((from, to)) = range {
binds.push(from);
binds.push((from).into());
conds.push(format!("p.capture_time >= ?{}", binds.len()));
binds.push(to);
binds.push((to).into());
conds.push(format!("p.capture_time < ?{}", binds.len()));
}
if favorites {
@@ -549,9 +710,9 @@ impl Db {
sql.push_str(" WHERE ");
sql.push_str(&conds.join(" AND "));
}
binds.push(limit as i64);
binds.push((limit as i64).into());
let limit_pos = binds.len();
binds.push(offset as i64);
binds.push((offset as i64).into());
let offset_pos = binds.len();
sql.push_str(&format!(
" ORDER BY p.seq LIMIT ?{limit_pos} OFFSET ?{offset_pos}"
@@ -580,6 +741,25 @@ impl Db {
Ok(rows)
}
/// Include associated files so a public preview cannot reveal a hidden
/// RAW, JPEG or Live Photo through another part of the same shot.
pub fn photo_group_members_for(&self, uids: &HashSet<String>) -> Result<HashSet<String>> {
if uids.is_empty() {
return Ok(HashSet::new());
}
let ids = serde_json::to_string(uids).expect("file identifiers serialize");
let conn = self.read();
let mut query = conn.prepare(
"SELECT uid FROM photos WHERE COALESCE(group_key, uid) IN (
SELECT COALESCE(group_key, uid) FROM photos
WHERE uid IN (SELECT value FROM json_each(?1))
)",
)?;
Ok(query
.query_map([ids], |row| row.get(0))?
.collect::<rusqlite::Result<_>>()?)
}
/// The whole persisted timeline, in stored order — every file, not one row
/// per group. Only for whole-library passes (the photo re-date); the gallery
/// pages with [`Db::photos_page`].
@@ -601,27 +781,38 @@ impl Db {
pub fn photos_months(
&self,
kind: Option<crate::control::PhotoKind>,
) -> Result<Vec<(i32, i32, usize)>> {
self.photos_months_visible(kind, "[]")
}
pub fn photos_months_visible(
&self,
kind: Option<crate::control::PhotoKind>,
excluded: &str,
) -> Result<Vec<(i32, i32, usize)>> {
let conn = self.read();
// The scrubber has to agree with the page it scrubs, so it counts what
// the page shows: groups everywhere except on the Raw tab.
let (filter, binds): (&str, Vec<i64>) = match kind {
Some(crate::control::PhotoKind::Raw) => (" WHERE kind = 2", Vec::new()),
Some(k) => (
" WHERE uid = COALESCE(group_key, uid) AND kind = ?1",
vec![k.as_i64()],
),
None => (" WHERE uid = COALESCE(group_key, uid)", Vec::new()),
let filter = match kind {
Some(crate::control::PhotoKind::Raw) => "kind = ?2",
Some(_) => "uid = COALESCE(group_key, uid) AND kind = ?2",
None => "uid = COALESCE(group_key, uid) AND ?2 IS NULL",
};
let sql = format!(
"SELECT CAST(strftime('%Y', capture_time, 'unixepoch', 'localtime') AS INTEGER) AS y, \
"WITH hidden_groups AS (
SELECT COALESCE(group_key, uid) FROM photos
WHERE uid IN (SELECT value FROM json_each(?1))
)
SELECT CAST(strftime('%Y', capture_time, 'unixepoch', 'localtime') AS INTEGER) AS y, \
CAST(strftime('%m', capture_time, 'unixepoch', 'localtime') AS INTEGER) AS m, \
COUNT(*) \
FROM photos{filter} GROUP BY y, m ORDER BY y DESC, m DESC"
FROM photos WHERE {filter}
AND COALESCE(group_key, uid) NOT IN (SELECT * FROM hidden_groups)
GROUP BY y, m ORDER BY y DESC, m DESC"
);
let mut stmt = conn.prepare(&sql)?;
let rows = stmt
.query_map(rusqlite::params_from_iter(binds), |r| {
.query_map(params![excluded, kind.map(|kind| kind.as_i64())], |r| {
Ok((
r.get::<_, i32>(0)?,
r.get::<_, i32>(1)?,
@@ -637,6 +828,15 @@ impl Db {
/// pick the date, and its own year is left out, because today's photos are
/// what the timeline already opens on. One row per group, like the timeline.
pub fn photos_on_this_day(&self, today: i64, limit: usize) -> Result<Vec<StoredPhoto>> {
self.photos_on_this_day_visible(today, limit, "[]")
}
pub fn photos_on_this_day_visible(
&self,
today: i64,
limit: usize,
excluded: &str,
) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
let sql = format!(
"{PHOTO_SELECT} WHERE p.uid = COALESCE(p.group_key, p.uid) \
@@ -644,11 +844,12 @@ impl Db {
= strftime('%m-%d', ?1, 'unixepoch', 'localtime') \
AND strftime('%Y', p.capture_time, 'unixepoch', 'localtime') \
< strftime('%Y', ?1, 'unixepoch', 'localtime') \
AND p.uid NOT IN (SELECT value FROM json_each(?3))
ORDER BY p.seq LIMIT ?2"
);
let mut stmt = conn.prepare(&sql)?;
let rows = stmt
.query_map(params![today, limit as i64], stored_photo)?
.query_map(params![today, limit as i64, excluded], stored_photo)?
.collect::<rusqlite::Result<_>>()?;
Ok(rows)
}
@@ -657,6 +858,10 @@ impl Db {
/// cover, most photographed first. Counted per group, like the timeline;
/// photos without a location, or taken far from any town, are in none.
pub fn photo_places(&self) -> Result<Vec<StoredPlace>> {
self.photo_places_visible("[]")
}
pub fn photo_places_visible(&self, excluded: &str) -> Result<Vec<StoredPlace>> {
let rows: Vec<(u32, usize, String)> = {
let conn = self.read();
// SQLite fills the bare `uid` from the row `MIN(seq)` picked, which
@@ -664,9 +869,10 @@ impl Db {
let mut stmt = conn.prepare(
"SELECT place_id, COUNT(*), uid, MIN(seq) FROM photos \
WHERE uid = COALESCE(group_key, uid) AND place_id IS NOT NULL \
AND uid NOT IN (SELECT value FROM json_each(?1))
GROUP BY place_id ORDER BY COUNT(*) DESC, MIN(seq)",
)?;
stmt.query_map([], |r| {
stmt.query_map([excluded], |r| {
Ok((r.get(0)?, r.get::<_, i64>(1)? as usize, r.get(2)?))
})?
.collect::<rusqlite::Result<_>>()?
@@ -695,15 +901,29 @@ impl Db {
id: u32,
offset: usize,
limit: usize,
) -> Result<Vec<StoredPhoto>> {
self.photos_at_place_visible(id, offset, limit, "[]")
}
pub fn photos_at_place_visible(
&self,
id: u32,
offset: usize,
limit: usize,
excluded: &str,
) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
let sql = format!(
"{PHOTO_SELECT} WHERE p.uid = COALESCE(p.group_key, p.uid) AND p.place_id = ?1 \
"{PHOTO_SELECT} WHERE p.uid = COALESCE(p.group_key, p.uid) AND p.place_id = ?1
AND p.uid NOT IN (SELECT value FROM json_each(?4))
ORDER BY p.seq LIMIT ?2 OFFSET ?3"
);
let mut stmt = conn.prepare(&sql)?;
let rows = stmt
.query_map(params![id, limit as i64, offset as i64], stored_photo)?
.query_map(
params![id, limit as i64, offset as i64, excluded],
stored_photo,
)?
.collect::<rusqlite::Result<_>>()?;
Ok(rows)
}
@@ -884,6 +1104,10 @@ impl Db {
/// Per-tab counts for the Photos page subtitle: `(photos, videos, raw)`.
pub fn photos_counts(&self) -> Result<(usize, usize, usize)> {
self.photos_counts_visible("[]")
}
pub fn photos_counts_visible(&self, excluded: &str) -> Result<(usize, usize, usize)> {
use crate::control::PhotoKind;
let conn = self.read();
// Photos and Videos count groups — one shot is one entry, however many
@@ -891,9 +1115,12 @@ impl Db {
// them individually.
let mut stmt = conn.prepare(
"SELECT kind, COUNT(*) FROM photos \
WHERE uid = COALESCE(group_key, uid) OR kind = 2 GROUP BY kind",
WHERE (uid = COALESCE(group_key, uid) OR kind = 2)
AND uid NOT IN (SELECT value FROM json_each(?1)) GROUP BY kind",
)?;
let rows = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?)))?;
let rows = stmt.query_map([excluded], |r| {
Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?))
})?;
let (mut photos, mut videos, mut raw) = (0usize, 0usize, 0usize);
for row in rows {
let (kind, n) = row?;
+231 -5
View File
@@ -337,6 +337,10 @@ fn places_group_photos_by_the_nearest_town() {
.map(|p| p.uid)
.collect();
assert_eq!(berlin, ["b2", "b1-jpeg"]);
let visible = db.photo_places_visible(r#"["b2","p1"]"#).unwrap();
assert_eq!(visible.len(), 1);
assert_eq!(visible[0].count, 1);
assert_eq!(visible[0].cover.uid, "b1-jpeg");
}
/// A town's photos come with where each was taken, one per RAW+JPEG shot.
@@ -646,6 +650,54 @@ fn trash_replace_lists_folders_first() {
assert!(db.trash_list().unwrap().is_empty());
}
fn stored_trash(uid: &str, name: &str, trashed_at: Option<i64>) -> StoredTrash {
StoredTrash {
uid: uid.into(),
name: name.into(),
is_dir: false,
size: 1,
mtime: 1,
parent_uid: None,
trashed_at,
}
}
#[test]
fn trash_upsert_keeps_other_rows_and_retain_drops_unlisted_ones() {
let db = Db::open_in_memory().unwrap();
db.trash_upsert(&[
stored_trash("a", "a.txt", Some(1)),
stored_trash("b", "b.txt", None),
])
.unwrap();
// A later refresh adds what is new without touching what it kept.
db.trash_upsert(&[
stored_trash("c", "c.txt", Some(3)),
stored_trash("b", "b2.txt", Some(2)),
])
.unwrap();
let mut names: Vec<String> = db
.trash_list()
.unwrap()
.into_iter()
.map(|i| i.name)
.collect();
names.sort();
assert_eq!(names, ["a.txt", "b2.txt", "c.txt"]);
// Rows the server no longer lists go; the listed ones stay.
let keep: std::collections::HashSet<String> = ["a", "c"].map(String::from).into();
assert_eq!(db.trash_retain(&keep).unwrap(), 1);
assert_eq!(db.trash_remove(&["a".into(), "gone".into()]).unwrap(), 1);
let left: Vec<String> = db
.trash_list()
.unwrap()
.into_iter()
.map(|i| i.uid)
.collect();
assert_eq!(left, ["c"]);
}
#[test]
fn state_stamps_round_trip_and_clear() {
let db = Db::open_in_memory().unwrap();
@@ -2740,15 +2792,34 @@ fn atomic_trash_replacement_returns_owned_blobs_after_commit() {
#[test]
fn a_queued_trash_holds_its_name_until_it_lands() {
let db = Db::open_in_memory().unwrap();
db.upsert_node(&file("held-name", "root", "report.txt", 1))
.unwrap();
let node = uid("held-name").to_string();
assert!(!db.has_pending_trash_named("report.txt").unwrap());
let root = uid("root").to_string();
assert!(
!db.has_pending_trash_named("report.txt", Some(&root))
.unwrap()
);
let (trash_id, _) = db.replace_ops_with_trash(&node, "report.txt", 1).unwrap();
assert!(db.has_pending_trash_named("report.txt").unwrap());
assert!(!db.has_pending_trash_named("other.txt").unwrap());
assert!(
db.has_pending_trash_named("report.txt", Some(&root))
.unwrap()
);
assert!(
!db.has_pending_trash_named("other.txt", Some(&root))
.unwrap()
);
assert!(
!db.has_pending_trash_named("report.txt", Some(&uid("other").to_string()))
.unwrap()
);
db.delete_op(trash_id).unwrap();
assert!(!db.has_pending_trash_named("report.txt").unwrap());
assert!(
!db.has_pending_trash_named("report.txt", Some(&root))
.unwrap()
);
}
/// A create that found its name held by a queued trash backed off. The trash
@@ -6094,7 +6165,7 @@ fn cache_eviction_candidates_are_lru_ordered_and_limited() {
}
db.cache_touch("other", "block", 100, 0).unwrap();
let batch = db.cache_eviction_candidates("blob", 3).unwrap();
let batch = db.cache_eviction_candidates("blob", 3, 0).unwrap();
assert_eq!(batch.len(), 3, "honours the limit");
let keys: Vec<&str> = batch.iter().map(|(k, _)| k.as_str()).collect();
assert_eq!(keys, ["k9", "k8", "k7"], "least-recently-accessed first");
@@ -6925,6 +6996,54 @@ fn two_workers_never_share_a_node() {
);
}
#[test]
fn trashing_a_folder_waits_for_moves_out_of_its_subtree() {
let db = Db::open_in_memory().unwrap();
db.upsert_nodes(&[
folder("root", None, "My Files"),
folder("source", Some("root"), "source"),
folder("nested", Some("source"), "nested"),
folder("destination", Some("root"), "destination"),
file("photo", "destination", "photo.jpg", 1),
])
.unwrap();
let source = uid("source").to_string();
let destination = uid("destination").to_string();
let (move_id, _) = db
.enqueue_op(&PendingOp {
kind: OP_RENAME.to_string(),
uid: uid("photo").to_string(),
parent_uid: Some(destination.clone()),
name: Some("photo.jpg".into()),
blob_path: None,
meta_json: Some(
serde_json::to_string(&RenameMeta {
original_parent_uid: uid("nested").to_string(),
original_name: Some("photo.jpg".into()),
})
.unwrap(),
),
next_attempt_at: 20,
..bulk_op(0, &destination)
})
.unwrap();
assert!(db.has_pending_moves_out_of(&source).unwrap());
assert!(!db.has_pending_moves_out_of(&destination).unwrap());
assert!(
!db.has_pending_moves_out_of(&uid("root").to_string())
.unwrap()
);
let (trash_id, _) = db.replace_ops_with_trash(&source, "source", 2).unwrap();
assert!(db.claim_next_due_op(10).unwrap().is_none());
assert_eq!(db.earliest_due_at().unwrap(), Some(20));
assert_eq!(db.claim_next_due_op(20).unwrap().unwrap().id, move_id);
assert!(db.claim_next_due_op(20).unwrap().is_none());
assert_eq!(db.earliest_due_at().unwrap(), None);
db.delete_op(move_id).unwrap();
assert_eq!(db.claim_next_due_op(20).unwrap().unwrap().id, trash_id);
}
#[test]
fn claims_do_not_survive_the_process_that_took_them() {
let db = Db::open_in_memory().unwrap();
@@ -7098,6 +7217,29 @@ fn a_raw_and_its_jpeg_are_one_entry_in_the_grid() {
["jpeg", "raw"]
);
assert_eq!(db.photos_group("alone").unwrap().len(), 1);
let hidden = db
.photo_group_members_for(&["raw".to_owned()].into_iter().collect())
.unwrap();
assert_eq!(
hidden,
["raw".to_owned(), "jpeg".to_owned()].into_iter().collect()
);
let excluded = serde_json::to_string(&hidden).unwrap();
assert_eq!(db.photos_counts_visible(&excluded).unwrap(), (1, 0, 0));
let months = db.photos_months_visible(None, &excluded).unwrap();
assert_eq!(months.len(), 1);
assert_eq!(months[0].2, 1);
let public = db
.photos_page_visible(0, 1, None, None, false, false, &excluded)
.unwrap();
assert_eq!(
public
.iter()
.map(|photo| photo.uid.as_str())
.collect::<Vec<_>>(),
["alone"]
);
}
/// Pixel names the two files of one raw capture after the role each plays, so
@@ -7778,3 +7920,87 @@ fn migration_v45_keeps_existing_device_folders_and_location_ids() {
db.migrate().unwrap();
assert_eq!(db.mount_list().unwrap().len(), 2);
}
#[test]
fn photo_feed_failure_keeps_the_cursor_and_a_stale_listing_cannot_restore_deleted_rows() {
let db = Db::open_in_memory().unwrap();
let rows = vec![TimelineRow::new("gone", 200), TimelineRow::new("keep", 100)];
db.photos_replace(&rows).unwrap();
db.albums_replace(&[StoredAlbum {
uid: "album".into(),
name: "Test album".into(),
photo_count: 2,
cover_uid: Some("gone".into()),
last_activity: None,
shared: false,
}])
.unwrap();
db.album_photos_replace(
"album",
&[
("gone".into(), 200, None, None),
("keep".into(), 100, None, None),
],
)
.unwrap();
db.set_state_str("photos-cursor-test", "old").unwrap();
let stale = db.photo_projection_generation().unwrap();
db.conn.lock().execute_batch(
"CREATE TEMP TRIGGER reject_cursor BEFORE INSERT ON sync_state WHEN NEW.key='photos-cursor-test'
BEGIN SELECT RAISE(ABORT, 'simulated cursor failure'); END;",
).unwrap();
let gone = ["gone".into()];
assert!(
db.photos_apply_events("photos-cursor-test", "next", &gone, &[], false, false)
.is_err()
);
assert_eq!(
db.state_str("photos-cursor-test").unwrap().as_deref(),
Some("old")
);
assert_eq!(db.photos_count().unwrap(), 2);
assert!(db.photos_contains("gone", Some("album")).unwrap());
db.conn
.lock()
.execute("DROP TRIGGER reject_cursor", [])
.unwrap();
db.photos_apply_events("photos-cursor-test", "next", &gone, &[], false, false)
.unwrap();
assert_eq!(
db.state_str("photos-cursor-test").unwrap().as_deref(),
Some("next")
);
assert!(!db.photos_contains("gone", None).unwrap());
assert_eq!(
db.albums_list().unwrap()[0].cover_uid.as_deref(),
Some("keep")
);
assert!(!db.photos_replace_at(&rows, stale, true, 1).unwrap());
assert!(
!db.album_photos_replace_at("album", &[("gone".into(), 200, None, None)], stale, 1)
.unwrap()
);
assert!(!db.photos_contains("gone", None).unwrap());
db.photos_apply_events("photos-cursor-test", "restore", &[], &gone, false, false)
.unwrap();
let current = db.photo_projection_generation().unwrap();
assert!(db.photos_replace_at(&rows, current, true, 2).unwrap());
assert!(db.photos_contains("gone", None).unwrap());
}
#[test]
fn warm_photo_months_exclude_hidden_groups_with_thirteen_thousand_rows() {
let db = Db::open_in_memory().unwrap();
let mut rows: Vec<_> = (0..13_000)
.map(|i| TimelineRow::new(format!("volume~{i}"), 1_700_000_000 + i))
.collect();
rows[1].main_uid = Some(rows[0].uid.clone());
db.photos_replace(&rows).unwrap();
let started = std::time::Instant::now();
let months = db.photos_months_visible(None, r#"["volume~1"]"#).unwrap();
assert_eq!(months.iter().map(|month| month.2).sum::<usize>(), 12_998);
assert!(started.elapsed() < std::time::Duration::from_secs(1));
}
+55 -1
View File
@@ -1,6 +1,6 @@
//! Trashed nodes, persisted so the trash view opens without a round-trip.
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use rusqlite::params;
@@ -48,6 +48,60 @@ impl Db {
Ok(())
}
/// Add or update trashed nodes, keeping every other row.
pub fn trash_upsert(&self, items: &[StoredTrash]) -> Result<()> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
{
let mut stmt = tx.prepare(
"INSERT INTO trash (uid, name, is_dir, size, mtime, parent_uid, trashed_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
ON CONFLICT(uid) DO UPDATE SET
name = excluded.name, is_dir = excluded.is_dir, size = excluded.size,
mtime = excluded.mtime, parent_uid = excluded.parent_uid,
trashed_at = excluded.trashed_at",
)?;
for item in items {
stmt.execute(params![
item.uid,
item.name,
item.is_dir as i64,
item.size,
item.mtime,
item.parent_uid,
item.trashed_at
])?;
}
}
tx.commit()?;
Ok(())
}
/// Drop the trashed nodes with these uids. Returns how many rows went.
pub fn trash_remove(&self, uids: &[String]) -> Result<usize> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let mut removed = 0;
{
let mut stmt = tx.prepare("DELETE FROM trash WHERE uid = ?1")?;
for uid in uids {
removed += stmt.execute(params![uid])?;
}
}
tx.commit()?;
Ok(removed)
}
/// Drop every trashed node not in `keep`. Returns how many rows went.
pub fn trash_retain(&self, keep: &HashSet<String>) -> Result<usize> {
let gone: Vec<String> = self
.trash_seen()?
.into_keys()
.filter(|uid| !keep.contains(uid))
.collect();
self.trash_remove(&gone)
}
/// The persisted trash listing, folders first then by name — the order the
/// Trash page shows it in.
pub fn trash_list(&self) -> Result<Vec<StoredTrash>> {
+17
View File
@@ -42,6 +42,13 @@ pub enum Error {
#[error("that mailbox password does not unlock your account")]
WrongMailboxPassword,
/// The account's only second factor is a security key, which needs
/// WebAuthn. Asking for a code would only fail as a wrong one.
#[error(
"this account's only second factor is a security key, which pdfs cannot use yet — add an authenticator app in your Proton account settings"
)]
SecurityKeyOnly,
/// The login was gated behind human verification. Not a failure the user can
/// fix by retrying: the challenge has to be solved and the login restarted
/// with the resulting token (see [`auth::login_verified`](crate::auth::login_verified)).
@@ -220,6 +227,7 @@ fn core_kind(e: &Error) -> ErrorKind {
| Error::ReloginRequired
| Error::Locked
| Error::WrongMailboxPassword
| Error::SecurityKeyOnly
| Error::HumanVerificationRequired(_) => ErrorKind::Denied,
Error::Proton(proton) => proton_kind(proton),
_ => ErrorKind::Internal,
@@ -347,6 +355,15 @@ pub fn sign_in_blocked(e: &proton_sdk::ProtonError) -> bool {
/// The result every request-serving method in the daemon returns.
pub type CoreResult<T> = std::result::Result<T, CoreError>;
impl From<proton_auth_rs::Error> for Error {
fn from(error: proton_auth_rs::Error) -> Self {
match error {
proton_auth_rs::Error::Keyring(error) => Self::Keyring(error),
other => Self::Other(other.to_string()),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
-133
View File
@@ -1,133 +0,0 @@
//! When a photo was taken, as its own EXIF tags say.
//!
//! The fallback for a photo nothing else dates: a Takeout photo without a
//! sidecar or a date in its name, and a photo uploaded from the app. Without
//! it the first was filed under the moment of the import and the second under
//! the file's modification time, which a copy resets.
use std::io::{BufRead, Seek};
use std::path::Path;
use crate::takeout::days_from_civil;
/// The capture time in the EXIF of the file at `path`, in epoch seconds.
pub fn capture_time_of(path: &Path) -> Option<i64> {
let file = std::fs::File::open(path).ok()?;
capture_time(&mut std::io::BufReader::new(file))
}
/// The capture time in the EXIF `reader` holds: `DateTimeOriginal`, then
/// `DateTimeDigitized`, each adjusted by its `OffsetTime…` tag when the camera
/// wrote one. Without an offset the time is read as UTC, as a date in a file
/// name is (`takeout::capture_time_from_name`).
pub fn capture_time(reader: &mut (impl BufRead + Seek)) -> Option<i64> {
let exif = exif::Reader::new().read_from_container(reader).ok()?;
let ascii = |tag: exif::Tag| -> Option<String> {
let field = exif.get_field(tag, exif::In::PRIMARY)?;
match &field.value {
exif::Value::Ascii(parts) => parts.first().map(|part| {
String::from_utf8_lossy(part)
.trim_matches('\0')
.trim()
.to_string()
}),
_ => None,
}
};
[
(exif::Tag::DateTimeOriginal, exif::Tag::OffsetTimeOriginal),
(exif::Tag::DateTimeDigitized, exif::Tag::OffsetTimeDigitized),
]
.into_iter()
.find_map(|(time, offset)| {
let local = parse_exif_time(&ascii(time)?)?;
let offset = ascii(offset).and_then(|o| parse_offset(&o)).unwrap_or(0);
Some(local - offset)
})
}
/// `YYYY:MM:DD HH:MM:SS` as epoch seconds, read as UTC. `None` for the
/// all-zero or blank value cameras write when they do not know.
fn parse_exif_time(text: &str) -> Option<i64> {
let bytes = text.as_bytes();
if bytes.len() < 19 || bytes[4] != b':' || bytes[7] != b':' || bytes[13] != b':' {
return None;
}
let number = |range: std::ops::Range<usize>| text.get(range)?.parse::<i64>().ok();
let (year, month, day) = (number(0..4)?, number(5..7)?, number(8..10)?);
let (hour, minute, second) = (number(11..13)?, number(14..16)?, number(17..19)?);
if !(1800..=9999).contains(&year)
|| !(1..=12).contains(&month)
|| !(1..=31).contains(&day)
|| hour > 23
|| minute > 59
|| second > 60
{
return None;
}
Some(days_from_civil(year, month, day) * 86_400 + hour * 3600 + minute * 60 + second)
}
/// `+HH:MM` or `-HH:MM` as seconds east of UTC.
fn parse_offset(text: &str) -> Option<i64> {
let sign = match text.as_bytes().first()? {
b'+' => 1,
b'-' => -1,
_ => return None,
};
let (hours, minutes) = text.get(1..)?.split_once(':')?;
let (hours, minutes) = (hours.parse::<i64>().ok()?, minutes.parse::<i64>().ok()?);
(hours <= 14 && minutes < 60).then_some(sign * (hours * 3600 + minutes * 60))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn an_exif_time_reads_as_utc_and_its_offset_moves_it() {
// 2023-06-01 12:00:00 UTC.
let noon = 1_685_620_800;
assert_eq!(parse_exif_time("2023:06:01 12:00:00"), Some(noon));
assert_eq!(parse_offset("+02:00"), Some(7200));
assert_eq!(parse_offset("-05:30"), Some(-19_800));
assert_eq!(
parse_exif_time("2023:06:01 14:00:00").map(|t| t - parse_offset("+02:00").unwrap()),
Some(noon)
);
}
#[test]
fn an_unknown_exif_time_is_no_time() {
assert_eq!(parse_exif_time("0000:00:00 00:00:00"), None);
assert_eq!(parse_exif_time(" : : : : "), None);
assert_eq!(parse_exif_time(""), None);
assert_eq!(parse_offset("02:00"), None);
}
#[test]
fn a_photo_is_dated_by_its_original_time_and_offset() {
let ascii = |tag, text: &str| exif::Field {
tag,
ifd_num: exif::In::PRIMARY,
value: exif::Value::Ascii(vec![text.as_bytes().to_vec()]),
};
let taken = ascii(exif::Tag::DateTimeOriginal, "2023:06:01 14:00:00");
let offset = ascii(exif::Tag::OffsetTimeOriginal, "+02:00");
let digitized = ascii(exif::Tag::DateTimeDigitized, "2024:01:01 00:00:00");
let mut writer = exif::experimental::Writer::new();
writer.push_field(&taken);
writer.push_field(&offset);
writer.push_field(&digitized);
let mut tiff = std::io::Cursor::new(Vec::new());
writer.write(&mut tiff, false).unwrap();
tiff.set_position(0);
assert_eq!(capture_time(&mut tiff), Some(1_685_620_800));
}
#[test]
fn a_file_without_exif_has_no_capture_time() {
let mut png = std::io::Cursor::new(b"\x89PNG\r\n\x1a\n".to_vec());
assert_eq!(capture_time(&mut png), None);
}
}
+3 -5
View File
@@ -10,7 +10,6 @@ pub mod config;
pub mod control;
pub mod db;
pub mod error;
pub mod exiftime;
pub mod kernelkey;
pub mod keycheck;
pub mod localindex;
@@ -19,21 +18,20 @@ pub mod mountfail;
pub mod mountpoint;
pub mod mounts;
pub mod opener;
pub mod places;
pub mod profile;
pub mod sdkcache;
pub mod search;
mod secretservice;
pub mod service;
pub mod shell;
pub mod similar;
pub mod syncignore;
pub mod syncroot;
pub mod takeout;
pub mod timestamp;
pub mod tray;
pub mod update;
// Photo logic shared with Proton Gallery through the proton-rs crates.
pub use proton_gallery_rs::{exiftime, places, similar, takeout};
pub use access::{Access, access_for, perm_bits};
pub use error::{CoreError, CoreResult, Error, Result};
+1 -3
View File
@@ -40,9 +40,7 @@ pub fn path(state_dir: &Path) -> PathBuf {
/// and renamed, so a reader never sees half of it.
pub fn record(state_dir: &Path, failure: &MountFailure) -> std::io::Result<()> {
let json = serde_json::to_vec(failure).map_err(std::io::Error::other)?;
let tmp = state_dir.join(format!("{FILE_NAME}.tmp"));
std::fs::write(&tmp, json)?;
std::fs::rename(&tmp, path(state_dir))
proton_base_rs::atomic::write_with_mode(&path(state_dir), &json, 0o644)
}
/// Forget the failure: a mount is up, or a new daemon has started.
-185
View File
@@ -1,185 +0,0 @@
//! Offline reverse geocoding: the town a photo was taken in, from its GPS
//! position, without asking any service.
//!
//! The towns are GeoNames' `cities15000` dump (every place with more than
//! 15,000 people), trimmed to id, name, position and country in
//! `data/cities15000.tsv` by `scripts/geonames-cities.py`. The script drops the
//! parts of a city, such as Kreuzberg or "Paris 15 Vaugirard", so that a photo
//! lands in the town people would name. GeoNames publishes the dump under
//! CC BY 4.0, which is why the About dialog credits it.
//!
//! `data/countries.tsv` maps each country code to its ISO 3166-1 English name,
//! as the `iso-codes` project spells it, so a front end can translate it
//! through that project's `iso_3166-1` gettext domain.
use std::collections::HashMap;
use std::sync::OnceLock;
/// How far from the nearest town a photo may be and still count as taken
/// there, in km. Farther out (at sea, in the wilderness) it has no place.
pub const MAX_DISTANCE_KM: f64 = 50.0;
/// Mean Earth radius in km, for the haversine distance.
const EARTH_RADIUS_KM: f64 = 6371.0;
const CITIES: &str = include_str!("../data/cities15000.tsv");
const COUNTRIES: &str = include_str!("../data/countries.tsv");
/// One town of the table.
#[derive(Debug, Clone, PartialEq)]
pub struct City {
/// The GeoNames id, stable across dumps.
pub id: u32,
pub name: &'static str,
pub latitude: f64,
pub longitude: f64,
/// ISO 3166-1 alpha-2 country code.
pub country: &'static str,
}
/// The towns, and an index of them by whole-degree cell.
struct Table {
cities: Vec<City>,
cells: HashMap<(i32, i32), Vec<usize>>,
}
fn table() -> &'static Table {
static TABLE: OnceLock<Table> = OnceLock::new();
TABLE.get_or_init(|| {
let cities: Vec<City> = CITIES.lines().filter_map(parse_city).collect();
let mut cells: HashMap<(i32, i32), Vec<usize>> = HashMap::new();
for (index, city) in cities.iter().enumerate() {
cells
.entry(cell(city.latitude, city.longitude))
.or_default()
.push(index);
}
Table { cities, cells }
})
}
fn parse_city(line: &'static str) -> Option<City> {
let mut fields = line.split('\t');
Some(City {
id: fields.next()?.parse().ok()?,
name: fields.next()?,
latitude: fields.next()?.parse().ok()?,
longitude: fields.next()?.parse().ok()?,
country: fields.next()?,
})
}
fn cell(latitude: f64, longitude: f64) -> (i32, i32) {
(latitude.floor() as i32, longitude.floor() as i32)
}
/// Great-circle distance between two positions, in km.
pub fn distance_km(a: (f64, f64), b: (f64, f64)) -> f64 {
let (lat1, lat2) = (a.0.to_radians(), b.0.to_radians());
let dlat = lat2 - lat1;
let dlon = (b.1 - a.1).to_radians();
let h = (dlat / 2.0).sin().powi(2) + lat1.cos() * lat2.cos() * (dlon / 2.0).sin().powi(2);
2.0 * EARTH_RADIUS_KM * h.sqrt().min(1.0).asin()
}
/// The town nearest to a position, if one lies within [`MAX_DISTANCE_KM`].
pub fn nearest(latitude: f64, longitude: f64) -> Option<&'static City> {
if !latitude.is_finite() || !longitude.is_finite() {
return None;
}
let table = table();
// A degree of latitude is about 111 km everywhere; a degree of longitude
// shrinks towards the poles, so more cells are needed to cover the radius.
let lat_span = (MAX_DISTANCE_KM / 111.0).ceil() as i32;
let lon_span = {
let scale = latitude.to_radians().cos().abs().max(0.01);
((MAX_DISTANCE_KM / (111.0 * scale)).ceil() as i32).min(180)
};
let (row, column) = cell(latitude, longitude);
let mut best: Option<(f64, &City)> = None;
for dlat in -lat_span..=lat_span {
for dlon in -lon_span..=lon_span {
// Longitude wraps at the antimeridian: -181 is 179.
let lon_cell = (column + dlon + 180).rem_euclid(360) - 180;
let Some(indices) = table.cells.get(&(row + dlat, lon_cell)) else {
continue;
};
for &index in indices {
let city = &table.cities[index];
let d = distance_km((latitude, longitude), (city.latitude, city.longitude));
if d <= MAX_DISTANCE_KM && best.is_none_or(|(bd, _)| d < bd) {
best = Some((d, city));
}
}
}
}
best.map(|(_, city)| city)
}
/// The town with this GeoNames id.
pub fn city(id: u32) -> Option<&'static City> {
let cities = &table().cities;
// The table is sorted by id.
cities
.binary_search_by_key(&id, |city| city.id)
.ok()
.map(|index| &cities[index])
}
/// The English ISO 3166-1 name of a country code, such as "Germany" for "DE".
pub fn country_name(code: &str) -> Option<&'static str> {
COUNTRIES.lines().find_map(|line| {
let (c, name) = line.split_once('\t')?;
(c == code).then_some(name)
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_position_in_a_town_resolves_to_that_town() {
let berlin = nearest(52.52, 13.405).expect("Berlin is a town");
assert_eq!(berlin.name, "Berlin");
assert_eq!(berlin.country, "DE");
assert_eq!(city(berlin.id), Some(berlin));
}
#[test]
fn the_open_sea_has_no_town() {
assert_eq!(nearest(0.0, -30.0), None);
assert_eq!(nearest(f64::NAN, 0.0), None);
}
#[test]
fn a_town_across_the_antimeridian_is_still_found() {
// Just east of 180°, the nearest towns in Fiji are west of it.
let town = nearest(-16.8, 179.99);
assert!(town.is_none_or(|t| t.country == "FJ"));
let suva = nearest(-18.14, 178.44).expect("Suva is a town");
assert_eq!(suva.country, "FJ");
}
#[test]
fn every_town_has_a_named_country() {
for city in &table().cities {
assert!(
country_name(city.country).is_some(),
"{} has no country name for {}",
city.name,
city.country
);
}
assert_eq!(country_name("DE"), Some("Germany"));
}
#[test]
fn distances_are_great_circle_km() {
let d = distance_km((52.52, 13.405), (48.137, 11.575));
assert!(
(d - 504.0).abs() < 5.0,
"Berlin to Munich is about 504 km, got {d}"
);
}
}
-135
View File
@@ -1,135 +0,0 @@
//! Saving a new secret into an existing Secret Service item by replacing the
//! item, not by editing its secret (`docs/BUGS.md` B291).
//!
//! oo7-daemon before 0.7.0 applies `SetSecret` only in memory: the keyring
//! file keeps the old secret, and it comes back once the daemon restarts.
//! Fedora 45 ships such a build (`0.7.0~beta^1.e91b98a`). Every refreshed
//! sign-in pdfs saved there was gone after a reboot, and the next start found
//! a refresh token Proton had already spent.
//!
//! `CreateItem` with `replace` is written to the file there, and every
//! provider implements it. So a secret that changes is written as a new item
//! that replaces the old one: in the old one's collection, with its label and
//! all of its attributes, so the provider recognises it as the same entry.
use dbus_secret_service::{Collection, EncryptionType, Path, SecretService};
use std::collections::HashMap;
/// The content type the keyring store writes its secrets with.
const CONTENT_TYPE: &str = "application/octet-stream";
/// What [`replace`] did.
#[derive(Debug, PartialEq, Eq)]
pub(crate) enum Replaced {
/// The secret is saved.
Saved,
/// Nothing was changed; save it the usual way. Says why, for the log.
Skipped(String),
}
/// Save `secret` into the one item for `service`/`user` by replacing it.
///
/// Skips, changing nothing, when there is no such item yet (the store creates
/// it), when there is more than one, when it is locked, or when the Secret
/// Service fails before the new item exists. Never shows a prompt: one would
/// block the caller, so it is dismissed and the write is skipped.
pub(crate) fn replace(service: &str, user: &str, secret: &[u8]) -> Replaced {
match try_replace(service, user, secret) {
Ok(replaced) => replaced,
Err(e) => Replaced::Skipped(e.to_string()),
}
}
fn try_replace(
service: &str,
user: &str,
secret: &[u8],
) -> Result<Replaced, dbus_secret_service::Error> {
let ss = SecretService::connect_with_max_prompt_timeout(EncryptionType::Dh, 0)?;
let lookup = HashMap::from([("service", service), ("username", user)]);
let found = ss.search_items(lookup.clone())?;
if !found.locked.is_empty() {
return Ok(Replaced::Skipped("the item is locked".into()));
}
let [old] = found.unlocked.as_slice() else {
return Ok(Replaced::Skipped(format!(
"{} items match, not one",
found.unlocked.len()
)));
};
let Some(collection) = collection_of(&old.path) else {
return Ok(Replaced::Skipped(format!(
"no collection in the item path {}",
old.path
)));
};
let label = old.get_label()?;
let attributes = old.get_attributes()?;
let attributes: HashMap<&str, &str> = attributes
.iter()
.map(|(k, v)| (k.as_str(), v.as_str()))
.collect();
let collection = Collection::new(&ss, collection);
let new = collection.create_item(&label, attributes, secret, true, CONTENT_TYPE)?;
// A provider that did not match the old item leaves it next to the new
// one, and two items make every read ambiguous.
let after = ss.search_items(lookup)?;
for item in after.unlocked.iter().chain(&after.locked) {
if item.path == new.path {
continue;
}
if let Err(e) = item.delete() {
// The old item still holds the previous secret, so taking the new
// one out leaves things as they were, for the usual write.
if let Err(undo) = new.delete() {
tracing::error!(
error = %undo,
"could not remove a duplicate keyring item; remove one of the two \
`proton-drive-linux` entries in your keyring and sign in again"
);
}
return Ok(Replaced::Skipped(format!(
"could not remove the replaced item: {e}"
)));
}
}
Ok(Replaced::Saved)
}
/// The collection an item path belongs to: the path without its last part,
/// as long as it is a Secret Service object path.
fn collection_of(item: &Path<'_>) -> Option<Path<'static>> {
let (collection, _) = item.rsplit_once('/')?;
if !collection.starts_with("/org/freedesktop/secrets/") || collection.ends_with('/') {
return None;
}
Path::new(collection.to_owned()).ok()
}
#[cfg(test)]
mod tests {
use super::*;
fn collection(item: &str) -> Option<String> {
collection_of(&Path::new(item).unwrap()).map(|p| p.to_string())
}
#[test]
fn an_item_belongs_to_the_collection_its_path_names() {
assert_eq!(
collection("/org/freedesktop/secrets/collection/login/14").as_deref(),
Some("/org/freedesktop/secrets/collection/login")
);
assert_eq!(
collection("/org/freedesktop/secrets/aliases/default/3").as_deref(),
Some("/org/freedesktop/secrets/aliases/default")
);
}
#[test]
fn a_path_outside_the_secret_service_names_no_collection() {
assert_eq!(collection("/org/example/item/1"), None);
assert_eq!(collection("/item"), None);
}
}
-193
View File
@@ -1,193 +0,0 @@
//! Telling photos apart by what they show rather than by their bytes.
//!
//! The server's content hash only matches byte-identical files. A photo that
//! was resized, re-encoded or sent through a messenger is another file, so the
//! similar-photo finder hashes each thumbnail instead: a 64-bit difference
//! hash ("dHash"), which compares neighbouring brightness in a 9×8 grid. Two
//! copies of one picture land a few bits apart, and different pictures about
//! half of the bits apart.
/// Hashes this many bits apart or fewer show the same picture.
pub const SIMILAR_BITS: u32 = 6;
/// Stored for a photo whose thumbnail was looked at and gave no usable hash
/// (it couldn't be decoded, or is one flat colour), so it isn't looked at
/// again. A real hash is never 0: a picture that would hash to 0 has no
/// brightness changes, which [`difference_hash`] turns down as flat.
pub const NO_HASH: u64 = 0;
/// Samples of the grid that differ by less than this in brightness, all of
/// them, make a flat picture — a black frame, a blank page — whose hash is
/// noise and would match every other flat picture.
const FLAT: u8 = 8;
/// The difference hash of an 8-bit greyscale image `width` px wide, rows
/// stored one after another in `luma`. `None` for an empty or flat image.
pub fn difference_hash(width: usize, height: usize, luma: &[u8]) -> Option<u64> {
if width == 0 || height == 0 || luma.len() < width * height {
return None;
}
// Box-average down to 9×8, so every source pixel counts once and the hash
// doesn't depend on which pixels a nearest-neighbour pick happens to hit.
let mut grid = [[0u8; 9]; 8];
for (gy, row) in grid.iter_mut().enumerate() {
let (y0, y1) = span(gy, 8, height);
for (gx, cell) in row.iter_mut().enumerate() {
let (x0, x1) = span(gx, 9, width);
let mut sum = 0u64;
for y in y0..y1 {
sum += luma[y * width + x0..y * width + x1]
.iter()
.map(|&v| u64::from(v))
.sum::<u64>();
}
*cell = (sum / ((y1 - y0) * (x1 - x0)) as u64) as u8;
}
}
let (min, max) = grid
.iter()
.flatten()
.fold((u8::MAX, 0), |(min, max), &v| (min.min(v), max.max(v)));
if max - min < FLAT {
return None;
}
let mut hash = 0u64;
for row in &grid {
for pair in row.windows(2) {
hash = hash << 1 | u64::from(pair[0] < pair[1]);
}
}
(hash != NO_HASH).then_some(hash)
}
/// The source pixels `from..to` that cell `i` of `cells` covers along an edge
/// of `len` px; never empty, even for an edge shorter than the grid.
fn span(i: usize, cells: usize, len: usize) -> (usize, usize) {
let from = (i * len / cells).min(len - 1);
let to = ((i + 1) * len / cells).max(from + 1);
(from, to)
}
/// Whether two hashes show the same picture.
pub fn alike(a: u64, b: u64) -> bool {
(a ^ b).count_ones() <= SIMILAR_BITS
}
/// Group `hashes` into sets of alike ones, by index, each set in index order
/// and the sets by their first index. A hash alike to none other is in no
/// set. Alike is taken as catching: a burst of shots each a little different
/// from the last is one set.
pub fn sets(hashes: &[u64]) -> Vec<Vec<usize>> {
// Two hashes at most SIMILAR_BITS apart agree on at least one of their
// eight bytes, as long as SIMILAR_BITS < 8. So only hashes sharing some
// byte are compared, not every pair of the library.
const _: () = assert!(SIMILAR_BITS < 8);
let mut parent: Vec<usize> = (0..hashes.len()).collect();
fn root(parent: &mut [usize], mut i: usize) -> usize {
while parent[i] != i {
parent[i] = parent[parent[i]];
i = parent[i];
}
i
}
for byte in 0..8 {
let mut buckets: Vec<Vec<usize>> = vec![Vec::new(); 256];
for (i, &hash) in hashes.iter().enumerate() {
buckets[(hash >> (byte * 8)) as u8 as usize].push(i);
}
for bucket in &buckets {
for (n, &a) in bucket.iter().enumerate() {
for &b in &bucket[n + 1..] {
if alike(hashes[a], hashes[b]) {
let (ra, rb) = (root(&mut parent, a), root(&mut parent, b));
if ra != rb {
parent[ra.max(rb)] = ra.min(rb);
}
}
}
}
}
}
let mut by_root: std::collections::BTreeMap<usize, Vec<usize>> = Default::default();
for i in 0..hashes.len() {
let r = root(&mut parent, i);
by_root.entry(r).or_default().push(i);
}
by_root.into_values().filter(|set| set.len() > 1).collect()
}
#[cfg(test)]
mod tests {
use super::*;
/// A 90×80 picture: a bright disc on a gradient.
fn picture(width: usize, height: usize) -> Vec<u8> {
let mut luma = Vec::with_capacity(width * height);
for y in 0..height {
for x in 0..width {
let (fx, fy) = (x as f64 / width as f64, y as f64 / height as f64);
let disc = ((fx - 0.3).powi(2) + (fy - 0.6).powi(2)).sqrt() < 0.2;
let v = if disc { 240.0 } else { 40.0 + 120.0 * fx * fy };
luma.push(v as u8);
}
}
luma
}
#[test]
fn a_resized_copy_hashes_alike() {
let big = difference_hash(900, 800, &picture(900, 800)).unwrap();
let small = difference_hash(91, 79, &picture(91, 79)).unwrap();
assert!(
alike(big, small),
"{} bits apart",
(big ^ small).count_ones()
);
}
#[test]
fn a_different_picture_hashes_apart() {
let one = difference_hash(90, 80, &picture(90, 80)).unwrap();
let mirrored: Vec<u8> = picture(90, 80)
.chunks(90)
.flat_map(|row| row.iter().rev().copied().collect::<Vec<_>>())
.collect();
let other = difference_hash(90, 80, &mirrored).unwrap();
assert!(!alike(one, other));
}
#[test]
fn a_flat_picture_has_no_hash() {
assert_eq!(difference_hash(64, 64, &[17; 64 * 64]), None);
assert_eq!(difference_hash(0, 0, &[]), None);
assert_eq!(difference_hash(4, 4, &[0; 3]), None);
}
#[test]
fn a_picture_smaller_than_the_grid_still_hashes() {
let luma: Vec<u8> = (0..4 * 3).map(|i| (i * 20) as u8).collect();
assert!(difference_hash(4, 3, &luma).is_some());
}
#[test]
fn alike_hashes_form_sets_and_catch_on() {
let a = 0x0123_4567_89ab_cdef_u64;
let hashes = [
a,
!a,
a ^ 0b111,
a ^ 0b111 ^ (0b1111 << 40),
0x5555_0000_ffff_aaaa,
];
assert_eq!(sets(&hashes), vec![vec![0, 2, 3]]);
}
#[test]
fn hashes_apart_in_most_bytes_are_still_found() {
// One bit apart in each of six bytes: six bits in all, and only the
// top two bytes agree.
let a = 1u64 << 63;
let b = (0..6).fold(a, |h, byte| h | 1 << (byte * 8));
assert_eq!(sets(&[a, b]), vec![vec![0, 1]]);
}
}
File diff suppressed because it is too large. Load diff
+9
View File
@@ -14,12 +14,21 @@ pdfs-core.workspace = true
zip.workspace = true
sha1.workspace = true
proton-drive-rs.workspace = true
proton-gallery-app-rs.workspace = true
proton-gallery-rs.workspace = true
proton-media-rs.workspace = true
hyper = { version = "1", features = ["http1", "server"] }
hyper-util = { version = "0.1", features = ["tokio"] }
http-body-util = "0.1"
getrandom = "0.4"
zeroize.workspace = true
futures.workspace = true
async-trait.workspace = true
tokio.workspace = true
tracing.workspace = true
fuser.workspace = true
libc.workspace = true
rustix = { version = "1", features = ["net", "process"] }
serde.workspace = true
serde_json.workspace = true
base64.workspace = true
+111 -92
View File
@@ -12,7 +12,7 @@
//! than looked up in our own timeline, where a shared album's photos never
//! appear.
use std::collections::{HashMap, HashSet};
use std::collections::HashSet;
use std::sync::atomic::Ordering;
use pdfs_core::control::{AlbumInfo, PhotoItem, Topic};
@@ -24,8 +24,8 @@ use proton_drive_rs::proton_sdk::ids::NodeUid;
use tracing::warn;
use super::{
ALBUM_SYNCED_PREFIX, ALBUMS_SYNCED_MS, Core, PHOTOS_AVAILABLE, PHOTOS_SYNCED_MS,
TIMELINE_ENRICH_CHUNK, TIMELINE_TTL, now_ms, parse_uid,
ALBUM_SYNCED_PREFIX, ALBUMS_SYNCED_MS, Core, PHOTOS_AVAILABLE, TIMELINE_ENRICH_CHUNK,
TIMELINE_TTL, link, now_ms, parse_uid,
};
impl Core {
@@ -34,7 +34,13 @@ impl Core {
/// [`Core::photos_timeline`] gives.
pub(crate) fn albums(&self) -> CoreResult<Option<Vec<AlbumInfo>>> {
let count = self.db.albums_count().map_err(CoreError::from)?;
if count == 0 {
if count == 0
&& self
.db
.state_i64(ALBUMS_SYNCED_MS)
.map_err(CoreError::from)?
.is_none()
{
// Nothing on disk, so this one request waits for the fetch — unless
// the account is already known to have no photos volume.
let known_empty = self.db.state_i64(PHOTOS_AVAILABLE).ok().flatten() == Some(0);
@@ -56,77 +62,71 @@ impl Core {
/// the persisted listing. Returns false when the account has no photos
/// volume.
pub(crate) async fn refresh_albums(&self) -> CoreResult<bool> {
let generation = self
.db
.photo_projection_generation()
.map_err(CoreError::from)?;
let photos = self.photos();
if photos
.get_photos_root()
if link::bounded(photos.get_photos_root())
.await
.map_err(|e| CoreError::from_api(&e, "photos root"))?
.is_none()
{
let _ = self.db.set_state_i64(PHOTOS_AVAILABLE, 0);
let _ = self.db.set_state_i64(PHOTOS_SYNCED_MS, now_ms());
let _ = self.db.set_state_i64(ALBUMS_SYNCED_MS, now_ms());
self.db
.albums_replace_at(&[], generation, now_ms())
.map_err(CoreError::from)?;
return Ok(false);
}
let mut uids = photos
.enumerate_album_node_uids()
let mut uids = link::bounded(photos.enumerate_album_node_uids())
.await
.map_err(|e| CoreError::from_api(&e, "albums"))?;
let shared = link::bounded(photos.enumerate_shared_with_me_album_uids())
.await
.map_err(|e| CoreError::from_api(&e, "shared albums"))?;
let shared_keys: HashSet<_> = shared.iter().map(ToString::to_string).collect();
uids.extend(shared);
let mut unique = HashSet::new();
uids.retain(|uid| unique.insert(uid.clone()));
// Shared-with-me albums are a separate listing on the sharer's volume.
// Best-effort: an account whose sharing listing fails still gets its own
// albums rather than an error page.
let shared: HashSet<String> = match photos.enumerate_shared_with_me_album_uids().await {
Ok(shared) => {
let keys = shared.iter().map(|u| u.to_string()).collect();
uids.extend(shared);
keys
}
Err(e) => {
warn!(error = %e, "enumerating shared-with-me albums failed");
HashSet::new()
}
};
// The listings carry uids only; the name, photo count and cover come from
// the album nodes themselves. A chunk that fails to resolve drops those
// albums from this refresh rather than failing the whole listing.
let mut albums: Vec<StoredAlbum> = Vec::with_capacity(uids.len());
let mut albums = Vec::with_capacity(uids.len());
for chunk in uids.chunks(TIMELINE_ENRICH_CHUNK) {
match photos.enumerate_nodes(chunk).await {
Ok(nodes) => {
for node in nodes {
let key = node.uid.to_string();
let props = node.album.as_ref();
albums.push(StoredAlbum {
shared: shared.contains(&key),
uid: key,
name: node.name,
photo_count: props.map_or(0, |a| a.photo_count.max(0) as usize),
cover_uid: props
.and_then(|a| a.cover_photo_uid.as_ref())
.map(|u| u.to_string()),
last_activity: props.and_then(|a| a.last_activity_time),
});
}
}
Err(e) => warn!(error = %e, "resolving an album chunk failed"),
let nodes = link::bounded(photos.enumerate_nodes(chunk))
.await
.map_err(|e| CoreError::from_api(&e, "album metadata"))?;
if nodes.len() != chunk.len() {
return Err(CoreError::remote(
"Some album details are still unavailable.",
));
}
for node in nodes.into_iter().filter(|node| !node.trashed) {
let uid = node.uid.to_string();
let props = node.album.as_ref();
albums.push(StoredAlbum {
shared: shared_keys.contains(&uid),
uid,
name: node.name,
photo_count: props.map_or(0, |a| a.photo_count.max(0) as usize),
cover_uid: props
.and_then(|a| a.cover_photo_uid.as_ref())
.map(ToString::to_string),
last_activity: props.and_then(|a| a.last_activity_time),
});
}
}
// Newest activity first, and albums the server gave no activity time for
// last — an album nobody has touched belongs below the ones in use. Ties
// fall back to the name so the order is stable between refreshes.
albums.sort_by(|a, b| {
b.last_activity
.cmp(&a.last_activity)
.then_with(|| a.name.to_lowercase().cmp(&b.name.to_lowercase()))
});
self.db.albums_replace(&albums).map_err(CoreError::from)?;
let _ = self.db.set_state_i64(PHOTOS_AVAILABLE, 1);
let _ = self.db.set_state_i64(ALBUMS_SYNCED_MS, now_ms());
if !self
.db
.albums_replace_at(&albums, generation, now_ms())
.map_err(CoreError::from)?
{
return Err(CoreError::remote(
"Photos changed during the album refresh. Try again shortly.",
));
}
Ok(true)
}
@@ -154,11 +154,21 @@ impl Core {
album: &NodeUid,
offset: usize,
limit: usize,
excluded: &str,
) -> CoreResult<Vec<PhotoItem>> {
let key = album.to_string();
if self.db.photo_removed(&key).map_err(CoreError::from)? {
return Ok(Vec::new());
}
let stamp = format!("{ALBUM_SYNCED_PREFIX}{key}");
let count = self.db.album_photos_count(&key).map_err(CoreError::from)?;
if count == 0 {
if count == 0
&& self
.db
.state_i64(&stamp)
.map_err(CoreError::from)?
.is_none()
{
self.rt.block_on(self.refresh_album(album))?;
} else if self.listing_stale(&stamp, TIMELINE_TTL) {
let core = self.clone();
@@ -172,7 +182,7 @@ impl Core {
let page = self
.db
.album_photos_page(&key, offset, limit)
.album_photos_page_visible(&key, offset, limit, excluded)
.map_err(CoreError::from)?;
Ok(page.into_iter().map(|p| self.photo_item(p)).collect())
}
@@ -213,47 +223,56 @@ impl Core {
/// Re-enumerate one album's photos and replace its persisted contents.
pub(crate) async fn refresh_album(&self, album: &NodeUid) -> CoreResult<()> {
let generation = self
.db
.photo_projection_generation()
.map_err(CoreError::from)?;
let photos = self.photos();
let items = photos
.enumerate_album(album)
let items = link::bounded(photos.enumerate_album(album))
.await
.map_err(|e| CoreError::from_api(&e, "album"))?;
// Same enrichment as the timeline: the listing carries a uid and a
// capture time, and the gallery needs a name and media type to classify
// each entry. Best-effort per chunk.
let uids: Vec<NodeUid> = items.iter().map(|it| it.uid.clone()).collect();
let mut meta: HashMap<String, (Option<String>, Option<String>)> = HashMap::new();
let key = album.to_string();
let mut meta = self
.db
.album_photo_metadata(&key)
.map_err(CoreError::from)?;
let uids: Vec<NodeUid> = items
.iter()
.filter(|item| !meta.contains_key(&item.uid.to_string()))
.map(|item| item.uid.clone())
.collect();
for chunk in uids.chunks(TIMELINE_ENRICH_CHUNK) {
match photos.enumerate_nodes(chunk).await {
Ok(nodes) => {
for node in nodes {
let media_type = match &node.kind {
NodeKind::File { media_type, .. } => Some(media_type.clone()),
NodeKind::Folder => None,
};
meta.insert(node.uid.to_string(), (Some(node.name), media_type));
}
}
Err(e) => warn!(error = %e, "resolving album photo metadata failed"),
let nodes = link::bounded(photos.enumerate_nodes(chunk))
.await
.map_err(|e| CoreError::from_api(&e, "album photo metadata"))?;
for node in nodes.into_iter().filter(|node| !node.trashed) {
let media_type = match node.kind {
NodeKind::File { media_type, .. } => Some(media_type),
NodeKind::Folder => None,
};
meta.insert(node.uid.to_string(), (node.name, media_type));
}
}
let rows: Vec<(String, i64, Option<String>, Option<String>)> = items
.iter()
.map(|it| {
let key = it.uid.to_string();
let (name, media_type) = meta.get(&key).cloned().unwrap_or((None, None));
(key, it.capture_time, name, media_type)
let rows: Vec<_> = items
.into_iter()
.map(|item| {
let uid = item.uid.to_string();
let (name, media_type) = meta
.get(&uid)
.map(|(name, mime)| (Some(name.clone()), mime.clone()))
.unwrap_or_default();
(uid, item.capture_time, name, media_type)
})
.collect();
let key = album.to_string();
self.db
.album_photos_replace(&key, &rows)
.map_err(CoreError::from)?;
let _ = self
if !self
.db
.set_state_i64(&format!("{ALBUM_SYNCED_PREFIX}{key}"), now_ms());
.album_photos_replace_at(&key, &rows, generation, now_ms())
.map_err(CoreError::from)?
{
return Err(CoreError::remote(
"Photos changed during the album refresh. Try again shortly.",
));
}
Ok(())
}
@@ -283,7 +302,7 @@ impl Core {
.block_on(self.photos().delete_album(album, delete_photos))
.map_err(|e| CoreError::from_api(&e, "delete album"))?;
let key = album.to_string();
if let Err(error) = self.db.album_photos_replace(&key, &[]) {
if let Err(error) = self.db.photos_delete(&[key]) {
warn!(%album, %error, "a deleted album's photos could not be dropped");
}
self.after_album_change(None);
+68 -81
View File
@@ -517,32 +517,15 @@ pub(super) async fn run_event_sync(
}
}
/// The state key holding the photos volume's event cursor. Separate from the
/// Drive cursor because the two volumes have their own event streams, and one
/// cursor cannot address both.
const PHOTOS_EVENT_CURSOR: &str = "photos_event_cursor";
/// Poll the photos volume's event stream, so a photo deleted or added on another
/// device shows up in the gallery within one poll instead of on the timeline's
/// freshness window.
///
/// Deliberately much smaller than [`run_event_sync`]: the photos volume is not
/// part of the FUSE mount, so there is no inode space to converge and nothing to
/// invalidate in the kernel. A trash or a delete removes the rows behind the
/// gallery; anything else only drops the timeline's freshness stamp, and the
/// next read re-enumerates. Getting that wrong costs a stale tile, not a wrong
/// file.
/// Follow the Photos volume and commit changes with its durable event cursor.
pub(super) async fn run_photos_event_sync(core: Core) {
let photos = core.photos();
// The photos volume only exists once the user has used Photos at all, and
// the daemon may well start before the network does.
let scope = loop {
match photos.get_photos_root().await {
match link::bounded(photos.get_photos_root()).await {
Ok(Some(root)) => break root.tree_event_scope_id(),
Ok(None) => {
debug!("no photos volume; photo event sync not started");
return;
}
Ok(None) => core.next_poll(POLL_INTERVAL).await,
Err(error) if session_revoked(&error) => {
core.mark_session_expired("read the photos root", &error);
return;
@@ -554,39 +537,71 @@ pub(super) async fn run_photos_event_sync(core: Core) {
}
};
let mut cursor: Option<DriveEventId> = core
.db
.state_str(PHOTOS_EVENT_CURSOR)
.ok()
.flatten()
.map(DriveEventId::from);
info!(?cursor, "photo event sync started");
let key = format!("photos_event_cursor:{}", scope.volume_id());
let mut cursor = loop {
match core.db.state_str(&key) {
Ok(cursor) => break cursor.map(DriveEventId::from),
Err(error) => {
warn!(%error, "reading the photo event checkpoint failed");
core.next_poll(POLL_INTERVAL).await;
}
}
};
info!("photo event sync started");
loop {
core.next_poll(POLL_INTERVAL).await;
let events = match link::bounded(photos.enumerate_events(&scope, cursor.as_ref())).await {
Ok(events) => events,
Err(error) if core.lost_link(&error, "poll photo events") => continue,
Err(error) => {
warn!(%error, "photo event poll failed; retrying after interval");
if !core.lost_link(&error, "poll photo events") {
warn!(%error, "photo event poll failed; retrying after interval");
}
core.next_poll(POLL_INTERVAL).await;
continue;
}
};
let Some(head) = events.last().map(|event| event.id().clone()) else {
continue;
};
apply_photo_events(&core, &events);
core.events.publish(&[Topic::Photos]);
// One cursor write per batch, after the batch is applied: a crash in
// between replays events whose effects are idempotent.
if let Err(error) = core.db.set_state_str(PHOTOS_EVENT_CURSOR, head.as_str()) {
warn!(%error, "persisting the photo event cursor failed");
continue;
if let Some(head) = events.last().map(|event| event.id().clone()) {
if let Err(error) = invalidate_photo_events(&core, &events).await {
warn!(%error, "invalidating photo event metadata failed");
core.next_poll(POLL_INTERVAL).await;
continue;
}
// A failed database write must replay all changes, including echoes.
let plan = classify_photo_events(&events, |_| false);
if let Err(error) = core.db.photos_apply_events(
&key,
head.as_str(),
&plan.gone,
&plan.touched,
plan.full,
cursor.is_none(),
) {
warn!(%error, "committing the photo event checkpoint failed");
core.next_poll(POLL_INTERVAL).await;
continue;
}
cursor = Some(head);
if !plan.is_empty() {
core.events.publish(&[Topic::Photos]);
}
}
cursor = Some(head);
if core.db.photos_need_work().unwrap_or(true) {
core.spawn_timeline_refresh();
}
core.next_poll(POLL_INTERVAL).await;
}
}
async fn invalidate_photo_events(
core: &Core,
events: &[DriveEvent],
) -> proton_drive_rs::proton_sdk::error::Result<()> {
for event in events {
core.client.invalidate_caches_for_event(event).await?;
}
Ok(())
}
/// What one batch of photos-volume events asks the gallery's tables to do.
#[derive(Debug, Default, PartialEq)]
pub(super) struct PhotoEventPlan {
@@ -616,14 +631,22 @@ fn classify_photo_events(
mut is_echo: impl FnMut(&NodeUid) -> bool,
) -> PhotoEventPlan {
let mut plan = PhotoEventPlan::default();
for event in events {
let mut seen = HashSet::new();
for event in events.iter().rev() {
match event {
DriveEvent::NodeDeleted { node_uid, .. } => plan.gone.push(node_uid.to_string()),
DriveEvent::NodeDeleted { node_uid, .. } => {
if seen.insert(node_uid) {
plan.gone.push(node_uid.to_string());
}
}
DriveEvent::NodeUpdated {
node_uid,
is_trashed,
..
} => {
if !seen.insert(node_uid) {
continue;
}
if *is_trashed {
plan.gone.push(node_uid.to_string());
} else if !is_echo(node_uid) {
@@ -641,47 +664,11 @@ fn classify_photo_events(
DriveEvent::CursorAdvanced { .. } | DriveEvent::SharedWithMeUpdated { .. } => {}
}
}
plan.gone.reverse();
plan.touched.reverse();
plan
}
/// Apply one batch of photos-volume events to the gallery's tables.
fn apply_photo_events(core: &Core, events: &[DriveEvent]) {
let plan = classify_photo_events(events, |uid| core.take_self_change(uid));
let PhotoEventPlan {
gone,
touched,
full,
} = &plan;
if !gone.is_empty() {
match core.db.photos_delete(gone) {
Ok(removed) if removed > 0 => {
debug!(removed, "photos removed by a remote event");
}
Ok(_) => {}
Err(error) => warn!(%error, "removing remotely deleted photos failed"),
}
}
// A stale mark is what makes the next refresh cheap: it reads these photos'
// nodes and carries every other photo's metadata as it stands.
if !touched.is_empty() {
match core.db.photos_mark_unresolved(touched) {
Ok(marked) if marked > 0 => debug!(marked, "photos changed by a remote event"),
Ok(_) => {}
Err(error) => warn!(%error, "marking remotely changed photos failed"),
}
}
if *full {
match core.db.photos_mark_all_unresolved() {
Ok(marked) => debug!(marked, "the photo event stream lost continuity"),
Err(error) => warn!(%error, "marking the whole timeline unresolved failed"),
}
}
if !plan.is_empty() {
core.invalidate_photos();
core.invalidate_albums();
}
}
/// Keep the local-file index fresh for the launcher prompt's "This computer"
/// results. Rebuilds the index whenever it is older than [`LOCAL_INDEX_TTL`],
/// then sleeps; runs on its own thread for the life of the daemon.
+171 -17
View File
@@ -396,6 +396,11 @@ fn touched_topics(request: &CtlRequest) -> &'static [Topic] {
use CtlRequest as R;
match request {
R::Pin { .. } | R::Unpin { .. } => &[Topic::Files],
// Only locking invalidates a hidden view. Authentication does not
// change the public catalog and must not emit this notification.
R::DriveGallery {
action: pdfs_core::control::GalleryAction::Lock,
} => &[Topic::GalleryVisibility],
R::CreateAlbum { .. }
| R::RenameAlbum { .. }
| R::DeleteAlbum { .. }
@@ -552,6 +557,68 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
// Registered before the request is even parsed, so a handler that wedges
// inside the daemon shows up by name in `pdfs diagnostics`.
let _inflight = super::diagnostics::InflightGuard::new(super::diagnostics::request_kind(&line));
if let Ok(CtlRequest::PhotoMonths { kind }) = &request {
let started = std::time::Instant::now();
let visibility = core.cached_gallery_visibility();
let visibility_ms = started.elapsed().as_millis() as u64;
let mut database_ms = None;
let mut response_started = std::time::Instant::now();
let response = match visibility {
Ok(visibility) => {
let excluded = serde_json::to_string(&visibility.media)
.expect("a set of file identifiers serializes");
let query = std::time::Instant::now();
let months = core.db.photos_months_visible(*kind, &excluded);
response_started = std::time::Instant::now();
database_ms = Some(response_started.duration_since(query).as_millis() as u64);
match months {
Ok(months) => CtlResponse::PhotoMonths {
months: months
.into_iter()
.map(|(year, month, count)| PhotoMonth { year, month, count })
.collect(),
},
Err(error) => CtlResponse::error(error.into()),
}
}
Err(error) => CtlResponse::error(error),
};
write_response(stream, response);
tracing::info!(
visibility_ms,
database_ms,
response_ms = response_started.elapsed().as_millis() as u64,
total_ms = started.elapsed().as_millis() as u64,
"PhotoMonths timings"
);
return;
}
let visibility = if matches!(
&request,
Ok(CtlRequest::PhotosTimeline { .. }
| CtlRequest::PhotoDuplicates
| CtlRequest::PhotoSimilar
| CtlRequest::PhotoPlaces
| CtlRequest::PlacePhotos { .. }
| CtlRequest::PlaceLocations { .. }
| CtlRequest::PhotosOnThisDay { .. }
| CtlRequest::PhotoGroup { .. }
| CtlRequest::PhotoAlbums
| CtlRequest::AlbumPhotos { .. }
| CtlRequest::PhotoThumbs { .. }
| CtlRequest::OpenGalleryImage { private: false, .. }
| CtlRequest::OpenPhoto { .. })
) {
core.gallery_visibility()
} else {
Ok(std::sync::Arc::default())
};
let visibility = match visibility {
Ok(visibility) => visibility,
Err(error) => return write_response(stream, CtlResponse::error(error)),
};
let excluded =
serde_json::to_string(&visibility.media).expect("a set of file identifiers serializes");
let response = match request {
Ok(CtlRequest::Status) => status_response(core, username, mountpoint),
Ok(CtlRequest::Subscribe) => {
@@ -627,7 +694,7 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
// A full refresh is "forget what you know", not a second code
// path: the ordinary refresh then finds every photo stale.
if let Some(Err(e)) = full.then(|| core.db.photos_mark_all_unresolved()) {
warn!(error = %e, "marking the whole timeline unresolved failed");
return write_response(stream, CtlResponse::error(e.into()));
}
// Whatever changed the timeline can just as easily have
// changed an album; one scope covers the whole photos view.
@@ -668,11 +735,12 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
range,
favorites,
not_in_album,
&excluded,
) {
Ok(Some(items)) => CtlResponse::Photos {
available: true,
items,
counts: core.db.photos_counts().ok(),
counts: core.db.photos_counts_visible(&excluded).ok(),
},
Ok(None) => CtlResponse::Photos {
available: false,
@@ -693,7 +761,7 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PhotoPlaces) => match core.photo_places() {
Ok(CtlRequest::PhotoPlaces) => match core.photo_places(&excluded) {
Ok(items) => CtlResponse::Places {
items,
mapping: core.timeline_progress(),
@@ -701,7 +769,7 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PlacePhotos { id, offset, limit }) => {
match core.place_photos(id, offset, clamp_limit(limit)) {
match core.place_photos(id, offset, clamp_limit(limit), &excluded) {
Ok(items) => CtlResponse::Photos {
available: true,
items,
@@ -715,7 +783,7 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PhotosOnThisDay { limit }) => {
match core.photos_on_this_day(clamp_limit(limit)) {
match core.photos_on_this_day(clamp_limit(limit), &excluded) {
Ok(items) => CtlResponse::Photos {
available: true,
items,
@@ -760,7 +828,10 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
// An album page is a timeline page as far as the front-end is
// concerned, so it comes back in the same reply — with `counts`
// unset, which describe the whole timeline rather than this album.
Some(album) => match core.album_photos(&album, offset, clamp_limit(limit)) {
Some(album) => match visibility
.permits(&uid)
.and_then(|()| core.album_photos(&album, offset, clamp_limit(limit), &excluded))
{
Ok(items) => CtlResponse::Photos {
available: true,
items,
@@ -802,17 +873,28 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
core.remove_from_album(album, photos)
})
}
Ok(CtlRequest::PhotoMonths { kind }) => match core.db.photos_months(kind) {
Ok(months) => CtlResponse::PhotoMonths {
months: months
.into_iter()
.map(|(year, month, count)| PhotoMonth { year, month, count })
.collect(),
Ok(CtlRequest::PhotoPresence { uid, album }) => match core.cached_gallery_visibility() {
Ok(visibility)
if visibility.media.contains(&uid)
|| album
.as_ref()
.is_some_and(|uid| visibility.albums.contains(uid)) =>
{
CtlResponse::PhotoPresence { present: false }
}
Ok(_) => match core.db.photos_contains(&uid, album.as_deref()) {
Ok(present) => CtlResponse::PhotoPresence { present },
Err(error) => CtlResponse::error(error.into()),
},
Err(e) => CtlResponse::error(e.into()),
Err(error) => CtlResponse::error(error),
},
Ok(CtlRequest::PhotoMonths { .. }) => unreachable!("handled before visibility requests"),
Ok(CtlRequest::PhotoThumbs { uids }) => {
let parsed: Vec<NodeUid> = uids.iter().filter_map(|u| parse_uid(u)).collect();
let parsed: Vec<NodeUid> = uids
.iter()
.filter(|uid| !visibility.media.contains(*uid))
.filter_map(|u| parse_uid(u))
.collect();
CtlResponse::Thumbs {
items: core.photo_thumbs(&parsed),
}
@@ -862,7 +944,7 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
running: core.timeline_refreshing(),
},
Ok(CtlRequest::OpenPhoto { uid }) => match parse_uid(&uid) {
Some(u) => match core.open_photo(&u) {
Some(u) => match visibility.permits(&uid).and_then(|()| core.open_photo(&u)) {
Ok(p) => CtlResponse::FilePath {
path: p.display().to_string(),
},
@@ -870,6 +952,38 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
},
None => CtlResponse::error(CoreError::invalid(format!("bad uid: {uid}"))),
},
Ok(CtlRequest::DriveGallery {
action: pdfs_core::control::GalleryAction::AuthenticateHidden,
}) => core
.authenticate_hidden_gallery(&stream)
.unwrap_or_else(CtlResponse::error),
Ok(CtlRequest::DriveGallery { action }) => core
.drive_gallery(action)
.unwrap_or_else(CtlResponse::error),
Ok(CtlRequest::OpenGalleryVideo { uid, private }) => core
.open_gallery_video(&uid, private)
.unwrap_or_else(CtlResponse::error),
Ok(CtlRequest::OpenGalleryImage { uid, private: true }) => {
core.gallery_image(&uid).unwrap_or_else(CtlResponse::error)
}
Ok(CtlRequest::OpenGalleryImage {
uid,
private: false,
}) => match visibility
.permits(&uid)
.and_then(|()| core.open_shared_file(&uid))
{
Ok(path) => CtlResponse::FilePath {
path: path.display().to_string(),
},
Err(error) => CtlResponse::error(error),
},
Ok(CtlRequest::CloseGalleryVideo { token }) => core
.close_gallery_video(&token)
.unwrap_or_else(CtlResponse::error),
Ok(CtlRequest::GalleryThumbnail { uid, private }) => core
.gallery_thumbnail(&uid, private)
.unwrap_or_else(CtlResponse::error),
// The scan alone walks every archive's central directory, so even the
// dry run goes to a background thread and is read back through
// `ImportStatus` rather than answered inline.
@@ -986,7 +1100,8 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
// timeline, and the GUI reloads the gallery the moment this
// reply lands — so refresh now rather than leaving it to a
// background pass that would land just after that reload.
if let Err(e) = core.rt.block_on(core.refresh_timeline()) {
core.invalidate_photos();
if let Err(e) = core.refresh_timeline_joined() {
warn!(error = %e, "timeline refresh after upload failed");
}
CtlResponse::Ok {
@@ -1227,6 +1342,10 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Ok(entries) => CtlResponse::Entries { entries },
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::ListTrashedFolder { uid }) => match core.list_trashed_folder(&uid) {
Ok(entries) => CtlResponse::Entries { entries },
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::Restore { uids }) => match core.restore(&uids) {
Ok(n) => {
core.log_activity(ActivityKind::Restore, format!("{n} item(s)"), "", true);
@@ -1929,9 +2048,14 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
// The request did not parse: the caller sent something malformed.
Err(e) => CtlResponse::error(CoreError::invalid(format!("bad request: {e}"))),
};
let response = filter_gallery_response(response, &visibility);
if !touched.is_empty() && !matches!(response, CtlResponse::Error { .. }) {
core.events.publish(touched);
}
write_response(stream, response);
}
fn write_response(mut stream: UnixStream, response: CtlResponse) {
let mut out = match serde_json::to_vec(&response) {
Ok(v) => v,
Err(e) => {
@@ -1940,10 +2064,40 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
}
};
out.push(b'\n');
let mut stream = stream;
let _ = stream.write_all(&out);
}
fn filter_gallery_response(
mut response: CtlResponse,
visibility: &super::gallery::Visibility,
) -> CtlResponse {
let visible = |item: &pdfs_core::control::PhotoItem| !visibility.media.contains(&item.uid);
match &mut response {
CtlResponse::Photos { items, .. } => items.retain(visible),
CtlResponse::Albums { items, .. } => {
items.retain(|item| !visibility.albums.contains(&item.uid));
for album in items {
if album
.cover_uid
.as_ref()
.is_some_and(|uid| visibility.media.contains(uid))
{
album.cover_uid = None;
}
}
}
CtlResponse::PhotoDuplicates { sets } | CtlResponse::PhotoSimilar { sets, .. } => {
for set in sets.iter_mut() {
set.retain(visible);
}
sets.retain(|set| set.len() > 1);
}
CtlResponse::PhotoLocations { items } => items.retain(|located| visible(&located.photo)),
_ => {}
}
response
}
/// Listen on the control socket, serving one command per connection, each on its
/// own thread. Runs on its own thread; returns only if the listener itself fails.
///
+101 -16
View File
@@ -20,7 +20,7 @@
//! are kept as a conflict copy rather than discarded — see
//! [`Core::revision_conflict`].
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::fs::File;
use std::path::{Path, PathBuf};
use std::sync::Arc;
@@ -53,6 +53,21 @@ use super::{
use proton_drive_rs::proton_sdk::api::ResponseCode;
use proton_drive_rs::proton_sdk::error::ProtonError;
#[derive(Debug)]
struct TrashedMove(String);
impl std::fmt::Display for TrashedMove {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"‘{}’ is in Proton Drive Trash. Restore it before retrying this move.",
self.0
)
}
}
impl std::error::Error for TrashedMove {}
const DRAIN_ACCESS_RECHECK: Duration = Duration::from_secs(5);
/// How long an op may stay access-deferred before the deferral is reported as a
@@ -795,13 +810,13 @@ impl Core {
let parent = parse_node_uid(parent_str).ok_or("rename op has an unparseable parent")?;
let name = op.name.clone().ok_or("rename op has no name")?;
// The node we were asked to rename may be gone or trashed by now. Either
// way there is nothing to rename and nothing to lose — a rename holds no
// bytes — so the op is satisfied rather than retried forever.
// A trashed node can be restored. Only retire the operation if the node
// is gone entirely.
let node = match self.fetch_node_remote(&uid)? {
Some(n) if !n.trashed => n,
_ => {
warn!(%uid, name, "renamed node is gone or trashed remotely; dropping the rename");
Some(n) if n.trashed => return Err(TrashedMove(n.name).into()),
Some(n) => n,
None => {
warn!(%uid, name, "renamed node is gone remotely; dropping the rename");
self.db.delete_op(op.id)?;
return Ok(());
}
@@ -827,23 +842,38 @@ impl Core {
// no window for the requirements to go stale between two calls —
// the race B46 queued this op to get away from.
let target = (node.name != name).then_some(name.as_str());
match self.move_rename_remote(&uid, &parent, target) {
let try_move = |target| {
self.move_rename_remote(&uid, &parent, target).map_err(
|error| -> Box<dyn std::error::Error> {
if (is_gone(&error) || refusal(&error) == Some(Some(SyncIssue::Access)))
&& let Some(name) = trashed_move_folder(
node.parent_uid.iter().chain(std::iter::once(&parent)),
|uid| self.fetch_node_remote(uid).ok().flatten(),
)
{
return TrashedMove(name).into();
}
error.into()
},
)
};
match try_move(target) {
Ok(()) => name.clone(),
// The destination holds that name already. Landing under a
// *different* name is the non-destructive resolution: it neither
// clobbers their file nor drops ours, and it is visible.
Err(e) if is_already_exists(&e) => {
Err(e) if is_already_exists(e.as_ref()) => {
if self.name_is_held(op, Some(&parent), &name)? {
return Err(held_by_queued_change(&name));
}
// What held it may have let go since Drive answered
// (docs/BUGS.md B144).
match self.move_rename_remote(&uid, &parent, target) {
match try_move(target) {
Ok(()) => name.clone(),
Err(e) if is_already_exists(&e) => {
Err(e) if is_already_exists(e.as_ref()) => {
let alt = conflict_name(&name, now_secs());
warn!(%uid, name, alt, "destination already holds that name; using a conflict name");
self.move_rename_remote(&uid, &parent, Some(&alt))?;
try_move(Some(&alt))?;
self.adopt_drained_name(&uid, &alt);
self.log_activity(
ActivityKind::Rename,
@@ -853,7 +883,7 @@ impl Core {
);
alt
}
Err(e) => return Err(e.into()),
Err(e) => return Err(e),
}
}
// The destination folder is gone. Leaving the node in its current
@@ -862,7 +892,7 @@ impl Core {
// the user chose still applies there. Retrying the move could
// only fail again — the folder is not coming back — and would
// wedge the queue.
Err(e) if is_gone(&e) => {
Err(e) if is_gone(e.as_ref()) => {
warn!(%uid, name, %parent, "move destination is gone; leaving the node where it is");
let landed = match self.drain_rename_in_place(
op,
@@ -885,7 +915,7 @@ impl Core {
);
landed
}
Err(e) => return Err(e.into()),
Err(e) => return Err(e),
}
};
self.db.delete_op(op.id)?;
@@ -1349,7 +1379,11 @@ impl Core {
parent: Option<&NodeUid>,
name: &str,
) -> Result<bool, Box<dyn std::error::Error>> {
if self.db.has_pending_trash_named(name)? {
let parent_uid = parent.map(NodeUid::to_string);
if self
.db
.has_pending_trash_named(name, parent_uid.as_deref())?
{
return Ok(true);
}
if op.attempts >= NAME_HOLD_ATTEMPTS {
@@ -2695,6 +2729,29 @@ fn moved_elsewhere(meta: &RenameMeta, node: &Node, parent: &NodeUid, name: &str)
!at(&meta.original_parent_uid, original_name) && !at(&parent.to_string(), name)
}
/// A child can have a normal metadata record even when an ancestor is trashed.
/// Check both live parent chains before interpreting a refused move as an
/// access problem. Shared ancestors only need to be fetched once.
fn trashed_move_folder<'a>(
parents: impl IntoIterator<Item = &'a NodeUid>,
mut fetch: impl FnMut(&NodeUid) -> Option<Node>,
) -> Option<String> {
let mut seen = HashSet::new();
for parent in parents {
let mut next = Some(parent.clone());
while let Some(uid) = next.take()
&& seen.insert(uid.clone())
&& let Some(node) = fetch(&uid)
{
if node.trashed {
return Some(node.name);
}
next = node.parent_uid;
}
}
None
}
/// The answer from Drive somewhere in `e`'s chain, if it got that far.
fn proton_error<'e>(e: &'e (dyn std::error::Error + 'static)) -> Option<&'e ProtonError> {
std::iter::successors(Some(e), |e| e.source()).find_map(|e| e.downcast_ref::<ProtonError>())
@@ -2704,6 +2761,9 @@ fn proton_error<'e>(e: &'e (dyn std::error::Error + 'static)) -> Option<&'e Prot
/// could not be reached, which says nothing; otherwise the issue Drive's answer
/// is, or `Some(None)` for a failure that is not a refusal.
fn refusal(e: &(dyn std::error::Error + 'static)) -> Option<Option<SyncIssue>> {
if e.is::<TrashedMove>() {
return Some(Some(SyncIssue::Trashed));
}
let proton = proton_error(e)?;
// A refused session refuses everything, so it says nothing about this
// op either; the mount is offline for it (`Core::lost_link`).
@@ -3111,6 +3171,31 @@ mod tests {
}
}
#[test]
fn a_refused_move_identifies_a_trashed_ancestor_on_either_side() {
let source = folder_node("source", Some("old"));
let destination = folder_node("destination", Some("new"));
let mut old = folder_node("old", None);
let mut new = folder_node("new", None);
let find = |old: &Node, new: &Node| {
trashed_move_folder([&source.uid, &destination.uid], |uid| {
[&source, &destination, old, new]
.into_iter()
.find(|node| node.uid == *uid)
.cloned()
})
};
old.trashed = true;
assert_eq!(find(&old, &new).as_deref(), Some("old"));
old.trashed = false;
new.trashed = true;
let error = TrashedMove(find(&old, &new).unwrap());
assert_eq!(refusal(&error), Some(Some(SyncIssue::Trashed)));
assert!(error.to_string().contains("‘new’"));
new.trashed = false;
assert_eq!(find(&old, &new), None);
}
#[test]
fn queued_cross_parent_rename_retains_viewer_source_authority_after_local_move() {
use std::sync::atomic::{AtomicU64, Ordering};
+26 -1
View File
@@ -29,7 +29,7 @@ use proton_drive_rs::proton_sdk::error::Result;
use proton_drive_rs::proton_sdk::ids::{DriveEventId, NodeUid, ShareId};
use proton_drive_rs::{
AvailableNameTarget, Device, DeviceType, DriveEvent, DriveEventScopeId, Node, NodeMoveItem,
ProtonDriveClient, RevisionReader, Thumbnail, ThumbnailType,
ProtonDriveClient, RevisionReader, Thumbnail, ThumbnailType, TrashedNode,
};
/// One outcome per node of a batch request; see [`ProtonDriveClient::trash_nodes`].
@@ -180,6 +180,23 @@ pub(crate) trait DriveApi: Send + Sync {
async fn enumerate_trash_node_uids(&self) -> Result<Vec<NodeUid>>;
/// Permanently delete everything in the trash.
async fn empty_trash(&self) -> Result<()>;
/// As [`DriveApi::enumerate_nodes_light`], for nodes in the trash, each
/// with the time it was trashed when the server reports one.
async fn enumerate_trashed_nodes_light(&self, uids: &[NodeUid]) -> Result<Vec<TrashedNode>> {
Ok(self
.enumerate_nodes_light(uids)
.await?
.into_iter()
.map(|node| TrashedNode {
node,
trash_time: None,
})
.collect())
}
/// The account's storage, across every Proton product.
async fn quota(&self) -> Result<Quota>;
@@ -538,6 +555,14 @@ impl DriveApi for ProtonDriveClient {
ProtonDriveClient::enumerate_trash_node_uids(self).await
}
async fn enumerate_trashed_nodes_light(&self, uids: &[NodeUid]) -> Result<Vec<TrashedNode>> {
ProtonDriveClient::enumerate_trashed_nodes_light(self, uids).await
}
async fn empty_trash(&self) -> Result<()> {
ProtonDriveClient::empty_trash(self).await
}
async fn quota(&self) -> Result<Quota> {
ProtonDriveClient::quota(self).await
}
+8 -6
View File
@@ -2372,17 +2372,19 @@ impl ProtonFs {
}
}
}
// Offline: queue it. Trashing is the one mutation a user expects to work
// regardless — the file is gone from their point of view the moment the
// command returns. The same when the network goes
// away under the remote call below.
// Queue a folder trash while Drive still holds children that were moved
// out locally. The drain waits for those moves before sending it.
let queue_it = |reply: ReplyEmpty| match self.core.queue_trash(&uid, name, open_now) {
Ok(()) => reply.ok(),
Err(e) => reply.error(e),
};
if !self.core.sends_inline() {
queue_it(reply);
return;
return queue_it(reply);
}
match self.core.has_pending_moves_out_of(&uid) {
Ok(false) => {}
Ok(true) => return queue_it(reply),
Err(error) => return reply.error(error),
}
if let Err(e) = self.core.require_uid_writable(&uid) {
reply.error(e);
+653
View File
@@ -0,0 +1,653 @@
//! Host Proton Gallery's account and catalog services in the Drive daemon.
mod video;
use std::collections::HashSet;
use std::path::PathBuf;
use std::sync::{
Arc, RwLock,
atomic::{AtomicU64, Ordering},
};
use std::time::{Duration, Instant};
use pdfs_core::control::{
GalleryAction, GalleryAlbum, GalleryFolder, GalleryPlace, PhotoItem, PhotoKind, Response,
};
use pdfs_core::{CoreError, CoreResult};
use proton_drive_rs::proton_sdk::session::ProtonApiSession;
use proton_gallery_app_rs::{App, View};
use proton_gallery_rs::CancellationToken;
use proton_gallery_rs::model::{AssetKey, SourceId};
use tokio::sync::{Mutex, Notify, OnceCell};
use super::{Core, NodeUid, parse_uid};
pub(super) struct Gallery {
root: PathBuf,
session: ProtonApiSession,
cancel: CancellationToken,
app: OnceCell<App>,
checked: Mutex<Option<Instant>>,
visibility: RwLock<Option<Arc<Visibility>>>,
refresh: Notify,
access: AtomicU64,
video: video::Server,
}
#[derive(Default, PartialEq, Eq)]
pub(super) struct Visibility {
pub(super) media: HashSet<String>,
pub(super) albums: HashSet<String>,
}
impl Visibility {
pub(super) fn permits(&self, uid: &str) -> CoreResult<()> {
if self.media.contains(uid) || self.albums.contains(uid) {
return Err(CoreError::invalid("Unlock hidden albums to open this item"));
}
Ok(())
}
}
impl Gallery {
pub(super) fn new(root: PathBuf, session: ProtonApiSession) -> Self {
Self {
root,
session,
cancel: CancellationToken::new(),
app: OnceCell::new(),
checked: Mutex::new(None),
visibility: RwLock::default(),
refresh: Notify::new(),
access: AtomicU64::new(0),
video: video::Server::default(),
}
}
pub(super) fn cancel(&self) {
self.cancel.cancel();
self.video.close_all();
}
async fn app(&self, core: &Core) -> CoreResult<&App> {
self.app
.get_or_try_init(|| async {
App::open_hosted(
self.root.clone(),
self.session.clone(),
core.client.clone(),
self.cancel.child_token(),
)
.await
.map_err(gallery_error)
})
.await
}
async fn ensure_visibility(
&self,
core: &Core,
app: &App,
force: bool,
) -> CoreResult<Arc<Visibility>> {
if self.cancel.is_cancelled() {
return Err(CoreError::remote("The Gallery session expired"));
}
let mut checked = self.checked.lock().await;
if !force
&& let Some(instant) = *checked
&& instant.elapsed()
< if self.snapshot().is_some() {
Duration::from_secs(60)
} else {
Duration::from_secs(5)
}
{
return self.snapshot().ok_or_else(visibility_loading);
}
if self.snapshot().is_none()
&& app
.gallery_visibility_cached()
.await
.map_err(gallery_error)?
{
self.publish_visibility(core, self.read_visibility(core, app).await?);
}
let started = Instant::now();
let result = tokio::select! {
biased;
_ = self.cancel.cancelled() => Err(CoreError::remote("The Gallery session expired")),
result = tokio::time::timeout(Duration::from_secs(20), async {
app.refresh_gallery_configuration(false).await.map_err(gallery_error)?;
self.read_visibility(core, app).await
}) => result.unwrap_or_else(|_| Err(CoreError::remote("Gallery visibility check timed out"))),
};
tracing::debug!(
elapsed_ms = started.elapsed().as_millis() as u64,
success = result.is_ok(),
"Gallery visibility refresh"
);
*checked = Some(Instant::now());
if let Ok(visibility) = result {
self.publish_visibility(core, visibility.clone());
return Ok(visibility);
}
if !force && let Some(visibility) = self.snapshot() {
return Ok(visibility);
}
result
}
fn snapshot(&self) -> Option<Arc<Visibility>> {
self.visibility
.read()
.unwrap_or_else(|e| e.into_inner())
.clone()
}
fn publish_visibility(&self, core: &Core, visibility: Arc<Visibility>) {
let mut snapshot = self.visibility.write().unwrap_or_else(|e| e.into_inner());
if snapshot.as_ref() != Some(&visibility) {
*snapshot = Some(visibility);
drop(snapshot);
self.video.close_all();
core.events
.publish(&[pdfs_core::control::Topic::GalleryVisibilitySynced]);
}
}
async fn read_visibility(&self, core: &Core, app: &App) -> CoreResult<Arc<Visibility>> {
let account = self.account()?;
let mut visibility = Visibility::default();
for (volume, node) in app
.catalog
.hidden_assets(account.clone())
.await
.map_err(gallery_error)?
{
visibility
.media
.insert(NodeUid::new(volume.into(), node.into()).to_string());
}
visibility.media.extend(
core.db
.photo_group_members_for(&visibility.media)
.map_err(CoreError::from)?,
);
for album in app
.catalog
.hidden_configuration_albums(account)
.await
.map_err(gallery_error)?
{
visibility.albums.insert(
NodeUid::new(album.volume.as_str().into(), album.node.as_str().into()).to_string(),
);
}
Ok(Arc::new(visibility))
}
fn account(&self) -> CoreResult<proton_gallery_rs::model::AccountId> {
proton_gallery_rs::model::AccountId::new(self.session.user_id().as_str())
.map_err(gallery_error)
}
async fn page(
&self,
app: &App,
album: Option<GalleryPlace>,
proton: bool,
access: Option<u64>,
offset: usize,
limit: usize,
) -> CoreResult<Response> {
let private = access.is_some();
let parent = album
.as_ref()
.map(|a| (a.gallery.as_str(), a.album.as_str()));
let mut albums = Vec::new();
if offset == 0 && !proton {
if private && parent.is_none() {
albums.extend(
app.photo_albums_with_hidden(true)
.await
.map_err(gallery_error)?
.into_iter()
.filter(|entry| entry.settings.hidden)
.map(|entry| gallery_album(entry, true)),
);
albums.extend(
app.hidden_drive_albums()
.await
.map_err(gallery_error)?
.into_iter()
.map(|entry| gallery_album(entry, false)),
);
} else {
albums.extend(
app.drive_albums(parent, private)
.await
.map_err(gallery_error)?
.into_iter()
.map(|entry| gallery_album(entry, false)),
);
}
}
let (items, total) = if let Some(GalleryPlace { gallery, album }) = album {
let view = if proton && private {
let id = SourceId::new(&album).map_err(gallery_error)?;
if !app
.catalog
.album_settings(id)
.await
.map_err(gallery_error)?
.hidden
{
return Err(CoreError::not_found("Hidden album"));
}
View::PhotoAlbum(album)
} else {
if private
&& !app
.catalog
.drive_album_is_private(
SourceId::new(&gallery).map_err(gallery_error)?,
SourceId::new(&album).map_err(gallery_error)?,
)
.await
.map_err(gallery_error)?
{
return Err(CoreError::not_found("Hidden gallery folder"));
}
View::DriveAlbum { gallery, album }
};
let outline = app
.outline_with_visibility(view, false, private)
.await
.map_err(gallery_error)?;
let total = outline.len();
let items = app
.outline_items(&outline, offset.min(total), limit.clamp(1, 250))
.await
.map_err(gallery_error)?
.into_iter()
.flatten()
.map(|item| PhotoItem {
uid: asset_uid(&item.key),
capture_time: item.sort_time,
thumb_path: None,
name: Some(item.name),
ratio: item.aspect_ratio,
no_thumb: false,
kind: if item.is_video {
PhotoKind::Video
} else {
PhotoKind::Photo
},
favorite: item.favorite,
group_size: 1,
has_raw: false,
place_id: None,
})
.collect();
(items, total)
} else {
(Vec::new(), 0)
};
self.require_access(access)?;
Ok(Response::GalleryPage {
albums,
items,
total,
})
}
fn lock(&self) {
let _ = self
.access
.try_update(Ordering::SeqCst, Ordering::SeqCst, |value| {
Some((value | 1).wrapping_add(1))
});
self.video.close_all();
}
fn private_access(&self) -> CoreResult<u64> {
let access = self.access.load(Ordering::SeqCst);
if access & 1 == 0 {
return Err(CoreError::invalid("Unlock hidden albums to open this item"));
}
Ok(access)
}
fn require_access(&self, access: Option<u64>) -> CoreResult<()> {
if let Some(access) = access
&& self.private_access()? != access
{
return Err(CoreError::invalid("Hidden albums were locked"));
}
Ok(())
}
pub(super) async fn poll(self: Arc<Self>, core: Core) {
loop {
tokio::select! {
biased;
_ = self.cancel.cancelled() => break,
_ = self.refresh.notified() => {},
_ = tokio::time::sleep(Duration::from_secs(60)) => {}
}
if let Ok(Ok(app)) = tokio::time::timeout(Duration::from_secs(5), self.app(&core)).await
{
let _ = self.ensure_visibility(&core, app, false).await;
}
}
}
fn asset(&self, uid: &NodeUid) -> CoreResult<AssetKey> {
Ok(AssetKey {
account: self.account()?,
volume: proton_gallery_rs::model::VolumeId::new(uid.volume_id.as_str())
.map_err(gallery_error)?,
node: proton_gallery_rs::model::NodeId::new(uid.link_id.as_str())
.map_err(gallery_error)?,
})
}
}
fn gallery_album(entry: proton_gallery_rs::library::Album, proton: bool) -> GalleryAlbum {
GalleryAlbum {
place: GalleryPlace {
gallery: entry.gallery.to_string(),
album: entry.source.id.to_string(),
},
proton,
folder: GalleryFolder {
uid: NodeUid::new(
entry.source.volume.as_str().into(),
entry.source.root.as_str().into(),
)
.to_string(),
name: entry.settings.title.unwrap_or(entry.source.label),
},
cover: entry.cover.map(|asset| asset_uid(&asset.key)),
items: entry.direct_items,
folders: entry.child_albums,
hidden: entry.settings.hidden,
}
}
fn gallery_error(error: impl std::fmt::Display) -> CoreError {
CoreError::remote(error.to_string())
}
fn uid(value: &str) -> CoreResult<NodeUid> {
parse_uid(value).ok_or_else(|| CoreError::invalid("Invalid gallery node"))
}
fn done() -> Response {
Response::Ok {
message: "Gallery updated".into(),
}
}
fn asset_uid(asset: &AssetKey) -> String {
NodeUid::new(asset.volume.as_str().into(), asset.node.as_str().into()).to_string()
}
fn visibility_loading() -> CoreError {
CoreError::new(
pdfs_core::control::ErrorKind::Offline,
"Gallery visibility is loading. Try again shortly.",
)
}
impl Core {
pub(super) fn cached_gallery_visibility(&self) -> CoreResult<Arc<Visibility>> {
if self.gallery.is_none() {
return Ok(Arc::default());
}
let gallery = self.gallery_host()?;
match gallery.snapshot() {
Some(snapshot) => Ok(snapshot),
None => {
gallery.refresh.notify_one();
Err(visibility_loading())
}
}
}
pub(super) fn gallery_visibility(&self) -> CoreResult<Arc<Visibility>> {
if self.gallery.is_none() {
return Ok(Arc::default());
}
let gallery = self.gallery_host()?;
self.rt.block_on(async {
let app = gallery.app(self).await?;
gallery.ensure_visibility(self, app, false).await
})
}
fn gallery_host(&self) -> CoreResult<&Gallery> {
self.gallery
.as_deref()
.filter(|gallery| !gallery.cancel.is_cancelled())
.ok_or_else(|| CoreError::remote("The Gallery session is unavailable"))
}
pub(super) fn authenticate_hidden_gallery(
&self,
stream: &std::os::unix::net::UnixStream,
) -> CoreResult<Response> {
let gallery = self.gallery_host()?;
let access = gallery.access.load(Ordering::SeqCst);
let peer = rustix::net::sockopt::socket_peercred(stream)
.map_err(|_| CoreError::invalid("Couldn’t identify the authentication requester."))?;
if peer.uid != rustix::process::geteuid() {
return Err(CoreError::invalid(
"Use your Linux sign-in to unlock Hidden albums.",
));
}
let subject = proton_gallery_app_rs::desktop_auth::Subject::new(
peer.pid.as_raw_nonzero().get() as u32,
peer.uid.as_raw(),
)
.map_err(gallery_error)?;
self.rt
.block_on(subject.authenticate("io.narl.proton-drive-linux.unlock-hidden-albums"))
.map_err(gallery_error)?;
if gallery.cancel.is_cancelled() {
return Err(CoreError::invalid("The Gallery session has ended."));
}
gallery
.access
.compare_exchange(access, access | 1, Ordering::SeqCst, Ordering::SeqCst)
.map_err(|_| CoreError::invalid("Hidden albums were locked"))?;
Ok(done())
}
pub(super) fn drive_gallery(&self, action: GalleryAction) -> CoreResult<Response> {
let gallery = self.gallery_host()?;
let action = match action {
GalleryAction::Lock => {
gallery.lock();
return Ok(done());
}
other => other,
};
self.rt.block_on(async {
let app = gallery.app(self).await?;
gallery
.ensure_visibility(self, app, matches!(action, GalleryAction::Refresh))
.await?;
match action {
GalleryAction::AuthenticateHidden | GalleryAction::Lock => unreachable!(),
GalleryAction::List {
album,
offset,
limit,
} => gallery.page(app, album, false, None, offset, limit).await,
GalleryAction::PrivateList {
album,
proton,
offset,
limit,
} => {
let access = gallery.private_access()?;
gallery
.page(app, album, proton, Some(access), offset, limit)
.await
}
GalleryAction::Folders { parent } => {
let list = app
.browse_drive_folders(parent.as_deref().map(uid).transpose()?)
.await
.map_err(gallery_error)?;
Ok(Response::GalleryFolders {
folders: list
.children
.into_iter()
.map(|folder| GalleryFolder {
uid: folder.uid.to_string(),
name: folder.name,
})
.collect(),
})
}
GalleryAction::Add { uid: node } => {
app.add_drive_folder(uid(&node)?)
.await
.map_err(gallery_error)?;
gallery.ensure_visibility(self, app, true).await?;
Ok(done())
}
GalleryAction::Remove { gallery: id } => {
app.forget_drive_gallery(SourceId::new(id).map_err(gallery_error)?)
.await
.map_err(gallery_error)?;
gallery.ensure_visibility(self, app, true).await?;
Ok(done())
}
GalleryAction::Refresh => {
for album in app.drive_albums(None, true).await.map_err(gallery_error)? {
let root = NodeUid::new(
album.source.volume.as_str().into(),
album.source.root.as_str().into(),
);
app.scan_drive_folder(root).await.map_err(gallery_error)?;
}
gallery.ensure_visibility(self, app, true).await?;
Ok(done())
}
GalleryAction::Cover { album, uid: node } => {
let id = SourceId::new(&album.album).map_err(gallery_error)?;
let root = SourceId::new(&album.gallery).map_err(gallery_error)?;
app.checked_album(album.gallery, album.album)
.await
.map_err(gallery_error)?;
let access = if app
.catalog
.drive_album_is_private(root.clone(), id.clone())
.await
.map_err(gallery_error)?
{
Some(gallery.private_access()?)
} else {
None
};
let mut settings = app
.catalog
.album_settings(id.clone())
.await
.map_err(gallery_error)?;
gallery.require_access(access)?;
settings.cover = Some(gallery.asset(&uid(&node)?)?);
app.catalog
.set_album_settings_at(Some(root), id, settings)
.await
.map_err(gallery_error)?;
gallery.ensure_visibility(self, app, true).await?;
Ok(done())
}
}
})
}
pub(super) fn open_gallery_video(&self, node: &str, private: bool) -> CoreResult<Response> {
let gallery = self.gallery_host()?;
let uid = uid(node)?;
self.rt.block_on(async {
let app = gallery.app(self).await?;
let visibility = gallery.ensure_visibility(self, app, false).await?;
let access = if private {
Some(gallery.private_access()?)
} else {
visibility.permits(node)?;
None
};
let stream = app
.stream_video(&gallery.asset(&uid)?)
.await
.map_err(gallery_error)?;
let response = gallery.video.open(stream, &gallery.cancel).await?;
if let Err(error) = gallery.require_access(access) {
if let Response::GalleryVideo { token, .. } = &response {
gallery.video.close(token);
}
return Err(error);
}
Ok(response)
})
}
pub(super) fn gallery_image(&self, node: &str) -> CoreResult<Response> {
use base64::{Engine, engine::general_purpose::STANDARD};
let gallery = self.gallery_host()?;
let access = gallery.private_access()?;
let asset = gallery.asset(&uid(node)?)?;
self.rt.block_on(async {
let app = gallery.app(self).await?;
let bytes = tokio::time::timeout(Duration::from_secs(30), async {
if let Some(bytes) = app.preview(&asset).await? {
Ok(bytes)
} else {
app.original_in_memory(&asset, 8 * 1024 * 1024).await
}
})
.await
.map_err(|_| CoreError::remote("Proton took too long to open this photo"))?
.map_err(gallery_error)?;
gallery.require_access(Some(access))?;
Ok(Response::GalleryThumbnail {
data: Some(STANDARD.encode(bytes)),
})
})
}
pub(super) fn close_gallery_video(&self, token: &str) -> CoreResult<Response> {
self.gallery_host()?.video.close(token);
Ok(done())
}
pub(super) fn gallery_thumbnail(&self, node: &str, private: bool) -> CoreResult<Response> {
use base64::{Engine, engine::general_purpose::STANDARD};
let gallery = self.gallery_host()?;
let asset = gallery.asset(&uid(node)?)?;
self.rt.block_on(async {
let app = gallery.app(self).await?;
let visibility = gallery.ensure_visibility(self, app, false).await?;
let access = if private {
Some(gallery.private_access()?)
} else {
visibility.permits(node)?;
None
};
let data = app
.thumbnail(&asset)
.await
.map_err(gallery_error)?
.map(|bytes| STANDARD.encode(bytes));
gallery.require_access(access)?;
Ok(Response::GalleryThumbnail { data })
})
}
}
+205
View File
@@ -0,0 +1,205 @@
use std::{collections::VecDeque, io, sync::Arc, time::Duration};
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use http_body_util::{BodyExt, Empty, StreamBody, combinators::UnsyncBoxBody};
use hyper::{
Method, Request, Response, StatusCode,
body::{Bytes, Frame, Incoming},
header,
server::conn::http1,
service::service_fn,
};
use hyper_util::rt::TokioIo;
use pdfs_core::{CoreResult, control::Response as ControlResponse};
use proton_gallery_app_rs::video::VideoStream;
use proton_gallery_rs::CancellationToken;
use tokio::net::TcpListener;
type Body = UnsyncBoxBody<Bytes, io::Error>;
const LEASE: Duration = Duration::from_secs(4 * 60 * 60);
const MAX_STREAMS: usize = 4;
const CHUNK: u32 = 256 * 1024;
#[derive(Default)]
pub(super) struct Server {
active: parking_lot::Mutex<VecDeque<(String, CancellationToken)>>,
}
impl Server {
pub(super) fn close(&self, token: &str) {
let mut active = self.active.lock();
if let Some(index) = active.iter().position(|(id, _)| id == token)
&& let Some((_, cancel)) = active.remove(index)
{
cancel.cancel();
}
}
pub(super) fn close_all(&self) {
for (_, cancel) in self.active.lock().drain(..) {
cancel.cancel();
}
}
pub(super) async fn open(
&self,
stream: Arc<VideoStream>,
account: &CancellationToken,
) -> CoreResult<ControlResponse> {
let listener = TcpListener::bind((std::net::Ipv4Addr::LOCALHOST, 0))
.await
.map_err(super::gallery_error)?;
let address = listener.local_addr().map_err(super::gallery_error)?;
let mut nonce = [0; 32];
getrandom::fill(&mut nonce).map_err(super::gallery_error)?;
let token = URL_SAFE_NO_PAD.encode(nonce);
let path: Arc<str> = format!("/{token}").into();
let cancel = account.child_token();
{
let mut active = self.active.lock();
active.retain(|(_, cancel)| !cancel.is_cancelled());
if active.len() >= MAX_STREAMS
&& let Some((_, oldest)) = active.pop_front()
{
oldest.cancel();
}
active.push_back((token.clone(), cancel.clone()));
}
let size = stream.length();
tokio::spawn(async move {
let mut connections = tokio::task::JoinSet::new();
let lifetime = tokio::time::sleep(LEASE);
tokio::pin!(lifetime);
loop {
tokio::select! {
biased;
_ = cancel.cancelled() => break,
_ = &mut lifetime => break,
_ = connections.join_next(), if !connections.is_empty() => {},
incoming = listener.accept(), if connections.len() < 8 => {
let Ok((socket, _)) = incoming else { break };
let stream = stream.clone();
let path = path.clone();
connections.spawn(async move {
let service = service_fn(move |request| reply(request, stream.clone(), path.clone()));
let _ = http1::Builder::new()
.timer(hyper_util::rt::TokioTimer::new())
.header_read_timeout(Duration::from_secs(10))
.keep_alive(false)
.serve_connection(TokioIo::new(socket), service).await;
});
}
}
}
stream.stop();
cancel.cancel();
connections.abort_all();
});
Ok(ControlResponse::GalleryVideo {
url: format!("http://{address}/{token}"),
token,
size,
})
}
}
fn range(value: Option<&str>, size: u64) -> Option<(u64, u64)> {
let Some(value) = value else {
return Some((0, size));
};
let (first, last) = value.strip_prefix("bytes=")?.split_once('-')?;
if first.is_empty() {
let suffix = last.parse::<u64>().ok()?.min(size);
return (suffix > 0).then_some((size - suffix, size));
}
let start = first.parse::<u64>().ok()?;
let end = if last.is_empty() {
size
} else {
last.parse::<u64>().ok()?.saturating_add(1).min(size)
};
(start < end).then_some((start, end))
}
fn empty() -> Body {
Empty::new().map_err(|never| match never {}).boxed_unsync()
}
async fn reply(
request: Request<Incoming>,
stream: Arc<VideoStream>,
path: Arc<str>,
) -> Result<Response<Body>, hyper::http::Error> {
let response = Response::builder().header(header::CACHE_CONTROL, "no-store");
if request
.uri()
.path_and_query()
.is_none_or(|actual| actual.as_str() != path.as_ref())
{
return response.status(StatusCode::NOT_FOUND).body(empty());
}
if !matches!(*request.method(), Method::HEAD | Method::GET) {
return response
.status(StatusCode::METHOD_NOT_ALLOWED)
.body(empty());
}
let size = stream.length();
let requested = request.headers().get(header::RANGE);
let parsed = match requested {
Some(value) => value
.to_str()
.ok()
.and_then(|value| range(Some(value), size)),
None => range(None, size),
};
let Some((start, end)) = parsed else {
return response
.status(StatusCode::RANGE_NOT_SATISFIABLE)
.header(header::CONTENT_RANGE, format!("bytes */{size}"))
.body(empty());
};
let response = response
.header(header::ACCEPT_RANGES, "bytes")
.header(header::CONTENT_LENGTH, end - start)
.header(header::CONTENT_TYPE, "application/octet-stream");
let response = if requested.is_some() {
response.status(StatusCode::PARTIAL_CONTENT).header(
header::CONTENT_RANGE,
format!("bytes {start}-{}/{size}", end - 1),
)
} else {
response
};
if request.method() == Method::HEAD {
return response.body(empty());
}
let body = futures::stream::try_unfold((stream, start), move |(stream, at)| async move {
if at == end {
return Ok(None);
}
let read = (end - at).min(u64::from(CHUNK)) as u32;
let bytes = stream.read(at, read).await.map_err(io::Error::other)?;
if bytes.is_empty() {
return Err(io::Error::from(io::ErrorKind::UnexpectedEof));
}
let next = at + bytes.len() as u64;
Ok(Some((Frame::data(Bytes::from(bytes)), (stream, next))))
});
response.body(BodyExt::boxed_unsync(StreamBody::new(body)))
}
#[cfg(test)]
mod tests {
use super::range;
#[test]
fn resolves_seek_ranges_and_rejects_invalid_ranges() {
assert_eq!(range(None, 10), Some((0, 10)));
assert_eq!(range(Some("bytes=3-5"), 10), Some((3, 6)));
assert_eq!(range(Some("bytes=9-"), 10), Some((9, 10)));
assert_eq!(range(Some("bytes=-3"), 10), Some((7, 10)));
assert_eq!(range(Some("bytes=10-"), 10), None);
assert_eq!(range(Some("bytes=0-1,3-4"), 10), None);
}
}
+129
View File
@@ -0,0 +1,129 @@
//! Hold off a shutdown while queued changes exist only on this computer.
//!
//! A change made through the mount is staged here and sent by the drain. Until
//! it lands, this computer has the only copy, so powering off then risks it
//! more than the user expects (`docs/BUGS.md` B66). While the queue holds work
//! the drain can still send, the daemon takes a logind shutdown inhibitor
//! through `systemd-inhibit`, which desktops show in their power-off dialog
//! with the reason. It is let go as soon as the queue empties.
//!
//! Work that cannot drain on its own — a temporary file waiting for its final
//! name, or a change Drive refused or that keeps failing — never holds the
//! lock, or a shutdown would be held up for something that is not coming.
//! Without `systemd-inhibit` nothing happens.
use std::process::{Child, Command, Stdio};
use std::time::Duration;
use pdfs_core::db::PendingCounts;
use tracing::{debug, warn};
use super::Core;
/// How often the queue is looked at.
const CHECK_EVERY: Duration = Duration::from_secs(5);
/// Changes the drain can still send by itself.
fn sendable(counts: &PendingCounts) -> i64 {
(counts.uploads + counts.changes - counts.parked - counts.failing).max(0)
}
/// Run until the daemon stops, holding the inhibitor while changes wait.
pub(crate) fn run(core: Core) {
let mut held: Option<Child> = None;
loop {
let waiting = core
.db
.pending_op_counts()
.map(|counts| sendable(&counts))
.unwrap_or(0);
// The helper can go away (logind restarted, or it was killed); the
// lock is then taken again on the next round.
if let Some(child) = &mut held
&& !matches!(child.try_wait(), Ok(None))
{
held = None;
}
match (&mut held, waiting > 0) {
(None, true) => held = take(waiting),
(Some(child), false) => {
release(child);
held = None;
}
_ => {}
}
if !core.shutdown.sleep(CHECK_EVERY) {
break;
}
}
if let Some(mut child) = held {
release(&mut child);
}
}
fn take(waiting: i64) -> Option<Child> {
let why = if waiting == 1 {
"Sending 1 change to Proton Drive".to_string()
} else {
format!("Sending {waiting} changes to Proton Drive")
};
match Command::new("systemd-inhibit")
.args([
"--what=shutdown",
"--who=Proton Drive",
&format!("--why={why}"),
"--mode=block",
"sleep",
"infinity",
])
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.spawn()
{
Ok(child) => {
debug!(waiting, "holding off shutdown while changes are queued");
Some(child)
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
Err(error) => {
warn!(%error, "taking the shutdown inhibitor failed");
None
}
}
}
fn release(child: &mut Child) {
let _ = child.kill();
let _ = child.wait();
debug!("queue drained; shutdown no longer held off");
}
#[cfg(test)]
mod tests {
use super::*;
fn counts(uploads: i64, changes: i64, parked: i64, failing: i64) -> PendingCounts {
PendingCounts {
uploads,
changes,
parked,
failing,
last_error: None,
}
}
#[test]
fn only_work_the_drain_can_send_holds_off_a_shutdown() {
assert_eq!(sendable(&counts(0, 0, 0, 0)), 0);
assert_eq!(sendable(&counts(2, 1, 0, 0)), 3);
assert_eq!(sendable(&counts(1, 0, 1, 0)), 0);
assert_eq!(sendable(&counts(2, 0, 0, 2)), 0);
assert_eq!(sendable(&counts(3, 2, 1, 1)), 3);
}
#[test]
fn counts_that_overlap_never_go_negative() {
assert_eq!(sendable(&counts(1, 0, 1, 1)), 0);
}
}
+216 -53
View File
@@ -75,7 +75,7 @@ use proton_drive_rs::proton_sdk::error::ProtonError;
use proton_drive_rs::proton_sdk::ids::{DriveEventId, LinkId, NodeUid, ShareId, VolumeId};
use proton_drive_rs::{
AvailableNameTarget, DriveEvent, DriveEventScopeId, MemberRole, Node, NodeKind, NodeMoveItem,
ProtonDriveClient, ProtonPhotosClient, SharedWithMeItem, ThumbnailType,
ProtonDriveClient, ProtonPhotosClient, SharedWithMeItem, ThumbnailType, TrashedNode,
};
mod albums;
@@ -91,6 +91,8 @@ mod drive;
use drive::{DriveApi, RevisionRead};
mod events;
mod filesystem;
mod gallery;
mod inhibit;
mod keeper;
mod link;
pub use filesystem::ProtonFs;
@@ -499,6 +501,7 @@ struct Core {
/// The concrete client, for what [`DriveApi`] does not cover: sharing,
/// Photos, devices, invitations and public links.
client: ProtonDriveClient,
gallery: Option<Arc<gallery::Gallery>>,
rt: tokio::runtime::Handle,
/// The daemon's My Files root. Forked on-demand sessions retain this value
/// so ownership checks never mistake their own mount root for account scope.
@@ -4012,6 +4015,15 @@ impl Core {
Ok(())
}
fn has_pending_moves_out_of(&self, uid: &NodeUid) -> Result<bool, Errno> {
self.db
.has_pending_moves_out_of(&uid.to_string())
.map_err(|error| {
error!(%uid, %error, "checking for moves out of a trash target failed");
Errno::EIO
})
}
/// Queue trashing a node the server knows about, and drop it from the tree
/// now.
///
@@ -4599,6 +4611,7 @@ impl Core {
shared_by_unverified: false,
trashed_at: 0,
trashed_from: None,
trashed_parent: None,
issue: state::ino_lid(ino)
.and_then(|lid| issues.get(&lid))
.map(|issue| SyncIssue::parse(issue)),
@@ -5390,6 +5403,11 @@ impl Core {
&parent_uid,
)
.map_err(|error| self.errno_error(error, "trash access"))?;
if self.db.has_pending_moves_out_of(&uid.to_string())? {
return Err(CoreError::conflict(
"wait for pending moves out of this folder to sync before trashing it",
));
}
self.rt
.block_on(self.drive.trash_nodes(std::slice::from_ref(&uid)))
.and_then(batch::into_unit)
@@ -5454,11 +5472,13 @@ impl Core {
} else {
uid
};
// Local-first or offline, or the network goes away under the call:
// queue the trash. The rename that follows queues as well, behind it.
// Local-first or offline: queue the trash and the rename that follows.
if !self.sends_inline() {
return self.queue_trash(uid, name, open);
}
if self.has_pending_moves_out_of(uid)? {
return Err(Errno::EBUSY);
}
self.require_uid_writable(uid)?;
if let Err(e) = self
.block_on_bounded(self.drive.trash_nodes(std::slice::from_ref(uid)))
@@ -5585,20 +5605,29 @@ impl Core {
self.await_trash_refresh(TRASH_FIRST_WAIT);
}
let items = self.db.trash_list().map_err(CoreError::from)?;
let trashed: HashSet<&str> = items.iter().map(|item| item.uid.as_str()).collect();
let trashed_parents: Vec<Option<String>> = items
.iter()
.map(|item| {
item.parent_uid
.clone()
.filter(|parent| trashed.contains(parent.as_str()))
})
.collect();
// Most of a trash comes from a handful of folders.
let mut origins: HashMap<String, Option<String>> = HashMap::new();
Ok(self
.db
.trash_list()
.map_err(CoreError::from)?
Ok(items
.into_iter()
.map(|item| DirEntry {
.zip(trashed_parents)
.map(|(item, trashed_parent)| DirEntry {
trashed_from: item.parent_uid.as_ref().and_then(|parent| {
origins
.entry(parent.clone())
.or_insert_with(|| self.db.node_path(parent).ok().flatten())
.clone()
}),
trashed_parent,
trashed_at: item.trashed_at.unwrap_or(0),
name: item.name,
is_dir: item.is_dir,
@@ -5620,13 +5649,86 @@ impl Core {
.collect())
}
/// What a trashed folder holds: its children on the server, and the
/// stored trash entries trashed inside it before it went.
fn list_trashed_folder(&self, uid: &str) -> CoreResult<Vec<DirEntry>> {
let folder =
parse_uid(uid).ok_or_else(|| CoreError::invalid(format!("invalid uid: {uid}")))?;
let children = self
.rt
.block_on(within_trash_call(
"enumerate trashed folder",
self.drive.enumerate_folder_children_node_uids(&folder),
))?
.map_err(|e| CoreError::from_api(&e, "enumerate trashed folder"))?;
let mut nodes = Vec::with_capacity(children.len());
for chunk in children.chunks(TRASH_MATERIALIZE_CHUNK) {
nodes.extend(
self.rt
.block_on(within_trash_call(
"enumerate nodes",
self.drive.enumerate_trashed_nodes_light(chunk),
))?
.map_err(|e| CoreError::from_api(&e, "enumerate nodes"))?,
);
}
let stored = self.db.trash_list().map_err(CoreError::from)?;
let stored_at: HashMap<&str, Option<i64>> = stored
.iter()
.map(|item| (item.uid.as_str(), item.trashed_at))
.collect();
let mut entries: Vec<DirEntry> = nodes
.iter()
.map(|TrashedNode { node, trash_time }| {
let uid = node.uid.to_string();
let trashed_at = trash_time
.or_else(|| stored_at.get(uid.as_str()).copied().flatten())
.unwrap_or(0);
trashed_entry(
uid,
node.name.clone(),
node.is_folder(),
node_size(node) as i64,
node.modification_time,
trashed_at,
folder.to_string(),
)
})
.collect();
let listed: HashSet<String> = entries.iter().map(|e| e.uid.clone()).collect();
entries.extend(
stored
.into_iter()
.filter(|item| {
item.parent_uid.as_deref() == Some(uid) && !listed.contains(&item.uid)
})
.map(|item| {
trashed_entry(
item.uid,
item.name,
item.is_dir,
item.size,
item.mtime,
item.trashed_at.unwrap_or(0),
folder.to_string(),
)
}),
);
entries.sort_by(|a, b| {
b.is_dir
.cmp(&a.is_dir)
.then_with(|| a.name.to_lowercase().cmp(&b.name.to_lowercase()))
});
Ok(entries)
}
/// Re-fetch the trash listing from the server and persist it.
///
/// Materialized in chunks of [`TRASH_MATERIALIZE_CHUNK`], each persisted as
/// it lands: decrypting a trashed node costs an S2K unlock, so a trash of a
/// few thousand nodes takes minutes, and an all-or-nothing write would throw
/// that away on the first failure and show the user nothing until the very
/// end. Every batch is a usable listing.
/// Only nodes the stored listing does not hold yet are read, and light: a
/// trashed file needs no content key, so none is unlocked. Each chunk of
/// [`TRASH_MATERIALIZE_CHUNK`] is stored as it lands, so a large trash
/// fills the page while it loads. Rows the server no longer lists go once
/// the whole listing is in, never earlier.
async fn refresh_trash(&self) -> CoreResult<()> {
let started = Instant::now();
let uids = within_trash_call("enumerate trash", self.drive.enumerate_trash_node_uids())
@@ -5638,8 +5740,6 @@ impl Core {
"trash refresh: enumerated"
);
// What the listing held before, and whether it was the whole trash: the
// chunks below overwrite it, so both are read up front.
let seen = self.db.trash_seen().unwrap_or_default();
let complete = self
.db
@@ -5647,36 +5747,51 @@ impl Core {
.ok()
.flatten()
.is_some();
let _ = self.db.clear_state(TRASH_COMPLETE_MS);
let now = now_secs();
if uids.is_empty() {
self.db.trash_replace(&[]).map_err(CoreError::from)?;
}
let mut items: Vec<StoredTrash> = Vec::with_capacity(uids.len());
for chunk in uids.chunks(TRASH_MATERIALIZE_CHUNK) {
let chunk_started = Instant::now();
let nodes = within_trash_call("enumerate nodes", self.drive.enumerate_nodes(chunk))
.await?
.map_err(|e| CoreError::from_api(&e, "enumerate nodes"))?;
items.extend(nodes.into_iter().map(|node| StoredTrash {
uid: node.uid.to_string(),
name: node.name.clone(),
is_dir: node.is_folder(),
size: node_size(&node) as i64,
mtime: node.modification_time,
// The folder the node will go back to, which is also what tells
// a restore whether that folder is itself in the trash.
parent_uid: node.parent_uid.as_ref().map(|uid| uid.to_string()),
trashed_at: first_seen_in_trash(&seen, &node.uid.to_string(), complete, now),
}));
// Cumulative, so the table is always a prefix of the real trash
// rather than a mix of this refresh and the last one.
self.db.trash_replace(&items).map_err(CoreError::from)?;
let listed: HashSet<String> = uids.iter().map(ToString::to_string).collect();
let removed = self.db.trash_retain(&listed).map_err(CoreError::from)?;
if removed > 0 {
self.trash_progress.notify_waiters();
}
let missing: Vec<NodeUid> = uids
.into_iter()
.filter(|uid| !seen.contains_key(&uid.to_string()))
.collect();
let mut materialized = 0usize;
for chunk in missing.chunks(TRASH_MATERIALIZE_CHUNK) {
let chunk_started = Instant::now();
let nodes = within_trash_call(
"enumerate nodes",
self.drive.enumerate_trashed_nodes_light(chunk),
)
.await?
.map_err(|e| CoreError::from_api(&e, "enumerate nodes"))?;
let items: Vec<StoredTrash> = nodes
.into_iter()
.map(|TrashedNode { node, trash_time }| {
let uid = node.uid.to_string();
StoredTrash {
trashed_at: trash_time
.or_else(|| first_seen_in_trash(&seen, &uid, complete, now)),
uid,
name: node.name.clone(),
is_dir: node.is_folder(),
size: node_size(&node) as i64,
mtime: node.modification_time,
// The folder the node will go back to, which is also what
// tells a restore whether that folder is itself in the trash.
parent_uid: node.parent_uid.as_ref().map(|uid| uid.to_string()),
}
})
.collect();
materialized += items.len();
self.db.trash_upsert(&items).map_err(CoreError::from)?;
self.trash_progress.notify_waiters();
self.events.publish(&[Topic::Trash]);
debug!(
materialized = items.len(),
total = uids.len(),
materialized,
total = missing.len(),
chunk_ms = chunk_started.elapsed().as_millis() as u64,
"trash refresh: batch"
);
@@ -5685,7 +5800,9 @@ impl Core {
let _ = self.db.set_state_i64(TRASH_SYNCED_MS, now_ms());
let _ = self.db.set_state_i64(TRASH_COMPLETE_MS, now_ms());
info!(
count = items.len(),
count = listed.len(),
new = materialized,
removed,
elapsed_ms = started.elapsed().as_millis() as u64,
"trash refresh: done"
);
@@ -5754,7 +5871,7 @@ impl Core {
/// Drop the persisted photos timeline's freshness stamp, so the next timeline
/// read fetches rather than serving what it already has.
fn invalidate_photos(&self) {
let _ = self.db.clear_state(PHOTOS_SYNCED_MS);
let _ = self.db.photos_invalidate();
}
/// Drop one folder's cached child listing (`rel` is mountpoint-relative), so
@@ -5866,6 +5983,8 @@ impl Core {
}
}
});
let back: Vec<String> = restored.iter().map(ToString::to_string).collect();
let _ = self.db.trash_remove(&back);
self.invalidate_trash();
Ok(restored.len())
}
@@ -5879,6 +5998,7 @@ impl Core {
// daemon interrupted mid-batch has already forgotten exactly the nodes
// the server destroyed — never more, never fewer.
let mut deleted = 0usize;
let mut gone: Vec<String> = Vec::new();
let mut first_error: Option<ProtonError> = None;
self.rt
.block_on(async {
@@ -5888,6 +6008,7 @@ impl Core {
match outcome {
Ok(()) => {
self.drop_local(std::slice::from_ref(&uid));
gone.push(uid.to_string());
deleted += 1;
}
Err(e) => {
@@ -5904,23 +6025,34 @@ impl Core {
{
return Err(CoreError::from_api(&error, "delete"));
}
let _ = self.db.trash_remove(&gone);
self.invalidate_trash();
Ok(deleted)
}
/// Permanently delete everything in the trash. The uids are listed first so
/// the blobs of items trashed by *another* client — which this daemon may
/// still hold in its cache — are reclaimed too, not just the ones it trashed.
/// Permanently delete everything in the trash.
///
/// The stored listing says what goes, so the blobs of items trashed by
/// another client, which this daemon may still hold, are reclaimed too. It
/// is not read from the server again first: on a large trash that listing
/// is what kept the delete from ever being sent (`docs/BUGS.md` B293).
/// The stored listing is cleared at once, and the server is asked again
/// after [`TRASH_TTL`], so anything it still holds then shows up again.
fn empty_trash(&self) -> CoreResult<usize> {
let uids = self
.rt
.block_on(self.drive.enumerate_trash_node_uids())
.map_err(|e| CoreError::from_api(&e, "enumerate trash"))?;
let uids: Vec<String> = self
.db
.trash_seen()
.map_err(CoreError::from)?
.into_keys()
.collect();
self.rt
.block_on(self.client.empty_trash())
.block_on(within_trash_call("empty trash", self.drive.empty_trash()))?
.map_err(|e| CoreError::from_api(&e, "empty trash"))?;
self.drop_local(&uids);
self.invalidate_trash();
let parsed: Vec<NodeUid> = uids.iter().filter_map(|uid| parse_uid(uid)).collect();
self.drop_local(&parsed);
let _ = self.db.trash_replace(&[]);
let _ = self.db.set_state_i64(TRASH_SYNCED_MS, now_ms());
self.events.publish(&[Topic::Trash]);
Ok(uids.len())
}
@@ -6822,6 +6954,37 @@ async fn within_trash_call<T>(what: &str, call: impl Future<Output = T>) -> Core
})
}
/// An entry inside a trashed folder, for [`Core::list_trashed_folder`].
fn trashed_entry(
uid: String,
name: String,
is_dir: bool,
size: i64,
mtime: i64,
trashed_at: i64,
parent: String,
) -> DirEntry {
DirEntry {
name,
is_dir,
size: size.max(0) as u64,
modified: mtime,
pinned: false,
pinned_by_folder: false,
cached: false,
uid,
path: String::new(),
role: String::new(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
trashed_at,
trashed_from: None,
trashed_parent: Some(parent),
issue: None,
}
}
/// When a node a trash refresh found went to the trash, as far as the daemon
/// can tell. A node the stored listing already had keeps the time it had. A new
/// one went there since the last refresh, so now is close, but only when that
+3
View File
@@ -232,6 +232,9 @@ impl Core {
/// Unless the keyring holds a newer session than the refused one: then
/// the mount ends, for the daemon to mount again on that one.
pub(crate) fn mark_session_expired(&self, what: &str, error: &dyn std::fmt::Display) {
if let Some(gallery) = &self.gallery {
gallery.cancel();
}
if self.link.resume_stored.load(Ordering::Relaxed) {
return self.mark_offline(what, error);
}
+23
View File
@@ -310,6 +310,7 @@ enum Start {
pub struct MountOptions {
/// Proton account the session belongs to, for display and per-user paths.
pub username: String,
pub gallery_session: Option<proton_drive_rs::proton_sdk::session::ProtonApiSession>,
/// What the background conflict sweep may do. See `docs/BUGS.md` B71.
pub sweep_mode: SweepMode,
/// Upload and download caps in bytes per second (`0` = no cap).
@@ -456,6 +457,7 @@ pub(crate) fn mount_with(
) -> std::io::Result<MountOutcome> {
let MountOptions {
username,
gallery_session,
sweep_mode,
upload_limit,
download_limit,
@@ -480,9 +482,13 @@ pub(crate) fn mount_with(
let (sync_tx, sync_rx) = std::sync::mpsc::channel::<sync::SyncMsg>();
let paused_until = pause::load_paused_until(&db);
let gallery_dir = control_socket.with_file_name("gallery-catalog");
let gallery =
gallery_session.map(|session| Arc::new(super::gallery::Gallery::new(gallery_dir, session)));
let core = Core {
drive,
client,
gallery,
rt: rt.clone(),
maintenance: Arc::new(Mutex::new(Default::default())),
primary_root_uid: root.uid.clone(),
@@ -636,6 +642,9 @@ pub(crate) fn mount_with(
// Keep the launcher prompt's "This computer" index fresh. Its own thread:
// the walk is I/O-heavy and must never sit in front of a FUSE callback.
if matches!(host, Host::Daemon) {
if let Some(gallery) = &core.gallery {
rt.spawn(gallery.clone().poll(core.clone()));
}
let db = core.db.clone();
let indexing = core.indexing.clone();
let transfers = core.transfers.clone();
@@ -784,6 +793,17 @@ pub(crate) fn mount_with(
warn!(%error, "start My Files share-id repair failed");
}
}
// Hold off a shutdown while changes exist only here (B66). Only the real
// daemon: a simulation's queue is not the user's data.
if matches!(host, Host::Daemon) {
let inhibit_core = core.clone();
if let Err(error) = std::thread::Builder::new()
.name("pdfs-inhibit".into())
.spawn(move || crate::inhibit::run(inhibit_core))
{
warn!(%error, "start the shutdown inhibitor failed");
}
}
// Stop signals (SIGTERM from `systemctl --user stop`, SIGINT from Ctrl-C)
// are delivered onto the async runtime; bridge them onto a sync channel so
@@ -954,6 +974,9 @@ fn keep_or_abandon(
/// Signal every worker to stop, then join them up to [`WORKER_JOIN_DEADLINE`].
fn halt_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::JoinHandle<()>>) {
core.shutdown.stop();
if let Some(gallery) = &core.gallery {
gallery.cancel();
}
core.wake_drain();
let _ = core.sync_tx.send(sync::SyncMsg::Stop);
// Connect while the socket still exists — that is the whole point of the
+139 -171
View File
@@ -32,7 +32,7 @@ use tracing::{debug, info, warn};
use super::drive::RevisionRead;
use super::{
Core, PHOTOS_AVAILABLE, PHOTOS_SYNCED_MS, TIMELINE_ENRICH_CHUNK, TIMELINE_TTL, WriteAuthority,
Core, PHOTOS_AVAILABLE, PHOTOS_SYNCED_MS, TIMELINE_ENRICH_CHUNK, WriteAuthority, link,
node_size, now_ms, parse_uid,
};
@@ -1144,18 +1144,6 @@ impl Drop for ProgressGuard<'_> {
}
}
/// What a timeline enrichment pass resolves for one photo, beyond what the
/// timeline listing already carries.
#[derive(Clone, Debug)]
struct PhotoMeta {
name: Option<String>,
media_type: Option<String>,
favorite: bool,
content_hash: Option<String>,
/// Where the photo was taken, as its uploader recorded it.
location: Option<(f64, f64)>,
}
impl Core {
/// [`upload_thumbnails`] for a file about to upload, staging any RAW copy
/// under this cache. Blocking.
@@ -1175,6 +1163,7 @@ impl Core {
.unwrap_or_default()
}
#[allow(clippy::too_many_arguments)]
pub(crate) fn photos_timeline(
&self,
offset: usize,
@@ -1183,21 +1172,29 @@ impl Core {
range: Option<(i64, i64)>,
favorites: bool,
not_in_album: bool,
excluded: &str,
) -> CoreResult<Option<Vec<PhotoItem>>> {
let count = self.db.photos_count().map_err(CoreError::from)?;
if count == 0 {
// Nothing to serve, so this one request has to wait for the fetch —
// unless we already know the account has no photos volume and the
// answer is a fresh "no".
let known_empty = self.db.state_i64(PHOTOS_AVAILABLE).ok().flatten() == Some(0);
if known_empty && !self.listing_stale(PHOTOS_SYNCED_MS, TIMELINE_TTL) {
return Ok(None);
let listed = self
.db
.state_i64(PHOTOS_SYNCED_MS)
.map_err(CoreError::from)?
.is_some();
if !listed {
if count == 0 {
if !self.refresh_timeline_joined()? {
return Ok(None);
}
} else {
self.spawn_timeline_refresh();
}
if !self.refresh_timeline_joined()? {
return Ok(None);
}
} else if self.listing_stale(PHOTOS_SYNCED_MS, TIMELINE_TTL) {
self.spawn_timeline_refresh();
} else if self
.db
.state_i64(PHOTOS_AVAILABLE)
.map_err(CoreError::from)?
== Some(0)
{
return Ok(None);
}
// The filter is only as good as the album contents on disk, and those
// are fetched lazily, per album opened. Fill in the missing ones before
@@ -1208,7 +1205,15 @@ impl Core {
let page = self
.db
.photos_page(offset, limit, kind, range, favorites, not_in_album)
.photos_page_visible(
offset,
limit,
kind,
range,
favorites,
not_in_album,
excluded,
)
.map_err(CoreError::from)?;
Ok(Some(page.into_iter().map(|p| self.photo_item(p)).collect()))
}
@@ -1365,8 +1370,11 @@ impl Core {
/// The towns the photos were taken in, as wire items. A town the bundled
/// table no longer knows (the stored id came from an older table) is left
/// out rather than shown without a name.
pub(crate) fn photo_places(&self) -> CoreResult<Vec<PlaceInfo>> {
let places = self.db.photo_places().map_err(CoreError::from)?;
pub(crate) fn photo_places(&self, excluded: &str) -> CoreResult<Vec<PlaceInfo>> {
let places = self
.db
.photo_places_visible(excluded)
.map_err(CoreError::from)?;
Ok(places
.into_iter()
.filter_map(|place| {
@@ -1390,10 +1398,11 @@ impl Core {
id: u32,
offset: usize,
limit: usize,
excluded: &str,
) -> CoreResult<Vec<PhotoItem>> {
let photos = self
.db
.photos_at_place(id, offset, limit)
.photos_at_place_visible(id, offset, limit, excluded)
.map_err(CoreError::from)?;
Ok(photos.into_iter().map(|p| self.photo_item(p)).collect())
}
@@ -1417,13 +1426,17 @@ impl Core {
/// Photos taken on today's date in earlier years, as wire items. Served
/// from the stored timeline only: the strip is a nicety, not worth a wait
/// on the network, and the timeline's own load keeps that store fresh.
pub(crate) fn photos_on_this_day(&self, limit: usize) -> CoreResult<Vec<PhotoItem>> {
pub(crate) fn photos_on_this_day(
&self,
limit: usize,
excluded: &str,
) -> CoreResult<Vec<PhotoItem>> {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map_or(0, |d| d.as_secs() as i64);
let photos = self
.db
.photos_on_this_day(now, limit)
.photos_on_this_day_visible(now, limit, excluded)
.map_err(CoreError::from)?;
Ok(photos.into_iter().map(|p| self.photo_item(p)).collect())
}
@@ -2477,162 +2490,113 @@ impl Core {
}
}
/// Whether the listing stamped under `key` is older than `ttl` (or was never
/// fetched).
/// Commit the lightweight listing before resolving pending nodes. The
/// feed generation fences both commits against events received in flight.
pub(crate) async fn refresh_timeline(&self) -> CoreResult<bool> {
let started = Instant::now();
let generation = self
.db
.photo_projection_generation()
.map_err(CoreError::from)?;
let photos = self.photos();
if photos
.get_photos_root()
.await
.map_err(|e| CoreError::from_api(&e, "photos root"))?
if self
.db
.state_i64(PHOTOS_SYNCED_MS)
.map_err(CoreError::from)?
.is_none()
{
let _ = self.db.set_state_i64(PHOTOS_AVAILABLE, 0);
let _ = self.db.set_state_i64(PHOTOS_SYNCED_MS, now_ms());
let available = link::bounded(photos.get_photos_root())
.await
.map_err(|e| CoreError::from_api(&e, "photos root"))?
.is_some();
let items = if available {
link::bounded_by(super::BATCH_CALL_TIMEOUT, photos.enumerate_timeline())
.await
.map_err(|e| CoreError::from_api(&e, "timeline"))?
.into_iter()
.map(|item| db::TimelineRow::new(item.uid.to_string(), item.capture_time))
.collect()
} else {
Vec::new()
};
if !self
.db
.photos_replace_at(&items, generation, available, now_ms())
.map_err(CoreError::from)?
{
return Err(CoreError::remote(
"Photos changed during the refresh. Try again shortly.",
));
}
if !available {
return Ok(false);
}
} else if self
.db
.state_i64(PHOTOS_AVAILABLE)
.map_err(CoreError::from)?
== Some(0)
{
return Ok(false);
}
let items = photos
.enumerate_timeline()
.await
.map_err(|e| CoreError::from_api(&e, "timeline"))?;
// The timeline DTO carries only a uid and capture time, but the gallery
// has to split into its tabs — which needs each photo's name and media
// type. Resolve those in batches off the request path. Best-effort: a
// photo whose node we fail to resolve keeps whatever was learned before
// (or classifies from nothing, i.e. a still photo), so a partial resolve
// never blanks the timeline.
//
// Only photos the daemon has never read, and ones a remote event marked
// stale, are read. Resolving all of them every time is what made a
// 12k-photo refresh take minutes (`docs/BUGS.md` B92): this metadata does
// not change on its own, and when it does change the event feed names the
// uid that changed (`Db::photos_mark_unresolved`).
let known: HashSet<String> = self
.db
.photos_resolved()
.map_err(CoreError::from)?
.into_iter()
.collect();
let uids: Vec<NodeUid> = items
.iter()
.filter(|it| !known.contains(&it.uid.to_string()))
.map(|it| it.uid.clone())
.collect();
let resolved_at = now_ms();
let mut meta: HashMap<String, PhotoMeta> = HashMap::new();
// A main photo names its related photos rather than the other way round,
// so the relation is recorded from whichever end resolves — the two ends
// can land in different chunks, and one of them may not resolve at all.
let mut main_of: HashMap<String, String> = HashMap::new();
let mut stopped = false;
// Photos read before count as done, so the bar picks up where an
// interrupted pass left off. Cleared however the pass ends.
let skipped = items.len() - uids.len();
let pending = self.db.photos_pending().map_err(CoreError::from)?;
let total = self.db.photos_count().map_err(CoreError::from)?;
let skipped = total.saturating_sub(pending.len());
let mut resolved = 0;
let progress = ProgressGuard(&self.timeline_progress);
if !uids.is_empty() {
*progress.0.lock() = Some((skipped, items.len()));
}
for (index, chunk) in uids.chunks(TIMELINE_ENRICH_CHUNK).enumerate() {
// Teardown cancels the runtime under this loop, and every chunk
// still in flight then fails its decrypt task — a burst of
// "skipping undecryptable photo ... task was cancelled" warnings
// that read like corruption and are only a shutdown. Stop asking
// for chunks instead. What resolved so far is kept below; the rest
// stay unresolved and are read by the next refresh.
for (index, chunk) in pending.chunks(TIMELINE_ENRICH_CHUNK).enumerate() {
if self.shutdown.is_stopping() {
stopped = true;
break;
}
match photos.enumerate_nodes(chunk).await {
Ok(nodes) => {
for node in nodes {
let media_type = match &node.kind {
NodeKind::File { media_type, .. } => Some(media_type.clone()),
NodeKind::Folder => None,
};
// The favorite state is a tag on the photo node, so it
// rides along with the metadata resolve rather than
// costing a listing of its own.
let favorite = node
.photo
.as_ref()
.is_some_and(|p| p.tags.contains(&PhotoTag::Favorite));
let key = node.uid.to_string();
let content_hash = node.photo.as_ref().and_then(|p| p.content_hash.clone());
let location = node
.photo
.as_ref()
.and_then(|p| p.location)
.map(|l| (l.latitude, l.longitude));
if let Some(photo) = node.photo.as_ref() {
if let Some(main) = photo.main_photo_uid.as_ref() {
main_of.insert(key.clone(), main.to_string());
}
for related in &photo.related_photo_uids {
main_of.insert(related.to_string(), key.clone());
}
}
meta.insert(
key,
PhotoMeta {
name: Some(node.name),
media_type,
favorite,
content_hash,
location,
},
);
*progress.0.lock() = Some((skipped + index * TIMELINE_ENRICH_CHUNK, total));
let uids: Vec<_> = chunk.iter().filter_map(|uid| parse_uid(uid)).collect();
let nodes = link::bounded(photos.enumerate_nodes(&uids))
.await
.map_err(|e| CoreError::from_api(&e, "read photo metadata"))?;
let rows: Vec<_> = nodes
.into_iter()
.filter_map(|node| {
let NodeKind::File { media_type, .. } = node.kind else {
return None;
};
if node.trashed {
return None;
}
}
Err(e) => warn!(error = %e, "resolving photo metadata for a timeline chunk failed"),
let photo = node.photo.as_ref();
Some(db::TimelineRow {
name: Some(node.name),
media_type: Some(media_type),
favorite: Some(photo.is_some_and(|p| p.tags.contains(&PhotoTag::Favorite))),
content_hash: photo.and_then(|p| p.content_hash.clone()),
location: photo
.and_then(|p| p.location)
.map(|l| (l.latitude, l.longitude)),
main_uid: photo
.and_then(|p| p.main_photo_uid.as_ref())
.map(ToString::to_string),
resolved_at: Some(now_ms()),
..db::TimelineRow::new(node.uid.to_string(), 0)
})
})
.collect();
if !self
.db
.photos_resolve_at(generation, &rows)
.map_err(CoreError::from)?
{
return Err(CoreError::remote(
"Photos changed during the refresh. Try again shortly.",
));
}
let done = (skipped + (index + 1) * TIMELINE_ENRICH_CHUNK).min(items.len());
*progress.0.lock() = Some((done, items.len()));
resolved += rows.len();
}
drop(progress);
let rows: Vec<db::TimelineRow> = items
.iter()
.map(|it| {
let key = it.uid.to_string();
let main_uid = main_of.get(&key).cloned();
// An unresolved photo keeps whatever was learned before, for the
// favorite flag as much as for the name and media type.
match meta.get(&key).cloned() {
Some(m) => db::TimelineRow {
name: m.name,
media_type: m.media_type,
favorite: Some(m.favorite),
content_hash: m.content_hash,
location: m.location,
main_uid,
resolved_at: Some(resolved_at),
..db::TimelineRow::new(key, it.capture_time)
},
None => db::TimelineRow {
main_uid,
..db::TimelineRow::new(key, it.capture_time)
},
}
})
.collect();
self.db.photos_replace(&rows).map_err(CoreError::from)?;
let _ = self.db.set_state_i64(PHOTOS_AVAILABLE, 1);
let _ = self.db.set_state_i64(PHOTOS_SYNCED_MS, now_ms());
// How long this took is the first thing to know when a refresh looks
// like it never happened, and the chunk count is what it scales with.
info!(
photos = rows.len(),
// `resolved` is what came back, which is short of `wanted` when
// shutdown cut the pass off or a chunk failed.
resolved = meta.len(),
wanted = uids.len(),
skipped = rows.len().saturating_sub(uids.len()),
chunks = uids.len().div_ceil(TIMELINE_ENRICH_CHUNK),
photos = total,
resolved,
wanted = pending.len(),
skipped,
elapsed_ms = started.elapsed().as_millis() as u64,
stopped,
"photos timeline refreshed"
);
Ok(true)
@@ -2656,7 +2620,11 @@ impl Core {
// A refresh that landed stamped the timeline fresh. One that failed
// did not, and the caller gets this refresh's own error instead of
// an empty timeline.
if !self.listing_stale(PHOTOS_SYNCED_MS, TIMELINE_TTL)
if self
.db
.state_i64(PHOTOS_SYNCED_MS)
.map_err(CoreError::from)?
.is_some()
|| self.timeline_refreshing.swap(true, Ordering::SeqCst)
{
return Ok(self.db.state_i64(PHOTOS_AVAILABLE).ok().flatten() != Some(0));
+8 -76
View File
@@ -212,15 +212,12 @@ impl BlockRing {
}
/// The result of fetching one block, shared between everyone who wanted it.
type BlockResult = Result<Arc<Vec<u8>>, Errno>;
pub(super) type BlockResult = Result<Arc<Vec<u8>>, Errno>;
/// Identifies a block *of a revision*: node, validity tag, and the plaintext
/// range the block covers — see [`RingKey`] for why it is the range and not the
/// index.
type BlockKey = (NodeUid, i64, u64, BlockSpan);
/// A claim on a block key: who owns the fetch, and where its result is published.
type Claim = (u64, tokio::sync::watch::Receiver<Option<BlockResult>>);
pub(super) type BlockKey = (NodeUid, i64, u64, BlockSpan);
/// Block fetches in flight, so a block is downloaded and decrypted once however
/// many readers want it at that moment (audit T1).
@@ -231,34 +228,7 @@ type Claim = (u64, tokio::sync::watch::Receiver<Option<BlockResult>>);
/// for. Without this each of them pays the full round-trip and decrypt and all
/// but one result is discarded — roughly double the bandwidth and CPU in steady
/// state. Same shape as [`ReaderSlot::Pending`] one level up.
#[derive(Default)]
pub(super) struct BlockFlight {
inflight: Mutex<HashMap<BlockKey, Claim>>,
}
/// Distinguishes attempts at the same key, so a leader only retires its own.
static NEXT_FLIGHT_ID: AtomicU64 = AtomicU64::new(0);
/// The leader's claim on a key. Retires it on drop — including when the task is
/// cancelled mid-fetch, which is what keeps a dropped `JoinSet` from leaving a
/// key nobody will ever publish.
struct Flight {
flight: Arc<BlockFlight>,
key: BlockKey,
id: u64,
}
impl Drop for Flight {
fn drop(&mut self) {
let mut inflight = self.flight.inflight.lock();
if inflight
.get(&self.key)
.is_some_and(|(id, _)| *id == self.id)
{
inflight.remove(&self.key);
}
}
}
pub(super) type BlockFlight = proton_media_rs::SingleFlight<BlockKey, BlockResult>;
/// Where each file's reader is expected to read next, so sequential reads can be
/// told from random ones and only the former prefetched (audit T2).
@@ -581,49 +551,11 @@ impl Core {
cache_blocks: bool,
) -> BlockResult {
let key: BlockKey = (uid.clone(), mtime, fsize, span);
let claim = {
let mut inflight = self.block_flight.inflight.lock();
match inflight.get(&key) {
Some((_, rx)) => Err(rx.clone()),
None => {
let id = NEXT_FLIGHT_ID.fetch_add(1, Ordering::Relaxed);
let (tx, rx) = tokio::sync::watch::channel(None);
inflight.insert(key.clone(), (id, rx));
Ok((
tx,
Flight {
flight: self.block_flight.clone(),
key: key.clone(),
id,
},
))
}
}
};
let (tx, flight) = match claim {
Ok(lead) => lead,
Err(mut rx) => {
loop {
if let Some(result) = rx.borrow_and_update().clone() {
return result;
}
if rx.changed().await.is_err() {
break;
}
}
return self
.read_block(reader, uid, mtime, fsize, span, cache_blocks)
.await;
}
};
let result = self
.read_block(reader, uid, mtime, fsize, span, cache_blocks)
.await;
// Retire the claim before publishing, so a racer that wakes and re-enters
// finds no stale entry to join.
drop(flight);
let _ = tx.send(Some(result.clone()));
result
self.block_flight
.join(key, || {
self.read_block(reader, uid, mtime, fsize, span, cache_blocks)
})
.await
}
/// Download and decrypt one block, then put it where the next reader will
+1
View File
@@ -77,6 +77,7 @@ fn shared_entry_of(core: &Core, n: Node) -> DirEntry {
shared_by_unverified,
trashed_at: 0,
trashed_from: None,
trashed_parent: None,
issue: None,
}
}
+138
View File
@@ -89,6 +89,7 @@ impl Daemon {
db,
MountOptions {
username: "sim".into(),
gallery_session: None,
sweep_mode: AppConfig::default().resolved_conflict_sweep(),
upload_limit: 0,
download_limit: 0,
@@ -1917,6 +1918,143 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn the_trash_lists_by_folder_and_empties() {
let drive = FakeDrive::new();
let dir = scratch("trash-by-folder");
let daemon = Daemon::start(&dir, drive.client(1, Faults::none())).unwrap();
let mount = &daemon.mountpoint;
std::fs::create_dir_all(mount.join("album/inner")).unwrap();
std::fs::write(mount.join("album/inner/a.jpg"), b"a").unwrap();
std::fs::write(mount.join("album/b.jpg"), b"b").unwrap();
std::fs::write(mount.join("loose.txt"), b"c").unwrap();
assert!(wait_until(Duration::from_secs(30), || daemon.drained()
&& drive.lookup("album/inner/a.jpg").is_some()
&& drive.lookup("loose.txt").is_some()));
let album = drive.lookup("album").unwrap();
let inner = drive.lookup("album/inner").unwrap();
// A file trashed on its own, then the folder that held it.
drive.device().trash("album/b.jpg");
drive.device().trash("album");
std::fs::remove_file(mount.join("loose.txt")).unwrap();
assert!(wait_until(Duration::from_secs(30), || daemon.drained()));
let list = |request: &Request| match daemon.request(request) {
Ok(Response::Entries { entries }) => entries,
other => panic!("unexpected reply: {other:?}"),
};
daemon
.request(&Request::Refresh {
scope: RefreshScope::Trash,
})
.unwrap();
let mut trash = Vec::new();
assert!(wait_until(Duration::from_secs(30), || {
trash = list(&Request::ListTrash);
trash.len() == 3
}));
let b = trash.iter().find(|e| e.name == "b.jpg").unwrap();
assert_eq!(
b.trashed_parent.as_deref(),
Some(album.to_string().as_str())
);
let top: Vec<&str> = trash
.iter()
.filter(|e| e.trashed_parent.is_none())
.map(|e| e.name.as_str())
.collect();
assert_eq!(top.len(), 2);
assert!(top.contains(&"album") && top.contains(&"loose.txt"));
let inside = list(&Request::ListTrashedFolder {
uid: album.to_string(),
});
let names: Vec<&str> = inside.iter().map(|e| e.name.as_str()).collect();
assert!(names.contains(&"inner"), "{names:?}");
assert!(names.contains(&"b.jpg"), "{names:?}");
assert!(inside.iter().any(|e| e.uid == inner.to_string()));
assert!(matches!(
daemon.request(&Request::EmptyTrash),
Ok(Response::Ok { .. })
));
assert!(list(&Request::ListTrash).is_empty());
daemon
.request(&Request::Refresh {
scope: RefreshScope::Trash,
})
.unwrap();
assert!(list(&Request::ListTrash).is_empty());
assert!(drive.lookup("album").is_none());
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_move_blocked_by_trash_can_be_retried_after_restore() {
use pdfs_core::control::SyncIssue;
let drive = FakeDrive::new();
let dir = scratch("move-out-of-trash");
let daemon = Daemon::start(&dir, drive.client(1, Faults::none())).unwrap();
let mount = &daemon.mountpoint;
std::fs::create_dir_all(mount.join("source/nested")).unwrap();
std::fs::create_dir(mount.join("destination")).unwrap();
std::fs::write(mount.join("source/nested/wallpaper.jpg"), b"image").unwrap();
assert!(wait_until(Duration::from_secs(30), || daemon.drained()
&& drive.lookup("source/nested/wallpaper.jpg").is_some()));
let source = drive.lookup("source").unwrap();
let pause = |paused| Request::SetSyncPaused {
paused,
until: None,
};
daemon.request(&pause(true)).unwrap();
std::fs::rename(
mount.join("source/nested/wallpaper.jpg"),
mount.join("destination/wallpaper.jpg"),
)
.unwrap();
drive.device().trash("source");
daemon.request(&pause(false)).unwrap();
assert!(wait_until(Duration::from_secs(30), || daemon
.pending()
.is_ok_and(|ops| ops.iter().any(|op| op.issue
== Some(SyncIssue::Trashed)
&& op
.last_error
.as_deref()
.is_some_and(|message| message.contains("‘source’"))))));
daemon
.request(&Request::RetryPendingOp { id: None })
.unwrap();
assert!(wait_until(Duration::from_secs(30), || daemon
.pending()
.is_ok_and(|ops| ops.iter().any(|op| op.attempts >= 2))));
daemon
.request(&Request::Restore {
uids: vec![source.to_string()],
})
.unwrap();
daemon
.request(&Request::RetryPendingOp { id: None })
.unwrap();
assert!(wait_until(Duration::from_secs(30), || daemon.drained()
&& drive.lookup("destination/wallpaper.jpg").is_some()));
assert_eq!(
std::fs::read(mount.join("destination/wallpaper.jpg")).unwrap(),
b"image"
);
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_write_over_a_create_whose_read_back_failed_is_not_a_conflict() {
+22
View File
@@ -160,6 +160,17 @@ impl Server {
.is_some_and(|remote| remote.file.is_none() && !remote.trashed)
}
fn in_trash(&self, uid: &NodeUid) -> bool {
let mut node = self.current(uid);
while let Some(remote) = node {
if remote.trashed {
return true;
}
node = remote.parent.as_ref().and_then(|uid| self.current(uid));
}
false
}
fn is_below(&self, uid: &NodeUid, ancestor: &NodeUid) -> bool {
let mut cursor = Some(uid.clone());
while let Some(at) = cursor {
@@ -1037,6 +1048,12 @@ impl FakeClient {
}
let parent = new_parent.cloned().or(remote.parent.clone());
let name = new_name.unwrap_or(&remote.name).to_owned();
if server.in_trash(uid) || parent.as_ref().is_some_and(|uid| server.in_trash(uid)) {
return Err(api(
ResponseCode::NotEnoughPermissions,
"an item involved in the move is in Trash",
));
}
if let Some(parent) = &parent {
if !server.is_folder(parent) {
return Err(api(
@@ -1729,6 +1746,11 @@ impl DriveApi for FakeClient {
.await
}
async fn empty_trash(&self) -> Result<()> {
let trashed = self.enumerate_trash_node_uids().await?;
self.batch(&trashed, delete_one).await.map(|_| ())
}
async fn enumerate_trash_node_uids(&self) -> Result<Vec<NodeUid>> {
self.request(false, |server| {
Ok(server
+4
View File
@@ -112,6 +112,8 @@ struct Run {
/// Run `profile` on `seed`. It answers a report of what went wrong, if
/// anything did.
pub(crate) fn run(seed: u64, profile: &'static Profile) -> Result<(), String> {
// The simulation suite runs serially. Discard logs left by earlier tests.
take_logged();
let dir = scratch(&format!("{}-{seed}", profile.name));
let mut rng = Rng::new(seed);
let drive = FakeDrive::new();
@@ -137,6 +139,7 @@ pub(crate) fn run(seed: u64, profile: &'static Profile) -> Result<(), String> {
let _ = daemon.stop();
}
}
run.logged.extend(take_logged());
match outcome {
Ok(()) => {
let _ = std::fs::remove_dir_all(&dir);
@@ -432,6 +435,7 @@ impl Run {
fn report(&self, failure: &str, dir: &Path) -> String {
let _ = std::fs::write(dir.join("steps.log"), self.log.join("\n") + "\n");
let _ = std::fs::write(dir.join("daemon.log"), self.logged.join("\n") + "\n");
let tail = &self.log[self.log.len().saturating_sub(LOG_TAIL)..];
format!(
"{} seed {} failed: {failure}\nreplay with PDFS_SIM_SEED={}; state kept in {}\nlast steps:\n{}",
+222 -85
View File
@@ -256,6 +256,62 @@ impl Reporter {
}
}
/// Photos uploaded at once. Each upload waits on Proton between its blocks,
/// so two keep the link busy where one leaves it idle; more would mostly
/// stage more copies on disk.
const IMPORT_UPLOADS: usize = 2;
/// Where an upload reads the photo from.
enum Source<'a> {
/// A copy staged on disk, which a thumbnail can be made from.
Staged(&'a Path),
/// Straight out of the open archive.
Archive(&'a mut zip::ZipArchive<std::io::BufReader<std::fs::File>>),
}
/// A staged photo handed to an upload worker.
struct Upload {
index: usize,
path: PathBuf,
photo: TakeoutPhoto,
tags: Vec<PhotoTag>,
main_uid: Option<NodeUid>,
}
impl Upload {
fn landed(self, result: CoreResult<NodeUid>) -> Landed {
Landed {
index: self.index,
live: self.main_uid.is_some(),
photo: self.photo,
result,
}
}
fn streamed(
index: usize,
photo: TakeoutPhoto,
main_uid: Option<NodeUid>,
result: CoreResult<NodeUid>,
) -> Landed {
Landed {
index,
live: main_uid.is_some(),
photo,
result,
}
}
}
/// A finished upload, to be counted on the import thread.
struct Landed {
index: usize,
/// The video of a live pair, filed with its still.
live: bool,
photo: TakeoutPhoto,
result: CoreResult<NodeUid>,
}
/// How a planned photo relates to a live or motion photo pair.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum LiveRole {
@@ -527,7 +583,7 @@ impl Core {
let scan = match scan_archives_until(archives, || self.import_should_stop(cancel)) {
Ok(scan) => scan,
Err(_) if self.import_should_stop(cancel) => return Ok(true),
Err(e) => return Err(e),
Err(e) => return Err(CoreError::conflict(e.to_string())),
};
*planned = plan(&scan);
*albums = scan.albums.clone();
@@ -714,72 +770,133 @@ impl Core {
}
}
for (archive_index, indices) in by_archive {
if self.import_should_stop(cancel) {
stopped = true;
break;
}
let mut archive = match open_archive(archives, archive_index) {
Ok(archive) => archive,
Err(e) => {
// The archive was readable at scan time; whatever broke
// it since costs its own photos, not the whole run.
warn!(error = %e, photos = indices.len(), "cannot reopen a takeout archive");
summary.failed += indices.len();
for _ in &indices {
job.step();
report.step();
// Staged photos go to IMPORT_UPLOADS workers while this thread
// reads the next one out of its archive. A round ends only when
// every upload in it has landed, so a pair's video always finds
// its still's uid.
let (jobs, queue) = std::sync::mpsc::sync_channel::<Upload>(0);
let queue = std::sync::Mutex::new(queue);
std::thread::scope(|scope| {
let (landed_tx, landed) = std::sync::mpsc::channel::<Landed>();
for _ in 0..IMPORT_UPLOADS {
let queue = &queue;
let landed_tx = landed_tx.clone();
scope.spawn(move || {
loop {
let next = queue.lock().map(|queue| queue.recv());
let Ok(Ok(upload)) = next else { break };
let result = self.upload_takeout_photo(
Source::Staged(&upload.path),
&upload.photo,
upload.tags.clone(),
upload.main_uid.clone(),
);
let _ = std::fs::remove_file(&upload.path);
if landed_tx.send(upload.landed(result)).is_err() {
break;
}
}
report.counts(summary);
continue;
}
};
});
}
drop(landed_tx);
for index in indices {
'archives: for (archive_index, indices) in by_archive {
if self.import_should_stop(cancel) {
stopped = true;
break;
}
let main_uid = match planned[index].live {
Some(LiveRole::Video(still)) => planned[still].uid.clone(),
_ => None,
};
let photo = planned[index].photo.clone();
job.detail(photo.name.clone());
report.current(&photo.name);
let uploaded =
self.upload_takeout_photo(&mut archive, &planned[index], main_uid.clone());
job.step();
report.step();
match uploaded {
Ok(uid) => {
self.note_self_change(&uid);
summary.uploaded += 1;
summary.bytes += photo.size;
if main_uid.is_some() {
summary.live_photos += 1;
}
if photo.favorite {
favorites.push(uid.clone());
}
planned[index].uid = Some(uid);
if summary.uploaded.is_multiple_of(TIMELINE_REFRESH_EVERY) {
self.spawn_timeline_refresh();
}
if summary.uploaded.is_multiple_of(CHECKPOINT_EVERY) {
report.counts(summary);
report.checkpoint();
}
}
let mut archive = match open_archive(archives, archive_index) {
Ok(archive) => archive,
Err(e) => {
summary.failed += 1;
warn!(name = photo.name, error = %e, "takeout photo upload failed");
// The archive was readable at scan time; whatever
// broke it since costs its own photos, not the run.
warn!(error = %e, photos = indices.len(), "cannot reopen a takeout archive");
summary.failed += indices.len();
for _ in &indices {
job.step();
report.step();
}
report.counts(summary);
continue;
}
};
for index in indices {
for done in landed.try_iter() {
self.record_upload(planned, done, summary, &mut favorites, job, report);
}
if self.import_should_stop(cancel) {
stopped = true;
break 'archives;
}
let item = &planned[index];
let main_uid = match item.live {
Some(LiveRole::Video(still)) => planned[still].uid.clone(),
_ => None,
};
let tags = match item.live {
Some(LiveRole::Still(tag)) => vec![tag],
_ => Vec::new(),
};
let photo = item.photo.clone();
job.detail(photo.name.clone());
report.current(&photo.name);
// A thumbnail is made from a file, so a photo that can
// have one is copied out of the archive and uploaded
// from the copy by a worker. Without one, Proton's apps
// show a blank tile until they make their own.
let staged = can_make_thumbnail(&photo.name, photo.size)
.then(|| self.stage_takeout_entry(&mut archive, &photo.entry))
.and_then(|staged| {
staged
.inspect_err(|e| {
warn!(error = %e,
"copying a takeout photo out failed; uploading it without a thumbnail")
})
.ok()
});
match staged {
Some(path) => {
let upload = Upload {
index,
path,
photo,
tags,
main_uid,
};
if let Err(std::sync::mpsc::SendError(upload)) = jobs.send(upload) {
let _ = std::fs::remove_file(&upload.path);
}
}
None => {
let result = self.upload_takeout_photo(
Source::Archive(&mut archive),
&photo,
tags,
main_uid.clone(),
);
let done = Upload::streamed(index, photo, main_uid, result);
self.record_upload(
planned,
done,
summary,
&mut favorites,
job,
report,
);
}
}
}
report.counts(summary);
}
drop(jobs);
for done in landed.iter() {
self.record_upload(planned, done, summary, &mut favorites, job, report);
}
});
if stopped {
break;
}
}
@@ -787,14 +904,55 @@ impl Core {
stopped
}
/// Upload one planned photo out of its open archive.
/// Count one finished upload and file its uid in the plan.
fn record_upload(
&self,
planned: &mut [Planned],
done: Landed,
summary: &mut ImportSummary,
favorites: &mut Vec<NodeUid>,
job: &crate::transfers::JobGuard,
report: &Reporter,
) {
job.step();
report.step();
match done.result {
Ok(uid) => {
self.note_self_change(&uid);
summary.uploaded += 1;
summary.bytes += done.photo.size;
if done.live {
summary.live_photos += 1;
}
if done.photo.favorite {
favorites.push(uid.clone());
}
planned[done.index].uid = Some(uid);
if summary.uploaded.is_multiple_of(TIMELINE_REFRESH_EVERY) {
self.spawn_timeline_refresh();
}
if summary.uploaded.is_multiple_of(CHECKPOINT_EVERY) {
report.counts(summary);
report.checkpoint();
}
}
Err(e) => {
summary.failed += 1;
warn!(error = %e, "takeout photo upload failed");
}
}
report.counts(summary);
}
/// Upload one planned photo, from its staged copy or straight out of its
/// open archive.
fn upload_takeout_photo(
&self,
archive: &mut zip::ZipArchive<std::io::BufReader<std::fs::File>>,
item: &Planned,
source: Source<'_>,
photo: &TakeoutPhoto,
tags: Vec<PhotoTag>,
main_uid: Option<NodeUid>,
) -> CoreResult<NodeUid> {
let photo = &item.photo;
let photos = self.photos();
let guard = self.transfers.begin(
photo.name.clone(),
@@ -802,10 +960,6 @@ impl Core {
TransferDirection::Upload,
photo.size,
);
let tags = match item.live {
Some(LiveRole::Still(tag)) => vec![tag],
_ => Vec::new(),
};
let metadata = |capture_time: Option<i64>| PhotoUploadMetadata {
capture_time,
last_modification_time: capture_time.and_then(pdfs_core::timestamp::modification_time),
@@ -820,21 +974,8 @@ impl Core {
replace_existing_draft: true,
};
// A thumbnail is made from a file, so a photo that can have one is
// copied out of the archive first and uploaded from the copy. Without
// one, Proton's apps show a blank tile until they make their own.
let staged = can_make_thumbnail(&photo.name, photo.size)
.then(|| self.stage_takeout_entry(archive, &photo.entry))
.and_then(|staged| {
staged
.inspect_err(|e| {
warn!(name = photo.name, error = %e,
"copying a takeout photo out failed; uploading it without a thumbnail")
})
.ok()
});
let uploaded = match &staged {
Some(path) => {
match source {
Source::Staged(path) => {
let thumbnails = self.upload_thumbnails(path, &photo.name);
// No sidecar date and none in the name: the photo's own EXIF,
// then when Google received it, before Drive files it under
@@ -860,7 +1001,7 @@ impl Core {
.map_err(|e| CoreError::from_api(&e, "upload"))
})
}
None => read_entry(archive, &photo.entry, |reader| {
Source::Archive(archive) => read_entry(archive, &photo.entry, |reader| {
let counting = CountingReader::new(reader, &guard);
self.rt
.block_on(photos.upload_photo_from(
@@ -875,11 +1016,7 @@ impl Core {
.map_err(|e| CoreError::from_api(&e, "upload"))
})
.and_then(|result| result),
};
if let Some(path) = staged {
let _ = std::fs::remove_file(path);
}
uploaded
}
/// Mark photos as favorites. A separate call: the upload seal takes
+8
View File
@@ -22,6 +22,10 @@ name = "pdfs-prompt"
path = "src/prompt.rs"
[dependencies]
proton-gallery-rs.workspace = true
# The offline basemap under the Places map.
proton-map-rs.workspace = true
base64.workspace = true
pdfs-core.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
@@ -41,3 +45,7 @@ pangocairo = "0.22"
reqwest = { version = "0.13", default-features = false, features = ["rustls", "webpki-roots"] }
# The system libintl (glibc) rather than a bundled gettext build.
gettext-rs = { version = "0.8", features = ["gettext-system"] }
[dev-dependencies]
# Builds small basemaps for the map tests.
proton-map-rs = { workspace = true, features = ["writer"] }
+4
View File
@@ -118,11 +118,15 @@ scale.browser-status-meter trough { min-width: 140px; min-height: 6px; }
.memory-card { padding: 0; border-radius: 12px; }
.memory-cover { border-radius: 12px; background: alpha(currentColor, 0.06); }
.memory-label { font-weight: 700; color: white; text-shadow: 0 1px 3px rgba(0, 0, 0, 0.9); padding: 22px 10px 6px 10px; background: linear-gradient(to top, rgba(0, 0, 0, 0.6), rgba(0, 0, 0, 0)); }
/* The button handles hover and clicks. The FlowBox cell can be wider than it. */
flowbox.photo-album-grid > flowboxchild:hover,
flowbox.photo-album-grid > flowboxchild:active { background: transparent; }
.album-card { padding: 0; border-radius: 16px; transition: background 180ms ease; }
.album-card:hover { background: alpha(currentColor, 0.07); }
.album-cover { border-radius: 16px; background: alpha(currentColor, 0.06); box-shadow: 0 4px 16px rgba(0, 0, 0, 0.32); transition: box-shadow 180ms ease; }
.album-card:hover .album-cover { box-shadow: 0 10px 26px rgba(0, 0, 0, 0.5); }
.album-card:hover .photo-thumb { transform: scale(1.05); }
listview.drive-gallery-rows > row { padding: 0; }
.view-switch button, .view-switch toggle { padding-left: 12px; padding-right: 12px; }
/* Shared with Me: what a share lets me do, as a tinted pill beside the name. */
+40 -5
View File
@@ -100,6 +100,9 @@ fn on_feed(ui: &Rc<Ui>, state: &Rc<FeedState>, feed: Feed, hidden: bool) {
follow_takeout(ui, state, &jobs);
}
Feed::Event(Event::Changed { topics }) if hidden => {
if topics.contains(&Topic::Photos) {
check_photo_viewer(ui);
}
state.deferred.borrow_mut().extend(topics);
}
Feed::Event(Event::Changed { topics }) => follow_changes(ui, state, &topics),
@@ -133,10 +136,44 @@ fn follow_changes(ui: &Rc<Ui>, state: &FeedState, topics: &[Topic]) {
refresh_file_thumbnails(ui);
}
let page = ui.stack.visible_child_name();
// A timeline out of sight is read again when the user comes back to it;
// one on screen is left alone (below).
if changed(Topic::Photos) && (page.as_deref() != Some("gallery") || !photo_grid_shown(ui)) {
if changed(Topic::GalleryVisibility) || changed(Topic::GalleryVisibilitySynced) {
if let Some(browser) = ui.drive_galleries.borrow().upgrade() {
browser.refresh_visibility(changed(Topic::GalleryVisibility));
}
for window in gtk4::Window::list_toplevels() {
if let Ok(window) = window.downcast::<gtk4::Window>()
&& window.has_css_class("photo-viewer-window")
{
window.close();
}
}
ui.gallery
.thumb_generation
.set(ui.gallery.thumb_generation.get().wrapping_add(1));
ui.gallery.thumb_queue.borrow_mut().clear();
ui.gallery.decode_queue.borrow_mut().clear();
ui.gallery.thumb_wanted.borrow_mut().clear();
ui.gallery.photo_tex.borrow_mut().clear();
ui.gallery.model.remove_all();
ui.gallery.groups.remove_all();
ui.gallery.row_by_photo.borrow_mut().clear();
ui.gallery.timeline_stale.set(true);
if page.as_deref() == Some("gallery") {
match ui.gallery.content.visible_child_name().as_deref() {
Some("albums") => load_albums(ui),
Some("drive-galleries") => {}
Some("places") => load_places(ui),
_ => load_gallery(ui, false),
}
}
}
if changed(Topic::Photos) {
check_photo_viewer(ui);
if page.as_deref() != Some("gallery") || !photo_grid_shown(ui) {
ui.gallery.timeline_stale.set(true);
} else {
refresh_visible_photos(ui);
}
}
match page.as_deref() {
// A search on screen is left alone: reloading would swap its hits for
@@ -144,8 +181,6 @@ fn follow_changes(ui: &Rc<Ui>, state: &FeedState, topics: &[Topic]) {
Some("browser") if changed(Topic::Files) && ui.browser.search.text().trim().is_empty() => {
load_browser(ui)
}
// The timeline is left alone: reloading it would throw away the place
// the user scrolled to. F5 reloads it on request.
Some("gallery")
if changed(Topic::Photos)
&& ui.gallery.content.visible_child_name().as_deref() == Some("albums") =>
+248
View File
@@ -0,0 +1,248 @@
//! A least-recently-used cache with a cost budget.
//!
//! Each entry carries a cost (bytes for a texture cache, `1` for a cache capped
//! by count). Inserting past the budget drops the entries used longest ago.
//! Lookups and touches are logarithmic, so a cache of a few thousand textures
//! can be touched on every tile bind without a linear scan.
use std::borrow::Borrow;
use std::collections::{BTreeMap, HashMap};
use std::hash::Hash;
struct Entry<V> {
value: V,
cost: usize,
stamp: u64,
}
pub(crate) struct Lru<K, V> {
entries: HashMap<K, Entry<V>>,
/// Entries by the stamp of their last use, oldest first.
order: BTreeMap<u64, K>,
next: u64,
used: usize,
budget: usize,
}
impl<K: Eq + Hash + Clone, V> Lru<K, V> {
pub(crate) fn new(budget: usize) -> Self {
Self {
entries: HashMap::new(),
order: BTreeMap::new(),
next: 0,
used: 0,
budget,
}
}
/// Look an entry up without marking it used.
pub(crate) fn get<Q>(&self, key: &Q) -> Option<&V>
where
K: Borrow<Q>,
Q: Eq + Hash + ?Sized,
{
self.entries.get(key).map(|entry| &entry.value)
}
pub(crate) fn contains_key<Q>(&self, key: &Q) -> bool
where
K: Borrow<Q>,
Q: Eq + Hash + ?Sized,
{
self.entries.contains_key(key)
}
/// Mark an entry as just used, so it is the last to be evicted. Returns
/// whether it was there.
pub(crate) fn touch<Q>(&mut self, key: &Q) -> bool
where
K: Borrow<Q>,
Q: Eq + Hash + ?Sized,
{
let stamp = self.next;
let Some(entry) = self.entries.get_mut(key) else {
return false;
};
if let Some(held) = self.order.remove(&entry.stamp) {
self.order.insert(stamp, held);
}
entry.stamp = stamp;
self.next += 1;
true
}
/// Look an entry up and mark it used.
pub(crate) fn fetch<Q>(&mut self, key: &Q) -> Option<&V>
where
K: Borrow<Q>,
Q: Eq + Hash + ?Sized,
{
if !self.touch(key) {
return None;
}
self.get(key)
}
/// Store `value` as the most recently used entry, then evict the least
/// recently used ones until the total cost fits the budget again. The new
/// entry itself is never evicted, even when it alone is over budget.
pub(crate) fn insert(&mut self, key: K, value: V, cost: usize) {
self.remove(&key);
let stamp = self.next;
self.next += 1;
self.order.insert(stamp, key.clone());
self.entries.insert(key, Entry { value, cost, stamp });
self.used += cost;
while self.used > self.budget && self.order.len() > 1 {
let Some((_, oldest)) = self.order.pop_first() else {
break;
};
if let Some(entry) = self.entries.remove(&oldest) {
self.used -= entry.cost;
}
}
}
pub(crate) fn remove<Q>(&mut self, key: &Q) -> Option<V>
where
K: Borrow<Q>,
Q: Eq + Hash + ?Sized,
{
let entry = self.entries.remove(key)?;
self.order.remove(&entry.stamp);
self.used -= entry.cost;
Some(entry.value)
}
/// Keep only the entries `keep` accepts.
pub(crate) fn retain(&mut self, mut keep: impl FnMut(&K, &V) -> bool) {
let order = &mut self.order;
let used = &mut self.used;
self.entries.retain(|key, entry| {
let kept = keep(key, &entry.value);
if !kept {
order.remove(&entry.stamp);
*used -= entry.cost;
}
kept
});
}
pub(crate) fn clear(&mut self) {
self.entries.clear();
self.order.clear();
self.used = 0;
}
#[cfg(test)]
pub(crate) fn len(&self) -> usize {
self.entries.len()
}
#[cfg(test)]
pub(crate) fn used(&self) -> usize {
self.used
}
}
#[cfg(test)]
mod tests {
use super::Lru;
#[test]
fn evicts_least_recently_used_past_budget() {
let mut cache = Lru::new(3);
cache.insert("a", 1, 1);
cache.insert("b", 2, 1);
cache.insert("c", 3, 1);
assert!(cache.touch("a"));
cache.insert("d", 4, 1);
assert!(cache.contains_key("a"));
assert!(!cache.contains_key("b"));
assert!(cache.contains_key("c"));
assert!(cache.contains_key("d"));
assert_eq!(cache.len(), 3);
}
#[test]
fn budget_counts_cost_not_entries() {
let mut cache = Lru::new(100);
cache.insert("small", (), 10);
cache.insert("medium", (), 40);
cache.insert("large", (), 60);
// 110 > 100: the oldest goes, and that is enough.
assert!(!cache.contains_key("small"));
assert_eq!(cache.used(), 100);
cache.insert("huge", (), 90);
assert_eq!(cache.used(), 90);
assert_eq!(cache.len(), 1);
}
#[test]
fn keeps_a_single_entry_over_budget() {
let mut cache = Lru::new(10);
cache.insert("a", (), 5);
cache.insert("big", (), 50);
assert!(cache.contains_key("big"));
assert!(!cache.contains_key("a"));
assert_eq!(cache.used(), 50);
}
#[test]
fn remove_and_clear_keep_cost_in_step() {
let mut cache = Lru::new(100);
cache.insert("a", (), 10);
cache.insert("b", (), 20);
assert_eq!(cache.remove("a"), Some(()));
assert_eq!(cache.used(), 20);
cache.clear();
assert_eq!(cache.used(), 0);
assert_eq!(cache.len(), 0);
}
#[test]
fn reinsert_replaces_cost_and_refreshes() {
let mut cache = Lru::new(30);
cache.insert("a", 1, 10);
cache.insert("b", 2, 10);
cache.insert("a", 3, 15);
assert_eq!(cache.used(), 25);
assert_eq!(cache.get("a"), Some(&3));
cache.insert("c", 4, 10);
// "b" was used longest ago now.
assert!(!cache.contains_key("b"));
assert!(cache.contains_key("a"));
}
#[test]
fn fetch_touches_and_get_does_not() {
let mut cache = Lru::new(2);
cache.insert("a", 1, 1);
cache.insert("b", 2, 1);
assert_eq!(cache.get("a"), Some(&1));
cache.insert("c", 3, 1);
assert!(!cache.contains_key("a"));
assert_eq!(cache.fetch("b"), Some(&2));
cache.insert("d", 4, 1);
assert!(cache.contains_key("b"));
assert!(!cache.contains_key("c"));
assert_eq!(cache.fetch("missing"), None);
}
#[test]
fn retain_drops_rejected_entries_and_their_cost() {
let mut cache = Lru::new(100);
cache.insert((true, "a".to_string()), (), 10);
cache.insert((false, "b".to_string()), (), 20);
cache.insert((true, "c".to_string()), (), 30);
cache.retain(|(private, _), _| !private);
assert_eq!(cache.used(), 20);
assert_eq!(cache.len(), 1);
assert!(cache.remove(&(false, "b".to_string())).is_some());
assert_eq!(cache.used(), 0);
cache.insert((true, "d".to_string()), (), 10);
cache.clear();
assert_eq!(cache.used(), 0);
assert_eq!(cache.len(), 0);
}
}
+53 -24
View File
@@ -6,6 +6,7 @@ pub(crate) mod dates;
pub(crate) mod events;
#[path = "../i18n.rs"]
pub(crate) mod i18n;
pub(crate) mod lru;
pub(crate) mod pages;
pub(crate) mod reload;
#[path = "../theme.rs"]
@@ -15,6 +16,7 @@ pub(crate) mod widgets;
use compat::*;
use events::*;
use lru::Lru;
use pages::activity::*;
use pages::albums::*;
use pages::browser::*;
@@ -160,6 +162,7 @@ struct Ui {
pub(crate) details: DetailsState,
pub(crate) trash: TrashState,
pub(crate) gallery: GalleryState,
drive_galleries: RefCell<std::rc::Weak<pages::drive_galleries::Browser>>,
pub(crate) shared: SharedState,
pub(crate) shared_by_me: SharedByMeState,
pub(crate) devices: DevicesState,
@@ -189,16 +192,11 @@ impl Ui {
/// Scrolling back over a day then repaints from memory rather than decoding
/// the same JPEGs off disk again.
fn store_texture(&self, uid: &str, texture: gtk4::gdk::Texture) {
let mut cache = self.gallery.photo_tex.borrow_mut();
let mut order = self.gallery.photo_tex_order.borrow_mut();
if cache.insert(uid.to_string(), texture).is_none() {
order.push_back(uid.to_string());
}
while order.len() > TEXTURE_CACHE_MAX {
if let Some(old) = order.pop_front() {
cache.remove(&old);
}
}
let bytes = texture_bytes(texture.width(), texture.height());
self.gallery
.photo_tex
.borrow_mut()
.insert(uid.to_string(), texture, bytes);
}
/// Mark a cached thumbnail as freshly used, so eviction takes the tiles
@@ -207,14 +205,7 @@ impl Ui {
/// Insertion order alone evicts the top of the timeline first — exactly what
/// a scroll back up then asks for again.
fn touch_texture(&self, uid: &str) {
let mut order = self.gallery.photo_tex_order.borrow_mut();
if order.back().is_some_and(|back| back == uid) {
return;
}
if let Some(at) = order.iter().position(|held| held == uid) {
order.remove(at);
order.push_back(uid.to_string());
}
self.gallery.photo_tex.borrow_mut().touch(uid);
}
}
@@ -459,6 +450,7 @@ fn build_window(app: &adw::Application) {
nav: split.clone(),
narrow: Cell::new(false),
file_thumbs: FileThumbnailState::new(),
drive_galleries: RefCell::default(),
login: login_widgets,
status: StatusState {
status_inflight: Cell::new(false),
@@ -555,10 +547,21 @@ fn build_window(app: &adw::Application) {
selection_bar: trash_widgets.selection_bar.clone(),
selection_label: trash_widgets.selection_label.clone(),
loader: Loader::new(&trash_widgets.content),
back: trash_widgets.back.clone(),
crumb: trash_widgets.crumb.clone(),
search: trash_widgets.search.clone(),
grouped: trash_widgets.grouped.clone(),
all: RefCell::new(Vec::new()),
nav: RefCell::new(Vec::new()),
folder_rows: RefCell::new(None),
},
gallery: GalleryState {
model: gallery_widgets.model.clone(),
groups: gallery_widgets.groups.clone(),
row_by_photo: RefCell::default(),
rendered_layout: Cell::new(None),
rendered_photos: Cell::new(0),
align_tick: RefCell::default(),
row_height: Cell::new(ROW_DEFAULT),
scrolling_down: Cell::new(true),
scroll_offset: Cell::new(0.0),
@@ -615,23 +618,29 @@ fn build_window(app: &adw::Application) {
memory_cards: gallery_widgets.memory_cards.clone(),
has_memories: Cell::new(false),
date_suppress: Cell::new(false),
months_loading: Cell::new(false),
months_pending: Cell::new(false),
changes_pending: Cell::new(false),
viewer: RefCell::new(std::rc::Weak::new()),
reload_pending: Cell::new(false),
loading: Cell::new(false),
has_more: Cell::new(false),
library: Cell::new(true),
page_waiters: RefCell::new(Vec::new()),
width: Cell::new(0),
photo_tex: RefCell::new(HashMap::new()),
photo_tex_order: RefCell::new(VecDeque::new()),
photo_tex: RefCell::new(Lru::new(TEXTURE_CACHE_BYTES)),
photo_nothumb: RefCell::new(HashSet::new()),
thumb_wanted: RefCell::new(HashMap::new()),
thumb_queue: RefCell::new(VecDeque::new()),
thumb_inflight: Cell::new(false),
thumb_inflight: RefCell::default(),
thumb_generation: Cell::new(0),
decode_queue: RefCell::new(VecDeque::new()),
decode_idle: Cell::new(false),
decode_inflight: RefCell::default(),
thumb_source: RefCell::new(None),
relayout_source: RefCell::new(None),
bound: RefCell::new(BTreeMap::new()),
visible_tiles_pending: Cell::new(0),
tile_pools: RefCell::new(HashMap::new()),
list: gallery_widgets.list.clone(),
selecting: Cell::new(false),
@@ -640,6 +649,7 @@ fn build_window(app: &adw::Application) {
select_btn: gallery_widgets.select_btn.clone(),
selection: gallery_widgets.selection.clone(),
more: gallery_widgets.more.clone(),
gallery_actions: gallery_widgets.gallery_actions.clone(),
upload_face: gallery_widgets.upload_face.clone(),
float_heading: gallery_widgets.float_heading.clone(),
float_label: gallery_widgets.float_label.clone(),
@@ -725,6 +735,10 @@ fn build_window(app: &adw::Application) {
resume_button: takeout_widgets.resume_button.clone(),
inflight: Cell::new(false),
running: Cell::new(false),
restored: Cell::new(false),
save_seq: Cell::new(0),
saved_seq: Default::default(),
repaint_seq: Cell::new(0),
},
update,
});
@@ -787,6 +801,11 @@ fn build_window(app: &adw::Application) {
// thumbnail generation as it paints.
cancel_file_thumbnails(&ui_nav);
sync_sidebar(&ui_nav);
if st.visible_child_name().as_deref() != Some("gallery")
&& let Some(browser) = ui_nav.drive_galleries.borrow().upgrade()
{
browser.close();
}
match st.visible_child_name().as_deref() {
Some("browser") => reload_browser_view(&ui_nav),
// The Albums and Places listings are re-read in place. Loading the
@@ -796,7 +815,7 @@ fn build_window(app: &adw::Application) {
// sight (see `follow_changes`) or nothing is loaded yet.
Some("gallery") if !ui_nav.gallery.in_collection() => {
match ui_nav.gallery.view_switch.active() {
VIEW_ALBUMS => load_albums(&ui_nav),
VIEW_ALBUMS => show_albums(&ui_nav),
VIEW_PLACES => load_places(&ui_nav),
_ if ui_nav.gallery.timeline_stale.replace(false)
|| ui_nav.gallery.model.n_items() == 0 =>
@@ -833,6 +852,14 @@ fn build_window(app: &adw::Application) {
.height_request(WindowState::MIN_HEIGHT)
.content(&toasts)
.build();
pages::drive_galleries::wire(
&ui,
&window,
&gallery_widgets.drive_galleries,
&gallery_widgets.drive_status,
&gallery_widgets.add_gallery,
&gallery_widgets.hidden_albums,
);
remember_window_state(&ui, app, &window);
// A wide window docks the details pane beside the files instead of laying it
// over them.
@@ -1456,7 +1483,7 @@ fn install_window_actions(ui: &Rc<Ui>, window: &adw::ApplicationWindow) {
None,
gtk4::License::Custom,
Some(&gettext(
"The places map is drawn from <a href=\"https://www.naturalearthdata.com/\">Natural Earth</a> land outlines and country borders, which are in the public domain.",
"The places map is drawn from <a href=\"https://www.naturalearthdata.com/\">Natural Earth</a> land, water and borders, which are in the public domain.",
)),
);
dialog.add_legal_section(
@@ -1673,7 +1700,9 @@ fn install_shortcuts(ui: &Rc<Ui>, window: &adw::ApplicationWindow) {
let on_trash = page.as_deref() == Some("trash");
let on_gallery = page.as_deref() == Some("gallery");
// Pages with a level to go back to, besides My files' own history.
let in_collection = on_gallery && ui.gallery.in_collection();
let in_collection = on_gallery
&& (ui.gallery.in_collection()
|| ui.gallery.content.visible_child_name().as_deref() == Some("drive-galleries"));
let in_shared_folder =
page.as_deref() == Some("shared") && !ui.shared.nav.borrow().is_empty();
let in_device_folder =
+35 -8
View File
@@ -18,6 +18,9 @@ const COVER_EDGE: i32 = 200;
pub(crate) fn show_albums(ui: &Rc<Ui>) {
show_album_grid(ui);
load_albums(ui);
if let Some(browser) = ui.drive_galleries.borrow().upgrade() {
browser.load_root();
}
}
/// Switch to the album grid as it was last filled, keeping its scroll
@@ -102,7 +105,7 @@ fn fill_albums(ui: &Rc<Ui>, albums: &[AlbumInfo]) {
ui.gallery.albums.remove(&child);
}
*ui.gallery.album_list.borrow_mut() = albums.to_vec();
if ui.gallery.album.borrow().is_none() {
if ui.gallery.content.visible_child_name().as_deref() == Some("albums") {
ui.gallery.title.set_subtitle(&ngettext_f(
"{n} album",
"{n} albums",
@@ -125,7 +128,12 @@ fn fill_albums(ui: &Rc<Ui>, albums: &[AlbumInfo]) {
/// One album as a clickable card: its cover, its name, and how many photos it
/// holds. An album shared with us says so — it lives on someone else's volume,
/// and that is worth knowing before opening it.
fn album_card(ui: &Rc<Ui>, album: &AlbumInfo) -> gtk4::Button {
pub(crate) fn collection_card(
title: &str,
subtitle: &str,
icon: &str,
(width, height): (i32, i32),
) -> (gtk4::Button, gtk4::Picture) {
let picture = gtk4::Picture::builder()
// Fills and crops the cover square, like a timeline tile.
.content_fit(gtk4::ContentFit::Cover)
@@ -136,7 +144,7 @@ fn album_card(ui: &Rc<Ui>, album: &AlbumInfo) -> gtk4::Button {
picture.add_css_class("photo-thumb");
let placeholder = gtk4::Image::builder()
.icon_name("pdfs-album-symbolic")
.icon_name(icon)
.pixel_size(32)
.halign(gtk4::Align::Center)
.valign(gtk4::Align::Center)
@@ -146,12 +154,12 @@ fn album_card(ui: &Rc<Ui>, album: &AlbumInfo) -> gtk4::Button {
let cover = gtk4::Overlay::new();
cover.set_child(Some(&placeholder));
cover.add_overlay(&picture);
cover.set_size_request(COVER_EDGE, COVER_EDGE);
cover.set_size_request(width, height);
cover.set_overflow(gtk4::Overflow::Hidden);
cover.add_css_class("album-cover");
let name = gtk4::Label::builder()
.label(&album.name)
.label(title)
.halign(gtk4::Align::Start)
.xalign(0.0)
.ellipsize(gtk4::pango::EllipsizeMode::End)
@@ -159,7 +167,7 @@ fn album_card(ui: &Rc<Ui>, album: &AlbumInfo) -> gtk4::Button {
name.add_css_class("heading");
let count = gtk4::Label::builder()
.label(album_subtitle(album))
.label(subtitle)
.halign(gtk4::Align::Start)
.xalign(0.0)
.ellipsize(gtk4::pango::EllipsizeMode::End)
@@ -172,10 +180,23 @@ fn album_card(ui: &Rc<Ui>, album: &AlbumInfo) -> gtk4::Button {
card.append(&name);
card.append(&count);
let button = gtk4::Button::builder().child(&card).build();
let button = gtk4::Button::builder()
.child(&card)
.halign(gtk4::Align::Start)
.build();
button.add_css_class("flat");
button.add_css_class("album-card");
button.set_tooltip_text(Some(&album.name));
button.set_tooltip_text(Some(title));
(button, picture)
}
fn album_card(ui: &Rc<Ui>, album: &AlbumInfo) -> gtk4::Button {
let (button, picture) = collection_card(
&album.name,
&album_subtitle(album),
"pdfs-album-symbolic",
(COVER_EDGE, COVER_EDGE),
);
if let Some(uid) = album.cover_uid.clone() {
want_cover(ui, uid, &picture);
@@ -693,6 +714,12 @@ pub(crate) fn wire_albums(ui: &Rc<Ui>) {
// where it was, instead of reloading it.
let ui_back = ui.clone();
ui.gallery.back.clone().connect_clicked(move |_| {
if ui_back.gallery.content.visible_child_name().as_deref() == Some("drive-galleries") {
if let Some(browser) = ui_back.drive_galleries.borrow().upgrade() {
browser.back();
}
return;
}
if ui_back.gallery.place.borrow().is_some() {
show_place_grid(&ui_back);
return;
+2
View File
@@ -3185,6 +3185,7 @@ pub(crate) fn repaint_search(ui: &Rc<Ui>, hits: &[SearchHit]) {
shared_by_unverified: false,
trashed_at: 0,
trashed_from: None,
trashed_parent: None,
issue: None,
})
.collect();
@@ -3249,6 +3250,7 @@ mod tests {
shared_by_unverified: false,
trashed_at: 0,
trashed_from: None,
trashed_parent: None,
issue: None,
}
}
File diff suppressed because it is too large. Load diff
@@ -788,6 +788,9 @@ pub(crate) fn issue_text(issue: SyncIssue) -> String {
SyncIssue::Access => gettext(
"You can no longer change this on Proton Drive. Ask the owner for edit access, or export the file to keep your version.",
),
SyncIssue::Trashed => {
gettext("Restore the item or folder from Trash, then retry the move.")
}
SyncIssue::Missing => gettext(
"The folder this change belongs in is gone from Proton Drive. Export the file to keep your version.",
),
+43 -3
View File
@@ -336,6 +336,9 @@ pub(crate) fn login_error_message(error: &pdfs_core::Error) -> String {
pdfs_core::Error::WrongMailboxPassword => {
gettext("That mailbox password didn't unlock your account.")
}
pdfs_core::Error::SecurityKeyOnly => gettext(
"This account's only second factor is a security key, which this app can't use yet. Add an authenticator app in your Proton account settings, then sign in with its code.",
),
pdfs_core::Error::Other(message) if message.contains("verification") => gettext(
"Sign-in cancelled: the verification wasn't completed. Sign in again to get a new one.",
),
@@ -672,17 +675,51 @@ fn verify_in_browser(
/// Sign out, after a confirmation: disable+stop the mount service (so the
/// daemon isn't left running without credentials), forget the stored session,
/// and drop back to the login page.
///
/// Changes still queued are named first: they stay staged here and are sent
/// after the next sign-in, but until then they exist nowhere else.
pub(crate) fn sign_out(ui: &Rc<Ui>) {
let rx = spawn_request(ui.dirs.control_socket(), Request::Status);
let ui = ui.clone();
glib::spawn_future_local(async move {
let queued = match rx.recv().await {
Ok(Ok(Response::Status {
pending_uploads,
pending_changes,
parked_uploads,
..
})) => pending_uploads + pending_changes + parked_uploads,
_ => 0,
};
confirm_sign_out(&ui, queued);
});
}
fn confirm_sign_out(ui: &Rc<Ui>, queued: u64) {
let Some(window) = ui_window(ui) else { return };
let ui = ui.clone();
let mut body = gettext(
"Proton Drive will disconnect and stop syncing until you sign in again. Files already on this computer stay where they are.",
);
if queued > 0 {
let warning = ngettext_f(
"{n} change is not on Proton Drive yet. It stays on this computer and is sent after you sign in again.",
"{n} changes are not on Proton Drive yet. They stay on this computer and are sent after you sign in again.",
queued,
&[],
);
body = format!("{warning}\n\n{body}");
}
confirm_destructive(
&window,
&gettext("Sign Out?"),
&gettext(
"Proton Drive will disconnect and stop syncing until you sign in again. Files already on this computer stay where they are.",
),
&body,
&gettext("Sign Out"),
move || {
let galleries = ui.drive_galleries.borrow().upgrade();
if let Some(galleries) = galleries {
galleries.reset();
}
// systemctl and the keyring both block, so run them on a worker.
let ui = ui.clone();
glib::spawn_future_local(async move {
@@ -707,6 +744,9 @@ mod tests {
#[test]
fn a_cancelled_prompt_says_which_step_was_skipped() {
assert!(
login_error_message(&pdfs_core::Error::SecurityKeyOnly).contains("authenticator app")
);
let totp = pdfs_core::Error::Other("two-factor entry cancelled".into());
assert!(login_error_message(&totp).contains("two-factor code"));
let hv = pdfs_core::Error::Other("verification cancelled".into());
+211 -22
View File
@@ -1,7 +1,12 @@
//! The online street map under the Places map: vector tiles from OpenFreeMap,
//! fetched and drawn on a worker thread, and handed to the map as finished
//! images.
//! The tiles under the Places map, made on worker threads and handed to the
//! map as finished images. They come from one of two sources.
//!
//! The offline basemap is a PMTiles file installed with the app and drawn by
//! `proton-map-rs`, the same map Proton Gallery shows: seas, lakes, rivers,
//! built-up areas, borders and town names from Natural Earth and GeoNames.
//! It sends nothing anywhere. Its names are drawn into the tiles.
//!
//! The online street map is vector tiles from OpenFreeMap.
//! It is off unless turned on in Preferences, since every tile fetched tells
//! OpenFreeMap which part of the world is being looked at. OpenFreeMap needs
//! no account or key, and its tiles are OpenStreetMap data in the
@@ -20,9 +25,17 @@ use std::sync::{Arc, Mutex};
use std::time::Duration;
use gtk4::cairo;
use proton_map::{Basemap, Style, TileId};
use super::mvt::{self, Kind, Layer};
/// The basemap's file name, wherever it is installed.
pub(crate) const BASEMAP_FILE: &str = "basemap-v1.pmtiles";
/// How many threads draw basemap tiles. Drawing is all CPU; two keep the map
/// quick without taking the machine.
const BASEMAP_THREADS: usize = 2;
/// Where OpenFreeMap describes its current tile set.
const TILEJSON_URL: &str = "https://tiles.openfreemap.org/planet";
@@ -254,8 +267,20 @@ struct Job {
dark: bool,
}
/// The worker thread making tiles. Dropping it stops the thread once the
/// fetches under way are done.
/// Where a tile worker gets its tiles.
pub(crate) enum Source {
/// OpenFreeMap's street map, cached under `cache_dir`, with names in
/// `language` where the map has them.
Online {
cache_dir: PathBuf,
language: String,
},
/// The basemap installed at this path.
Offline(PathBuf),
}
/// The worker threads making tiles. Dropping it stops them once the tiles
/// under way are done.
pub(crate) struct Tiles {
jobs: async_channel::Sender<Job>,
wanted: Arc<Mutex<HashSet<TileKey>>>,
@@ -264,27 +289,35 @@ pub(crate) struct Tiles {
}
impl Tiles {
/// Start the worker, caching under `cache_dir`, drawing names in
/// `language` where the map has them. Finished tiles arrive on the
/// returned channel.
pub(crate) fn start(
cache_dir: PathBuf,
language: String,
scale: i32,
) -> (Self, async_channel::Receiver<Done>) {
/// Start the worker for `source`. Finished tiles arrive on the returned
/// channel.
pub(crate) fn start(source: Source, scale: i32) -> (Self, async_channel::Receiver<Done>) {
let (jobs, job_rx) = async_channel::unbounded();
let (done_tx, done_rx) = async_channel::unbounded();
let wanted = Arc::new(Mutex::new(HashSet::new()));
let worker = Worker {
dir: cache_dir,
language,
scale,
wanted: wanted.clone(),
done: done_tx,
let spawned = match source {
Source::Online {
cache_dir,
language,
} => {
let worker = Worker {
dir: cache_dir,
language,
scale,
wanted: wanted.clone(),
done: done_tx,
};
std::thread::Builder::new()
.name("map-tiles".into())
.spawn(move || worker.run(job_rx))
}
Source::Offline(path) => {
let wanted = wanted.clone();
std::thread::Builder::new()
.name("map-basemap".into())
.spawn(move || draw_basemap(&path, scale, &wanted, &job_rx, &done_tx))
}
};
let spawned = std::thread::Builder::new()
.name("map-tiles".into())
.spawn(move || worker.run(job_rx));
if let Err(e) = spawned {
tracing::warn!("map tiles: cannot start the worker: {e}");
}
@@ -463,6 +496,126 @@ impl Worker {
}
}
/// Where the basemap may be installed, most specific first.
fn basemap_paths() -> Vec<PathBuf> {
let mut paths = Vec::new();
if let Some(path) = std::env::var_os("PDFS_BASEMAP") {
paths.push(PathBuf::from(path));
}
if let Some(dir) = std::env::current_exe()
.ok()
.and_then(|exe| exe.parent().map(Path::to_path_buf))
{
paths.push(dir.join("../share/proton-drive-linux").join(BASEMAP_FILE));
}
paths.push(Path::new("/usr/share/proton-drive-linux").join(BASEMAP_FILE));
if let Some(dir) = std::env::var_os("XDG_DATA_HOME")
.map(PathBuf::from)
.or_else(|| std::env::var_os("HOME").map(|home| Path::new(&home).join(".local/share")))
{
paths.push(dir.join("proton-drive-linux").join(BASEMAP_FILE));
}
if cfg!(debug_assertions) {
// `scripts/fetch-basemap.sh` puts it here for development builds.
paths
.push(Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../data")).join(BASEMAP_FILE));
}
paths
}
/// The installed basemap's path, if there is one.
pub(crate) fn find_basemap() -> Option<PathBuf> {
basemap_paths().into_iter().find(|path| path.is_file())
}
/// The basemap, read once per run: the map restarts its worker when the
/// window moves to a screen of another density, and the file is 25 MB.
fn open_basemap(path: &Path) -> Option<Arc<Basemap>> {
static OPENED: Mutex<Option<(PathBuf, Arc<Basemap>)>> = Mutex::new(None);
let mut opened = OPENED.lock().ok()?;
if let Some((at, basemap)) = opened.as_ref()
&& at == path
{
return Some(basemap.clone());
}
match Basemap::open(path) {
Ok(basemap) => {
let basemap = Arc::new(basemap);
*opened = Some((path.to_path_buf(), basemap.clone()));
Some(basemap)
}
Err(e) => {
tracing::warn!("map tiles: cannot open the basemap: {e}");
None
}
}
}
/// Open the basemap at `path` and draw the tiles asked for on
/// [`BASEMAP_THREADS`] threads until the map lets go of the worker. Without
/// a readable basemap every tile fails, and the outline shows.
fn draw_basemap(
path: &Path,
scale: i32,
wanted: &Mutex<HashSet<TileKey>>,
jobs: &async_channel::Receiver<Job>,
done: &async_channel::Sender<Done>,
) {
let basemap = open_basemap(path);
std::thread::scope(|scope| {
for _ in 0..BASEMAP_THREADS {
let basemap = basemap.clone();
scope.spawn(move || {
while let Ok(job) = jobs.recv_blocking() {
let key = job.key;
let still_wanted = wanted.lock().map(|w| w.contains(&key)).unwrap_or(true);
let result = match &basemap {
_ if !still_wanted => Done::Dropped(key),
Some(basemap) => Done::Tile(draw_basemap_tile(basemap, job, scale)),
None => Done::Failed(key),
};
if done.send_blocking(result).is_err() {
break;
}
}
});
}
});
}
/// One basemap tile at `scale` device px per logical px.
fn draw_basemap_tile(basemap: &Basemap, job: Job, scale: i32) -> Raster {
let style = if job.dark {
Style::dark()
} else {
Style::light()
};
let key = job.key;
let tile = proton_map::render_tile(
basemap,
TileId::new(key.z, key.x, key.y),
scale.max(1) as f32,
&style,
);
Raster {
key,
dark: job.dark,
edge: tile.width as i32,
pixels: rgba_to_cairo(tile.rgba),
labels: Vec::new(),
}
}
/// Premultiplied RGBA bytes to cairo's premultiplied ARGB, one native-endian
/// word per pixel.
fn rgba_to_cairo(mut pixels: Vec<u8>) -> Vec<u8> {
for px in pixels.as_chunks_mut::<4>().0 {
let [r, g, b, a] = *px;
*px = u32::from_be_bytes([a, r, g, b]).to_ne_bytes();
}
pixels
}
/// The first tile URL template in a TileJSON document.
fn tile_template(json: &[u8]) -> Option<String> {
let doc: serde_json::Value = serde_json::from_slice(json).ok()?;
@@ -823,6 +976,42 @@ fn label_name(layer: &Layer, feature: &mvt::Feature, language: &str) -> Option<S
mod tests {
use super::*;
fn empty_basemap() -> Basemap {
Basemap::from_bytes(proton_map::pmtiles::writer::Writer::new().finish(
"{}",
[-180.0, -85.0, 180.0, 85.0],
[0.0, 0.0],
))
.expect("empty archive opens")
}
#[test]
fn rgba_pixels_become_cairo_words() {
let pixels = rgba_to_cairo(vec![0x11, 0x22, 0x33, 0xff]);
let word = u32::from_ne_bytes(pixels.try_into().expect("one pixel"));
assert_eq!(word, 0xff11_2233);
}
#[test]
fn basemap_tiles_come_at_the_screen_density_in_the_theme_asked() {
let basemap = empty_basemap();
let key = TileKey { z: 3, x: 4, y: 2 };
for dark in [false, true] {
let raster = draw_basemap_tile(&basemap, Job { key, dark }, 2);
assert_eq!((raster.key, raster.dark), (key, dark));
assert_eq!(raster.edge, 512);
assert_eq!(raster.pixels.len(), 512 * 512 * 4);
// An empty basemap is all open sea.
let sea = if dark { Style::dark() } else { Style::light() }.water[0];
let word = u32::from_ne_bytes(raster.pixels[..4].try_into().expect("a pixel"));
assert_eq!(
word & 0x00ff_ffff,
u32::from_be_bytes([0, sea.r, sea.g, sea.b])
);
assert!(raster.surface().is_some());
}
}
#[test]
fn deep_tiles_draw_from_their_ancestor_at_the_data_zoom() {
let key = TileKey {
+1
View File
@@ -3,6 +3,7 @@ pub(crate) mod activity_text;
pub(crate) mod albums;
pub(crate) mod browser;
pub(crate) mod devices;
pub(crate) mod drive_galleries;
pub(crate) mod duplicates;
pub(crate) mod locations;
pub(crate) mod login;
+128 -16
View File
@@ -31,6 +31,11 @@ pub(crate) struct Viewer {
pub(crate) scroller: gtk4::ScrolledWindow,
pub(crate) picture: gtk4::Picture,
pub(crate) video: gtk4::Video,
video_token: RefCell<Option<String>>,
load_generation: Cell<u64>,
presence_loading: Cell<bool>,
presence_pending: Cell<bool>,
album: Option<String>,
pub(crate) spinner: Spinner,
pub(crate) status: gtk4::Label,
pub(crate) title: gtk4::Label,
@@ -272,12 +277,14 @@ fn viewer_item(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: &str) -> Option<PhotoItem>
/// full-size file, and swap it in — plus its EXIF — when it lands. A video
/// plays in place once its file is down.
pub(crate) fn load_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: String) {
let generation = viewer.load_generation.get().wrapping_add(1);
viewer.load_generation.set(generation);
viewer.spinner.set_visible(true);
viewer.status.set_visible(false);
viewer.loading.set(true);
*viewer.path.borrow_mut() = None;
clear_info(viewer);
stop_video(viewer);
stop_video(ui, viewer);
viewer.media.set_visible_child_name("photo");
sync_strip(viewer);
reset_zoom(viewer);
@@ -302,17 +309,25 @@ pub(crate) fn load_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: String) {
None => viewer.picture.set_paintable(gtk4::gdk::Paintable::NONE),
}
let rx = spawn_request(
ui.dirs.control_socket(),
Request::OpenPhoto { uid: uid.clone() },
);
let request = if is_video {
Request::OpenGalleryVideo {
uid: uid.clone(),
private: false,
}
} else {
Request::OpenPhoto { uid: uid.clone() }
};
let rx = spawn_request(ui.dirs.control_socket(), request);
let viewer = viewer.clone();
let ui = ui.clone();
glib::spawn_future_local(async move {
let result = rx.recv().await;
// The user may have moved on while this was in flight; that photo's own
// request owns the viewer now.
if *viewer.uid.borrow() != uid {
if viewer.load_generation.get() != generation {
if let Ok(Ok(Response::GalleryVideo { token, .. })) = result {
close_video(&ui, token);
}
return;
}
viewer.spinner.set_visible(false);
@@ -326,6 +341,14 @@ pub(crate) fn load_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>, uid: String) {
viewer.status.set_visible(true);
};
let path = match result {
Ok(Ok(Response::GalleryVideo { url, token, size })) => {
*viewer.video_token.borrow_mut() = Some(token);
viewer.video.set_file(Some(&gio::File::for_uri(&url)));
viewer.media.set_visible_child_name("video");
sync_strip(&viewer);
show_info(&viewer, Some(size), ExifInfo::default());
return;
}
Ok(Ok(Response::FilePath { path })) => path,
Ok(Ok(Response::Error { message, .. })) => return fail(&message),
Ok(Ok(_)) => return fail(&gettext("Unexpected reply from the Proton Drive service.")),
@@ -481,8 +504,18 @@ fn next_slide(ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
/// Stop whatever the player was playing, so stepping off a video (or closing
/// the lightbox) doesn't leave its sound running.
fn stop_video(viewer: &Rc<Viewer>) {
fn close_video(ui: &Ui, token: String) {
let _reply = spawn_request(
ui.dirs.control_socket(),
Request::CloseGalleryVideo { token },
);
}
fn stop_video(ui: &Ui, viewer: &Rc<Viewer>) {
viewer.video.set_media_stream(gtk4::MediaStream::NONE);
if let Some(token) = viewer.video_token.borrow_mut().take() {
close_video(ui, token);
}
}
/// Warm the cache with the photo `delta` steps away, so stepping there is
@@ -1210,6 +1243,43 @@ fn build_strip(ui: &Rc<Ui>) -> (gtk4::ListView, gtk4::SingleSelection) {
///
/// Returns the lightbox's window, for a caller that has to put the page
/// behind it back once it closes.
pub(crate) fn check_photo_viewer(ui: &Rc<Ui>) {
let Some(viewer) = ui.gallery.viewer.borrow().upgrade() else {
return;
};
if !viewer.window.is_visible() {
return;
}
if viewer.presence_loading.replace(true) {
viewer.presence_pending.set(true);
return;
}
let uid = viewer.uid.borrow().clone();
let rx = spawn_request(
ui.dirs.control_socket(),
Request::PhotoPresence {
uid: uid.clone(),
album: viewer.album.clone(),
},
);
let ui = ui.clone();
let viewer = Rc::downgrade(&viewer);
glib::spawn_future_local(async move {
let result = rx.recv().await;
let Some(viewer) = viewer.upgrade() else {
return;
};
viewer.presence_loading.set(false);
if *viewer.uid.borrow() == uid
&& matches!(result, Ok(Ok(Response::PhotoPresence { present: false })))
{
viewer.window.close();
} else if viewer.presence_pending.replace(false) {
check_photo_viewer(&ui);
}
});
}
pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) -> gtk4::Window {
let parent = ui.stack.root().and_downcast::<gtk4::Window>().unwrap();
@@ -1473,6 +1543,16 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) -> gtk4::Windo
scroller: scroller.clone(),
picture: picture.clone(),
video,
video_token: RefCell::new(None),
load_generation: Cell::new(0),
presence_loading: Cell::new(false),
presence_pending: Cell::new(false),
album: ui
.gallery
.album
.borrow()
.as_ref()
.map(|album| album.uid.clone()),
spinner,
status: status_label,
title: title_label,
@@ -1515,6 +1595,7 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) -> gtk4::Windo
toasts,
});
*ui.gallery.viewer.borrow_mut() = Rc::downgrade(&viewer);
let initial_idx = find_photo_index(&ui.gallery.model, &initial_uid).unwrap_or(0);
let capture_time = viewer_item(ui, &viewer, &initial_uid).map_or(0, |item| item.capture_time);
show_photo_position(ui, &viewer, initial_idx, capture_time);
@@ -1700,7 +1781,10 @@ pub(crate) fn open_photo_viewer(ui: &Rc<Ui>, initial_uid: String) -> gtk4::Windo
}
if let Some(viewer) = keep.borrow_mut().take() {
reveal_photo(&ui_close, &viewer.uid.borrow());
stop_video(&viewer);
viewer
.load_generation
.set(viewer.load_generation.get().wrapping_add(1));
stop_video(&ui_close, &viewer);
disarm_slide(&viewer);
if let Some(source) = viewer.chrome_source.borrow_mut().take() {
source.remove();
@@ -1952,7 +2036,7 @@ fn open_viewer_menu(ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
}
menu.section();
// These need the full-size file, which may still be downloading.
if viewer.path.borrow().is_some() {
if viewer.path.borrow().is_some() || viewer.video_token.borrow().is_some() {
let still = viewer.media.visible_child_name().as_deref() == Some("photo");
if still {
// Translators: menu item that puts the photo on the clipboard as an image.
@@ -1999,17 +2083,45 @@ fn open_viewer_menu(ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
});
}
fn save_shown_photo(_ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
if let Some(path) = viewer.path.borrow().as_deref() {
let name = viewer.name.borrow();
save_photo_to_disk(viewer, path, name.as_deref().unwrap_or("photo"));
}
fn save_shown_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
with_viewer_file(ui, viewer, |viewer, path, name| {
save_photo_to_disk(viewer, path, name.unwrap_or("photo"))
});
}
fn open_shown_photo(_ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
fn open_shown_photo(ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
with_viewer_file(ui, viewer, |viewer, path, name| {
open_with(&viewer.window, path, name)
});
}
fn with_viewer_file(
ui: &Rc<Ui>,
viewer: &Rc<Viewer>,
use_file: impl FnOnce(&Rc<Viewer>, &str, Option<&str>) + 'static,
) {
let name = viewer.name.borrow().clone();
if let Some(path) = viewer.path.borrow().as_deref() {
open_with(&viewer.window, path, viewer.name.borrow().as_deref());
use_file(viewer, path, name.as_deref());
return;
}
if viewer.video_token.borrow().is_none() {
return;
}
let uid = viewer.uid.borrow().clone();
let viewer = viewer.clone();
let rx = spawn_request(ui.dirs.control_socket(), Request::OpenPhoto { uid });
glib::spawn_future_local(async move {
match rx.recv().await {
Ok(Ok(Response::FilePath { path })) => use_file(&viewer, &path, name.as_deref()),
Ok(Ok(Response::Error { message, .. })) => {
viewer.toasts.add_toast(adw::Toast::new(&message))
}
_ => viewer
.toasts
.add_toast(adw::Toast::new(&gettext("Couldn't open the file."))),
}
});
}
fn add_shown_photo_to_album(ui: &Rc<Ui>, viewer: &Rc<Viewer>) {
File diff suppressed because it is too large. Load diff
+185 -130
View File
@@ -2,13 +2,13 @@
//! their newest photo, and once the map is zoomed in close, each photo where
//! it was taken.
//!
//! Underneath is one of two maps. The built-in one is drawn here from Natural
//! Underneath are up to two maps. The outline is drawn here from Natural
//! Earth's 1:50m land polygons and country borders (public domain), trimmed
//! into `data/land.txt` and `data/borders.txt` by
//! `scripts/natural-earth-land.py`; it needs no network and tells nobody
//! anything. The street map from OpenFreeMap ([`super::map_tiles`]) is
//! opt-in, in Preferences, and draws over the built-in one wherever its tiles
//! have arrived.
//! `scripts/natural-earth-land.py`, and is always there. Over it go tiles
//! ([`super::map_tiles`]) wherever they have arrived: the offline basemap
//! installed with the app, which needs no network and tells nobody anything,
//! or, when turned on in Preferences, the street map from OpenFreeMap.
//!
//! The projection is Web Mercator, the one every other map uses, so the
//! world looks the way people expect. Markers that would overlap on screen
@@ -17,6 +17,8 @@
//! a single photo opens it in its town, and photos taken on one spot, which
//! never part, are listed to pick from.
// Markers merge the same way in every Proton gallery app.
use proton_gallery_rs::places::{Bubble, bubble_radius, cluster, compact_count};
use std::collections::{HashMap, HashSet};
use std::path::PathBuf;
use std::sync::OnceLock;
@@ -25,7 +27,7 @@ use std::time::{Duration, Instant};
use gtk4::cairo;
use pdfs_core::control::PhotoLocation;
use super::map_tiles::{self, Done, Label, LabelClass, Palette, TILE_EDGE, TileKey, Tiles};
use super::map_tiles::{self, Done, Label, LabelClass, Palette, Source, TILE_EDGE, TileKey, Tiles};
use crate::*;
const LAND: &str = include_str!("../../../data/land.txt");
@@ -35,8 +37,8 @@ const BORDERS: &str = include_str!("../../../data/borders.txt");
const MAX_LATITUDE: f64 = 85.0;
/// How far in the map zooms, as a multiple of the whole world fitting the
/// widget: far enough for photos in one town to part on the built-in map,
/// and down to single streets on the street map.
/// widget: far enough for photos in one town to part on the outline and the
/// basemap, and down to single streets on the street map.
const MAX_ZOOM: f64 = 8192.0;
const ONLINE_MAX_ZOOM: f64 = 65536.0;
@@ -64,9 +66,14 @@ const MIN_SEGMENT: f64 = 0.75;
/// Edge in px of the square a marker's photo is scaled down to once.
const COVER_PX: i32 = 128;
/// How many street-map tiles stay in memory.
/// How many map tiles stay in memory.
const TILE_CACHE: usize = 160;
/// Photo marker covers kept, 64 KiB each. A town's own cover is never
/// dropped (it only arrives with the town's card), but the photos zoomed past
/// one by one would otherwise pile up for as long as the map is open.
const COVER_CACHE: usize = 1024;
/// How long a tile that couldn't be had is left alone before it is tried
/// again.
const TILE_RETRY: Duration = Duration::from_secs(60);
@@ -132,13 +139,6 @@ fn parse_ring(line: &str) -> Option<Ring> {
Some(Ring { points, min, max })
}
/// Radius in px of a marker holding `photos` photos: big enough to make the
/// photo out, and growing with the count, but slowly, so a home town with
/// thousands doesn't cover a continent.
fn bubble_radius(photos: usize) -> f64 {
18.0 + 2.0 * (photos.max(1) as f64).ln()
}
/// What the map shows: which part of the world, at what size.
#[derive(Clone, Copy, Debug)]
struct View {
@@ -190,18 +190,7 @@ struct Marker {
target: Target,
}
/// One marker as last drawn, standing for one or more [`Marker`]s: where, how
/// big, and which (indices into [`MapState::markers`], the one with the most
/// photos first).
struct Bubble {
x: f64,
y: f64,
radius: f64,
photos: usize,
members: Vec<usize>,
}
/// A street-map tile ready to draw, and the names on it.
/// A map tile ready to draw, and the names on it.
struct CachedTile {
surface: cairo::ImageSurface,
labels: Rc<Vec<Label>>,
@@ -209,9 +198,11 @@ struct CachedTile {
used: u64,
}
/// The street map, while it is turned on.
struct Online {
/// The tiles drawn over the outline, from the basemap or the street map.
struct TileLayer {
tiles: Tiles,
/// Whether they come from the street map.
online: bool,
cache: HashMap<TileKey, CachedTile>,
pending: HashSet<TileKey>,
failed: HashMap<TileKey, Instant>,
@@ -232,8 +223,11 @@ struct MapState {
/// for from close up.
locations: HashMap<u32, Vec<(PhotoItem, World)>>,
asked_locations: HashSet<u32>,
/// Marker photos, scaled and cropped to a square, by photo uid.
covers: HashMap<String, cairo::ImageSurface>,
/// Marker photos, scaled and cropped to a square, by photo uid, with the
/// draw they were last shown in.
covers: HashMap<String, (cairo::ImageSurface, u64)>,
/// Counts draws, for dropping the covers shown longest ago.
draws: u64,
asked_thumbs: HashSet<String>,
view: View,
/// False until the view has been fitted to the places at a real size, and
@@ -247,7 +241,11 @@ struct MapState {
drag_origin: Option<View>,
/// The scale a pinch started from.
pinch_origin: Option<View>,
online: Option<Online>,
tiles: Option<TileLayer>,
/// Whether the street map is turned on in Preferences.
street_map: bool,
/// The installed basemap, if any.
basemap: Option<PathBuf>,
cache_dir: Option<PathBuf>,
/// The spot [`PlacesMap::show_spot`] went to, ringed until the map is
/// fitted to all places again.
@@ -303,32 +301,11 @@ impl PlacesMap {
buttons.append(&zoom);
buttons.append(&fit);
let link = |url: &str, name: &str| format!("<a href=\"{url}\">{name}</a>");
let credit = gtk4::Label::builder()
.use_markup(true)
.halign(gtk4::Align::End)
.valign(gtk4::Align::End)
.visible(false)
.label(gettext_f(
// Translators: the credit on the street map. {openfreemap},
// {openmaptiles} and {openstreetmap} are the linked names of
// the projects, which stay as they are.
"{openfreemap} © {openmaptiles} Data from {openstreetmap}",
&[
(
"openfreemap",
&link("https://openfreemap.org", "OpenFreeMap"),
),
(
"openmaptiles",
&link("https://www.openmaptiles.org/", "OpenMapTiles"),
),
(
"openstreetmap",
&link("https://www.openstreetmap.org/copyright", "OpenStreetMap"),
),
],
))
.build();
credit.add_css_class("places-map-credit");
credit.add_css_class("caption");
@@ -349,6 +326,7 @@ impl PlacesMap {
locations: HashMap::new(),
asked_locations: HashSet::new(),
covers: HashMap::new(),
draws: 0,
asked_thumbs: HashSet::new(),
view: View {
center: World { x: 0.5, y: 0.5 },
@@ -361,7 +339,9 @@ impl PlacesMap {
pointer: (0.0, 0.0),
drag_origin: None,
pinch_origin: None,
online: None,
tiles: None,
street_map: false,
basemap: map_tiles::find_basemap(),
cache_dir: None,
pin: None,
}),
@@ -486,43 +466,68 @@ impl PlacesMap {
let Some(cover) = square_cover(texture) else {
return;
};
self.state.borrow_mut().covers.insert(uid.to_owned(), cover);
let mut state = self.state.borrow_mut();
let state = &mut *state;
state.covers.insert(uid.to_owned(), (cover, state.draws));
let towns: HashSet<&str> = state
.places
.iter()
.map(|(place, _)| place.cover.uid.as_str())
.collect();
for dropped in evict_covers(&mut state.covers, &towns, state.draws) {
// Asked for again once it is back on screen.
state.asked_thumbs.remove(&dropped);
}
self.area.queue_draw();
}
/// Draw OpenFreeMap's street map under the markers, or stop, keeping its
/// tiles in `cache_dir`.
/// Draw OpenFreeMap's street map under the markers, keeping its tiles in
/// `cache_dir`, or else the installed basemap.
pub(crate) fn set_online(self: &Rc<Self>, on: bool, cache_dir: PathBuf) {
{
let mut state = self.state.borrow_mut();
state.cache_dir = Some(cache_dir);
if on == state.online.is_some() {
let started = state.tiles.is_some() || state.basemap.is_none();
if on == state.street_map && started {
return;
}
state.online = None;
}
self.credit.set_visible(on);
if on {
self.start_tiles();
state.street_map = on;
state.tiles = None;
}
self.start_tiles();
self.clamp_zoom();
self.area.queue_draw();
}
/// Start a tile worker for the widget's current pixel density.
/// Start a tile worker for the widget's current pixel density, and show
/// the credit for what it draws.
fn start_tiles(self: &Rc<Self>) {
let Some(dir) = self.state.borrow().cache_dir.clone() else {
let source = {
let state = self.state.borrow();
match (&state.cache_dir, &state.basemap) {
(Some(dir), _) if state.street_map => {
let language = gtk4::pango::Language::default().to_string();
let language = language.split(['-', '_']).next().unwrap_or("en").to_owned();
Some(Source::Online {
cache_dir: dir.join("map-tiles"),
language,
})
}
(_, Some(basemap)) => Some(Source::Offline(basemap.clone())),
_ => None,
}
};
let Some(source) = source else {
self.credit.set_visible(false);
return;
};
let language = gtk4::pango::Language::default().to_string();
let language = language.split(['-', '_']).next().unwrap_or("en").to_owned();
let (tiles, done) = Tiles::start(
dir.join("map-tiles"),
language,
self.area.scale_factor().max(1),
);
self.state.borrow_mut().online = Some(Online {
let online = matches!(source, Source::Online { .. });
self.credit.set_label(&credit_text(online));
self.credit.set_visible(true);
let (tiles, done) = Tiles::start(source, self.area.scale_factor().max(1));
self.state.borrow_mut().tiles = Some(TileLayer {
tiles,
online,
cache: HashMap::new(),
pending: HashSet::new(),
failed: HashMap::new(),
@@ -542,7 +547,7 @@ impl PlacesMap {
fn tile_done(&self, done: Done) {
let mut state = self.state.borrow_mut();
let Some(online) = state.online.as_mut() else {
let Some(online) = state.tiles.as_mut() else {
return;
};
match done {
@@ -728,7 +733,7 @@ impl PlacesMap {
}
fn max_scale(&self) -> f64 {
let zoom = if self.state.borrow().online.is_some() {
let zoom = if self.state.borrow().tiles.as_ref().is_some_and(|t| t.online) {
ONLINE_MAX_ZOOM
} else {
MAX_ZOOM
@@ -823,11 +828,11 @@ impl PlacesMap {
let restart = self
.state
.borrow()
.online
.tiles
.as_ref()
.is_some_and(|o| o.tiles.scale != area.scale_factor().max(1));
if restart {
self.state.borrow_mut().online = None;
self.state.borrow_mut().tiles = None;
self.start_tiles();
}
@@ -840,8 +845,9 @@ impl PlacesMap {
let mut state = self.state.borrow_mut();
let state = &mut *state;
let view = state.view;
state.draws += 1;
paint_outline(cr, view, size, colors);
let labels = match state.online.as_mut() {
let labels = match state.tiles.as_mut() {
Some(online) => paint_tiles(cr, online, view, size, dark, area.scale_factor()),
None => Vec::new(),
};
@@ -863,7 +869,10 @@ impl PlacesMap {
}
for bubble in &state.bubbles {
let marker = &state.markers[bubble.members[0]];
let cover = state.covers.get(&marker.cover);
let cover = state.covers.get_mut(&marker.cover).map(|(cover, drawn)| {
*drawn = state.draws;
&*cover
});
if cover.is_none()
&& let Target::Photo { place, index } = marker.target
&& state.asked_thumbs.insert(marker.cover.clone())
@@ -991,16 +1000,16 @@ fn paint_outline(cr: &cairo::Context, view: View, size: (f64, f64), colors: &Pal
let _ = cr.stroke();
}
/// The street-map zoom whose tiles come closest to their own size on screen.
/// The tile zoom whose tiles come closest to their own size on screen.
fn tile_zoom(scale: f64) -> u8 {
(scale / TILE_EDGE).log2().round().clamp(0.0, 18.0) as u8
}
/// Draw the street-map tiles on screen, asking for the missing ones and
/// standing in their nearest ancestor meanwhile. Returns the names on them.
/// Draw the map tiles on screen, asking for the missing ones and standing in
/// their nearest ancestor meanwhile. Returns the names on them.
fn paint_tiles(
cr: &cairo::Context,
online: &mut Online,
online: &mut TileLayer,
view: View,
size: (f64, f64),
dark: bool,
@@ -1121,6 +1130,35 @@ fn evict(cache: &mut HashMap<TileKey, CachedTile>, frame: u64) {
}
}
/// Drop the photo covers drawn longest ago, down to [`COVER_CACHE`]. Never
/// one drawn in the current draw, and never a town's cover. Returns the uids
/// dropped.
fn evict_covers<T>(
covers: &mut HashMap<String, (T, u64)>,
towns: &HashSet<&str>,
draw: u64,
) -> Vec<String> {
if covers.len() <= COVER_CACHE {
return Vec::new();
}
let mut ages: Vec<(u64, &String)> = covers
.iter()
.filter(|(uid, (_, drawn))| *drawn < draw && !towns.contains(uid.as_str()))
.map(|(uid, (_, drawn))| (*drawn, uid))
.collect();
ages.sort_unstable_by_key(|(drawn, _)| *drawn);
let excess = covers.len() - COVER_CACHE;
let dropped: Vec<String> = ages
.into_iter()
.take(excess)
.map(|(_, uid)| uid.clone())
.collect();
for uid in &dropped {
covers.remove(uid);
}
dropped
}
/// Set the names of the tiles on screen, the most important first, leaving
/// out any that would overlap one already set.
fn paint_labels(
@@ -1528,6 +1566,47 @@ impl PlacesMap {
/// How many town names a merged marker's tooltip lists.
const TOOLTIP_NAMES: usize = 5;
/// The credit for the map's tiles, as Pango markup.
fn credit_text(online: bool) -> String {
let link = |url: &str, name: &str| format!("<a href=\"{url}\">{name}</a>");
if online {
gettext_f(
// Translators: the credit on the street map. {openfreemap},
// {openmaptiles} and {openstreetmap} are the linked names of
// the projects, which stay as they are.
"{openfreemap} © {openmaptiles} Data from {openstreetmap}",
&[
(
"openfreemap",
&link("https://openfreemap.org", "OpenFreeMap"),
),
(
"openmaptiles",
&link("https://www.openmaptiles.org/", "OpenMapTiles"),
),
(
"openstreetmap",
&link("https://www.openstreetmap.org/copyright", "OpenStreetMap"),
),
],
)
} else {
gettext_f(
// Translators: the credit on the offline map. {naturalearth} and
// {geonames} are the linked names of the projects, which stay as
// they are.
"Map data © {naturalearth} (public domain) and {geonames} (CC BY 4.0)",
&[
(
"naturalearth",
&link("https://www.naturalearthdata.com/", "Natural Earth"),
),
("geonames", &link("https://www.geonames.org/", "GeoNames")),
],
)
}
}
fn map_button(icon: &str, tooltip: &str) -> gtk4::Button {
let button = gtk4::Button::from_icon_name(icon);
button.set_tooltip_text(Some(tooltip));
@@ -1595,52 +1674,6 @@ fn spread(points: &[World]) -> f64 {
span(&mut xs.into_iter()).max(span(&mut ys.into_iter()))
}
/// Merge markers whose circles would overlap on screen, the one with the
/// most photos claiming its neighbours (the first of equals, when counts
/// tie). Markers off screen are left out.
fn cluster(screen: &[(f64, f64)], counts: &[usize], width: f64, height: f64) -> Vec<Bubble> {
let mut order: Vec<usize> = (0..screen.len()).collect();
order.sort_by(|a, b| counts[*b].cmp(&counts[*a]));
let mut bubbles: Vec<Bubble> = Vec::new();
for index in order {
let (x, y) = screen[index];
let reach = bubble_radius(counts[index]);
if x < -reach || y < -reach || x > width + reach || y > height + reach {
continue;
}
match bubbles
.iter_mut()
.find(|b| (b.x - x).hypot(b.y - y) < b.radius + reach * 0.5)
{
Some(bubble) => {
bubble.photos += counts[index];
bubble.radius = bubble_radius(bubble.photos);
bubble.members.push(index);
}
None => bubbles.push(Bubble {
x,
y,
radius: reach,
photos: counts[index],
members: vec![index],
}),
}
}
// Smallest last, so a small marker next to a big one stays clickable.
bubbles.sort_by(|a, b| b.radius.total_cmp(&a.radius));
bubbles
}
/// "7", "840", "1.2k", "35k": short enough to fit a marker. Digits and the
/// "k" read the same in every language the app ships.
fn compact_count(n: usize) -> String {
match n {
0..1_000 => n.to_string(),
1_000..10_000 => format!("{:.1}k", n as f64 / 1_000.0),
_ => format!("{}k", n / 1_000),
}
}
fn set_color(cr: &cairo::Context, color: &gtk4::gdk::RGBA, alpha: f32) {
cr.set_source_rgba(
f64::from(color.red()),
@@ -1654,6 +1687,28 @@ fn set_color(cr: &cairo::Context, color: &gtk4::gdk::RGBA, alpha: f32) {
mod tests {
use super::*;
#[test]
fn covers_past_the_cap_drop_the_stalest_photos_but_no_town() {
let mut covers: HashMap<String, ((), u64)> = (0..COVER_CACHE as u64 + 3)
.map(|n| (format!("p{n}"), ((), n)))
.collect();
let towns: HashSet<&str> = ["p0"].into_iter().collect();
let mut dropped = evict_covers(&mut covers, &towns, COVER_CACHE as u64);
dropped.sort();
assert_eq!(dropped, ["p1", "p2", "p3"]);
assert_eq!(covers.len(), COVER_CACHE);
assert!(covers.contains_key("p0"));
}
#[test]
fn covers_drawn_this_frame_are_kept_even_past_the_cap() {
let mut covers: HashMap<String, ((), u64)> = (0..COVER_CACHE + 2)
.map(|n| (format!("p{n}"), ((), 7)))
.collect();
assert!(evict_covers(&mut covers, &HashSet::new(), 7).is_empty());
assert_eq!(covers.len(), COVER_CACHE + 2);
}
#[test]
fn projection_puts_the_equator_and_meridian_in_the_middle() {
let p = project(0.0, 0.0);
@@ -635,6 +635,7 @@ pub(crate) fn shared_item_as_entry(item: &SharedItem) -> DirEntry {
shared_by_unverified: false,
trashed_at: 0,
trashed_from: None,
trashed_parent: None,
issue: None,
}
}
+1 -1
View File
@@ -245,7 +245,7 @@ pub(crate) fn build_main_page() -> MainWidgets {
let online_map_row = adw::SwitchRow::builder()
.title(gettext("Street map for Places"))
.subtitle(gettext(
"Show streets and town names from OpenFreeMap. It learns which parts of the world you look at.",
"Show streets from OpenFreeMap instead of the offline map. It learns which parts of the world you look at.",
))
.build();
appearance_group.add(&online_map_row);
+122 -19
View File
@@ -52,8 +52,9 @@ fn staged_path(ui: &Rc<Ui>) -> PathBuf {
/// Read the remembered set, dropping any archive that has since been moved or
/// deleted — a stale path would only fail at launch, and silently is worse.
fn load_staged(ui: &Rc<Ui>) -> Vec<PathBuf> {
let Ok(text) = std::fs::read_to_string(staged_path(ui)) else {
/// Blocking: run it off the GTK thread.
fn load_staged(path: &Path) -> Vec<PathBuf> {
let Ok(text) = std::fs::read_to_string(path) else {
return Vec::new();
};
let paths: Vec<String> = serde_json::from_str(&text).unwrap_or_default();
@@ -64,9 +65,39 @@ fn load_staged(ui: &Rc<Ui>) -> Vec<PathBuf> {
.collect()
}
/// Restore whatever was staged when the window last closed, read on a worker.
/// Anything the user staged meanwhile is kept alongside it.
fn restore_staged(ui: &Rc<Ui>) {
let path = staged_path(ui);
let ui = ui.clone();
glib::spawn_future_local(async move {
let restored = gio::spawn_blocking(move || load_staged(&path))
.await
.unwrap_or_default();
{
let mut staged = ui.takeout.archives.borrow_mut();
for path in restored {
if !staged.contains(&path) {
staged.push(path);
}
}
staged.sort();
}
ui.takeout.restored.set(true);
repaint_archives(&ui);
});
}
/// Remember the staged set. Best-effort: failing to write it costs the user a
/// re-pick next time, which is not worth interrupting them over now.
///
/// Written on a worker. Nothing is written before the remembered set has been
/// read back, so the empty list shown meanwhile can't overwrite it, and a
/// write that lands after a newer one is skipped.
fn save_staged(ui: &Rc<Ui>) {
if !ui.takeout.restored.get() {
return;
}
let paths: Vec<String> = ui
.takeout
.archives
@@ -75,12 +106,24 @@ fn save_staged(ui: &Rc<Ui>) {
.map(|p| p.display().to_string())
.collect();
let path = staged_path(ui);
let written = serde_json::to_string(&paths)
.map_err(|e| e.to_string())
.and_then(|text| std::fs::write(&path, text).map_err(|e| e.to_string()));
if let Err(e) = written {
tracing::warn!("cannot remember staged Takeout archives: {e}");
}
let seq = ui.takeout.save_seq.get() + 1;
ui.takeout.save_seq.set(seq);
let saved = ui.takeout.saved_seq.clone();
gio::spawn_blocking(move || {
let Ok(mut last) = saved.lock() else {
return;
};
if *last > seq {
return;
}
*last = seq;
let written = serde_json::to_string(&paths)
.map_err(|e| e.to_string())
.and_then(|text| std::fs::write(&path, text).map_err(|e| e.to_string()));
if let Err(e) = written {
tracing::warn!("cannot remember staged Takeout archives: {e}");
}
});
}
fn empty_list_description() -> String {
@@ -121,6 +164,16 @@ pub(crate) struct TakeoutState {
pub(crate) inflight: Cell<bool>,
/// Whether the daemon reported an import running on the last poll.
pub(crate) running: Cell<bool>,
/// Whether the remembered staged set has been read back yet; until then
/// nothing is saved over it.
pub(crate) restored: Cell<bool>,
/// The last save started, and the last one written by a worker, so an
/// older save that lands late doesn't overwrite a newer one.
pub(crate) save_seq: Cell<u64>,
pub(crate) saved_seq: std::sync::Arc<std::sync::Mutex<u64>>,
/// Bumped by every repaint of the archive list, so the sizes of an older
/// list that arrive late are not painted over a newer one.
pub(crate) repaint_seq: Cell<u64>,
}
/// Widgets the Import page hands back for wiring.
@@ -490,10 +543,10 @@ pub(crate) fn wire_takeout(ui: &Rc<Ui>, widgets: &TakeoutWidgets) {
.back_button
.connect_clicked(move |_| ui_back.stack.set_visible_child_name("gallery"));
// Restore whatever was staged when the window last closed, before the first
// paint, so the page comes up as the user left it.
*ui.takeout.archives.borrow_mut() = load_staged(ui);
// Restore whatever was staged when the window last closed, so the page
// comes up as the user left it.
repaint_archives(ui);
restore_staged(ui);
}
/// Stage `paths`, keeping only `.zip` files and skipping ones already staged.
@@ -503,15 +556,36 @@ pub(crate) fn wire_takeout(ui: &Rc<Ui>, widgets: &TakeoutWidgets) {
/// the scan reads the archives' central directories, and an extracted tree has
/// none — silently importing half of it would be worse than saying no.
pub(crate) fn add_archives(ui: &Rc<Ui>, paths: Vec<PathBuf>) {
// Whether each is a regular file is asked of the filesystem, which may be
// a slow network mount: on a worker.
let ui = ui.clone();
glib::spawn_future_local(async move {
let checked = gio::spawn_blocking(move || {
paths
.into_iter()
.map(|path| {
let file = path.is_file();
(path, file)
})
.collect::<Vec<_>>()
})
.await
.unwrap_or_default();
stage_archives(&ui, checked);
});
}
/// Stage the archives [`add_archives`] checked, as `(path, is a file)`.
fn stage_archives(ui: &Rc<Ui>, paths: Vec<(PathBuf, bool)>) {
let mut added = 0usize;
let mut rejected = 0usize;
{
let mut staged = ui.takeout.archives.borrow_mut();
for path in paths {
for (path, file) in paths {
let is_zip = path
.extension()
.is_some_and(|ext| ext.eq_ignore_ascii_case("zip"));
if !is_zip || !path.is_file() {
if !is_zip || !file {
rejected += 1;
continue;
}
@@ -542,12 +616,44 @@ pub(crate) fn add_archives(ui: &Rc<Ui>, paths: Vec<PathBuf>) {
}
/// Rebuild the staged-archive list and re-derive which actions are available.
///
/// The archives' sizes are read on a worker, and the list is rebuilt once
/// they are in.
pub(crate) fn repaint_archives(ui: &Rc<Ui>) {
let seq = ui.takeout.repaint_seq.get().wrapping_add(1);
ui.takeout.repaint_seq.set(seq);
let staged = ui.takeout.archives.borrow().clone();
save_staged(ui);
sync_takeout_actions(ui);
if staged.is_empty() {
paint_archives(ui, Vec::new());
return;
}
let ui = ui.clone();
glib::spawn_future_local(async move {
let sized = gio::spawn_blocking(move || {
staged
.into_iter()
.map(|path| {
let size = std::fs::metadata(&path).map(|m| m.len()).unwrap_or(0);
(path, size)
})
.collect::<Vec<_>>()
})
.await
.unwrap_or_default();
if ui.takeout.repaint_seq.get() == seq {
paint_archives(&ui, sized);
}
});
}
/// Fill the staged-archive list with `staged`, as `(path, size)`.
fn paint_archives(ui: &Rc<Ui>, staged: Vec<(PathBuf, u64)>) {
for row in ui.takeout.rows.borrow_mut().drain(..) {
ui.takeout.list_group.remove(&row);
}
let staged = ui.takeout.archives.borrow().clone();
let mut rows: Vec<adw::PreferencesRow> = Vec::new();
if staged.is_empty() {
ui.takeout
@@ -558,8 +664,8 @@ pub(crate) fn repaint_archives(ui: &Rc<Ui>) {
rows.push(row.upcast());
} else {
let mut total = 0u64;
for path in &staged {
let size = std::fs::metadata(path).map(|m| m.len()).unwrap_or(0);
for (path, size) in &staged {
let size = *size;
total += size;
let row = adw::ActionRow::builder()
.title(
@@ -601,9 +707,6 @@ pub(crate) fn repaint_archives(ui: &Rc<Ui>) {
)));
}
*ui.takeout.rows.borrow_mut() = rows;
save_staged(ui);
sync_takeout_actions(ui);
}
/// Enable exactly the actions that make sense right now: nothing to launch
+503 -24
View File
@@ -16,6 +16,28 @@ pub(crate) struct TrashState {
pub(crate) selection_label: gtk4::Label,
/// Runs the loads; the rows stay up while one runs.
pub(crate) loader: Rc<Loader>,
/// Back, the breadcrumb and the search entry over the list.
pub(crate) back: gtk4::Button,
pub(crate) crumb: gtk4::Box,
pub(crate) search: gtk4::SearchEntry,
/// On: the trash as the folders things were deleted from. Off: one list.
pub(crate) grouped: gtk4::ToggleButton,
/// The whole trash as last read, newest deletion first.
pub(crate) all: RefCell<Vec<DirEntry>>,
/// Where the list is, below the top level.
pub(crate) nav: RefCell<Vec<TrashNav>>,
/// What the trashed folder on screen holds, once read.
pub(crate) folder_rows: RefCell<Option<Vec<DirEntry>>>,
}
/// One step down from the top of the Trash page.
#[derive(Clone, Debug, PartialEq)]
pub(crate) enum TrashNav {
/// The things deleted from one folder: its path, `""` for My files, `None`
/// when the folder is not known.
Location { key: Option<String>, label: String },
/// Inside a trashed folder.
Folder { uid: String, name: String },
}
/// The widgets of the Trash page that a load repaints.
@@ -30,8 +52,18 @@ pub(crate) struct TrashWidgets {
pub(crate) selection_label: gtk4::Label,
pub(crate) restore_selected: gtk4::Button,
pub(crate) delete_selected: gtk4::Button,
pub(crate) back: gtk4::Button,
pub(crate) crumb: gtk4::Box,
pub(crate) search: gtk4::SearchEntry,
pub(crate) grouped: gtk4::ToggleButton,
}
/// The uid prefix of a row that stands for a folder things were deleted from.
/// Such a row is made here, never sent to the daemon.
const LOCATION_UID: &str = "trash-location:";
/// The location key of the folder that is not known.
const UNKNOWN_LOCATION: &str = "\u{0}";
/// The Trash page: everything Drive is holding in the trash, as a list with
/// the folder each item came from and when it was deleted, or as a grid, where
/// photos are easier to recognise. Restore and Delete Permanently sit in the
@@ -75,11 +107,41 @@ pub(crate) fn build_trash_page() -> (gtk4::Widget, TrashWidgets) {
content.add_named(&files.views, Some("list"));
content.add_named(&status, Some("status"));
let back = gtk4::Button::builder()
.icon_name("go-previous-symbolic")
.tooltip_text(gettext("Back"))
.visible(false)
.build();
back.add_css_class("flat");
let crumb = gtk4::Box::new(gtk4::Orientation::Horizontal, 2);
crumb.set_valign(gtk4::Align::Center);
let crumb_scroll = gtk4::ScrolledWindow::builder()
.hscrollbar_policy(gtk4::PolicyType::External)
.vscrollbar_policy(gtk4::PolicyType::Never)
.hexpand(true)
.child(&crumb)
.build();
let search = gtk4::SearchEntry::builder()
.placeholder_text(gettext("Search the trash"))
.width_chars(24)
.build();
let path_bar = gtk4::Box::new(gtk4::Orientation::Horizontal, 8);
path_bar.append(&back);
path_bar.append(&crumb_scroll);
path_bar.append(&search);
let grouped = gtk4::ToggleButton::builder()
.icon_name("folder-symbolic")
.tooltip_text(gettext("Group by original folder"))
.active(true)
.build();
let inner = gtk4::Box::new(gtk4::Orientation::Vertical, 12);
inner.set_margin_top(12);
inner.set_margin_bottom(12);
inner.set_margin_start(12);
inner.set_margin_end(12);
inner.append(&path_bar);
inner.append(&content);
// Bulk Delete Permanently always confirms with the count, so a stale
@@ -101,6 +163,7 @@ pub(crate) fn build_trash_page() -> (gtk4::Widget, TrashWidgets) {
let (frame, header, subtitle) = page_frame(&gettext("Trash"), &inner);
header.pack_start(&empty);
header.pack_end(&layout);
header.pack_end(&grouped);
(
frame.upcast(),
@@ -115,6 +178,10 @@ pub(crate) fn build_trash_page() -> (gtk4::Widget, TrashWidgets) {
selection_label,
restore_selected,
delete_selected,
back,
crumb,
search,
grouped,
},
)
}
@@ -151,7 +218,10 @@ pub(crate) fn wire_trash(ui: &Rc<Ui>, widgets: &TrashWidgets) {
files
.column_view
.append_column(&text_column(&pgettext("column", "Size"), |e| {
if e.is_dir {
if location_key(&e.uid).is_some() {
// A location row carries how many items it holds as its size.
ngettext_f("{n} item", "{n} items", e.size, &[])
} else if e.is_dir {
"—".to_string()
} else {
human_bytes(e.size)
@@ -163,6 +233,21 @@ pub(crate) fn wire_trash(ui: &Rc<Ui>, widgets: &TrashWidgets) {
.empty
.connect_clicked(move |_| prompt_empty_trash(&ui_empty));
let ui_back = ui.clone();
widgets.back.connect_clicked(move |_| {
ui_back.trash.nav.borrow_mut().pop();
show_trash_place(&ui_back);
});
let ui_search = ui.clone();
widgets
.search
.connect_search_changed(move |_| render_trash(&ui_search));
let ui_grouped = ui.clone();
widgets.grouped.connect_toggled(move |_| {
ui_grouped.trash.nav.borrow_mut().clear();
show_trash_place(&ui_grouped);
});
let ui_sel = ui.clone();
files
.selection
@@ -193,17 +278,52 @@ fn trashed_from_label(entry: &DirEntry) -> String {
}
}
/// A trashed file can't be opened; say what would let it be.
/// Open a location or a trashed folder in place. A trashed file can't be
/// opened; say what would let it be.
fn activate_trashed(ui: &Rc<Ui>, entry: &DirEntry) {
toast(
ui,
if let Some(key) = location_key(&entry.uid) {
ui.trash.nav.borrow_mut().push(TrashNav::Location {
key,
label: entry.name.clone(),
});
show_trash_place(ui);
} else if entry.is_dir {
ui.trash.nav.borrow_mut().push(TrashNav::Folder {
uid: entry.uid.clone(),
name: entry.name.clone(),
});
show_trash_place(ui);
} else {
let name = match outer_folder(ui) {
Some(folder) if !in_trash(ui, &entry.uid) => folder.name,
_ => entry.name.clone(),
};
// Translators: {name} is a file or folder name.
&gettext_f("Restore “{name}” to open it", &[("name", &entry.name)]),
);
toast(
ui,
&gettext_f("Restore “{name}” to open it", &[("name", &name)]),
);
}
}
fn trash_entry_menu(ui: &Rc<Ui>, entry: &DirEntry) -> ActionMenu {
let mut menu = ActionMenu::new();
if location_key(&entry.uid).is_none()
&& !in_trash(ui, &entry.uid)
&& let Some(folder) = outer_folder(ui)
{
// Went to the trash with the folder, so only the folder goes back.
let ui_c = ui.clone();
let label = gettext_f(
// Translators: {name} is a folder name.
"Restore “{name}”",
&[("name", &folder.name)],
);
menu.item(&label, move || {
restore_entries(&ui_c, std::slice::from_ref(&folder))
});
return menu;
}
let (ui_c, entry_c) = (ui.clone(), entry.clone());
menu.item(&pgettext("verb", "Restore"), move || {
restore_entries(&ui_c, std::slice::from_ref(&entry_c))
@@ -244,7 +364,7 @@ fn trash_background_menu(ui: &Rc<Ui>) -> ActionMenu {
});
let ui_c = ui.clone();
menu.item(&gettext("Refresh"), move || reload_current_page(&ui_c));
if ui.trash.files.model.n_items() > 0 {
if !ui.trash.all.borrow().is_empty() {
menu.section();
let ui_c = ui.clone();
menu.item(&gettext("Empty Trash…"), move || {
@@ -300,7 +420,17 @@ pub(crate) fn load_trash(ui: &Rc<Ui>) {
clear_trash(&ui);
}
match result {
Ok(Ok(Response::Entries { entries })) => repaint_trash(&ui, &entries),
Ok(Ok(Response::Entries { entries })) => {
let mut entries = entries;
sort_trash(&mut entries);
*ui.trash.all.borrow_mut() = entries;
keep_trash_place(&ui);
if matches!(ui.trash.nav.borrow().last(), Some(TrashNav::Folder { .. })) {
load_trashed_folder(&ui);
} else {
render_trash(&ui);
}
}
Ok(Ok(Response::Error { message, .. })) => trash_status(
&ui,
"dialog-warning-symbolic",
@@ -338,14 +468,125 @@ pub(crate) fn trash_status(ui: &Rc<Ui>, icon: &str, title: &str, description: &s
ui.trash.subtitle.set_subtitle("");
}
/// Repopulate the trash, most recently deleted first: the order in which a user
/// looks for what they just deleted.
pub(crate) fn repaint_trash(ui: &Rc<Ui>, entries: &[DirEntry]) {
/// Go to the place `nav` now points at: read a trashed folder, or show
/// what the stored listing already holds.
fn show_trash_place(ui: &Rc<Ui>) {
if !ui.trash.search.text().is_empty() {
ui.trash.search.set_text("");
}
ui.trash.files.selection.unselect_all();
ui.trash.files.scroll_to_top();
if matches!(ui.trash.nav.borrow().last(), Some(TrashNav::Folder { .. })) {
load_trashed_folder(ui);
} else {
render_trash(ui);
}
}
/// Read what the trashed folder on screen holds, then show it.
fn load_trashed_folder(ui: &Rc<Ui>) {
let Some(TrashNav::Folder { uid, .. }) = ui.trash.nav.borrow().last().cloned() else {
return;
};
ui.trash.folder_rows.borrow_mut().take();
let ui_p = ui.clone();
let ticket = ui.trash.loader.replace(move || {
clear_trash(&ui_p);
trash_status(
&ui_p,
"folder-open-symbolic",
&gettext("Loading…"),
&gettext("Reading this folder."),
false,
);
});
ui.busy_begin();
let rx = spawn_request(
ui.dirs.control_socket(),
Request::ListTrashedFolder { uid: uid.clone() },
);
let ui = ui.clone();
glib::spawn_future_local(async move {
let result = rx.recv().await;
ui.busy_end();
if !ticket.is_current() {
return;
}
let still_here = matches!(
ui.trash.nav.borrow().last(),
Some(TrashNav::Folder { uid: on, .. }) if *on == uid
);
if !still_here {
return;
}
match result {
Ok(Ok(Response::Entries { entries })) => {
*ui.trash.folder_rows.borrow_mut() = Some(entries);
render_trash(&ui);
}
Ok(Ok(Response::Error { message, .. })) => {
clear_trash(&ui);
trash_status(
&ui,
"dialog-warning-symbolic",
&gettext("Couldn't open the folder"),
&message,
false,
);
}
Ok(Ok(_)) => {
clear_trash(&ui);
trash_status(
&ui,
"dialog-warning-symbolic",
&gettext("Couldn't open the folder"),
&gettext("Unexpected reply from the Proton Drive service."),
false,
);
}
Ok(Err(_)) | Err(_) => trash_unreachable(&ui),
}
});
}
/// After a new listing, step back out of places that are gone from it.
fn keep_trash_place(ui: &Rc<Ui>) {
let all = ui.trash.all.borrow();
let mut nav = ui.trash.nav.borrow_mut();
let keep = kept_steps(&nav, &all);
nav.truncate(keep);
}
/// How many steps of `nav` still lead somewhere in `all`. A location stays
/// while something deleted from it is listed, and the trashed folder opened
/// from it while that folder is; the folders below it went with it.
fn kept_steps(nav: &[TrashNav], all: &[DirEntry]) -> usize {
let mut inside = false;
for (i, step) in nav.iter().enumerate() {
let there = match step {
TrashNav::Location { key, .. } => all
.iter()
.any(|e| e.trashed_parent.is_none() && e.trashed_from == *key),
TrashNav::Folder { uid, .. } => inside || all.iter().any(|e| e.uid == *uid),
};
if !there {
return i;
}
inside |= matches!(step, TrashNav::Folder { .. });
}
nav.len()
}
/// Show the rows for the current place, search and grouping.
pub(crate) fn render_trash(ui: &Rc<Ui>) {
let all = ui.trash.all.borrow();
// An empty Trash has nothing to empty, so the button goes away rather than
// sitting there greyed out.
ui.trash.empty.set_sensitive(true);
ui.trash.empty.set_visible(!entries.is_empty());
if entries.is_empty() {
ui.trash.empty.set_visible(!all.is_empty());
repaint_trash_crumb(ui);
if all.is_empty() {
drop(all);
trash_status(
ui,
"user-trash-symbolic",
@@ -356,22 +597,198 @@ pub(crate) fn repaint_trash(ui: &Rc<Ui>, entries: &[DirEntry]) {
clear_trash(ui);
return;
}
ui.trash.content.set_visible_child_name("list");
ui.trash.subtitle.set_subtitle(&ngettext_f(
"{n} item",
"{n} items",
entries.len() as u64,
&[],
));
let subtitle = ngettext_f("{n} item", "{n} items", all.len() as u64, &[]);
let mut sorted = entries.to_vec();
sort_trash(&mut sorted);
let query = ui.trash.search.text().to_lowercase();
let nav = ui.trash.nav.borrow().clone();
let rows: Vec<DirEntry> = if !query.is_empty() {
all.iter()
.filter(|e| e.name.to_lowercase().contains(&query))
.cloned()
.collect()
} else if !ui.trash.grouped.is_active() {
all.clone()
} else {
match nav.last() {
None => trash_locations(&all),
Some(TrashNav::Location { key, .. }) => all
.iter()
.filter(|e| e.trashed_parent.is_none() && e.trashed_from == *key)
.cloned()
.collect(),
Some(TrashNav::Folder { .. }) => {
ui.trash.folder_rows.borrow().clone().unwrap_or_default()
}
}
};
drop(all);
if rows.is_empty() {
clear_trash(ui);
if query.is_empty() {
trash_status(
ui,
"folder-open-symbolic",
&gettext("This folder is empty"),
&gettext("Nothing else was in this folder when it was deleted."),
false,
);
} else {
trash_status(
ui,
"edit-find-symbolic",
&gettext("No results"),
&gettext("Nothing in the trash has that name."),
false,
);
}
ui.trash.subtitle.set_subtitle(&subtitle);
return;
}
ui.trash.content.set_visible_child_name("list");
ui.trash.subtitle.set_subtitle(&subtitle);
// Only the rows that changed are swapped, so a refresh keeps the
// selection and the scroll position.
replace_items(&ui.trash.files.model, &sorted);
replace_items(&ui.trash.files.model, &rows);
sync_trash_selection(ui);
}
/// One row per folder things were deleted from, the most recent first. Items
/// trashed inside a trashed folder are found by opening that folder.
fn trash_locations(all: &[DirEntry]) -> Vec<DirEntry> {
let mut rows: Vec<DirEntry> = Vec::new();
let mut index: HashMap<Option<String>, usize> = HashMap::new();
for entry in all.iter().filter(|e| e.trashed_parent.is_none()) {
let key = entry.trashed_from.clone();
let at = *index.entry(key.clone()).or_insert_with(|| {
rows.push(location_row(key));
rows.len() - 1
});
let row = &mut rows[at];
row.size += 1;
row.trashed_at = row.trashed_at.max(entry.trashed_at);
row.modified = row.modified.max(entry.modified);
}
sort_trash(&mut rows);
rows
}
/// The row that stands for the folder `key`, still holding nothing.
fn location_row(key: Option<String>) -> DirEntry {
let (uid, name) = match key.as_deref() {
None => (
format!("{LOCATION_UID}{UNKNOWN_LOCATION}"),
gettext("Unknown folder"),
),
Some("") => (LOCATION_UID.to_string(), gettext("My files")),
Some(path) => (
format!("{LOCATION_UID}{path}"),
path.rsplit('/').next().unwrap_or(path).to_string(),
),
};
DirEntry {
name,
is_dir: true,
size: 0,
modified: 0,
pinned: false,
pinned_by_folder: false,
cached: false,
uid,
path: String::new(),
role: String::new(),
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
trashed_at: 0,
trashed_from: key,
trashed_parent: None,
issue: None,
}
}
/// The folder a location row stands for, or `None` for any other row.
fn location_key(uid: &str) -> Option<Option<String>> {
let key = uid.strip_prefix(LOCATION_UID)?;
Some((key != UNKNOWN_LOCATION).then(|| key.to_string()))
}
/// Whether the stored listing holds `uid`, so it can be restored on its own.
fn in_trash(ui: &Ui, uid: &str) -> bool {
ui.trash.all.borrow().iter().any(|e| e.uid == uid)
}
/// The trashed folder the user opened from a location, when inside one.
fn outer_folder(ui: &Ui) -> Option<DirEntry> {
let nav = ui.trash.nav.borrow();
let uid = nav.iter().find_map(|step| match step {
TrashNav::Folder { uid, .. } => Some(uid.clone()),
TrashNav::Location { .. } => None,
})?;
ui.trash.all.borrow().iter().find(|e| e.uid == uid).cloned()
}
/// What acting on `entries` acts on: a location row stands for everything
/// deleted from that folder, and an item that went with its folder for the
/// folder. Each node once.
fn trashed_targets(ui: &Ui, entries: &[DirEntry]) -> Vec<DirEntry> {
let all = ui.trash.all.borrow();
let outer = outer_folder(ui);
let mut seen = HashSet::new();
let mut targets = Vec::new();
for entry in entries {
let picked: Vec<DirEntry> = if let Some(key) = location_key(&entry.uid) {
all.iter()
.filter(|e| e.trashed_parent.is_none() && e.trashed_from == key)
.cloned()
.collect()
} else if all.iter().any(|e| e.uid == entry.uid) {
vec![entry.clone()]
} else {
outer.clone().into_iter().collect()
};
for target in picked {
if seen.insert(target.uid.clone()) {
targets.push(target);
}
}
}
targets
}
/// Rebuild the path bar for `nav`: "Trash", then each place down to the
/// current one. Hidden at the top, where there is nowhere to go back to.
fn repaint_trash_crumb(ui: &Rc<Ui>) {
let nav = ui.trash.nav.borrow();
let crumb = &ui.trash.crumb;
while let Some(child) = crumb.first_child() {
crumb.remove(&child);
}
ui.trash.back.set_visible(!nav.is_empty());
crumb.append(&trash_crumb_node(ui, &gettext("Trash"), 0, nav.is_empty()));
for (i, step) in nav.iter().enumerate() {
let label = match step {
TrashNav::Location { label, .. } => label,
TrashNav::Folder { name, .. } => name,
};
crumb.append(&crumb_separator());
crumb.append(&trash_crumb_node(ui, label, i + 1, i + 1 == nav.len()));
}
}
fn trash_crumb_node(ui: &Rc<Ui>, label: &str, depth: usize, current: bool) -> gtk4::Widget {
if current {
return crumb_current(label);
}
let button = gtk4::Button::builder().label(label).build();
button.add_css_class("flat");
let ui = ui.clone();
button.connect_clicked(move |_| {
ui.trash.nav.borrow_mut().truncate(depth);
show_trash_place(&ui);
});
button.upcast()
}
/// Newest deletion first. Entries with no known deletion time go last, newest
/// modification first among themselves.
fn sort_trash(entries: &mut [DirEntry]) {
@@ -393,6 +810,10 @@ fn clear_trash(ui: &Rc<Ui>) {
/// Restore trashed entries to the folders they were trashed from. When they
/// all came from one known folder, the toast offers to show it.
pub(crate) fn restore_entries(ui: &Rc<Ui>, entries: &[DirEntry]) {
let entries = &trashed_targets(ui, entries)[..];
if entries.is_empty() {
return;
}
let message = match entries {
// Translators: {name} is a file or folder name.
[one] => gettext_f("Restored “{name}”", &[("name", &one.name)]),
@@ -430,6 +851,10 @@ fn common_origin(entries: &[DirEntry]) -> Option<String> {
/// Confirm, then permanently delete trashed entries. Irreversible, so it asks.
pub(crate) fn prompt_delete_forever(ui: &Rc<Ui>, entries: &[DirEntry]) {
let entries = &trashed_targets(ui, entries)[..];
if entries.is_empty() {
return;
}
let win = ui_window(ui);
let uids: Vec<String> = entries.iter().map(|e| e.uid.clone()).collect();
let count = entries.len() as u64;
@@ -485,7 +910,7 @@ pub(crate) fn prompt_delete_forever(ui: &Rc<Ui>, entries: &[DirEntry]) {
/// Confirm, then permanently delete everything in the trash.
pub(crate) fn prompt_empty_trash(ui: &Rc<Ui>) {
let win = ui_window(ui);
let count = ui.trash.files.model.n_items();
let count = ui.trash.all.borrow().len();
let dialog = adw::AlertDialog::builder()
.heading(gettext("Empty Trash?"))
.body(ngettext_f(
@@ -536,10 +961,64 @@ mod tests {
shared_by_unverified: false,
trashed_at,
trashed_from: from.map(str::to_string),
trashed_parent: None,
issue: None,
}
}
fn inside(name: &str, parent: &str) -> DirEntry {
DirEntry {
trashed_parent: Some(parent.into()),
..trashed(name, 5, 1, None)
}
}
#[test]
fn locations_group_top_level_items_by_folder() {
let all = vec![
trashed("a", 30, 1, Some("Photos/2019")),
trashed("b", 10, 1, Some("Photos/2019")),
trashed("c", 20, 1, Some("")),
trashed("d", 5, 1, None),
inside("e", "a"),
];
let rows = trash_locations(&all);
let summary: Vec<(&str, u64, i64)> = rows
.iter()
.map(|r| (r.name.as_str(), r.size, r.trashed_at))
.collect();
assert_eq!(
summary,
[
("2019", 2, 30),
("My files", 1, 20),
("Unknown folder", 1, 5)
]
);
assert_eq!(location_key(&rows[0].uid), Some(Some("Photos/2019".into())));
assert_eq!(location_key(&rows[1].uid), Some(Some(String::new())));
assert_eq!(location_key(&rows[2].uid), Some(None));
assert_eq!(location_key("vol~link"), None);
}
#[test]
fn a_place_stays_only_while_the_listing_still_leads_there() {
let all = vec![trashed("album", 1, 1, Some("Photos")), inside("x", "album")];
let location = TrashNav::Location {
key: Some("Photos".into()),
label: "Photos".into(),
};
let folder = |uid: &str| TrashNav::Folder {
uid: uid.into(),
name: uid.into(),
};
// Folders below the trashed one are not listed themselves.
let nav = [location.clone(), folder("album"), folder("child")];
assert_eq!(kept_steps(&nav, &all), 3);
assert_eq!(kept_steps(&[location.clone(), folder("gone")], &all), 1);
assert_eq!(kept_steps(&nav, &[]), 0);
}
#[test]
fn trash_lists_newest_deletion_first_and_undated_last() {
let mut entries = vec![
+20
View File
@@ -419,6 +419,10 @@ impl Tray for DriveTray {
self.state.phase.icon().into()
}
fn attention_icon_name(&self) -> String {
self.icon_name()
}
fn status(&self) -> Status {
match self.state.phase {
Phase::Attention | Phase::SessionExpired => Status::NeedsAttention,
@@ -672,6 +676,22 @@ mod tests {
assert_eq!(phase_of(false, 0, true, false), Phase::Synced);
}
#[test]
fn attention_states_supply_the_icon_the_tray_requests() {
let mut tray = DriveTray {
state: status_state(&status(""), &[], &[], Path::new("/m")),
socket: PathBuf::new(),
};
for (phase, icon) in [
(Phase::Attention, "dialog-warning-symbolic"),
(Phase::SessionExpired, "dialog-password-symbolic"),
] {
tray.state.phase = phase;
assert_eq!(tray.status(), Status::NeedsAttention);
assert_eq!(tray.attention_icon_name(), icon);
}
}
fn status(extra: &str) -> Response {
status_from(env!("CARGO_PKG_VERSION"), extra)
}
+1
View File
@@ -26,6 +26,7 @@ const STOCK: &[&str] = &[
// Adwaita
"application-x-executable-symbolic",
"audio-x-generic-symbolic",
"changes-prevent-symbolic",
"checkbox-checked-symbolic",
"checkbox-symbolic",
"computer-symbolic",
+55 -4
View File
@@ -74,7 +74,6 @@ original wording.
| [B134](#b134--a-file-deleted-online-while-the-drain-checks-its-write-is-kept-as-a-conflict-copy) | A file deleted online while the drain checks its write is kept as a conflict copy | Only with `local_first` off |
| [B91](#b91--the-daemon-hung-overnight-and-nothing-could-say-what-it-was-waiting-on) | The daemon hung overnight, and nothing could say what it was waiting on | The cause is not diagnosed; the daemon now reports where it is stuck and is restarted |
| [B67](#b67--fresh-state-restore-can-silently-select-a-new-device-high-22) | Fresh-state restore can silently select a new device (HIGH-22) | An ambiguous device after a loss of local state is not asked about |
| [B66](#b66--local-only-pending-data-is-not-protected-at-shutdown-high-21) | Local-only pending data is not protected at shutdown (HIGH-21) | No shutdown protection and no `pdfs sync flush` |
| [B65](#b65--release-artifacts-lack-supply-chain-verification-med-11) | Release artifacts lack supply-chain verification (MED-11) | No SBOM, provenance or checksums |
| [B15](#b15--an-empty-but-listed-folder-and-duplicated-folder-uids) | An empty-but-`listed` folder, and duplicated folder uids | Cause not attributed |
| [B10](#b10--glib-critical-when-the-launcher-closes-over-an-in-flight-open) | GLib critical when the launcher closes over an in-flight open | |
@@ -109,6 +108,8 @@ a live account since.
| Entry | Title |
|---|---|
| [B66](#b66--local-only-pending-data-is-not-protected-at-shutdown-high-21) | Local-only pending data is not protected at shutdown (HIGH-21) |
| [B293](#b293--a-large-trash-never-lists-and-empty-trash-never-empties-it) | A large trash never lists, and Empty Trash never empties it |
| [B291](#b291--on-fedora-45s-oo7-every-saved-sign-in-is-lost-at-the-next-reboot) | On Fedora 45's oo7, every saved sign-in is lost at the next reboot |
| [B289](#b289--the-takeout-import-mixes-in-other-services-and-drops-location-and-live-photos) | The Takeout import mixes in other services and drops location and live photos |
| [B288](#b288--an-interrupted-takeout-import-reads-as-finished-and-one-error-ends-it) | An interrupted Takeout import reads as finished, and one error ends it |
@@ -395,6 +396,38 @@ a live account since.
Newest first.
### B293 — A large trash never lists, and Empty Trash never empties it
**Status:** Fixed in code, live check pending
**Found:** 2026-11-22. The user reported that the Trash page always showed the same 64 items and
that Empty Trash did nothing.
**Where:** `proton-drive-rs` `enumerate_trash_node_uids`; `crates/pdfs-fuse/src/lib.rs`
`Core::refresh_trash` and `Core::empty_trash`
**Repro:** An account with a few thousand items in the trash. The log shows `background trash
refresh failed error=enumerate trash: no answer within 120s` on every refresh, and Empty Trash
shows `a control request has been running for a long time request=EmptyTrash age_secs=417`.
**Cause:** The SDK sent the trash paging parameters in lower case, unlike every other paged
request, and kept asking while pages came back full. On a large trash the listing never
finished. The refresh therefore timed out every time, and the page kept the rows of the last
refresh that had got anywhere. Empty Trash listed the trash first, with no time limit, so the
`DELETE` was never sent. A refresh also decrypted every trashed file again, and rewrote the whole
table after every chunk.
**Fix:** SDK 0.12.5 sends `Page` and `PageSize` and stops on a page that brings nothing new.
The refresh reads only nodes it has not stored yet, without unlocking file keys, and keeps the
time the server says each was trashed. Rows go only once the whole listing is in. Empty Trash
sends the `DELETE` at once, under the trash call time limit, and clears the stored listing.
Restore and Delete Permanently drop their rows right away. Schema v47 indexes `trash.parent_uid`
and drops the stored rows once, so they are read again with real deletion times.
**Verified:** `cargo test -p pdfs-fuse the_trash_lists_by_folder_and_empties -- --ignored`, the
new `trash_upsert_keeps_other_rows_and_retain_drops_unlisted_ones`, and the SDK paging tests on
2026-11-22. Pending: a live listing and empty on the affected account.
### B292 — Two programs changing settings at once lose one of the changes
**Status:** Fixed
@@ -432,7 +465,7 @@ the build Fedora 45 and 46 ship as stable, from before oo7's
[7fb11dd](https://github.com/linux-credentials/oo7/commit/7fb11dd111c4be8fec01996e3caa31c44fa962b3)
("server: Write item changes to the keyring file"). See
[the investigation](ISSUE-78.md#setsecret-is-kept-only-in-memory-in-fedoras-oo7).
**Where:** `crates/pdfs-core/src/auth.rs`, `write_stored`; `crates/pdfs-core/src/secretservice.rs`
**Where:** `crates/pdfs-core/src/auth.rs`, `write_stored`; `proton-auth-rs` (`secret_service.rs`, moved from `crates/pdfs-core/src/secretservice.rs`)
**Repro:** With oo7-daemon at e91b98a, save the session blob, save it again with a new refresh
token, and restart oo7-daemon. The read after the restart returns the first blob. The keyring
@@ -988,11 +1021,16 @@ timing.
**Where:** not located; the drain's conflict check in `crates/pdfs-fuse/src/drain.rs` is the
first suspect
**Repro:** Only the CI run above. The last steps created `x/z/y/a.txt` (37 bytes), wrote 42
**Repro:** The first CI failure created `x/z/y/a.txt` (37 bytes), wrote 42
bytes into it at offset 20, and brought every link up. Drive then held the file at its first
37 bytes, and the full content was in `x/z/y/a (sync-conflict …).txt`. No other client wrote
to it.
On 2026-11-19, seed 1 of `one-client-echoes` showed the same symptom with delayed,
duplicated, and reordered events. A two-byte overwrite of `x/d` appeared only in a
conflict copy. Local replays passed. The simulation now retains its operation trace
and daemon logs on failure. [CI run](https://github.com/narrrl/proton-drive-linux/actions/runs/44427290348)
**Cause.** Not known. Nothing was lost, since the conflict copy holds the bytes, but the file
conflicted with its own create. B183 and B184 found the same symptom after a lost create
reply.
@@ -7470,7 +7508,7 @@ duplicates, renamed folders, multiple devices, and byte-identical restoration.
### B66 — Local-only pending data is not protected at shutdown (HIGH-21)
**Status:** Open — 1.0 data-safety/UX blocker
**Status:** Fixed, unverified live
**Found:** 2026-07-22, 1.0 recovery audit
**Where:** shutdown, CLI/GUI status, logout/unregister workflows
@@ -7483,6 +7521,19 @@ and last errors, and protect destructive lifecycle actions. Test offline stop,
logout, unregister, reboot, retry exhaustion, and drain. Credential removal must
never delete staged bytes.
**Fix:** `crates/pdfs-fuse/src/inhibit.rs` takes a logind shutdown inhibitor
(`systemd-inhibit --what=shutdown --mode=block`) while the queue holds changes
the drain can send, and lets it go once they land. Parked transient files and
changes that need the user are left out, so they never hold a power-off up for
good. `pdfs sync flush` retries failed changes and waits for the queue, with
its own exit codes for "needs you" and timeout. `pdfs logout` and the app's
Sign Out dialog say how many changes are still queued and that they stay
staged. Credential removal never touched `staging/`.
**Verified:** unit tests for which work holds the lock; `systemd-inhibit` in
block mode taken by a user process on 2026-11-21. A power-off with a queued
upload is still owed.
---
### B65 — Release artifacts lack supply-chain verification (MED-11)
+103
View File
@@ -11,6 +11,107 @@ scratch (user data in `staging/` and `recovery/` is never touched by this).
## [Unreleased]
### Added
- `pdfs sync flush` waits until every queued change is on Proton Drive, retrying
failed ones at once. It exits 3 when what is left needs you and 4 on `--timeout`.
- While changes are queued that only this computer has, the daemon holds off a
shutdown through logind, and desktops show "Sending N changes to Proton Drive" in
their power-off dialog. Changes waiting on you never hold it up (B66).
- Signing out, in the app or with `pdfs logout`, says how many changes are not on
Proton Drive yet. They stay on this computer and are sent after the next sign-in.
### Changed
- The Trash page shows the folders things were deleted from. Open one to see what
was deleted there, and open a deleted folder to see what it held. A switch in
the header goes back to one list, and a search box finds anything in the trash
by name. Restore and Delete Permanently work on a whole folder row. `pdfs trash`
takes `--by-folder`, and `--folder UID` for a deleted folder.
- The trash loads faster: a refresh reads only what is new, without unlocking file
keys, and shows the time each item was deleted as Proton Drive reports it.
- The Places map draws an offline basemap, the one Proton Gallery shows: seas by
depth, lakes, rivers, built-up areas, borders and town names, in light and dark,
rendered in the background through `proton-map-rs`. It ships with every package
(about 25 MB, fetched by `scripts/fetch-basemap.sh` against a pinned SHA-256) and
sends no request. The street map stays opt-in and replaces it when turned on.
Without the file the map falls back to the coarse outlines.
- A Google Photos import uploads two photos at once instead of one, while the next
photo is read out of its archive. Live-photo videos still go after their stills, and
cancelling still lets the uploads in flight finish.
- proton-sdk 0.12.4: thumbnails need fewer requests to Proton (0.12.3), and the SDK
can list photos that have no thumbnail (0.12.4).
- Signing in runs through `proton_sdk::login` (proton-sdk 0.12.2), the sequence Proton
Gallery and Proton Stream use too. The stored session, the lock/unlock choice for two-password
accounts, the three mailbox-password tries and the messages stay as they were.
- An account whose only second factor is a security key now gets a clear message at
sign-in instead of a code prompt that could only fail.
- Block fetches shared by concurrent readers now use `SingleFlight` from
`proton-media-rs`, the same code Proton Stream uses. Same behaviour: one
download per block, every reader gets its result, and a reader whose fetch was
cancelled downloads the block itself.
- The shared Gallery engine now comes from the `proton-gallery-rs` and
`proton-gallery-app-rs` crates on crates.io instead of a pinned Git revision of
Proton Gallery. Builds, vendoring and the Nix package no longer fetch from Git.
- RAW+JPEG grouping, Google Takeout reading, EXIF capture times, look-alike
detection and the offline town lookup now come from `proton-gallery-rs`,
shared with Proton Gallery. No behaviour change.
- The keyring code (one Secret Service connection that reconnects after the provider
restarts, B285, and replacing the session item instead of editing it, B291) now comes
from `proton-auth-rs`, shared with Proton Gallery and Proton Stream. No behaviour change.
- Config, machine-id and mount-failure files are written through `proton-base-rs`'s
atomic writer (temporary file, sync, rename, directory sync). The mount-failure file
is now synced before the rename too, so a crash cannot leave it empty.
### Fixed
- A large trash listed only its first items and never refreshed, and Empty Trash
never emptied it (B293). Needs proton-drive-rs 0.12.5.
## [3.9.0] — 2026-11-19
Schema: **46**. Photos adds a local database migration for incremental metadata sync.
### Added
- Open **Drive galleries** in Photos → Albums to browse saved My files folders and their
subfolders, choose album covers, and manage public links. Saved galleries and
covers sync with Proton Gallery on desktop and Android.
- Unlock hidden albums from the Albums view with your Linux sign-in.
### Changed
- Photos uses Proton Gallery's justified row planner. The Photos viewer starts
and seeks through supported videos without caching the whole original.
- Gallery-hidden albums and their contents are filtered from Photos and Drive
Galleries until unlocked.
- Photo grids defer work for off-screen tiles and keep the viewed photo anchored
during zoom.
### Fixed
- Unlocking Hidden albums keeps the private view open while its albums load.
- Hovering over an album highlights its card without shading the surrounding grid cell.
## [3.8.1] — 2026-11-19
Schema: **45**. No database migration in this release.
### Changed
- Refreshed the application icon and project artwork, and updated the documentation screenshots.
### Fixed
- The cache skips pinned files when looking for space to reclaim and reapplies the configured
limit when the service starts. Pinned files and recovery data are preserved.
- Trashing a folder waits for queued moves out of it, including moves from nested subfolders.
If a move is blocked by an item or folder already in Trash, Sync and `pdfs sync issues`
identify what to restore before retrying.
- The system tray displays a warning icon when sync needs attention or the session has expired.
- Updated the Proton SDK to 0.12.1 so a fast local clock no longer causes Proton to reject
uploads of new files and photos.
## [3.8.0] — 2026-11-14
Schema: **45**. Synced folders now tell device backups apart from folders attached to My files.
@@ -2353,6 +2454,8 @@ First stable release: FUSE files-on-demand mount, sync daemon under `proton-driv
- The outstanding FUSE defects tracked in `docs/BUGS.md`, plus a truncate defect, validated
by a new POSIX compliance suite for the filesystem.
[3.9.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v3.9.0
[3.8.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v3.8.1
[3.5.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v3.5.0
[3.4.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v3.4.1
[3.4.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v3.4.0
+3 -2
View File
@@ -70,7 +70,7 @@ The script runs `pdfs` from `PATH` on every Tab, so it keeps working after an up
| `pdfs login [-u EMAIL] [--remember]` | Sign in (password, then a 2FA code if the account has one) and store the session in the system keyring. Restarts the service if it is running. `--remember` keeps a two-password account unlocked across logins. |
| `pdfs unlock [--remember]` | Enter the mailbox password of a two-password account, so the service can open your files. Needed once per login session, or once with `--remember`. |
| `pdfs lock` | Forget the unlock held in memory; the account is locked again. |
| `pdfs logout` | End the session on Proton and remove the stored session. |
| `pdfs logout [--yes]` | End the session on Proton and remove the stored session. Asks first while changes are still queued; they stay on this computer and are sent after the next sign-in. |
| `pdfs status` | Account, mount, queue and cache state. |
| `pdfs quota` | Storage used across all Proton products. |
| `pdfs locations` | Every local place the service occupies: the Proton Drive folder and each synced folder, with mode and state. |
@@ -125,6 +125,7 @@ commands are for scripts and for cases where the mount is not available.
| `pdfs sync ignore check PATH` | Say whether a file in a synced folder is synced, and which pattern decides. |
| `pdfs sync limit [--up RATE] [--down RATE]` | Cap bandwidth, for example `500K` or `2M` per second. `0` removes a cap. Without options, prints the current caps. |
| `pdfs sync queue` | List changes that have not reached Proton Drive yet. |
| `pdfs sync flush [--timeout SECS]` | Retry failed changes now and wait until the queue is empty. Exits 0 when everything is on Proton Drive, 3 when what is left needs you (see `sync issues`), 4 on timeout. Says so while sync is paused or offline. |
| `pdfs sync issues` | List queued changes Proton Drive refused or that keep failing, and what to do. |
| `pdfs sync export ID DEST` | Save a copy of a queued upload's content. `DEST` may be a folder; an existing file is never overwritten. The upload stays queued. |
| `pdfs sync discard ID [--yes]` | Drop a queued change and undo it here: a new file or folder goes, anything else goes back to what Proton Drive has. Content not exported first is lost. |
@@ -153,7 +154,7 @@ commands are for scripts and for cases where the mount is not available.
| Command | Description |
|---|---|
| `pdfs trash` | List the Trash with the UIDs the next commands take. |
| `pdfs trash` | List the Trash with the UIDs the next commands take. `--by-folder` groups it by original folder; `--folder UID` lists a deleted folder. |
| `pdfs restore UIDS...` | Put items back where they were. |
| `pdfs delete-forever UIDS...` | Delete items permanently. |
| `pdfs empty-trash` | Delete everything in the Trash permanently. |
+1
View File
@@ -167,6 +167,7 @@ invented account, so they never show anyone's files:
```bash
scripts/screenshots.sh # all of them, into images/
scripts/screenshots.sh --only photos,files # just these
scripts/screenshots.sh --debug --skip-build # reuse already-built debug binaries
scripts/screenshots.sh --website ../proton-drive-linux-website # and copy them to the website
```
+9 -2
View File
@@ -71,13 +71,20 @@ terminal: `pdfs copy PATH... NEW_PARENT`.
## Trash
Deleted files go to the Trash, both from `~/ProtonDrive` and from the app. The **Trash** page
lists them with the date they were deleted.
shows the folders things were deleted from, with how many items each holds and when the last
one went. Open a folder to see what was deleted from it, and open a deleted folder to see what
it held. The folder button in the header switches to one list of everything, newest deletion
first, and the search box finds an item anywhere in the Trash by name.
- **Restore** puts an item back where it was, together with any trashed folders above it.
- **Delete Forever** and **Empty Trash** are permanent.
- Restore or delete a whole folder row to act on everything deleted from that folder.
From a terminal: `pdfs trash` lists the Trash with an ID for each item, and `pdfs restore`,
`pdfs delete-forever` and `pdfs empty-trash` take those IDs.
`pdfs delete-forever` and `pdfs empty-trash` take those IDs. `pdfs trash --by-folder` groups the
list by the folder each item was deleted from, and `pdfs trash --folder ID` shows what a deleted
folder holds.
## Earlier versions of a file
+5
View File
@@ -360,8 +360,12 @@ git clone https://github.com/narrrl/proton-drive-linux.git
cd proton-drive-linux
cargo build --release --locked
po/build.sh target/locale # optional: compile the translations
scripts/fetch-basemap.sh # optional: the offline Places map, into data/
```
Install `data/basemap-v1.pmtiles` as `~/.local/share/proton-drive-linux/basemap-v1.pmtiles`, or
point `PDFS_BASEMAP` at it. Without it Places shows coarse outlines only.
The four programs are in `target/release/`: `pdfs`, `pdfs-app`, `pdfs-tray` and `pdfs-prompt`.
### Build a package from the working tree
@@ -376,6 +380,7 @@ cd packaging && makepkg -fi
```bash
sudo dnf install -y rpm-build
scripts/fetch-basemap.sh
rpmbuild -bb packaging/proton-drive-linux.spec \
--define "git_dir $PWD" \
--define "_rpmdir $PWD/packaging/out" \
+40 -3
View File
@@ -1,5 +1,39 @@
# Photos
## How Photos is organized
Timeline, Albums, and Places share the same Photos workspace. The Timeline opens
with the media type, Filters, and, where available, **On this day**. Albums
contains the two collections **Proton albums** and **Drive galleries**. Add a
gallery by choosing a folder from **My files**. Opening it or any subfolder
keeps you in Photos. Back or Escape returns to the parent and its scroll
position.
**Hidden albums** opens with your Linux sign-in. The app uses the desktop
polkit prompt, so it needs a running authentication agent and the installed
polkit policy. Existing hidden albums remain hidden after upgrading. This view
includes hidden Proton albums and hidden Drive folders, including folders
inside public galleries. Leaving Hidden albums or switching away from the
window locks it and closes its previews.
For a system package, the policy is installed with the app. For a source build
or a per-user tarball install, have your system administrator install
`packaging/io.narl.proton-drive-linux.policy` (or the corresponding file under the
tarball's `share/polkit-1/actions/`) into the system polkit actions directory.
On NixOS, enable `security.polkit` and install the package system-wide to
register its policy.
GTK pages Timeline, Proton albums, and Places through the Drive daemon's Photos
control requests backed by its SQLite database. Saved Drive galleries come from
the shared Proton Gallery catalog, which the daemon hosts for the signed-in
account. Photos applies that catalog's visibility rules to public queries.
Hidden albums use separate, unlock-gated requests. Thumbnails in the hidden
view are kept in memory and cleared on lock.
Ctrl+mousewheel changes timeline tile size. GTK updates only the changed row
range and keeps the visible photo in place. As you scroll, each new page
updates the last affected section and adds new sections below it.
The **Photos** page of the app shows your Proton Photos library. Photos are stored apart from
My files, so they do not appear in `~/ProtonDrive`.
@@ -34,11 +68,14 @@ albums, and add or remove photos. Deleting an album keeps its photos in the time
**Places** groups photos by the town they were taken in, as cards or on a map. Town names come
from [GeoNames](https://www.geonames.org/) data (CC BY 4.0) and the map from
[Natural Earth](https://www.naturalearthdata.com/) (public domain). Both ship with the app, so
nothing is fetched online.
[Natural Earth](https://www.naturalearthdata.com/) (public domain): seas by depth, lakes, rivers,
built-up areas, borders and town names, in light and dark. Both ship with the app as
`/usr/share/proton-drive-linux/basemap-v1.pmtiles`, so nothing is fetched online. Without that file,
for example in a build from source that skipped `scripts/fetch-basemap.sh`, the map shows coarse
land outlines only. `PDFS_BASEMAP` points the app at another copy.
A street map from [OpenFreeMap](https://openfreemap.org/) (© OpenStreetMap contributors) can be
turned on under **Preferences → General → Appearance**. It is off by default, because the map
turned on instead under **Preferences → General → Appearance**. It is off by default, because the map
server learns which areas you look at.
## Viewer and duplicates
+2 -2
View File
@@ -43,7 +43,7 @@ them along.
| Item | State | Notes |
|---|---|---|
| Protect queued changes at shutdown | Not started | A systemd inhibitor lock while `staging/` holds unsent data, a warning when signing out, and a `pdfs sync flush` that waits for the queue. Today the queue is visible in `pdfs sync queue`, the tray and **Sync → Overview**, but nothing stops a shutdown (B66) |
| Protect queued changes at shutdown | Done, live check owed | The daemon holds a logind shutdown inhibitor while the queue holds changes it can send, signing out warns with the count, and `pdfs sync flush` waits for the queue (B66) |
| Restore pins from the machine profile | Not started | `profile.json` records pins; `pdfs sync restore` does not apply them yet |
| Profile backup health | Not started | A failed profile upload is a `WARN` in the log and nothing in the app (B68) |
| Rotated credentials survive a keyring failure | Open | Report and retry keyring write failures, so a rotated single-use refresh token is not lost (B61) |
@@ -63,7 +63,7 @@ them along.
| Add someone else's photo to your album | Blocked | Needs `copyPhoto` in the SDK |
| Photo name search | Not started | Photos search by date and filters only |
| Automatic photo upload from a folder | Not started | |
| Faster Google Photos import | Not started | Uploads run one at a time. Trash folders of locales missing from `TRASH_FOLDERS` import as photos |
| Faster Google Photos import | Partly done | Two uploads run at once. Trash folders of locales missing from `TRASH_FOLDERS` import as photos |
| More of a Google Photos export | Not started | `.tgz` exports and extracted folders; shared albums told apart from own ones; album descriptions and covers; photo descriptions |
| Content-addressed block cache | Idea | Store blocks by hash to deduplicate identical content |
+2 -1
View File
@@ -75,7 +75,8 @@ PDFS_SIM_SEED=7 RUST_LOG=pdfs_fuse=debug cargo test -p pdfs-fuse --lib \
| `PDFS_SIM_MEASURE` | Run `a_thousand_files_drain_as_fast_on_wifi_as_on_lan`, which times 1,000 files in 100 folders on a LAN and a Wi-Fi profile and prints the times |
A failure is a seed that can be replayed, but the daemon uses real threads, so a race may need
several replays. In CI, the `simulation` job uploads the stacks of a hung run as an artifact.
several replays. In CI, the `simulation` job uploads the failed seed's steps and daemon logs,
along with thread stacks when the watchdog catches a hang.
## Acceptance suite
+40
View File
@@ -0,0 +1,40 @@
# Violet / Copper vector branding
Drive, Stream, and Gallery share frontal violet–pink faces, shallow metallic copper backing, and narrow edge highlights. The icons are editable vector artwork with no embedded images or raster effects.
| App | Mark |
| --- | --- |
| Proton Drive for Linux | Open folder with lilac documents |
| Proton Stream | Rounded play prism |
| Proton Gallery | Stacked photo plates, mountains, and a copper sun |
## Master artwork
Edit `images/logo.svg`. It uses a 256-unit grid with transparent margins, a `mark` group, and reusable linear gradients. The copper backing extends six units up and right. SVG paths, rounded rectangles, circles, strokes, translation, and the photo clip are retained when exporting Android VectorDrawables. Unsupported SVG features cause generation to fail rather than silently change the artwork.
Earlier artwork is preserved in `images/branding-archive/`. The new assets replace the active branding without removing previous designs.
## Regeneration
```sh
python scripts/render-branding.py
python scripts/render-branding.py --check
```
Requires Python 3, librsvg (`rsvg-convert`), and ImageMagick (`magick`). Stream's TV banner also requires fontconfig and DejaVu Sans. Keep the generator identical across the three repositories; every checkout renders independently and requires no sibling checkout. Edit the canonical master rather than generated platform assets.
## Platform treatment
- Desktop SVGs and PNGs retain transparent margins and the exact approved geometry. The same colored artwork works on light and dark surfaces.
- Android adaptive icons use a pale lavender full-bleed background and a separate foreground containing the complete object. The 256-unit canvas maps to the central 64dp of the 108dp adaptive canvas; the painted geometry stays within the central safe circle. Android supplies the outer mask.
- Themed launcher icons and Stream's status-bar notification icon use a white silhouette without colored bevel highlights.
- iOS uses the same play prism over an opaque pale lavender square, exported at 1024px. The system supplies the corner mask.
- The Android TV banner pairs the play prism with the full Proton Stream name.
Keep full product names in launcher metadata. Application UI typography and photo-content icons are independent of this branding.
## Outputs
`packaging/io.narl.proton-drive-linux.svg` is installed by the desktop packages and embedded in the GTK resource bundle for in-app branding.
The Drive cover (`images/cover.svg` and `images/cover.png`) also embeds the canonical icon. The generator updates its `app-logo` group while preserving the surrounding cover layout.
Binary file not shown.

After

Width:  |  Height:  |  Size: 363 KiB

@@ -0,0 +1,156 @@
<svg width="1920" height="1080" viewBox="0 0 1920 1080" xmlns="http://www.w3.org/2000/svg">
<defs>
<linearGradient id="bg" x1="0" y1="0" x2="1920" y2="1080" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#1A1240"/>
<stop offset="1" stop-color="#0A0618"/>
</linearGradient>
<radialGradient id="glow1" cx="1390" cy="520" r="640" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#6D4AFF" stop-opacity="0.55"/>
<stop offset="0.5" stop-color="#6D4AFF" stop-opacity="0.15"/>
<stop offset="1" stop-color="#6D4AFF" stop-opacity="0"/>
</radialGradient>
<radialGradient id="glow2" cx="160" cy="1080" r="700" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#A855F7" stop-opacity="0.25"/>
<stop offset="1" stop-color="#A855F7" stop-opacity="0"/>
</radialGradient>
<pattern id="dots" width="32" height="32" patternUnits="userSpaceOnUse">
<circle cx="2" cy="2" r="1.6" fill="#FFFFFF"/>
</pattern>
<radialGradient id="dotfade" cx="1390" cy="520" r="900" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#FFFFFF" stop-opacity="1"/>
<stop offset="1" stop-color="#FFFFFF" stop-opacity="0"/>
</radialGradient>
<mask id="dotmask" maskUnits="userSpaceOnUse" x="0" y="0" width="1920" height="1080">
<rect width="1920" height="1080" fill="url(#dotfade)"/>
</mask>
<linearGradient id="title2" x1="140" y1="0" x2="760" y2="0" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#C9BBFF"/>
<stop offset="1" stop-color="#8A6CFF"/>
</linearGradient>
<!-- app logo -->
<linearGradient id="pdl-face" x1="12" y1="12" x2="116" y2="112" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#A48BFF"/>
<stop offset="0.55" stop-color="#7552FF"/>
<stop offset="1" stop-color="#5B36F2"/>
</linearGradient>
<linearGradient id="pdl-cloud" x1="0" y1="34" x2="0" y2="88" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#FFFFFF"/>
<stop offset="1" stop-color="#ECE6FF"/>
</linearGradient>
<mask id="pdl-cut" maskUnits="userSpaceOnUse" x="0" y="0" width="128" height="128">
<g fill="#FFFFFF">
<circle cx="66" cy="57" r="23"/>
<circle cx="42" cy="68" r="16"/>
<circle cx="88" cy="69" r="15"/>
<rect x="26" y="68" width="77" height="20" rx="10"/>
</g>
<g stroke="#000000" stroke-width="7.5" stroke-linecap="round" stroke-linejoin="round" fill="none">
<path d="M64 49v26"/>
<path d="M54.5 65.5l9.5 9.5 9.5-9.5"/>
</g>
</mask>
<filter id="logoshadow" x="-50%" y="-50%" width="200%" height="200%">
<feGaussianBlur stdDeviation="6"/>
</filter>
<filter id="chipshadow" x="-50%" y="-50%" width="200%" height="200%">
<feGaussianBlur stdDeviation="14"/>
</filter>
</defs>
<!-- background -->
<rect width="1920" height="1080" fill="url(#bg)"/>
<rect width="1920" height="1080" fill="url(#glow2)"/>
<rect width="1920" height="1080" fill="url(#glow1)"/>
<rect width="1920" height="1080" fill="url(#dots)" opacity="0.07" mask="url(#dotmask)"/>
<!-- orbit rings -->
<g fill="none" stroke="#FFFFFF">
<circle cx="1390" cy="520" r="300" stroke-opacity="0.10" stroke-width="2"/>
<circle cx="1390" cy="520" r="420" stroke-opacity="0.06" stroke-width="2" stroke-dasharray="4 14" stroke-linecap="round"/>
</g>
<!-- logo, 128 -> 384 -->
<g transform="translate(1198 328) scale(3)">
<rect x="16" y="24" width="96" height="92" rx="24" fill="#2A1080" opacity="0.8" filter="url(#logoshadow)"/>
<rect x="12" y="16" width="104" height="100" rx="26" fill="#4424C0"/>
<rect x="12" y="12" width="104" height="100" rx="26" fill="url(#pdl-face)"/>
<rect x="12.5" y="12.5" width="103" height="99" rx="25.5" fill="none" stroke="#FFFFFF" stroke-opacity="0.2"/>
<rect y="3" width="128" height="128" fill="#2E1596" fill-opacity="0.3" mask="url(#pdl-cut)"/>
<rect width="128" height="128" fill="url(#pdl-cloud)" mask="url(#pdl-cut)"/>
</g>
<!-- feature chips around the logo -->
<g font-family="Adwaita Sans" font-size="26" font-weight="600" fill="#FFFFFF">
<!-- folder, top left -->
<g transform="translate(1000 200)">
<rect x="0" y="10" width="236" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="236" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#6D4AFF"/>
<path d="M8 13a2 2 0 0 1 2-2h5l2.5 2.5H26a2 2 0 0 1 2 2V25a2 2 0 0 1-2 2H10a2 2 0 0 1-2-2z" fill="#FFFFFF"/>
</g>
<text x="74" y="47">On demand</text>
</g>
<!-- sync, top right -->
<g transform="translate(1590 290)">
<rect x="0" y="10" width="164" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="164" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#1EA885"/>
<g fill="none" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round">
<path d="M26 15a8.5 8.5 0 0 0-15-2"/>
<path d="M10 9v4.5h4.5"/>
<path d="M10 21a8.5 8.5 0 0 0 15 2"/>
<path d="M26 27v-4.5h-4.5"/>
</g>
</g>
<text x="74" y="47">Sync</text>
</g>
<!-- photos, bottom right -->
<g transform="translate(1560 690)">
<rect x="0" y="10" width="206" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="206" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#E8618C"/>
<rect x="8" y="9" width="20" height="18" rx="3" fill="none" stroke="#FFFFFF" stroke-width="2.4"/>
<circle cx="14" cy="15" r="2" fill="#FFFFFF"/>
<path d="M9.5 25l6-6 4 4 3-3 4 4" fill="none" stroke="#FFFFFF" stroke-width="2.4" stroke-linejoin="round"/>
</g>
<text x="74" y="47">Photos</text>
</g>
<!-- terminal, bottom left -->
<g transform="translate(980 790)">
<rect x="0" y="10" width="206" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="206" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#3B3F5C"/>
<path d="M9 12l6 6-6 6" fill="none" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round"/>
<path d="M18 25h9" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round"/>
</g>
<text x="74" y="48" font-family="JetBrainsMono NF" font-size="25" font-weight="700">pdfs</text>
<text x="150" y="48" font-weight="500" fill="#B9B0D9">CLI</text>
</g>
</g>
<!-- text block -->
<g font-family="Adwaita Sans">
<g transform="translate(140 292)">
<rect width="410" height="48" rx="24" fill="#6D4AFF" fill-opacity="0.18" stroke="#8A6CFF" stroke-opacity="0.6" stroke-width="1.5"/>
<circle cx="28" cy="24" r="6" fill="#A48BFF"/>
<text x="46" y="32" font-size="21" font-weight="700" letter-spacing="2.4" fill="#D6CCFF">UNOFFICIAL · OPEN SOURCE</text>
</g>
<text x="134" y="478" font-size="124" font-weight="800" letter-spacing="-3" fill="#FFFFFF">Proton Drive</text>
<text x="134" y="610" font-size="124" font-weight="800" letter-spacing="-3" fill="url(#title2)">for Linux</text>
<text x="140" y="702" font-size="38" font-weight="400" fill="#B9B0D9">Your Drive as a folder, and a native GTK4 app.</text>
<text x="140" y="752" font-size="38" font-weight="400" fill="#B9B0D9">No Electron.</text>
</g>
<!-- footer -->
<g font-family="JetBrainsMono NF" font-size="24" fill="#8C83B0">
<text x="140" y="968">github.com/narrrl/proton-drive-linux</text>
</g>
<g font-family="Adwaita Sans" font-size="22" fill="#6F6894">
<text x="1780" y="968" text-anchor="end">Not affiliated with Proton AG</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 8.3 KiB

@@ -0,0 +1,27 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128" viewBox="0 0 128 128"
role="img" aria-label="Proton Drive for Linux" data-app="drive">
<title>Proton Drive for Linux</title>
<desc>A custom faceted D with an inset mount channel, in violet warming to copper.</desc>
<!-- Canonical artwork. Regenerate platform assets with scripts/render-branding.py. -->
<defs>
<linearGradient id="face" x1="18" y1="12" x2="112" y2="116" gradientUnits="userSpaceOnUse">
<stop stop-color="#F6F2FC"/>
<stop offset="1" stop-color="#E5DDEE"/>
</linearGradient>
<linearGradient id="material" x1="31" y1="35" x2="96" y2="95" gradientUnits="userSpaceOnUse">
<stop stop-color="#9170E2"/>
<stop offset=".38" stop-color="#6D4AFF"/>
<stop offset=".6" stop-color="#7044D1"/>
<stop offset=".82" stop-color="#A66598"/>
<stop offset="1" stop-color="#B56D51"/>
</linearGradient>
</defs>
<rect x="12" y="12" width="104" height="104" rx="27" fill="url(#face)"/>
<rect x="12.75" y="12.75" width="102.5" height="102.5" rx="26.25"
fill="none" stroke="#A18CB7" stroke-width="1.5"/>
<g id="mark">
<path d="M 43.32 33.92 H 64.94 C 73.4 33.92 80.92 36.74 86.56 42.38 L 96.9 52.72 C 103.48 59.3 103.48 68.7 96.9 75.28 L 86.56 85.62 C 80.92 91.26 73.4 94.08 64.94 94.08 H 43.32 C 38.15 94.08 33.92 89.85 33.92 84.68 V 75.28 H 47.08 V 79.04 C 47.08 80.074 47.926 80.92 48.96 80.92 H 64.94 C 69.64 80.92 73.4 79.04 77.16 75.28 L 85.62 66.82 C 87.218 65.222 87.218 62.778 85.62 61.18 L 77.16 52.72 C 73.4 48.96 69.64 47.08 64.94 47.08 H 48.96 C 47.926 47.08 47.08 47.926 47.08 48.96 V 52.72 H 33.92 V 43.32 C 33.92 38.15 38.15 33.92 43.32 33.92 Z" fill="url(#material)" fill-rule="nonzero"/>
<path d="M 29.22 58.36 H 62.12 C 65.222 58.36 67.76 60.898 67.76 64 C 67.76 67.102 65.222 69.64 62.12 69.64 H 29.22 C 26.118 69.64 23.58 67.102 23.58 64 C 23.58 60.898 26.118 58.36 29.22 58.36 Z" fill="url(#material)" fill-rule="nonzero"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 2.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 51 KiB

After

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 137 KiB

After

Width:  |  Height:  |  Size: 138 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 363 KiB

After

Width:  |  Height:  |  Size: 407 KiB

+157 -149
View File
@@ -1,156 +1,164 @@
<svg width="1920" height="1080" viewBox="0 0 1920 1080" xmlns="http://www.w3.org/2000/svg">
<defs>
<linearGradient id="bg" x1="0" y1="0" x2="1920" y2="1080" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#1A1240"/>
<stop offset="1" stop-color="#0A0618"/>
</linearGradient>
<radialGradient id="glow1" cx="1390" cy="520" r="640" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#6D4AFF" stop-opacity="0.55"/>
<stop offset="0.5" stop-color="#6D4AFF" stop-opacity="0.15"/>
<stop offset="1" stop-color="#6D4AFF" stop-opacity="0"/>
</radialGradient>
<radialGradient id="glow2" cx="160" cy="1080" r="700" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#A855F7" stop-opacity="0.25"/>
<stop offset="1" stop-color="#A855F7" stop-opacity="0"/>
</radialGradient>
<pattern id="dots" width="32" height="32" patternUnits="userSpaceOnUse">
<circle cx="2" cy="2" r="1.6" fill="#FFFFFF"/>
</pattern>
<radialGradient id="dotfade" cx="1390" cy="520" r="900" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#FFFFFF" stop-opacity="1"/>
<stop offset="1" stop-color="#FFFFFF" stop-opacity="0"/>
</radialGradient>
<mask id="dotmask" maskUnits="userSpaceOnUse" x="0" y="0" width="1920" height="1080">
<rect width="1920" height="1080" fill="url(#dotfade)"/>
</mask>
<linearGradient id="title2" x1="140" y1="0" x2="760" y2="0" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#C9BBFF"/>
<stop offset="1" stop-color="#8A6CFF"/>
</linearGradient>
<!-- app logo -->
<linearGradient id="pdl-face" x1="12" y1="12" x2="116" y2="112" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#A48BFF"/>
<stop offset="0.55" stop-color="#7552FF"/>
<stop offset="1" stop-color="#5B36F2"/>
</linearGradient>
<linearGradient id="pdl-cloud" x1="0" y1="34" x2="0" y2="88" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#FFFFFF"/>
<stop offset="1" stop-color="#ECE6FF"/>
</linearGradient>
<mask id="pdl-cut" maskUnits="userSpaceOnUse" x="0" y="0" width="128" height="128">
<g fill="#FFFFFF">
<circle cx="66" cy="57" r="23"/>
<circle cx="42" cy="68" r="16"/>
<circle cx="88" cy="69" r="15"/>
<rect x="26" y="68" width="77" height="20" rx="10"/>
</g>
<g stroke="#000000" stroke-width="7.5" stroke-linecap="round" stroke-linejoin="round" fill="none">
<path d="M64 49v26"/>
<path d="M54.5 65.5l9.5 9.5 9.5-9.5"/>
</g>
</mask>
<filter id="logoshadow" x="-50%" y="-50%" width="200%" height="200%">
<feGaussianBlur stdDeviation="6"/>
</filter>
<filter id="chipshadow" x="-50%" y="-50%" width="200%" height="200%">
<feGaussianBlur stdDeviation="14"/>
</filter>
</defs>
<!-- background -->
<rect width="1920" height="1080" fill="url(#bg)"/>
<rect width="1920" height="1080" fill="url(#glow2)"/>
<rect width="1920" height="1080" fill="url(#glow1)"/>
<rect width="1920" height="1080" fill="url(#dots)" opacity="0.07" mask="url(#dotmask)"/>
<!-- orbit rings -->
<g fill="none" stroke="#FFFFFF">
<circle cx="1390" cy="520" r="300" stroke-opacity="0.10" stroke-width="2"/>
<circle cx="1390" cy="520" r="420" stroke-opacity="0.06" stroke-width="2" stroke-dasharray="4 14" stroke-linecap="round"/>
</g>
<!-- logo, 128 -> 384 -->
<g transform="translate(1198 328) scale(3)">
<rect x="16" y="24" width="96" height="92" rx="24" fill="#2A1080" opacity="0.8" filter="url(#logoshadow)"/>
<rect x="12" y="16" width="104" height="100" rx="26" fill="#4424C0"/>
<rect x="12" y="12" width="104" height="100" rx="26" fill="url(#pdl-face)"/>
<rect x="12.5" y="12.5" width="103" height="99" rx="25.5" fill="none" stroke="#FFFFFF" stroke-opacity="0.2"/>
<rect y="3" width="128" height="128" fill="#2E1596" fill-opacity="0.3" mask="url(#pdl-cut)"/>
<rect width="128" height="128" fill="url(#pdl-cloud)" mask="url(#pdl-cut)"/>
</g>
<!-- feature chips around the logo -->
<g font-family="Adwaita Sans" font-size="26" font-weight="600" fill="#FFFFFF">
<!-- folder, top left -->
<g transform="translate(1000 200)">
<rect x="0" y="10" width="236" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="236" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#6D4AFF"/>
<path d="M8 13a2 2 0 0 1 2-2h5l2.5 2.5H26a2 2 0 0 1 2 2V25a2 2 0 0 1-2 2H10a2 2 0 0 1-2-2z" fill="#FFFFFF"/>
</g>
<text x="74" y="47">On demand</text>
<?xml version="1.0" encoding="utf-8"?>
<!-- Generated by scripts/render-branding.py; edit images/logo.svg. -->
<svg xmlns="http://www.w3.org/2000/svg" width="1920" height="1080" viewBox="0 0 1920 1080">
<defs>
<linearGradient id="bg" x1="0" y1="0" x2="1920" y2="1080" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#1A1240" />
<stop offset="1" stop-color="#0A0618" />
</linearGradient>
<radialGradient id="glow1" cx="1390" cy="520" r="640" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#6D4AFF" stop-opacity="0.55" />
<stop offset="0.5" stop-color="#6D4AFF" stop-opacity="0.15" />
<stop offset="1" stop-color="#6D4AFF" stop-opacity="0" />
</radialGradient>
<radialGradient id="glow2" cx="160" cy="1080" r="700" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#A855F7" stop-opacity="0.25" />
<stop offset="1" stop-color="#A855F7" stop-opacity="0" />
</radialGradient>
<pattern id="dots" width="32" height="32" patternUnits="userSpaceOnUse">
<circle cx="2" cy="2" r="1.6" fill="#FFFFFF" />
</pattern>
<radialGradient id="dotfade" cx="1390" cy="520" r="900" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#FFFFFF" stop-opacity="1" />
<stop offset="1" stop-color="#FFFFFF" stop-opacity="0" />
</radialGradient>
<mask id="dotmask" maskUnits="userSpaceOnUse" x="0" y="0" width="1920" height="1080">
<rect width="1920" height="1080" fill="url(#dotfade)" />
</mask>
<linearGradient id="title2" x1="140" y1="0" x2="760" y2="0" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#C9BBFF" />
<stop offset="1" stop-color="#8A6CFF" />
</linearGradient>
<filter id="chipshadow" x="-50%" y="-50%" width="200%" height="200%">
<feGaussianBlur stdDeviation="14" />
</filter>
</defs>
<rect width="1920" height="1080" fill="url(#bg)" />
<rect width="1920" height="1080" fill="url(#glow2)" />
<rect width="1920" height="1080" fill="url(#glow1)" />
<rect width="1920" height="1080" fill="url(#dots)" opacity="0.07" mask="url(#dotmask)" />
<g fill="none" stroke="#FFFFFF">
<circle cx="1390" cy="520" r="300" stroke-opacity="0.10" stroke-width="2" />
<circle cx="1390" cy="520" r="420" stroke-opacity="0.06" stroke-width="2" stroke-dasharray="4 14" stroke-linecap="round" />
</g>
<!-- sync, top right -->
<g transform="translate(1590 290)">
<rect x="0" y="10" width="164" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="164" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#1EA885"/>
<g fill="none" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round">
<path d="M26 15a8.5 8.5 0 0 0-15-2"/>
<path d="M10 9v4.5h4.5"/>
<path d="M10 21a8.5 8.5 0 0 0 15 2"/>
<path d="M26 27v-4.5h-4.5"/>
<g id="app-logo" transform="translate(1198 328) scale(1.5)">
<defs>
<linearGradient id="drive-face" x1="36" y1="212" x2="220" y2="54" gradientUnits="userSpaceOnUse">
<stop stop-color="#6240DA" />
<stop offset=".52" stop-color="#8056EF" />
<stop offset="1" stop-color="#D58BDF" />
</linearGradient>
<linearGradient id="drive-rear" x1="26" y1="42" x2="218" y2="220" gradientUnits="userSpaceOnUse">
<stop stop-color="#8C68EE" />
<stop offset="1" stop-color="#5533B5" />
</linearGradient>
<linearGradient id="drive-paper" x1="60" y1="80" x2="180" y2="162" gradientUnits="userSpaceOnUse">
<stop stop-color="#E7DFFA" />
<stop offset="1" stop-color="#BDA5EF" />
</linearGradient>
<linearGradient id="drive-copper" x1="32" y1="32" x2="232" y2="224" gradientUnits="userSpaceOnUse">
<stop stop-color="#703E30" />
<stop offset=".16" stop-color="#B47456" />
<stop offset=".28" stop-color="#E6B296" />
<stop offset=".32" stop-color="#F8D8C3" />
<stop offset=".37" stop-color="#BB7C5A" />
<stop offset=".56" stop-color="#8A4D36" />
<stop offset=".69" stop-color="#D69B77" />
<stop offset=".75" stop-color="#F2C9AA" />
<stop offset=".81" stop-color="#B87956" />
<stop offset="1" stop-color="#713F30" />
</linearGradient>
<linearGradient id="drive-sun" x1="172" y1="82" x2="201" y2="112" gradientUnits="userSpaceOnUse">
<stop stop-color="#8D513A" />
<stop offset=".33" stop-color="#DDA581" />
<stop offset=".47" stop-color="#F7D8B8" />
<stop offset=".53" stop-color="#D69B74" />
<stop offset="1" stop-color="#925439" />
</linearGradient>
<linearGradient id="drive-mountain" x1="44" y1="148" x2="186" y2="218" gradientUnits="userSpaceOnUse">
<stop stop-color="#D9C9FA" />
<stop offset="1" stop-color="#A582E9" />
</linearGradient>
<linearGradient id="drive-far-mountain" x1="142" y1="152" x2="211" y2="216" gradientUnits="userSpaceOnUse">
<stop stop-color="#E8B4EB" />
<stop offset="1" stop-color="#BB7BD7" />
</linearGradient>
<clipPath id="drive-photo-clip">
<rect x="28" y="54" width="194" height="172" rx="24" />
</clipPath>
</defs>
<g id="mark">
<path d="M48 48H91Q99 48 105 54L117 66Q123 72 132 72H208Q228 72 228 92V200Q228 220 208 220H48Q28 220 28 200V68Q28 48 48 48Z" fill="url(#drive-copper)" transform="translate(6 -6)" stroke="#7D4936" stroke-width=".8" stroke-linejoin="round" />
<path d="M48 48H91Q99 48 105 54L117 66Q123 72 132 72H208Q228 72 228 92V200Q228 220 208 220H48Q28 220 28 200V68Q28 48 48 48Z" fill="url(#drive-rear)" stroke="#5E3CBD" stroke-width=".8" />
<rect x="72" y="82" width="132" height="112" rx="10" fill="url(#drive-paper)" stroke="#EFE9FC" stroke-width="1" />
<rect x="51" y="97" width="140" height="109" rx="10" fill="url(#drive-paper)" stroke="#EFE9FC" stroke-width="1" />
<path d="M48 94H89Q97 94 103 100L115 112Q121 118 130 118H208Q228 118 228 138V206Q228 226 208 226H48Q28 226 28 206V114Q28 94 48 94Z" fill="url(#drive-copper)" transform="translate(6 -6)" stroke="#7D4936" stroke-width=".8" stroke-linejoin="round" />
<path d="M48 94H89Q97 94 103 100L115 112Q121 118 130 118H208Q228 118 228 138V206Q228 226 208 226H48Q28 226 28 206V114Q28 94 48 94Z" fill="url(#drive-face)" stroke="#5936B7" stroke-width=".9" />
<path d="M32 113Q32 98 48 98H89Q95 98 100 103L112 115Q120 122 130 122H208Q224 122 224 138" fill="none" stroke="#F3DDF9" stroke-opacity=".52" stroke-width="1.5" stroke-linecap="round" />
</g>
</g>
<text x="74" y="47">Sync</text>
</g>
<!-- photos, bottom right -->
<g transform="translate(1560 690)">
<rect x="0" y="10" width="206" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="206" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#E8618C"/>
<rect x="8" y="9" width="20" height="18" rx="3" fill="none" stroke="#FFFFFF" stroke-width="2.4"/>
<circle cx="14" cy="15" r="2" fill="#FFFFFF"/>
<path d="M9.5 25l6-6 4 4 3-3 4 4" fill="none" stroke="#FFFFFF" stroke-width="2.4" stroke-linejoin="round"/>
</g>
<text x="74" y="47">Photos</text>
<g font-family="Adwaita Sans" font-size="26" font-weight="600" fill="#FFFFFF">
<g transform="translate(1000 200)">
<rect x="0" y="10" width="236" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)" />
<rect x="0" y="0" width="236" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5" />
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#6D4AFF" />
<path d="M8 13a2 2 0 0 1 2-2h5l2.5 2.5H26a2 2 0 0 1 2 2V25a2 2 0 0 1-2 2H10a2 2 0 0 1-2-2z" fill="#FFFFFF" />
</g>
<text x="74" y="47">On demand</text>
</g>
<g transform="translate(1590 290)">
<rect x="0" y="10" width="164" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)" />
<rect x="0" y="0" width="164" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5" />
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#1EA885" />
<g fill="none" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round">
<path d="M26 15a8.5 8.5 0 0 0-15-2" />
<path d="M10 9v4.5h4.5" />
<path d="M10 21a8.5 8.5 0 0 0 15 2" />
<path d="M26 27v-4.5h-4.5" />
</g>
</g>
<text x="74" y="47">Sync</text>
</g>
<g transform="translate(1560 690)">
<rect x="0" y="10" width="206" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)" />
<rect x="0" y="0" width="206" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5" />
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#E8618C" />
<rect x="8" y="9" width="20" height="18" rx="3" fill="none" stroke="#FFFFFF" stroke-width="2.4" />
<circle cx="14" cy="15" r="2" fill="#FFFFFF" />
<path d="M9.5 25l6-6 4 4 3-3 4 4" fill="none" stroke="#FFFFFF" stroke-width="2.4" stroke-linejoin="round" />
</g>
<text x="74" y="47">Photos</text>
</g>
<g transform="translate(980 790)">
<rect x="0" y="10" width="206" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)" />
<rect x="0" y="0" width="206" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5" />
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#3B3F5C" />
<path d="M9 12l6 6-6 6" fill="none" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round" />
<path d="M18 25h9" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round" />
</g>
<text x="74" y="48" font-family="JetBrainsMono NF" font-size="25" font-weight="700">pdfs</text>
<text x="150" y="48" font-weight="500" fill="#B9B0D9">CLI</text>
</g>
</g>
<!-- terminal, bottom left -->
<g transform="translate(980 790)">
<rect x="0" y="10" width="206" height="76" rx="22" fill="#000000" opacity="0.35" filter="url(#chipshadow)"/>
<rect x="0" y="0" width="206" height="76" rx="22" fill="#2A2058" fill-opacity="0.85" stroke="#FFFFFF" stroke-opacity="0.14" stroke-width="1.5"/>
<g transform="translate(22 20)">
<rect width="36" height="36" rx="10" fill="#3B3F5C"/>
<path d="M9 12l6 6-6 6" fill="none" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round"/>
<path d="M18 25h9" stroke="#FFFFFF" stroke-width="2.6" stroke-linecap="round"/>
</g>
<text x="74" y="48" font-family="JetBrainsMono NF" font-size="25" font-weight="700">pdfs</text>
<text x="150" y="48" font-weight="500" fill="#B9B0D9">CLI</text>
<g font-family="Adwaita Sans">
<g transform="translate(140 292)">
<rect width="410" height="48" rx="24" fill="#6D4AFF" fill-opacity="0.18" stroke="#8A6CFF" stroke-opacity="0.6" stroke-width="1.5" />
<circle cx="28" cy="24" r="6" fill="#A48BFF" />
<text x="46" y="32" font-size="21" font-weight="700" letter-spacing="2.4" fill="#D6CCFF">UNOFFICIAL · OPEN SOURCE</text>
</g>
<text x="134" y="478" font-size="124" font-weight="800" letter-spacing="-3" fill="#FFFFFF">Proton Drive</text>
<text x="134" y="610" font-size="124" font-weight="800" letter-spacing="-3" fill="url(#title2)">for Linux</text>
<text x="140" y="702" font-size="38" font-weight="400" fill="#B9B0D9">Your Drive as a folder, and a native GTK4 app.</text>
<text x="140" y="752" font-size="38" font-weight="400" fill="#B9B0D9">No Electron.</text>
</g>
</g>
<!-- text block -->
<g font-family="Adwaita Sans">
<g transform="translate(140 292)">
<rect width="410" height="48" rx="24" fill="#6D4AFF" fill-opacity="0.18" stroke="#8A6CFF" stroke-opacity="0.6" stroke-width="1.5"/>
<circle cx="28" cy="24" r="6" fill="#A48BFF"/>
<text x="46" y="32" font-size="21" font-weight="700" letter-spacing="2.4" fill="#D6CCFF">UNOFFICIAL · OPEN SOURCE</text>
<g font-family="JetBrainsMono NF" font-size="24" fill="#8C83B0">
<text x="140" y="968">github.com/narrrl/proton-drive-linux</text>
</g>
<g font-family="Adwaita Sans" font-size="22" fill="#6F6894">
<text x="1780" y="968" text-anchor="end">Not affiliated with Proton AG</text>
</g>
<text x="134" y="478" font-size="124" font-weight="800" letter-spacing="-3" fill="#FFFFFF">Proton Drive</text>
<text x="134" y="610" font-size="124" font-weight="800" letter-spacing="-3" fill="url(#title2)">for Linux</text>
<text x="140" y="702" font-size="38" font-weight="400" fill="#B9B0D9">Your Drive as a folder, and a native GTK4 app.</text>
<text x="140" y="752" font-size="38" font-weight="400" fill="#B9B0D9">No Electron.</text>
</g>
<!-- footer -->
<g font-family="JetBrainsMono NF" font-size="24" fill="#8C83B0">
<text x="140" y="968">github.com/narrrl/proton-drive-linux</text>
</g>
<g font-family="Adwaita Sans" font-size="22" fill="#6F6894">
<text x="1780" y="968" text-anchor="end">Not affiliated with Proton AG</text>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 8.3 KiB

After

Width:  |  Height:  |  Size: 11 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 115 KiB

After

Width:  |  Height:  |  Size: 115 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 33 KiB

+43 -35
View File
@@ -1,36 +1,44 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg width="128" height="128" viewBox="0 0 128 128" fill="none" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Proton Drive for Linux">
<defs>
<linearGradient id="pdl-face" x1="12" y1="12" x2="116" y2="112" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#A48BFF"/>
<stop offset="0.55" stop-color="#7552FF"/>
<stop offset="1" stop-color="#5B36F2"/>
</linearGradient>
<linearGradient id="pdl-cloud" x1="0" y1="34" x2="0" y2="88" gradientUnits="userSpaceOnUse">
<stop offset="0" stop-color="#FFFFFF"/>
<stop offset="1" stop-color="#ECE6FF"/>
</linearGradient>
<!-- the cloud, with the arrow cut out of it as negative space -->
<mask id="pdl-cut" maskUnits="userSpaceOnUse" x="0" y="0" width="128" height="128">
<g fill="#FFFFFF">
<circle cx="66" cy="57" r="23"/>
<circle cx="42" cy="68" r="16"/>
<circle cx="88" cy="69" r="15"/>
<rect x="26" y="68" width="77" height="20" rx="10"/>
</g>
<g stroke="#000000" stroke-width="7.5" stroke-linecap="round" stroke-linejoin="round">
<path d="M64 49v26"/>
<path d="M54.5 65.5l9.5 9.5 9.5-9.5"/>
</g>
</mask>
</defs>
<!-- base: a rounded tile with a darker lower edge for depth -->
<rect x="12" y="16" width="104" height="100" rx="26" fill="#4424C0"/>
<rect x="12" y="12" width="104" height="100" rx="26" fill="url(#pdl-face)"/>
<rect x="12.5" y="12.5" width="103" height="99" rx="25.5" stroke="#FFFFFF" stroke-opacity="0.2"/>
<!-- cloud shadow, then the cloud itself -->
<rect y="3" width="128" height="128" fill="#2E1596" fill-opacity="0.3" mask="url(#pdl-cut)"/>
<rect width="128" height="128" fill="url(#pdl-cloud)" mask="url(#pdl-cut)"/>
<svg xmlns="http://www.w3.org/2000/svg" width="256" height="256" viewBox="0 0 256 256" data-app="drive" role="img" aria-labelledby="drive-title drive-desc">
<title id="drive-title">Proton Drive for Linux</title>
<desc id="drive-desc">Frontal violet and pink vector icon with shallow metallic copper backing.</desc>
<defs>
<linearGradient id="drive-face" x1="36" y1="212" x2="220" y2="54" gradientUnits="userSpaceOnUse">
<stop stop-color="#6240DA"/><stop offset=".52" stop-color="#8056EF"/><stop offset="1" stop-color="#D58BDF"/>
</linearGradient>
<linearGradient id="drive-rear" x1="26" y1="42" x2="218" y2="220" gradientUnits="userSpaceOnUse">
<stop stop-color="#8C68EE"/><stop offset="1" stop-color="#5533B5"/>
</linearGradient>
<linearGradient id="drive-paper" x1="60" y1="80" x2="180" y2="162" gradientUnits="userSpaceOnUse">
<stop stop-color="#E7DFFA"/><stop offset="1" stop-color="#BDA5EF"/>
</linearGradient>
<linearGradient id="drive-copper" x1="32" y1="32" x2="232" y2="224" gradientUnits="userSpaceOnUse">
<stop stop-color="#703E30"/><stop offset=".16" stop-color="#B47456"/>
<stop offset=".28" stop-color="#E6B296"/><stop offset=".32" stop-color="#F8D8C3"/>
<stop offset=".37" stop-color="#BB7C5A"/><stop offset=".56" stop-color="#8A4D36"/>
<stop offset=".69" stop-color="#D69B77"/><stop offset=".75" stop-color="#F2C9AA"/>
<stop offset=".81" stop-color="#B87956"/><stop offset="1" stop-color="#713F30"/>
</linearGradient>
<linearGradient id="drive-sun" x1="172" y1="82" x2="201" y2="112" gradientUnits="userSpaceOnUse">
<stop stop-color="#8D513A"/><stop offset=".33" stop-color="#DDA581"/>
<stop offset=".47" stop-color="#F7D8B8"/><stop offset=".53" stop-color="#D69B74"/>
<stop offset="1" stop-color="#925439"/>
</linearGradient>
<linearGradient id="drive-mountain" x1="44" y1="148" x2="186" y2="218" gradientUnits="userSpaceOnUse">
<stop stop-color="#D9C9FA"/><stop offset="1" stop-color="#A582E9"/>
</linearGradient>
<linearGradient id="drive-far-mountain" x1="142" y1="152" x2="211" y2="216" gradientUnits="userSpaceOnUse">
<stop stop-color="#E8B4EB"/><stop offset="1" stop-color="#BB7BD7"/>
</linearGradient>
<clipPath id="drive-photo-clip"><rect x="28" y="54" width="194" height="172" rx="24"/></clipPath>
</defs>
<!-- Canonical vector artwork; regenerate platform assets with scripts/render-branding.py. -->
<g id="mark">
<path d="M48 48H91Q99 48 105 54L117 66Q123 72 132 72H208Q228 72 228 92V200Q228 220 208 220H48Q28 220 28 200V68Q28 48 48 48Z" fill="url(#drive-copper)" transform="translate(6 -6)" stroke="#7D4936" stroke-width=".8" stroke-linejoin="round"/>
<path d="M48 48H91Q99 48 105 54L117 66Q123 72 132 72H208Q228 72 228 92V200Q228 220 208 220H48Q28 220 28 200V68Q28 48 48 48Z" fill="url(#drive-rear)" stroke="#5E3CBD" stroke-width=".8"/>
<rect x="72" y="82" width="132" height="112" rx="10" fill="url(#drive-paper)" stroke="#EFE9FC" stroke-width="1"/>
<rect x="51" y="97" width="140" height="109" rx="10" fill="url(#drive-paper)" stroke="#EFE9FC" stroke-width="1"/>
<path d="M48 94H89Q97 94 103 100L115 112Q121 118 130 118H208Q228 118 228 138V206Q228 226 208 226H48Q28 226 28 206V114Q28 94 48 94Z" fill="url(#drive-copper)" transform="translate(6 -6)" stroke="#7D4936" stroke-width=".8" stroke-linejoin="round"/>
<path d="M48 94H89Q97 94 103 100L115 112Q121 118 130 118H208Q228 118 228 138V206Q228 226 208 226H48Q28 226 28 206V114Q28 94 48 94Z" fill="url(#drive-face)" stroke="#5936B7" stroke-width=".9"/>
<path d="M32 113Q32 98 48 98H89Q95 98 100 103L112 115Q120 122 130 122H208Q224 122 224 138" fill="none" stroke="#F3DDF9" stroke-opacity=".52" stroke-width="1.5" stroke-linecap="round"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 1.8 KiB

After

Width:  |  Height:  |  Size: 3.6 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 MiB

After

Width:  |  Height:  |  Size: 1.6 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 283 KiB

After

Width:  |  Height:  |  Size: 702 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 96 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 97 KiB

+10 -3
View File
@@ -9,14 +9,14 @@
# Build + install: cd packaging && makepkg -fi
pkgname=proton-drive-linux
pkgver=3.8.0
pkgver=3.9.0
pkgrel=1
pkgdesc="Unofficial Proton Drive client for Linux: files-on-demand FUSE mount, CLI, GTK4 app + tray"
arch=('x86_64')
url="https://proton-drive.narl.io"
license=('MIT')
depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'perl-image-exiftool')
makedepends=('cargo' 'git' 'gettext')
depends=('fuse3' 'polkit' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'perl-image-exiftool')
makedepends=('cargo' 'git' 'gettext' 'curl')
provides=('pdfs')
# Files-on-demand mount + keyring need these at runtime.
optdepends=('gnome-keyring: credential storage via Secret Service'
@@ -77,6 +77,13 @@ package() {
install -Dm644 "packaging/proton-drive.service" \
"$pkgdir/usr/lib/systemd/user/proton-drive.service"
install -Dm644 "packaging/io.narl.proton-drive-linux.policy" \
"$pkgdir/usr/share/polkit-1/actions/io.narl.proton-drive-linux.policy"
# The offline Places basemap, fetched and checked against its pinned
# SHA-256 by scripts/fetch-basemap.sh.
scripts/fetch-basemap.sh "$pkgdir/usr/share/proton-drive-linux/basemap-v1.pmtiles"
# Translations: compiled from po/ into /usr/share/locale.
po/build.sh "$pkgdir/usr/share/locale"
@@ -1,12 +1,13 @@
pkgbase = proton-drive-for-linux-bin
pkgdesc = Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries)
pkgver = 3.7.0
pkgver = 3.8.0
pkgrel = 1
url = https://proton-drive.narl.io
install = proton-drive.install
arch = x86_64
license = MIT
depends = fuse3
depends = polkit
depends = gtk4
depends = libadwaita
depends = webkitgtk-6.0
@@ -22,15 +23,15 @@ pkgbase = proton-drive-for-linux-bin
optdepends = xdg-utils: "Open folder" action in the tray menu
optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf)
optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding
provides = proton-drive-for-linux=3.7.0
provides = proton-drive-linux=3.7.0
provides = proton-drive-for-linux=3.8.0
provides = proton-drive-linux=3.8.0
provides = pdfs
conflicts = proton-drive-linux
conflicts = proton-drive-for-linux
conflicts = proton-drive-for-linux-git
options = !strip
options = !debug
source = proton-drive-linux-3.7.0-x86_64.tar.gz::https://github.com/narrrl/proton-drive-linux/releases/download/v3.7.0/proton-drive-linux-3.7.0-x86_64.tar.gz
sha256sums = SKIP
source = proton-drive-linux-3.8.0-x86_64.tar.gz::https://github.com/narrrl/proton-drive-linux/releases/download/v3.8.0/proton-drive-linux-3.8.0-x86_64.tar.gz
sha256sums = 99348fbf9121624701ae43e52776e5d1c469ff5cd5e52cd216bf8cb969e0bab2
pkgname = proton-drive-for-linux-bin
@@ -1,14 +1,14 @@
# Maintainer: Nils Pukropp <contact@narl.io>
pkgname=proton-drive-for-linux-bin
pkgver=3.7.0
pkgver=3.8.0
pkgrel=1
pkgdesc="Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries)"
arch=('x86_64')
url="https://proton-drive.narl.io"
_repo="https://github.com/narrrl/proton-drive-linux"
license=('MIT')
depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc'
depends=('fuse3' 'polkit' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc'
'hicolor-icon-theme')
optdepends=('perl-image-exiftool: thumbnails for camera RAW files'
'ffmpeg: video thumbnails'
@@ -29,7 +29,7 @@ _tag="v$pkgver"
# (desktop files, icon, user service, translations, completions) in a /usr-like
# tree under one directory.
source=("proton-drive-linux-$pkgver-x86_64.tar.gz::$_repo/releases/download/$_tag/proton-drive-linux-$pkgver-x86_64.tar.gz")
sha256sums=('SKIP')
sha256sums=('99348fbf9121624701ae43e52776e5d1c469ff5cd5e52cd216bf8cb969e0bab2')
package() {
cd "$srcdir/proton-drive-linux-$pkgver-x86_64"
@@ -49,6 +49,10 @@ package() {
install -Dm644 lib/systemd/user/proton-drive.service \
"$pkgdir/usr/lib/systemd/user/proton-drive.service"
# The offline Places basemap.
install -Dm644 share/proton-drive-linux/basemap-v1.pmtiles \
"$pkgdir/usr/share/proton-drive-linux/basemap-v1.pmtiles"
# Translations and shell completion for bash, zsh and fish.
install -dm755 "$pkgdir/usr/share"
cp -r --no-preserve=ownership share/locale share/bash-completion share/zsh share/fish \
Loaded 100 of 124 files, more files were not shown because too many files have changed in this diff. Show more