Compare commits

...
Author SHA1 Message Date
nvrl ed3477d1b3 docs: point contributors at the simulation runs and the ledger rules
DEVELOPMENT.md lists the daemon's current modules, asks for the
simulation runs on FUSE, queue and sync changes, and adds how to handle
a bug fix and a user-visible change, including the six pages the wiki
publisher copies. The release checklist moves to TESTING.md.
2026-10-06 19:29:02 +02:00
nvrl aa384ae2eb chore: cite docs/BUGS.md and drop references to notes outside the repo
66 comments cited `bugs.md B<n>`, the ledger's old name; they now say
`docs/BUGS.md B<n>`, and every cited number has an entry. About 90 more
cited offline.md, devices.md, features.md, mount-architecture.md,
plan.md and improvements.md, planning notes that were never in the
repository. Those citations are removed; the comments keep their text.
2026-10-06 19:29:02 +02:00
nvrl 10c69b0d64 test(acceptance): explain the metadata floor for local-first mounts
The throughput case said every create and unlink on a Drive mount waits
for Drive. Since 3.0.0 that holds only with local_first off; the floor
of 3 operations a second stays for that case.
2026-10-06 19:29:02 +02:00
nvrl 39be354bd2 docs(bugs): give the ledger an index and one order
The ledger ran B193 down to B78, then B1 up to B75, with T1 to T4 in
the middle and a stray top-level heading inside B14. Status lines used a
dozen wordings for the same few states, so finding what is still open
meant reading 6,600 lines.

Add a header: what the numbers promise (permanent, never reused; B76
and B77 were never assigned), the status vocabulary, and how to add an
entry, with a template. Add an index sorted by status, with what is left
for every open or partly fixed entry. Order all entries newest first
under one Entries section, and move the acceptance-runner findings to
their own section. Titles are in sentence case; anchors are unchanged.

Entry text is kept as written, except B89: the file it described,
proton-dev/CLAUDE.md, no longer exists, so it is closed.
2026-10-06 19:29:02 +02:00
nvrl 2cca8afba9 docs: describe local-first changes and sync issues for users
The user guide said only writes were queued. Since 3.0.0 creating,
renaming and deleting through the mount return at once too, and changes
Drive refuses become sync issues. Add a section on changes waiting for
Proton Drive: retry, export, discard, and the two cases that land under
another name or place instead of failing.

Troubleshooting covers changes that have not reached other devices yet,
`pdfs sync issues` and the local_first fallback, and names the real
error for symlinks and hard links (EPERM, not EOPNOTSUPP). CONFIGURATION
lists MemoryHigh and the test variables; CLI notes that `sync export`
takes a folder.
2026-10-06 19:14:29 +02:00
nvrl 6e9beb4b16 docs: rework contributor docs for the 3.0.0 local-first mount
ARCHITECTURE.md described the 2.x write path, where namespace changes
waited for Drive and every mount had its own inode space. Rewrite it
around the local tree (lid, inode = lid + 1, local~<lid> stand-ins), the
queue and its 16 drain workers, sync issues, and the remaining split
between the mount and the mirror engine. Drop the pins.json claim; pins
moved into the database at schema v5.

TESTING.md gains the simulation runs and loses the per-case narrative
that --list and BUGS.md already carry. ROADMAP.md takes over the
post-3.0.0 items from the milestone status file.

Remove MILESTONE-3.0.0-STATUS.md (a working log) and UI_UX_PLAN.md (the
2.x design audit). Code comments and CI that cited the untracked
docs/MILESTONE-3.0.0.md now point at TESTING.md, ARCHITECTURE.md and
ROADMAP.md. Threat-model links follow the section to §10.
2026-10-06 19:14:29 +02:00
nvrl 63484bf4b6 chore(images): add cover image
1920×1080 cover for link previews, social posts and the GitHub social preview.
The website renders its og.png from cover.svg.
2026-10-06 14:46:56 +02:00
nvrl 37c66b7220 chore(support) addd buy me a coffee badge + link 2026-10-06 10:49:24 +02:00
Nils Pukropp 41b6edbecb chore(aur): update packages to 3.0.0 2026-10-05 05:12:08 +00:00
nvrl 217738318e chore(release): 3.0.0 2026-10-05 05:16:57 +02:00
nvrl 9291580fd6 test(fuse): drop the known-bug matchers the runs confirmed fixed
The confirmation run on 812eb3d (PDFS_SIM_KNOWN=fail, twelve seeds)
passed every profile, so no open bug is tolerated any more: the call,
hang and damage matchers for B125 to B134 go, and any of them now fails
the seed. B126, B128 and B133 were fixed with B138, B162 and B137;
B134 stays open with local_first off only, which the runs do not mount.
2026-10-05 05:15:19 +02:00
nvrl 812eb3d0bc fix(fuse): a deleted file's trash leaves a file of the user's under the name it was sent as (B193) 2026-10-05 04:50:17 +02:00
nvrl 31863a1da1 fix(fuse): a new file closed empty reaches Drive at once (B192) 2026-10-05 04:50:10 +02:00
nvrl 57a825887c fix(core): a file deleted after its failed create was written again leaves Drive (B191) 2026-10-05 04:50:02 +02:00
nvrl 1436c5f37e fix(fuse): a stop queues the writes whose release the kernel never sent (B190) 2026-10-05 03:55:52 +02:00
nvrl f0a43fb923 fix(fuse): changes that failed on a short outage go out once the link is back (B189) 2026-10-05 03:48:13 +02:00
nvrl b47620651f fix(fuse): a restart no longer brings back the write of a file deleted while open (B188) 2026-10-05 03:37:41 +02:00
nvrl 2baad6720d fix(core): leave no temp copy when the cache adopts a blob it already holds (B187) 2026-10-05 02:52:51 +02:00
nvrl c271a1897c fix(fuse): keep what an open file reads when its delete is queued (B186) 2026-10-05 02:51:55 +02:00
nvrl d26dc8ada4 fix(fuse,core): retry a create at once when a write replaced its blob (B185) 2026-10-05 02:34:05 +02:00
nvrl 93f02672e6 docs: describe 3.0.0's local-speed writes and leave batched trashes and moves for later 2026-10-05 02:02:34 +02:00
nvrl 307305282e fix(fuse,core): what a create sent, and where, outlives a restart and a rename (B182, B183, B184)
The known-bug confirmation run failed one_client_on_a_flaky_link seeds 6
and 11 on three bugs around a create whose answer was lost:

- B182: after a restart, a write over a file whose create landed but was
  never read back forked a conflict copy. A revision of a file that has
  no revision of Drive's now adopts the landed node first.
- B183: the SHA-1s a create sent were kept in memory, so after a restart
  its trash or retry no longer knew the node Drive made. They are now on
  the op's row (pending_op.sent_sha1, schema 42).
- B184: a rename rewrote the op's target, and the trash and the retry
  looked for the node under the last name only. Each attempt's parent
  and name are recorded (pending_op.sent_to); the trash looks under all
  of them, and a retry adopts the node an earlier attempt made and moves
  it to the current name.
2026-10-05 02:01:50 +02:00
nvrl a8ed6060af feat(core,fuse,gui): show a sync issue on its node in the file browser
A refused change was only on the Sync page. A folder listing now carries
each node's issue (by local id, so a file not yet on Drive has it too),
and the browser's badge ranks it first, folders included, with the
issue's explanation as its tooltip. The event sampler publishes Files
when the set of issues changes, so the badge comes and goes with it.
2026-10-05 01:06:03 +02:00
nvrl 9dfe4c5450 feat(core,fuse,cli,gui): discard a queued change and undo it here
A change Drive will not take could be exported or retried, never let go
of. The Sync page's Discard button (with a confirmation) and
`pdfs sync discard <id>` drop it: a node made here goes, as if deleted
before it was sent; a node Drive has goes back to what Drive has, under
the same inode, with every change queued on it, while what is queued
below it stays. A change on the wire or an open file is refused.
2026-10-05 00:57:02 +02:00
nvrl da013c29d6 fix(gui): word a queued edit kept as a conflict copy in the user's language (B181)
The daemon logs it as "<reason>; local changes uploaded as <copy>" or
"...kept at <path>", which the Activity page showed as written, in
English. It now reads both and the three reasons, one sentence each.
2026-10-05 00:17:11 +02:00
nvrl 457a9d26f4 ci: keep the stacks of a hung simulation run
CI on f9b1638 failed slow-link seed 2 on a step that hung for 150 s, and
the stacks that would say where were left on the runner.
2026-10-05 00:13:42 +02:00
nvrl a68dae9422 fix(fuse): a kernel notice sent from the runtime hands its worker's tasks on (B130) 2026-10-05 00:02:56 +02:00
nvrl e743cd60cf fix(fuse): a folder whose mkdir is queued is listed from the database (B125) 2026-10-05 00:02:40 +02:00
nvrl f9b1638603 docs: record the 903452a account run and CI in the 3.0.0 status 2026-10-04 23:48:58 +02:00
nvrl 6539c7314b test(acceptance): a queue wait after a restart waits for the daemon to answer
A mirror target's root is plain local storage, so the durability case's
wait for the mount passed at once after the restart. The queue wait then
read `pdfs --json status` before the new daemon listened, took its null
"mount" for an empty queue, and the run's cleanup found no control socket.
2026-10-04 23:47:57 +02:00
nvrl 903452a15b fix(fuse): a stop wakes every drain worker (B180) 2026-10-04 23:00:07 +02:00
nvrl 4e0fd3c0b1 fix(fuse): a file read just as its create lands has its bytes (B179) 2026-10-04 22:52:00 +02:00
nvrl a6f019ebe6 fix(fuse): a folder trashed while sync is paused drops the creates in it (B178) 2026-10-04 22:51:47 +02:00
nvrl 0f3c06f982 fix(core): a folder listed just as its create lands is listed from the database (B177) 2026-10-04 22:41:57 +02:00
nvrl 4f141c78ca fix(fuse): a write closed just as its file's create lands is queued (B176) 2026-10-04 22:31:49 +02:00
nvrl ce60e006ac fix(fuse): a file replaced just as its create lands is trashed on Drive (B175) 2026-10-04 22:24:28 +02:00
nvrl 1612a9d07a fix(core): a file renamed just as its create lands keeps its access (B174) 2026-10-04 22:15:01 +02:00
nvrl 378667fb37 fix(fuse): a file deleted just as its create lands is trashed on Drive (B173) 2026-10-04 21:51:52 +02:00
nvrl 3b81cc5240 test(fuse): a simulated daemon does not ask the real API for its share id 2026-10-04 21:51:36 +02:00
nvrl d850265225 fix(fuse): a file renamed while its upload is read back keeps its new name (B172) 2026-10-04 21:34:20 +02:00
nvrl e32f72bfca fix(core,fuse): a deleted file's trash is not held back by its create (B171)
A file deleted after its create was sent keeps the op as a trash. The op
kept the create's attempts and backoff, and the drain recorded the failure
of a create attempt still on the wire on the row that was by then the trash.
The trash waited out the create's backoff.

The trash starts due, as a create that becomes a revision does, and the
drain records a failed attempt only if the op is still the kind it ran as
(`Db::record_attempt_failure`).
2026-10-04 21:23:10 +02:00
nvrl 2502af6e3c fix(fuse): a write opened as an upload lands takes the landed base (B170)
An upload that landed let go of its blob before the tree had its node. A
write open in between took the base it sampled at its start, the revision
the upload replaced, found none of its bytes cached, and queued an
incomplete revision; the next write open failed with EIO.

The drain keeps the landing node aside until the tree has it, and a write
open with no queued revision takes its base from that node, or from the
tree as it is when the handle goes in. The fake Drive can hold the drain
in that window (`hold_next_landing`).
2026-10-04 21:14:09 +02:00
nvrl 4757b957d2 fix(fuse): end a mirror pass at the stop (B169)
A stop that came while a mirror pass was uploading let the pass go on to
the next level of folders until the stop gave up on it and the runtime's
end cancelled the uploads under it. Once the daemon is stopping, a batch
starts none of its waiting ops, lets the ones sending finish, and ends
the pass; the remote walk and a new pass stop too.
2026-10-04 21:01:38 +02:00
nvrl 8e54ed0517 fix(fuse): do not send a file removed with its folder again (B168)
A delete of a file whose create was on the wire withdraws the create into
a trash. When its folder was removed too and that trash landed first,
Drive refused the create, and the drain re-homed the deleted file in the
root; only its missing bytes stopped it. The withdrawn create's trash
then read the trashed folder back into the cache before finding nothing
to trash.

A withdrawn create is no longer re-homed, and its trash is done once the
folder it was sent to is trashed or gone.
2026-10-04 20:42:04 +02:00
nvrl 9bbc994a4f perf(fuse): rebuild the hydrated listings in one pass (B167)
The hydrate rebuilt each listed folder's listing with a pass over every
entry. With 4,600 listed folders and 37,000 entries that took 1.7 s at
every start before the mount came up; one pass grouping the entries by
parent takes 3 ms.
2026-10-04 20:27:58 +02:00
nvrl ffa0fe698d test(fuse): a listing from before a create commits keeps the landed file (B166)
The account run's 0-byte read of a new file was B166 too: Drive's
listing had the node from before its create was committed, and the
queue no longer held it when read. The fake Drive can now hold a
folder's listing on its way back with one file listed unrevised.
2026-10-04 20:17:49 +02:00
nvrl 25c46cfe0a fix(fuse): list a file whose create lands while its folder is listed (B166)
A listing from Drive was overlaid with the queue as it was after Drive
answered. A create that landed in between was in neither, and a wide
folder on Wi-Fi dropped files that were never unlinked. The queue is now
read before Drive is asked too; a node that left it meanwhile is listed
as the DB has it.
2026-10-04 20:10:14 +02:00
nvrl 573a29e596 fix(fuse,cli): bound the stop's wait for busy threads (B165)
A stop joined every background thread for as long as each took. The sync
engine's mirror pass calls Drive with no deadline, and a stop that waited for
it was killed by systemd at TimeoutStopSec. The stop now waits up to 10 s for
the background threads and FUSE workers together, names any still running, and
the runtime shuts down within 2 s.
2026-10-04 19:12:19 +02:00
nvrl ce2bc0ac84 fix(fuse): list a new file once while its create lands (B164)
A listing from Drive overlays what is still queued for the folder. A create
Drive has made but whose answer is on its way, or which the drain is still
reading back, was in both: Drive's node under its new uid and the queue's
stand-in, under one name. A node queued in the folder now holds its name there.
2026-10-04 19:03:33 +02:00
nvrl b37d5e7928 fix(fuse): do not take a new file Drive lists before its commit as empty (B163) 2026-10-04 18:51:39 +02:00
nvrl 18587124ac fix(fuse): never adopt a node we trashed as an unanswered create's twin (B162)
A folder removed and made again met the old folder while its trash was on
the wire and adopted it, so the new folder's contents went into the old
one and were trashed with it. The twin search now skips hidden nodes, as
listings do.

The fake Drive can hold a trash (hold_next_trash).
2026-10-04 18:26:45 +02:00
nvrl cd18029944 fix(fuse): keep a move that lands while its create is read back (B161)
adopt_real_uid asked whether a move was queued only after reading the
node back. A move that landed in between left no op, and the answer still
had the old name, so the mount showed the file under it. Ask first.

The fake Drive can hold a read's answer (hold_answer_to_next_read) so the
test can land the move while the answer is on its way.
2026-10-04 18:20:56 +02:00
nvrl d6dd41407f fix(fuse): wake the drain when a folder is made (B160)
Queueing a folder's create left the workers asleep until their 30 s idle
poll; every other enqueue woke them. A file's create still waits for the
release that attaches its bytes.
2026-10-04 18:14:00 +02:00
nvrl 60445d3b26 fix(fuse): list again when a listing changed while it was published (B159)
A lookup in the root publishes the "Shared with me" entry only if the root
listing did not change while its plan was read, and answered EAGAIN when it
had. Right after a restart the root is listed again as the first lookups come
in, so a write could fail with "Resource temporarily unavailable".
ensure_children now takes the listing again, up to four times.
2026-10-04 17:50:39 +02:00
nvrl c2f93449ad fix(core): drop a rename replaced while its failure was being recorded (B158)
B141 drops a rename that fails with a newer one queued behind it. The
worker records the failure and releases its claim in two steps, and a
rename queued between them waited behind the old one, which was kept
with a backoff. Released, the old one was not due and the new one was,
so the new one landed first and the old one then put the first name
back.

Releasing the claim now drops the old rename too when a newer one is
queued, and both checks match the node by local id as well as by uid.
2026-10-04 17:23:51 +02:00
nvrl 4676a4e32b fix(fuse): cache an upload whose read-back failed, and know it as ours (B157)
When the read-back after an upload went unanswered, the drain let go of
the staged blob without caching it. A partial write opened next had
nothing to fill its gaps from and queued an incomplete revision, so the
write open after it was refused with EIO.

The bytes are now cached under the size and mtime the tree still has,
which is the base the next write takes. Their SHA-1 stands in for the
revision id the read-back did not bring, so that write chains onto the
upload and does not fork a conflict copy of itself.
2026-10-04 16:20:54 +02:00
nvrl 16fca10e6c perf(fuse): drain with 16 workers, and wake the idle ones when an op lands 2026-10-04 15:05:17 +02:00
nvrl dbc4d9a8c5 fix(core,fuse): a node written under its stand-in after landing updates its landed row 2026-10-04 13:38:54 +02:00
nvrl 9c93860cd8 refactor(core): rows and ops name a node's parent by local id; a landing rewrites nothing below it 2026-10-04 13:26:19 +02:00
nvrl 85ec91d6b3 fix(fuse): adopt a twin holding bytes an earlier attempt of the create sent (B156) 2026-10-04 13:14:29 +02:00
nvrl d5d3b2da3f refactor(core,fuse): find a queued op by its node's local id under either uid 2026-10-04 13:10:19 +02:00
nvrl e5f3f612f8 refactor(core,fuse): a node made here goes by its local id until it lands (schema 41)
A file or folder made on the mount used to get a minted `local~<ms>-<n>`
placeholder. It now gets `local~<lid>`, a function of the row it is given
first (`Db::add_local_row`), so the stand-in never changes and always leads
back to the row, and through it to the real uid once the node lands.
`Db::lid_of` reads it without a lookup.

Migration V41 rewrites the placeholders still stored to their row's
stand-in: the row and its node_json, its children, the ops on it or into
it (uid, parent, sidecar uid, a rename's original parent), and its pins.

Step 6a of the plan for 3.0.0 phase 3.
2026-10-04 12:32:20 +02:00
nvrl 3f74b95b34 fix(fuse): wait for a create on the wire that holds the name (B154); a write whose answer was lost is not a conflict (B155)
B154 is a regression from the B153 fix: after a failure on a held name the
op woke the ops waiting for it, but the database cannot see a create on the
wire, so the op was due again at once and spent its waits in milliseconds.
No wake while another create on the wire holds the name.

B155: a revision upload that reached Drive but lost its answer left the
file at a revision this daemon never heard of, and the retry kept the write
as a conflict copy of itself. A complete write whose bytes the remote holds,
by size and SHA-1, is sent again instead.

The fake Drive gains hold_reply_to_next_create and
lose_reply_to_next_revision for the two new sim tests.
2026-10-04 12:19:06 +02:00
nvrl 7269cb5118 fix(fuse): keep a rename made as its create lands (B152); wake an op once the name it waits for is free (B153)
Adoption asks again under the tree lock whether a rename is queued, and
writes the node row through the lock's outbox, so a rename made while
the create landed is no longer undone in the tree and the row.

An op that finds its name held fails as NameHeld: it backs off by the
minimum without doubling, and wakes the waiters for its name again once
its failure is recorded, so a trash that landed in between is not missed.
2026-10-04 06:04:20 +02:00
nvrl 6d75a56b00 fix(fuse,core): adopt a twin holding the create's bytes (B127); trash the node of a withdrawn create that lost its answer (B151)
A retried create now adopts the file its lost answer made when Drive's
SHA-1 matches the staged blob, not only when the file is empty. A delete
after a sent create keeps the op as a trash that finds and trashes the
node that create may have made; a new create of that name waits for it.
The simulation's settle now reads the log, so known damage is explained.
2026-10-04 05:41:57 +02:00
nvrl eb1c105bbf docs: record the full a6f36d5 account run in the 3.0.0 status 2026-10-04 04:45:48 +02:00
nvrl 85e80d8aac test(acceptance): retry only the sources a busy mirror refused; a mirror-to-mirror move carries an unsynced copy along 2026-10-04 04:45:48 +02:00
nvrl 23dc5c3723 fix(fuse): land a conflict copy in the root when its folder is gone (B150) 2026-10-04 04:45:48 +02:00
nvrl 50ae25bb88 fix(fuse): queue pdfs rename and move behind the mount's changes (B149)
With changes through the mount queued, `pdfs rename` and `pdfs move` went
straight to Drive ahead of a rename the mount still had queued, which then
landed and put its name back: the account run's B121 case on every on-demand
mount. They are now queued whenever the mount would queue, or the node has
anything queued, and rewrite a still-queued create's target.

The B113 acceptance case no longer requires creates to reach Drive at once;
under local-first they are queued and the moves rewrite them.
2026-10-04 04:45:48 +02:00
nvrl 9b2b9f24c5 feat(core): node rows name their parent by local id
Schema 40 adds nodes.parent_lid, backfilled from parent_uid. An upsert
and a new row's stub keep it in step, a folder's new row links the rows
stored before it, and a row merged into a landing folder hands its
children over. A restart places each node under its parent's row as it
is now, so a child whose stored node still names a landed folder's
placeholder lands under the folder's real uid.
2026-10-04 04:45:47 +02:00
nvrl ae6650f7e0 feat(core,fuse): queued ops name their node and folder by local id
Schema 39 adds pending_op.lid and parent_lid, backfilled from nodes. Every
op queued or retargeted gets both, and a create gets its row before it
is queued. The claim reads a parent's uid through parent_lid, and a
landing folder rewrites the ops made inside it by local id instead of by
its placeholder string. Ops queued against a row Drive listed follow it
when it is merged into the landing one.
2026-10-04 04:45:47 +02:00
nvrl f029e652a1 fix(core): keep a folder's pin when the folder lands (B148)
A folder pinned before its mkdir landed kept the placeholder uid in
pins.uid, which named no node once the landing moved the row to the
real uid. The landing now moves the pin with it.
2026-10-04 04:45:47 +02:00
nvrl 9b3651e4e8 fix(fuse): put a deadline on every remote call the drain makes
create_folder, trash_nodes, the twin listing and the uploads waited on
Drive without a limit, so one call that never answered held a drain
thread for good. They now give up after INTERACTIVE_CALL_TIMEOUT, and
uploads after that plus their size at 32 KiB/s; a missed deadline reads
as RequestTimeout, so the op backs off like any lost link.
2026-10-04 04:45:47 +02:00
nvrl 0dc476dfb7 test(sim): a steady Wi-Fi run and a drain-speed measurement
one_client_on_a_slow_link runs the Wi-Fi faults without outages and has
to drain within each settle. a_thousand_files_drain_as_fast_on_wifi_as_on_lan
times 100 folders and 1,000 files on LAN and Wi-Fi; it only prints for
now and runs when PDFS_SIM_MEASURE is set.
2026-10-04 04:45:47 +02:00
nvrl de258cbd60 ci(aur): retry the AUR host key scan
ssh-keyscan sometimes returns no key from the runner, which failed the
AUR publish for 2.8.0 and 2.10.0 at the fingerprint check. Retry the
scan up to five times with a longer timeout before checking the pin.
2026-10-04 03:43:30 +02:00
Nils Pukropp 4ea18e9187 chore(aur): update packages to 2.10.0 2026-10-04 01:42:24 +00:00
nvrl a6f36d56f3 feat(core,fuse,cli,gui): report changes Drive refuses as sync issues
A change Drive refuses because the storage is full, access was taken away,
its folder is gone, a folder limit is reached or the change was rejected
used to look like any other failure until six attempts had passed. The
drain now sorts the refusal into an issue and stores it on the queued op
(schema 38), so it needs attention at once in the app, tray and CLI. The
change stays queued and is retried; nothing is deleted.

The queue listing carries the issue and whether the op has file content.
ExportPendingOp copies that content to a new file: the Sync page offers it
as an Export button next to each issue, and the CLI as
`pdfs sync export <id> <dest>`. `pdfs sync issues` lists what needs
attention, and `pdfs sync queue` shows the issue per change.

docs/MILESTONE-3.0.0-STATUS.md tracks what is left for 3.0.0.
2026-10-04 03:18:57 +02:00
nvrl 77d0f254fc fix(fuse): never adopt a node another queued create is making
A file renamed away from a.txt while its create was on the wire, and a new
a.txt made after it, could both land on the first file's node: the second
create found a fresh, empty a.txt and took it for its own unanswered create.
The first file's rename then carried the second one's bytes away.

A create now leaves alone a node that another create is making under the
same name, and one the tree already knows. It waits for the name like for
any other queued change (docs/BUGS.md B147).
2026-10-04 03:15:50 +02:00
nvrl 4eccdeeda6 test(acceptance): retry a move while a mirror folder is busy syncing 2026-10-04 02:49:33 +02:00
nvrl 473842eefc fix(fuse): keep parked creates, renames and freed names from racing the drain
- B144: when Drive says a name is taken but nothing local holds it any more,
  the rename, move or create is sent once more under the wanted name before
  falling back to a conflict name. The trash that freed it could land between
  Drive's answer and the check.
- B145: the park sweep no longer drops a parked create whose node row is not
  written yet. It waits a minute, and keeps open files, before taking a
  missing row for a deleted file. The account run's B70 check failed with
  EBUSY on this.
- B146: a rename queued while an upload's read-back is on the wire is kept;
  the tree no longer takes back the name Drive still had.
2026-10-04 02:49:33 +02:00
nvrl 5e0f30cc2c feat(fuse,core): record namespace changes locally and send them from the queue
A new folder, a new file, a rename or a delete through the mount no longer
waits for Drive: it is recorded locally, the call returns at once, and the
drain sends it as it does offline. `"local_first": false` in config.json
brings back the old path for this release.

Fixes found by the simulation runs on the way:
- B129: a file deleted while its first upload is on the wire is trashed once
  the upload lands.
- B132, B140: files made in a new folder survive a restart; a landed folder
  hands its queued children over in the stored node too.
- B138: a create waits for the rename or trash that frees its name.
- B139: a file closed while its first upload lands keeps its last write.
- B141: a rename a worker holds stays queued, and the newer one waits behind
  it instead of racing it.
- B142: a rename of a file whose create lands meanwhile goes to the uid Drive
  gave it instead of answering EBUSY.
- B143: a create whose read-back fails as the link drops is adopted before
  the next op for it is sent, so its next write is not a conflict copy.
2026-10-04 01:50:38 +02:00
nvrl b3e33a7002 fix(core): never give out a pending_op id twice
A plain rowid is the largest one plus one, so a trash queued in the same
transaction that dropped the newest op took its id. The drain retires the
op it sent by id, so a rename landing deleted the trash queued over it and
the file stayed on Drive (B137). Schema 37 rebuilds pending_op with
AUTOINCREMENT, keeping every id and index.
2026-10-03 23:44:12 +02:00
nvrl f6456829fb chore(release): 2.10.0 2026-10-03 21:57:23 +02:00
nvrl 327f538fed test(core): migrate a database 2.8.2 wrote, with ops in every state
A dump of a database the 2.8.2 build wrote, with queued ops due, parked,
failing, deferred for access and claimed, and creates queued below a
folder made offline. Its migration checks every column of the queue and
the sealed revisions, as the real-database test now does too.
2026-10-03 21:55:15 +02:00
nvrl ad8b225cfd refactor(fuse): one tree for every mount
On-demand sync folders kept their own State, which the drain, the
event applier and the sweeps had to visit one by one (B74). Every mount
now serves from the primary's tree. A session keeps only its root, which
the kernel knows as inode 1, and its notification channel; work that
changes the tree tells every mounted session. Size upgrades become
daemon-wide, and moves compare mounts by root instead of by state.
2026-10-03 16:53:31 +02:00
nvrl ea27f2bcd0 test(core): migrate a copy of a real database
An ignored test takes a copy of the database named by PDFS_MIGRATE_DB
with VACUUM INTO over a read-only connection, migrates it, and checks
that every queued op survives unchanged, every node keeps its row
number as its lid, and no node is lost or doubled.
2026-10-03 16:53:31 +02:00
nvrl e4062cc97b feat(fuse,core): number inodes by local id
A node's inode is now derived from its row's local id instead of a
per-run counter, so the same node keeps the same inode across restarts
and across every place that interns it. A lid handed out before the
row is stored reads as no node.
2026-10-03 16:41:26 +02:00
nvrl c34cc93c45 fix(fuse,core): keep a rename made while its create is on the wire
A rename of a node whose create is still queued rewrites the create's
target, but the drain had already copied the op. When the create
landed, the row was deleted unread and the rename was lost (B136).

finish_create now learns where the create was sent and where it landed,
and queues a rename of the real uid when the row has moved on since.
adopt_real_uid keeps the tree's name and folder while that rename is
queued. A landed folder's children also stop naming its placeholder as
their parent in memory, which put the placeholder back on their rows.
2026-10-03 16:20:00 +02:00
nvrl 7e3df7602d feat(core): give every node row a local id that never changes
First step of phase 2 of docs/MILESTONE-3.0.0.md. Schema 36 rebuilds
`nodes` with `lid INTEGER PRIMARY KEY AUTOINCREMENT`. Each row takes its
old rowid as its lid, so the search index, keyed by rowid, stays valid.
AUTOINCREMENT keeps a deleted row's lid from coming back. `uid` stays
unique and becomes nullable, for nodes that are not on Drive yet.

A landed create's row now takes the real uid in place, keeping its lid.
When a listing had already stored the real node, its content is kept
under the placeholder's lid, which the node was known by first.

Nothing reads the lid yet apart from `Db::lid_of`.
2026-10-03 15:10:51 +02:00
nvrl 61a0d31408 chore(release): 2.9.0 2026-10-03 15:00:38 +02:00
nvrl 644a47eb36 fix(ci): quote the simulation command so the workflow parses
The ': ' in 'sim:: --' read as a YAML mapping, so GitHub rejected
ci.yml and ran no job at all since 2c46d62.
2026-10-03 14:55:13 +02:00
nvrl cd90c566cd fix(fuse,core): renames across folders, replaced files, and writes after a create
Found by the seeded simulation runs (docs/BUGS.md).

- B122: a rename across folders goes to Drive as one move-multiple
  request, so a sibling holding the new name in the old folder no longer
  fails it with EIO. The drain uses the same request.
- B123: a second write open on a file just written no longer fails with
  EIO. An empty file counts as complete, our own renames, moves and
  online creates record their echo, a duplicated event is skipped, a
  rename keeps the file's mtime while its revision stays, and a landed
  upload is cached before open handles are rebased onto it.
- B124: a node put back after a failed replacing rename leaves `hidden`
  and its folder is listed again, so it shows in the mount.
- B131: a landed create is read back with the same short retry as an
  online create, and its bytes are cached under the real uid before an
  open handle is repointed at it.
- B135: an online rename settles the tree through the state guard, so
  the database has both folders unlisted before the reply. A lookup
  right after it no longer rebuilds the destination without the file.

The simulation runs no longer tolerate B122 or B131.
2026-10-03 14:50:12 +02:00
nvrl 1c262eb601 refactor(fuse,core): keep queued-op preconditions in the database
Phase 1 of docs/MILESTONE-3.0.0.md, without a change in behaviour.

- Own sealed revisions live only in the own_sealed_rev table. The
  in-memory map in front of it is gone, so revision_conflict gives the
  same answer before and after a restart.
- A queued rename records the node's original name next to its original
  parent, and keeps both across renames that replace the row. The drain
  logs when Drive has the node neither where the rename found it nor
  where it takes it.

The SDK's NodeUpdated event carries no revision id, so echoes of our own
changes stay recognised by the self-change timer.
2026-10-03 14:49:58 +02:00
nvrl bf73a395e5 docs: record B132-B134 from the simulation runs
- B132: a folder whose mkdir is still queued is gone after a restart.
  The root's listing, served offline, was invalidated when the link
  came back, and the restart listed it on Drive without the queued
  folder.
- B133: a file deleted while its upload is in flight can still come
  back as a recovered copy. B105's check goes by op id, and SQLite gives
  the next op queued the id the delete freed.
- B134: a file deleted online while the drain checks its write is kept
  as a conflict copy. The unlink trashes the file on Drive before it
  drops the queued write, so B105's check still finds the op.
2026-10-03 09:07:56 +02:00
nvrl a5138639a8 test(fuse): delete files as they upload in the simulation runs
The runs did not find B105 with its fix reverted. Drive lists a trash
later than the round trips take, so the drain never saw its own
client's delete while it still held the write.

The FakeDrive can now hold the next read of one node on its way to
Drive. A new profile, one-client-deletes, writes a file Drive holds,
waits for the drain to read the node back, deletes the file, and lets
the read go on once Drive shows the trash. A conflict the drain keeps
because a client's own file was trashed now fails the run as a false
conflict. With B105's fix reverted, seeds 1 and 2 fail on it.

Outside the step, such a conflict is counted as B134 when the copy is
named after the file: the drain looked after an online unlink had
trashed the file on Drive but before it dropped the write. Seed 3 of
three-clients came up with it once.

Half the time the step makes the file again before the read goes on.
That is B133: the new file's write takes the op id the delete freed,
and B105's check finds it. The runs count it.

The runs also count B132, an ENOENT after a restart that restored
queued ops, in a seed that lost the link. Seed 4 of one-client-flaky
failed on it. The known-bug matchers now also see what was logged
before the call.
2026-10-03 09:07:56 +02:00
nvrl 988b78e810 docs: record the bugs the simulation runs found (B122-B131)
The first simulation runs found ten open bugs. They are documented here
and not fixed yet, as phase 0 of docs/MILESTONE-3.0.0.md changes no
behaviour. The runs count each one and end the seed where the mount
leaves the model behind.

- B122: a rename across folders fails with EIO when the source folder
  holds the new name.
- B123: a second write open on a file just written fails with EIO.
- B124: after a replacing rename fails, the file it was replacing is gone
  from the mount.
- B125: a folder made offline fails with EIO once its listing is
  invalidated.
- B126: a file created offline under a name an offline rename freed
  lands as a conflict copy.
- B127: a file created offline lands twice when the answer to its upload
  is lost.
- B128: a file created offline over one deleted offline can be lost.
- B129: a file deleted or replaced while its create is on the wire stays
  on Drive.
- B130: a read of a file that a remote change invalidates can hang the
  daemon. The event sync blocks a runtime worker in inval_inode on the
  read's page lock. Nothing then drives the runtime's timers, which the
  read's fetch is waiting for.
- B131: a write to a file whose create could not be read back lands as a
  conflict copy. The create's op is retired before the read-back, so a
  failed read-back leaves the tree on the placeholder's mtime and no
  revision id.
2026-10-03 07:47:02 +02:00
nvrl 2c46d62429 ci: run the simulation runs outside pull requests
A `simulation` job runs the seeded simulation runs of
docs/MILESTONE-3.0.0.md section 8.2 on pushes to main and manual runs. They
mount FUSE against the in-memory FakeDrive, so they need fuse3 but no
account. Like the acceptance suite they stay off pull requests. The job
has a 30 minute limit, and the runs' own watchdog turns a hang into a
failed seed well inside it.
2026-10-03 07:47:02 +02:00
nvrl 97743f17e3 test(fuse): add seeded simulation runs over the FakeDrive
The second half of the simulation harness in docs/MILESTONE-3.0.0.md
section 8.2. A run starts one or more real FUSE daemons on a shared
FakeDrive. Each client owns a folder `c<n>` that only it writes. A seed
then draws syscalls through the mounts, link flaps, restarts, pauses and
settles. Every syscall is checked against a model of the client's tree as
it happens. After the last step every link comes up and the run checks
no loss, POSIX behaviour, no false conflicts, convergence and liveness.

- sim::model is the POSIX model: the tree, what each call should answer,
  and how it changes the tree.
- sim::run has the profiles: one client on a good link, one on a flaky
  link with flaps and restarts, and three clients. PDFS_SIM_SEEDS sets
  how many seeds each runs, and PDFS_SIM_SEED replays one.
- sim::watchdog catches a call that never returns. It takes every
  thread's stack in the process, since no tracer can stop a thread
  blocked on a FUSE request. It then aborts the run's FUSE connections
  and writes the stacks next to the run's state.
- The daemons' logs at INFO and above are kept, so a known open bug is
  told apart from a new failure. Known bugs are counted, and the seed ends
  where the mount leaves the model behind. PDFS_SIM_KNOWN=fail fails on
  them instead.
- A restart retries opening the database while the last run still holds
  it.
- The FakeDrive restamps a node's modification time on a rename or move,
  as Drive does (B113).

The runs mount FUSE, so they are ignored by default. Run them with
`cargo test -p pdfs-fuse --lib sim:: -- --ignored --test-threads=1`.
Test-only; no behaviour change.
2026-10-03 07:47:01 +02:00
nvrl 3d10338807 test(fuse): run real FUSE daemons against the FakeDrive
mount() becomes a thin wrapper over mount_with(), which takes the
DriveApi to reach Drive through and a Host. The daemon's Host is
unchanged: stopped by SIGTERM or SIGINT, installed at the same point as
before, and indexing the user's home. A simulated daemon is stopped over
a channel and skips the home index, so several can share one test
process without walking the developer's home.

The account quota moves onto DriveApi. It was the only call outside the
trait that marks the link lost, so a simulated daemon would have gone
offline on it; the FakeDrive now also models a full account
(InsufficientQuota).

sim::daemon starts one daemon on its own state directory with a real
FUSE mount, and a smoke test (ignored; needs /dev/fuse) checks a local
write lands, a remote write arrives, and a write made offline lands once
the link is back.
2026-10-03 05:09:02 +02:00
nvrl d53ba1be21 test(fuse): add an in-memory FakeDrive with Drive's known faults
The first half of the simulation harness in docs/MILESTONE-3.0.0.md
section 8.1. FakeDrive is one shared server; each simulated daemon reaches
it through a FakeClient that implements DriveApi and carries its own link
state, latency, faults and an SDK-style entity cache.

The faults are the ones the account runs met: listings that lag (B118,
B48), renames and moves refused for a stale name hash because the cache
is only emptied by events (B111, B121), names that refuse a replace (B13,
B104, B106), events delayed, duplicated, reordered and echoed (B70, B105),
requests that stall, fail, or change Drive and lose their reply (B107,
B121), and non-uniform block sizes (B84, B85, B87).

Test-only; no behaviour change.
2026-10-03 04:51:35 +02:00
nvrl c73d5c9436 refactor(fuse): reach Drive through a DriveApi trait
`Core` held a concrete `ProtonDriveClient`, so no test could make Drive
answer late, answer stale or not answer. This is the first step of
phase 0 in docs/MILESTONE-3.0.0.md.

`DriveApi` (`crates/pdfs-fuse/src/drive.rs`) covers the calls the sync
core makes: listings, events, creates, uploads, downloads, renames,
moves, trashes and restores. `ProtonDriveClient` implements it by
forwarding. `Core::drive` holds it, and every call of those methods now
goes through it. Sharing, Photos, devices, invitations and public links
stay on `Core::client`.

The trait takes readers and writers as trait objects, and
`impl dyn DriveApi` puts the client's generic signatures back, so call
sites read the same. Block reads go through `RevisionRead` instead of
the SDK's `RevisionReader`, which only the SDK can construct.

No behaviour change.
2026-10-03 04:37:37 +02:00
nvrl 8d92c69279 fix(fuse): keep an upload pending until its bytes are cached (B119)
The drain retired an upload's op first, and only then read the node back
and linked the staged blob into the content cache. For that one round
trip, the status reported an empty queue for a file that wasn't cached
yet, so anything that waited for an empty queue before reading the file
could still need the network. The account run of 2026-10-03 hit it over
a slow link: "regression B119" failed in the same second the upload
landed.

`Core::landing_uploads` now counts an upload from just before its op is
retired until its bytes are cached. The status adds it to
`pending_uploads`, and reads the queue before the count, so it always
sees the upload in one of them. A drop guard uncounts it on every exit
path.

The op still goes first. An op restored after a crash would re-send a
revision this daemon hasn't recorded as its own yet, and
`revision_conflict` would keep it as a conflict copy of the user's file.
2026-10-03 03:59:05 +02:00
nvrl e32959e40a fix(fuse): land a move right after a rename, and rename across folders quickly (B121)
The SDK caches each node, including the name hash it sends with the next
rename or move, and only an event evicts it. Renames and moves didn't
update that copy. So after a rename, the next change to the same node
went out with the old name hash, Drive refused it as out of date (422),
and the retry kept sending the stale hash until the event feed came
round, up to 10 seconds later.

- Every remote rename and move goes through `Core::rename_remote` or
  `Core::move_remote`. Both drop the SDK's copy of the node after a
  successful change and before each out-of-date retry, so the retry
  sends the current hash.
- A plain move was not retried at all, so a move right after a rename
  failed with EIO through the mount and with "out of date" from
  `pdfs move`. It now retries like a rename.
- The drain's combined move-and-rename also drops the SDK's copy once
  it lands.
- A rename that also changes folders through the mount replies only
  once both halves have landed. When the move fails, the file keeps
  its new name in the source folder, the tree is told so, and the
  rename reports EIO. A dropped link still queues the change.

B111's cause in docs/BUGS.md is corrected to the same SDK cache. The
acceptance case "regression B121" renames and moves a file at once
through the mount and through `pdfs`, and checks names and bytes.
2026-10-03 03:58:38 +02:00
nvrl 0b27c884a7 test(acceptance): turn B114 to B117 into regressions and ride out a dropped link
New live cases:

- regression B48, B80, B94, B95, B98, B102 and B111 replay the bugs
  their entries describe. B48 and B111 also run against the reference.
- regression B114 to B117 replace the open B114 and B115 cases, and
  fail with `check` when the bug comes back.
- regression B119: files written through the mount stay cached once
  they are uploaded.
- regression B86: after a mirror folder switches to on-demand, the
  mount lists what the mirror uploaded just before.

`--journal-check` also counts three warnings as errors: a conflict
copy (B113), a FUSE job held for a long time (B110) and an authority
re-read because the local tree lacked it (B117, B120).

A timeout no longer ends a target when the mount still answers a stat
and a listing. Cases that make many round trips get twice the limit.

A write is queued when the kernel releases the file, after close(2)
has returned, so cases now wait for the ops they expect
(`Daemon.queued`). "regression B102" saw one revision of two because
it read the queue once.

On flaky Wi-Fi, the run crashed twice on a read that the daemon
correctly failed with EIO once its read budget ran out. The end-of-run
read-back and the mode matrix's file checks now record a result
instead. The mode matrix tries such a read once more after the daemon
is back online, and notes it.
2026-10-03 02:10:49 +02:00
nvrl 37c04f1e69 fix(fuse,core): fix B114 to B120 from the account runs
- B114: `pdfs rm`, `pdfs rename` and `pdfs move` tell each kernel
  session that the old name is gone, so it stops resolving at once
  instead of for up to 30 seconds (`Core::forget_everywhere`,
  `Core::relocate_everywhere`).
- B115: `pdfs ls` and the app's file browser wait up to 10 seconds for
  the listing's size fetch, as `stat` does, so they show a file's real
  size instead of its encrypted one.
- B116: trashing a folder also drops the creates queued in its
  subfolders. A parked one stayed queued with its bytes, and a due one
  would have landed in the root.
- B117: `pdfs move` and `pdfs rename` move the node in each mount
  instead of forgetting it, so its queued upload keeps its path in
  `pdfs sync queue` and is not put off once after the resume.
- B118: when a create or mkdir reads back nothing, the mount asks again
  and then queues it as it does offline, instead of failing with ENOENT
  for something Drive has made.
- B119: a landed upload keeps its staged bytes as the file's cached
  content, unless it is larger than a sixteenth of the cache, so
  reading a file just written does not need the network.
- B120: a landed folder's row passes to its real uid in the same
  transaction, so a file queued inside it is not put off for five
  seconds.

docs/BUGS.md records B117 to B120. The account run of 2026-10-03
verified B114 to B117 and B119 in My files.
2026-10-03 02:10:23 +02:00
nvrl 625776081b test(acceptance): cover B113, B88 and the closed pipe, and report more readably
New live cases:

- regression B113: files renamed and moved while sync is paused upload
  without a conflict copy.
- regression B88: pdfs mkdir, rename and rm by local path, the
  mount-root guard, a path outside every mount, and the refusal in a
  mirrored folder.
- pdfs output into a closed pipe exits 0 and prints nothing on stderr.
- open B114 and open B115 check two bugs that are still open. While
  one reproduces, its case ends as "known", which is listed but does
  not fail the run.

The console output names the pdfs version, the storage and the daemon
state of each target, and prints one aligned line per case, with its
details beneath. A failed case points to its docs/BUGS.md entry. The
run ends with a summary table, the slowest cases, the known issues
and the numbered failures. A queue wait now fails at once on a parked
op instead of waiting out its timeout.

docs/BUGS.md records B114, B115 and B116, which the first live run
found, and marks B113 and B88 as verified live.
2026-10-02 22:16:36 +02:00
nvrl 8cde79b8e3 fix(cli): stop quietly when stdout is closed
`pdfs ls | head` ended in a "Broken pipe" panic once `head` had its
lines, because std's `println!` panics on a failed write.

The CLI now shadows `println!` and `print!` with versions that exit 0 on
a closed pipe and report any other write error, such as a full disk,
with exit 1. Restoring the default SIGPIPE would also stop the panic,
but the same binary runs the daemon, which must not die when a socket's
peer goes away.
2026-10-02 21:16:34 +02:00
nvrl eb53d122cc fix(fuse,cli): let rm, rename and mkdir take paths in on-demand folders
`pdfs rm ~/Downloads/x` answered "is not under the mountpoint" while
`pdfs move` took the same path: the Delete, Rename and CreateFolder
handlers still read paths against the primary mount only (B88).

They now go through `route_change`, the router ResolveConflict already
used, renamed from `route_conflict`. It answers the request in the
inode space of the mount that owns the path.

An on-demand folder's root has a parent on Drive, so `pdfs rm
~/Downloads` would have trashed the whole synced folder from under its
mount. rm and rename refuse a path that names a mount's root.
2026-10-02 19:54:37 +02:00
nvrl c64252193f fix(fuse): keep the revision id of a file created through the mount
The write handle a create opens was given no revision id, so the drain's
conflict check fell back to (mtime, size). Drive advances a node's mtime
on a move or rename, so a new file moved before its first upload drained
landed as a (sync-conflict) copy next to its own empty original. Take the
minted node's revision id, and rebase handles still open on a queued
create onto the node it landed as (B113).
2026-10-02 19:25:33 +02:00
nvrl cc70a21c05 fix(aur): version -bin source filenames
makepkg reuses a cached source file of the same name, so the unversioned
proton-drive.service from an older build failed the 2.8.2 checksum. Carry
$pkgver in every file fetched from the tagged tree, like LICENSE already
did, and bump pkgrel to 2.
2026-10-01 23:50:06 +02:00
Nils Pukropp 644995cc56 chore(aur): update packages to 2.8.2 2026-10-01 19:49:42 +00:00
nvrl 50483bf3e1 chore(release): 2.8.2 2026-10-01 21:39:05 +02:00
nvrl 4caba0a34c chore(deps): require proton SDK 0.7.2
0.7.2 resolves each shared root with its own key. With 0.7.1, when
several accepted shares came from the same owner's volume, only the first
appeared under Shared with me and the rest were skipped with "missing
key" (B112, proton-sdk-rs#2).
2026-10-01 18:17:25 +02:00
Nils Pukropp e8554d6434 chore(aur): update packages to 2.8.1 2026-10-01 14:58:09 +00:00
Claude ba1fdd65be chore(release): 2.8.1 2026-09-30 17:02:31 +00:00
nvrl 0d4163b9bf fix(service): start with the graphical session, not default.target (#24)
default.target is reached before the desktop exports DISPLAY/WAYLAND_DISPLAY
to the user manager, so After=graphical-session.target never applied. The
daemon's first keyring call D-Bus-activated the Secret Service without a
display; on KDE ksecretd aborted, the first start failed, and every login
left a coredump.

Install into graphical-session.target instead, and use reenable when the
front-ends enable the unit so existing default.target.wants links are
replaced rather than kept alongside the new one.

Fixes #23
2026-09-30 19:01:50 +02:00
Nils Pukropp f3c7a809f4 chore(aur): update packages to 2.8.0 2026-09-30 02:15:19 +00:00
nvrl c1b5378045 fix(aur): download sources from GitHub, not the homepage
73e60f1 pointed url at proton-drive.narl.io, and all three AUR PKGBUILDs
built their source URLs from $url, so the release tarball, the binary
archive and the git clone were fetched from the homepage and 404ed. The
sources now come from a separate _repo on GitHub; url stays the homepage.
2026-09-30 04:13:06 +02:00
nvrl 3d6f53c16d chore(release): 2.8.0 2026-09-30 03:56:47 +02:00
nvrl 8ceee5285b chore(fuse): log how long each queued upload takes
"pending upload landed" now carries upload_ms, the time on the wire, and
total_ms, the whole op including the conflict check and gap-fill. A slow
drain then shows where its time goes without a profiler.
2026-09-30 03:55:48 +02:00
nvrl a181f1a5f0 fix(core,fuse): keep working through network drops
The mount only ever went from offline to online: nothing marked it
offline again, so after the Wi-Fi dropped every create, mkdir, delete and
rename called Drive, waited out the SDK's retries for up to two minutes
and answered EIO. Background loops kept polling the dead link.

Connectivity now lives in pdfs-fuse's new link module and goes both
ways. A call that gets no answer (a transport error or a request
timeout) marks the mount offline; a probe thread checks every 2 s,
backing off to 30 s, and on recovery re-lists folders served from the
database, retries failed operations and wakes the drain. Interactive
calls get a 20 s deadline, so a dead link costs one slow call instead of
many.

While offline:
- create, mkdir, unlink, rmdir and rename fall back to the offline
  queue instead of answering EIO;
- folders list from the database even when not marked listed, and are
  re-listed once the link returns;
- reads of uncached data wait up to 30 s for the link, then retry once
  within the 120 s read deadline;
- statfs serves the last quota instead of fetching, and quota refreshes
  are single-flight;
- the event and photo event loops wait for the link instead of polling.

A create whose request timed out may have landed anyway. When the drain
replays it and finds a same-named twin of the same kind, created within
the last two minutes and still empty, it adopts that node instead of
making a conflict copy. The drain also leaves alone a name still held
by a queued trash.
2026-09-30 03:25:00 +02:00
nvrl 7251f00e91 chore(deps): require proton SDK 0.7.1
0.7.1 sends TCP keepalives on its connections, so a link that dies
silently is noticed instead of leaving a request waiting on a dead
socket until its timeout.
2026-09-30 03:25:00 +02:00
nvrl cb38f79aa3 fix(fuse,scripts): log why a create finds no parent and fit the metadata floor to FUSE
A create that answered ENOENT left no trace in the log. It now warns
when the parent folder is no longer known, and when the parent left the
tree while its children were being listed, naming the parent inode and
the new name.

The acceptance script's metadata throughput floor was 20 operations per
second, which a FUSE mount cannot reach: each create and unlink waits for
Drive before it returns, about 350 ms apiece. On a FUSE mount the floor
is now 3 operations per second; other paths keep 20. The mount type is
read from /proc/self/mountinfo.
2026-09-29 21:22:54 +02:00
nvrl 54e9921efb fix(fuse): stop reads deadlocking on size upgrades and retry stale renames
A read that settles a provisional size invalidated the file's page cache
while the kernel held that read's page lock, so the worker waited on
itself forever (B110). Size upgrades now invalidate attributes only.

Renaming a node right after renaming it could be refused by Drive as out
of date, because the name hash read just before still held the old name.
Every remote rename now goes through Core::rename_remote, which retries
InvalidRequirements for up to ten seconds (B111).
2026-09-29 19:47:20 +02:00
nvrl f252b09757 fix(fuse): keep the size of a file open for writing across re-listings
B109: an online create seals an empty file on Drive, and until the last
close the written bytes live only in the open handle. A re-listing of
the parent, a size upgrade or a create landing re-interned the node with
Drive's empty size, the kernel capped reads at that size, and SQLite
reported "database disk image is malformed". State::keep_open_write_size
now carries the handle's size and local mtime over in all three places.

The acceptance script waits for My files to mount after a daemon restart
instead of failing at once.
2026-09-29 16:49:13 +02:00
nvrl a2992b3b9d fix(core,fuse): keep replaced and vanished files from failing saves, reads and syncs
Fixes from the account acceptance runs:

- B104: a reader open across a rename-over keeps working; the replaced
  node is unlinked like any other open file instead of forgotten.
- B105: a drain worker holding a revision withdrawn by an unlink no
  longer uploads it as a recovered-… conflict copy (Db::op_exists).
- B106: an open victim's queued ops are withdrawn at trash time, so a
  repeated editor save no longer fails with AlreadyExists; the staged
  blob goes with the last close.
- B107: a remote read gives up after 120 s with EIO instead of hanging
  its caller; the timer is built inside the runtime.
- B108: a mirror upload whose local file was deleted after the scan is
  skipped rather than counted as an error.

The acceptance script reports the failing line for non-assert errors,
ignores queue ops that predate the run, purges with delete-forever
first, and diagnoses a malformed SQLite database against a local replay.
2026-09-29 16:20:51 +02:00
nvrl 7b67db5ced fix(core,fuse,cli): harden writes, trash and cleanup found by account acceptance runs
The FUSE acceptance suite now runs on the signed-in account by default and
cleans up after itself; its first live runs found three daemon bugs.

- B101: a pwrite past EOF now authors the hole it leaves, so truncated bytes
  no longer come back from the base. A write attached to a create while that
  create uploads becomes a revision of the real uid instead of being dropped.
- B102: removing a folder drops the queued revisions of files below it, found
  through the node tree, with their blobs and in-flight uploads. `pdfs rm`
  now drops queued ops too.
- B103: each server call of a trash refresh times out after 120 s, so one hung
  call can no longer freeze the trash listing until restart.

Acceptance suite: account mode with a manifest, lock and reaper; cleanup by
uid; queued ops naming a test root count as leftovers; new edge-case,
throughput and B100/B101 cases; the wide-directory case is split from the
deep tree and gets a doubled budget. Adds `pdfs trash --json`.
2026-09-29 10:09:44 +02:00
nvrl 84c9ade8e3 fix(fuse): settle provisional sizes before reads and write opens
A file listed by the cheap enumeration reports its ciphertext size until
the size upgrade lands. Since B84 a read on that size fails with EIO, and
a write open started its handle at it. Reads and write opens now fetch
the real size first. A folder past the size-upgrade cap is queued instead
of dropped, so a large copy no longer leaves most folders provisional
(B100).
2026-09-29 07:37:41 +02:00
nvrl 9e26cd0285 chore(release): 2.7.0 2026-09-29 05:30:38 +02:00
nvrl 73e60f1151 chore: point homepage metadata at proton-drive.narl.io
The About dialog's website link, the README, and the package metadata
(Cargo, PKGBUILDs, AUR .SRCINFO, .deb, .rpm) now use the project
homepage. Source code, issue, and release links stay on GitHub.
2026-09-29 05:30:38 +02:00
nvrl 088e8f3b0d test(acceptance): cover moves between locations in the managed matrix
Every managed mode pair now also moves trees between the two folders in both
directions, moves several sources in one command, and passes a tree through
My files and out again. Each case waits for Drive before and after, then
checks the tree is only at the destination, byte for byte, with no conflict
copies. Refusal cases cover a taken name, a folder into itself, a mirror
source holding a symlink, and an on-demand file still open for writing.
2026-09-29 00:38:40 +02:00
nvrl 261d5dbf84 feat(core,fuse,cli): reuse cached content and take several sources in moves
- A mirror download first copies the file out of the content cache when the
  same revision is cached whole or block by block, so moving a file out of an
  on-demand folder into a mirror costs no traffic for what was already read.
- `pdfs move a b c DEST` moves every source, reports each failure and exits
  non-zero if any failed.
- Mirror-to-mirror moves rename with RENAME_NOREPLACE, so a name that
  appeared at the destination after the check is never overwritten.
- The local copy a move out of a mirror drops is checked again just before
  it is deleted; anything written into it meanwhile keeps the copy and is
  uploaded as new by the next pass.
2026-09-29 00:38:40 +02:00
nvrl 5f9d07aff1 feat(fuse,core,cli,gui): move between locations without transferring content
A move request with two absolute paths is resolved against every
location (My files, on-demand and mirror synced folders) and made as a
server-side move_node. The local side is levelled afterwards: mounts
forget and re-list, a mirror folder renames its copy and baseline along
or drops the copy once Drive holds it.

Anything only this disk holds is refused up front: queued ops, open
writes, unsynced or ignored mirror files and symlinks. The GUI Move
dialog gains a top-level list of locations.
2026-09-29 00:26:56 +02:00
nvrl 3fb7b9be6b chore(release): 2.6.2 2026-09-28 23:58:26 +02:00
nvrl 6c3a50ce7f fix(gui): let the Sync page use the full width
The transfer label did not ellipsize, so a long file name set the
column's minimum width and pushed it past the clamp. The page width
changed with every file.

The label now ellipsizes in the middle, and the Sync page drops the
clamp so Overview and Folders fill the space next to the sidebar.
2026-09-28 23:58:26 +02:00
nvrl f8304a0e86 fix(gui): keep Albums and Places selected when returning to Photos
Every visit to the Photos page loaded the timeline, and a finished
timeline load always shows the timeline. Leaving Albums or Places and
coming back therefore showed the timeline under the Albums or Places
toggle.

A visit now reloads the view the toggle selects. An open album or place
still loads its photos as before.
2026-09-28 23:58:26 +02:00
Nils Pukropp e63f39f8c2 chore(aur): update packages to 2.6.1 2026-09-28 21:44:18 +00:00
nvrl 6f100dbe5e chore(images) updated prompt image 2026-09-28 23:35:53 +02:00
nvrl 7c5ef269cd chore(release): 2.6.1 2026-09-28 23:35:14 +02:00
nvrl f13b6fc0ef feat(gui): add pdfs-prompt --preload for session autostart
--preload registers the resident GTK launcher and realizes its window
without showing it, so the first shortcut opens it as fast as later
ones. When a launcher is already running it exits without activating
it.

The application hold now happens in startup, which only the primary
instance runs. Taken before registration, it also kept every remote
invocation alive after it had forwarded its activation.
2026-09-28 23:35:14 +02:00
nvrl 3d61fc637b ci(release): reuse cached builds from main
The release job never hit a cache: its rust-cache key contained the job
name, tags cannot restore caches of earlier tags, and rpmbuild rebuilt
the whole workspace under its own compiler flags.

CI on main now saves shared debug and release caches that the release
workflow restores. Quality gates and the package build run in parallel,
and the GitHub release is created once both pass. The RPM spec skips
its cargo build when the caller passes --define "prebuilt 1".
2026-09-28 23:26:24 +02:00
Nils Pukropp 889b2aa4a9 chore(aur): update packages to 2.6.0 2026-09-28 21:22:30 +00:00
nvrl 8bd3867cb3 fix(gui): keep the launcher preview at its fixed width
A wrapping label asks for its whole text on one line as its natural
width, so a long location in the preview pane widened it, squeezed the
results list and stretched the window. Cap the preview labels' natural
width, limit detail values to three lines with a tooltip for the full
location, and keep the pane from taking extra width.
2026-09-28 23:04:48 +02:00
nvrl 9577387c2f chore(release): 2.6.0 2026-09-28 22:57:28 +02:00
nvrl 7a087b8827 feat(gui): rewrite the search launcher with a preview pane
The launcher (pdfs-prompt) now follows the app's theme: the theme code
moved into a shared theme.rs, the prompt's stylesheet lives in
resources/prompt.css on the libadwaita palette, the Proton-purple accent
pin is gone, and every activation re-reads the Proton theme and open_with
settings so changes in the app apply without restarting the launcher.

The window is rebuilt in src/hud/:
- one list with Recent, Pinned and Best matches sections; a click
  selects, a double-click or Enter opens, focus stays in the entry
- rows show a thumbnail, the name with its matched characters in bold,
  location, sync badge, size and age
- a preview pane with a large thumbnail, type, size, modified date,
  location and availability, plus Open (Enter), Show in Folder
  (Ctrl+Enter, via FileManager1) and Copy Path (Ctrl+C / Ctrl+Shift+C)
- thumbnails from the freedesktop cache, local decoding (written back
  to the shared cache) or the daemon, on two worker threads
- a history of opened files that fills the Recent section and adds a
  frecency bonus to search ranking
- the mount check before opening a Drive file runs off the main thread
  with a 2 s timeout, and Escape cancels a slow open
- rows can be dragged out as files
- the search debounce drops to 80 ms

New strings are translated in every catalog.
2026-09-28 22:53:46 +02:00
nvrl 7099106802 docs: show the Places map screenshot
Rename photos_mpa.png to photos_map.png and add it to the README
screenshot table.
2026-09-28 22:10:14 +02:00
nvrl 43385d36ee chore(images) added up to date images 2026-09-28 22:09:07 +02:00
nvrl 4b97f401cb docs: refresh screenshots
Replace the Photos screenshot with one of placeholder stock photos, add
Sync, conflict history and Computers screenshots, and drop the retired
Activity, Settings, Shared and restore-devices images from the README.
2026-09-28 22:08:16 +02:00
nvrl af869944cf docs: restructure and update documentation for 2.5.0
Split the README into a short front page and task-oriented pages under
docs/: installation, user guide, CLI reference, configuration and
troubleshooting, with an index in docs/README.md. Rewrite RECOVERY.md
around Continue This Backup Here and Restore Folders, and DEVELOPMENT.md
around running, debugging and releasing from source.

Bring ARCHITECTURE.md up to date with the read path, push events, the
thread map, conflict copies and the machine profile, and move the feature
design notes there. Collect planned work in ROADMAP.md. Link the threat
model from SECURITY.md and the translation rules from CONTRIBUTING.md.

Add scripts/publish-wiki.sh, which mirrors the user pages to the GitHub
wiki with rewritten links and only pushes with --push.
2026-09-28 20:04:22 +02:00
Nils Pukropp 4b90abf3b5 chore(aur): update packages to 2.5.0 2026-09-28 16:56:58 +00:00
nvrl 61266fc9b1 chore(release): 2.5.0 2026-09-28 16:37:23 +02:00
nvrl d18655538e feat(core,fuse,gui): push daemon events, polish Photos, adopt newer widgets
The app and the tray stop polling. pdfs-core adds Request::Subscribe: the
connection stays open and carries Event lines (status, transfers, and
Changed with a Topic), with a heartbeat while quiet. In pdfs-fuse, an
EventHub gives each subscriber a mailbox, code that changes something
publishes its topic, and a sampler publishes the status, transfers and sync
progress when they differ. The app follows the feed on one thread, paints
status and transfers as they arrive, and reloads the page on screen when its
topic changes, including after changes made with the pdfs CLI. A hidden
window keeps the changes and catches up when focused. Front-ends fall back
to polling a daemon that predates subscriptions. Refresh buttons leave the
page headers; F5 and Ctrl+R still reload from the server.

Photos gets a scrubber overlaid on the right edge of the timeline in place
of the slider and scrollbar. It shows the years, appears when the pointer
comes near or the timeline scrolls, names the month under the pointer and
jumps to it on drag. Upload becomes a split button whose menu holds the
Google Photos import. Tiles are pooled per row and rebound instead of
rebuilt, hover scales the thumbnail to 1.02 without a shadow, and tiles
lose their tooltip.

On libadwaita 1.8 or newer the app uses AdwSpinner, AdwToggleGroup,
AdwButtonRow and AdwShortcutsDialog, and keeps look-alike stand-ins on
older releases (src/compat.rs, widgets/compat.rs). It still builds against
libadwaita 1.5 and picks at run time, creating the newer widgets by class
name through GtkBuilder, so every package gets them. PDFS_STAND_IN_WIDGETS
forces the stand-ins.

The Sync and share-link icons draw in full again: GTK drops group
transforms from symbolic icons, so their coordinates are baked into the
paths. All catalogs are updated and complete.
2026-09-28 16:37:23 +02:00
nvrl da7ea0ec56 feat(core,gui): split Sync into views, tidy Activity, rework Computers
The Sync page gets Overview, Folders and History views under a switcher in
the header. Overview holds the status card, conflicts, transfers and the
queue; Folders holds the folder list, with Add Folder and a popover that
explains the sync modes; History is the former Activity page, which leaves
the sidebar. A banner in Folders and History points back to Overview when
changes fail or conflict.

Activity hides save-by-rename temporaries (pdfs-core gains
syncignore::is_scratch_name), with a toggle to show them again. Runs of
similar events fold into one expandable row, "Show Older" pages further back
through what the daemon keeps, and titles and details are translated
sentences built from the daemon's targets instead of its raw text.
Conflict copies say when they were made.

Computers lists every device in one group, this computer first with a
"This computer" pill; its row opens Sync's folder list. Another computer's
row browses its backup read-only in the shared file list, and its menu
offers Browse Files, Restore, Rename, "Continue This Backup Here…" (the old
identity adoption, now explained in its dialog) and Remove. Subtitles read
"last backup" or "no backups yet", and an empty list links to the download
page. DirEntry derives Default for the browse view's synthesized entries.

The sidebar shows badges for sync problems and waiting invitations, polled
every five minutes away from their pages. All catalogs are updated and
complete.
2026-09-28 15:14:59 +02:00
nvrl 127b71efd0 feat(core,fuse,gui): rebuild Trash and shared pages on one file list
The list and grid view from My files moves into widgets/file_list.rs as
FileList, so Trash, Shared with me and Shared by me share its columns,
selection, context menus, keyboard handling and list/grid switch instead of
each drawing its own rows.

Trash shows where each item came from and when it was deleted. Proton does
not expose the trash time through the SDK yet, so the daemon records when it
first sees a node in the trash (migration V35, trash_seen) and reports that;
the date becomes exact once proton-sdk-rs exposes Node::trash_time. Restoring
offers a Show action, and opening a trashed item explains that it needs to be
restored first.

Shared with me splits files and saved links into two views. Files show who
shared them, with a warning pill for unverified senders, the access they give
and when they were shared. Pending invitations sit in a banner that opens a
dialog to accept or decline each one. Several items can be left at once.
Saved links get their own list with open, copy and remove actions.

Shared by me gains a filter (all, links, people, expired links) and columns
for location, people and link state, with a copy button for live links.
Several items can stop being shared at once, the page refreshes after the
share dialog closes, and a reload rebinds only the rows that changed so the
selection survives.

All catalogs are updated and complete.
2026-09-28 14:35:37 +02:00
nvrl 2339b17d97 feat(core,gui): keep pages up while they reload
Pages no longer swap their content for a "Loading…" status on every load.
A new Loader keeps the last rows on screen and only shows the loading
state if the reply takes longer than 300 ms; a newer load always
supersedes an older one, so a late reply is dropped instead of painted.
Reloading what is on screen keeps it usable; loading something else greys
the old rows out and clears the selection, since their paths would
resolve against the new folder.

My files, search results and Trash now swap only the rows that changed,
so selection, scroll position and thumbnails survive a refresh. Activity
is one sectioned ListView: open conflicts first, then a section per day,
with the filter bar pinned above the list.

Polls give up after 5 s instead of 120 s. A daemon that is running but
doesn't answer now reads as "Not responding" in the app and the tray
rather than as disconnected.
2026-09-28 13:51:28 +02:00
nvrl bd4f797d1f feat(gui): polish pass over wording, icons and responsiveness
The Sync page gives each folder one card with its state, a Synced /
Online only switch and a menu for the rest; switching either way asks
first and the download prompt shows the free space on the disk.

Enabling, disabling and restarting the Proton Drive service ran
systemctl on GTK's main thread and froze the window. They now run in
the background, and every Retry button goes through one shared restart
helper.

Ten icons are bundled under resources/icons for glyphs no stock theme
has or that read ambiguously (sync, share, people, link, menu, map and
so on). A new test fails when the app or search prompt asks for a
symbolic icon that is neither bundled nor on a list checked against
Adwaita 50, libadwaita and GTK, or when a bundled icon is missing from
the resource manifest. The tray's sync icon, which Adwaita 50 no longer
ships, is now view-refresh.

Wording follows one glossary, recorded in docs/TRANSLATING.md: "Proton
Drive service" instead of "mount service" or "daemon", "Online only"
instead of "On-demand", "Proton Drive folder" instead of "mountpoint".
Dialog headings and buttons use title case and page titles match the
sidebar. Dates go through one formatter (app/dates.rs) and sizes
through i18n::human_bytes, and the cache label is translated.

The status tick stops while the window is hidden or minimized and runs
every fifth tick while it is unfocused, refreshing as soon as it gains
focus. "Sync complete" is announced only for large batches and only
when the window isn't focused. Smaller fixes: Empty Trash hides when
there is nothing to empty, copy buttons confirm with a check mark, the
sidebar status spins until the first report, and Photos moves Import
and Find Duplicates into a menu beside a labelled Select toggle.

All catalogs are updated and complete.
2026-09-28 13:26:59 +02:00
nvrl e872cad058 ci: move to Node 24 versions of checkout, upload-artifact and action-gh-release
actions/checkout@v4, actions/upload-artifact@v4 and
softprops/action-gh-release@v2 still target Node.js 20, which the runners
now force onto Node.js 24 with a deprecation warning. Their current majors
(checkout v7, upload-artifact v7, action-gh-release v3) run on Node.js 24.
2026-09-28 12:16:54 +02:00
Nils Pukropp 081c8fb889 chore(aur): update packages to 2.4.0 2026-09-28 07:33:52 +00:00
nvrl da35a05b84 chore(release): 2.4.0 2026-09-28 09:16:54 +02:00
nvrl a90eeecda9 feat(core,fuse,gui): find photos that look alike
The duplicate finder gains an Exact / Similar switch. Similar asks the
daemon for PhotoSimilar, which starts a background pass that hashes the
thumbnail of every shot not hashed yet and replies at once with the sets
found so far and how far hashing has got. The dialog shows a progress bar
and asks again until the pass is done, then lists the sets.

The pass takes thumbnails from the server only: a photo without one is
skipped for the run instead of downloaded whole, and the dialog says how
many photos could not be compared. Decoding and hashing are spread over
the machine's cores.

Every similar photo starts ticked Keep, since photos that look alike can
be different shots. A similar photo stands for its whole shot, so one not
kept goes to Trash with the rest of its shot.
2026-09-28 09:16:54 +02:00
nvrl c98ed0d83b feat(core): perceptual hashes for finding similar photos
A 64-bit difference hash of each thumbnail, stored in the new
photos.similar_hash column (schema 34), and the queries the similar-photo
finder needs: what is left to hash, progress, and the sets of shots that
look alike. Sets made only of byte-identical copies are left to the
duplicate finder. A refresh keeps a hash unless the photo's bytes changed.

The daemon pass, the control request and the GUI are not built yet; see
PLAN.md.
2026-09-28 09:16:54 +02:00
Nils Pukropp c46b2c1082 chore(aur): update packages to 2.3.0 2026-09-28 01:42:04 +00:00
nvrl fde9c2d271 feat(core,fuse,gui): Proton theme, photos on the map and an opt-in street map
- The Proton accent preference becomes a whole Proton theme that also
  holds against a GTK theme imported from gtk.css; folders get Proton's
  purple folder icon while it is on.
- Places map markers show one of each town's photos with a count, and
  zoomed in on a town every photo sits where it was taken
  (Request::PlaceLocations). Photos taken on one spot are listed in a
  popover. "Show on map" in the photo viewer goes to the Places map.
- The built-in map draws country borders, sea and land in their own
  colours.
- An opt-in street map from OpenFreeMap (vector tiles drawn with cairo,
  cached on disk up to 256 MB), with the OpenStreetMap credit on the map
  and in About.
- Translations for all catalogs; changelog folded into 2.3.0.
2026-09-28 03:12:57 +02:00
nvrl 4fd14eb745 chore(release): 2.3.0 2026-09-28 02:12:59 +02:00
nvrl b3c03a7162 feat(core,fuse,gui): places on a world map
The Places view gets a Grid/Map toggle. The map shows a bubble per town,
sized by its number of photos, over the outline of the world's land. Towns
whose bubbles would overlap merge into one that counts all their photos, and
clicking it zooms in until they part; clicking a single town opens it like a
card does. Drag to pan, scroll or pinch to zoom, and the buttons zoom in,
zoom out or frame every place again. Hovering names the town, or the first
few towns of a merged bubble.

No tile server is involved: none that is free has terms allowing an app like
this one to use it. The map is drawn from Natural Earth's 1:50m land
polygons (public domain), trimmed and simplified by
scripts/natural-earth-land.py into crates/pdfs-gui/data/land.txt, in Web
Mercator. The About dialog credits Natural Earth.

PlaceInfo carries the town's latitude and longitude; a daemon older than the
map sends none, and its towns are left off the map.
2026-09-28 02:12:59 +02:00
nvrl 8749d6c2c6 feat(core,fuse,gui): show how far photo locations have been read
The first timeline refresh after the Places update reads the metadata of
the whole library, and until it is done the Places view has nothing to show
and no sign that anything is happening.

The refresh now records how many photos it has resolved out of how many,
counting photos read by an earlier pass as done. Response::Places carries
it as mapping (absent from older daemons, which parse as None). The Places
view shows "Mapping photos: 120 of 480" with a bar while it is set, asks
again every two seconds, and loads the new places once the pass ends. An
empty listing during the pass says the photos are being mapped rather than
that there are no places.
2026-09-28 02:12:59 +02:00
nvrl 4ee2aed29b fix(gui): scroll the "On this day" strip with the photos timeline
The strip sat above the timeline's list and was hidden as soon as the list
left its top. Hiding it grew the viewport, which moved the list back to its
top and brought the strip back, so scrolling up near the top shook the whole
page between the two heights.

The strip is now the first row of the list (GalleryRow::Memories), so it
scrolls away with the grid and never resizes the viewport. It is still only
there atop the whole, unfiltered timeline.
2026-09-28 02:12:59 +02:00
nvrl 753056418c feat(core,fuse,gui): places view from photo locations
Photos now know where they were taken. proton-drive-rs 0.7.0 reads the
Location from a photo's extended attributes, and the timeline refresh stores
it with the photo (schema v33). The migration marks every photo unresolved
once, so the next refresh backfills locations from node metadata without
downloading anything.

Towns are looked up offline: pdfs-core bundles GeoNames' cities15000 table,
trimmed by scripts/geonames-cities.py, and files each photo under the
nearest town within 50 km. The new PhotoPlaces and PlacePhotos requests list
the towns and page one town's photos.

The Photos page gets a third view, Places, with a card per town (newest
photo, town, country, count). A place opens in the gallery like an album.
Country names come translated from the iso-codes catalogs, now an optional
dependency. The About dialog credits GeoNames (CC BY 4.0).
2026-09-28 02:12:59 +02:00
nvrl 2757977fd6 feat(gui): one Filters popover for the photos timeline
Favorites, "not in an album" and the month move from separate pills and a
dropdown into a Filters popover next to the kind tabs. The kind tabs stay
in the row because their counts are worth a glance. The button carries a
badge with the number of active filters, kind included, and the popover
has a "Clear filters" action that resets everything in one reload.

Flipping a filter while a page was still loading used to drop the reload,
leaving the timeline answering the old filter. The request is now queued
and the stale page is discarded.
2026-09-28 02:12:59 +02:00
nvrl 2f14c1688f feat(core,fuse,gui): find and trash duplicate photos
A "Find duplicates" button in the Photos header opens a review of every
set of files that share a content hash. Each set shows its copies side by
side with a Keep toggle; the copy in the most albums, then in the larger
RAW+JPEG group, then the oldest, is kept by default, and a set can't lose
its last kept copy. The copies not kept move to Trash one file at a time.

- core: Db::photo_duplicates, Request::PhotoDuplicates, and a files_only
  flag on TrashNodes (defaults to false, so old clients still trash whole
  groups).
- core: deleting a group's representative now clears the group key of its
  remaining files instead of leaving them pointing at a missing uid.
- fuse: handlers for both requests.
2026-09-28 02:12:59 +02:00
nvrl 7dbf818d83 feat(core,fuse,gui): "On this day" strip above the photos timeline
Add a PhotosOnThisDay control request, served from the stored timeline:
photos whose local capture date matches today's in earlier years, one
row per group. The GUI shows a card per year ("3 years ago") with the
year's newest photo as its cover; clicking a card scrolls the timeline
to that day. The strip is revealed only at the top of the whole,
unfiltered timeline, so it gives the grid its room back once you
scroll or filter.
2026-09-28 02:12:59 +02:00
nvrl c3ec82392a fix(gui): keep the photos timeline at its top across a relayout
Anchoring on the first photo row after a relayout scrolled the heading
above it out of view, so the timeline opened a heading's height down.
2026-09-28 02:12:58 +02:00
nvrl ce4656e46a fix(gui): square photo grid laid out to the viewport
Lay the timeline out as a grid of square tiles that spans the
scrolled window's width exactly, instead of justified rows sized from
learned aspect ratios. Track the width from the horizontal adjustment
so the widest row can no longer push the window wider, and put the
scrubber beside the list rather than over it.

Load the app stylesheet above user priority: a theme imported from the
user's gtk.css padded every button, tiles included. Keep the top photo
at its screen offset across a relayout, and only drop thumbnail
requests the unbound row still owns.
2026-09-28 02:12:58 +02:00
nvrl 557eeb5f1d feat(gui): slideshow in the photo viewer
A new top-bar button, or F5, plays the loaded photos as a slideshow: it goes
fullscreen, hides the controls and moves on every five seconds. A video plays
to its end first, and the slideshow waits for each photo to load before it
starts counting. Stepping by hand restarts the count. Escape stops the
slideshow instead of closing the viewer, and fullscreen is left only if the
slideshow entered it. The slideshow ends on the last photo of the timeline.
2026-09-28 02:12:58 +02:00
nvrl b47e6c91a0 feat(gui): group the photos timeline by zoom level, select ranges
Zooming out now changes how the timeline is sectioned: by day at the default
row height, by month further out and by year at the smallest tiles, which
gain one more zoom step. Under a month or year heading the tile caption shows
the day instead of the clock time.

Shift+click selects every photo from the last one clicked, and dragging
across the timeline selects the photos swept over, scrolling at the top and
bottom edges. Sweeping back gives photos up again.
2026-09-28 02:12:58 +02:00
nvrl 5c1da9d7a5 feat(core,fuse,gui,cli): filter the photos timeline to photos in no album
A new pill next to the favorites filter narrows the timeline to the photos
no album of ours holds yet, so what is left to file is easy to find. A
RAW+JPEG pair counts as filed when any of its files is in an album.

PhotosTimeline gains an optional not_in_album flag; older front-ends omit it
and get the whole timeline. Album contents are fetched lazily, so the first
filtered page fetches the contents of any album not yet listed. Photos added
to an album while the filter is on leave the timeline at once.
2026-09-28 02:12:58 +02:00
Nils Pukropp 0135c29a4f chore(aur): update packages to 2.2.2 2026-09-27 21:44:59 +00:00
189 changed files with 111837 additions and 27478 deletions

No files matched your search

+8 -2
View File
@@ -45,7 +45,7 @@ jobs:
run: pacman -Syu --noconfirm --needed git openssh pacman-contrib
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
with:
# The updated recipes are committed back to main, not to the tag.
ref: main
@@ -67,7 +67,13 @@ jobs:
User aur
EOF
# Pin the host key to the fingerprint published on aur.archlinux.org.
ssh-keyscan -t ed25519 aur.archlinux.org > "$HOME/.ssh/known_hosts"
# The scan sometimes returns nothing from the runner, so retry it.
for attempt in 1 2 3 4 5; do
ssh-keyscan -T 15 -t ed25519 aur.archlinux.org > "$HOME/.ssh/known_hosts" &&
[ -s "$HOME/.ssh/known_hosts" ] && break
echo "ssh-keyscan attempt $attempt returned no key; retrying"
sleep 10
done
ssh-keygen -lf "$HOME/.ssh/known_hosts" | grep -q 'SHA256:RFzBCUItH9LZS0cKB5UE6ceAYhBD5C8GeOBip8Z11+4'
chown -R builder: "$HOME/.ssh"
chmod 600 "$HOME/.ssh/aur"
+88 -1
View File
@@ -13,7 +13,7 @@ jobs:
rust:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install native dependencies
run: |
@@ -32,8 +32,14 @@ jobs:
with:
toolchain: '1.96'
components: rustfmt, clippy
cache: false
# Tags can only restore caches saved on main, so only main saves one. The
# release workflow restores this key for its quality gates.
- uses: swatinem/rust-cache@v2
with:
shared-key: debug
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Formatting
run: cargo fmt --all -- --check
@@ -57,3 +63,84 @@ jobs:
# Real Proton credentials and /dev/fuse deliberately do not run on pull
# requests. Follow docs/TESTING.md on a dedicated test-account runner.
# The seeded simulation runs (docs/TESTING.md): real FUSE
# daemons against an in-memory Drive, so they need /dev/fuse but no account.
simulation:
if: github.event_name != 'pull_request'
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@v7
- name: Install native dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
pkg-config \
fuse3 \
libfuse3-dev \
libgtk-4-dev \
libadwaita-1-dev \
libwebkitgtk-6.0-dev \
libsecret-1-dev \
libdbus-1-dev
- name: Install minimum supported Rust
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: '1.96'
cache: false
- uses: swatinem/rust-cache@v2
with:
shared-key: debug
save-if: false
- name: Simulation runs
run: 'cargo test -p pdfs-fuse --lib --locked sim:: -- --ignored --test-threads=1'
# A hung step writes every daemon thread's stack next to the run's
# state; without them a hang that does not replay cannot be told apart.
- name: Keep the stacks of a hung run
if: failure()
uses: actions/upload-artifact@v7
with:
name: simulation-stacks
path: /tmp/pdfs-sim-*/stacks.txt
if-no-files-found: ignore
# Keeps the release-profile dependencies cached on main, where the release
# workflow can restore them; a tag cannot reuse the caches of earlier tags.
release-cache:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
- name: Install native dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
pkg-config \
libfuse3-dev \
libgtk-4-dev \
libadwaita-1-dev \
libwebkitgtk-6.0-dev \
libsecret-1-dev \
libdbus-1-dev
- name: Install minimum supported Rust
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: '1.96'
cache: false
- uses: swatinem/rust-cache@v2
with:
shared-key: release
- name: Release build
run: cargo build --release --locked
+76 -25
View File
@@ -10,14 +10,66 @@ permissions:
contents: write
jobs:
checks:
name: Release quality gates
runs-on: ubuntu-24.04
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
pkg-config \
libfuse3-dev \
libgtk-4-dev \
libadwaita-1-dev \
libwebkitgtk-6.0-dev \
libsecret-1-dev \
libdbus-1-dev
- name: Set up Rust
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: '1.96'
components: rustfmt, clippy
cache: false
# A tag can read the caches of main but its own caches are useless to the next
# tag, so this only restores what CI saved on main.
- name: Cache Cargo target
uses: swatinem/rust-cache@v2
with:
shared-key: debug
save-if: false
- name: Verify tag and manifest versions
if: startsWith(github.ref, 'refs/tags/')
shell: bash
run: |
tag_version="${GITHUB_REF_NAME#v}"
manifest_version="$(sed -n 's/^version = "\([^"]*\)"/\1/p' Cargo.toml | head -n1)"
pkgbuild_version="$(sed -n 's/^pkgver=//p' packaging/PKGBUILD | head -n1)"
test "$tag_version" = "$manifest_version"
test "$tag_version" = "$pkgbuild_version"
- name: Release quality gates
run: |
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --locked
scripts/fuse-acceptance.sh --offline-only
build-release:
name: Build and Release
name: Build packages
runs-on: ubuntu-24.04
environment: 'rpm signing'
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Install system dependencies
run: |
@@ -38,27 +90,14 @@ jobs:
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
toolchain: '1.96'
components: rustfmt, clippy
cache: false
# Restores the release-profile dependencies that CI builds on main.
- name: Cache Cargo target
uses: swatinem/rust-cache@v2
- name: Verify tag and manifest versions
if: startsWith(github.ref, 'refs/tags/')
shell: bash
run: |
tag_version="${GITHUB_REF_NAME#v}"
manifest_version="$(sed -n 's/^version = "\([^"]*\)"/\1/p' Cargo.toml | head -n1)"
pkgbuild_version="$(sed -n 's/^pkgver=//p' packaging/PKGBUILD | head -n1)"
test "$tag_version" = "$manifest_version"
test "$tag_version" = "$pkgbuild_version"
- name: Release quality gates
run: |
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace --locked
scripts/fuse-acceptance.sh --offline-only
with:
shared-key: release
save-if: false
- name: Build release binaries
run: |
@@ -127,7 +166,7 @@ jobs:
Maintainer: Nils Pukropp <contact@narl.io>
Depends: fuse3, libgtk-4-1 (>= 4.12.0), libadwaita-1-0 (>= 1.5.0), libsecret-1-0, libwebkitgtk-6.0-4, libimage-exiftool-perl
Installed-Size: ${INSTALLED_SIZE}
Homepage: https://github.com/narrrl/proton-drive-linux
Homepage: https://proton-drive.narl.io
Description: Unofficial Proton Drive client for Linux
Files-on-demand Proton Drive for the Linux desktop, unaffiliated with Proton AG:
- FUSE virtual mount with block-level caching
@@ -152,6 +191,7 @@ jobs:
--define "_topdir $PWD/rpmbuild" \
--define "_sourcedir $PWD" \
--define "git_dir $PWD" \
--define "prebuilt 1" \
--define "version ${RPM_VERSION}" \
--define "_dbpath $PWD/rpmbuild/db" \
packaging/proton-drive-linux.spec
@@ -181,7 +221,7 @@ jobs:
- name: Upload Workflow Artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: proton-drive-linux-builds
path: |
@@ -190,9 +230,20 @@ jobs:
*.rpm
if-no-files-found: error
publish:
name: Create GitHub Release
needs: [checks, build-release]
if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-24.04
steps:
- name: Download packages
uses: actions/download-artifact@v8
with:
name: proton-drive-linux-builds
- name: Create GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
files: |
*.deb
@@ -205,7 +256,7 @@ jobs:
aur:
name: Publish AUR packages
needs: build-release
needs: publish
if: startsWith(github.ref, 'refs/tags/')
uses: ./.github/workflows/aur.yml
with:
+17 -8
View File
@@ -6,9 +6,10 @@ deliberately high — everything else is ordinary Rust review.
## Before you start
- Read [`AGENTS.md`](AGENTS.md) for repository conventions and
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for how the daemon, front ends, and SDK fit
together.
- Read [`AGENTS.md`](AGENTS.md) for repository conventions,
[`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md) for building, running and debugging from source,
and [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for how the daemon, front ends, and SDK fit
together. [`docs/README.md`](docs/README.md) indexes the rest.
- Check [`docs/BUGS.md`](docs/BUGS.md) — it is the authoritative issue ledger, and your problem may
already be tracked there with context.
- For anything larger than a fix, open an issue first so the design can be discussed before you
@@ -23,8 +24,9 @@ cargo build --workspace
```
GUI and FUSE targets need the GTK4, libadwaita, WebKitGTK 6, libsecret, D-Bus, and FUSE3
development packages listed in the [README](README.md#prerequisites). Minimum supported Rust
version is 1.96.
development packages listed in [`docs/INSTALL.md`](docs/INSTALL.md#build-dependencies). Minimum
supported Rust version is 1.96. [`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md#running-from-source)
explains how to run your build next to, or instead of, the installed service.
## Quality gates
@@ -37,9 +39,9 @@ cargo test --workspace --locked
scripts/fuse-acceptance.sh --offline-only
```
Changes to the FUSE layer, sync, or the pending-operation queue should also be exercised against a
real mount before review — see [`docs/TESTING.md`](docs/TESTING.md). Live runs are destructive:
use a dedicated test account, never one holding data you cannot lose.
Changes to the FUSE layer, sync, or the queue should also pass the simulation runs and be
exercised against a real mount before review — see [`docs/TESTING.md`](docs/TESTING.md). Live
runs are destructive: use a dedicated test account, never one holding data you cannot lose.
## Making changes
@@ -51,6 +53,13 @@ use a dedicated test account, never one holding data you cannot lose.
control-socket work stays off GTK's main thread.
- **`staging/` and `recovery/` may hold the only copy of user data.** Nothing may clear or purge
them opportunistically.
- **The GUI is fully translated.** Wrap every user-visible string in the helpers from
`crates/pdfs-gui/src/i18n.rs`, run `po/update.sh`, and translate the new entries in every
catalog in `po/LINGUAS` in the same change. See [`docs/TRANSLATING.md`](docs/TRANSLATING.md).
The CLI and daemon messages stay in English.
- **Document what users see.** A new command, setting or behaviour belongs in the matching page
under `docs/` (user guide, CLI reference or configuration) and in the `[Unreleased]` section of
[`docs/CHANGELOG.md`](docs/CHANGELOG.md).
## Commits and pull requests
Generated
+37 -8
View File
@@ -3389,6 +3389,31 @@ dependencies = [
"system-deps 9.0.0",
]
[[package]]
name = "pangocairo"
version = "0.22.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bd199a0fef965b3ebda91a929a1ab5e7729538882c68e8de38e1904d2e094757"
dependencies = [
"cairo-rs",
"glib",
"pango",
"pangocairo-sys",
]
[[package]]
name = "pangocairo-sys"
version = "0.22.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a649a616d67ae68539bc85f00fb45d36fba742ed20fc20e6712a15d8ee52bd3f"
dependencies = [
"cairo-sys-rs",
"glib-sys",
"libc",
"pango-sys",
"system-deps 9.0.0",
]
[[package]]
name = "parking"
version = "2.2.1"
@@ -3449,7 +3474,7 @@ checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4"
[[package]]
name = "pdfs-cli"
version = "2.2.2"
version = "3.0.0"
dependencies = [
"anyhow",
"clap",
@@ -3464,7 +3489,7 @@ dependencies = [
[[package]]
name = "pdfs-core"
version = "2.2.2"
version = "3.0.0"
dependencies = [
"async-trait",
"dbus-secret-service-keyring-store",
@@ -3488,8 +3513,9 @@ dependencies = [
[[package]]
name = "pdfs-fuse"
version = "2.2.2"
version = "3.0.0"
dependencies = [
"async-trait",
"base64 0.23.1",
"fuser",
"futures",
@@ -3503,12 +3529,13 @@ dependencies = [
"sha1 0.11.0",
"tokio",
"tracing",
"tracing-subscriber",
"zip",
]
[[package]]
name = "pdfs-gui"
version = "2.2.2"
version = "3.0.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3517,7 +3544,9 @@ dependencies = [
"kamadak-exif",
"ksni",
"libadwaita",
"pangocairo",
"pdfs-core",
"reqwest",
"serde",
"serde_json",
"tokio",
@@ -3746,9 +3775,9 @@ dependencies = [
[[package]]
name = "proton-drive-rs"
version = "0.6.6"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34d587f1ef2ffa9183f48f0cca407c4b641b636be516dde8b13e082de5c83757"
checksum = "d6ce113d234474821d34787436ee80e1458e3d8a8f97212219615ed315eb70be"
dependencies = [
"async-trait",
"base64 0.23.1",
@@ -3773,9 +3802,9 @@ dependencies = [
[[package]]
name = "proton-sdk"
version = "0.6.6"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7decd4f7ec188815192a1d2a16a83f2d52944141c5b8e648184ef17061c9ecce"
checksum = "2a33c67df9694f90ce60e5b34984275956244138511b4640ca0d100e1dac8b3d"
dependencies = [
"aes-gcm 0.11.1",
"async-trait",
+4 -4
View File
@@ -3,20 +3,20 @@ resolver = "2"
members = ["crates/pdfs-core", "crates/pdfs-fuse", "crates/pdfs-cli", "crates/pdfs-gui"]
[workspace.package]
version = "2.2.2"
version = "3.0.0"
edition = "2024"
license = "MIT"
authors = ["Nils Pukropp <contact@narl.io>"]
rust-version = "1.96"
repository = "https://github.com/narrrl/proton-drive-linux"
homepage = "https://github.com/narrrl/proton-drive-linux"
homepage = "https://proton-drive.narl.io"
readme = "README.md"
keywords = ["proton", "drive", "fuse", "sync", "linux"]
categories = ["filesystem", "command-line-utilities"]
[workspace.dependencies]
proton-sdk = { version = "0.6.6" }
proton-drive-rs = { version = "0.6.6" }
proton-sdk = { version = "0.7.2" }
proton-drive-rs = { version = "0.7.2" }
# Internal crates.
pdfs-core = { path = "crates/pdfs-core" }
+110 -448
View File
@@ -4,512 +4,174 @@
# Proton Drive for Linux
**Files-on-demand Proton Drive for the Linux desktop** — a FUSE virtual mount with block-level
caching, a scriptable CLI, and a native GTK4 app with tray and search launcher.
**Proton Drive on the Linux desktop.** Your files as a folder that downloads on demand, synced
local folders, Photos, sharing, and a native GTK4 app with tray and search launcher, plus a
scriptable CLI.
**[proton-drive.narl.io](https://proton-drive.narl.io)**
[![CI](https://github.com/narrrl/proton-drive-linux/actions/workflows/ci.yml/badge.svg)](https://github.com/narrrl/proton-drive-linux/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/narrrl/proton-drive-linux?sort=semver)](https://github.com/narrrl/proton-drive-linux/releases/latest)
[![AUR](https://img.shields.io/aur/version/proton-drive-for-linux)](https://aur.archlinux.org/packages/proton-drive-for-linux)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Rust 1.96+](https://img.shields.io/badge/rust-1.96%2B-orange.svg)](https://www.rust-lang.org)
[![Platform: Linux](https://img.shields.io/badge/platform-linux-lightgrey.svg)](#prerequisites)
[![Buy me a coffee](https://img.shields.io/badge/Buy%20me%20a%20coffee-FFDD00?logo=buymeacoffee&logoColor=black)](https://buymeacoffee.com/narl)
<img src="images/files.png" alt="The GTK4 file browser" width="820">
<img src="images/files.png" alt="My files in the Proton Drive app" width="820">
</div>
> [!IMPORTANT]
> This is an **unofficial**, community-built client. It is not affiliated with, endorsed by, or
> supported by Proton AG. "Proton" and "Proton Drive" are trademarks of Proton AG, used here only
> to describe what the software talks to. Use a test account first, and read
> [`docs/RECOVERY.md`](docs/RECOVERY.md) before trusting it with data you cannot lose.
## Install
```bash
# Debian / Ubuntu
sudo apt install ./proton-drive-linux_*.deb
# Fedora
sudo dnf install ./proton-drive-linux-*.rpm
# Arch Linux (AUR)
paru -S proton-drive-for-linux # or -git / -bin
```
Packages for each release are attached to the [latest
release](https://github.com/narrrl/proton-drive-linux/releases/latest); see
[Installation & Packages](#installation--packages) for details and
[Building from Source](#building-from-source) to compile it yourself.
```bash
pdfs login # sign in (SRP + 2FA, credentials go to the system keyring)
pdfs status # mount state, queued writes, cache usage
pdfs-app # the desktop app
```
## Contents
- [Features](#features) · [Quick search](#quick-search-and-media-opening) · [Opening files](#choosing-how-files-open)
- [Selective sync](#selective-sync-pdfsignore) · [Diagnostics](#diagnostics--maintenance) · [Scripting](#scripting---json)
- [Performance](#performance--caching) · [Filesystem safety](#filesystem-safety) · [Screenshots](#screenshots)
- [Prerequisites](#prerequisites) · [Building](#building-from-source) · [Packages](#installation--packages) · [Releases](#automated-releases-cicd)
**Documentation:** [Architecture](docs/ARCHITECTURE.md) · [Testing](docs/TESTING.md) ·
[Recovery](docs/RECOVERY.md) · [Development](docs/DEVELOPMENT.md) · [Translating](docs/TRANSLATING.md) ·
[Known issues](docs/BUGS.md) · [Changelog](docs/CHANGELOG.md) ·
[Contributing](CONTRIBUTING.md) · [Security policy](SECURITY.md)
> to describe what the software talks to. Try it with data you have another copy of first, and
> read [what survives a lost machine](docs/RECOVERY.md) before you rely on it.
## Features
- **Files-on-Demand (FUSE)**: Mount your Proton Drive as a virtual filesystem where files are downloaded only when opened, utilizing smart block-level caching and disk-backed writes.
- **Command-Line Interface (CLI)**: Manage your drive, authenticate, and monitor sync status directly from the terminal.
- **Non-Blocking GTK4 Desktop App**: Browse files, manage pins, configure options, and monitor active transfers through a modern, native GUI with a fully non-blocking asynchronous main loop.
- **File Thumbnails**: Browse locally cached image previews in Files, Shared, and Trash, including embedded previews from camera RAW files. Build a whole folder tree from the Files toolbar and resize the grid from the status bar.
- **Shared Links & Invites**: Browse files shared with you by other users, and view/manage your own public shared links directly in the GUI.
- **Local Backup (Computers)**: Sync and back up local directories (like Downloads, Documents, Pictures, etc.) directly to your Proton Drive account.
- **Locations**: One page (and `pdfs locations`) listing every place Proton Drive occupies on this machine — the main mount plus each backed-up folder — with its mode, sync state, and whether it is read-only. Switching a folder between a full local copy and on-demand happens here.
- **System Tray Integration**: Background indicator for status monitoring, quick actions, and fast sync controls.
- **Unified Search Launcher (HUD)**: A resident Google Drive-style launcher (`pdfs-prompt`) that searches Proton Drive and local files together, ranks the best matches, tolerates abbreviations and typos, and is ideal for a system-wide hotkey. It can also present the same search in your own launcher (`--dmenu`) or, searching live as you type, in `fzf` (`--fzf`).
- **Secure Credential Storage**: Integrates with the system Secret Service (GNOME Keyring, KWallet, etc.) with smart in-memory credential caching to avoid UI thread blockages.
- **Proton Photos Support**: Access your Proton Photos timeline, view thumbnails, and download backed-up media natively (available in the GUI as a navigation tab and via the CLI).
- **File Version History**: Every revision Proton Drive still holds for a file, from the browser's details pane (**Versions**) or `pdfs versions list|restore|save|rm`. Restoring is server-side — no re-upload — and an old version can be written out to a local file without touching the live one.
- **Photo Favourites**: Star a photo in the lightbox, filter the gallery to favourites, or use `pdfs favorite <uid>` / `pdfs photos --favorites`.
- **Photo Albums**: Browse your albums — including the ones shared with you — from the Albums view of the Photos page, or with `pdfs albums` / `pdfs album <uid>`. Album contents open in the same gallery as the timeline.
- **Human Verification (CAPTCHA) Recovery**: Detects sign-in gates (VPN/new IP challenges) and launches an embedded `WebKitWebView` dialog to safely complete the challenge, transparently retrying authentication with the earned token.
- **Selective Sync (`.pdfsignore`)**: Keep build trees, dependency directories, and editor leftovers out of synced folders using gitignore-style rules.
- **Data-Safe Offline Writes**: Durable scratch/staging files and a transactional pending queue preserve acknowledged writes across network failures and restarts.
**Files**
## Quick Search and Media Opening
- **Your Drive as a folder.** `~/ProtonDrive` works with every program. Files download when they
are opened, a block at a time, so a video starts playing and a large archive can be read
without fetching all of it.
- **Available offline.** Keep chosen files and folders fully downloaded; everything else is
cached within a size limit you set.
- **Synced folders.** Back up folders such as `~/Documents` under your computer in Proton Drive,
as a full two-way copy or online only. Gitignore-style `.pdfsignore` rules, conflict copies
instead of overwrites, pause and bandwidth limits.
- **Works at local speed.** Making, renaming and deleting files and folders returns at once, online
or offline. Changes are staged on disk and queued before they are acknowledged, so they survive
a lost connection, a crash or a reboot and reach Drive when they can.
`pdfs-prompt` searches Drive metadata and the local home-directory index in one request. Results are ranked together rather than split by source. Matching is case-insensitive and supports exact names, prefixes, parent paths, multiple terms, punctuation-separated words, ordered abbreviations, common typos, and adjacent transpositions.
**Desktop app**
The prompt stays resident after its first launch, so invoking the shortcut again reuses and resets the existing window. Drive folders open from the mount. Audio and video results also open through FUSE, allowing media players to seek and request ranges without downloading the entire file first; ordinary Drive files are materialized into the local cache before opening.
- **Browse** My files, Shared with me, Shared by me, Computers and Trash, with thumbnails
(including camera RAW), drag and drop, details and **version history**.
- **Photos.** Timeline with favorites, albums, and Places on a map; RAW+JPEG pairs shown as one
photo; duplicate finder; slideshow; **import from Google Photos** (Takeout).
- **Sharing.** Invite people as viewer, editor or admin, create public links with password and
expiry, accept invitations.
- **Sync overview.** What is uploading, what is waiting, what failed and why, and a history of
what happened. A change Drive refuses (storage full, access taken away) shows at once, on the
file too, and can be exported or discarded.
- **Tray icon** with status, pause and quick actions, and a **search launcher** (`pdfs-prompt`)
that searches Proton Drive and your home folder together, as its own window, through fuzzel,
rofi or wofi, or live in `fzf`.
- **Restore a computer.** Continue a backup on a new machine and bring its folders back in one
step.
- **Sign-in** with two-factor authentication and CAPTCHA; the session is kept in the system
keyring.
- Available in English, German, Spanish, French, Italian, Dutch, Polish and Brazilian
Portuguese.
### Using your own launcher (fuzzel, rofi, wofi, …)
**Command line**
`pdfs-prompt --dmenu` runs the same search through an external dmenu-style
launcher instead of the built-in window, so the HUD matches the rest of a
tiling-WM desktop:
- About 55 `pdfs` commands cover files, sync, sharing, Photos, Trash, versions and diagnostics.
Query commands print JSON with `--json`.
```bash
pdfs-prompt --dmenu # fuzzel/rofi/wofi/tofi/bemenu/dmenu, whichever is installed
pdfs-prompt --dmenu --menu 'fuzzel --dmenu --width 60'
pdfs-prompt --dmenu --query invoice # skip straight to a search
```
## Install
A launcher filters a fixed list — it cannot ask for a new one per keystroke — so
searching takes two steps, and the prompt says which step you are on:
- `Search Drive ›` lists your pinned files. Type anything that isn't one of them
and press Enter to search for it.
- `Drive: invoice ›` lists the results. Enter opens one; typing something else
and pressing Enter searches again. Escape closes.
`--query` skips the first step. fuzzel and rofi also get file-type icons.
To make an existing keybinding use it without re-binding, and to pin the
launcher command, set them in `config.json`:
```json
{
"prompt": {
"mode": "dmenu",
"menu": ["fuzzel", "--dmenu", "--width", "60"],
"menu_limit": 50
}
}
```
`--gtk` overrides `"mode": "dmenu"` for one invocation. A `{prompt}` token
anywhere in `menu` is replaced by the prompt text; without one, the launcher's
own prompt flag is appended.
### Live launcher search (fzf)
The two-step above exists because no dmenu-style launcher can call back for new
results per keystroke. `fzf` can, so `pdfs-prompt --fzf` searches as you type,
exactly like the built-in window:
```bash
pdfs-prompt --fzf # pinned files, then live results as you type
pdfs-prompt --fzf --query invoice # start on a search
```
Every keystroke re-queries the daemon (debounced, like the GTK prompt) and fzf's
own matcher is switched off, so the ordering you see is the daemon's ranking —
abbreviations and typos included, which a launcher filtering a fixed list cannot
do. Enter opens; Escape closes.
fzf is a terminal program, so when it is started without one — from a WM
keybinding — it spawns a terminal for itself: foot, ghostty, kitty, alacritty,
wezterm or xterm, whichever is installed. Each is launched with the app-id/class
`pdfs-prompt`, so one window rule floats it whichever terminal you have. In
Hyprland:
```
windowrulev2 = float, class:^(pdfs-prompt)$
windowrulev2 = size 900 500, class:^(pdfs-prompt)$
windowrulev2 = center, class:^(pdfs-prompt)$
bind = SUPER, space, exec, pdfs-prompt --fzf
```
Set it as the default front end, and pick the terminal explicitly, in
`config.json`:
```json
{
"prompt": {
"mode": "fzf",
"terminal": ["foot", "--app-id=pdfs-prompt", "--"],
"menu_limit": 50
}
}
```
A `{cmd}` token anywhere in `terminal` is replaced by the command to run;
without one it is appended, which is what every detected terminal expects.
## Choosing How Files Open
By default every result is handed to `xdg-open`. An `open_with` block in
`config.json` overrides that per file type — for instance opening text in
Neovim inside Alacritty, while everything else still goes to the desktop:
```json
{
"open_with": {
"terminal": ["alacritty", "-e"],
"rules": [
{ "match": ["@text"], "command": ["nvim"], "terminal": true },
{ "match": ["*.png", "*.jpg"], "command": ["imv"] }
]
}
}
```
- `match` takes file-name globs (`*.md`, `notes-*.txt`) or the classes `@dir`,
`@text`, `@document`, `@image`, `@media`, `@any`. First matching rule wins.
- `command` is argv, not a shell line. A `{}` token is replaced by the path;
without one the path is appended. `$VAR` tokens expand from the environment,
so `["$EDITOR"]` follows your editor.
- `"terminal": true` wraps the command in `terminal`. When that is unset, the
terminal comes from `$PDFS_TERMINAL`, then `$TERMINAL`, then the first known
emulator on `PATH`; a bare name like `TERMINAL=alacritty` gains the right
"run this" flag automatically.
- `"default": [...]` replaces `xdg-open` for everything unmatched.
The rules apply to the prompt (both front ends) and to the GTK browser, so a
file opens the same way wherever it was picked. Patterns are matched against the
Drive name — a downloaded file is stored in the cache under its content hash,
which says nothing about its type, so matching that would send everything to
`xdg-open`.
## Selective Sync (`.pdfsignore`)
Two-way synced folders skip paths matched by ignore rules, so syncing a project
directory does not upload `node_modules/`, `target/`, or `.git/`.
Rules come from two places, and both apply:
1. **Per folder** — a `.pdfsignore` file at the root of the synced folder
(`.protonignore` also works). Gitignore syntax, including negation:
```gitignore
# everything build-related
build/
*.log
# ...except this one
!important.log
```
2. **Globally** — an `ignore_patterns` list in `config.json`, applied to every
synced folder. When unset, sensible defaults apply: `.git/`, `.hg/`, `.svn/`,
`node_modules/`, `target/`, `.venv/`, `__pycache__/`, `*~`, `*.swp`, `*.tmp`,
`.DS_Store`, and `Thumbs.db`.
```json
{
"ignore_patterns": ["node_modules/", "target/", "*.iso"]
}
```
Set it to `[]` to opt out of the defaults entirely.
Rules are re-read at the start of every sync pass, so edits take effect on the
next pass without restarting the daemon.
**Ignoring is never destructive.** If a rule starts matching a file that was
already synced, its copy on Drive is left untouched — the file simply stops
being tracked. Removing the rule later picks the existing remote file back up
rather than re-uploading it.
## Diagnostics & Maintenance
When something looks wrong, `pdfs diagnose` checks the installation and prints a
report. It runs without a daemon on purpose — the state worth diagnosing is
usually the state where the daemon will not start:
```console
$ pdfs diagnose
Paths
[ok ] state dir: /home/you/.local/state/proton-drive-linux
[ok ] database: /home/you/.local/state/proton-drive-linux/cache.db (170.3 MiB)
Account
[ok ] keyring session: you@proton.me
Daemon
[ok ] daemon responding
[ok ] mounted at: /home/you/ProtonDrive
[ok ] queued writes: none
No problems found.
```
It exits non-zero if any check fails, so it works in a health-check script.
For the local metadata database and content cache:
| Command | What it does |
| Distribution | Package |
|---|---|
| `pdfs cache inspect` | Database size, reclaimable space, per-table row counts, cache usage against budget |
| `pdfs cache inspect --deep` | Also runs SQLite's integrity check — reads every page, so it is slow on a large database |
| `pdfs cache vacuum` | Checkpoints the write-ahead log and compacts the database |
| `pdfs cache clear` | Deletes cached file content, keeping pinned files |
| Debian, Ubuntu 24.04 or newer | `.deb` from the [latest release](https://github.com/narrrl/proton-drive-linux/releases/latest) |
| Fedora 44 or newer | `.rpm` from the [latest release](https://github.com/narrrl/proton-drive-linux/releases/latest) |
| Arch Linux | [`proton-drive-for-linux`](https://aur.archlinux.org/packages/proton-drive-for-linux) (also `-bin` and `-git`) in the AUR |
| Anything else | [Build from source](docs/INSTALL.md#build-from-source) |
`vacuum` takes a write lock for its duration and needs room for a second copy of
the database while it runs, so it is a deliberate operation rather than
something the daemon does on a timer.
Before a release, run the real-kernel acceptance suite against a dedicated test
account and mount. See [`docs/TESTING.md`](docs/TESTING.md); it includes optional
two-mount convergence checks and confines destructive operations to a fresh
test directory.
## Scripting (`--json`)
Query commands accept a global `--json` flag and emit machine-readable output:
```console
$ pdfs --json sync list | jq '.items[] | select(.state != "idle") | .local_path'
$ pdfs --json cache inspect | jq '.db_reclaimable_bytes'
$ pdfs --json status | jq -r '.mount.mountpoint'
```bash
sudo apt install ./proton-drive-linux_*.deb # Debian, Ubuntu
sudo dnf install ./proton-drive-linux-*.rpm # Fedora
paru -S proton-drive-for-linux # Arch
```
Supported on `status`, `ls`, `pins`, `sync list`, `devices list`, `transfers`,
`activity`, and `cache inspect`. Commands that perform an action keep their
human output — a script that needs to know whether one worked has the exit code.
Then open **Proton Drive** from your applications menu and sign in. The app starts the background
service, and your files appear in `~/ProtonDrive`.
Two things worth relying on:
Signing in from a terminal instead? `pdfs login` stores the session but does not start the
service. Enable it once:
- **The payload is unwrapped.** Output is `{"items": […]}`, not the daemon's
internal `{"SyncFolders": {"items": […]}}`, so scripts never name an internal
variant.
- **Errors still fail.** A daemon-side error prints its JSON body (with a
machine-readable `kind`) on stdout *and* exits non-zero, so `set -e` and
`if pdfs …` behave as expected rather than treating a failed lookup as
success.
```bash
pdfs login
systemctl --user enable --now proton-drive.service
pdfs status
```
## Performance & Caching
[docs/INSTALL.md](docs/INSTALL.md) has the requirements, optional extras (tray on GNOME, RAW and
video thumbnails), headless use and uninstalling.
The client includes several optimizations designed for high efficiency, a low memory footprint, and a responsive user experience:
## Documentation
- **On-Demand Block Cache**: Files are read in fixed 4 MiB blocks. For unpinned files, the client fetches and caches only the blocks containing the requested byte ranges. This enables fast sequential and sparse reads (e.g. media streaming or metadata scanning) without downloading entire files.
- **Disk-Backed Writes**: Large file writes are staged on disk in temporary scratch files (rather than fully buffered in RAM) and track modified byte intervals. Only the unedited remote segments are fetched at commit time, keeping memory usage minimal.
- **Non-Blocking GTK4 Loop**: To prevent UI freezes, all synchronous D-Bus credential checks, control socket requests, and cache usage calculations are offloaded to background worker threads or fetched asynchronously.
- **Flicker-Free UI Rendering**: The GTK4 application performs differential rendering of the pins list, only modifying list rows when the list content changes, preserving the user's scroll position and widget focus.
- **Durable Staging**: Scratch metadata and staged writes are synced and atomically published before an upload is acknowledged locally. Rapid revisions supersede pending work transactionally.
## Filesystem Safety
Version 1.0 strengthens the boundaries around local-only data and destructive reconciliation:
- Truncate composes with queued revisions, preserving the authoritative prefix and correct zero-filled growth.
- Combined move-and-rename operations are queued durably and tolerate partially completed remote state.
- Session tombstones prevent successfully unlinked entries from reappearing through eventually consistent listings.
- Names are valid UTF-8 and no longer than Linux's 255-byte component limit.
- Incomplete mirror scans are non-destructive, and total-wipe protection covers every non-empty baseline.
- Failed conflict preservation, local deletion, or staging publication does not settle a new baseline or discard the only local copy.
- State, cache, and config directories must be real, current-user-owned `0700` directories; control sockets are `0600`.
| | |
|---|---|
| [User guide](docs/USER_GUIDE.md) | Everything the app, tray and launcher do |
| [Command-line reference](docs/CLI.md) | Every `pdfs` command and scripting with `--json` |
| [Configuration](docs/CONFIGURATION.md) | `config.json`, file locations, environment variables, the service |
| [Troubleshooting](docs/TROUBLESHOOTING.md) | Diagnosing problems and fixing common ones |
| [Recovering on a new computer](docs/RECOVERY.md) | Getting your synced folders back after a reinstall or a lost machine |
| [Changelog](docs/CHANGELOG.md) | What changed in each release |
For contributors: [development guide](docs/DEVELOPMENT.md), [architecture](docs/ARCHITECTURE.md),
[testing](docs/TESTING.md), [translating](docs/TRANSLATING.md), [roadmap](docs/ROADMAP.md) and the
[bug ledger](docs/BUGS.md). The [documentation index](docs/README.md) lists everything.
## Screenshots
### GUI Application & Launcher
<table>
<tr>
<td align="center" width="50%"><img src="images/login.png" alt="Login Screen" width="100%"><br><sub><b>Login Screen</b></sub></td>
<td align="center" width="50%"><img src="images/files.png" alt="Files Browser" width="100%"><br><sub><b>Files Browser</b></sub></td>
<td align="center" width="50%"><img src="images/files.png" alt="My files" width="100%"><br><sub><b>My files</b></sub></td>
<td align="center" width="50%"><img src="images/photos.png" alt="Photos timeline" width="100%"><br><sub><b>Photos</b></sub></td>
</tr>
<tr>
<td align="center" width="50%"><img src="images/shared.png" alt="Shared Links" width="100%"><br><sub><b>Shared Links</b></sub></td>
<td align="center" width="50%"><img src="images/shared_with_me.png" alt="Shared with me" width="100%"><br><sub><b>Shared with me</b></sub></td>
<td align="center" width="50%"><img src="images/photos_map.png" alt="Photos places map" width="100%"><br><sub><b>Places</b></sub></td>
<td align="center" width="50%"><img src="images/computers.png" alt="Computers" width="100%"><br><sub><b>Computers</b></sub></td>
</tr>
<tr>
<td align="center" width="50%"><img src="images/restore_devices.png" alt="Local Backups (Computers)" width="100%"><br><sub><b>Local Backups (Computers)</b></sub></td>
<td align="center" width="50%"><img src="images/photos.png" alt="Photos Timeline" width="100%"><br><sub><b>Photos Timeline</b></sub></td>
<td align="center" width="50%"><img src="images/sync.png" alt="Sync folders" width="100%"><br><sub><b>Sync</b></sub></td>
<td align="center" width="50%"><img src="images/conflicts.png" alt="Sync history showing resolved conflicts" width="100%"><br><sub><b>Sync history</b></sub></td>
</tr>
<tr>
<td align="center" width="50%"><img src="images/prompt_gtk.png" alt="Search Launcher Prompt (GTK)" width="100%"><br><sub><b>Search Launcher Prompt (GTK)</b></sub></td>
<td align="center" width="50%"><img src="images/prompt_fzf.png" alt="Search Launcher Prompt (fzf)" width="100%"><br><sub><b>Search Launcher Prompt (fzf)</b></sub></td>
<td align="center" width="50%"><img src="images/prompt_gtk.png" alt="Search launcher" width="100%"><br><sub><b>Search launcher</b></sub></td>
<td align="center" width="50%"><img src="images/prompt_fzf.png" alt="Search launcher in fzf" width="100%"><br><sub><b>Search launcher in fzf</b></sub></td>
</tr>
<tr>
<td align="center" width="50%"><img src="images/activity.png" alt="Activity" width="100%"><br><sub><b>Activity</b></sub></td>
<td align="center" width="50%"><img src="images/settings.png" alt="Settings" width="100%"><br><sub><b>Settings</b></sub></td>
<td align="center" width="50%"><img src="images/login.png" alt="Sign in" width="100%"><br><sub><b>Sign in</b></sub></td>
<td></td>
</tr>
</table>
---
## How it works
## Prerequisites
A background service (`proton-drive.service`, the `pdfs daemon` command) holds your session,
mounts `~/ProtonDrive` through FUSE, runs the upload queue and the sync engine, and follows
Proton's event stream for changes made elsewhere. The app, the tray, the launcher and the CLI are
clients of that service over a local socket. Everything is end-to-end encrypted with your Proton
keys before it leaves the machine, through a Rust implementation of Proton's Drive SDK.
To compile the application from source or run the built binaries, ensure you have the following system libraries installed on your distribution:
### Ubuntu / Debian (24.04+)
```bash
sudo apt-get update
sudo apt-get install -y \
pkg-config \
libfuse3-dev \
libgtk-4-dev \
libadwaita-1-dev \
libwebkitgtk-6.0-dev \
libsecret-1-dev \
libdbus-1-dev \
libimage-exiftool-perl
```
### Arch Linux
```bash
sudo pacman -S --needed pkgconf fuse3 gtk4 libadwaita libsecret dbus webkitgtk-6.0 perl-image-exiftool
```
### Fedora (44+)
```bash
sudo dnf install -y \
pkgconf-pkg-config fuse3-devel gtk4-devel libadwaita-devel \
webkitgtk6.0-devel libsecret-devel dbus-devel glib2-devel \
perl-Image-ExifTool cargo rust
```
Runtime extras (pick your desktop):
```bash
# GNOME — keyring + tray (AppIndicator)
sudo dnf install -y gnome-keyring gnome-shell-extension-appindicator xdg-utils
# KDE Plasma — KWallet (Secret Service); tray works via built-in SNI
sudo dnf install -y kwallet xdg-utils
```
---
## Building from Source
Ensure you have Rust and Cargo installed (minimum supported Rust version is 1.96).
1. Clone the repository and navigate into the project directory:
```bash
git clone https://github.com/narrrl/proton-drive-linux.git
cd proton-drive-linux
```
2. Build the workspace in release mode:
```bash
cargo build --release --locked
```
The compiled binaries will be available under `target/release/`:
- `pdfs`: The CLI utility.
- `pdfs-app`: The GTK4 application.
- `pdfs-tray`: The tray status notifier.
- `pdfs-prompt`: The launcher prompt for quick HUD search.
---
## Installation & Packages
### 1. Debian / Ubuntu (.deb)
Install the debian package via `dpkg` or `apt`:
```bash
sudo apt install ./proton-drive-linux_*.deb
```
### 2. Arch Linux (AUR)
Three AUR packages cover the usual choices; they conflict with each other, so
install one:
| Package | Contents |
| --- | --- |
| [`proton-drive-for-linux`](https://aur.archlinux.org/packages/proton-drive-for-linux) | builds the latest tagged release from source |
| [`proton-drive-for-linux-git`](https://aur.archlinux.org/packages/proton-drive-for-linux-git) | builds `main` from git |
| [`proton-drive-for-linux-bin`](https://aur.archlinux.org/packages/proton-drive-for-linux-bin) | installs the release binaries, no compile |
```bash
paru -S proton-drive-for-linux # or any AUR helper
```
The package sources live in `packaging/aur/`; see
[`packaging/aur/README.md`](packaging/aur/README.md) for how they are updated and
published. A local `PKGBUILD` that compiles the working tree in place is still
available for development builds:
```bash
cd packaging && makepkg -fi
```
### 3. Fedora (local RPM)
A local `.spec` is available under `packaging/`. From the repository root:
```bash
sudo dnf install -y rpm-build
rpmbuild -bb packaging/proton-drive-linux.spec \
--define "git_dir $PWD" \
--define "_rpmdir $PWD/packaging/out" \
--define "_builddir $PWD/packaging/build" \
--define "_sourcedir $PWD" \
--define "_specdir $PWD/packaging" \
--define "_srcrpmdir $PWD/packaging/out"
sudo dnf install packaging/out/x86_64/proton-drive-linux-*.rpm
```
---
## Automated Releases (CI/CD)
This project has a GitHub Actions CI workflow configured under `.github/workflows/release.yml`.
### How it works:
1. **Triggers**:
- Pushing a git tag matching `v*` (e.g. `git tag v0.1.0 && git push origin v0.1.0`).
- Manual runs via the **Actions** tab in GitHub (**workflow_dispatch**).
2. **Quality Gates and Build Process**:
- For tagged builds, verifies that the tag, workspace version, and `packaging/PKGBUILD` version agree.
- Runs `cargo fmt`, Clippy with warnings denied, the locked workspace test suite, and the account-free FUSE acceptance contract.
- Spawns an Ubuntu runner and installs GTK4, Libadwaita, FUSE3, and Secret Service packages.
- Sets up the Rust compiler and caches build targets to speed up runs.
- Compiles the workspace members in release mode.
3. **Artifact Packaging**:
- Generates a `.tar.gz` containing the raw binaries (`pdfs`, `pdfs-app`, `pdfs-tray`, `pdfs-prompt`).
- Packs them into Debian (`.deb`) and Fedora (`.rpm`) packages.
- Includes the systemd user service and tray autostart entry in the Debian package.
4. **Publishing**:
- Creates a GitHub Release matching the pushed tag and uploads the `.deb`, `.rpm`, and `.tar.gz` packages as release assets.
- For manual runs, compiles and exposes the packages as workflow run artifacts for testing.
---
Files you open are decrypted into a local cache so programs can read them. What that means for
the security of the machine is spelled out in the
[threat model](docs/ARCHITECTURE.md#10-threat-model-what-this-client-writes-to-disk-in-plaintext).
## Contributing
Bug reports, packaging fixes, and patches are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for
the workflow and the quality gates CI enforces, and [SECURITY.md](SECURITY.md) for reporting a
vulnerability privately.
Bug reports, translations and patches are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for the
workflow and the checks CI runs, and [SECURITY.md](SECURITY.md) to report a vulnerability
privately.
The app, tray and search prompt are available in English, German, Spanish, French, Italian, Dutch,
Polish and Brazilian Portuguese. [docs/TRANSLATING.md](docs/TRANSLATING.md) explains how to improve
a translation or add a language.
## Support
Proton Drive Linux is free and built in spare time. If it saves you a few headaches, you can
[buy me a coffee](https://buymeacoffee.com/narl).
<a href="https://buymeacoffee.com/narl"><img src="https://img.shields.io/badge/Buy%20me%20a%20coffee-FFDD00?logo=buymeacoffee&logoColor=black" alt="Buy me a coffee"></a>
## License
Released under the [MIT License](LICENSE).
The Places map uses [GeoNames](https://www.geonames.org/) data (CC BY 4.0) and
[Natural Earth](https://www.naturalearthdata.com/) outlines (public domain). The optional street
map comes from [OpenFreeMap](https://openfreemap.org/), © [OpenStreetMap](https://www.openstreetmap.org/copyright)
contributors.
Proton Drive is a service of Proton AG. This project is an independent client and carries no
affiliation with or endorsement from Proton AG.
+5
View File
@@ -30,5 +30,10 @@ the interesting boundaries are:
- **Cryptography.** Key derivation, signature handling, and anything that could publish plaintext
or the wrong key material to Proton.
What the client deliberately keeps in plaintext on disk, and why, is documented in the
[threat model](docs/ARCHITECTURE.md#10-threat-model-what-this-client-writes-to-disk-in-plaintext).
Behaviour described there is known and not a vulnerability by itself; a way to make it worse
than described is.
Out of scope: vulnerabilities in Proton's own service (report those to Proton AG), and issues that
require an attacker who already has your user account on the machine.
+212 -20
View File
@@ -14,12 +14,45 @@ use pdfs_core::cache::ContentCache;
use pdfs_core::config::AppDirs;
use pdfs_core::control::{
ConflictKeep, ErrorKind, MountKind, RefreshScope, Request as CtlRequest,
Response as CtlResponse, RestoreItem as CtlRestoreItem, ShareEntryKind, SyncPhase,
Response as CtlResponse, RestoreItem as CtlRestoreItem, ShareEntryKind, SyncIssue, SyncPhase,
pending_summary,
};
use pdfs_core::db::Db;
use pdfs_core::service;
/// `println!` for this binary, shadowing std's, which panics when stdout is
/// closed: `pdfs ls | head` ended in a "Broken pipe" panic once `head` had its
/// lines. This one hands the failure to [`stdout_written`] instead.
///
/// Restoring the default `SIGPIPE` would also stop the panic, but the same
/// binary runs the daemon, which must not die when a socket's peer goes away.
macro_rules! println {
($($arg:tt)*) => {{
use std::io::Write as _;
$crate::stdout_written(writeln!(std::io::stdout(), $($arg)*))
}};
}
/// [`println!`] without the newline.
macro_rules! print {
($($arg:tt)*) => {{
use std::io::Write as _;
$crate::stdout_written(write!(std::io::stdout(), $($arg)*))
}};
}
/// Exit on a failed write to stdout. A closed pipe means the reader has all it
/// wanted, so that exit is a success.
fn stdout_written(result: std::io::Result<()>) {
if let Err(error) = result {
if error.kind() == std::io::ErrorKind::BrokenPipe {
std::process::exit(0);
}
eprintln!("Error: writing to stdout: {error}");
std::process::exit(1);
}
}
#[derive(Parser)]
#[command(
name = "pdfs",
@@ -30,7 +63,7 @@ struct Cli {
/// Emit machine-readable JSON instead of formatted text.
///
/// Applies to the query commands (`status`, `ls`, `pins`, `sync list`,
/// `devices list`, `locations`, `transfers`, `activity`, `cache inspect`). Commands that
/// `devices list`, `locations`, `transfers`, `activity`, `trash`, `cache inspect`). Commands that
/// perform an action keep their human output — a script that needs to know
/// whether one succeeded has the exit code.
#[arg(long, global = true)]
@@ -285,26 +318,34 @@ enum Command {
},
/// Rename a file or folder via the running daemon.
Rename {
/// File/folder path, inside the mountpoint or relative to it.
/// File/folder path, inside My files or an on-demand synced folder, or
/// relative to the mountpoint.
path: PathBuf,
/// New name (a single path component).
new_name: String,
},
/// Move a file or folder into another folder via the running daemon.
/// Move files or folders into another folder via the running daemon.
///
/// The paths may be in different locations: My files and any synced
/// folder, on-demand or mirrored. The move happens on Proton Drive, so
/// nothing is downloaded or uploaded again, unlike `mv` between two
/// locations. The last path is the destination folder.
Move {
/// File/folder path, inside the mountpoint or relative to it.
path: PathBuf,
/// Destination folder path, inside the mountpoint or relative to it.
new_parent: PathBuf,
/// Files/folders to move, then the destination folder: each inside a
/// location, or relative to the mountpoint.
#[arg(required = true, num_args = 2.., value_name = "PATH")]
paths: Vec<PathBuf>,
},
/// Trash a file or folder via the running daemon.
Rm {
/// File/folder path, inside the mountpoint or relative to it.
/// File/folder path, inside My files or an on-demand synced folder, or
/// relative to the mountpoint.
path: PathBuf,
},
/// Create a folder via the running daemon.
Mkdir {
/// Parent folder path, inside the mountpoint or relative to it.
/// Parent folder path, inside My files or an on-demand synced folder,
/// or relative to the mountpoint.
parent: PathBuf,
/// New folder name (a single path component).
name: String,
@@ -636,6 +677,25 @@ enum SyncCmd {
},
/// List uploads and changes not yet on Proton Drive.
Queue,
/// List queued changes Proton Drive refused or that keep failing, and what to do.
Issues,
/// Save the content of a queued upload (by id from `sync queue`) to a file.
Export {
/// Queue id.
id: i64,
/// Where to save it: a new file, or an existing folder to save it in.
dest: PathBuf,
},
/// Drop a queued change (by id from `sync queue`) and undo it here: a new file
/// or folder goes, anything else goes back to what Proton Drive has. Its
/// content is lost unless saved with `sync export` first.
Discard {
/// Queue id.
id: i64,
/// Do not ask first.
#[arg(long)]
yes: bool,
},
/// Retry a queued operation now (by id from `sync queue`), or every failed one.
Retry {
/// Queue id; omit to retry every failed operation.
@@ -800,7 +860,10 @@ fn main() -> Result<()> {
scope,
} => cmd_search(query, limit, scope),
Command::Rename { path, new_name } => cmd_rename(path, new_name),
Command::Move { path, new_parent } => cmd_move(path, new_parent),
Command::Move { mut paths } => {
let new_parent = paths.pop().expect("clap requires at least two paths");
cmd_move(paths, new_parent)
}
Command::Rm { path } => cmd_rm(path),
Command::Mkdir { parent, name } => cmd_mkdir(parent, name),
Command::Upload { sources, parent } => cmd_upload(sources, parent),
@@ -984,7 +1047,17 @@ fn cmd_sync(action: SyncCmd) -> Result<()> {
})?)?
}
SyncCmd::Queue => return cmd_sync_queue(),
SyncCmd::Issues => return cmd_sync_issues(),
SyncCmd::Export { id, dest } => return cmd_sync_export(id, &dest),
SyncCmd::Limit { up, down } => return cmd_sync_limit(up, down),
SyncCmd::Discard { id, yes } => {
if !yes {
confirm(&format!(
"Discard queued change {id} and undo it here? Content not exported is lost."
))?;
}
ok_or_bail(control_request(CtlRequest::DiscardPendingOp { id })?)?
}
SyncCmd::Retry { id } => ok_or_bail(control_request(CtlRequest::RetryPendingOp { id })?)?,
SyncCmd::Resume { folder: None } => {
ok_or_bail(control_request(CtlRequest::SetSyncPaused {
@@ -1062,6 +1135,9 @@ fn cmd_sync_queue() -> Result<()> {
(false, _) => "due now".to_string(),
};
println!("[{}] {} {} ({state})", op.id, op.kind, op.path);
if let Some(issue) = op.issue {
println!(" needs attention: {}", issue_text(issue));
}
if let Some(error) = &op.last_error {
println!(" failed {} time(s): {error}", op.attempts);
}
@@ -1073,6 +1149,91 @@ fn cmd_sync_queue() -> Result<()> {
Ok(())
}
/// `pdfs sync issues`: the queued changes that need the user, and what to do.
fn cmd_sync_issues() -> Result<()> {
let response = control_request(CtlRequest::ListPendingOps)?;
let items = match response {
CtlResponse::PendingOps { items } => items,
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
};
let issues: Vec<_> = items.into_iter().filter(|op| op.failing).collect();
if json_enabled() {
println!("{}", serde_json::to_string_pretty(&issues)?);
return Ok(());
}
if issues.is_empty() {
println!("No sync issues.");
return Ok(());
}
for op in issues {
println!("[{}] {} {}", op.id, op.kind, op.path);
match op.issue {
Some(issue) => println!(" {}", issue_text(issue)),
None => println!(" keeps failing; it is retried."),
}
if let Some(error) = &op.last_error {
println!(" last error: {error}");
}
if op.exportable {
println!(
" save a copy: pdfs sync export {} <file or folder>",
op.id
);
}
}
Ok(())
}
/// What a sync issue means, and what the user can do about it.
fn issue_text(issue: SyncIssue) -> &'static str {
match issue {
SyncIssue::Quota => {
"Proton Drive storage is full. Free up space or upgrade; the change is retried."
}
SyncIssue::Access => {
"you can no longer change this on Proton Drive, for example because a share was \
revoked. Save a copy to keep the content."
}
SyncIssue::Missing => {
"the file or its folder is gone from Proton Drive. Save a copy to keep the content."
}
SyncIssue::Limit => {
"the folder holds too many items or is nested too deep. Move the file to another \
folder."
}
SyncIssue::Rejected => {
"Proton Drive rejected the change, for example the name. Rename the file."
}
SyncIssue::Unknown => "Proton Drive refused the change.",
}
}
/// `pdfs sync export`: save a queued upload's content. A folder as `dest`
/// gets the file under its own name.
fn cmd_sync_export(id: i64, dest: &Path) -> Result<()> {
let mut dest =
std::path::absolute(dest).with_context(|| format!("resolving {}", dest.display()))?;
if dest.is_dir() {
let items = match control_request(CtlRequest::ListPendingOps)? {
CtlResponse::PendingOps { items } => items,
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
};
let Some(op) = items.iter().find(|op| op.id == id) else {
bail!("no queued change {id}; see `pdfs sync queue`");
};
let name = Path::new(&op.path)
.file_name()
.with_context(|| format!("{} has no file name", op.path))?;
dest.push(name);
}
ok_or_bail(control_request(CtlRequest::ExportPendingOp {
id,
dest: dest.display().to_string(),
})?)
}
/// `pdfs sync restore`: list what this machine's device holds, confirm where
/// each folder should land, then hand the mapping to the daemon.
///
@@ -1939,6 +2100,7 @@ fn mount_once(mountpoint: Option<PathBuf>) -> Result<pdfs_fuse::MountOutcome> {
sweep_mode: config.resolved_conflict_sweep(),
upload_limit: config.upload_limit.unwrap_or(0),
download_limit: config.download_limit.unwrap_or(0),
local_first: config.local_first.unwrap_or(true),
},
);
@@ -1946,6 +2108,9 @@ fn mount_once(mountpoint: Option<PathBuf>) -> Result<pdfs_fuse::MountOutcome> {
if let Err(e) = rt.block_on(auth::persist(&session)) {
tracing::debug!(error = %e, "no new tokens to persist on shutdown");
}
// Dropping the runtime waits for every blocking task, however long; a call
// still stuck on Drive would hold the stop until systemd kills it (B165).
rt.shutdown_timeout(std::time::Duration::from_secs(2));
let outcome = result.context("mount failed")?;
Ok(outcome)
@@ -2074,6 +2239,7 @@ fn cmd_photos(limit: usize, offset: usize, favorites: bool) -> Result<()> {
kind: None,
range: None,
favorites,
not_in_album: false,
})? {
CtlResponse::Photos {
available: false, ..
@@ -2438,14 +2604,33 @@ fn cmd_rename(path: PathBuf, new_name: String) -> Result<()> {
Ok(())
}
fn cmd_move(path: PathBuf, new_parent: PathBuf) -> Result<()> {
match control_request(CtlRequest::Move {
path: path_arg(&path)?,
new_parent: path_arg(&new_parent)?,
})? {
CtlResponse::Ok { message } => println!("{message}"),
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
/// Move each of `sources` into `new_parent`, one request each. A refusal names
/// its source and the rest still move; the command fails if any did not.
fn cmd_move(sources: Vec<PathBuf>, new_parent: PathBuf) -> Result<()> {
let new_parent = path_arg(&new_parent)?;
let mut failed = 0;
for source in &sources {
let outcome = path_arg(source).and_then(|path| {
match control_request(CtlRequest::Move {
path,
new_parent: new_parent.clone(),
})? {
CtlResponse::Ok { message } => Ok(message),
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
});
match outcome {
Ok(message) => println!("{message}"),
Err(e) if sources.len() == 1 => return Err(e),
Err(e) => {
eprintln!("{}: {e:#}", source.display());
failed += 1;
}
}
}
if failed > 0 {
bail!("{failed} of {} could not be moved", sources.len());
}
Ok(())
}
@@ -2523,7 +2708,11 @@ fn cmd_diagnostics() -> Result<()> {
}
fn cmd_trash() -> Result<()> {
match control_request(CtlRequest::ListTrash)? {
let response = control_request(CtlRequest::ListTrash)?;
if emit_json(&response)? {
return Ok(());
}
match response {
CtlResponse::Entries { entries } if entries.is_empty() => println!("(trash is empty)"),
CtlResponse::Entries { entries } => {
for e in entries {
@@ -3363,6 +3552,9 @@ mod login_wait_tests {
shared_by: String::new(),
shared_at: 0,
shared_by_unverified: false,
trashed_at: 0,
trashed_from: None,
issue: None,
};
assert_eq!(shared_by_line(&entry), None, "no invitation, no line");
entry.shared_by = "alice@proton.me".into();
File diff suppressed because it is too large. Load diff
+250
View File
@@ -0,0 +1,250 @@
AD Andorra
AE United Arab Emirates
AF Afghanistan
AG Antigua and Barbuda
AI Anguilla
AL Albania
AM Armenia
AO Angola
AQ Antarctica
AR Argentina
AS American Samoa
AT Austria
AU Australia
AW Aruba
AX Åland Islands
AZ Azerbaijan
BA Bosnia and Herzegovina
BB Barbados
BD Bangladesh
BE Belgium
BF Burkina Faso
BG Bulgaria
BH Bahrain
BI Burundi
BJ Benin
BL Saint Barthélemy
BM Bermuda
BN Brunei Darussalam
BO Bolivia, Plurinational State of
BQ Bonaire, Sint Eustatius and Saba
BR Brazil
BS Bahamas
BT Bhutan
BV Bouvet Island
BW Botswana
BY Belarus
BZ Belize
CA Canada
CC Cocos (Keeling) Islands
CD Congo, The Democratic Republic of the
CF Central African Republic
CG Congo
CH Switzerland
CI Côte d'Ivoire
CK Cook Islands
CL Chile
CM Cameroon
CN China
CO Colombia
CR Costa Rica
CU Cuba
CV Cabo Verde
CW Curaçao
CX Christmas Island
CY Cyprus
CZ Czechia
DE Germany
DJ Djibouti
DK Denmark
DM Dominica
DO Dominican Republic
DZ Algeria
EC Ecuador
EE Estonia
EG Egypt
EH Western Sahara
ER Eritrea
ES Spain
ET Ethiopia
FI Finland
FJ Fiji
FK Falkland Islands (Malvinas)
FM Micronesia, Federated States of
FO Faroe Islands
FR France
GA Gabon
GB United Kingdom
GD Grenada
GE Georgia
GF French Guiana
GG Guernsey
GH Ghana
GI Gibraltar
GL Greenland
GM Gambia
GN Guinea
GP Guadeloupe
GQ Equatorial Guinea
GR Greece
GS South Georgia and the South Sandwich Islands
GT Guatemala
GU Guam
GW Guinea-Bissau
GY Guyana
HK Hong Kong
HM Heard Island and McDonald Islands
HN Honduras
HR Croatia
HT Haiti
HU Hungary
ID Indonesia
IE Ireland
IL Israel
IM Isle of Man
IN India
IO British Indian Ocean Territory
IQ Iraq
IR Iran, Islamic Republic of
IS Iceland
IT Italy
JE Jersey
JM Jamaica
JO Jordan
JP Japan
KE Kenya
KG Kyrgyzstan
KH Cambodia
KI Kiribati
KM Comoros
KN Saint Kitts and Nevis
KP Korea, Democratic People's Republic of
KR Korea, Republic of
KW Kuwait
KY Cayman Islands
KZ Kazakhstan
LA Lao People's Democratic Republic
LB Lebanon
LC Saint Lucia
LI Liechtenstein
LK Sri Lanka
LR Liberia
LS Lesotho
LT Lithuania
LU Luxembourg
LV Latvia
LY Libya
MA Morocco
MC Monaco
MD Moldova, Republic of
ME Montenegro
MF Saint Martin (French part)
MG Madagascar
MH Marshall Islands
MK North Macedonia
ML Mali
MM Myanmar
MN Mongolia
MO Macao
MP Northern Mariana Islands
MQ Martinique
MR Mauritania
MS Montserrat
MT Malta
MU Mauritius
MV Maldives
MW Malawi
MX Mexico
MY Malaysia
MZ Mozambique
NA Namibia
NC New Caledonia
NE Niger
NF Norfolk Island
NG Nigeria
NI Nicaragua
NL Netherlands
NO Norway
NP Nepal
NR Nauru
NU Niue
NZ New Zealand
OM Oman
PA Panama
PE Peru
PF French Polynesia
PG Papua New Guinea
PH Philippines
PK Pakistan
PL Poland
PM Saint Pierre and Miquelon
PN Pitcairn
PR Puerto Rico
PS Palestine, State of
PT Portugal
PW Palau
PY Paraguay
QA Qatar
RE Réunion
RO Romania
RS Serbia
RU Russian Federation
RW Rwanda
SA Saudi Arabia
SB Solomon Islands
SC Seychelles
SD Sudan
SE Sweden
SG Singapore
SH Saint Helena, Ascension and Tristan da Cunha
SI Slovenia
SJ Svalbard and Jan Mayen
SK Slovakia
SL Sierra Leone
SM San Marino
SN Senegal
SO Somalia
SR Suriname
SS South Sudan
ST Sao Tome and Principe
SV El Salvador
SX Sint Maarten (Dutch part)
SY Syrian Arab Republic
SZ Eswatini
TC Turks and Caicos Islands
TD Chad
TF French Southern Territories
TG Togo
TH Thailand
TJ Tajikistan
TK Tokelau
TL Timor-Leste
TM Turkmenistan
TN Tunisia
TO Tonga
TR Türkiye
TT Trinidad and Tobago
TV Tuvalu
TW Taiwan, Province of China
TZ Tanzania, United Republic of
UA Ukraine
UG Uganda
UM United States Minor Outlying Islands
US United States
UY Uruguay
UZ Uzbekistan
VA Holy See (Vatican City State)
VC Saint Vincent and the Grenadines
VE Venezuela, Bolivarian Republic of
VG Virgin Islands, British
VI Virgin Islands, U.S.
VN Viet Nam
VU Vanuatu
WF Wallis and Futuna
WS Samoa
XK Kosovo
YE Yemen
YT Mayotte
ZA South Africa
ZM Zambia
ZW Zimbabwe
1 AD Andorra
2 AE United Arab Emirates
3 AF Afghanistan
4 AG Antigua and Barbuda
5 AI Anguilla
6 AL Albania
7 AM Armenia
8 AO Angola
9 AQ Antarctica
10 AR Argentina
11 AS American Samoa
12 AT Austria
13 AU Australia
14 AW Aruba
15 AX Åland Islands
16 AZ Azerbaijan
17 BA Bosnia and Herzegovina
18 BB Barbados
19 BD Bangladesh
20 BE Belgium
21 BF Burkina Faso
22 BG Bulgaria
23 BH Bahrain
24 BI Burundi
25 BJ Benin
26 BL Saint Barthélemy
27 BM Bermuda
28 BN Brunei Darussalam
29 BO Bolivia, Plurinational State of
30 BQ Bonaire, Sint Eustatius and Saba
31 BR Brazil
32 BS Bahamas
33 BT Bhutan
34 BV Bouvet Island
35 BW Botswana
36 BY Belarus
37 BZ Belize
38 CA Canada
39 CC Cocos (Keeling) Islands
40 CD Congo, The Democratic Republic of the
41 CF Central African Republic
42 CG Congo
43 CH Switzerland
44 CI Côte d'Ivoire
45 CK Cook Islands
46 CL Chile
47 CM Cameroon
48 CN China
49 CO Colombia
50 CR Costa Rica
51 CU Cuba
52 CV Cabo Verde
53 CW Curaçao
54 CX Christmas Island
55 CY Cyprus
56 CZ Czechia
57 DE Germany
58 DJ Djibouti
59 DK Denmark
60 DM Dominica
61 DO Dominican Republic
62 DZ Algeria
63 EC Ecuador
64 EE Estonia
65 EG Egypt
66 EH Western Sahara
67 ER Eritrea
68 ES Spain
69 ET Ethiopia
70 FI Finland
71 FJ Fiji
72 FK Falkland Islands (Malvinas)
73 FM Micronesia, Federated States of
74 FO Faroe Islands
75 FR France
76 GA Gabon
77 GB United Kingdom
78 GD Grenada
79 GE Georgia
80 GF French Guiana
81 GG Guernsey
82 GH Ghana
83 GI Gibraltar
84 GL Greenland
85 GM Gambia
86 GN Guinea
87 GP Guadeloupe
88 GQ Equatorial Guinea
89 GR Greece
90 GS South Georgia and the South Sandwich Islands
91 GT Guatemala
92 GU Guam
93 GW Guinea-Bissau
94 GY Guyana
95 HK Hong Kong
96 HM Heard Island and McDonald Islands
97 HN Honduras
98 HR Croatia
99 HT Haiti
100 HU Hungary
101 ID Indonesia
102 IE Ireland
103 IL Israel
104 IM Isle of Man
105 IN India
106 IO British Indian Ocean Territory
107 IQ Iraq
108 IR Iran, Islamic Republic of
109 IS Iceland
110 IT Italy
111 JE Jersey
112 JM Jamaica
113 JO Jordan
114 JP Japan
115 KE Kenya
116 KG Kyrgyzstan
117 KH Cambodia
118 KI Kiribati
119 KM Comoros
120 KN Saint Kitts and Nevis
121 KP Korea, Democratic People's Republic of
122 KR Korea, Republic of
123 KW Kuwait
124 KY Cayman Islands
125 KZ Kazakhstan
126 LA Lao People's Democratic Republic
127 LB Lebanon
128 LC Saint Lucia
129 LI Liechtenstein
130 LK Sri Lanka
131 LR Liberia
132 LS Lesotho
133 LT Lithuania
134 LU Luxembourg
135 LV Latvia
136 LY Libya
137 MA Morocco
138 MC Monaco
139 MD Moldova, Republic of
140 ME Montenegro
141 MF Saint Martin (French part)
142 MG Madagascar
143 MH Marshall Islands
144 MK North Macedonia
145 ML Mali
146 MM Myanmar
147 MN Mongolia
148 MO Macao
149 MP Northern Mariana Islands
150 MQ Martinique
151 MR Mauritania
152 MS Montserrat
153 MT Malta
154 MU Mauritius
155 MV Maldives
156 MW Malawi
157 MX Mexico
158 MY Malaysia
159 MZ Mozambique
160 NA Namibia
161 NC New Caledonia
162 NE Niger
163 NF Norfolk Island
164 NG Nigeria
165 NI Nicaragua
166 NL Netherlands
167 NO Norway
168 NP Nepal
169 NR Nauru
170 NU Niue
171 NZ New Zealand
172 OM Oman
173 PA Panama
174 PE Peru
175 PF French Polynesia
176 PG Papua New Guinea
177 PH Philippines
178 PK Pakistan
179 PL Poland
180 PM Saint Pierre and Miquelon
181 PN Pitcairn
182 PR Puerto Rico
183 PS Palestine, State of
184 PT Portugal
185 PW Palau
186 PY Paraguay
187 QA Qatar
188 RE Réunion
189 RO Romania
190 RS Serbia
191 RU Russian Federation
192 RW Rwanda
193 SA Saudi Arabia
194 SB Solomon Islands
195 SC Seychelles
196 SD Sudan
197 SE Sweden
198 SG Singapore
199 SH Saint Helena, Ascension and Tristan da Cunha
200 SI Slovenia
201 SJ Svalbard and Jan Mayen
202 SK Slovakia
203 SL Sierra Leone
204 SM San Marino
205 SN Senegal
206 SO Somalia
207 SR Suriname
208 SS South Sudan
209 ST Sao Tome and Principe
210 SV El Salvador
211 SX Sint Maarten (Dutch part)
212 SY Syrian Arab Republic
213 SZ Eswatini
214 TC Turks and Caicos Islands
215 TD Chad
216 TF French Southern Territories
217 TG Togo
218 TH Thailand
219 TJ Tajikistan
220 TK Tokelau
221 TL Timor-Leste
222 TM Turkmenistan
223 TN Tunisia
224 TO Tonga
225 TR Türkiye
226 TT Trinidad and Tobago
227 TV Tuvalu
228 TW Taiwan, Province of China
229 TZ Tanzania, United Republic of
230 UA Ukraine
231 UG Uganda
232 UM United States Minor Outlying Islands
233 US United States
234 UY Uruguay
235 UZ Uzbekistan
236 VA Holy See (Vatican City State)
237 VC Saint Vincent and the Grenadines
238 VE Venezuela, Bolivarian Republic of
239 VG Virgin Islands, British
240 VI Virgin Islands, U.S.
241 VN Viet Nam
242 VU Vanuatu
243 WF Wallis and Futuna
244 WS Samoa
245 XK Kosovo
246 YE Yemen
247 YT Mayotte
248 ZA South Africa
249 ZM Zambia
250 ZW Zimbabwe
+147 -15
View File
@@ -91,7 +91,7 @@ const TOUCH_FLUSH_AT: usize = 256;
///
/// Public because it is the *contract* a block has to satisfy, not a detail of
/// this cache: the network read path validates against it too, so a block that
/// comes back short is refused before it can truncate a file (bugs.md B84).
/// comes back short is refused before it can truncate a file (`docs/BUGS.md` B84).
pub fn block_len(size: u64, idx: u64) -> u64 {
let start = idx.saturating_mul(BLOCK_SIZE);
size.saturating_sub(start).min(BLOCK_SIZE)
@@ -311,8 +311,7 @@ struct PinFile {
pins: BTreeMap<String, Pin>,
}
/// What a staged file actually contains, written beside it as `<name>.json`
/// (offline.md Phase 2).
/// What a staged file actually contains, written beside it as `<name>.json`.
///
/// A staged file is **not** necessarily valid whole-file content, and that is
/// the whole reason this exists. A partial overwrite commits by filling the
@@ -365,6 +364,14 @@ pub struct StagedWrite {
pub based_on: Option<Baseline>,
}
impl StagedWrite {
/// Whether `authored` ranges cover a file of `len` bytes, so the staged
/// bytes are its complete new content. An empty file has nothing to cover.
pub fn covers(authored: &[(u64, u64)], len: u64) -> bool {
len == 0 || authored == [(0, len)]
}
}
/// The identity of a remote revision, as far as we can observe it.
///
/// `revision_id` is the authoritative identity when present: the server assigns a
@@ -431,9 +438,9 @@ pub struct ContentCache {
/// Subdirectory for write-handle scratch files (disk-backed write buffers).
/// Emptied on open so a crashed run leaves no orphans.
scratch_dir: PathBuf,
/// Subdirectory holding the bytes of writes that have not been uploaded yet
/// (offline.md Phase 2/3) — every released write passes through here, since
/// the upload is a queued op performed later. Unlike `scratch_dir` this is
/// Subdirectory holding the bytes of writes that have not been uploaded yet.
/// Every released write passes through here, since the upload is a queued
/// op performed later. Unlike `scratch_dir` this is
/// **never** emptied on open: these are the only copy of content the user
/// authored, and the whole point is that they outlive the daemon.
staging_dir: PathBuf,
@@ -468,7 +475,7 @@ pub struct ContentCache {
thumb_bytes: AtomicU64,
/// Unified metadata DB. Its `cache_entries` table is the LRU index: every
/// store/read/evict updates it, and the budget enforcers query it instead of
/// scanning the cache directories (plan.md P4).
/// scanning the cache directories.
db: Arc<Db>,
/// LRU touches not yet written to `cache_entries`, keyed by cache key.
///
@@ -730,6 +737,60 @@ impl ContentCache {
Some(buf)
}
/// Write the whole content of `uid` at `(mtime, size)` to `dst` from the
/// cache alone, and report whether that worked.
///
/// A whole-file blob is cloned. Otherwise every block of the revision has to
/// be cached, under its recorded geometry or, failing that, the uniform one
/// the read path would use. Anything short of the full content leaves
/// nothing at `dst`, so a caller falls back to a download with a clean slate.
///
/// This is what lets a mirror folder take a file an on-demand mount has
/// already read, after a move between them, without fetching it again. The
/// trust is the read path's: the same `(mtime, size)` key, and blocks that
/// were validated when they were fetched.
pub fn copy_cached_to(&self, uid: &NodeUid, mtime: i64, size: u64, dst: &Path) -> bool {
if size == 0 {
return false;
}
if let Some(blob) = self.valid_blob(uid, mtime, size) {
if Self::clone_or_copy(&blob, dst).is_ok_and(|len| len == size) {
self.touch(Self::key(uid));
return true;
}
let _ = std::fs::remove_file(dst);
return false;
}
let geometry = self
.block_geometry(uid, mtime, size)
.unwrap_or_else(|| BlockGeometry::uniform(size));
if geometry.size() != size {
return false;
}
let spans = geometry.spans(0, size);
// Checked up front so a file with one block missing costs stats, not a
// copy of every block before it.
if !spans
.iter()
.all(|&span| self.has_block(uid, mtime, size, span))
{
return false;
}
let written = (|| -> Option<()> {
use std::io::Write;
let mut out = std::fs::File::create(dst).ok()?;
for span in spans {
out.write_all(&self.cached_block(uid, mtime, size, span)?)
.ok()?;
}
(out.metadata().ok()?.len() == size).then_some(())
})();
if written.is_none() {
let _ = std::fs::remove_file(dst);
}
written.is_some()
}
/// Store `bytes` as the cached content for `uid`, tagged with `(mtime,
/// size)`. The blob is written to a temp file then renamed so a concurrent
/// reader never sees a partial blob; the meta tag is written last so a
@@ -783,6 +844,9 @@ impl ContentCache {
}
let bytes = std::fs::metadata(&tmp)?.len();
std::fs::rename(&tmp, &blob)?;
// Linked from the blob already cached, the temp name was a second name
// of it, and the rename did nothing (`docs/BUGS.md` B187).
let _ = std::fs::remove_file(&tmp);
let meta = serde_json::to_vec(&Meta { mtime, size })?;
std::fs::write(self.meta_path(uid), meta)?;
self.db
@@ -924,7 +988,7 @@ impl ContentCache {
/// a miss ([`cached_block`](Self::cached_block) checks), so a bad block
/// cannot escape this cache even though this does not check. Validating a
/// block is the *reader's* job, at the point it is fetched — see
/// `Core::read_block` (bugs.md B84).
/// `Core::read_block` (`docs/BUGS.md` B84).
pub fn store_block(
&self,
uid: &NodeUid,
@@ -1426,9 +1490,8 @@ impl ContentCache {
/// This is what makes a write survive its upload: the caller is releasing a
/// write handle and would otherwise delete the file, so until the bytes are
/// on the remote, staging holds the only copy. Every dirty handle goes
/// through here — the upload is a queued op performed later (offline.md
/// Phase 3), and a staged file is also what a human can recover from if the
/// queue never drains.
/// through here — the upload is a queued op performed later, and a staged
/// file is also what a human can recover from if the queue never drains.
///
/// Falls back to a copy when the rename crosses a filesystem boundary; a
/// failure here means we could not save the bytes at all, so it is reported
@@ -2002,6 +2065,17 @@ mod tests {
NodeUid::new(VolumeId::from("vol"), LinkId::from(link))
}
#[test]
fn an_empty_write_covers_the_file_and_a_gap_does_not() {
assert!(
StagedWrite::covers(&[], 0),
"an empty file has nothing to cover"
);
assert!(StagedWrite::covers(&[(0, 7)], 7));
assert!(!StagedWrite::covers(&[(0, 3), (4, 7)], 7));
assert!(!StagedWrite::covers(&[], 7));
}
/// A unique temp directory removed on drop; avoids a dev-dependency.
struct TempDir(PathBuf);
impl TempDir {
@@ -2124,6 +2198,24 @@ mod tests {
assert_eq!(c.read_range(&u, 200, 5, 0, 5).unwrap(), b"newer");
}
/// A landed create and the revision after it can adopt the same staged
/// blob. The link to the temp name is then a second name of the cached
/// blob, and a rename between two names of one file does nothing: the
/// temp name stayed until the next start swept it (`docs/BUGS.md` B187).
#[test]
fn store_file_twice_from_the_same_blob_leaves_no_temp_file() {
let (c, d) = cache();
let u = uid("a");
let src = d.path().join("staged-blob");
std::fs::write(&src, b"staged").unwrap();
c.store_file(&u, 100, 6, &src).unwrap();
c.store_file(&u, 100, 6, &src).unwrap();
assert!(!c.blob_path(&u).with_extension("tmp").exists());
assert_eq!(c.read_range(&u, 100, 6, 0, 6).unwrap(), b"staged");
}
#[test]
fn reconcile_rebuilds_index_from_disk() {
let dir = TempDir::new();
@@ -2561,6 +2653,46 @@ mod tests {
assert!(c.cached_block(&u, 100, 5000, blk(5000, 0)).is_none());
}
#[test]
fn a_cached_blob_is_copied_out_whole() {
let (c, d) = cache();
let u = uid("blob");
c.store(&u, 7, 5, b"hello").unwrap();
let dst = d.path().join("out");
assert!(c.copy_cached_to(&u, 7, 5, &dst));
assert_eq!(std::fs::read(&dst).unwrap(), b"hello");
// Another revision is not this one.
assert!(!c.copy_cached_to(&u, 8, 5, &d.path().join("stale")));
assert!(!d.path().join("stale").exists());
}
#[test]
fn cached_blocks_are_copied_out_in_order() {
let (c, d) = cache();
let u = uid("blocks");
let size = BLOCK_SIZE + 3;
let first = vec![1_u8; BLOCK_SIZE as usize];
c.store_block(&u, 7, size, blk(size, 0), &first).unwrap();
c.store_block(&u, 7, size, blk(size, 1), b"end").unwrap();
let dst = d.path().join("out");
assert!(c.copy_cached_to(&u, 7, size, &dst));
let bytes = std::fs::read(&dst).unwrap();
assert_eq!(bytes.len() as u64, size);
assert_eq!(&bytes[..BLOCK_SIZE as usize], first.as_slice());
assert_eq!(&bytes[BLOCK_SIZE as usize..], b"end");
}
#[test]
fn a_file_with_a_block_missing_is_not_copied_out() {
let (c, d) = cache();
let u = uid("partial");
let size = BLOCK_SIZE + 3;
c.store_block(&u, 7, size, blk(size, 1), b"end").unwrap();
let dst = d.path().join("out");
assert!(!c.copy_cached_to(&u, 7, size, &dst));
assert!(!dst.exists(), "a partial copy must not be left behind");
}
/// The [`BlockSpan`] the uniform geometry gives block `idx` of a file of
/// `size` bytes — the shape these tests were written against, and the one
/// this client's own uploads still have.
@@ -2573,7 +2705,7 @@ mod tests {
}
/// The geometry the whole read path validates against, here and over the
/// network (bugs.md B84): full blocks, then whatever is left.
/// network (`docs/BUGS.md` B84): full blocks, then whatever is left.
#[test]
fn block_len_is_full_blocks_then_the_remainder() {
let size = BLOCK_SIZE * 2 + 17;
@@ -2653,7 +2785,7 @@ mod tests {
/// A cached block is addressed by the range it covers, not by its index
/// alone. Re-planning a file onto its real geometry must therefore miss the
/// blocks the uniform fallback cached, rather than serve their bytes at the
/// wrong offset (bugs.md B85).
/// wrong offset (`docs/BUGS.md` B85).
#[test]
fn a_cached_block_is_not_served_to_a_different_geometry() {
let (c, _d) = cache();
@@ -2720,7 +2852,7 @@ mod tests {
/// A block of the wrong length cannot escape the cache even though the
/// store does not check: the read validates it and reports a miss, so the
/// next read refetches rather than serving a hole (bugs.md B84).
/// next read refetches rather than serving a hole (`docs/BUGS.md` B84).
#[test]
fn a_block_of_the_wrong_length_reads_as_a_miss() {
let (c, _d) = cache();
@@ -2785,7 +2917,7 @@ mod tests {
);
}
/// A failed upload must never cost the user their bytes (offline.md Phase 2):
/// A failed upload must never cost the user their bytes:
/// the scratch file moves to staging intact, and staging survives a reopen
/// the way scratch deliberately does not.
#[test]
+38 -11
View File
@@ -128,11 +128,12 @@ pub struct AppConfig {
/// field.
#[serde(default)]
pub prompt: Option<PromptConfig>,
/// Paint the desktop app in Proton purple instead of the system accent
/// colour. `None` means "follow the system". Defaulted for configs
/// predating the field.
#[serde(default)]
pub proton_accent: Option<bool>,
/// Paint the desktop app in Proton's colours instead of the system theme
/// and accent colour. `None` means "follow the system". Defaulted for
/// configs predating the field; also read from `proton_accent`, its name
/// while it only changed the accent.
#[serde(default, alias = "proton_accent")]
pub proton_theme: Option<bool>,
/// How the desktop app's My Files page lays out and orders a folder.
/// Defaulted for configs predating the field.
#[serde(default)]
@@ -147,6 +148,19 @@ pub struct AppConfig {
/// Defaulted for configs predating the field.
#[serde(default)]
pub language: Option<String>,
/// Draw the Places map from OpenFreeMap's street map instead of the
/// built-in outline. Off by default: the tiles come from a third party,
/// which then learns which parts of the world are looked at. Defaulted for
/// configs predating the field.
#[serde(default)]
pub online_map: bool,
/// Record a new folder, a new file, a rename or a delete through the mount
/// locally and send it from the queue, instead of waiting for Drive in the
/// syscall. `None` means on. `false` brings back the old path for one
/// release (docs/ROADMAP.md). Defaulted for configs predating
/// the field.
#[serde(default)]
pub local_first: Option<bool>,
}
/// The My Files page's view choices, remembered between runs.
@@ -216,10 +230,12 @@ impl Default for AppConfig {
conflict_sweep: None,
open_with: None,
prompt: None,
proton_accent: None,
proton_theme: None,
files_view: FilesView::default(),
tray_hidden: false,
language: None,
online_map: false,
local_first: None,
}
}
}
@@ -416,7 +432,7 @@ impl AppDirs {
/// typical umask makes them `0755`. Until this was fixed, the only thing
/// keeping another local user out was `~/.cache` and `~/.local/state`
/// happening to be `0700`, which is a convention of the user's system and
/// not something this client established (bugs.md B6).
/// not something this client established (`docs/BUGS.md` B6).
///
/// Applied on every start, not just at creation: a directory that already
/// exists with a permissive mode — restored from a backup that flattened
@@ -467,7 +483,7 @@ fn restrict_dir(dir: &Path) -> Result<()> {
/// session — enumerate, read, upload, trash, share — without presenting any
/// credential, since the daemon already holds one. It is an authority boundary,
/// not merely private data, and `UnixListener::bind` applies the umask like any
/// other file (bugs.md B6).
/// other file (`docs/BUGS.md` B6).
///
/// There is a window between `bind` and this call during which the socket
/// carries the umask's mode. It is closed in practice by the containing
@@ -488,7 +504,7 @@ mod tests {
std::fs::metadata(p).unwrap().permissions().mode() & 0o777
}
/// bugs.md B6. A permissive mode here is not a cosmetic problem: these
/// `docs/BUGS.md` B6. A permissive mode here is not a cosmetic problem: these
/// directories hold decrypted content, a plaintext index of every node name,
/// and the control socket that commands the daemon's session.
#[test]
@@ -566,7 +582,7 @@ mod tests {
conflict_sweep: Some(SweepMode::Enforce),
open_with: None,
prompt: None,
proton_accent: Some(true),
proton_theme: Some(true),
files_view: FilesView {
list: true,
sort: FileSort::Modified,
@@ -575,13 +591,16 @@ mod tests {
},
tray_hidden: true,
language: Some("de".to_string()),
online_map: true,
local_first: Some(false),
};
let json = serde_json::to_string(&config).unwrap();
let decoded: AppConfig = serde_json::from_str(&json).unwrap();
assert_eq!(decoded.app_version, "external-drive-test-client@1.0.0");
assert_eq!(decoded.user_agent, "test-agent/1.0");
assert_eq!(decoded.proton_accent, Some(true));
assert_eq!(decoded.proton_theme, Some(true));
assert!(decoded.tray_hidden);
assert!(decoded.online_map);
assert_eq!(decoded.language.as_deref(), Some("de"));
assert_eq!(decoded.upload_limit, Some(500_000));
assert_eq!(decoded.download_limit, None);
@@ -598,6 +617,14 @@ mod tests {
assert!(partial.folders_first);
}
#[test]
fn the_old_proton_accent_setting_turns_on_the_proton_theme() {
let decoded: AppConfig =
serde_json::from_str(r#"{"app_version":"a","user_agent":"b","proton_accent":true}"#)
.unwrap();
assert_eq!(decoded.proton_theme, Some(true));
}
#[test]
fn test_default_app_config() {
let default_config = AppConfig::default();
+632 -17
View File
@@ -31,6 +31,13 @@ const WRITE_TIMEOUT: Duration = Duration::from_secs(2);
/// answers.
const READ_TIMEOUT: Duration = Duration::from_secs(120);
/// Cap on the read half for the requests front ends poll ([`Request::is_poll`]).
/// The daemon answers those from memory or its database, so a reply that takes
/// longer than this means it is stuck. Waiting the full [`READ_TIMEOUT`] would
/// freeze the poller's read-out for two minutes; failing fast lets it say "not
/// responding" and ask again on its next tick.
const POLL_READ_TIMEOUT: Duration = Duration::from_secs(5);
/// A command sent from the CLI to the daemon.
#[derive(Serialize, Deserialize, Debug)]
pub enum Request {
@@ -67,6 +74,10 @@ pub enum Request {
/// get the whole timeline, as before.
#[serde(default)]
favorites: bool,
/// Restrict the page to photos no album of ours holds — what is left to
/// file. Older front-ends omit it and get the whole timeline, as before.
#[serde(default)]
not_in_album: bool,
},
/// Every photo of one photo's group — the JPEG and the RAW of one shot, a
/// live photo and its clip. Replies with [`Response::Photos`], holding one
@@ -80,6 +91,34 @@ pub enum Request {
#[serde(default)]
kind: Option<PhotoKind>,
},
/// Photos taken on today's date in earlier years, newest first, at most
/// `limit` of them — the "On this day" strip above the timeline. Replies
/// with [`Response::Photos`]; a front-end groups them by year itself.
PhotosOnThisDay { limit: usize },
/// Sets of photos that are byte-identical copies of each other, going by
/// the server's content hash. Replies with [`Response::PhotoDuplicates`].
PhotoDuplicates,
/// Sets of shots that look alike going by their thumbnails: resized or
/// re-encoded copies, and bursts of one moment. Starts hashing the
/// thumbnails not hashed yet in the background and replies at once with
/// the sets found so far; a front-end asks again until the reply's
/// `hashing` is `None`. Replies with [`Response::PhotoSimilar`].
PhotoSimilar,
/// The towns the photos were taken in, the one with the most photos first.
/// Only photos whose uploader recorded a location count. Replies with
/// [`Response::Places`].
PhotoPlaces,
/// Fetch a page of the photos taken in one town, newest first. `id` is the
/// town's id from [`Request::PhotoPlaces`]. Replies with
/// [`Response::Photos`], like an album page.
PlacePhotos {
id: u32,
offset: usize,
limit: usize,
},
/// Every photo taken in one town with where exactly it was taken, for the
/// map's close-up. Replies with [`Response::PhotoLocations`].
PlaceLocations { id: u32 },
/// List the account's photo albums, newest activity first, including the
/// albums other people share with us. Metadata only — an album's cover
/// thumbnail is fetched with [`Request::PhotoThumbs`] like any other photo.
@@ -126,8 +165,14 @@ pub enum Request {
///
/// A photo that belongs to a group takes the whole group with it: trashing
/// the JPEG of a shot and leaving its RAW behind is not what anyone means by
/// deleting that photo.
TrashNodes { uids: Vec<String> },
/// deleting that photo. With `files_only` set, exactly the listed files
/// go and their groups stay: the duplicate finder removes one copy of a
/// shot, not the shot. Older front-ends omit it and trash whole groups.
TrashNodes {
uids: Vec<String>,
#[serde(default)]
files_only: bool,
},
/// Fetch thumbnails for ordinary Drive image files shown outside the Photos
/// timeline. The modification time is the cache validity tag, so replacing
/// an image can never reuse its previous revision's thumbnail.
@@ -294,7 +339,10 @@ pub enum Request {
/// single path component (no separators). Replies with [`Response::Ok`].
Rename { path: String, new_name: String },
/// Move a file or folder into a new parent folder. Both `path` and
/// `new_parent` are mountpoint-relative. Replies with [`Response::Ok`].
/// `new_parent` are mountpoint-relative, or both absolute local paths; two
/// absolute paths may lie in different locations (My files, an on-demand
/// or a mirror synced folder), and the move is still made on Proton Drive
/// without transferring content. Replies with [`Response::Ok`].
Move { path: String, new_parent: String },
/// Trash a file or folder. `path` is mountpoint-relative. Replies with
/// [`Response::Ok`].
@@ -395,11 +443,11 @@ pub enum Request {
DeleteDevice { uid: String },
/// Adopt an existing device as this machine's, pinning its uid in
/// `config.json` so a hostname change or reinstall re-attaches to it instead
/// of registering a duplicate (features.md 5.1). `uid: None` clears the pin.
/// of registering a duplicate. `uid: None` clears the pin.
/// Replies with [`Response::Ok`].
AdoptDevice { uid: Option<String> },
// ---- device folder sync (devices.md) ----------------------------------
// ---- device folder sync ----------------------------------
/// Add a local folder to this machine's device, uploading its tree and
/// registering the device on first use. Replies with [`Response::Ok`].
AddSyncFolder { local_path: String },
@@ -429,6 +477,15 @@ pub enum Request {
/// Retry a backed-off queued op now instead of waiting out its backoff;
/// every failed op when `id` is `None`. Replies with [`Response::Ok`].
RetryPendingOp { id: Option<i64> },
/// Copy the content of a queued upload (`id` from
/// [`Request::ListPendingOps`]) to `dest`, an absolute path that must not
/// exist yet. The op stays queued. Replies with [`Response::Ok`].
ExportPendingOp { id: i64, dest: String },
/// Drop a queued change (`id` from [`Request::ListPendingOps`]) and undo
/// it locally: a node made here goes, a node Drive has goes back to what
/// Drive has, along with every other change queued on it. Its content is
/// lost unless it was exported first. Replies with [`Response::Ok`].
DiscardPendingOp { id: i64 },
/// List the `(sync-conflict …)` copies under My Files and in synced
/// folders. Replies with
/// [`Response::Conflicts`].
@@ -439,7 +496,7 @@ pub enum Request {
/// [`Response::Ok`].
ResolveConflict { path: String, keep: ConflictKeep },
/// List the folders under this machine's device that can be synced here,
/// each with a proposed local path (features.md 5.2). Replies with
/// each with a proposed local path. Replies with
/// [`Response::RestorableFolders`].
ListRestorableFolders,
/// Attach the given remote device folders to local paths and sync them down.
@@ -634,8 +691,88 @@ pub enum Request {
/// passes, …), so a front-end can show a running "what happened" feed
/// without re-deriving it from anywhere.
ListActivity { limit: usize },
// ---- push -------------------------------------------------------------
/// Keep the connection open and stream [`Event`] lines instead of one
/// [`Response`]. See [`subscribe`] for the client side.
Subscribe,
}
/// Something a front-end shows that the daemon can say has changed, so the
/// front-end reads it again. Carried by [`Event::Changed`].
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
#[serde(rename_all = "snake_case")]
pub enum Topic {
/// Folder listings in the mount: something was created, renamed, moved,
/// trashed or pinned, here or on another device.
Files,
/// The photos timeline or albums.
Photos,
Trash,
/// Shares, links, invitations and saved links, in either direction.
Shares,
/// The account's computers.
Devices,
/// Synced folders, their mode and their live progress.
Locations,
/// The pending-op queue ([`Request::ListPendingOps`]).
Queue,
Conflicts,
/// The activity log gained entries.
Activity,
/// A topic from a newer daemon, which this front-end cannot act on.
#[serde(other)]
Unknown,
}
impl Topic {
/// Every topic a front-end can act on. A new subscription is told all of
/// them have changed, so it catches up on whatever happened while it was
/// not connected.
pub const ALL: [Topic; 9] = [
Topic::Files,
Topic::Photos,
Topic::Trash,
Topic::Shares,
Topic::Devices,
Topic::Locations,
Topic::Queue,
Topic::Conflicts,
Topic::Activity,
];
}
/// One line on a [`Request::Subscribe`] connection.
///
/// The two things front ends show continuously, status and transfers, come
/// with their contents, so following them costs no requests. Everything else is
/// a [`Topic`] to re-read.
#[derive(Serialize, Deserialize, Debug)]
pub enum Event {
/// A fresh [`Response::Status`]. Sent when a subscription starts and
/// whenever any of its fields changes.
Status(Box<Response>),
/// A fresh [`Response::Transfers`] snapshot. Sent when a subscription
/// starts and whenever the snapshot changes, which is continuously while
/// something moves.
Transfers {
items: Vec<TransferItem>,
jobs: Vec<JobItem>,
},
/// These topics changed. Changes close together arrive as one event.
Changed { topics: Vec<Topic> },
/// Nothing happened. Sent every [`HEARTBEAT`] so a client can tell a quiet
/// daemon from a stuck one.
Heartbeat,
}
/// How often a quiet subscription sends [`Event::Heartbeat`].
pub const HEARTBEAT: Duration = Duration::from_secs(10);
/// How long [`Subscription::next_event`] waits for a line before it gives the
/// daemon up as stuck: a few missed heartbeats.
const SUBSCRIBE_READ_TIMEOUT: Duration = Duration::from_secs(25);
/// Which kind of share entry a [`ShareEntry`] is, and which collection an
/// [`Request::UpdateShareRole`]/[`Request::RemoveShareEntry`] targets.
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
@@ -701,7 +838,7 @@ pub struct DeviceInfo {
/// One `(sync-conflict …)` copy and the file it is a copy of (in
/// [`Response::Conflicts`]).
#[derive(Serialize, Deserialize, Debug, Clone)]
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
pub struct ConflictInfo {
/// Mountpoint-relative path of the copy, or its absolute local path when it
/// sits in a synced folder.
@@ -752,6 +889,59 @@ pub struct PendingOpInfo {
pub parked: bool,
/// Failed often enough to count as stuck.
pub failing: bool,
/// Why Drive refused it, when it did; such an op needs attention at once.
#[serde(default)]
pub issue: Option<SyncIssue>,
/// Carries file content that [`Request::ExportPendingOp`] can save.
#[serde(default)]
pub exportable: bool,
}
/// Why Drive refused a queued change (in [`PendingOpInfo`]). The change stays
/// queued and keeps being retried; nothing is deleted.
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum SyncIssue {
/// The account's storage is full.
Quota,
/// The account may no longer change the file or its folder, for example
/// because a share was revoked.
Access,
/// The file or the folder it goes into is gone from Drive.
Missing,
/// The folder holds too many items, or is nested too deep.
Limit,
/// Drive rejected the change, for example a name it does not accept.
Rejected,
/// An issue from a newer daemon.
#[serde(other)]
Unknown,
}
impl SyncIssue {
/// The name stored in the database and sent on the wire.
pub fn as_str(self) -> &'static str {
match self {
SyncIssue::Quota => "quota",
SyncIssue::Access => "access",
SyncIssue::Missing => "missing",
SyncIssue::Limit => "limit",
SyncIssue::Rejected => "rejected",
SyncIssue::Unknown => "unknown",
}
}
/// The issue stored as `name`.
pub fn parse(name: &str) -> Self {
match name {
"quota" => SyncIssue::Quota,
"access" => SyncIssue::Access,
"missing" => SyncIssue::Missing,
"limit" => SyncIssue::Limit,
"rejected" => SyncIssue::Rejected,
_ => SyncIssue::Unknown,
}
}
}
/// One synced local folder on this machine's device (in [`Response::SyncFolders`]).
@@ -860,7 +1050,7 @@ pub struct ShareEntry {
}
/// A node's public link (in [`Response::Share`] / [`Response::PublicLink`]).
#[derive(Serialize, Deserialize, Debug, Clone)]
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
pub struct PublicLinkInfo {
/// Public-link id — the handle for [`Request::RemovePublicLink`].
pub id: String,
@@ -873,6 +1063,9 @@ pub struct PublicLinkInfo {
pub expires: Option<i64>,
/// Whether a custom password additionally protects the link.
pub has_password: bool,
/// When the link was made, epoch seconds; 0 when unknown.
#[serde(default)]
pub created: i64,
}
/// One invitation addressed to me (in [`Response::Invitations`]).
@@ -905,7 +1098,7 @@ pub struct BookmarkInfo {
/// the share's state so the "Shared" view renders in one pass: how many people
/// have access, how many invitations are still pending, and the public link if
/// the node has one.
#[derive(Serialize, Deserialize, Debug, Clone)]
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
pub struct SharedItem {
/// Node uid in `volume~link` form — the handle for opening the node's share
/// (via its mount path) or its details.
@@ -987,7 +1180,7 @@ pub enum ActivityKind {
/// One line in the daemon's activity log (in [`Response::Activity`]). Newest
/// first. Records a mutation or transfer the daemon performed, with enough
/// context to read as a sentence: "Uploaded report.pdf to /docs".
#[derive(Serialize, Deserialize, Debug, Clone)]
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
pub struct ActivityEntry {
/// When it happened, epoch seconds.
pub time: i64,
@@ -1045,7 +1238,7 @@ pub struct TransferItem {
}
/// One entry in a [`Request::ListDir`] listing.
#[derive(Serialize, Deserialize, Debug, Clone)]
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Default)]
pub struct DirEntry {
/// Decrypted node name.
pub name: String,
@@ -1093,6 +1286,20 @@ pub struct DirEntry {
/// warning worth showing.
#[serde(default)]
pub shared_by_unverified: bool,
/// When a [`Request::ListTrash`] entry went to the trash, epoch seconds; 0
/// when unknown, and for anything not in the trash. The API does not report
/// it to this client, so the daemon notes when it first saw the node there.
#[serde(default)]
pub trashed_at: i64,
/// The folder a [`Request::ListTrash`] entry goes back to on restore, as a
/// mountpoint-relative path (`""` for the root). `None` when that folder is
/// not known, and for anything not in the trash.
#[serde(default)]
pub trashed_from: Option<String>,
/// Why Drive refused a change queued on the node, for a [`Request::ListDir`]
/// entry; `None` otherwise.
#[serde(default)]
pub issue: Option<SyncIssue>,
}
/// One hit in a [`Request::Search`] result. Like [`DirEntry`] but carries the
@@ -1418,6 +1625,45 @@ pub struct AlbumInfo {
pub shared: bool,
}
/// One town in a [`Response::Places`] listing.
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct PlaceInfo {
/// The town's GeoNames id — what [`Request::PlacePhotos`] takes.
pub id: u32,
/// The town's name, as the town itself spells it.
pub name: String,
/// ISO 3166-1 alpha-2 code of the town's country.
pub country: String,
/// Where the town is, in decimal degrees, for the map. Zero from a daemon
/// older than the map.
#[serde(default)]
pub latitude: f64,
#[serde(default)]
pub longitude: f64,
/// How many photos were taken there, counting a RAW+JPEG shot once.
pub photo_count: usize,
/// The newest photo taken there, shown as the place's cover.
pub cover: PhotoItem,
}
/// One photo and where it was taken, in decimal degrees. Part of
/// [`Response::PhotoLocations`].
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct PhotoLocation {
pub photo: PhotoItem,
pub latitude: f64,
pub longitude: f64,
}
/// How many of the timeline's photos a running refresh has read the metadata
/// (and so the location) of, out of how many. Part of [`Response::Places`],
/// and of [`Response::PhotoSimilar`] for how many shots have been hashed.
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
pub struct MappingProgress {
pub done: usize,
pub total: usize,
}
/// One photo in a [`Request::PhotosTimeline`] page.
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct PhotoItem {
@@ -1657,17 +1903,17 @@ pub enum Response {
/// The pin registry.
pins: Vec<Pin>,
/// False when the daemon is serving the cached tree because the API is
/// unreachable (offline.md Phase 1). Cached and pinned content still
/// unreachable. Cached and pinned content still
/// reads; anything else fails until the network is back.
#[serde(default = "default_online")]
online: bool,
/// Writes accepted locally but not yet uploaded (offline.md Phase 3).
/// Writes accepted locally but not yet uploaded.
/// Non-zero means the mount is ahead of the remote — either a copy is
/// still draining, or it cannot drain because we are offline.
#[serde(default)]
pending_uploads: u64,
/// Queued mutations that carry no bytes: `mkdir`, `rename`, `trash`
/// (offline.md Phase 3b). Counted apart from `pending_uploads` because
/// Queued mutations that carry no bytes: `mkdir`, `rename`, `trash`.
/// Counted apart from `pending_uploads` because
/// calling a queued `mkdir` an upload is a lie.
#[serde(default)]
pending_changes: u64,
@@ -1776,6 +2022,31 @@ pub enum Response {
/// The months the timeline spans (reply to [`Request::PhotoMonths`]),
/// newest first.
PhotoMonths { months: Vec<PhotoMonth> },
/// The duplicate sets (reply to [`Request::PhotoDuplicates`]), newest
/// first. Each set lists the copy worth keeping first.
PhotoDuplicates { sets: Vec<Vec<PhotoItem>> },
/// The sets of alike shots (reply to [`Request::PhotoSimilar`]), newest
/// first, each in capture order; and how far hashing has got, `None`
/// once no pass is running. `uncompared` counts the shots left out
/// because no thumbnail could be read for them.
PhotoSimilar {
sets: Vec<Vec<PhotoItem>>,
#[serde(default)]
hashing: Option<MappingProgress>,
#[serde(default)]
uncompared: usize,
},
/// The towns the photos were taken in (reply to [`Request::PhotoPlaces`]),
/// and how far a running timeline refresh has got reading photo locations
/// — `None` when none is reading any.
Places {
items: Vec<PlaceInfo>,
#[serde(default)]
mapping: Option<MappingProgress>,
},
/// The photos of one town, each with its location (reply to
/// [`Request::PlaceLocations`]).
PhotoLocations { items: Vec<PhotoLocation> },
/// Thumbnails for a [`Request::PhotoThumbs`] batch.
Thumbs { items: Vec<PhotoThumb> },
/// Per-node outcome of a [`Request::TrashNodes`] batch. Reported per uid
@@ -1944,13 +2215,45 @@ impl ErrorKind {
}
}
impl Request {
/// Whether this is one of the cheap state reads that front ends repeat on a
/// timer. The daemon answers these without touching the network, so they
/// get the short [`POLL_READ_TIMEOUT`].
pub fn is_poll(&self) -> bool {
matches!(
self,
Request::Status
| Request::GetQueueStatus
| Request::ListLocations
| Request::ListPendingOps
| Request::ListConflicts
| Request::ListActivity { .. }
| Request::ThumbnailBuildStatus
| Request::PhotosRefreshStatus
| Request::RedateStatus
| Request::ImportStatus
)
}
/// How long to wait for the daemon's reply to this request.
pub fn read_timeout(&self) -> Duration {
if self.is_poll() {
POLL_READ_TIMEOUT
} else {
READ_TIMEOUT
}
}
}
/// Send one [`Request`] to the daemon listening on `socket` and read its
/// [`Response`]. Returns a crate [`crate::Error`] if no daemon is listening.
/// [`Response`]. Returns a crate [`crate::Error`] if no daemon is listening, and
/// one for which [`crate::Error::is_timeout`] holds if the daemon took longer
/// than [`Request::read_timeout`] to answer.
///
/// Shared by the CLI and GUI so both speak the wire format identically.
pub fn send(socket: &Path, req: &Request) -> Result<Response> {
let mut stream = UnixStream::connect(socket)?;
stream.set_read_timeout(Some(READ_TIMEOUT))?;
stream.set_read_timeout(Some(req.read_timeout()))?;
stream.set_write_timeout(Some(WRITE_TIMEOUT))?;
let mut line = serde_json::to_vec(req)?;
line.push(b'\n');
@@ -1962,10 +2265,232 @@ pub fn send(socket: &Path, req: &Request) -> Result<Response> {
Ok(serde_json::from_str(resp.trim())?)
}
/// An open [`Request::Subscribe`] connection. See [`subscribe`].
pub struct Subscription {
reader: BufReader<UnixStream>,
line: String,
}
/// Start following the daemon listening on `socket`. Fails if no daemon is
/// listening. A daemon too old to know [`Request::Subscribe`] answers it with an
/// error, which the first [`Subscription::next_event`] returns as
/// [`crate::Error::Other`].
pub fn subscribe(socket: &Path) -> Result<Subscription> {
let mut stream = UnixStream::connect(socket)?;
stream.set_read_timeout(Some(SUBSCRIBE_READ_TIMEOUT))?;
stream.set_write_timeout(Some(WRITE_TIMEOUT))?;
let mut line = serde_json::to_vec(&Request::Subscribe)?;
line.push(b'\n');
stream.write_all(&line)?;
stream.flush()?;
Ok(Subscription {
reader: BufReader::new(stream),
line: String::new(),
})
}
impl Subscription {
/// Wait for the next event. `Ok(None)` means the daemon closed the
/// connection; a timeout means it went [`SUBSCRIBE_READ_TIMEOUT`] without
/// even a heartbeat. A line from a newer daemon that this client cannot
/// read is skipped rather than ending the subscription.
pub fn next_event(&mut self) -> Result<Option<Event>> {
loop {
self.line.clear();
if self.reader.read_line(&mut self.line)? == 0 {
return Ok(None);
}
let line = self.line.trim();
if let Ok(event) = serde_json::from_str::<Event>(line) {
return Ok(Some(event));
}
if let Ok(Response::Error { message, .. }) = serde_json::from_str::<Response>(line) {
return Err(crate::Error::Other(message));
}
}
}
}
/// What [`follow`] reports.
#[derive(Debug)]
pub enum Feed {
/// A subscription started. The events after it describe the daemon's
/// current state, so a front-end catches up from them.
Connected,
Event(Event),
/// The subscription ended: the daemon stopped, or stopped answering.
Disconnected,
}
/// How often [`follow`] polls a daemon that cannot push events.
const LEGACY_POLL: Duration = Duration::from_secs(3);
/// The longest [`follow`] waits between attempts to reach a daemon.
const RECONNECT_MAX: Duration = Duration::from_secs(5);
/// Follow the daemon on `socket` for as long as `on` returns true, across
/// daemon restarts. Blocks, so run it on a thread of its own.
///
/// A daemon that predates [`Request::Subscribe`] (still running after an
/// upgrade, until its service restarts) is polled for its status and transfers
/// instead, so a front-end sees the same events either way, only later.
pub fn follow(socket: &Path, mut on: impl FnMut(Feed) -> bool) {
let mut connected = false;
let mut backoff = Duration::from_secs(1);
loop {
let ended = match subscribe(socket) {
Ok(mut subscription) => {
let mut first = true;
loop {
match subscription.next_event() {
Ok(Some(event)) => {
if first {
first = false;
connected = true;
backoff = Duration::from_secs(1);
if !on(Feed::Connected) {
return;
}
}
if !on(Feed::Event(event)) {
return;
}
}
Ok(None) => break Ok(()),
Err(e) => break Err(e),
}
}
}
Err(e) => Err(e),
};
if let Err(crate::Error::Other(_)) = ended {
if !connected {
connected = true;
if !on(Feed::Connected) {
return;
}
}
if !poll_legacy(socket, &mut on) {
return;
}
std::thread::sleep(LEGACY_POLL);
continue;
}
if connected {
connected = false;
if !on(Feed::Disconnected) {
return;
}
}
std::thread::sleep(backoff);
backoff = (backoff * 2).min(RECONNECT_MAX);
}
}
/// One round of [`follow`]'s fallback: the status and transfers a subscription
/// would have pushed. Returns what `on` returned.
fn poll_legacy(socket: &Path, on: &mut impl FnMut(Feed) -> bool) -> bool {
match send(socket, &Request::Status) {
Ok(status @ Response::Status { .. }) => {
if !on(Feed::Event(Event::Status(Box::new(status)))) {
return false;
}
}
_ => return true,
}
if let Ok(Response::Transfers { items, jobs }) = send(socket, &Request::GetQueueStatus) {
return on(Feed::Event(Event::Transfers { items, jobs }));
}
true
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn polls_wait_briefly_and_real_work_waits_long() {
assert_eq!(Request::Status.read_timeout(), POLL_READ_TIMEOUT);
assert_eq!(
Request::ListActivity { limit: 200 }.read_timeout(),
POLL_READ_TIMEOUT
);
assert_eq!(
Request::OpenFile {
path: "a.txt".into(),
uid: None,
}
.read_timeout(),
READ_TIMEOUT
);
}
/// A daemon on a fresh socket that reads the subscription request and
/// answers with `lines`, then hangs up.
fn fake_daemon(name: &str, lines: &'static str) -> std::path::PathBuf {
let socket = std::env::temp_dir().join(format!("pdfs-{name}-{}.sock", std::process::id()));
let _ = std::fs::remove_file(&socket);
let listener = std::os::unix::net::UnixListener::bind(&socket).unwrap();
std::thread::spawn(move || {
let (stream, _) = listener.accept().unwrap();
let mut reader = BufReader::new(stream);
let mut request = String::new();
reader.read_line(&mut request).unwrap();
assert!(matches!(
serde_json::from_str::<Request>(request.trim()),
Ok(Request::Subscribe)
));
reader.get_mut().write_all(lines.as_bytes()).unwrap();
});
socket
}
#[test]
fn a_subscription_skips_what_it_cannot_read() {
let socket = fake_daemon(
"subscribe",
"not json\n\
{\"Changed\":{\"topics\":[\"files\",\"from_a_newer_daemon\"]}}\n\
{\"NewEvent\":{}}\n\
\"Heartbeat\"\n",
);
let mut subscription = subscribe(&socket).unwrap();
let first = subscription.next_event().unwrap();
let second = subscription.next_event().unwrap();
let end = subscription.next_event().unwrap();
let _ = std::fs::remove_file(&socket);
match first {
Some(Event::Changed { topics }) => {
assert_eq!(topics, vec![Topic::Files, Topic::Unknown]);
}
other => panic!("expected a change, got {other:?}"),
}
assert!(matches!(second, Some(Event::Heartbeat)), "{second:?}");
assert!(end.is_none(), "{end:?}");
}
#[test]
fn a_daemon_without_subscriptions_is_told_apart_from_a_dead_one() {
let socket = fake_daemon("legacy", "{\"Error\":{\"message\":\"unknown request\"}}\n");
let mut subscription = subscribe(&socket).unwrap();
let answer = subscription.next_event();
let _ = std::fs::remove_file(&socket);
assert!(matches!(answer, Err(crate::Error::Other(_))), "{answer:?}");
}
#[test]
fn a_daemon_that_never_answers_times_out_as_not_responding() {
let socket = std::env::temp_dir().join(format!("pdfs-silent-{}.sock", std::process::id()));
let _ = std::fs::remove_file(&socket);
// Bound but never accepted: connect succeeds, no reply ever comes.
let _listener = std::os::unix::net::UnixListener::bind(&socket).unwrap();
let started = std::time::Instant::now();
let err = send(&socket, &Request::Status).unwrap_err();
let _ = std::fs::remove_file(&socket);
assert!(err.is_timeout(), "{err}");
assert!(started.elapsed() < READ_TIMEOUT);
}
#[test]
fn photo_kind_classifies_by_extension_then_mime() {
use PhotoKind::*;
@@ -2172,7 +2697,17 @@ mod tests {
fn trash_requests_and_outcomes_roundtrip() {
let request = Request::TrashNodes {
uids: vec!["volume~first".into(), "volume~second".into()],
files_only: false,
};
// An older front-end's request trashes whole groups.
let old: Request = serde_json::from_str(r#"{"TrashNodes":{"uids":["v~a"]}}"#).unwrap();
assert!(matches!(
old,
Request::TrashNodes {
files_only: false,
..
}
));
let line = serde_json::to_string(&request).unwrap();
assert!(!line.contains('\n'), "wire form must be a single line");
let decoded: Request = serde_json::from_str(&line).unwrap();
@@ -2459,6 +2994,55 @@ mod tests {
}
}
/// A place listing and a place page cross the wire intact: a mangled id
/// would open the wrong town.
#[test]
fn places_requests_and_reply_roundtrip() {
for req in [
Request::PhotoPlaces,
Request::PlacePhotos {
id: 2950159,
offset: 60,
limit: 30,
},
Request::PlaceLocations { id: 2950159 },
] {
let line = serde_json::to_string(&req).unwrap();
let back: Request = serde_json::from_str(&line).unwrap();
assert_eq!(line, serde_json::to_string(&back).unwrap());
}
let line = r#"{"Places":{"items":[{"id":2950159,"name":"Berlin","country":"DE","photo_count":2,
"cover":{"uid":"vol~b2","capture_time":1700000000,"thumb_path":null,"name":"b2.jpg","ratio":null,"no_thumb":false}}]}}"#;
let Response::Places { items, mapping } = serde_json::from_str(line).unwrap() else {
panic!("not a place listing");
};
assert_eq!(items.len(), 1);
assert_eq!((items[0].id, items[0].country.as_str()), (2950159, "DE"));
assert_eq!(items[0].cover.uid, "vol~b2");
// A daemon from before the progress field still parses: no mapping.
assert_eq!(mapping, None);
let reply = Response::Places {
items: Vec::new(),
mapping: Some(MappingProgress {
done: 120,
total: 480,
}),
};
let line = serde_json::to_string(&reply).unwrap();
let Response::Places { mapping, .. } = serde_json::from_str(&line).unwrap() else {
panic!("not a place listing");
};
assert_eq!(
mapping,
Some(MappingProgress {
done: 120,
total: 480
})
);
}
/// The sharing and devices requests must survive the same line-delimited JSON
/// round-trip: a mangled id or role would touch the wrong share or member.
#[test]
@@ -2772,6 +3356,17 @@ mod tests {
},
r#"{"SaveRevisionAsByUid":{"uid":"vol~link","revision_id":"rev-3","dest":"/tmp/out.bin"}}"#,
),
(
Request::ExportPendingOp {
id: 7,
dest: "/tmp/out.bin".into(),
},
r#"{"ExportPendingOp":{"id":7,"dest":"/tmp/out.bin"}}"#,
),
(
Request::DiscardPendingOp { id: 7 },
r#"{"DiscardPendingOp":{"id":7}}"#,
),
];
for (request, wire) in cases {
assert_eq!(serde_json::to_string(&request).unwrap(), wire);
@@ -2780,6 +3375,25 @@ mod tests {
}
}
/// A daemon that predates sync issues sends neither field, and a newer one
/// may send an issue this build does not know.
#[test]
fn a_queued_op_from_another_daemon_version_still_decodes() {
let old = r#"{"id":1,"kind":"create","path":"a.txt","attempts":0,"last_error":null,
"queued_at":5,"next_attempt_at":5,"parked":false,"failing":false}"#;
let op: PendingOpInfo = serde_json::from_str(old).unwrap();
assert_eq!(op.issue, None);
assert!(!op.exportable);
let newer = old.replace(r#""failing":false"#, r#""failing":true,"issue":"embargo""#);
let op: PendingOpInfo = serde_json::from_str(&newer).unwrap();
assert_eq!(op.issue, Some(SyncIssue::Unknown));
assert_eq!(
SyncIssue::parse(SyncIssue::Quota.as_str()),
SyncIssue::Quota
);
}
/// A front-end that predates the favorites filter sends no `favorites`
/// field, and must keep getting the whole timeline rather than an empty page.
#[test]
@@ -2790,6 +3404,7 @@ mod tests {
decoded,
Request::PhotosTimeline {
favorites: false,
not_in_album: false,
kind: None,
range: None,
..
+249
View File
@@ -0,0 +1,249 @@
/* WARNING: Script requires that SQLITE_DBCONFIG_DEFENSIVE be disabled */
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE sync_state (key TEXT PRIMARY KEY, value TEXT);
INSERT INTO sync_state VALUES('schema_version','35');
CREATE TABLE nodes (
uid TEXT PRIMARY KEY,
parent_uid TEXT,
name TEXT NOT NULL,
is_dir INTEGER NOT NULL,
size INTEGER,
mtime INTEGER NOT NULL,
revision_hash TEXT,
trashed INTEGER NOT NULL DEFAULT 0
, node_json TEXT, listed INTEGER NOT NULL DEFAULT 0, path TEXT);
INSERT INTO nodes VALUES('vol~root',NULL,'My Files',1,0,200,NULL,0,'{"uid":{"volume_id":"vol","link_id":"root"},"parent_uid":null,"kind":"Folder","name":"My Files","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',0,'');
INSERT INTO nodes VALUES('vol~docs','vol~root','Docs',1,0,200,NULL,0,'{"uid":{"volume_id":"vol","link_id":"docs"},"parent_uid":{"volume_id":"vol","link_id":"root"},"kind":"Folder","name":"Docs","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',0,'Docs');
INSERT INTO nodes VALUES('vol~report','vol~docs','report.txt',0,10,200,NULL,0,'{"uid":{"volume_id":"vol","link_id":"report"},"parent_uid":{"volume_id":"vol","link_id":"docs"},"kind":{"File":{"media_type":"text/plain","total_size_on_storage":20,"active_revision_state":null,"active_revision_id":null,"claimed_size":10,"claimed_modification_time":null,"content_sha1":null}},"name":"report.txt","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',0,'Docs/report.txt');
INSERT INTO nodes VALUES('vol~old','vol~docs','old.txt',0,3,200,NULL,0,'{"uid":{"volume_id":"vol","link_id":"old"},"parent_uid":{"volume_id":"vol","link_id":"docs"},"kind":{"File":{"media_type":"text/plain","total_size_on_storage":13,"active_revision_state":null,"active_revision_id":null,"claimed_size":3,"claimed_modification_time":null,"content_sha1":null}},"name":"old.txt","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',0,'Docs/old.txt');
INSERT INTO nodes VALUES('vol~shared','vol~docs','shared.txt',0,4,200,NULL,0,'{"uid":{"volume_id":"vol","link_id":"shared"},"parent_uid":{"volume_id":"vol","link_id":"docs"},"kind":{"File":{"media_type":"text/plain","total_size_on_storage":14,"active_revision_state":null,"active_revision_id":null,"claimed_size":4,"claimed_modification_time":null,"content_sha1":null}},"name":"shared.txt","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',0,'Docs/shared.txt');
INSERT INTO nodes VALUES('local~dir','vol~root','Offline folder',1,0,200,NULL,0,'{"uid":{"volume_id":"local","link_id":"dir"},"parent_uid":{"volume_id":"vol","link_id":"root"},"kind":"Folder","name":"Offline folder","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',1,'Offline folder');
INSERT INTO nodes VALUES('local~inner','local~dir','inner.txt',0,5,200,NULL,0,'{"uid":{"volume_id":"local","link_id":"inner"},"parent_uid":{"volume_id":"local","link_id":"dir"},"kind":{"File":{"media_type":"text/plain","total_size_on_storage":15,"active_revision_state":null,"active_revision_id":null,"claimed_size":5,"claimed_modification_time":null,"content_sha1":null}},"name":"inner.txt","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',0,'Offline folder/inner.txt');
INSERT INTO nodes VALUES('local~part','vol~root','movie.part',0,7,200,NULL,0,'{"uid":{"volume_id":"local","link_id":"part"},"parent_uid":{"volume_id":"vol","link_id":"root"},"kind":{"File":{"media_type":"text/plain","total_size_on_storage":17,"active_revision_state":null,"active_revision_id":null,"claimed_size":7,"claimed_modification_time":null,"content_sha1":null}},"name":"movie.part","creation_time":100,"modification_time":200,"trashed":false,"is_shared":false,"is_shared_publicly":false,"signature_email":null,"membership":null,"direct_role":null,"share_id":null,"photo":null,"album":null,"verification":{"name":"NotSigned","passphrase":"NotSigned","content_key":null,"extended_attributes":null}}',0,'movie.part');
CREATE TABLE cache_entries (
cache_key TEXT PRIMARY KEY,
size_bytes INTEGER,
last_accessed INTEGER,
is_pinned INTEGER NOT NULL DEFAULT 0
, kind TEXT NOT NULL DEFAULT 'blob');
CREATE TABLE pins (
uid TEXT PRIMARY KEY,
path TEXT NOT NULL,
recursive INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE local_files (
id INTEGER PRIMARY KEY,
path TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
is_dir INTEGER NOT NULL,
size INTEGER NOT NULL DEFAULT 0,
mtime INTEGER NOT NULL DEFAULT 0,
scan_gen INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE photos (
uid TEXT PRIMARY KEY,
capture_time INTEGER NOT NULL,
name TEXT,
ratio REAL,
thumb_state INTEGER NOT NULL DEFAULT 0,
seq INTEGER NOT NULL
, media_type TEXT, kind INTEGER NOT NULL DEFAULT 0, favorite INTEGER NOT NULL DEFAULT 0, content_hash TEXT, main_uid TEXT, group_key TEXT, resolved_at INTEGER, latitude REAL, longitude REAL, place_id INTEGER, similar_hash INTEGER);
CREATE TABLE trash (
uid TEXT PRIMARY KEY,
name TEXT NOT NULL,
is_dir INTEGER NOT NULL,
size INTEGER NOT NULL DEFAULT 0,
mtime INTEGER NOT NULL DEFAULT 0
, parent_uid TEXT, trashed_at INTEGER);
CREATE TABLE device (
uid TEXT PRIMARY KEY,
share_id TEXT NOT NULL,
root_uid TEXT NOT NULL,
name TEXT NOT NULL,
created INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE sync_folder (
id INTEGER PRIMARY KEY,
local_path TEXT NOT NULL UNIQUE,
remote_uid TEXT NOT NULL,
remote_share_id TEXT NOT NULL,
mode TEXT NOT NULL DEFAULT 'mirror',
state TEXT NOT NULL DEFAULT 'idle',
last_sync INTEGER NOT NULL DEFAULT 0
, pending_mode TEXT);
CREATE TABLE sync_entry (
folder_id INTEGER NOT NULL,
rel_path TEXT NOT NULL,
remote_uid TEXT,
local_mtime INTEGER NOT NULL DEFAULT 0,
local_size INTEGER NOT NULL DEFAULT 0,
remote_hash TEXT,
remote_rev TEXT, local_mtime_ns INTEGER,
PRIMARY KEY (folder_id, rel_path)
);
CREATE TABLE activity (
id INTEGER PRIMARY KEY,
time INTEGER NOT NULL,
kind TEXT NOT NULL,
target TEXT NOT NULL,
detail TEXT NOT NULL DEFAULT '',
ok INTEGER NOT NULL DEFAULT 1
);
CREATE TABLE pending_op (
id INTEGER PRIMARY KEY,
kind TEXT NOT NULL,
uid TEXT NOT NULL,
blob_path TEXT,
meta_json TEXT,
created_at INTEGER NOT NULL,
attempts INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
next_attempt_at INTEGER NOT NULL DEFAULT 0
, parent_uid TEXT, name TEXT, claimed_at INTEGER NOT NULL DEFAULT 0, access_deferred_since INTEGER NOT NULL DEFAULT 0);
INSERT INTO pending_op VALUES(1,'mkdir','local~dir',NULL,NULL,1700000000000,0,NULL,0,'vol~root','Offline folder',4611686018427387903,0);
INSERT INTO pending_op VALUES(2,'create','local~inner','/blobs/inner','{"len":5,"mtime":1700000000,"based_on":null}',1700000000000,0,NULL,0,'local~dir','inner.txt',0,0);
INSERT INTO pending_op VALUES(3,'create','local~part','/blobs/part','{"len":5,"mtime":1700000000,"based_on":null}',1700000000000,0,NULL,8000000000000,'vol~root','movie.part',0,0);
INSERT INTO pending_op VALUES(4,'revision','vol~report','/blobs/report','{"len":5,"mtime":1700000000,"based_on":null}',1700000000000,2,'network unreachable',1700000900000,NULL,NULL,0,0);
INSERT INTO pending_op VALUES(5,'rename','vol~shared',NULL,'{"original_parent_uid":"vol~docs","original_name":"shared.txt"}',1700000000000,0,NULL,1700000700000,'vol~root','shared-moved.txt',0,1700000100000);
INSERT INTO pending_op VALUES(6,'trash','vol~old',NULL,NULL,1700000000000,0,NULL,0,NULL,NULL,0,0);
CREATE TABLE IF NOT EXISTS 'local_fts_data'(id INTEGER PRIMARY KEY, block BLOB);
INSERT INTO local_fts_data VALUES(1,x'000000');
INSERT INTO local_fts_data VALUES(10,x'00000000000000');
CREATE TABLE IF NOT EXISTS 'local_fts_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
CREATE TABLE IF NOT EXISTS 'local_fts_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
CREATE TABLE IF NOT EXISTS 'local_fts_config'(k PRIMARY KEY, v) WITHOUT ROWID;
INSERT INTO local_fts_config VALUES('version',4);
CREATE TABLE share_access (
root_uid TEXT PRIMARY KEY,
access TEXT NOT NULL
CHECK (access IN ('owner', 'editor', 'viewer', 'unknown'))
);
CREATE TABLE mount (
id INTEGER PRIMARY KEY,
kind TEXT NOT NULL
CHECK (kind IN ('myfiles', 'device', 'shared')),
sync_folder_id INTEGER REFERENCES sync_folder(id) ON DELETE CASCADE,
share_root_uid TEXT,
local_path TEXT NOT NULL DEFAULT '',
root_uid TEXT NOT NULL DEFAULT '',
root_share_id TEXT NOT NULL DEFAULT '',
mode TEXT NOT NULL DEFAULT 'unknown',
access TEXT NOT NULL DEFAULT 'rw'
CHECK (access IN ('rw', 'ro')),
CHECK (
(kind = 'myfiles' AND sync_folder_id IS NULL AND share_root_uid IS NULL)
OR
(kind = 'device' AND sync_folder_id IS NOT NULL AND share_root_uid IS NULL)
OR
(kind = 'shared' AND sync_folder_id IS NULL AND share_root_uid IS NOT NULL)
)
);
CREATE TABLE albums (
uid TEXT PRIMARY KEY,
name TEXT NOT NULL,
photo_count INTEGER NOT NULL DEFAULT 0,
cover_uid TEXT,
last_activity INTEGER,
shared INTEGER NOT NULL DEFAULT 0,
seq INTEGER NOT NULL
);
CREATE TABLE album_photos (
album_uid TEXT NOT NULL,
uid TEXT NOT NULL,
capture_time INTEGER NOT NULL,
name TEXT,
media_type TEXT,
kind INTEGER NOT NULL DEFAULT 0,
ratio REAL,
thumb_state INTEGER NOT NULL DEFAULT 0,
seq INTEGER NOT NULL,
PRIMARY KEY (album_uid, uid)
);
CREATE TABLE IF NOT EXISTS 'nodes_fts_data'(id INTEGER PRIMARY KEY, block BLOB);
INSERT INTO nodes_fts_data VALUES(1,x'083850');
INSERT INTO nodes_fts_data VALUES(10,x'000000000108080008010101020101030101040101050101060101070101080101');
INSERT INTO nodes_fts_data VALUES(137438953473,x'000000340430206669010204010366696c0102050103696c6501020601036c657301020701036d79200102020103792066010203040808080808');
INSERT INTO nodes_fts_data VALUES(274877906945,x'0000001a0430646f6302080201010201036f6373020803010103040b');
INSERT INTO nodes_fts_data VALUES(412316860417,x'0000008b04302e747803080801010d01032f72650306010106010363732f03060101040103646f630306010102010365706f03080301010801036f637303060101030202727403080501010a0103706f7203080401010901037265700308020101070202742e03080601010b0103732f7203060101050103742e7403080701010c0202787403080901010e040b0a0a0a0b0a0a0b0b0a0a0b');
INSERT INTO nodes_fts_data VALUES(549755813889,x'0000006b04302e747804080501010a01032f6f6c0406010106010363732f04060101040103642e7404080401010902026f63040601010201036c642e04080301010801036f6373040601010302026c640408020101070103732f6f0406010105010374787404080601010b040b0a0a0b090b0a0a0a');
INSERT INTO nodes_fts_data VALUES(687194767361,x'0000008c04302e747805080801010d01032f736805060101060103617265050804010109010363732f05060101040103642e7405080701010c02026f630506010102010365642e05080601010b010368617205080301010801036f63730506010103010372656405080501010a0103732f73050601010502026861050802010107010374787405080901010e040b0a0b0a0b090b0b0a0b0a0a');
INSERT INTO nodes_fts_data VALUES(824633720833,x'00000084043020666f060809010109010364657206080d01010d0103652066060808010108010366666c06080301010302026c6906080401010402026f6c06080a01010a0103696e6506080601010601036c646506080c01010c0202696e06080501010501036e652006080701010701036f666606080201010202026c6406080b01010b040b0b0b0b0a0a0b0b0a0b0b');
INSERT INTO nodes_fts_data VALUES(962072674305,x'000000da043020666f070601010901032e747807080701011601032f696e07060101100103646572070601010d010365206607060101080202722e07080501011403012f070601010e010366666c070601010302026c69070601010402026f6c070601010a0103696e65070601010603016e07080201011101036c6465070601010c0202696e070601010501036e6520070601010703017207080401011302026e6507080301011201036f6666070601010202026c64070601010b0103722e7407080601011502022f69070601010f0103747874070808010117040a0b0a0a0a0a080a09090a090a090a090a0a090b09');
INSERT INTO nodes_fts_data VALUES(1099511627777,x'0000005c04302e706108080701010701036172740808090101090103652e70080806010106010369652e08080501010501036d6f7608080201010201036f766908080301010301037061720808080101080103766965080804010104040b0b0b0b0b0b0b');
CREATE TABLE IF NOT EXISTS 'nodes_fts_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID;
INSERT INTO nodes_fts_idx VALUES(1,x'',2);
INSERT INTO nodes_fts_idx VALUES(2,x'',2);
INSERT INTO nodes_fts_idx VALUES(3,x'',2);
INSERT INTO nodes_fts_idx VALUES(4,x'',2);
INSERT INTO nodes_fts_idx VALUES(5,x'',2);
INSERT INTO nodes_fts_idx VALUES(6,x'',2);
INSERT INTO nodes_fts_idx VALUES(7,x'',2);
INSERT INTO nodes_fts_idx VALUES(8,x'',2);
CREATE TABLE IF NOT EXISTS 'nodes_fts_content'(id INTEGER PRIMARY KEY, c0, c1);
INSERT INTO nodes_fts_content VALUES(1,'My Files','');
INSERT INTO nodes_fts_content VALUES(2,'Docs','Docs');
INSERT INTO nodes_fts_content VALUES(3,'report.txt','Docs/report.txt');
INSERT INTO nodes_fts_content VALUES(4,'old.txt','Docs/old.txt');
INSERT INTO nodes_fts_content VALUES(5,'shared.txt','Docs/shared.txt');
INSERT INTO nodes_fts_content VALUES(6,'Offline folder','Offline folder');
INSERT INTO nodes_fts_content VALUES(7,'inner.txt','Offline folder/inner.txt');
INSERT INTO nodes_fts_content VALUES(8,'movie.part','movie.part');
CREATE TABLE IF NOT EXISTS 'nodes_fts_docsize'(id INTEGER PRIMARY KEY, sz BLOB);
INSERT INTO nodes_fts_docsize VALUES(1,x'0600');
INSERT INTO nodes_fts_docsize VALUES(2,x'0202');
INSERT INTO nodes_fts_docsize VALUES(3,x'080d');
INSERT INTO nodes_fts_docsize VALUES(4,x'050a');
INSERT INTO nodes_fts_docsize VALUES(5,x'080d');
INSERT INTO nodes_fts_docsize VALUES(6,x'0c0c');
INSERT INTO nodes_fts_docsize VALUES(7,x'0716');
INSERT INTO nodes_fts_docsize VALUES(8,x'0808');
CREATE TABLE IF NOT EXISTS 'nodes_fts_config'(k PRIMARY KEY, v) WITHOUT ROWID;
INSERT INTO nodes_fts_config VALUES('version',4);
CREATE TABLE own_sealed_rev (
uid TEXT PRIMARY KEY,
revision_id TEXT NOT NULL,
sealed_at INTEGER NOT NULL
);
INSERT INTO own_sealed_rev VALUES('vol~report','rev-1',1700000000000);
PRAGMA writable_schema=ON;
INSERT INTO sqlite_schema(type,name,tbl_name,rootpage,sql)VALUES('table','local_fts','local_fts',0,'CREATE VIRTUAL TABLE local_fts USING fts5(
name, path, content=''local_files'', content_rowid=''id'', tokenize=''trigram''
)');
INSERT INTO sqlite_schema(type,name,tbl_name,rootpage,sql)VALUES('table','nodes_fts','nodes_fts',0,'CREATE VIRTUAL TABLE nodes_fts USING fts5(
name, path, tokenize=''trigram''
)');
CREATE TRIGGER mount_sync_folder_insert
AFTER INSERT ON sync_folder
BEGIN
INSERT OR IGNORE INTO mount (kind, sync_folder_id)
VALUES ('device', NEW.id);
END;
CREATE INDEX idx_nodes_parent ON nodes(parent_uid);
CREATE INDEX idx_photos_seq ON photos(seq);
CREATE INDEX pending_op_uid ON pending_op(uid);
CREATE INDEX pending_op_parent ON pending_op(parent_uid);
CREATE INDEX cache_entries_lru ON cache_entries(kind, last_accessed);
CREATE INDEX idx_nodes_name_nocase ON nodes(name COLLATE NOCASE);
CREATE INDEX idx_local_files_name_nocase
ON local_files(name COLLATE NOCASE);
CREATE UNIQUE INDEX mount_myfiles
ON mount(kind) WHERE kind = 'myfiles';
CREATE UNIQUE INDEX mount_device
ON mount(sync_folder_id) WHERE kind = 'device';
CREATE UNIQUE INDEX mount_shared
ON mount(share_root_uid) WHERE kind = 'shared';
CREATE INDEX idx_album_photos_seq ON album_photos(album_uid, seq);
CREATE INDEX idx_album_photos_uid ON album_photos(uid);
CREATE INDEX idx_photos_favorite ON photos(favorite, seq);
CREATE INDEX idx_sync_entry_remote_uid ON sync_entry(remote_uid);
CREATE INDEX idx_local_files_scan_gen ON local_files(scan_gen);
CREATE INDEX idx_nodes_parent_live ON nodes(parent_uid) WHERE trashed = 0;
CREATE INDEX idx_pending_op_claimed
ON pending_op(uid) WHERE claimed_at <> 0;
CREATE INDEX idx_trash_parent ON trash(parent_uid);
CREATE INDEX idx_photos_group ON photos(group_key);
CREATE INDEX idx_photos_unresolved ON photos(resolved_at);
CREATE INDEX idx_photos_place ON photos(place_id);
PRAGMA writable_schema=OFF;
COMMIT;
+1 -1
View File
@@ -1,4 +1,4 @@
//! Database health reporting and repair (features.md 4.3).
//! Database health reporting and repair.
//!
//! Everything here exists for the case where something has already gone wrong:
//! a crash mid-write, a full disk, a cache that has grown out of proportion to
+363 -6
View File
@@ -5,11 +5,13 @@
//! is already on disk somewhere, so a change here would silently diverge from
//! what an existing install actually has. Add a new one instead.
use rusqlite::OptionalExtension;
use super::Db;
use crate::Result;
/// Current schema version. Bump on every forward migration added below.
pub(super) const SCHEMA_VERSION: i64 = 32;
pub(super) const SCHEMA_VERSION: i64 = 42;
impl Db {
pub(super) fn migrate(&self) -> Result<()> {
@@ -284,6 +286,160 @@ impl Db {
}
}
}
if current < 33 {
// Same guards as V26-V31.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('photos') WHERE name = 'place_id'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_photos: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'photos'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_photos && !has_column {
tx.execute_batch(MIGRATION_V33)?;
}
}
if current < 34 {
// Same guards as V26-V33.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('photos') WHERE name = 'similar_hash'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_photos: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'photos'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_photos && !has_column {
tx.execute_batch(MIGRATION_V34)?;
}
}
if current < 35 {
// Same guards as V26-V34.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('trash') WHERE name = 'trashed_at'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_trash: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'trash'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_trash && !has_column {
tx.execute_batch(MIGRATION_V35)?;
}
}
if current < 36 {
// Same guards as V26-V35.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('nodes') WHERE name = 'lid'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_nodes: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'nodes'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_nodes && !has_column {
tx.execute_batch(MIGRATION_V36)?;
}
}
if current < 37 {
// Same guards as V26-V36.
let sql: Option<String> = tx
.query_row(
"SELECT sql FROM sqlite_master WHERE type = 'table' AND name = 'pending_op'",
[],
|row| row.get(0),
)
.optional()?;
if sql.is_some_and(|sql| !sql.contains("AUTOINCREMENT")) {
tx.execute_batch(MIGRATION_V37)?;
}
}
if current < 38 {
// Same guards as V26-V37.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('pending_op') WHERE name = 'issue'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_ops: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'pending_op'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_ops && !has_column {
tx.execute_batch(MIGRATION_V38)?;
}
}
if current < 39 {
// Same guards as V26-V38.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('pending_op') WHERE name = 'lid'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_ops: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'pending_op'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_ops && !has_column {
tx.execute_batch(MIGRATION_V39)?;
}
}
if current < 40 {
// Same guards as V26-V39.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('nodes') WHERE name = 'parent_lid'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_nodes: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'nodes'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_nodes && !has_column {
tx.execute_batch(MIGRATION_V40)?;
}
}
if current < 41 {
// Needs both tables; on a database that never had them there is
// nothing to rewrite.
let tables: i64 = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master
WHERE type = 'table' AND name IN ('nodes', 'pending_op', 'pins')",
[],
|row| row.get(0),
)?;
if tables == 3 {
tx.execute_batch(MIGRATION_V41)?;
}
}
if current < 42 {
// Same guards as V26-V40.
let has_column: bool = tx.query_row(
"SELECT COUNT(*) FROM pragma_table_info('pending_op') WHERE name = 'sent_sha1'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
let has_ops: bool = tx.query_row(
"SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name = 'pending_op'",
[],
|row| row.get::<_, i64>(0),
)? > 0;
if has_ops && !has_column {
tx.execute_batch(MIGRATION_V42)?;
}
}
tx.execute(
"INSERT INTO sync_state (key, value) VALUES ('schema_version', ?1)
ON CONFLICT(key) DO UPDATE SET value = excluded.value",
@@ -421,7 +577,7 @@ CREATE TABLE trash (
);
";
/// Schema v8: device sync (devices.md). This machine registers as one Proton Drive
/// Schema v8: device sync. This machine registers as one Proton Drive
/// **Device** (a share + root folder on the main volume); `device` is a singleton
/// row cached so we reuse the same device across restarts instead of creating a
/// new one each run. `sync_folder` is one row per local folder the user added,
@@ -482,8 +638,8 @@ ALTER TABLE sync_folder ADD COLUMN pending_mode TEXT;
";
/// Schema v11: writes no longer upload inside the FUSE handler. `release` stages
/// the bytes and records the intended upload here, and a drain worker performs it
/// (offline.md Phase 3). The row outlives the process, so a write survives both a
/// the bytes and records the intended upload here, and a drain worker performs it.
/// The row outlives the process, so a write survives both a
/// dead network and a restart.
///
/// `blob_path` points into the content cache's `staging/` dir and `meta_json` is
@@ -510,7 +666,7 @@ CREATE INDEX pending_op_uid ON pending_op(uid);
/// Schema v12: `pending_op` also carries mutations that *create* a node, which a
/// revision op never had to describe — it always addressed a node the server had
/// already minted a uid for (offline.md Phase 3b).
/// already minted a uid for.
///
/// An offline `create`/`mkdir` cannot get a real uid, so the node is invented
/// locally under a `local~<uuid>` placeholder and the op records where it goes
@@ -900,7 +1056,7 @@ ALTER TABLE pending_op ADD COLUMN access_deferred_since INTEGER NOT NULL DEFAULT
/// as the last sync and left the file the same length — a fixed-size record
/// rewritten, an image re-exported, a database page updated in place — read as
/// *unchanged* and was never uploaded. The remote copy silently stayed at the
/// older content until something else about the file moved (bugs.md B25).
/// older content until something else about the file moved (`docs/BUGS.md` B25).
///
/// Nullable rather than a converted `local_mtime * 1_000_000_000`: the
/// nanosecond part of an existing baseline is genuinely unknown, and inventing
@@ -973,3 +1129,204 @@ UPDATE photos SET resolved_at = strftime('%s','now') WHERE name IS NOT NULL;
/// decoder, which could only fail, and that failure was recorded as the
/// permanent "no thumbnail" verdict. Run once per table in `{table}`.
const MIGRATION_V32: &str = "UPDATE {table} SET thumb_state = 0 WHERE kind = 1 AND thumb_state = 2";
/// Schema v33: where each photo was taken.
///
/// `latitude` and `longitude` come from the location other clients write into
/// a photo's extended attributes; `place_id` is the GeoNames id of the nearest
/// town (see `crate::places`), so the Places view is a plain `GROUP BY`.
///
/// Every photo is marked unresolved, so the next refresh reads the location of
/// the whole library once. That is the backfill: the metadata is on the node
/// the refresh reads anyway, and nothing has to be downloaded.
const MIGRATION_V33: &str = "
ALTER TABLE photos ADD COLUMN latitude REAL;
ALTER TABLE photos ADD COLUMN longitude REAL;
ALTER TABLE photos ADD COLUMN place_id INTEGER;
CREATE INDEX IF NOT EXISTS idx_photos_place ON photos(place_id);
UPDATE photos SET resolved_at = NULL;
";
/// Schema v34: what each photo looks like, for finding similar ones.
///
/// `similar_hash` is a 64-bit difference hash of the photo's thumbnail (see
/// `crate::similar`): resized and re-encoded copies of one picture land a few
/// bits apart, where the server's `content_hash` only matches byte-identical
/// files. `NULL` until the daemon has hashed the thumbnail, which it does as
/// thumbnails arrive and when the similar-photo finder asks for the rest.
const MIGRATION_V34: &str = "
ALTER TABLE photos ADD COLUMN similar_hash INTEGER;
";
/// Schema v35: when each trashed node went to the trash.
///
/// The API does not tell this client, so `trashed_at` is when the daemon first
/// saw the node in the trash, in epoch seconds. `NULL` when it cannot say: rows
/// from before this version, and nodes found by a refresh that did not follow a
/// complete one, which may have been in the trash for a long time already.
const MIGRATION_V35: &str = "
ALTER TABLE trash ADD COLUMN trashed_at INTEGER;
";
/// Schema v36: every node row has a local id (`lid`); see
/// docs/ARCHITECTURE.md §2.
///
/// The `uid` was the row's key, so a node had no identity until Drive minted
/// one, and a create that landed moved its row to a new key. The `lid` is given
/// when the row is first written and never changes or comes back, which is what
/// `AUTOINCREMENT` adds over a plain rowid. It takes the old rowid, so the
/// search index, keyed by rowid since v15, stays valid. The `uid` becomes
/// nullable for a node that is not on Drive yet.
const MIGRATION_V36: &str = "
CREATE TABLE nodes_v36 (
lid INTEGER PRIMARY KEY AUTOINCREMENT,
uid TEXT UNIQUE,
parent_uid TEXT,
name TEXT NOT NULL,
is_dir INTEGER NOT NULL,
size INTEGER,
mtime INTEGER NOT NULL,
revision_hash TEXT,
trashed INTEGER NOT NULL DEFAULT 0,
node_json TEXT,
listed INTEGER NOT NULL DEFAULT 0,
path TEXT
);
INSERT INTO nodes_v36
(lid, uid, parent_uid, name, is_dir, size, mtime, revision_hash, trashed, node_json, listed,
path)
SELECT rowid, uid, parent_uid, name, is_dir, size, mtime, revision_hash, trashed, node_json,
listed, path
FROM nodes;
DROP TABLE nodes;
ALTER TABLE nodes_v36 RENAME TO nodes;
CREATE INDEX idx_nodes_parent ON nodes(parent_uid);
CREATE INDEX idx_nodes_name_nocase ON nodes(name COLLATE NOCASE);
CREATE INDEX idx_nodes_parent_live ON nodes(parent_uid) WHERE trashed = 0;
";
/// Schema v37: a `pending_op` id is never given out twice.
///
/// A plain rowid is the largest one plus one, so an op queued right after the
/// newest one was dropped took its id. The drain retires the op it is
/// working on by id, and so deleted the new op: a trash queued over a rename
/// on the wire was lost when the rename landed. `AUTOINCREMENT` never reuses
/// an id. Ids are kept, as is every index.
const MIGRATION_V37: &str = "
CREATE TABLE pending_op_v37 (
id INTEGER PRIMARY KEY AUTOINCREMENT,
kind TEXT NOT NULL,
uid TEXT NOT NULL,
blob_path TEXT,
meta_json TEXT,
created_at INTEGER NOT NULL,
attempts INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
next_attempt_at INTEGER NOT NULL DEFAULT 0,
parent_uid TEXT,
name TEXT,
claimed_at INTEGER NOT NULL DEFAULT 0,
access_deferred_since INTEGER NOT NULL DEFAULT 0
);
INSERT INTO pending_op_v37
(id, kind, uid, blob_path, meta_json, created_at, attempts, last_error, next_attempt_at,
parent_uid, name, claimed_at, access_deferred_since)
SELECT id, kind, uid, blob_path, meta_json, created_at, attempts, last_error,
next_attempt_at, parent_uid, name, claimed_at, access_deferred_since
FROM pending_op;
DROP TABLE pending_op;
ALTER TABLE pending_op_v37 RENAME TO pending_op;
CREATE INDEX pending_op_uid ON pending_op(uid);
CREATE INDEX pending_op_parent ON pending_op(parent_uid);
CREATE INDEX idx_pending_op_claimed
ON pending_op(uid) WHERE claimed_at <> 0;
";
/// Schema v38: why Drive refused a queued op, when it did (a sync issue).
/// `NULL` for an op that has not been refused, which is every op migrated.
const MIGRATION_V38: &str = "
ALTER TABLE pending_op ADD COLUMN issue TEXT;
";
/// Schema v39: a queued op names its node and its parent by local id too.
///
/// A create made inside a folder that had not landed named the folder by its
/// placeholder uid, and every landing had to find and rewrite those strings.
/// The local id stays the same when the folder lands. An op whose node has no
/// row keeps `NULL` and is matched by uid, as before.
const MIGRATION_V39: &str = "
ALTER TABLE pending_op ADD COLUMN lid INTEGER;
ALTER TABLE pending_op ADD COLUMN parent_lid INTEGER;
UPDATE pending_op
SET lid = (SELECT lid FROM nodes WHERE nodes.uid = pending_op.uid),
parent_lid = (SELECT lid FROM nodes WHERE nodes.uid = pending_op.parent_uid);
CREATE INDEX pending_op_lid ON pending_op(lid);
CREATE INDEX pending_op_parent_lid ON pending_op(parent_lid);
";
/// Schema v40: a node row names its parent by local id too.
///
/// The local id stays when a folder made here lands and its row takes the uid
/// Drive gave it, so a child linked by it needs no rewrite. `NULL` while the
/// parent has no row, as `parent_uid` then names nothing either.
const MIGRATION_V40: &str = "
ALTER TABLE nodes ADD COLUMN parent_lid INTEGER;
UPDATE nodes SET parent_lid = (SELECT p.lid FROM nodes p WHERE p.uid = nodes.parent_uid);
CREATE INDEX idx_nodes_parent_lid ON nodes(parent_lid);
";
/// Schema v41: a node made on this machine goes by `local~<lid>` until it
/// lands, a function of its row, where it used to get a `local~<ms>-<n>`
/// placeholder. Every placeholder still stored is rewritten to its row's
/// stand-in: the row itself and its `node_json`, its children, the ops on it
/// or into it, and its pins. A placeholder op with no row keeps its uid and
/// drains as before.
const MIGRATION_V41: &str = "
CREATE TEMP TABLE local_v41 AS
SELECT uid AS old, 'local~' || lid AS new, lid FROM nodes
WHERE uid LIKE 'local~%' AND uid <> 'local~' || lid;
UPDATE pending_op SET uid = (SELECT new FROM local_v41 WHERE old = pending_op.uid)
WHERE uid IN (SELECT old FROM local_v41);
UPDATE pending_op SET parent_uid = (SELECT new FROM local_v41 WHERE old = pending_op.parent_uid)
WHERE parent_uid IN (SELECT old FROM local_v41);
UPDATE pending_op
SET meta_json = json_set(meta_json, '$.uid',
(SELECT new FROM local_v41 WHERE old = json_extract(pending_op.meta_json, '$.uid')))
WHERE json_valid(meta_json)
AND json_extract(meta_json, '$.uid') IN (SELECT old FROM local_v41);
UPDATE pending_op
SET meta_json = json_set(meta_json, '$.original_parent_uid',
(SELECT new FROM local_v41
WHERE old = json_extract(pending_op.meta_json, '$.original_parent_uid')))
WHERE json_valid(meta_json)
AND json_extract(meta_json, '$.original_parent_uid') IN (SELECT old FROM local_v41);
UPDATE pins SET uid = (SELECT new FROM local_v41 WHERE old = pins.uid)
WHERE uid IN (SELECT old FROM local_v41);
UPDATE nodes
SET node_json = json_set(node_json, '$.parent_uid.link_id',
(SELECT CAST(lid AS TEXT) FROM local_v41 WHERE old = nodes.parent_uid))
WHERE parent_uid IN (SELECT old FROM local_v41)
AND json_valid(node_json)
AND json_extract(node_json, '$.parent_uid.volume_id') = 'local';
UPDATE nodes SET parent_uid = (SELECT new FROM local_v41 WHERE old = nodes.parent_uid)
WHERE parent_uid IN (SELECT old FROM local_v41);
UPDATE nodes
SET node_json = json_set(node_json, '$.uid.link_id', CAST(lid AS TEXT))
WHERE lid IN (SELECT lid FROM local_v41)
AND json_valid(node_json)
AND json_extract(node_json, '$.uid.volume_id') = 'local';
UPDATE nodes SET uid = 'local~' || lid WHERE lid IN (SELECT lid FROM local_v41);
DROP TABLE local_v41;
";
/// Schema v42: the SHA-1 of each blob a queued create sent, space-separated.
/// A create whose answer was lost may have made its file with any of them,
/// and the op that finds that file again may run after a restart. `NULL` for
/// an op that has sent nothing, which is every op migrated: what the daemon
/// that wrote it sent is not known. Likewise the parent and name each attempt
/// was sent under, as a JSON list of pairs: a rename since then moves the op
/// but not the file Drive may have made.
const MIGRATION_V42: &str = "
ALTER TABLE pending_op ADD COLUMN sent_sha1 TEXT;
ALTER TABLE pending_op ADD COLUMN sent_to TEXT;
";
+4 -3
View File
@@ -56,10 +56,11 @@ pub use local::LocalFileHit;
pub use maintenance::{DbStats, VacuumOutcome};
pub use nodes::{PublishedSharedRoot, SearchHit, StoredNode};
pub use ops::{
AttachedBlob, FAILING_ATTEMPTS, LOCAL_VOLUME, OP_CREATE, OP_MKDIR, OP_RENAME, OP_REVISION,
OP_TRASH, PARK_EXPIRY_MS, PARK_UNTIL, PendingCounts, PendingOp, RenameMeta, op_supersedes,
AttachedBlob, CreateLanding, CreateRetired, FAILING_ATTEMPTS, LOCAL_VOLUME, OP_CREATE,
OP_MKDIR, OP_RENAME, OP_REVISION, OP_TRASH, PARK_EXPIRY_MS, PARK_UNTIL, PendingCounts,
PendingOp, RenameMeta, local_lid, local_uid, op_supersedes,
};
pub use photos::{StoredPhoto, THUMB_HAVE, THUMB_NONE, THUMB_UNKNOWN, TimelineRow};
pub use photos::{StoredPhoto, StoredPlace, THUMB_HAVE, THUMB_NONE, THUMB_UNKNOWN, TimelineRow};
pub use pins::PinRow;
pub use sync::{StoredSyncEntry, StoredSyncFolder};
pub use trash::StoredTrash;
+444 -92
View File
@@ -3,9 +3,11 @@
use std::collections::HashSet;
use rusqlite::types::Value;
use rusqlite::{OptionalExtension, Transaction, params};
use super::Db;
use super::ops::{OP_CREATE, OP_MKDIR, OP_RENAME, local_lid, local_uid, wake_ops_waiting_for_tx};
use crate::{Access, Result};
use proton_drive_rs::proton_sdk::ids::{LinkId, NodeUid, VolumeId};
use proton_drive_rs::{Node, NodeKind};
@@ -17,6 +19,8 @@ use super::utils::{
pub struct StoredNode {
pub node: Node,
pub listed: bool,
/// The row's local id ([`Db::lid_of`]).
pub lid: i64,
}
/// One full-text search match: the stored [`Node`] plus its mountpoint-relative
@@ -62,12 +66,13 @@ impl Db {
/// snapshot while offline, including after a live event expired its TTL.
pub fn visible_children(&self, parent: &NodeUid) -> Result<Vec<Node>> {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT node_json FROM nodes
WHERE parent_uid = ?1 AND trashed = 0 AND node_json IS NOT NULL
ORDER BY name, uid",
)?;
let rows = stmt.query_map([parent.to_string()], |row| row.get::<_, String>(0))?;
let (key, parent) = parent_key(&conn, &parent.to_string())?;
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW} FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.{key} = ?1 AND n.trashed = 0 AND n.node_json IS NOT NULL
ORDER BY n.name, n.uid"
))?;
let rows = stmt.query_map([parent], |row| row.get::<_, String>(0))?;
let mut nodes = Vec::new();
for row in rows {
nodes.push(serde_json::from_str(&row?)?);
@@ -264,10 +269,25 @@ impl Db {
/// resolve its shared-tree access. Removing the op first could resurrect
/// that row after a crash; removing the row first could strand a now
/// unauthorizable op. One transaction closes both windows.
pub fn complete_trash_op(&self, op_id: i64, uid: &NodeUid) -> Result<()> {
///
/// An op that found the name still held by this trash failed with a
/// backoff, and the name is free now. It is made due at once
/// ([`Db::wake_ops_waiting_for`]).
///
/// Returns the staged blob of the create the trash withdrew, if it kept
/// one, for the caller to discard.
pub fn complete_trash_op(&self, op_id: i64, uid: &NodeUid) -> Result<Option<String>> {
let uid = uid.to_string();
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let (name, blob): (Option<String>, Option<String>) = tx
.query_row(
"SELECT name, blob_path FROM pending_op WHERE id = ?1",
params![op_id],
|row| Ok((row.get(0)?, row.get(1)?)),
)
.optional()?
.unwrap_or_default();
let rowid: Option<i64> = tx
.query_row(
"SELECT rowid FROM nodes WHERE uid = ?1",
@@ -280,17 +300,23 @@ impl Db {
tx.execute("DELETE FROM nodes_fts WHERE rowid = ?1", params![rowid])?;
}
tx.execute("DELETE FROM pending_op WHERE id = ?1", params![op_id])?;
if let Some(name) = name {
wake_ops_waiting_for_tx(&tx, &name)?;
}
tx.commit()?;
Ok(())
Ok(blob)
}
/// Check if a folder node has any non-trashed children in the database.
pub fn has_children(&self, parent_uid: &NodeUid) -> Result<bool> {
let uid_str = parent_uid.to_string();
let conn = self.read();
let (key, parent) = parent_key(&conn, &parent_uid.to_string())?;
let count: i64 = conn.query_row(
"SELECT COUNT(*) FROM nodes WHERE parent_uid = ?1 AND trashed = 0",
params![uid_str],
&format!(
"SELECT COUNT(*) FROM nodes
WHERE {key} = ?1 AND trashed = 0 AND node_json IS NOT NULL"
),
params![parent],
|row| row.get(0),
)?;
Ok(count > 0)
@@ -305,9 +331,11 @@ impl Db {
pub fn node_path(&self, uid: &str) -> Result<Option<String>> {
let conn = self.read();
let exists: Option<i64> = conn
.query_row("SELECT 1 FROM nodes WHERE uid = ?1", params![uid], |r| {
r.get(0)
})
.query_row(
"SELECT 1 FROM nodes WHERE uid = ?1 AND node_json IS NOT NULL",
params![uid],
|r| r.get(0),
)
.optional()?;
if exists.is_none() {
return Ok(None);
@@ -325,11 +353,11 @@ impl Db {
pub fn path_relative_to(&self, ancestor_uid: &str, uid: &str) -> Result<Option<String>> {
let conn = self.read();
let mut stmt = conn.prepare(
"WITH RECURSIVE anc(uid, parent_uid, name, depth) AS (
SELECT uid, parent_uid, name, 0 FROM nodes WHERE uid = ?1
"WITH RECURSIVE anc(uid, parent_lid, name, depth) AS (
SELECT uid, parent_lid, name, 0 FROM nodes WHERE uid = ?1
UNION ALL
SELECT n.uid, n.parent_uid, n.name, anc.depth + 1
FROM nodes n JOIN anc ON n.uid = anc.parent_uid
SELECT n.uid, n.parent_lid, n.name, anc.depth + 1
FROM nodes n JOIN anc ON n.lid = anc.parent_lid
WHERE anc.depth < 1024
)
SELECT uid, name FROM anc ORDER BY depth",
@@ -378,12 +406,13 @@ impl Db {
let conn = self.read();
let rows: Vec<HitRow> = if query.chars().count() < TRIGRAM_MIN {
let pat = format!("%{}%", like_escape(query));
let mut stmt = conn.prepare(
"SELECT node_json, uid, path FROM nodes
WHERE name LIKE ?1 ESCAPE '\\' AND trashed = 0 AND node_json IS NOT NULL
AND (?3 IS NULL OR path IS NULL OR path LIKE ?3 ESCAPE '\\')
ORDER BY name LIMIT ?2",
)?;
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW}, n.uid, n.path
FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.name LIKE ?1 ESCAPE '\\' AND n.trashed = 0 AND n.node_json IS NOT NULL
AND (?3 IS NULL OR n.path IS NULL OR n.path LIKE ?3 ESCAPE '\\')
ORDER BY n.name LIMIT ?2"
))?;
collect_hits(stmt.query_map(params![pat, limit as i64, scope_pat], hit_row)?)?
} else {
// Escape double quotes and quote each term, then combine with AND so
@@ -393,13 +422,14 @@ impl Db {
.map(|word| format!("\"{}\"", word.replace('"', "\"\"")))
.collect::<Vec<_>>()
.join(" AND ");
let mut stmt = conn.prepare(
"SELECT n.node_json, n.uid, n.path
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW}, n.uid, n.path
FROM nodes_fts f JOIN nodes n ON n.rowid = f.rowid
LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE f.name MATCH ?1 AND n.trashed = 0 AND n.node_json IS NOT NULL
AND (?3 IS NULL OR n.path IS NULL OR n.path LIKE ?3 ESCAPE '\\')
ORDER BY f.rank LIMIT ?2",
)?;
ORDER BY f.rank LIMIT ?2"
))?;
collect_hits(stmt.query_map(params![phrase, limit as i64, scope_pat], hit_row)?)?
};
@@ -436,20 +466,22 @@ impl Db {
let lane_limit = limit.div_ceil(2);
let mut rows: Vec<HitRow> = if terms.is_empty() {
let pat = format!("%{}%", like_escape(query));
let mut stmt = conn.prepare(
"SELECT node_json, uid, path FROM nodes
WHERE name LIKE ?1 ESCAPE '\\' AND trashed = 0 AND node_json IS NOT NULL
ORDER BY name LIMIT ?2",
)?;
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW}, n.uid, n.path
FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.name LIKE ?1 ESCAPE '\\' AND n.trashed = 0 AND n.node_json IS NOT NULL
ORDER BY n.name LIMIT ?2"
))?;
collect_hits(stmt.query_map(params![pat, limit as i64], hit_row)?)?
} else {
let expression = terms.join(" OR ");
let mut stmt = conn.prepare(
"SELECT n.node_json, n.uid, n.path
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW}, n.uid, n.path
FROM nodes_fts f JOIN nodes n ON n.rowid = f.rowid
LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE nodes_fts MATCH ?1 AND n.trashed = 0 AND n.node_json IS NOT NULL
ORDER BY f.rank LIMIT ?2",
)?;
ORDER BY f.rank LIMIT ?2"
))?;
collect_hits(stmt.query_map(params![expression, lane_limit as i64], hit_row)?)?
};
// Trigram rank does not know that a name *starts* with the query, and a
@@ -459,12 +491,13 @@ impl Db {
if !terms.is_empty()
&& let Some(first) = query.chars().next()
{
let mut stmt = conn.prepare(
"SELECT node_json, uid, path FROM nodes
WHERE name COLLATE NOCASE LIKE ?1 ESCAPE '\\'
AND trashed = 0 AND node_json IS NOT NULL
ORDER BY name COLLATE NOCASE LIMIT ?2",
)?;
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW}, n.uid, n.path
FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.name COLLATE NOCASE LIKE ?1 ESCAPE '\\'
AND n.trashed = 0 AND n.node_json IS NOT NULL
ORDER BY n.name COLLATE NOCASE LIMIT ?2"
))?;
let mut extra = Vec::new();
for pattern in [
format!("{}%", like_escape(query)),
@@ -500,40 +533,50 @@ impl Db {
/// re-enumerates from the remote on next access.
pub fn set_listed(&self, uid: &NodeUid, listed: bool) -> Result<()> {
let conn = self.conn.lock();
let (key, uid) = row_key(&uid.to_string());
conn.execute(
"UPDATE nodes SET listed = ?2 WHERE uid = ?1",
params![uid.to_string(), listed as i64],
&format!("UPDATE nodes SET listed = ?2 WHERE {key} = ?1"),
params![uid, listed as i64],
)?;
Ok(())
}
/// Load every persisted node for cold-start hydration of the `State` maps.
///
/// Each node names its parent as the parent's row has it now, found by
/// local id: a folder that landed has its new uid there, while the node
/// stored below it may still hold the placeholder.
pub fn load_all(&self) -> Result<Vec<StoredNode>> {
let conn = self.read();
let mut stmt =
conn.prepare("SELECT node_json, listed FROM nodes WHERE node_json IS NOT NULL")?;
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW}, n.listed, n.lid
FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.node_json IS NOT NULL"
))?;
let rows = stmt.query_map([], |row| {
let json: String = row.get(0)?;
let listed: i64 = row.get(1)?;
Ok((json, listed != 0))
Ok((json, listed != 0, row.get(2)?))
})?;
let mut out = Vec::new();
for row in rows {
let (json, listed) = row?;
let (json, listed, lid) = row?;
let node: Node = serde_json::from_str(&json)?;
out.push(StoredNode { node, listed });
out.push(StoredNode { node, listed, lid });
}
Ok(out)
}
/// Load one persisted node back by uid. Used to recover the My Files root
/// when the API is unreachable, so the mount can still serve the cached tree
/// (offline.md Phase 1).
/// when the API is unreachable, so the mount can still serve the cached tree.
pub fn node_by_uid(&self, uid: &str) -> Result<Option<Node>> {
let conn = self.read();
let json: Option<String> = conn
.query_row(
"SELECT node_json FROM nodes WHERE uid = ?1 AND node_json IS NOT NULL",
&format!(
"SELECT {NODE_NOW} FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.uid = ?1 AND n.node_json IS NOT NULL"
),
params![uid],
|r| r.get(0),
)
@@ -544,31 +587,178 @@ impl Db {
}
}
/// The uid of the node with local id `lid`: `None` when it has no row, or
/// is not on Drive yet.
pub fn uid_of_lid(&self, lid: i64) -> Result<Option<String>> {
let conn = self.read();
Ok(conn
.query_row("SELECT uid FROM nodes WHERE lid = ?1", params![lid], |r| {
r.get(0)
})
.optional()?
.flatten())
}
/// The local id of the node stored under `uid`, or `None` when it has no
/// row. It stays the same for as long as the row exists, including when a
/// create lands and the row takes the uid Drive gave it: a node made on
/// this machine is found by its stand-in ([`local_uid`]) before and after.
pub fn lid_of(&self, uid: &str) -> Result<Option<i64>> {
node_lid(&self.read(), uid)
}
/// The local id of each node, in order, giving a row to each that has none.
///
/// A new row is only a stub: it has no `node_json`, so no listing or search
/// returns it until the node is upserted, which the caller still owes. That
/// keeps this cheap enough to call with the inode lock held, which is
/// where an inode number is first needed.
pub fn lids_for(&self, nodes: &[&Node]) -> Result<Vec<i64>> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let mut lids = Vec::with_capacity(nodes.len());
{
let mut known = tx.prepare_cached("SELECT lid FROM nodes WHERE uid = ?1")?;
let mut stub = tx.prepare_cached(
"INSERT INTO nodes (uid, parent_uid, name, is_dir, mtime, trashed, parent_lid)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)",
)?;
for node in nodes {
let uid = node.uid.to_string();
let lid = match known.query_row(params![uid], |r| r.get(0)).optional()? {
Some(lid) => lid,
None => {
let parent = node.parent_uid.as_ref().map(ToString::to_string);
let parent_lid = match &parent {
Some(parent) => node_lid(&tx, parent)?,
None => None,
};
stub.execute(params![
uid,
parent,
node.name,
node.is_folder() as i64,
node.modification_time,
node.trashed as i64,
parent_lid,
])?;
let lid = tx.last_insert_rowid();
adopt_children_tx(&tx, &uid, lid)?;
lid
}
};
lids.push(lid);
}
}
tx.commit()?;
Ok(lids)
}
/// Give a node being made on this machine its row under `parent_uid`, and
/// return its local id. The row goes by [`local_uid`] of that id until
/// the node lands.
///
/// Like [`Db::lids_for`], only a stub: the caller upserts the node.
pub fn add_local_row(
&self,
parent_uid: &str,
name: &str,
is_dir: bool,
mtime: i64,
) -> Result<i64> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
tx.execute(
"INSERT INTO nodes (uid, parent_uid, name, is_dir, mtime, trashed, parent_lid)
VALUES (NULL, ?1, ?2, ?3, ?4, 0,
COALESCE(?5, (SELECT lid FROM nodes WHERE uid = ?1)))",
params![
parent_uid,
name,
is_dir as i64,
mtime,
local_lid(parent_uid)
],
)?;
let lid = tx.last_insert_rowid();
tx.execute(
"UPDATE nodes SET uid = ?2 WHERE lid = ?1",
params![lid, local_uid(lid)],
)?;
tx.commit()?;
Ok(lid)
}
/// Read the persisted incremental-sync cursor (a `DriveEventId`), if any.
/// The daemon resumes from this on restart instead of reseeding to the
/// server head, so changes made while unmounted are still applied (P2).
pub fn children_if_listed(&self, parent: &NodeUid) -> Result<Option<Vec<Node>>> {
let conn = self.read();
let (key, folder) = row_key(&parent.to_string());
let listed: Option<i64> = conn
.query_row(
"SELECT listed FROM nodes WHERE uid = ?1",
params![parent.to_string()],
&format!("SELECT listed FROM nodes WHERE {key} = ?1"),
params![folder],
|r| r.get(0),
)
.optional()?;
if listed != Some(1) {
return Ok(None);
}
let mut stmt = conn.prepare(
"SELECT node_json FROM nodes
WHERE parent_uid = ?1 AND node_json IS NOT NULL AND trashed = 0",
)?;
let rows = stmt.query_map(params![parent.to_string()], |r| r.get::<_, String>(0))?;
drop(conn);
self.known_children(parent).map(Some)
}
/// The children the DB knows under `parent`, whether or not the folder's
/// listing is still marked complete. Only for when nothing better can be
/// had: a listing dropped as stale may miss or keep a child.
pub fn known_children(&self, parent: &NodeUid) -> Result<Vec<Node>> {
let conn = self.read();
let (key, parent) = parent_key(&conn, &parent.to_string())?;
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW} FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.{key} = ?1 AND n.node_json IS NOT NULL AND n.trashed = 0"
))?;
let rows = stmt.query_map(params![parent], |r| r.get::<_, String>(0))?;
let mut out = Vec::new();
for json in rows {
out.push(serde_json::from_str(&json?)?);
}
Ok(Some(out))
Ok(out)
}
/// What a queued change does to `parent`'s listing, which Drive does not
/// show until it lands: the children made or moved in by a queued create,
/// mkdir or rename, as the DB holds them, and the nodes a queued rename
/// moves elsewhere.
///
/// A listing read from Drive is laid over with these, or a folder listed
/// while a create in it is queued, after a restart say, loses the file
/// until the create lands (`docs/BUGS.md` B132).
pub fn queued_children(&self, parent: &NodeUid) -> Result<(Vec<Node>, HashSet<String>)> {
let conn = self.read();
let (key, parent) = parent_key(&conn, &parent.to_string())?;
let mut stmt = conn.prepare(&format!(
"SELECT {NODE_NOW} FROM nodes n LEFT JOIN nodes p ON p.lid = n.parent_lid
WHERE n.{key} = ?1 AND n.node_json IS NOT NULL AND n.trashed = 0
AND EXISTS (SELECT 1 FROM pending_op o
WHERE (o.lid = n.lid OR o.uid = n.uid) AND o.kind IN (?2, ?3, ?4))"
))?;
let rows = stmt.query_map(params![parent, OP_CREATE, OP_MKDIR, OP_RENAME], |r| {
r.get::<_, String>(0)
})?;
let mut here = Vec::new();
for json in rows {
here.push(serde_json::from_str(&json?)?);
}
let mut stmt = conn.prepare(&format!(
"SELECT DISTINCT n.uid FROM pending_op p JOIN nodes n ON n.lid = p.lid OR n.uid = p.uid
WHERE p.kind = ?2 AND n.{key} IS NOT ?1"
))?;
let gone = stmt
.query_map(params![parent, OP_RENAME], |r| r.get::<_, String>(0))?
.collect::<std::result::Result<HashSet<_>, _>>()?;
Ok((here, gone))
}
// --- Content-cache LRU index (P4) -------------------------------------
@@ -590,9 +780,157 @@ struct PriorRow {
trashed: bool,
}
/// [`Db::lid_of`] on `conn`, which may be a transaction.
pub(super) fn node_lid(conn: &rusqlite::Connection, uid: &str) -> Result<Option<i64>> {
let row = match local_lid(uid) {
Some(lid) => conn.query_row("SELECT lid FROM nodes WHERE lid = ?1", params![lid], |r| {
r.get(0)
}),
None => conn.query_row("SELECT lid FROM nodes WHERE uid = ?1", params![uid], |r| {
r.get(0)
}),
};
Ok(row.optional()?)
}
/// The column and value that find `uid`'s own row. A stand-in is found by its
/// local id: a create that lands gives the row the uid Drive made before the
/// tree has it, and the tree asks by the stand-in until then (`docs/BUGS.md`
/// B177).
fn row_key(uid: &str) -> (&'static str, Value) {
match local_lid(uid) {
Some(lid) => ("lid", Value::Integer(lid)),
None => ("uid", Value::Text(uid.to_owned())),
}
}
/// The column and value a row names `parent` by: its local id when it has a
/// row, which the children keep when it lands, or its uid when it has none,
/// as a device folder's root does.
fn parent_key(conn: &rusqlite::Connection, parent: &str) -> Result<(&'static str, Value)> {
Ok(match node_lid(conn, parent)? {
Some(lid) => ("parent_lid", Value::Integer(lid)),
None => ("parent_uid", Value::Text(parent.to_owned())),
})
}
/// `n.node_json` naming its parent as the parent's row `p`, joined by
/// `n.parent_lid`, has it now. A folder that lands takes its real uid in its
/// own row only, so a node stored below it still names the stand-in.
const NODE_NOW: &str = "CASE WHEN p.uid IS NULL OR p.uid = n.parent_uid THEN n.node_json
ELSE json_set(n.node_json, '$.parent_uid', json_object(
'volume_id', substr(p.uid, 1, instr(p.uid, '~') - 1),
'link_id', substr(p.uid, instr(p.uid, '~') + 1))) END";
/// Hand a drained placeholder's row to the real uid it landed as.
///
/// Nothing else is rewritten. The rows below it and the ops made inside it
/// name it by its local id, which stays; what reads them for a uid takes the
/// one the row has now ([`NODE_NOW`], `PARENT_NOW` in `ops.rs`).
///
/// Dropping the placeholder row and waiting for the server's copy of the real
/// node left a window with no row for either uid. A queued child drained in
/// that window found no authority for its parent, re-read it over the network
/// and was deferred five seconds behind a folder that had just landed
/// (`docs/BUGS.md` B120). The placeholder's own node, readdressed, stands in
/// until the server's copy is upserted over it. A row the real uid already
/// has is newer than the placeholder, so its node is kept, under the
/// placeholder's `lid`: the node was known by that one first.
pub(super) fn adopt_placeholder_row_tx(
tx: &Transaction<'_>,
local: &str,
real: &str,
) -> Result<()> {
// A folder pinned before it landed stays pinned: the pin keyed by the
// placeholder was dropped with it (`docs/BUGS.md` B148).
tx.execute(
"UPDATE OR IGNORE pins SET uid = ?2 WHERE uid = ?1",
params![local, real],
)?;
tx.execute("DELETE FROM pins WHERE uid = ?1", params![local])?;
let row: Option<(i64, Option<String>)> = tx
.query_row(
"SELECT lid, node_json FROM nodes WHERE uid = ?1",
params![local],
|row| Ok((row.get(0)?, row.get(1)?)),
)
.optional()?;
// Ops queued before schema 39 have no `parent_lid`, nor do ops made
// under a placeholder that never had a row.
tx.execute(
"UPDATE pending_op SET parent_uid = ?2 WHERE parent_lid IS NULL AND parent_uid = ?1",
params![local, real],
)?;
let Some((lid, json)) = row else {
return Ok(());
};
let known: Option<(i64, Option<String>)> = tx
.query_row(
"SELECT lid, node_json FROM nodes WHERE uid = ?1",
params![real],
|row| Ok((row.get(0)?, row.get(1)?)),
)
.optional()?;
if let Some((known_lid, _)) = known {
// Ops queued against the row Drive listed now name the landing one.
tx.execute(
"UPDATE pending_op SET lid = ?2 WHERE lid = ?1",
params![known_lid, lid],
)?;
tx.execute(
"UPDATE pending_op SET parent_lid = ?2 WHERE parent_lid = ?1",
params![known_lid, lid],
)?;
tx.execute(
"UPDATE nodes SET parent_lid = ?2 WHERE parent_lid = ?1",
params![known_lid, lid],
)?;
tx.execute("DELETE FROM nodes WHERE lid = ?1", params![known_lid])?;
tx.execute("DELETE FROM nodes_fts WHERE rowid = ?1", params![known_lid])?;
}
tx.execute(
"UPDATE nodes SET uid = ?2 WHERE lid = ?1",
params![lid, real],
)?;
let json = known.and_then(|(_, known_json)| known_json).or(json);
let (Some(json), Some(real)) = (json, parse_node_uid(real)) else {
return Ok(());
};
let mut node: Node = serde_json::from_str(&json)?;
node.uid = real;
upsert_node_tx(tx, &node)
}
/// Link the rows that name `uid` as their parent to its new row `lid`: they
/// were stored before it, or under a row of the same uid since dropped.
fn adopt_children_tx(tx: &Transaction<'_>, uid: &str, lid: i64) -> Result<()> {
tx.execute(
"UPDATE nodes SET parent_lid = ?2 WHERE parent_uid = ?1 AND parent_lid IS NOT ?2",
params![uid, lid],
)?;
Ok(())
}
fn upsert_node_tx(tx: &Transaction<'_>, node: &Node) -> Result<()> {
let json = serde_json::to_string(node)?;
let uid = node.uid.to_string();
// A node written under its stand-in after it landed is its row, which has
// the uid Drive gave it by now: written under the stand-in, it would bring
// the placeholder row back beside it.
if let Some(lid) = local_lid(&uid) {
let landed: Option<String> = tx
.query_row(
"SELECT uid FROM nodes WHERE lid = ?1 AND uid IS NOT ?2",
params![lid, uid],
|row| row.get(0),
)
.optional()?;
if let Some(real) = landed.as_deref().and_then(parse_node_uid) {
let mut node = node.clone();
node.uid = real;
return upsert_node_tx(tx, &node);
}
}
let json = serde_json::to_string(node)?;
let parent_uid = node.parent_uid.as_ref().map(|u| u.to_string());
let prior: Option<PriorRow> = tx
@@ -614,26 +952,34 @@ fn upsert_node_tx(tx: &Transaction<'_>, node: &Node) -> Result<()> {
// where resolving it from scratch is a recursive walk to the root. A node
// whose parent is not cached (a device folder's root lives in `device`, never
// in `nodes`) starts a path of its own, which is what the walk did too.
let path = match &parent_uid {
None => String::new(),
let (parent_lid, path) = match &parent_uid {
None => (None, String::new()),
Some(parent) => {
let parent_path: Option<Option<String>> = tx
.query_row(
"SELECT path FROM nodes WHERE uid = ?1",
params![parent],
|row| row.get(0),
)
.optional()?;
join_path(parent_path.flatten().as_deref().unwrap_or(""), &node.name)
let parent_row: Option<(i64, Option<String>)> = match node_lid(tx, parent)? {
Some(lid) => tx
.query_row(
"SELECT lid, path FROM nodes WHERE lid = ?1",
params![lid],
|row| Ok((row.get(0)?, row.get(1)?)),
)
.optional()?,
None => None,
};
let (lid, parent_path) = parent_row.unzip();
(
lid,
join_path(parent_path.flatten().as_deref().unwrap_or(""), &node.name),
)
}
};
tx.execute(
"INSERT INTO nodes
(uid, parent_uid, name, is_dir, size, mtime, trashed, node_json, path)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)
(uid, parent_uid, name, is_dir, size, mtime, trashed, node_json, path, parent_lid)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)
ON CONFLICT(uid) DO UPDATE SET
parent_uid = excluded.parent_uid,
parent_lid = excluded.parent_lid,
name = excluded.name,
is_dir = excluded.is_dir,
size = excluded.size,
@@ -651,6 +997,7 @@ fn upsert_node_tx(tx: &Transaction<'_>, node: &Node) -> Result<()> {
node.trashed as i64,
json,
path,
parent_lid,
],
)?;
let rowid: i64 = tx.query_row(
@@ -658,6 +1005,9 @@ fn upsert_node_tx(tx: &Transaction<'_>, node: &Node) -> Result<()> {
params![uid],
|row| row.get(0),
)?;
if prior.is_none() {
adopt_children_tx(tx, &uid, rowid)?;
}
// FTS5 has no UPSERT, so the node's own row is always replaced.
let indexable = node_is_indexable_tx(tx, &uid)?;
@@ -680,7 +1030,7 @@ fn upsert_node_tx(tx: &Transaction<'_>, node: &Node) -> Result<()> {
|| prior.trashed != node.trashed
});
if node.is_folder() && subtree_moved {
reindex_subtree_tx(tx, &uid, &path, indexable)?;
reindex_subtree_tx(tx, rowid, &path, indexable)?;
}
Ok(())
}
@@ -701,22 +1051,22 @@ fn upsert_node_tx(tx: &Transaction<'_>, node: &Node) -> Result<()> {
/// this deep.
fn reindex_subtree_tx(
tx: &Transaction<'_>,
folder: &str,
folder: i64,
folder_path: &str,
folder_indexable: bool,
) -> Result<()> {
let descendants: Vec<(i64, String, bool)> = {
let mut stmt = tx.prepare(
"WITH RECURSIVE sub(rowid, uid, name, path, blocked, depth) AS (
SELECT n.rowid, n.uid, n.name,
"WITH RECURSIVE sub(rowid, name, path, blocked, depth) AS (
SELECT n.rowid, n.name,
CASE WHEN ?2 = '' THEN n.name ELSE ?2 || '/' || n.name END,
n.trashed, 0
FROM nodes n WHERE n.parent_uid = ?1
FROM nodes n WHERE n.parent_lid = ?1
UNION ALL
SELECT n.rowid, n.uid, n.name,
SELECT n.rowid, n.name,
sub.path || '/' || n.name,
MAX(sub.blocked, n.trashed), sub.depth + 1
FROM nodes n JOIN sub ON n.parent_uid = sub.uid
FROM nodes n JOIN sub ON n.parent_lid = sub.rowid
WHERE sub.depth < 256
)
SELECT rowid, path, blocked FROM sub",
@@ -741,8 +1091,10 @@ fn reindex_subtree_tx(
let mut set_path = tx.prepare("UPDATE nodes SET path = ?2 WHERE rowid = ?1")?;
let mut unindex = tx.prepare("DELETE FROM nodes_fts WHERE rowid = ?1")?;
let mut index = tx.prepare(
// Not a stub row (`Db::lids_for`): it is indexed when its node is
// written.
"INSERT INTO nodes_fts (rowid, name, path)
SELECT rowid, name, ?2 FROM nodes WHERE rowid = ?1",
SELECT rowid, name, ?2 FROM nodes WHERE rowid = ?1 AND node_json IS NOT NULL",
)?;
for (rowid, path, blocked) in descendants {
set_path.execute(params![rowid, path])?;
@@ -755,7 +1107,8 @@ fn reindex_subtree_tx(
}
fn direct_child_uids_tx(tx: &Transaction<'_>, parent: &str) -> Result<Vec<String>> {
let mut stmt = tx.prepare("SELECT uid FROM nodes WHERE parent_uid = ?1")?;
let (key, parent) = parent_key(tx, parent)?;
let mut stmt = tx.prepare(&format!("SELECT uid FROM nodes WHERE {key} = ?1"))?;
let rows = stmt.query_map([parent], |row| row.get::<_, String>(0))?;
let mut uids = Vec::new();
for row in rows {
@@ -780,14 +1133,14 @@ fn direct_child_uids_tx(tx: &Transaction<'_>, parent: &str) -> Result<Vec<String
/// ends at one whose parent is a row we have already visited.
fn node_is_indexable_tx(tx: &Transaction<'_>, uid: &str) -> Result<bool> {
let indexable: i64 = tx.query_row(
"WITH RECURSIVE ancestors(uid, parent_uid, trashed, depth, path) AS (
SELECT uid, parent_uid, trashed, 0, char(31) || uid || char(31)
"WITH RECURSIVE ancestors(lid, parent_lid, trashed, depth, path) AS (
SELECT lid, parent_lid, trashed, 0, ',' || lid || ','
FROM nodes WHERE uid = ?1
UNION ALL
SELECT n.uid, n.parent_uid, n.trashed, a.depth + 1,
a.path || n.uid || char(31)
FROM ancestors a JOIN nodes n ON n.uid = a.parent_uid
WHERE instr(a.path, char(31) || n.uid || char(31)) = 0
SELECT n.lid, n.parent_lid, n.trashed, a.depth + 1,
a.path || n.lid || ','
FROM ancestors a JOIN nodes n ON n.lid = a.parent_lid
WHERE instr(a.path, ',' || n.lid || ',') = 0
)
SELECT CASE
WHEN COUNT(*) = 0 THEN 0
@@ -795,8 +1148,7 @@ fn node_is_indexable_tx(tx: &Transaction<'_>, uid: &str) -> Result<bool> {
WHEN EXISTS (
SELECT 1 FROM ancestors a
WHERE a.depth = (SELECT MAX(depth) FROM ancestors)
AND a.parent_uid IS NOT NULL
AND EXISTS (SELECT 1 FROM nodes p WHERE p.uid = a.parent_uid)
AND EXISTS (SELECT 1 FROM nodes p WHERE p.lid = a.parent_lid)
) THEN 0
ELSE 1
END
@@ -814,7 +1166,7 @@ fn tombstone_subtree_tx(tx: &Transaction<'_>, root_uid: &str) -> Result<()> {
SELECT rowid, uid, node_json FROM nodes WHERE uid = ?1
UNION
SELECT n.rowid, n.uid, n.node_json FROM nodes n
JOIN subtree s ON n.parent_uid = s.uid
JOIN subtree s ON n.parent_lid = s.rowid
)
SELECT rowid, uid, node_json FROM subtree",
)?;
File diff suppressed because it is too large. Load diff
+307 -5
View File
@@ -3,7 +3,7 @@
use rusqlite::params;
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use super::Db;
use crate::Result;
@@ -42,6 +42,15 @@ pub struct StoredPhoto {
pub has_raw: bool,
}
/// A town photos were taken in: its GeoNames id (see [`crate::places`]), how
/// many photos it holds, and the newest of them.
#[derive(Clone, Debug, PartialEq)]
pub struct StoredPlace {
pub id: u32,
pub count: usize,
pub cover: StoredPhoto,
}
/// One row of a timeline replacement. Everything but the uid and the capture
/// time is "what this refresh learned": a `None` keeps whatever is already
/// stored rather than clearing it, because the timeline DTO carries only the uid
@@ -58,6 +67,8 @@ pub struct TimelineRow {
/// The main photo this one is *related* to (a live-photo video, a burst
/// sibling), when the server says it belongs to one.
pub main_uid: Option<String>,
/// Where the photo was taken, as `(latitude, longitude)` in decimal degrees.
pub location: Option<(f64, f64)>,
/// When this refresh resolved the photo's node, if it did at all. `Some`
/// makes every other field authoritative — including a `None` that means
/// "the server no longer reports one" — while `None` means the refresh
@@ -86,6 +97,9 @@ struct Learned {
favorite: bool,
content_hash: Option<String>,
main_uid: Option<String>,
location: Option<(f64, f64)>,
/// What its thumbnail looks like (see [`crate::similar`]).
similar_hash: Option<i64>,
/// When this photo's node was last resolved, kept so a refresh that skips it
/// does not make it look unresolved again.
resolved_at: Option<i64>,
@@ -281,7 +295,7 @@ impl Db {
let learned: HashMap<String, Learned> = {
let mut stmt = tx.prepare(
"SELECT uid, name, ratio, thumb_state, media_type, favorite, content_hash, \
main_uid, resolved_at FROM photos",
main_uid, resolved_at, latitude, longitude, similar_hash FROM photos",
)?;
let rows = stmt.query_map([], |r| {
Ok((
@@ -295,6 +309,10 @@ impl Db {
content_hash: r.get(6)?,
main_uid: r.get(7)?,
resolved_at: r.get(8)?,
location: r
.get::<_, Option<f64>>(9)?
.zip(r.get::<_, Option<f64>>(10)?),
similar_hash: r.get(11)?,
},
))
})?;
@@ -317,14 +335,20 @@ impl Db {
carried.name = row.name.clone();
carried.media_type = row.media_type.clone();
carried.favorite = row.favorite.unwrap_or(false);
// New bytes may show another picture: hash it again.
if row.content_hash != carried.content_hash {
carried.similar_hash = None;
}
carried.content_hash = row.content_hash.clone();
carried.main_uid = row.main_uid.clone();
carried.location = row.location;
} else {
carried.name = row.name.clone().or(carried.name);
carried.media_type = row.media_type.clone().or(carried.media_type);
carried.favorite = row.favorite.unwrap_or(carried.favorite);
carried.content_hash = row.content_hash.clone().or(carried.content_hash);
carried.main_uid = row.main_uid.clone().or(carried.main_uid);
carried.location = row.location.or(carried.location);
}
carried.resolved_at = row.resolved_at.or(carried.resolved_at);
// The tab this photo lands in is derived here, once, so a page or
@@ -355,8 +379,10 @@ impl Db {
let mut stmt = tx.prepare(
"INSERT INTO photos
(uid, capture_time, name, ratio, thumb_state, seq, media_type, kind, favorite,
content_hash, main_uid, group_key, resolved_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13)",
content_hash, main_uid, group_key, resolved_at, latitude, longitude,
place_id, similar_hash)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15,
?16, ?17)",
)?;
for (seq, ((row, carried, kind), group_key)) in
resolved.iter().zip(group_keys.iter()).enumerate()
@@ -375,6 +401,13 @@ impl Db {
carried.main_uid,
group_key,
carried.resolved_at,
carried.location.map(|(lat, _)| lat),
carried.location.map(|(_, lon)| lon),
carried
.location
.and_then(|(lat, lon)| crate::places::nearest(lat, lon))
.map(|city| city.id),
carried.similar_hash,
])?;
}
}
@@ -447,6 +480,14 @@ impl Db {
albums.execute([uid])?;
}
}
// A group whose representative went is left pointing at nothing, which
// drops its other files off every page. Make them stand alone until the
// next refresh groups them again.
tx.execute(
"UPDATE photos SET group_key = NULL WHERE group_key IS NOT NULL \
AND group_key NOT IN (SELECT uid FROM photos)",
[],
)?;
tx.commit()?;
Ok(removed)
}
@@ -455,7 +496,8 @@ impl Db {
/// restricts the page to one tab (Photos / Videos / Raw); `range`, when set,
/// restricts it to a `[from, to)` capture-time window (epoch seconds) — the
/// date scrubber's jump. `offset` is relative to whatever the filters leave.
/// `favorites` restricts the page to favorited photos.
/// `favorites` restricts the page to favorited photos, and `not_in_album`
/// to the shots no album holds any file of.
///
/// The page holds one row per *group*: the RAW and the JPEG of one shot are
/// one tile. The Raw tab is the exception and lists every raw file, because
@@ -467,6 +509,7 @@ impl Db {
kind: Option<crate::control::PhotoKind>,
range: Option<(i64, i64)>,
favorites: bool,
not_in_album: bool,
) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
// Built up so any combination of the optional filters is one indexed
@@ -490,6 +533,15 @@ impl Db {
if favorites {
conds.push("p.favorite = 1".to_string());
}
// By group, not by file: a shot filed as its JPEG is filed, even though
// its RAW was never added anywhere.
if not_in_album {
conds.push(
"NOT EXISTS (SELECT 1 FROM album_photos a JOIN photos m ON m.uid = a.uid \
WHERE COALESCE(m.group_key, m.uid) = COALESCE(p.group_key, p.uid))"
.to_string(),
);
}
if !conds.is_empty() {
sql.push_str(" WHERE ");
sql.push_str(&conds.join(" AND "));
@@ -577,6 +629,256 @@ impl Db {
Ok(rows)
}
/// Photos taken on `today`'s calendar day in earlier years, newest first —
/// the "On this day" strip. `today` is epoch seconds: its local month and day
/// pick the date, and its own year is left out, because today's photos are
/// what the timeline already opens on. One row per group, like the timeline.
pub fn photos_on_this_day(&self, today: i64, limit: usize) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
let sql = format!(
"{PHOTO_SELECT} WHERE p.uid = COALESCE(p.group_key, p.uid) \
AND strftime('%m-%d', p.capture_time, 'unixepoch', 'localtime') \
= strftime('%m-%d', ?1, 'unixepoch', 'localtime') \
AND strftime('%Y', p.capture_time, 'unixepoch', 'localtime') \
< strftime('%Y', ?1, 'unixepoch', 'localtime') \
ORDER BY p.seq LIMIT ?2"
);
let mut stmt = conn.prepare(&sql)?;
let rows = stmt
.query_map(params![today, limit as i64], stored_photo)?
.collect::<rusqlite::Result<_>>()?;
Ok(rows)
}
/// Every town photos were taken in, with how many and the newest one as its
/// cover, most photographed first. Counted per group, like the timeline;
/// photos without a location, or taken far from any town, are in none.
pub fn photo_places(&self) -> Result<Vec<StoredPlace>> {
let rows: Vec<(u32, usize, String)> = {
let conn = self.read();
// SQLite fills the bare `uid` from the row `MIN(seq)` picked, which
// is the newest photo of the place.
let mut stmt = conn.prepare(
"SELECT place_id, COUNT(*), uid, MIN(seq) FROM photos \
WHERE uid = COALESCE(group_key, uid) AND place_id IS NOT NULL \
GROUP BY place_id ORDER BY COUNT(*) DESC, MIN(seq)",
)?;
stmt.query_map([], |r| {
Ok((r.get(0)?, r.get::<_, i64>(1)? as usize, r.get(2)?))
})?
.collect::<rusqlite::Result<_>>()?
};
let uids: Vec<String> = rows.iter().map(|(_, _, uid)| uid.clone()).collect();
let mut covers: HashMap<String, StoredPhoto> = self
.photos_by_uid(&uids)?
.into_iter()
.map(|photo| (photo.uid.clone(), photo))
.collect();
Ok(rows
.into_iter()
.filter_map(|(id, count, uid)| {
Some(StoredPlace {
id,
count,
cover: covers.remove(&uid)?,
})
})
.collect())
}
/// One page of the photos taken in a town, newest first, one per group.
pub fn photos_at_place(
&self,
id: u32,
offset: usize,
limit: usize,
) -> Result<Vec<StoredPhoto>> {
let conn = self.read();
let sql = format!(
"{PHOTO_SELECT} WHERE p.uid = COALESCE(p.group_key, p.uid) AND p.place_id = ?1 \
ORDER BY p.seq LIMIT ?2 OFFSET ?3"
);
let mut stmt = conn.prepare(&sql)?;
let rows = stmt
.query_map(params![id, limit as i64, offset as i64], stored_photo)?
.collect::<rusqlite::Result<_>>()?;
Ok(rows)
}
/// Every photo taken in a town, one per group as in [`Self::photos_at_place`],
/// with where exactly it was taken. A group's own location is that of the
/// photo standing for it.
pub fn photo_locations_at_place(&self, id: u32) -> Result<Vec<(StoredPhoto, (f64, f64))>> {
let photos = self.photos_at_place(id, 0, usize::MAX >> 1)?;
let located: HashMap<String, (f64, f64)> = {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT uid, latitude, longitude FROM photos \
WHERE place_id = ?1 AND latitude IS NOT NULL AND longitude IS NOT NULL",
)?;
stmt.query_map(params![id], |r| Ok((r.get(0)?, (r.get(1)?, r.get(2)?))))?
.collect::<rusqlite::Result<_>>()?
};
Ok(photos
.into_iter()
.filter_map(|photo| {
let at = *located.get(&photo.uid)?;
Some((photo, at))
})
.collect())
}
/// Sets of photos with the same content hash, each set ordered with the
/// copy worth keeping first, and the sets newest first.
///
/// The copy to keep is the one in the most albums, then the one that is part
/// of a larger group (the JPEG's RAW is filed with it, a stray re-upload is
/// not), then the earliest in server order. At most one file per group is
/// listed: the files of one RAW+JPEG shot are not copies of each other, and a
/// set left with one member is no set.
pub fn photo_duplicates(&self) -> Result<Vec<Vec<StoredPhoto>>> {
let rows: Vec<(String, String, String)> = {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT p.uid, p.content_hash, COALESCE(p.group_key, p.uid) AS k FROM photos p \
WHERE p.content_hash IN (SELECT content_hash FROM photos \
WHERE content_hash IS NOT NULL GROUP BY content_hash HAVING COUNT(*) > 1) \
ORDER BY p.content_hash, \
(SELECT COUNT(*) FROM album_photos a WHERE a.uid = p.uid) DESC, \
(SELECT COUNT(*) FROM photos m WHERE COALESCE(m.group_key, m.uid) = COALESCE(p.group_key, p.uid)) DESC, \
p.seq",
)?;
stmt.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)))?
.collect::<rusqlite::Result<_>>()?
};
let uids: Vec<String> = rows.iter().map(|(uid, _, _)| uid.clone()).collect();
let mut stored: HashMap<String, StoredPhoto> = self
.photos_by_uid(&uids)?
.into_iter()
.map(|photo| (photo.uid.clone(), photo))
.collect();
let mut sets: Vec<Vec<StoredPhoto>> = Vec::new();
let mut hash_of_set: Option<&str> = None;
let mut groups_in_set: HashSet<&str> = HashSet::new();
for (uid, hash, group) in &rows {
if hash_of_set != Some(hash.as_str()) {
hash_of_set = Some(hash);
groups_in_set.clear();
sets.push(Vec::new());
}
if !groups_in_set.insert(group) {
continue;
}
if let (Some(set), Some(photo)) = (sets.last_mut(), stored.remove(uid)) {
set.push(photo);
}
}
sets.retain(|set| set.len() > 1);
sets.sort_by_key(|set| std::cmp::Reverse(set.iter().map(|p| p.capture_time).max()));
Ok(sets)
}
/// Shots whose look hasn't been hashed yet, by the uid of the photo the
/// grid shows for each, newest first, at most `limit`. Videos, and photos
/// with no thumbnail to hash, are left out. A shot counts as hashed once
/// any of its files is, since they all show one picture.
pub fn photos_unhashed(&self, limit: usize) -> Result<Vec<StoredPhoto>> {
let uids: Vec<String> = {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT COALESCE(group_key, uid) AS k FROM photos \
GROUP BY k \
HAVING MAX(similar_hash) IS NULL \
AND MAX(kind != ?1 AND thumb_state != ?2) \
ORDER BY MAX(capture_time) DESC LIMIT ?3",
)?;
stmt.query_map(
params![
crate::control::PhotoKind::Video.as_i64(),
THUMB_NONE,
limit as i64
],
|r| r.get(0),
)?
.collect::<rusqlite::Result<_>>()?
};
self.photos_by_uid(&uids)
}
/// Record the look of photo `uid`: its difference hash, or
/// [`crate::similar::NO_HASH`] when its thumbnail gave none.
pub fn photo_set_similar_hash(&self, uid: &str, hash: u64) -> Result<()> {
let conn = self.conn.lock();
conn.execute(
"UPDATE photos SET similar_hash = ?2 WHERE uid = ?1",
params![uid, hash as i64],
)?;
Ok(())
}
/// How many shots have been hashed, out of how many could be.
pub fn similar_progress(&self) -> Result<(usize, usize)> {
let conn = self.read();
let (hashed, total): (i64, i64) = conn.query_row(
"SELECT COUNT(*) FILTER (WHERE hashed), COUNT(*) FROM \
(SELECT MAX(similar_hash) IS NOT NULL AS hashed FROM photos \
GROUP BY COALESCE(group_key, uid) \
HAVING MAX(similar_hash) IS NOT NULL \
OR MAX(kind != ?1 AND thumb_state != ?2))",
params![crate::control::PhotoKind::Video.as_i64(), THUMB_NONE],
|r| Ok((r.get(0)?, r.get(1)?)),
)?;
Ok((hashed.max(0) as usize, total.max(0) as usize))
}
/// Sets of shots that look alike (see [`crate::similar`]), one photo per
/// shot in capture order, and the sets newest first. A set whose photos
/// are all byte-identical copies is left to [`Self::photo_duplicates`].
pub fn photo_similar(&self) -> Result<Vec<Vec<StoredPhoto>>> {
let rows: Vec<(String, u64, Option<String>)> = {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT COALESCE(group_key, uid) AS k, MAX(similar_hash), \
MAX(content_hash) FILTER (WHERE uid = COALESCE(group_key, uid)) \
FROM photos GROUP BY k \
HAVING MAX(similar_hash) IS NOT NULL AND MAX(similar_hash) != ?1",
)?;
stmt.query_map(params![crate::similar::NO_HASH as i64], |r| {
Ok((r.get(0)?, r.get::<_, i64>(1)? as u64, r.get(2)?))
})?
.collect::<rusqlite::Result<_>>()?
};
let hashes: Vec<u64> = rows.iter().map(|(_, hash, _)| *hash).collect();
let found: Vec<Vec<usize>> = crate::similar::sets(&hashes)
.into_iter()
.filter(|set| {
let first = &rows[set[0]].2;
first.is_none() || set.iter().any(|&i| rows[i].2 != *first)
})
.collect();
let uids: Vec<String> = found.iter().flatten().map(|&i| rows[i].0.clone()).collect();
let mut stored: HashMap<String, StoredPhoto> = self
.photos_by_uid(&uids)?
.into_iter()
.map(|photo| (photo.uid.clone(), photo))
.collect();
let mut sets: Vec<Vec<StoredPhoto>> = found
.iter()
.map(|set| {
let mut photos: Vec<StoredPhoto> = set
.iter()
.filter_map(|&i| stored.remove(&rows[i].0))
.collect();
photos.sort_by_key(|p| p.capture_time);
photos
})
.filter(|set| set.len() > 1)
.collect();
sets.sort_by_key(|set| std::cmp::Reverse(set.iter().map(|p| p.capture_time).max()));
Ok(sets)
}
/// Per-tab counts for the Photos page subtitle: `(photos, videos, raw)`.
pub fn photos_counts(&self) -> Result<(usize, usize, usize)> {
use crate::control::PhotoKind;
+15 -13
View File
@@ -64,11 +64,11 @@ impl Db {
}
/// Whether `uid` is pinned — directly, or because a strict ancestor folder
/// is pinned recursively. The ancestor check walks `parent_uid` to the root
/// is pinned recursively. The ancestor check walks `parent_lid` to the root
/// via a CTE; a direct pin is honoured even when the node has no `nodes` row
/// yet (e.g. a CLI that never hydrates the node cache).
///
/// The walk is depth-capped like [`super::utils::path_of`]: a `parent_uid`
/// The walk is depth-capped like [`super::utils::path_of`]: a parent
/// cycle is corrupt data the API can hand us, and `UNION ALL` over one never
/// terminates — while holding the daemon's only SQLite connection, on a path
/// that runs per cached read.
@@ -85,11 +85,11 @@ impl Db {
let anc: Option<i64> = conn
.query_row(
&format!(
"WITH RECURSIVE anc(uid, parent_uid, depth) AS (
SELECT uid, parent_uid, 0 FROM nodes WHERE uid = ?1
"WITH RECURSIVE anc(uid, parent_lid, depth) AS (
SELECT uid, parent_lid, 0 FROM nodes WHERE uid = ?1
UNION ALL
SELECT n.uid, n.parent_uid, anc.depth + 1
FROM nodes n JOIN anc ON n.uid = anc.parent_uid
SELECT n.uid, n.parent_lid, anc.depth + 1
FROM nodes n JOIN anc ON n.lid = anc.parent_lid
WHERE anc.depth < {MAX_PATH_DEPTH}
)
SELECT 1 FROM anc a JOIN pins p ON p.uid = a.uid
@@ -109,10 +109,11 @@ impl Db {
pub fn pinned_uids(&self) -> Result<Vec<String>> {
let conn = self.read();
let mut stmt = conn.prepare(
"WITH RECURSIVE sub(uid) AS (
SELECT uid FROM pins WHERE recursive = 1
"WITH RECURSIVE sub(uid, lid) AS (
SELECT p.uid, n.lid FROM pins p LEFT JOIN nodes n ON n.uid = p.uid
WHERE p.recursive = 1
UNION
SELECT n.uid FROM nodes n JOIN sub ON n.parent_uid = sub.uid
SELECT n.uid, n.lid FROM nodes n JOIN sub ON n.parent_lid = sub.lid
)
SELECT uid FROM sub
UNION
@@ -126,15 +127,16 @@ impl Db {
Ok(out)
}
/// Every descendant uid of `folder` (all depths), via a `parent_uid` CTE.
/// Every descendant uid of `folder` (all depths), via a `parent_lid` CTE.
/// Used to evict a recursively-pinned subtree's cached blobs on unpin.
pub fn descendants(&self, folder: &str) -> Result<Vec<String>> {
let conn = self.read();
let mut stmt = conn.prepare(
"WITH RECURSIVE sub(uid) AS (
SELECT uid FROM nodes WHERE parent_uid = ?1
"WITH RECURSIVE sub(uid, lid) AS (
SELECT c.uid, c.lid FROM nodes c JOIN nodes f ON c.parent_lid = f.lid
WHERE f.uid = ?1
UNION
SELECT n.uid FROM nodes n JOIN sub ON n.parent_uid = sub.uid
SELECT n.uid, n.lid FROM nodes n JOIN sub ON n.parent_lid = sub.lid
)
SELECT uid FROM sub",
)?;
+26 -9
View File
@@ -95,11 +95,27 @@ impl Db {
/// Resolve a persisted node's effective access through its nearest recorded
/// share root. `None` means the node itself is not persisted, which callers
/// must not treat as owned: stale handles need to fail closed.
///
/// A stand-in ([`local_uid`](super::local_uid)) finds its row by local id,
/// so it still does once its create has landed and the row has the uid
/// Drive gave it. The tree learns that uid a moment later, and a rename in
/// between was refused (`docs/BUGS.md` B174).
pub fn effective_node_access(&self, uid: &NodeUid) -> Result<Option<Access>> {
let conn = self.read();
let key = uid.to_string();
let key = match super::local_lid(&key) {
Some(lid) => conn
.query_row("SELECT uid FROM nodes WHERE lid = ?1", [lid], |row| {
row.get::<_, Option<String>>(0)
})
.optional()?
.flatten()
.unwrap_or(key),
None => key,
};
let exists: bool = conn.query_row(
"SELECT EXISTS(SELECT 1 FROM nodes WHERE uid = ?1)",
[uid.to_string()],
"SELECT EXISTS(SELECT 1 FROM nodes WHERE uid = ?1 AND node_json IS NOT NULL)",
[&key],
|row| row.get(0),
)?;
if !exists {
@@ -107,25 +123,26 @@ impl Db {
}
let value = conn
.query_row(
"WITH RECURSIVE ancestors(uid, parent_uid, depth, path) AS (
SELECT uid, parent_uid, 0, char(31) || uid || char(31)
"WITH RECURSIVE ancestors(uid, parent_uid, parent_lid, depth, path) AS (
SELECT uid, parent_uid, parent_lid, 0, char(31) || uid || char(31)
FROM nodes WHERE uid = ?1
UNION ALL
SELECT a.parent_uid, n.parent_uid, a.depth + 1,
a.path || a.parent_uid || char(31)
SELECT COALESCE(n.uid, a.parent_uid), n.parent_uid, n.parent_lid,
a.depth + 1,
a.path || COALESCE(n.uid, a.parent_uid) || char(31)
FROM ancestors a
LEFT JOIN nodes n ON n.uid = a.parent_uid
LEFT JOIN nodes n ON n.lid = a.parent_lid
WHERE a.parent_uid IS NOT NULL
AND instr(
a.path,
char(31) || a.parent_uid || char(31)
char(31) || COALESCE(n.uid, a.parent_uid) || char(31)
) = 0
)
SELECT sa.access
FROM ancestors a JOIN share_access sa ON sa.root_uid = a.uid
ORDER BY a.depth ASC
LIMIT 1",
[uid.to_string()],
[&key],
|row| row.get::<_, String>(0),
)
.optional()?;
+86 -4
View File
@@ -25,8 +25,8 @@ pub struct StoredSyncFolder {
}
/// One per-file sync baseline row: what a path looked like on both sides at the
/// last successful sync, so the next reconcile can tell which side changed
/// (devices.md Phase 2). `remote_rev`/`remote_hash` hold the remote signature —
/// last successful sync, so the next reconcile can tell which side changed.
/// `remote_rev`/`remote_hash` hold the remote signature —
/// its modification time and size as strings — since no cheap content hash is
/// exposed; change detection is `(mtime, size)` on each side.
#[derive(Clone, Debug, PartialEq)]
@@ -39,7 +39,7 @@ pub struct StoredSyncEntry {
///
/// `local_mtime` alone cannot tell an edit that landed inside the same
/// second as the last sync — and left the file the same length — from no
/// edit at all, so those never uploaded (bugs.md B25). `None` on a row
/// edit at all, so those never uploaded (`docs/BUGS.md` B25). `None` on a row
/// written before schema 28, which the comparison reads as "compare whole
/// seconds", so upgrading does not make an entire mirror look changed.
pub local_mtime_ns: Option<i64>,
@@ -260,10 +260,92 @@ impl Db {
Ok(())
}
/// Drop the baseline rows of `rel_path` and everything under it.
///
/// Used when a subtree leaves a mirror folder through a move on Drive: its
/// local copy is about to be removed, and rows left behind would read as
/// "deleted locally" and trash the moved nodes at their new place.
pub fn sync_entries_remove_subtree(&self, folder_id: i64, rel_path: &str) -> Result<()> {
let conn = self.conn.lock();
conn.execute(
"DELETE FROM sync_entry
WHERE folder_id = ?1
AND (rel_path = ?2 OR substr(rel_path, 1, length(?2) + 1) = ?2 || '/')",
params![folder_id, rel_path],
)?;
Ok(())
}
/// Move the baseline rows of `from_rel` and everything under it to `to_rel`
/// in folder `to_folder`, in one transaction.
///
/// Used when a move on Drive took a mirror subtree along with its local
/// copy: the rows still describe both sides correctly, only under a new
/// path, so the next pass on either folder has nothing to do. The two
/// folders may be the same one.
pub fn sync_entries_move(
&self,
from_folder: i64,
from_rel: &str,
to_folder: i64,
to_rel: &str,
) -> Result<()> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
{
let mut select = tx.prepare(
"SELECT rel_path, remote_uid, local_mtime, local_size, remote_rev, remote_hash,
local_mtime_ns
FROM sync_entry
WHERE folder_id = ?1
AND (rel_path = ?2 OR substr(rel_path, 1, length(?2) + 1) = ?2 || '/')",
)?;
let rows = select
.query_map(params![from_folder, from_rel], |r| {
Ok(StoredSyncEntry {
rel_path: r.get(0)?,
remote_uid: r.get(1)?,
local_mtime: r.get(2)?,
local_size: r.get(3)?,
remote_rev: r.get(4)?,
remote_hash: r.get(5)?,
local_mtime_ns: r.get(6)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
let mut delete =
tx.prepare("DELETE FROM sync_entry WHERE folder_id = ?1 AND rel_path = ?2")?;
for row in &rows {
delete.execute(params![from_folder, row.rel_path])?;
}
let mut insert = tx.prepare(
"INSERT OR REPLACE INTO sync_entry
(folder_id, rel_path, remote_uid, local_mtime, local_size, remote_rev,
remote_hash, local_mtime_ns)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
)?;
for row in rows {
let rel = format!("{to_rel}{}", &row.rel_path[from_rel.len()..]);
insert.execute(params![
to_folder,
rel,
row.remote_uid,
row.local_mtime,
row.local_size,
row.remote_rev,
row.remote_hash,
row.local_mtime_ns,
])?;
}
}
tx.commit()?;
Ok(())
}
/// Drop the entire baseline for a folder. Used when flipping ondemand→mirror:
/// the local tree was evicted, so the old baseline is stale and would make the
/// next reconcile mistake "locally deleted" for "must re-download". Clearing it
/// leaves an empty baseline + full remote = pure download (devices.md P3).
/// leaves an empty baseline + full remote = pure download.
pub fn sync_entries_clear(&self, folder_id: i64) -> Result<()> {
let conn = self.conn.lock();
conn.execute(
File diff suppressed because it is too large. Load diff
+26 -5
View File
@@ -1,5 +1,7 @@
//! Trashed nodes, persisted so the trash view opens without a round-trip.
use std::collections::HashMap;
use rusqlite::params;
use super::Db;
@@ -15,6 +17,9 @@ pub struct StoredTrash {
/// row written before schema v29, and for a node whose parent the server
/// did not report.
pub parent_uid: Option<String>,
/// When the daemon first saw the node in the trash, epoch seconds. `None`
/// when it cannot say (see schema v35).
pub trashed_at: Option<i64>,
}
impl Db {
@@ -24,8 +29,8 @@ impl Db {
tx.execute("DELETE FROM trash", [])?;
{
let mut stmt = tx.prepare(
"INSERT INTO trash (uid, name, is_dir, size, mtime, parent_uid) \
VALUES (?1, ?2, ?3, ?4, ?5, ?6)",
"INSERT INTO trash (uid, name, is_dir, size, mtime, parent_uid, trashed_at) \
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)",
)?;
for item in items {
stmt.execute(params![
@@ -34,7 +39,8 @@ impl Db {
item.is_dir as i64,
item.size,
item.mtime,
item.parent_uid
item.parent_uid,
item.trashed_at
])?;
}
}
@@ -47,7 +53,7 @@ impl Db {
pub fn trash_list(&self) -> Result<Vec<StoredTrash>> {
let conn = self.read();
let mut stmt = conn.prepare(
"SELECT uid, name, is_dir, size, mtime, parent_uid FROM trash
"SELECT uid, name, is_dir, size, mtime, parent_uid, trashed_at FROM trash
ORDER BY is_dir DESC, name COLLATE NOCASE",
)?;
let rows = stmt.query_map([], |r| {
@@ -58,6 +64,7 @@ impl Db {
size: r.get(3)?,
mtime: r.get(4)?,
parent_uid: r.get(5)?,
trashed_at: r.get(6)?,
})
})?;
let mut items = Vec::new();
@@ -81,5 +88,19 @@ impl Db {
Ok(pairs)
}
// ---- device sync (devices.md) -----------------------------------------
/// When each node now in the trash was first seen there, keyed by uid, so
/// a refresh can carry the times over into the listing it writes.
pub fn trash_seen(&self) -> Result<HashMap<String, Option<i64>>> {
let conn = self.read();
let mut stmt = conn.prepare("SELECT uid, trashed_at FROM trash")?;
let rows = stmt.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))?;
let mut seen = HashMap::new();
for row in rows {
let (uid, at) = row?;
seen.insert(uid, at);
}
Ok(seen)
}
// ---- device sync -----------------------------------------
}
+6 -6
View File
@@ -72,18 +72,18 @@ pub(super) fn path_of(conn: &Connection, uid: &str) -> Result<String> {
}
}
/// Resolve a path by walking `parent_uid` to the root via a recursive CTE.
/// Resolve a path by walking `parent_lid` to the root via a recursive CTE.
///
/// A cycle in `parent_uid` is corrupt data the API can still hand us, and
/// A cycle in the parents is corrupt data the API can still hand us, and
/// `UNION ALL` over one never terminates — the walk is therefore depth-capped
/// and returns the truncated path rather than hanging the caller.
pub(super) fn walk_path_of(conn: &Connection, uid: &str) -> Result<String> {
let mut stmt = conn.prepare(&format!(
"WITH RECURSIVE anc(uid, parent_uid, name, depth) AS (
SELECT uid, parent_uid, name, 0 FROM nodes WHERE uid = ?1
"WITH RECURSIVE anc(parent_lid, parent_uid, name, depth) AS (
SELECT parent_lid, parent_uid, name, 0 FROM nodes WHERE uid = ?1
UNION ALL
SELECT n.uid, n.parent_uid, n.name, anc.depth + 1
FROM nodes n JOIN anc ON n.uid = anc.parent_uid
SELECT n.parent_lid, n.parent_uid, n.name, anc.depth + 1
FROM nodes n JOIN anc ON n.lid = anc.parent_lid
WHERE anc.depth < {MAX_PATH_DEPTH}
)
SELECT name, parent_uid FROM anc ORDER BY depth DESC"
+14
View File
@@ -66,6 +66,20 @@ impl Error {
}
}
impl Error {
/// Whether a control-socket round-trip gave up waiting for the reply: the
/// daemon is running but stuck, as opposed to not running at all.
pub fn is_timeout(&self) -> bool {
matches!(
self,
Error::Io(e) if matches!(
e.kind(),
std::io::ErrorKind::WouldBlock | std::io::ErrorKind::TimedOut
)
)
}
}
pub type Result<T> = std::result::Result<T, Error>;
/// A failure on its way to a front-end: prose for the user, plus the
+2
View File
@@ -14,11 +14,13 @@ pub mod localindex;
pub mod menu;
pub mod mounts;
pub mod opener;
pub mod places;
pub mod profile;
pub mod sdkcache;
pub mod search;
pub mod service;
pub mod shell;
pub mod similar;
pub mod syncignore;
pub mod takeout;
pub mod tray;
+185
View File
@@ -0,0 +1,185 @@
//! Offline reverse geocoding: the town a photo was taken in, from its GPS
//! position, without asking any service.
//!
//! The towns are GeoNames' `cities15000` dump (every place with more than
//! 15,000 people), trimmed to id, name, position and country in
//! `data/cities15000.tsv` by `scripts/geonames-cities.py`. The script drops the
//! parts of a city, such as Kreuzberg or "Paris 15 Vaugirard", so that a photo
//! lands in the town people would name. GeoNames publishes the dump under
//! CC BY 4.0, which is why the About dialog credits it.
//!
//! `data/countries.tsv` maps each country code to its ISO 3166-1 English name,
//! as the `iso-codes` project spells it, so a front end can translate it
//! through that project's `iso_3166-1` gettext domain.
use std::collections::HashMap;
use std::sync::OnceLock;
/// How far from the nearest town a photo may be and still count as taken
/// there, in km. Farther out (at sea, in the wilderness) it has no place.
pub const MAX_DISTANCE_KM: f64 = 50.0;
/// Mean Earth radius in km, for the haversine distance.
const EARTH_RADIUS_KM: f64 = 6371.0;
const CITIES: &str = include_str!("../data/cities15000.tsv");
const COUNTRIES: &str = include_str!("../data/countries.tsv");
/// One town of the table.
#[derive(Debug, Clone, PartialEq)]
pub struct City {
/// The GeoNames id, stable across dumps.
pub id: u32,
pub name: &'static str,
pub latitude: f64,
pub longitude: f64,
/// ISO 3166-1 alpha-2 country code.
pub country: &'static str,
}
/// The towns, and an index of them by whole-degree cell.
struct Table {
cities: Vec<City>,
cells: HashMap<(i32, i32), Vec<usize>>,
}
fn table() -> &'static Table {
static TABLE: OnceLock<Table> = OnceLock::new();
TABLE.get_or_init(|| {
let cities: Vec<City> = CITIES.lines().filter_map(parse_city).collect();
let mut cells: HashMap<(i32, i32), Vec<usize>> = HashMap::new();
for (index, city) in cities.iter().enumerate() {
cells
.entry(cell(city.latitude, city.longitude))
.or_default()
.push(index);
}
Table { cities, cells }
})
}
fn parse_city(line: &'static str) -> Option<City> {
let mut fields = line.split('\t');
Some(City {
id: fields.next()?.parse().ok()?,
name: fields.next()?,
latitude: fields.next()?.parse().ok()?,
longitude: fields.next()?.parse().ok()?,
country: fields.next()?,
})
}
fn cell(latitude: f64, longitude: f64) -> (i32, i32) {
(latitude.floor() as i32, longitude.floor() as i32)
}
/// Great-circle distance between two positions, in km.
pub fn distance_km(a: (f64, f64), b: (f64, f64)) -> f64 {
let (lat1, lat2) = (a.0.to_radians(), b.0.to_radians());
let dlat = lat2 - lat1;
let dlon = (b.1 - a.1).to_radians();
let h = (dlat / 2.0).sin().powi(2) + lat1.cos() * lat2.cos() * (dlon / 2.0).sin().powi(2);
2.0 * EARTH_RADIUS_KM * h.sqrt().min(1.0).asin()
}
/// The town nearest to a position, if one lies within [`MAX_DISTANCE_KM`].
pub fn nearest(latitude: f64, longitude: f64) -> Option<&'static City> {
if !latitude.is_finite() || !longitude.is_finite() {
return None;
}
let table = table();
// A degree of latitude is about 111 km everywhere; a degree of longitude
// shrinks towards the poles, so more cells are needed to cover the radius.
let lat_span = (MAX_DISTANCE_KM / 111.0).ceil() as i32;
let lon_span = {
let scale = latitude.to_radians().cos().abs().max(0.01);
((MAX_DISTANCE_KM / (111.0 * scale)).ceil() as i32).min(180)
};
let (row, column) = cell(latitude, longitude);
let mut best: Option<(f64, &City)> = None;
for dlat in -lat_span..=lat_span {
for dlon in -lon_span..=lon_span {
// Longitude wraps at the antimeridian: -181 is 179.
let lon_cell = (column + dlon + 180).rem_euclid(360) - 180;
let Some(indices) = table.cells.get(&(row + dlat, lon_cell)) else {
continue;
};
for &index in indices {
let city = &table.cities[index];
let d = distance_km((latitude, longitude), (city.latitude, city.longitude));
if d <= MAX_DISTANCE_KM && best.is_none_or(|(bd, _)| d < bd) {
best = Some((d, city));
}
}
}
}
best.map(|(_, city)| city)
}
/// The town with this GeoNames id.
pub fn city(id: u32) -> Option<&'static City> {
let cities = &table().cities;
// The table is sorted by id.
cities
.binary_search_by_key(&id, |city| city.id)
.ok()
.map(|index| &cities[index])
}
/// The English ISO 3166-1 name of a country code, such as "Germany" for "DE".
pub fn country_name(code: &str) -> Option<&'static str> {
COUNTRIES.lines().find_map(|line| {
let (c, name) = line.split_once('\t')?;
(c == code).then_some(name)
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_position_in_a_town_resolves_to_that_town() {
let berlin = nearest(52.52, 13.405).expect("Berlin is a town");
assert_eq!(berlin.name, "Berlin");
assert_eq!(berlin.country, "DE");
assert_eq!(city(berlin.id), Some(berlin));
}
#[test]
fn the_open_sea_has_no_town() {
assert_eq!(nearest(0.0, -30.0), None);
assert_eq!(nearest(f64::NAN, 0.0), None);
}
#[test]
fn a_town_across_the_antimeridian_is_still_found() {
// Just east of 180°, the nearest towns in Fiji are west of it.
let town = nearest(-16.8, 179.99);
assert!(town.is_none_or(|t| t.country == "FJ"));
let suva = nearest(-18.14, 178.44).expect("Suva is a town");
assert_eq!(suva.country, "FJ");
}
#[test]
fn every_town_has_a_named_country() {
for city in &table().cities {
assert!(
country_name(city.country).is_some(),
"{} has no country name for {}",
city.name,
city.country
);
}
assert_eq!(country_name("DE"), Some("Germany"));
}
#[test]
fn distances_are_great_circle_km() {
let d = distance_km((52.52, 13.405), (48.137, 11.575));
assert!(
(d - 504.0).abs() < 5.0,
"Berlin to Munich is about 504 km, got {d}"
);
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
//! The machine profile that travels with a device (features.md 5.3).
//! The machine profile that travels with a device.
//!
//! Sync folder mappings, pins, and the settings that shape them live in
//! `cache.db` and `config.json` — both on the local disk. When that disk dies,
+10 -3
View File
@@ -55,9 +55,10 @@ fn systemctl_query(args: &[&str]) -> bool {
/// keyring checks). `--now` is a no-op on an active unit, so the daemon would
/// keep sleeping and the mount would not appear until the next poll fired.
/// Restarting makes the new session take effect immediately, and starts the
/// unit when it is not running at all.
/// unit when it is not running at all. `reenable` for the reason given on
/// [`enable`].
pub fn enable_start() -> bool {
let enabled = systemctl(&["enable"]);
let enabled = systemctl(&["reenable"]);
systemctl(&["restart"]) && enabled
}
@@ -97,8 +98,14 @@ pub fn is_enabled() -> bool {
/// Enable the service so it auto-starts on future logins, without necessarily
/// starting it now beyond what `enable` does. Used by the Settings "Start on
/// login" switch; pairs with [`disable`](disable).
///
/// `reenable` rather than `enable`: older installs were enabled
/// into `default.target.wants`, and a plain `enable` would add the new
/// `graphical-session.target.wants` link while leaving the old one to start the
/// unit too early. `reenable` drops every existing link and recreates them from
/// the current `[Install]` section.
pub fn enable() -> bool {
systemctl(&["enable"])
systemctl(&["reenable"])
}
/// Disable auto-start without stopping the running mount. Used by the Settings
+193
View File
@@ -0,0 +1,193 @@
//! Telling photos apart by what they show rather than by their bytes.
//!
//! The server's content hash only matches byte-identical files. A photo that
//! was resized, re-encoded or sent through a messenger is another file, so the
//! similar-photo finder hashes each thumbnail instead: a 64-bit difference
//! hash ("dHash"), which compares neighbouring brightness in a 9×8 grid. Two
//! copies of one picture land a few bits apart, and different pictures about
//! half of the bits apart.
/// Hashes this many bits apart or fewer show the same picture.
pub const SIMILAR_BITS: u32 = 6;
/// Stored for a photo whose thumbnail was looked at and gave no usable hash
/// (it couldn't be decoded, or is one flat colour), so it isn't looked at
/// again. A real hash is never 0: a picture that would hash to 0 has no
/// brightness changes, which [`difference_hash`] turns down as flat.
pub const NO_HASH: u64 = 0;
/// Samples of the grid that differ by less than this in brightness, all of
/// them, make a flat picture — a black frame, a blank page — whose hash is
/// noise and would match every other flat picture.
const FLAT: u8 = 8;
/// The difference hash of an 8-bit greyscale image `width` px wide, rows
/// stored one after another in `luma`. `None` for an empty or flat image.
pub fn difference_hash(width: usize, height: usize, luma: &[u8]) -> Option<u64> {
if width == 0 || height == 0 || luma.len() < width * height {
return None;
}
// Box-average down to 9×8, so every source pixel counts once and the hash
// doesn't depend on which pixels a nearest-neighbour pick happens to hit.
let mut grid = [[0u8; 9]; 8];
for (gy, row) in grid.iter_mut().enumerate() {
let (y0, y1) = span(gy, 8, height);
for (gx, cell) in row.iter_mut().enumerate() {
let (x0, x1) = span(gx, 9, width);
let mut sum = 0u64;
for y in y0..y1 {
sum += luma[y * width + x0..y * width + x1]
.iter()
.map(|&v| u64::from(v))
.sum::<u64>();
}
*cell = (sum / ((y1 - y0) * (x1 - x0)) as u64) as u8;
}
}
let (min, max) = grid
.iter()
.flatten()
.fold((u8::MAX, 0), |(min, max), &v| (min.min(v), max.max(v)));
if max - min < FLAT {
return None;
}
let mut hash = 0u64;
for row in &grid {
for pair in row.windows(2) {
hash = hash << 1 | u64::from(pair[0] < pair[1]);
}
}
(hash != NO_HASH).then_some(hash)
}
/// The source pixels `from..to` that cell `i` of `cells` covers along an edge
/// of `len` px; never empty, even for an edge shorter than the grid.
fn span(i: usize, cells: usize, len: usize) -> (usize, usize) {
let from = (i * len / cells).min(len - 1);
let to = ((i + 1) * len / cells).max(from + 1);
(from, to)
}
/// Whether two hashes show the same picture.
pub fn alike(a: u64, b: u64) -> bool {
(a ^ b).count_ones() <= SIMILAR_BITS
}
/// Group `hashes` into sets of alike ones, by index, each set in index order
/// and the sets by their first index. A hash alike to none other is in no
/// set. Alike is taken as catching: a burst of shots each a little different
/// from the last is one set.
pub fn sets(hashes: &[u64]) -> Vec<Vec<usize>> {
// Two hashes at most SIMILAR_BITS apart agree on at least one of their
// eight bytes, as long as SIMILAR_BITS < 8. So only hashes sharing some
// byte are compared, not every pair of the library.
const _: () = assert!(SIMILAR_BITS < 8);
let mut parent: Vec<usize> = (0..hashes.len()).collect();
fn root(parent: &mut [usize], mut i: usize) -> usize {
while parent[i] != i {
parent[i] = parent[parent[i]];
i = parent[i];
}
i
}
for byte in 0..8 {
let mut buckets: Vec<Vec<usize>> = vec![Vec::new(); 256];
for (i, &hash) in hashes.iter().enumerate() {
buckets[(hash >> (byte * 8)) as u8 as usize].push(i);
}
for bucket in &buckets {
for (n, &a) in bucket.iter().enumerate() {
for &b in &bucket[n + 1..] {
if alike(hashes[a], hashes[b]) {
let (ra, rb) = (root(&mut parent, a), root(&mut parent, b));
if ra != rb {
parent[ra.max(rb)] = ra.min(rb);
}
}
}
}
}
}
let mut by_root: std::collections::BTreeMap<usize, Vec<usize>> = Default::default();
for i in 0..hashes.len() {
let r = root(&mut parent, i);
by_root.entry(r).or_default().push(i);
}
by_root.into_values().filter(|set| set.len() > 1).collect()
}
#[cfg(test)]
mod tests {
use super::*;
/// A 90×80 picture: a bright disc on a gradient.
fn picture(width: usize, height: usize) -> Vec<u8> {
let mut luma = Vec::with_capacity(width * height);
for y in 0..height {
for x in 0..width {
let (fx, fy) = (x as f64 / width as f64, y as f64 / height as f64);
let disc = ((fx - 0.3).powi(2) + (fy - 0.6).powi(2)).sqrt() < 0.2;
let v = if disc { 240.0 } else { 40.0 + 120.0 * fx * fy };
luma.push(v as u8);
}
}
luma
}
#[test]
fn a_resized_copy_hashes_alike() {
let big = difference_hash(900, 800, &picture(900, 800)).unwrap();
let small = difference_hash(91, 79, &picture(91, 79)).unwrap();
assert!(
alike(big, small),
"{} bits apart",
(big ^ small).count_ones()
);
}
#[test]
fn a_different_picture_hashes_apart() {
let one = difference_hash(90, 80, &picture(90, 80)).unwrap();
let mirrored: Vec<u8> = picture(90, 80)
.chunks(90)
.flat_map(|row| row.iter().rev().copied().collect::<Vec<_>>())
.collect();
let other = difference_hash(90, 80, &mirrored).unwrap();
assert!(!alike(one, other));
}
#[test]
fn a_flat_picture_has_no_hash() {
assert_eq!(difference_hash(64, 64, &[17; 64 * 64]), None);
assert_eq!(difference_hash(0, 0, &[]), None);
assert_eq!(difference_hash(4, 4, &[0; 3]), None);
}
#[test]
fn a_picture_smaller_than_the_grid_still_hashes() {
let luma: Vec<u8> = (0..4 * 3).map(|i| (i * 20) as u8).collect();
assert!(difference_hash(4, 3, &luma).is_some());
}
#[test]
fn alike_hashes_form_sets_and_catch_on() {
let a = 0x0123_4567_89ab_cdef_u64;
let hashes = [
a,
!a,
a ^ 0b111,
a ^ 0b111 ^ (0b1111 << 40),
0x5555_0000_ffff_aaaa,
];
assert_eq!(sets(&hashes), vec![vec![0, 2, 3]]);
}
#[test]
fn hashes_apart_in_most_bytes_are_still_found() {
// One bit apart in each of six bytes: six bits in all, and only the
// top two bytes agree.
let a = 1u64 << 63;
let b = (0..6).fold(a, |h, byte| h | 1 << (byte * 8));
assert_eq!(sets(&[a, b]), vec![vec![0, 1]]);
}
}
+40 -1
View File
@@ -1,4 +1,4 @@
//! Selective-sync ignore rules for mirror folders (features.md 1.1).
//! Selective-sync ignore rules for mirror folders.
//!
//! A mirror folder otherwise uploads every path beneath its root, which for a
//! project directory means `node_modules/`, `.git/`, and `target/` — quota
@@ -96,6 +96,26 @@ pub fn is_transient_name(name: &str) -> bool {
false
}
/// Whether `name` is scratch another program writes and throws away: the
/// [`is_transient_name`] set, plus the save-by-rename temporaries of GTK, Qt,
/// Chromium and Electron, generic `.tmp` files and Emacs lock files. None of
/// them is a file the user made, so a front-end can hide them from a history
/// of what happened to the user's files.
pub fn is_scratch_name(name: &str) -> bool {
if is_transient_name(name) {
return true;
}
let lower = name.to_ascii_lowercase();
lower.ends_with(".tmp")
|| lower.ends_with(".temp")
|| name.starts_with(".goutputstream-")
|| name.starts_with(".org.chromium.")
|| name.starts_with(".com.google.chrome.")
|| name.starts_with(".#")
|| (name.starts_with('#') && name.ends_with('#') && name.len() > 1)
|| (name.starts_with('.') && name.contains(".tmp."))
}
/// Filenames accepted as a folder's ignore file, in precedence order.
const IGNORE_FILE_NAMES: &[&str] = &[".pdfsignore", ".protonignore"];
@@ -341,6 +361,25 @@ mod tests {
}
}
#[test]
fn save_by_rename_temporaries_are_scratch() {
for name in [
".goutputstream-4XK2B2",
".org.chromium.Chromium.aBc123",
"~$report.docx",
".~lock.budget.ods#",
".#notes.org",
"#notes.org#",
"cache.tmp",
"page.html.crdownload",
] {
assert!(is_scratch_name(name), "{name}");
}
for name in ["report.docx", "notes.org", "#1 hits.txt", ".bashrc"] {
assert!(!is_scratch_name(name), "{name}");
}
}
#[test]
fn finished_names_are_not_transient() {
for name in [
+4
View File
@@ -15,6 +15,7 @@ zip.workspace = true
sha1.workspace = true
proton-drive-rs.workspace = true
futures.workspace = true
async-trait.workspace = true
tokio.workspace = true
tracing.workspace = true
fuser.workspace = true
@@ -24,3 +25,6 @@ base64.workspace = true
image.workspace = true
notify = "8"
parking_lot.workspace = true
[dev-dependencies]
tracing-subscriber.workspace = true
+40 -4
View File
@@ -15,7 +15,7 @@
use std::collections::{HashMap, HashSet};
use std::sync::atomic::Ordering;
use pdfs_core::control::{AlbumInfo, PhotoItem};
use pdfs_core::control::{AlbumInfo, PhotoItem, Topic};
use pdfs_core::db::StoredAlbum;
use pdfs_core::{CoreError, CoreResult};
@@ -137,10 +137,12 @@ impl Core {
}
let core = self.clone();
self.rt.spawn(async move {
if let Err(e) = core.refresh_albums().await {
warn!(error = %e, "background album refresh failed");
}
let refreshed = core.refresh_albums().await;
core.albums_refreshing.store(false, Ordering::SeqCst);
match refreshed {
Ok(_) => core.events.publish(&[Topic::Photos]),
Err(e) => warn!(error = %e, "background album refresh failed"),
}
});
}
@@ -175,6 +177,40 @@ impl Core {
Ok(page.into_iter().map(|p| self.photo_item(p)).collect())
}
/// Make sure every album of ours has its contents on disk, for a filter
/// that asks which photos no album holds. Contents are otherwise fetched
/// the first time an album is opened, so an album never opened would read
/// as empty and its photos as unfiled.
///
/// Shared-with-me albums are skipped: their photos live on the sharer's
/// volume and are never in our timeline. The freshness stamps are no guide
/// here — any album change clears all of them — so only an album whose
/// server count disagrees with an empty local copy is waited for.
pub(crate) fn ensure_album_contents(&self) -> CoreResult<()> {
if self.db.albums_count().map_err(CoreError::from)? == 0
&& !self.rt.block_on(self.refresh_albums())?
{
return Ok(());
}
let albums = self.db.albums_list().map_err(CoreError::from)?;
for album in albums.iter().filter(|a| !a.shared && a.photo_count > 0) {
let stored = self
.db
.album_photos_count(&album.uid)
.map_err(CoreError::from)?;
if stored > 0 {
continue;
}
let Some(uid) = parse_uid(&album.uid) else {
continue;
};
if let Err(error) = self.rt.block_on(self.refresh_album(&uid)) {
warn!(album = %album.uid, %error, "fetching an album's contents failed");
}
}
Ok(())
}
/// Re-enumerate one album's photos and replace its persisted contents.
pub(crate) async fn refresh_album(&self, album: &NodeUid) -> CoreResult<()> {
let photos = self.photos();
+110 -20
View File
@@ -9,7 +9,7 @@ use crate::shutdown::Shutdown;
///
/// Returns whether this mount actually held the node, so the caller can evict
/// its content blob once if *any* mount did.
fn forget_and_notify(st: &mut State, notify: &mut NotifyBatch, uid: &NodeUid) -> bool {
pub(crate) fn forget_and_notify(st: &mut State, notify: &mut NotifyBatch, uid: &NodeUid) -> bool {
// Capture the inode before `forget` clears the uid mapping.
let child = st.by_uid.get(uid).copied();
let Some((parent, name)) = st.forget(uid) else {
@@ -97,15 +97,13 @@ fn event_serializes_shared_publication(event: &DriveEvent, own_volume: &VolumeId
/// entry forces the next `lookup`/`readdir` to re-enumerate from the remote, so
/// most events only need to invalidate listings rather than re-fetch eagerly.
///
/// Applied to **every** mounted inode space, not just the primary one. This task
/// is per-daemon while node state is per-mount, and a `DriveEvent` names a uid,
/// not a mount — so a file trashed from another device has to be withdrawn from
/// whichever of our sessions is showing it, which for anything under a sync
/// folder is a [`Core::fork_state`] fork rather than `core.state`. Reaching for
/// the primary state alone left forks serving a deleted file indefinitely, the
/// read-side twin of `docs/BUGS.md` B74. Inode numbers are per-mount, so each
/// mount must be notified through its **own** channel; that pairing is what
/// [`Core::for_each_mount`] exists to preserve.
/// Applied to the tree every mount shares, and told to **every** mounted
/// session. A `DriveEvent` names a uid, not a mount, so a file trashed from
/// another device has to be withdrawn from whichever of our sessions is showing
/// it — which for anything under a sync folder is a [`Core::fork_state`] fork.
/// Telling the primary session alone left forks serving a deleted file, the
/// read-side twin of `docs/BUGS.md` B74. [`Core::for_each_mount`] sends what
/// the kernel needs to hear to every session.
fn apply_event(core: &Core, event: &DriveEvent, dirty: &mut DirtyParents) -> pdfs_core::Result<()> {
let foreign_delete = is_foreign_node_delete(event, &core.primary_root_uid.volume_id);
let serializes_shared_publication =
@@ -145,7 +143,7 @@ fn apply_event(core: &Core, event: &DriveEvent, dirty: &mut DirtyParents) -> pdf
// create, and re-fetching would replace the size and mtime of the
// write we just accepted with the stale revision's — making a file
// that was copied in seconds ago read as empty until its upload
// lands (offline.md Phase 3).
// lands.
//
// A write that has already drained is the same story one step later:
// the drain brought the tree level with the revision it sealed, so
@@ -233,6 +231,48 @@ fn acknowledge_applied_event(
Ok(id)
}
/// How many applied node-update ids [`RecentEvents`] remembers. Far more than a
/// replay can reach back: one comes from a batch whose last event was not its
/// newest, so it repeats at most what that batch held.
const RECENT_EVENTS: usize = 4096;
/// The node updates applied lately, by event id, so one that comes round again
/// is skipped rather than applied twice.
///
/// The feed can deliver the same event twice in a batch, and a batch whose
/// events come out of order leaves the cursor short of its newest event, so the
/// next poll replays what lay past it. An update applied twice is not harmless:
/// the first copy of our own change claims the echo [`Core::note_self_change`]
/// recorded, and the second, unclaimed, reads as someone else's edit and evicts
/// the bytes a partial write is about to gap-fill from (docs/BUGS.md B123).
///
/// Only node updates: a delete is idempotent, and a continuity loss names the
/// cursor it lost, which is an id already applied.
#[derive(Default)]
struct RecentEvents {
ids: HashSet<DriveEventId>,
order: VecDeque<DriveEventId>,
}
impl RecentEvents {
/// Record `event` as applied, reporting whether it was already.
fn repeat(&mut self, event: &DriveEvent) -> bool {
let DriveEvent::NodeUpdated { id, .. } = event else {
return false;
};
if !self.ids.insert(id.clone()) {
return true;
}
self.order.push_back(id.clone());
if self.order.len() > RECENT_EVENTS
&& let Some(old) = self.order.pop_front()
{
self.ids.remove(&old);
}
false
}
}
/// The folders one batch of events changed, collected so the listings are
/// dropped once at the end of the batch rather than once per event.
///
@@ -320,10 +360,10 @@ fn flush_dirty_parents(core: &Core, dirty: &DirtyParents) {
/// only on fatal error.
///
/// Takes the whole `Core` rather than the primary mount's pieces: an event has
/// to be applied to every mounted inode space, and the registry that enumerates
/// them lives on the `Core` (see [`apply_event`]).
/// to reach every mounted session, and the registry that enumerates them lives
/// on the `Core` (see [`apply_event`]).
pub(super) async fn run_event_sync(
client: ProtonDriveClient,
client: Arc<dyn DriveApi>,
scope: DriveEventScopeId,
core: Core,
) {
@@ -335,7 +375,7 @@ pub(super) async fn run_event_sync(
// server head; persist it so the next restart resumes instead of
// reseeding (which would skip everything that changed offline).
// Seeding needs the network, and this task also runs on mounts that
// started offline (offline.md Phase 1) — so retry rather than giving up,
// started offline — so retry rather than giving up,
// which used to disable live sync for the life of the daemon.
Ok(None) => {
// Nothing says what changed before this cursor, so a persisted SDK
@@ -377,10 +417,12 @@ pub(super) async fn run_event_sync(
};
info!(?cursor, "event sync started");
let mut recent = RecentEvents::default();
loop {
tokio::time::sleep(POLL_INTERVAL).await;
let events = match client.enumerate_events(&scope, cursor.as_ref()).await {
core.next_poll(POLL_INTERVAL).await;
let events = match link::bounded(client.enumerate_events(&scope, cursor.as_ref())).await {
Ok(events) => events,
Err(e) if core.lost_link(&e, "poll events") => continue,
Err(e) => {
warn!(error = %e, "event poll failed; retrying after interval");
continue;
@@ -396,6 +438,11 @@ pub(super) async fn run_event_sync(
// did apply.
let mut dirty = DirtyParents::default();
for event in &events {
if recent.repeat(event) {
debug!(id = %event.id().as_str(), "skipping an event already applied");
cursor = Some(event.id().clone());
continue;
}
// Converge the SDK's own caches (folder keys, entity cache) on the
// server before applying the event to our tree. Without this, a node
// re-keyed/moved by another client keeps a stale key in the SDK for
@@ -422,6 +469,10 @@ pub(super) async fn run_event_sync(
cursor = Some(applied);
}
flush_dirty_parents(&core, &dirty);
// Another device changed the tree: whatever a front-end lists may have
// moved, including what is shared and what is in the trash.
core.events
.publish(&[Topic::Files, Topic::Trash, Topic::Shares, Topic::Conflicts]);
}
}
@@ -467,9 +518,10 @@ pub(super) async fn run_photos_event_sync(core: Core) {
info!(?cursor, "photo event sync started");
loop {
tokio::time::sleep(POLL_INTERVAL).await;
let events = match photos.enumerate_events(&scope, cursor.as_ref()).await {
core.next_poll(POLL_INTERVAL).await;
let events = match link::bounded(photos.enumerate_events(&scope, cursor.as_ref())).await {
Ok(events) => events,
Err(error) if core.lost_link(&error, "poll photo events") => continue,
Err(error) => {
warn!(%error, "photo event poll failed; retrying after interval");
continue;
@@ -479,6 +531,7 @@ pub(super) async fn run_photos_event_sync(core: Core) {
continue;
};
apply_photo_events(&core, &events);
core.events.publish(&[Topic::Photos]);
// One cursor write per batch, after the batch is applied: a crash in
// between replays events whose effects are idempotent.
if let Err(error) = core.db.set_state_str(PHOTOS_EVENT_CURSOR, head.as_str()) {
@@ -607,7 +660,7 @@ pub(super) fn run_local_index(
scan_local_once(&db, &indexing, &transfers, &mountpoint);
}
// Interruptible, so teardown joins this thread instead of leaving it to
// keep rewriting the index for a mount that is gone (bugs.md B44).
// keep rewriting the index for a mount that is gone (`docs/BUGS.md` B44).
if !shutdown.sleep(LOCAL_INDEX_CHECK) {
return;
}
@@ -710,6 +763,43 @@ mod tests {
}
}
#[test]
fn a_node_update_seen_again_is_skipped_but_other_events_are_not() {
let update = |id: &str| DriveEvent::NodeUpdated {
id: DriveEventId::from(id),
node_uid: uid("file"),
parent_node_uid: None,
is_trashed: false,
is_shared: false,
};
let mut recent = RecentEvents::default();
assert!(!recent.repeat(&update("e1")));
assert!(!recent.repeat(&update("e2")));
assert!(recent.repeat(&update("e1")), "a duplicate or replay");
let lost = DriveEvent::ContinuityLost {
id: DriveEventId::from("e2"),
};
assert!(!recent.repeat(&lost), "names the cursor, which was applied");
}
#[test]
fn recent_events_forget_the_oldest_past_their_bound() {
let update = |n: usize| DriveEvent::NodeUpdated {
id: DriveEventId::from(format!("e{n}")),
node_uid: uid("file"),
parent_node_uid: None,
is_trashed: false,
is_shared: false,
};
let mut recent = RecentEvents::default();
for n in 0..=RECENT_EVENTS {
assert!(!recent.repeat(&update(n)));
}
assert_eq!(recent.ids.len(), RECENT_EVENTS);
assert!(!recent.repeat(&update(0)), "the oldest was forgotten");
assert!(recent.repeat(&update(RECENT_EVENTS)));
}
#[test]
fn foreign_delete_invalidates_an_earlier_refresh_generation() {
let own_volume = VolumeId::from("own");
+322 -56
View File
@@ -17,8 +17,8 @@ use std::time::{Duration, Instant};
use pdfs_core::config::AppDirs;
use pdfs_core::control::{
ActivityKind, PhotoMonth, RefreshScope, Request as CtlRequest, Response as CtlResponse,
TransferDirection, TrashFailure,
ActivityKind, Event, HEARTBEAT, PhotoMonth, RefreshScope, Request as CtlRequest,
Response as CtlResponse, Topic, TransferDirection, TrashFailure,
};
use pdfs_core::{CoreError, CoreResult};
use proton_drive_rs::proton_sdk::ids::NodeUid;
@@ -163,8 +163,8 @@ fn read_request_line(reader: &mut impl BufRead) -> std::io::Result<Option<String
/// Turn a CLI-supplied path into the mount that owns it and the path relative
/// to *that* mount's root.
///
/// [`rel_to_mount`] is the older, primary-only reading, and it is what every
/// mutating request still uses. It rejects anything outside the primary
/// [`rel_to_mount`] is the older, primary-only reading, and it is what most
/// requests still use. It rejects anything outside the primary
/// mountpoint, which leaves a secondary on-demand mount — a synced folder
/// switched out of mirror mode — with no way to be named at all: `pdfs ls` and
/// `pdfs refresh` both answered "is not under the mountpoint", so a folder
@@ -185,11 +185,14 @@ fn route_to_mount(core: &Core, mountpoint: &Path, path: &str) -> CoreResult<(Cor
rel_to_mount(mountpoint, path).map(|rel| (core.clone(), rel))
}
/// [`route_to_mount`] for a conflict copy. The listing names a copy inside a
/// synced folder by its absolute local path (`docs/BUGS.md` B99), so the
/// resolve has to run against the on-demand mount serving it. A mirror folder
/// has no mount to run against: its copy is an ordinary local file.
fn route_conflict(core: &Core, mountpoint: &Path, path: &str) -> CoreResult<(Core, PathBuf)> {
/// [`route_to_mount`] for a request that changes what it names: trash, rename,
/// create a folder, resolve a conflict. Each has to run against the on-demand
/// mount serving the path, and `pdfs rm ~/Downloads/x` answered "is not under
/// the mountpoint" while `pdfs move` took the same path. The conflict listing
/// names a copy in a synced folder by its absolute local path (`docs/BUGS.md`
/// B99), so resolving one needs this too. A mirror folder has no mount to run
/// against: a file there is an ordinary local file.
fn route_change(core: &Core, mountpoint: &Path, path: &str) -> CoreResult<(Core, PathBuf)> {
let routed = route_to_mount(core, mountpoint, path).map_err(|e| {
let p = Path::new(path);
let mirrored = core
@@ -200,7 +203,7 @@ fn route_conflict(core: &Core, mountpoint: &Path, path: &str) -> CoreResult<(Cor
.any(|folder| folder.mode == "mirror" && p.starts_with(&folder.local_path));
if mirrored {
CoreError::invalid(format!(
"{path} is in a mirrored folder; rename or delete the local file there"
"{path} is in a mirrored folder; change the local file there instead"
))
} else {
e
@@ -217,6 +220,27 @@ fn route_conflict(core: &Core, mountpoint: &Path, path: &str) -> CoreResult<(Cor
}
}
/// [`route_change`] for a request that trashes or renames the node it names.
/// A mount's root is My files or a whole synced folder. An on-demand folder's
/// root has a parent on Drive, so nothing further down would stop `pdfs rm
/// ~/Downloads` from trashing the folder out from under its own mount.
fn route_node(core: &Core, mountpoint: &Path, path: &str) -> CoreResult<(Core, PathBuf)> {
let routed = route_change(core, mountpoint, path)?;
if below_root(&routed.1) {
Ok(routed)
} else {
Err(CoreError::invalid(format!(
"{path} is the root of a synced location and cannot be changed"
)))
}
}
/// Whether a mount-relative path names something below the mount's root.
fn below_root(rel: &Path) -> bool {
rel.components()
.any(|component| matches!(component, Component::Normal(_)))
}
/// Turn a CLI-supplied path into a mountpoint-relative path. An absolute path
/// must live under `mountpoint`; a relative path is taken as already relative to
/// the mount root.
@@ -286,6 +310,159 @@ impl Drop for RefreshGuard {
}
}
/// The reply to [`CtlRequest::Status`], which subscriptions are also sent
/// whenever it changes.
pub(crate) fn status_response(core: &Core, username: &str, mountpoint: &Path) -> CtlResponse {
let pins = core.cache.list_pins();
let queued = core.db.pending_op_counts().unwrap_or_default();
// Read after the queue: the drain counts an upload as landing before it
// retires the op, so one of the two always has it (B119).
let landing = core.landing_uploads.load(Ordering::SeqCst);
let staging = core.cache.staging_usage();
CtlResponse::Status {
parked_uploads: queued.parked.max(0) as u64,
failing_ops: queued.failing.max(0) as u64,
failing_error: queued.last_error.clone(),
staged_bytes: staging.bytes,
staged_oldest_secs: staging.oldest_secs,
paused: core.sync_pause().is_some(),
paused_until: core.sync_pause().flatten(),
username: username.to_string(),
mountpoint: mountpoint.display().to_string(),
pinned: pins.len(),
used: core.cache.usage(),
budget: core.cache.budget(),
pins,
online: core.online.load(Ordering::Relaxed),
pending_uploads: queued.uploads.max(0) as u64 + landing,
pending_changes: queued.changes.max(0) as u64,
}
}
/// What a front-end shows that `request` changes when it succeeds. Requests
/// from the CLI publish the same way as the app's, so an open window follows
/// a change made in a terminal.
fn touched_topics(request: &CtlRequest) -> &'static [Topic] {
use CtlRequest as R;
match request {
R::Pin { .. } | R::Unpin { .. } => &[Topic::Files],
R::CreateAlbum { .. }
| R::RenameAlbum { .. }
| R::DeleteAlbum { .. }
| R::AddToAlbum { .. }
| R::RemoveFromAlbum { .. }
| R::SetPhotoFavorite { .. }
| R::UploadPhoto { .. } => &[Topic::Photos],
R::TrashNodes { .. } => &[Topic::Files, Topic::Photos, Topic::Trash, Topic::Conflicts],
R::Rename { .. } | R::Move { .. } | R::CreateFolder { .. } | R::UploadPaths { .. } => {
&[Topic::Files, Topic::Conflicts]
}
R::Delete { .. } => &[Topic::Files, Topic::Trash, Topic::Conflicts],
R::Restore { .. } | R::DeleteForever { .. } | R::EmptyTrash => {
&[Topic::Files, Topic::Photos, Topic::Trash, Topic::Conflicts]
}
R::Refresh { scope } => match scope {
RefreshScope::Dir { .. } => &[Topic::Files],
RefreshScope::Trash => &[Topic::Trash],
// A photos refresh answers once it has started; the timeline
// publishes again when it lands.
RefreshScope::Photos { .. } => &[],
},
R::RenameDevice { .. } | R::DeleteDevice { .. } => &[Topic::Devices],
R::AdoptDevice { .. } => &[Topic::Devices, Topic::Locations],
R::AddSyncFolder { .. }
| R::RemoveSyncFolder { .. }
| R::SetSyncFolderMode { .. }
| R::SetSyncFolderPaused { .. }
| R::RestoreSyncFolders { .. }
| R::RestoreDeviceFolders { .. } => &[Topic::Locations, Topic::Devices],
R::RetryPendingOp { .. } => &[Topic::Queue],
R::DiscardPendingOp { .. } => &[Topic::Queue, Topic::Files],
R::ResolveConflict { .. } => &[Topic::Files, Topic::Trash, Topic::Conflicts],
R::ShareNode { .. }
| R::ShareNodeByUid { .. }
| R::UpdateShareRole { .. }
| R::UpdateShareRoleByUid { .. }
| R::RemoveShareEntry { .. }
| R::RemoveShareEntryByUid { .. }
| R::CreatePublicLink { .. }
| R::CreatePublicLinkByUid { .. }
| R::RemovePublicLink { .. }
| R::RemovePublicLinkByUid { .. }
| R::StopSharing { .. }
| R::StopSharingByUid { .. } => &[Topic::Shares, Topic::Files],
R::LeaveShared { .. }
| R::AcceptInvitation { .. }
| R::RejectInvitation { .. }
| R::CreateBookmark { .. }
| R::DeleteBookmark { .. } => &[Topic::Shares],
R::RestoreRevision { .. }
| R::RestoreRevisionByUid { .. }
| R::SaveRevisionAs { .. }
| R::SaveRevisionAsByUid { .. } => &[Topic::Files],
_ => &[],
}
}
/// How long a subscription's writes may block. A client that stops reading
/// is dropped after this rather than holding its handler forever.
const SUBSCRIPTION_WRITE_TIMEOUT: Duration = Duration::from_secs(10);
/// Serve a [`CtlRequest::Subscribe`]: send the current status and transfers
/// and say every topic changed, so the client catches up, then send whatever
/// the [`EventHub`](super::events::EventHub) collects, with a heartbeat when
/// nothing does. Ends when the client goes away or the daemon stops.
fn serve_subscription(core: &Core, username: &str, mountpoint: &Path, mut stream: UnixStream) {
let Some(mailbox) = core.events.subscribe() else {
warn!("control: refusing a subscription at the limit");
if let Ok(mut out) = serde_json::to_vec(&CtlResponse::error(CoreError::remote(
"too many subscriptions",
))) {
out.push(b'\n');
let _ = stream.write_all(&out);
}
return;
};
struct Unsubscribe<'a>(&'a Core, std::sync::Arc<super::events::Mailbox>);
impl Drop for Unsubscribe<'_> {
fn drop(&mut self) {
self.0.events.unsubscribe(&self.1);
}
}
let _unsubscribe = Unsubscribe(core, mailbox.clone());
if let Err(e) = stream.set_write_timeout(Some(SUBSCRIPTION_WRITE_TIMEOUT)) {
warn!(error = %e, "control: could not set subscription timeout");
return;
}
let heartbeat = super::events::event_line(&Event::Heartbeat);
let mut lines: Vec<_> = [
super::events::event_line(&Event::Status(Box::new(status_response(
core, username, mountpoint,
)))),
super::events::transfers_line(core.transfers.snapshot(), core.jobs_snapshot()),
super::events::event_line(&Event::Changed {
topics: Topic::ALL.to_vec(),
}),
]
.into_iter()
.flatten()
.collect();
loop {
for line in &lines {
if stream.write_all(line.as_bytes()).is_err() {
return;
}
}
if core.shutdown.is_stopping() {
return;
}
lines = mailbox.take(HEARTBEAT);
if lines.is_empty() {
lines.extend(heartbeat.clone());
}
}
}
fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: UnixStream) {
if let Err(e) = stream.set_read_timeout(Some(CONTROL_READ_TIMEOUT)) {
warn!(error = %e, "control: could not set request timeout");
@@ -310,32 +487,21 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
return;
}
};
let request = serde_json::from_str::<CtlRequest>(line.trim());
// A subscription lasts as long as its client, so it is not a request in
// flight and stays out of the diagnostics' list of them.
if let Ok(CtlRequest::Subscribe) = request {
serve_subscription(core, username, mountpoint, stream);
return;
}
let touched = request.as_ref().map_or(&[][..], touched_topics);
// Registered before the request is even parsed, so a handler that wedges
// inside the daemon shows up by name in `pdfs diagnostics`.
let _inflight = super::diagnostics::InflightGuard::new(super::diagnostics::request_kind(&line));
let response = match serde_json::from_str::<CtlRequest>(line.trim()) {
Ok(CtlRequest::Status) => {
let pins = core.cache.list_pins();
let queued = core.db.pending_op_counts().unwrap_or_default();
let staging = core.cache.staging_usage();
CtlResponse::Status {
parked_uploads: queued.parked.max(0) as u64,
failing_ops: queued.failing.max(0) as u64,
failing_error: queued.last_error.clone(),
staged_bytes: staging.bytes,
staged_oldest_secs: staging.oldest_secs,
paused: core.sync_pause().is_some(),
paused_until: core.sync_pause().flatten(),
username: username.to_string(),
mountpoint: mountpoint.display().to_string(),
pinned: pins.len(),
used: core.cache.usage(),
budget: core.cache.budget(),
pins,
online: core.online.load(Ordering::Relaxed),
pending_uploads: queued.uploads.max(0) as u64,
pending_changes: queued.changes.max(0) as u64,
}
let response = match request {
Ok(CtlRequest::Status) => status_response(core, username, mountpoint),
Ok(CtlRequest::Subscribe) => {
CtlResponse::error(CoreError::invalid("a subscription is served on its own"))
}
Ok(CtlRequest::Pin { path }) => match rel_to_mount(mountpoint, &path) {
Ok(rel) => match core.pin(&rel) {
@@ -440,7 +606,15 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
kind,
range,
favorites,
}) => match core.photos_timeline(offset, clamp_limit(limit), kind, range, favorites) {
not_in_album,
}) => match core.photos_timeline(
offset,
clamp_limit(limit),
kind,
range,
favorites,
not_in_album,
) {
Ok(Some(items)) => CtlResponse::Photos {
available: true,
items,
@@ -453,6 +627,49 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PhotoDuplicates) => match core.photo_duplicates() {
Ok(sets) => CtlResponse::PhotoDuplicates { sets },
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PhotoSimilar) => match core.photo_similar() {
Ok((sets, hashing, uncompared)) => CtlResponse::PhotoSimilar {
sets,
hashing,
uncompared,
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PhotoPlaces) => match core.photo_places() {
Ok(items) => CtlResponse::Places {
items,
mapping: core.timeline_progress(),
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PlacePhotos { id, offset, limit }) => {
match core.place_photos(id, offset, clamp_limit(limit)) {
Ok(items) => CtlResponse::Photos {
available: true,
items,
counts: None,
},
Err(e) => CtlResponse::error(e),
}
}
Ok(CtlRequest::PlaceLocations { id }) => match core.place_locations(id) {
Ok(items) => CtlResponse::PhotoLocations { items },
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::PhotosOnThisDay { limit }) => {
match core.photos_on_this_day(clamp_limit(limit)) {
Ok(items) => CtlResponse::Photos {
available: true,
items,
counts: None,
},
Err(e) => CtlResponse::error(e),
}
}
Ok(CtlRequest::PhotoGroup { uid }) => match core.photo_group(&uid) {
Ok(items) => CtlResponse::Photos {
available: true,
@@ -546,8 +763,8 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
items: core.photo_thumbs(&parsed),
}
}
Ok(CtlRequest::TrashNodes { uids }) => match Core::parse_uids(&uids) {
Ok(parsed) => match core.trash_photos(&parsed) {
Ok(CtlRequest::TrashNodes { uids, files_only }) => match Core::parse_uids(&uids) {
Ok(parsed) => match core.trash_photos(&parsed, files_only) {
Ok((trashed, failed)) => CtlResponse::Trashed {
trashed,
failed: failed
@@ -770,8 +987,8 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::Rename { path, new_name }) => match rel_to_mount(mountpoint, &path) {
Ok(rel) => match core.rename(&rel, &new_name) {
Ok(CtlRequest::Rename { path, new_name }) => match route_node(core, mountpoint, &path) {
Ok((core, rel)) => match core.rename(&rel, &new_name) {
Ok(name) => {
core.log_activity(ActivityKind::Rename, &name, format!("was {path}"), true);
CtlResponse::Ok {
@@ -785,6 +1002,24 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
},
Err(e) => CtlResponse::error(e),
},
// Two absolute paths may name two different locations, which only
// `move_between` can resolve; a relative pair keeps the old meaning.
Ok(CtlRequest::Move { path, new_parent })
if Path::new(&path).is_absolute() && Path::new(&new_parent).is_absolute() =>
{
match core.move_between(Path::new(&path), Path::new(&new_parent)) {
Ok(name) => {
core.log_activity(ActivityKind::Move, &name, format!("to {new_parent}"), true);
CtlResponse::Ok {
message: format!("moved {name}"),
}
}
Err(e) => {
core.log_activity(ActivityKind::Move, &path, &e, false);
CtlResponse::error(e)
}
}
}
Ok(CtlRequest::Move { path, new_parent }) => {
match (
rel_to_mount(mountpoint, &path),
@@ -810,8 +1045,8 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
(Err(e), _) | (_, Err(e)) => CtlResponse::error(e),
}
}
Ok(CtlRequest::Delete { path }) => match rel_to_mount(mountpoint, &path) {
Ok(rel) => match core.delete(&rel) {
Ok(CtlRequest::Delete { path }) => match route_node(core, mountpoint, &path) {
Ok((core, rel)) => match core.delete(&rel) {
Ok(name) => {
core.log_activity(ActivityKind::Trash, &name, "", true);
CtlResponse::Ok {
@@ -825,21 +1060,23 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::CreateFolder { parent, name }) => match rel_to_mount(mountpoint, &parent) {
Ok(parent_rel) => match core.create_folder(&parent_rel, &name) {
Ok(name) => {
core.log_activity(ActivityKind::CreateFolder, &name, "", true);
CtlResponse::Ok {
message: format!("created folder {name}"),
Ok(CtlRequest::CreateFolder { parent, name }) => {
match route_change(core, mountpoint, &parent) {
Ok((core, parent_rel)) => match core.create_folder(&parent_rel, &name) {
Ok(name) => {
core.log_activity(ActivityKind::CreateFolder, &name, "", true);
CtlResponse::Ok {
message: format!("created folder {name}"),
}
}
}
Err(e) => {
core.log_activity(ActivityKind::CreateFolder, &name, &e, false);
CtlResponse::error(e)
}
},
Err(e) => CtlResponse::error(e),
},
Err(e) => {
core.log_activity(ActivityKind::CreateFolder, &name, &e, false);
CtlResponse::error(e)
}
},
Err(e) => CtlResponse::error(e),
}
}
Ok(CtlRequest::UploadPaths { parent, sources }) => {
match rel_to_mount(mountpoint, &parent) {
// Ack immediately and upload on a background thread: a directory tree
@@ -1156,7 +1393,7 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::ResolveConflict { path, keep }) => {
match route_conflict(core, mountpoint, &path) {
match route_change(core, mountpoint, &path) {
Ok((core, rel)) => match core.resolve_conflict(&rel, &keep) {
Ok(message) => CtlResponse::Ok { message },
Err(e) => CtlResponse::error(e),
@@ -1168,6 +1405,20 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Ok(items) => CtlResponse::PendingOps { items },
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::ExportPendingOp { id, dest }) => {
match core.export_pending_op(id, Path::new(&dest)) {
Ok(written) => CtlResponse::Ok {
message: format!("wrote {} to {dest}", human_bytes(written)),
},
Err(e) => CtlResponse::error(e),
}
}
Ok(CtlRequest::DiscardPendingOp { id }) => match core.discard_pending_op(id) {
Ok(path) => CtlResponse::Ok {
message: format!("discarded the queued change to {path}"),
},
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::RetryPendingOp { id }) => match core.retry_pending_ops(id) {
Ok(0) if id.is_some() => CtlResponse::Ok {
message: "nothing to retry: the operation is already due or waiting for a rename"
@@ -1533,6 +1784,9 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
// The request did not parse: the caller sent something malformed.
Err(e) => CtlResponse::error(CoreError::invalid(format!("bad request: {e}"))),
};
if !touched.is_empty() && !matches!(response, CtlResponse::Error { .. }) {
core.events.publish(touched);
}
let mut out = match serde_json::to_vec(&response) {
Ok(v) => v,
Err(e) => {
@@ -1562,7 +1816,7 @@ pub(crate) fn run_control_socket(
info!("control socket listening");
for conn in listener.incoming() {
// Checked on every accept, including the deliberate poke teardown sends
// to wake this thread out of a blocking accept (bugs.md B44).
// to wake this thread out of a blocking accept (`docs/BUGS.md` B44).
if core.shutdown.is_stopping() {
info!("control socket closing");
return;
@@ -1654,6 +1908,18 @@ mod request_limit_tests {
assert!(rel_to_mount(mountpoint, "Photos/../../escape").is_err());
assert!(rel_to_mount(mountpoint, "/mnt/elsewhere").is_err());
}
/// `pdfs rm` and `pdfs rename` reach on-demand mounts, whose root is a whole
/// synced folder with a parent on Drive. Only a path below a root may get
/// through to the trash or rename call.
#[test]
fn only_a_path_below_a_mount_root_names_a_node() {
assert!(below_root(Path::new("Photos")));
assert!(below_root(Path::new("./Photos/2026")));
assert!(!below_root(Path::new("")));
assert!(!below_root(Path::new(".")));
assert!(!below_root(Path::new("./.")));
}
}
#[cfg(test)]
+20 -32
View File
@@ -1,5 +1,5 @@
//! This machine as a Proton Drive Device, and the local folders it keeps in
//! step with the remote (devices.md).
//! step with the remote.
//!
//! A sync folder runs in one of two modes. `mirror` keeps a real local tree that
//! the engine in [`super::sync`] reconciles both ways; `ondemand` evicts that
@@ -27,7 +27,7 @@ use tracing::{info, warn};
use super::sync::{self, base_name};
use super::{
Core, SecondaryInsertRejection, SecondaryMount, State, SwitchBlocked, clear_stale_mount,
Core, SecondaryInsertRejection, SecondaryMount, SwitchBlocked, clear_stale_mount,
device_type_str, dir_is_empty, evict_dir_contents, fuse_connection_id, is_stale_mount,
now_secs, parse_uid, spawn_session, sync_folder_info, this_hostname,
};
@@ -184,8 +184,8 @@ impl Core {
}
/// Adopt an existing device as *this* machine's, pinning it in `config.json`
/// so hostname changes and reinstalls stop registering duplicates
/// (features.md 5.1). `None` clears the pin and returns to hostname matching.
/// so hostname changes and reinstalls stop registering duplicates.
/// `None` clears the pin and returns to hostname matching.
///
/// Refuses a uid that is not in the account: the whole point of the pin is
/// that it is authoritative, so accepting a bad one would defer the failure
@@ -260,7 +260,7 @@ impl Core {
})
}
// ---- device folder sync (devices.md, Phase 1) -------------------------
// ---- device folder sync -------------------------
/// Auto-register (or recover) this machine as a Proton Drive Device, caching
/// it so restarts reuse the same device. Recovery matches an existing remote
@@ -270,8 +270,7 @@ impl Core {
/// Resolution order: the *adopted* uid pinned in `config.json`, then the
/// cached DB row (validated remotely), then a hostname match, then create.
/// The pin exists because the hostname heuristic silently registers a second
/// device after a rename or reinstall, orphaning the first one's folders
/// (features.md 5.1).
/// device after a rename or reinstall, orphaning the first one's folders.
pub(crate) fn ensure_device(&self) -> CoreResult<StoredDevice> {
let name = this_hostname();
// Enumerate the remote devices once: used to validate the pin, validate
@@ -363,14 +362,14 @@ impl Core {
) -> CoreResult<Option<NodeUid>> {
let uids = self
.rt
.block_on(self.client.enumerate_folder_children_node_uids(root_uid))
.block_on(self.drive.enumerate_folder_children_node_uids(root_uid))
.map_err(|e| CoreError::from_api(&e, "list device root"))?;
if uids.is_empty() {
return Ok(None);
}
let nodes = self
.rt
.block_on(self.client.enumerate_nodes(&uids))
.block_on(self.drive.enumerate_nodes(&uids))
.map_err(|e| CoreError::from_api(&e, "resolve device root children"))?;
Ok(nodes
.into_iter()
@@ -446,10 +445,7 @@ impl Core {
}
None => self
.rt
.block_on(
self.client
.create_folder(&root_uid, &name, Some(now_secs())),
)
.block_on(self.drive.create_folder(&root_uid, &name, Some(now_secs())))
.map_err(|e| CoreError::from_api(&e, &format!("create device folder {name}")))?,
};
@@ -603,7 +599,7 @@ impl Core {
&& let Some(uid) = parse_uid(&folder.remote_uid)
&& let Err(e) = self
.rt
.block_on(self.client.trash_nodes(&[uid]))
.block_on(self.drive.trash_nodes(&[uid]))
.and_then(batch::into_unit)
{
warn!(id, error = %e, "delete remote device folder failed");
@@ -619,23 +615,16 @@ impl Core {
};
}
/// A sibling Core that shares this one's client/rt/cache/db (and transfer,
/// activity, mount registries) but gets a **fresh, empty `State`** — its own
/// inode space starting at [`ROOT_INO`]. Used to root a secondary FUSE session
/// at an `ondemand` sync folder without colliding with the main mount's inodes
/// (devices.md Phase 3).
/// A sibling Core for a secondary FUSE session rooted at an `ondemand` sync
/// folder. It shares everything with this one,
/// the tree included, except what belongs to a session: its root and its
/// kernel channel, both filled in when the session is built.
pub(crate) fn fork_state(&self) -> Core {
let mut fork = self.clone();
fork.primary = false;
let share_access = self.state().share_access.clone();
fork.state = Arc::new(Mutex::new(State::new(self.db.clone(), share_access, 2)));
// A fresh inode space needs a fresh notification channel: this fork's
// session is the only one that knows these inodes, so it must be the one
// notified about them. Filled in by `spawn_ondemand_mount`.
fork.root = Arc::new(OnceLock::new());
fork.notifier = Arc::new(OnceLock::new());
fork.session_live = Arc::new(std::sync::atomic::AtomicBool::new(false));
// Size upgrades are keyed by inode, which is per-fork too.
fork.size_upgrades = Arc::new(Mutex::new(HashMap::new()));
fork
}
@@ -858,7 +847,7 @@ impl Core {
})?;
let root = self
.rt
.block_on(self.client.enumerate_nodes(std::slice::from_ref(&root_uid)))
.block_on(self.drive.enumerate_nodes(std::slice::from_ref(&root_uid)))
.map_err(|e| SwitchBlocked::Failed(format!("fetch remote root: {e}")))?
.into_iter()
.next()
@@ -937,8 +926,8 @@ impl Core {
}
}
/// Spawn a secondary FUSE session rooted at `root` over `local` on a forked
/// inode space. Clears any stale kernel mount first (a crashed run can leave
/// Spawn a secondary FUSE session rooted at `root` over `local` from a
/// [`Core::fork_state`] sibling. Clears any stale kernel mount first (a crashed run can leave
/// one, which would fail the fresh mount with EBUSY).
pub(crate) fn spawn_ondemand_mount(
&self,
@@ -996,8 +985,7 @@ impl Core {
/// restart (their local dirs are empty on disk — the files live in the cloud).
/// Best-effort per folder: a missing local path or a failed remote fetch marks
/// the folder `error` and moves on rather than aborting the rest. Runs on its
/// own thread from `mount` so the network fetches never block startup
/// (devices.md Phase 4).
/// own thread from `mount` so the network fetches never block startup.
pub(crate) fn restore_ondemand_mounts(&self) {
let folders = match self.db.sync_folder_list() {
Ok(f) => f,
@@ -1017,7 +1005,7 @@ impl Core {
};
let root = match self
.rt
.block_on(self.client.enumerate_nodes(std::slice::from_ref(&root_uid)))
.block_on(self.drive.enumerate_nodes(std::slice::from_ref(&root_uid)))
{
Ok(v) => match v.into_iter().next() {
Some(n) => n,
File diff suppressed because it is too large. Load diff
+473
View File
@@ -0,0 +1,473 @@
//! The calls the sync core makes to Drive, behind a trait.
//!
//! `Core` used to hold a concrete [`ProtonDriveClient`], so no test could make
//! Drive answer late, answer stale, or not answer at all, and the interleavings
//! behind most recent bugs could only be found on a real account over a slow
//! link. [`DriveApi`] is the part of the
//! client the mount, the drain, the event feed and the mirror engine use; the
//! real client implements it by forwarding, and the simulation tests implement
//! it with an in-memory Drive that injects Drive's known faults.
//!
//! Sharing, Photos, devices, invitations and public links stay on the concrete
//! client (`Core::client`) until a test needs them faked.
//!
//! The method names match the client's own, so a call site reads the same
//! whichever it goes through. The client's generic methods (a reader or writer
//! of any type) cannot be part of an object-safe trait, so the trait takes them
//! as trait objects and `impl dyn DriveApi` puts the generic signatures back.
use std::io::{Read, Write};
use std::sync::Arc;
use async_trait::async_trait;
use futures::StreamExt as _;
use futures::stream::BoxStream;
use proton_drive_rs::proton_sdk::account::Quota;
use proton_drive_rs::proton_sdk::error::Result;
use proton_drive_rs::proton_sdk::ids::{DriveEventId, NodeUid};
use proton_drive_rs::{
DriveEvent, DriveEventScopeId, Node, NodeMoveItem, ProtonDriveClient, RevisionReader, Thumbnail,
};
/// One outcome per node of a batch request; see [`ProtonDriveClient::trash_nodes`].
pub(crate) type NodeOutcomes = Vec<(NodeUid, Result<()>)>;
/// The outcomes of a streaming batch request, one item per node as it lands.
pub(crate) type OutcomeStream<'a> = BoxStream<'a, Result<(NodeUid, Result<()>)>>;
/// Drive as the sync core sees it. See the module documentation.
#[async_trait]
pub(crate) trait DriveApi: Send + Sync {
async fn get_my_files_folder(&self) -> Result<Node>;
async fn get_node(&self, uid: &NodeUid) -> Result<Option<Node>>;
async fn enumerate_nodes(&self, uids: &[NodeUid]) -> Result<Vec<Node>>;
/// As [`DriveApi::enumerate_nodes`], but files carry no claimed size or
/// modification time; see [`ProtonDriveClient::enumerate_nodes_light`].
async fn enumerate_nodes_light(&self, uids: &[NodeUid]) -> Result<Vec<Node>>;
async fn enumerate_folder_children_node_uids(
&self,
folder_uid: &NodeUid,
) -> Result<Vec<NodeUid>>;
async fn enumerate_events(
&self,
scope: &DriveEventScopeId,
cursor: Option<&DriveEventId>,
) -> Result<Vec<DriveEvent>>;
async fn invalidate_caches_for_event(&self, event: &DriveEvent) -> Result<()>;
async fn create_folder(
&self,
parent_uid: &NodeUid,
name: &str,
last_modification_time: Option<i64>,
) -> Result<NodeUid>;
async fn upload_file(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
contents: &[u8],
) -> Result<NodeUid>;
#[allow(clippy::too_many_arguments)]
async fn upload_file_from_dyn(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
reader: &mut (dyn Read + Send),
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
aead: bool,
) -> Result<NodeUid>;
#[allow(clippy::too_many_arguments)]
async fn upload_file_replacing_draft_from_dyn(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
reader: &mut (dyn Read + Send),
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
aead: bool,
) -> Result<NodeUid>;
async fn upload_new_revision_from_dyn(
&self,
file_uid: &NodeUid,
reader: &mut (dyn Read + Send),
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
) -> Result<()>;
async fn open_revision(&self, uid: &NodeUid) -> Result<Arc<dyn RevisionRead>>;
async fn download_file_to_dyn(
&self,
uid: &NodeUid,
output: &mut (dyn Write + Send),
) -> Result<()>;
async fn download_revision_to_dyn(
&self,
file_uid: &NodeUid,
revision_id: &str,
writer: &mut (dyn Write + Send),
) -> Result<()>;
async fn rename_node(
&self,
uid: &NodeUid,
new_name: &str,
new_media_type: Option<&str>,
) -> Result<()>;
async fn move_node(&self, uid: &NodeUid, new_parent: &NodeUid) -> Result<()>;
fn move_nodes_streaming<'a>(
&'a self,
items: Vec<NodeMoveItem>,
new_parent: NodeUid,
) -> OutcomeStream<'a>;
async fn trash_nodes(&self, uids: &[NodeUid]) -> Result<NodeOutcomes>;
async fn restore_nodes(&self, uids: &[NodeUid]) -> Result<NodeOutcomes>;
fn restore_nodes_streaming<'a>(&'a self, uids: &[NodeUid]) -> OutcomeStream<'a>;
fn delete_nodes_streaming<'a>(&'a self, uids: &[NodeUid]) -> OutcomeStream<'a>;
async fn enumerate_trash_node_uids(&self) -> Result<Vec<NodeUid>>;
/// The account's storage, across every Proton product.
async fn quota(&self) -> Result<Quota>;
/// Called by the drain once it has let go of the blob of an upload to
/// `uid` that landed, or retired the create that made `uid`, before the
/// tree has the node, so a simulation can hold it there. Blocks the drain
/// worker it is called on.
#[cfg(test)]
fn landing(&self, _uid: &NodeUid) {}
}
/// The generic signatures of the client, over the object-safe ones above.
impl dyn DriveApi {
#[allow(clippy::too_many_arguments)]
pub(crate) async fn upload_file_from<R: Read + Send>(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
mut reader: R,
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
aead: bool,
) -> Result<NodeUid> {
self.upload_file_from_dyn(
parent_uid,
name,
media_type,
&mut reader,
intended_size,
thumbnails,
last_modification_time,
aead,
)
.await
}
#[allow(clippy::too_many_arguments)]
pub(crate) async fn upload_file_replacing_draft_from<R: Read + Send>(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
mut reader: R,
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
aead: bool,
) -> Result<NodeUid> {
self.upload_file_replacing_draft_from_dyn(
parent_uid,
name,
media_type,
&mut reader,
intended_size,
thumbnails,
last_modification_time,
aead,
)
.await
}
pub(crate) async fn upload_new_revision_from<R: Read + Send>(
&self,
file_uid: &NodeUid,
mut reader: R,
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
) -> Result<()> {
self.upload_new_revision_from_dyn(
file_uid,
&mut reader,
intended_size,
thumbnails,
last_modification_time,
)
.await
}
pub(crate) async fn download_file_to<W: Write + Send>(
&self,
uid: &NodeUid,
output: &mut W,
) -> Result<()> {
self.download_file_to_dyn(uid, output).await
}
pub(crate) async fn download_revision_to<W: Write + Send>(
&self,
file_uid: &NodeUid,
revision_id: &str,
writer: &mut W,
) -> Result<()> {
self.download_revision_to_dyn(file_uid, revision_id, writer)
.await
}
}
/// An open revision that block reads are served from; see [`RevisionReader`].
#[async_trait]
pub(crate) trait RevisionRead: Send + Sync {
/// Plaintext size of the revision.
fn size(&self) -> u64;
/// Plaintext size of each block, in block order.
fn block_sizes(&self) -> &[u64];
async fn read_at(&self, offset: u64, length: u64) -> Result<Vec<u8>>;
}
#[async_trait]
impl RevisionRead for RevisionReader {
fn size(&self) -> u64 {
RevisionReader::size(self)
}
fn block_sizes(&self) -> &[u64] {
RevisionReader::block_sizes(self)
}
async fn read_at(&self, offset: u64, length: u64) -> Result<Vec<u8>> {
RevisionReader::read_at(self, offset, length).await
}
}
#[async_trait]
impl DriveApi for ProtonDriveClient {
async fn get_my_files_folder(&self) -> Result<Node> {
ProtonDriveClient::get_my_files_folder(self).await
}
async fn get_node(&self, uid: &NodeUid) -> Result<Option<Node>> {
ProtonDriveClient::get_node(self, uid).await
}
async fn enumerate_nodes(&self, uids: &[NodeUid]) -> Result<Vec<Node>> {
ProtonDriveClient::enumerate_nodes(self, uids).await
}
async fn enumerate_nodes_light(&self, uids: &[NodeUid]) -> Result<Vec<Node>> {
ProtonDriveClient::enumerate_nodes_light(self, uids).await
}
async fn enumerate_folder_children_node_uids(
&self,
folder_uid: &NodeUid,
) -> Result<Vec<NodeUid>> {
ProtonDriveClient::enumerate_folder_children_node_uids(self, folder_uid).await
}
async fn enumerate_events(
&self,
scope: &DriveEventScopeId,
cursor: Option<&DriveEventId>,
) -> Result<Vec<DriveEvent>> {
ProtonDriveClient::enumerate_events(self, scope, cursor).await
}
async fn invalidate_caches_for_event(&self, event: &DriveEvent) -> Result<()> {
ProtonDriveClient::invalidate_caches_for_event(self, event).await
}
async fn create_folder(
&self,
parent_uid: &NodeUid,
name: &str,
last_modification_time: Option<i64>,
) -> Result<NodeUid> {
ProtonDriveClient::create_folder(self, parent_uid, name, last_modification_time).await
}
async fn upload_file(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
contents: &[u8],
) -> Result<NodeUid> {
ProtonDriveClient::upload_file(self, parent_uid, name, media_type, contents).await
}
async fn upload_file_from_dyn(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
reader: &mut (dyn Read + Send),
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
aead: bool,
) -> Result<NodeUid> {
ProtonDriveClient::upload_file_from(
self,
parent_uid,
name,
media_type,
reader,
intended_size,
thumbnails,
last_modification_time,
aead,
)
.await
}
async fn upload_file_replacing_draft_from_dyn(
&self,
parent_uid: &NodeUid,
name: &str,
media_type: &str,
reader: &mut (dyn Read + Send),
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
aead: bool,
) -> Result<NodeUid> {
ProtonDriveClient::upload_file_replacing_draft_from(
self,
parent_uid,
name,
media_type,
reader,
intended_size,
thumbnails,
last_modification_time,
aead,
)
.await
}
async fn upload_new_revision_from_dyn(
&self,
file_uid: &NodeUid,
reader: &mut (dyn Read + Send),
intended_size: i64,
thumbnails: Vec<Thumbnail>,
last_modification_time: Option<i64>,
) -> Result<()> {
ProtonDriveClient::upload_new_revision_from(
self,
file_uid,
reader,
intended_size,
thumbnails,
last_modification_time,
)
.await
}
async fn open_revision(&self, uid: &NodeUid) -> Result<Arc<dyn RevisionRead>> {
let reader = ProtonDriveClient::open_revision(self, uid).await?;
Ok(Arc::new(reader))
}
async fn download_file_to_dyn(
&self,
uid: &NodeUid,
output: &mut (dyn Write + Send),
) -> Result<()> {
ProtonDriveClient::download_file_to(self, uid, &mut &mut *output).await
}
async fn download_revision_to_dyn(
&self,
file_uid: &NodeUid,
revision_id: &str,
writer: &mut (dyn Write + Send),
) -> Result<()> {
ProtonDriveClient::download_revision_to(self, file_uid, revision_id, &mut &mut *writer)
.await
}
async fn rename_node(
&self,
uid: &NodeUid,
new_name: &str,
new_media_type: Option<&str>,
) -> Result<()> {
ProtonDriveClient::rename_node(self, uid, new_name, new_media_type).await
}
async fn move_node(&self, uid: &NodeUid, new_parent: &NodeUid) -> Result<()> {
ProtonDriveClient::move_node(self, uid, new_parent).await
}
fn move_nodes_streaming<'a>(
&'a self,
items: Vec<NodeMoveItem>,
new_parent: NodeUid,
) -> OutcomeStream<'a> {
ProtonDriveClient::move_nodes_streaming(self, items, new_parent).boxed()
}
async fn trash_nodes(&self, uids: &[NodeUid]) -> Result<NodeOutcomes> {
ProtonDriveClient::trash_nodes(self, uids).await
}
async fn restore_nodes(&self, uids: &[NodeUid]) -> Result<NodeOutcomes> {
ProtonDriveClient::restore_nodes(self, uids).await
}
fn restore_nodes_streaming<'a>(&'a self, uids: &[NodeUid]) -> OutcomeStream<'a> {
ProtonDriveClient::restore_nodes_streaming(self, uids).boxed()
}
fn delete_nodes_streaming<'a>(&'a self, uids: &[NodeUid]) -> OutcomeStream<'a> {
ProtonDriveClient::delete_nodes_streaming(self, uids).boxed()
}
async fn enumerate_trash_node_uids(&self) -> Result<Vec<NodeUid>> {
ProtonDriveClient::enumerate_trash_node_uids(self).await
}
async fn quota(&self) -> Result<Quota> {
ProtonDriveClient::quota(self).await
}
}
+333
View File
@@ -0,0 +1,333 @@
//! Push notifications for front-ends: the daemon's side of
//! [`Request::Subscribe`](pdfs_core::control::Request::Subscribe).
//!
//! Front-ends used to poll. The app asked for the status every two seconds, the
//! tray every three, and pages re-read themselves on a timer, whether or not
//! anything had changed. Now each subscription owns a [`Mailbox`] in the
//! [`EventHub`]. Code that changes something publishes a [`Topic`], and
//! [`run_event_sampler`] watches the few things that change without passing
//! through one place (the status, transfers, sync progress) and publishes them
//! when they differ.
use std::collections::{BTreeSet, HashMap};
use std::sync::Arc;
use std::time::Duration;
use parking_lot::{Condvar, Mutex, MutexGuard};
use pdfs_core::control::{Event, JobItem, Response as CtlResponse, Topic, TransferItem};
use tracing::warn;
use super::Core;
/// Most subscriptions served at once. Each holds a control handler for as long
/// as it lasts; the app and the tray need one apiece, and the cap keeps a
/// misbehaving client from spending the handlers that requests need.
const MAX_SUBSCRIBERS: usize = 8;
/// How long a subscription waits after a change before it sends, so a burst
/// (a folder of files trashed at once) goes out as one event.
const COALESCE: Duration = Duration::from_millis(150);
/// How often [`run_event_sampler`] looks at transfers. Status and sync
/// progress are looked at every other time.
const SAMPLE_INTERVAL: Duration = Duration::from_millis(500);
/// Every open subscription's mailbox. Shared by every clone of [`Core`],
/// on-demand forks included, so a change in any mount reaches every client.
#[derive(Default)]
pub(crate) struct EventHub {
mailboxes: Mutex<Vec<Arc<Mailbox>>>,
}
/// What one subscription has yet to send.
#[derive(Default)]
pub(crate) struct Mailbox {
inbox: Mutex<Inbox>,
ready: Condvar,
}
#[derive(Default)]
struct Inbox {
topics: BTreeSet<Topic>,
/// The newest [`Event::Status`] line. A newer one replaces it unsent: a
/// client wants the current status, not every status in between.
status: Option<Arc<str>>,
/// The newest [`Event::Transfers`] line, replaced the same way.
transfers: Option<Arc<str>>,
}
impl Inbox {
fn is_empty(&self) -> bool {
self.topics.is_empty() && self.status.is_none() && self.transfers.is_none()
}
}
/// `event` as one line of the wire format, newline included.
pub(crate) fn event_line(event: &Event) -> Option<Arc<str>> {
match serde_json::to_string(event) {
Ok(mut line) => {
line.push('\n');
Some(line.into())
}
Err(e) => {
warn!(error = %e, "events: serialize event failed");
None
}
}
}
impl EventHub {
/// Tell every subscriber that `topics` changed.
pub(crate) fn publish(&self, topics: &[Topic]) {
self.deliver(|inbox| inbox.topics.extend(topics.iter().copied()));
}
/// Hand every subscriber a fresh status.
fn publish_status(&self, line: Arc<str>) {
self.deliver(|inbox| inbox.status = Some(line.clone()));
}
/// Hand every subscriber a fresh transfers snapshot.
fn publish_transfers(&self, line: Arc<str>) {
self.deliver(|inbox| inbox.transfers = Some(line.clone()));
}
fn deliver(&self, put: impl Fn(&mut Inbox)) {
for mailbox in self.mailboxes.lock().iter() {
put(&mut mailbox.inbox.lock());
mailbox.ready.notify_one();
}
}
/// Open a mailbox, or `None` at [`MAX_SUBSCRIBERS`].
pub(crate) fn subscribe(&self) -> Option<Arc<Mailbox>> {
let mut mailboxes = self.mailboxes.lock();
if mailboxes.len() >= MAX_SUBSCRIBERS {
return None;
}
let mailbox = Arc::new(Mailbox::default());
mailboxes.push(mailbox.clone());
Some(mailbox)
}
pub(crate) fn unsubscribe(&self, mailbox: &Arc<Mailbox>) {
self.mailboxes
.lock()
.retain(|other| !Arc::ptr_eq(other, mailbox));
}
fn has_subscribers(&self) -> bool {
!self.mailboxes.lock().is_empty()
}
}
impl Mailbox {
/// Wait up to `timeout` for something to send, then take it as wire lines:
/// the status first, then transfers, then the changed topics. Empty when
/// nothing arrived in time.
pub(crate) fn take(&self, timeout: Duration) -> Vec<Arc<str>> {
let mut inbox = self.inbox.lock();
if inbox.is_empty() {
self.ready.wait_for(&mut inbox, timeout);
if inbox.is_empty() {
return Vec::new();
}
// Let the rest of a burst arrive before sending the first of it.
MutexGuard::unlocked(&mut inbox, || std::thread::sleep(COALESCE));
}
let mut lines = Vec::new();
lines.extend(inbox.status.take());
lines.extend(inbox.transfers.take());
if !inbox.topics.is_empty() {
let topics = std::mem::take(&mut inbox.topics).into_iter().collect();
lines.extend(event_line(&Event::Changed { topics }));
}
lines
}
}
/// The [`Event::Transfers`] line for a snapshot.
pub(crate) fn transfers_line(items: Vec<TransferItem>, jobs: Vec<JobItem>) -> Option<Arc<str>> {
event_line(&Event::Transfers { items, jobs })
}
/// Watch what changes without a single place to publish it from, and publish it
/// when it differs from the last look: the status (queue counts, cache use,
/// online, pause), the transfers snapshot, the synced folders with their
/// live progress, and which nodes have a sync issue. Looks only while someone
/// is subscribed.
pub(crate) fn run_event_sampler(core: Core, username: String, mountpoint: std::path::PathBuf) {
let mut transfers: Option<Arc<str>> = None;
let mut status: Option<String> = None;
let mut queue: Option<String> = None;
let mut locations: Option<String> = None;
let mut issues: Option<HashMap<i64, String>> = None;
let mut tick = 0u32;
while core.shutdown.sleep(SAMPLE_INTERVAL) {
if !core.events.has_subscribers() {
// A new subscription starts from a snapshot of its own, so there is
// nothing to compare against until then.
transfers = None;
status = None;
queue = None;
locations = None;
issues = None;
continue;
}
let line = transfers_line(core.transfers.snapshot(), core.jobs_snapshot());
if line.is_some() && line != transfers {
transfers = line.clone();
if let Some(line) = line {
core.events.publish_transfers(line);
}
}
tick = tick.wrapping_add(1);
if !tick.is_multiple_of(2) {
continue;
}
let now = super::control::status_response(&core, &username, &mountpoint);
let (key, queue_key) = status_keys(&now);
if status.as_ref().is_some_and(|last| *last != key)
&& let Some(line) = event_line(&Event::Status(Box::new(now)))
{
core.events.publish_status(line);
}
status = Some(key);
if queue.as_ref().is_some_and(|last| *last != queue_key) {
core.events.publish(&[Topic::Queue]);
}
queue = Some(queue_key);
// An issue shows on its node in the file browser: raised, cleared,
// landed or discarded, the listing is out of date.
if let Ok(now) = core.db.node_issues() {
if issues.as_ref().is_some_and(|last| *last != now) {
core.events.publish(&[Topic::Files]);
}
issues = Some(now);
}
if let Ok(now) = core.list_locations() {
let key = serde_json::to_string(&now).unwrap_or_default();
if locations.as_ref().is_some_and(|last| *last != key) {
core.events.publish(&[Topic::Locations]);
}
locations = Some(key);
}
}
}
/// What makes two statuses different enough to send, and what makes their
/// queues different. The age of the oldest staged write is left out of both:
/// it grows every second while anything is staged, which is not news.
fn status_keys(status: &CtlResponse) -> (String, String) {
let CtlResponse::Status {
parked_uploads,
failing_ops,
failing_error,
staged_bytes,
paused,
paused_until,
username,
mountpoint,
pinned,
used,
budget,
online,
pending_uploads,
pending_changes,
..
} = status
else {
return (String::new(), String::new());
};
let queue = format!(
"{pending_uploads}/{pending_changes}/{parked_uploads}/{failing_ops}/{failing_error:?}"
);
let key = format!(
"{queue}/{staged_bytes}/{paused}/{paused_until:?}/{username}/{mountpoint}/{pinned}/\
{used}/{budget}/{online}"
);
(key, queue)
}
#[cfg(test)]
mod tests {
use super::*;
fn changed(line: &str) -> Vec<Topic> {
match serde_json::from_str::<Event>(line.trim()) {
Ok(Event::Changed { topics }) => topics,
other => panic!("not a change event: {other:?}"),
}
}
#[test]
fn a_burst_of_changes_arrives_as_one_event() {
let hub = EventHub::default();
let mailbox = hub.subscribe().expect("room for a subscriber");
hub.publish(&[Topic::Trash, Topic::Files]);
hub.publish(&[Topic::Files]);
let lines = mailbox.take(Duration::from_secs(1));
assert_eq!(lines.len(), 1);
assert_eq!(changed(&lines[0]), vec![Topic::Files, Topic::Trash]);
assert!(mailbox.take(Duration::from_millis(10)).is_empty());
}
#[test]
fn a_newer_status_replaces_one_not_yet_sent() {
let hub = EventHub::default();
let mailbox = hub.subscribe().expect("room for a subscriber");
hub.publish_status("first\n".into());
hub.publish_status("second\n".into());
assert_eq!(
mailbox.take(Duration::from_secs(1)),
vec![Arc::<str>::from("second\n")]
);
}
#[test]
fn a_closed_subscription_gets_nothing_more() {
let hub = EventHub::default();
let mailbox = hub.subscribe().expect("room for a subscriber");
hub.unsubscribe(&mailbox);
hub.publish(&[Topic::Files]);
assert!(mailbox.take(Duration::from_millis(10)).is_empty());
assert!(!hub.has_subscribers());
}
#[test]
fn subscriptions_are_capped() {
let hub = EventHub::default();
let open: Vec<_> = (0..MAX_SUBSCRIBERS)
.map(|_| hub.subscribe().expect("room for a subscriber"))
.collect();
assert!(hub.subscribe().is_none());
hub.unsubscribe(&open[0]);
assert!(hub.subscribe().is_some());
}
#[test]
fn the_age_of_a_staged_write_is_not_a_change() {
let status = |staged_oldest_secs| CtlResponse::Status {
parked_uploads: 0,
failing_ops: 0,
failing_error: None,
staged_bytes: 10,
staged_oldest_secs,
paused: false,
paused_until: None,
username: "u".into(),
mountpoint: "/m".into(),
pinned: 0,
used: 0,
budget: 0,
pins: Vec::new(),
online: true,
pending_uploads: 1,
pending_changes: 0,
};
assert_eq!(status_keys(&status(5)), status_keys(&status(6)));
}
}
File diff suppressed because it is too large. Load diff
+2494 -845
View File
File diff suppressed because it is too large. Load diff
+374
View File
@@ -0,0 +1,374 @@
//! Noticing that the network went away mid-session, and that it came back.
//!
//! The mount used to learn it was offline only at startup: a daemon that had
//! connected once kept `online` true for the rest of its life, so a Wi-Fi drop
//! turned every create, mkdir, delete and rename into a two-minute wait for an
//! `EIO`, while the queue that exists for exactly this sat unused. Now any
//! remote call a handler waits on reports a failure to reach Proton through
//! [`Core::lost_link`], which flips the mount offline; the handler then takes its
//! queued path, and the probe thread here flips it back when Proton answers.
use std::future::Future;
use std::sync::atomic::Ordering;
use std::time::{Duration, Instant};
use parking_lot::{Condvar, Mutex};
use proton_drive_rs::proton_sdk::api::ResponseCode;
use proton_drive_rs::proton_sdk::error::{ProtonApiError, ProtonError};
use tracing::{debug, info, warn};
use crate::{Core, ONLINE_PROBE_MAX, ONLINE_PROBE_MIN};
/// How long a remote call a FUSE handler is waiting on may take before the link
/// is treated as down.
///
/// The SDK's own budget for one request is four attempts of up to 30 seconds
/// each, so a dead link held the caller for about two minutes. A metadata call
/// that has had no answer in 20 seconds is not going to be one the user wants to
/// keep waiting for; queueing the change is the better answer.
pub(crate) const INTERACTIVE_CALL_TIMEOUT: Duration = Duration::from_secs(20);
/// How long one probe may wait for an answer. A probe is a single small request,
/// and a probe stuck in the SDK's retry budget would delay noticing the link is
/// back by as long as it waits.
const PROBE_TIMEOUT: Duration = Duration::from_secs(10);
/// First delay between probes after a drop seen mid-session, and the ceiling it
/// doubles to while the outage is young. A blip should end in seconds, not in
/// the minutes [`ONLINE_PROBE_MAX`] allows.
const PROBE_AFTER_DROP: Duration = Duration::from_secs(2);
const PROBE_YOUNG_MAX: Duration = Duration::from_secs(30);
/// How long an outage counts as young. Past it the probe backs off to
/// [`ONLINE_PROBE_MAX`], because a laptop can sit offline for days and each
/// probe is a real API round trip.
const YOUNG_OUTAGE: Duration = Duration::from_secs(10 * 60);
/// How often the idle probe thread checks for teardown while the link is up.
const IDLE_CHECK: Duration = Duration::from_secs(1);
/// The shared half of the online flag: when the link went down, and a condvar
/// that fires on every change so waiters need not poll.
#[derive(Default)]
pub(crate) struct Link {
lost_at: Mutex<Option<Instant>>,
changed: Condvar,
/// Folders listed from the DB alone while offline, to relist once back.
pub(crate) stale_listings: Mutex<std::collections::HashSet<u64>>,
}
/// Whether a failed call failed because Proton could not be reached, rather
/// than because it answered and refused.
///
/// A transport error is the network. So is a request timeout: the SDK reports
/// Proton's own "try again" codes that way, and [`no_answer`] reports our
/// deadline that way. Anything else Proton actually said, and says again on a
/// retry, so it must reach the caller rather than be queued.
pub(crate) fn is_network_error(e: &ProtonError) -> bool {
match e {
ProtonError::Transport(_) => true,
ProtonError::Api(api) => {
matches!(
api.code,
ResponseCode::RequestTimeout | ResponseCode::Timeout
)
}
_ => false,
}
}
/// The error a call reports when it runs past `after` without an answer.
pub(crate) fn no_answer(after: Duration) -> ProtonError {
ProtonError::Api(ProtonApiError {
code: ResponseCode::RequestTimeout,
http_status: 408,
message: format!("no answer within {}s", after.as_secs()),
details: None,
})
}
/// The slowest upload rate an upload's deadline allows for, past the
/// metadata budget it starts with. Far below any link that works, so only a
/// transfer that has stopped moving runs into it.
const SLOWEST_UPLOAD: u64 = 32 * 1024;
/// How long an upload of `len` bytes may take before the link is treated as
/// down: the budget of a metadata call, plus the bytes at [`SLOWEST_UPLOAD`].
pub(crate) fn upload_deadline(len: u64) -> Duration {
INTERACTIVE_CALL_TIMEOUT + Duration::from_secs(len / SLOWEST_UPLOAD)
}
/// Run a remote call, giving up after [`INTERACTIVE_CALL_TIMEOUT`].
pub(crate) async fn bounded<T>(
call: impl Future<Output = Result<T, ProtonError>>,
) -> Result<T, ProtonError> {
bounded_by(INTERACTIVE_CALL_TIMEOUT, call).await
}
/// Run a remote call, giving up after `limit`.
pub(crate) async fn bounded_by<T>(
limit: Duration,
call: impl Future<Output = Result<T, ProtonError>>,
) -> Result<T, ProtonError> {
match tokio::time::timeout(limit, call).await {
Ok(result) => result,
Err(_) => Err(no_answer(limit)),
}
}
/// The next delay between probes, given the current one and how long the
/// outage has lasted.
fn next_probe_delay(delay: Duration, outage: Duration) -> Duration {
let cap = if outage < YOUNG_OUTAGE {
PROBE_YOUNG_MAX
} else {
ONLINE_PROBE_MAX
};
(delay * 2).min(cap)
}
impl Core {
/// Whether the mount currently believes Proton is reachable.
pub(crate) fn is_online(&self) -> bool {
self.online.load(Ordering::Relaxed)
}
/// Whether a namespace change through the mount goes to Drive inside the
/// syscall. Only with [`AppConfig::local_first`] off, and only online;
/// otherwise it is queued and the drain sends it.
///
/// [`AppConfig::local_first`]: pdfs_core::config::AppConfig::local_first
pub(crate) fn sends_inline(&self) -> bool {
!self.local_first && self.is_online()
}
/// Run a remote call a caller is waiting on, giving up after
/// [`INTERACTIVE_CALL_TIMEOUT`]. The timer is built inside the runtime: a
/// FUSE worker has no reactor of its own.
pub(crate) fn block_on_bounded<T>(
&self,
call: impl Future<Output = Result<T, ProtonError>>,
) -> Result<T, ProtonError> {
self.rt.block_on(bounded(call))
}
/// Run a remote call, giving up after `limit`: an upload, whose budget
/// grows with its size ([`upload_deadline`]).
pub(crate) fn block_on_within<T>(
&self,
limit: Duration,
call: impl Future<Output = Result<T, ProtonError>>,
) -> Result<T, ProtonError> {
self.rt.block_on(bounded_by(limit, call))
}
/// Report a failed remote call. Returns whether it failed for want of a
/// network, in which case the mount is now offline and the caller should
/// take its queued path instead of failing.
pub(crate) fn lost_link(&self, e: &ProtonError, what: &str) -> bool {
if !is_network_error(e) {
return false;
}
self.mark_offline(what, e);
true
}
/// Flip the mount offline and wake the probe. Only the first caller of an
/// outage logs it; the rest are the same outage.
pub(crate) fn mark_offline(&self, what: &str, error: &dyn std::fmt::Display) {
let mut lost_at = self.link.lost_at.lock();
if self.online.swap(false, Ordering::Relaxed) {
*lost_at = Some(Instant::now());
warn!(during = what, %error,
"lost the connection to Proton; queueing changes until it returns");
self.link.changed.notify_all();
}
}
/// Flip the mount back online, wake everything waiting for it, and let the
/// queue drain.
fn mark_online(&self) {
{
let mut lost_at = self.link.lost_at.lock();
self.online.store(true, Ordering::Relaxed);
match lost_at.take() {
Some(since) => {
info!(
offline_secs = since.elapsed().as_secs(),
"connection to Proton is back"
)
}
None => info!("back online"),
}
self.link.changed.notify_all();
}
self.relist_offline_listings();
// Anything written while offline is queued and waiting on exactly this,
// including ops that failed on the way down and are sitting out a
// backoff sized for a server that keeps refusing.
if let Err(e) = self.db.retry_failed_ops_now(crate::now_millis()) {
warn!(error = %e, "moving backed-off ops up after a reconnect failed");
}
self.wake_drain();
}
/// Wait up to `budget` for the link to come back. Returns whether it is up.
pub(crate) fn wait_online(&self, budget: Duration) -> bool {
let deadline = Instant::now() + budget;
let mut lost_at = self.link.lost_at.lock();
while !self.is_online() {
if self.shutdown.is_stopping()
|| self
.link
.changed
.wait_until(&mut lost_at, deadline)
.timed_out()
{
return self.is_online();
}
}
true
}
/// Wait out `interval` before a background poll of Proton, and while the
/// link is down, until it is back. Returns as soon as it comes back, so the
/// poll catches up on what changed during the outage at once instead of
/// one interval later. A poll into a dead link would only wait out the
/// SDK's retries and fail.
pub(crate) async fn next_poll(&self, interval: Duration) {
let due = Instant::now() + interval;
let mut was_offline = false;
loop {
if !self.is_online() {
was_offline = true;
} else if was_offline || Instant::now() >= due {
return;
}
tokio::time::sleep(IDLE_CHECK).await;
}
}
/// Watch the link for the life of the mount: sleep while it is up, and
/// probe it back while it is down.
///
/// A mount that started from the cache is down from the first moment; one
/// that started online goes down when [`Core::mark_offline`] says so.
pub(crate) fn run_online_probe(&self) {
// Only an offline start never had a chance to repair its share id.
let mut repair_share_id = !self.is_online();
loop {
{
let mut lost_at = self.link.lost_at.lock();
while self.is_online() {
if self.shutdown.is_stopping() {
return;
}
self.link.changed.wait_for(&mut lost_at, IDLE_CHECK);
}
}
if !self.probe_until_online() {
return;
}
if std::mem::take(&mut repair_share_id) {
self.repair_primary_share_id(&self.primary_root_uid);
}
}
}
/// Probe until Proton answers, then flip the mount online. Returns `false`
/// if the daemon is stopping.
fn probe_until_online(&self) -> bool {
let started = self.link.lost_at.lock().unwrap_or_else(Instant::now);
let mut delay = if started.elapsed() < YOUNG_OUTAGE {
PROBE_AFTER_DROP
} else {
ONLINE_PROBE_MIN
};
loop {
if !self.shutdown.sleep(delay) {
return false;
}
let probe = self.rt.block_on(async {
match tokio::time::timeout(PROBE_TIMEOUT, self.drive.get_my_files_folder()).await {
Ok(result) => result,
Err(_) => Err(no_answer(PROBE_TIMEOUT)),
}
});
match probe {
Ok(root) => {
{
let root_ino = self.root_ino();
let mut st = self.state();
if let Some(e) = st.entries.get_mut(&root_ino) {
e.node = root.clone();
}
}
if let Err(e) = self.db.upsert_node(&root) {
warn!(error = %e, "refresh root after reconnect failed");
}
self.mark_online();
return true;
}
Err(e) => {
debug!(error = %e, ?delay, "online probe failed; still offline");
delay = next_probe_delay(delay, started.elapsed());
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn api(code: ResponseCode) -> ProtonError {
ProtonError::Api(ProtonApiError {
code,
http_status: 422,
message: String::new(),
details: None,
})
}
#[test]
fn only_an_unanswered_call_counts_as_a_lost_link() {
assert!(is_network_error(&no_answer(INTERACTIVE_CALL_TIMEOUT)));
assert!(is_network_error(&api(ResponseCode::Timeout)));
assert!(!is_network_error(&api(ResponseCode::AlreadyExists)));
assert!(!is_network_error(&api(ResponseCode::InvalidRequirements)));
assert!(!is_network_error(&ProtonError::invalid_operation("bug")));
}
#[test]
fn an_upload_gets_longer_the_bigger_it_is() {
assert_eq!(upload_deadline(0), INTERACTIVE_CALL_TIMEOUT);
assert_eq!(
upload_deadline(1 << 30),
INTERACTIVE_CALL_TIMEOUT + Duration::from_secs(32_768)
);
}
#[test]
fn a_call_past_its_deadline_reads_as_a_lost_link() {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_time()
.build()
.unwrap();
let limit = Duration::from_millis(20);
let result: Result<(), _> = rt.block_on(bounded_by(limit, std::future::pending()));
assert!(result.is_err_and(|e| is_network_error(&e)));
}
#[test]
fn a_young_outage_is_probed_often_and_an_old_one_rarely() {
let young = Duration::from_secs(60);
let old = YOUNG_OUTAGE + Duration::from_secs(1);
assert_eq!(
next_probe_delay(PROBE_AFTER_DROP, young),
2 * PROBE_AFTER_DROP
);
assert_eq!(next_probe_delay(PROBE_YOUNG_MAX, young), PROBE_YOUNG_MAX);
assert_eq!(next_probe_delay(PROBE_YOUNG_MAX, old), 2 * PROBE_YOUNG_MAX);
assert_eq!(next_probe_delay(ONLINE_PROBE_MAX, old), ONLINE_PROBE_MAX);
}
}
+190 -56
View File
@@ -193,7 +193,7 @@ pub(super) fn clear_stale_mount(mountpoint: &Path) {
}
/// `sync_state` key holding the uid of the My Files root, so a later run can
/// recover it from `nodes` without the API (offline.md Phase 1).
/// recover it from `nodes` without the API.
const ROOT_UID_KEY: &str = "root_uid";
/// The My Files root, and whether we got it from the API (`true`) or from the
@@ -207,11 +207,11 @@ const ROOT_UID_KEY: &str = "root_uid";
/// means an empty tree, and mounting that would show the user an empty Drive
/// rather than an honest error.
fn fetch_or_recall_root(
client: &ProtonDriveClient,
drive: &dyn DriveApi,
rt: &tokio::runtime::Handle,
db: &Db,
) -> std::io::Result<(Node, bool)> {
let err = match rt.block_on(client.get_my_files_folder()) {
let err = match rt.block_on(drive.get_my_files_folder()) {
Ok(root) => {
if let Err(e) = db.set_state_str(ROOT_UID_KEY, &root.uid.to_string()) {
warn!(error = %e, "persist root uid failed");
@@ -244,6 +244,9 @@ pub struct MountOptions {
/// Upload and download caps in bytes per second (`0` = no cap).
pub upload_limit: u64,
pub download_limit: u64,
/// Namespace changes through the mount are queued, not sent in the
/// syscall. See [`pdfs_core::config::AppConfig::local_first`].
pub local_first: bool,
}
/// Spawn one FUSE session rooted at an arbitrary remote node.
@@ -276,13 +279,14 @@ pub(super) fn spawn_session(
info!(mountpoint = %mountpoint.display(), "mounting Proton Drive location");
let fs = ProtonFs::new(core.clone(), root);
if core.primary {
// Only My Files rehydrates the daemon-wide node cache. A secondary
// inode space is rooted at one device subtree and must not import every
// unrelated persisted node.
// Only My Files rehydrates the daemon-wide node cache, once. A
// secondary mount shares the tree it fills.
core.hydrate();
}
let session = Session::new(fs, mountpoint, &config)?.spawn()?;
let _ = core.notifier.set(session.notifier());
let _ = core
.notifier
.set(KernelChannel::new(session.notifier(), core.root_ino()));
core.session_live.store(true, Ordering::Release);
Ok(session)
}
@@ -339,17 +343,57 @@ pub fn mount(
control_socket: &Path,
db: Arc<Db>,
options: MountOptions,
) -> std::io::Result<MountOutcome> {
let drive = Arc::new(client.clone());
mount_with(
client,
drive,
rt,
mountpoint,
cache,
control_socket,
db,
options,
Host::Daemon,
)
}
/// What a mount runs in.
pub(crate) enum Host {
/// The daemon: stopped by SIGTERM or SIGINT, and indexing the user's home
/// for the launcher prompt.
Daemon,
/// One of several simulated daemons in a test process (`sim`): stopped by
/// a message on this channel, and with no home to index.
#[cfg(test)]
Simulation(std::sync::mpsc::Receiver<()>),
}
/// [`mount`], reaching Drive through `drive` and hosted by `host`; `client`
/// serves only what [`DriveApi`] does not cover.
#[allow(clippy::too_many_arguments)]
pub(crate) fn mount_with(
client: ProtonDriveClient,
drive: Arc<dyn DriveApi>,
rt: tokio::runtime::Handle,
mountpoint: &Path,
cache: ContentCache,
control_socket: &Path,
db: Arc<Db>,
options: MountOptions,
host: Host,
) -> std::io::Result<MountOutcome> {
let MountOptions {
username,
sweep_mode,
upload_limit,
download_limit,
local_first,
} = options;
// Start the uptime clock here rather than at the first `pdfs diagnostics`,
// so the age it reports is the daemon's own.
super::diagnostics::start_clock();
let (root, online) = fetch_or_recall_root(&client, &rt, &db)?;
let (root, online) = fetch_or_recall_root(drive.as_ref(), &rt, &db)?;
let scope = root.tree_event_scope_id();
db.mount_upsert_my_files(&mountpoint.to_string_lossy(), &root.uid.to_string(), None)
.map_err(|error| std::io::Error::other(format!("project My Files mount: {error}")))?;
@@ -357,19 +401,21 @@ pub fn mount(
.all_share_access()
.map_err(|error| std::io::Error::other(format!("load shared access: {error}")))?;
// The folder-sync engine (devices.md Phase 2) runs on its own thread and is
// The folder-sync engine runs on its own thread and is
// nudged over this channel; the sender lives in Core so control-socket
// handlers can trigger reconciles.
let (sync_tx, sync_rx) = std::sync::mpsc::channel::<sync::SyncMsg>();
let paused_until = pause::load_paused_until(&db);
let core = Core {
client: client.clone(),
drive,
client,
rt: rt.clone(),
maintenance: Arc::new(Mutex::new(Default::default())),
primary_root_uid: root.uid.clone(),
primary: true,
state: Arc::new(Mutex::new(State::new(db.clone(), share_access, 2))),
state: Arc::new(Mutex::new(State::new(db.clone(), share_access))),
root: Arc::new(OnceLock::new()),
cache: Arc::new(cache),
readers: Arc::new(Mutex::new(HashMap::new())),
block_ring: Arc::new(Mutex::new(BlockRing::default())),
@@ -383,20 +429,28 @@ pub fn mount(
shared_generation: Arc::new(AtomicU64::new(0)),
shared_refresh_deadlines: Arc::new(Mutex::new(SharedRefreshDeadlines::default())),
online: Arc::new(AtomicBool::new(online)),
link: Arc::new(Default::default()),
sync_paused_until: Arc::new(AtomicI64::new(paused_until)),
pending: Arc::new(Mutex::new(HashMap::new())),
hidden: Arc::new(Mutex::new(HashSet::new())),
creating: Arc::new(Mutex::new(HashMap::new())),
sealed_unread: Arc::new(Mutex::new(HashMap::new())),
landed: Arc::new(Mutex::new(HashMap::new())),
unadopted: Arc::new(Mutex::new(HashMap::new())),
drain_wake: Arc::new((Mutex::new(false), Condvar::new())),
shutdown: Arc::new(crate::shutdown::Shutdown::default()),
upload_times: Arc::new(Mutex::new(HashMap::new())),
upload_cancel: Arc::new(Mutex::new(HashMap::new())),
landing_uploads: Arc::new(AtomicU64::new(0)),
timeline_refreshing: Arc::new(AtomicBool::new(false)),
timeline_progress: Arc::new(Mutex::new(None)),
similar_hashing: Arc::new(AtomicBool::new(false)),
albums_refreshing: Arc::new(AtomicBool::new(false)),
trash_refreshing: Arc::new(AtomicBool::new(false)),
trash_progress: Arc::new(tokio::sync::Notify::new()),
conflict_notified: Arc::new(Mutex::new(HashSet::new())),
sweep_mode,
own_sealed_revs: Arc::new(Mutex::new(HashMap::new())),
local_first,
self_changes: Arc::new(Mutex::new(HashMap::new())),
thumb_gen: Arc::new(Mutex::new(HashSet::new())),
thumb_gen_budget: Arc::new(tokio::sync::Semaphore::new(
@@ -409,11 +463,13 @@ pub fn mount(
thumbnail_build_cancel: Arc::new(tokio::sync::Notify::new()),
thumbnail_misses: Arc::new(Mutex::new(Default::default())),
quota: Arc::new(Mutex::new(None)),
quota_refreshing: Arc::new(AtomicBool::new(false)),
size_upgrades: Arc::new(Mutex::new(HashMap::new())),
size_waiters: Arc::new(Default::default()),
notifier: Arc::new(OnceLock::new()),
session_live: Arc::new(AtomicBool::new(false)),
transfers: TransferRegistry::new(),
events: Arc::new(Default::default()),
indexing: Arc::new(AtomicBool::new(false)),
sync_progress: Arc::new(Mutex::new(HashMap::new())),
sync_tx,
@@ -422,7 +478,7 @@ pub fn mount(
states: Arc::new(StateRegistry::default()),
};
// Before anything can queue work against it: the drain thread below reaches
// every mount's inode space through this registry, not through `core.state`.
// every mount's session through this registry.
core.register_state(mountpoint);
core.transfers.set_limits(upload_limit, download_limit);
@@ -445,7 +501,7 @@ pub fn mount(
// Every long-lived thread started from here on is kept in `workers` and
// joined at the end of this function. Leaving them running turned an
// in-process remount into a second generation of drain workers, sync engines
// and sweeps all operating on a mount that no longer exists (bugs.md B44).
// and sweeps all operating on a mount that no longer exists (`docs/BUGS.md` B44).
let mut workers: Vec<std::thread::JoinHandle<()>> = Vec::new();
for worker in 0..DRAIN_WORKERS {
let core = core.clone();
@@ -481,9 +537,10 @@ pub fn mount(
);
}
// Mounted from the cache: watch for the network coming back so the mount can
// stop being read-only-ish without the user restarting the daemon.
if !online {
// Watch the link for the life of the mount: a mount from the cache waits for
// the network to come back, and an online one can lose it at any moment and
// must notice it coming back just the same.
{
let core = core.clone();
workers.push(
std::thread::Builder::new()
@@ -494,7 +551,7 @@ pub fn mount(
// Keep the launcher prompt's "This computer" index fresh. Its own thread:
// the walk is I/O-heavy and must never sit in front of a FUSE callback.
{
if matches!(host, Host::Daemon) {
let db = core.db.clone();
let indexing = core.indexing.clone();
let transfers = core.transfers.clone();
@@ -566,6 +623,22 @@ pub fn mount(
}
}
}
// Publishes what changes without one place to publish it from (status,
// transfers, sync progress) to the front-ends subscribed on that socket.
{
let sampler_core = core.clone();
let username = username.clone();
let mountpoint = mountpoint.to_path_buf();
match std::thread::Builder::new()
.name("pdfs-events".into())
.spawn(move || events::run_event_sampler(sampler_core, username, mountpoint))
{
Ok(handle) => workers.push(handle),
// Not fatal: front-ends still get every published change, only
// not the sampled ones.
Err(error) => warn!(error = %error, "could not start the event sampler"),
}
}
// The supervisor watches the pool and answers the systemd watchdog. Started
// after the listener, because its liveness probe is a round trip over that
// socket.
@@ -602,11 +675,13 @@ pub fn mount(
}
}
};
rt.spawn(run_event_sync(client, scope, core.clone()));
rt.spawn(run_event_sync(core.drive.clone(), scope, core.clone()));
// The photos volume has its own event stream; without it the gallery only
// learns about a phone-side deletion when the timeline goes stale.
rt.spawn(run_photos_event_sync(core.clone()));
if online {
// Not in a simulation: `client` there reaches no API, and the thread, still
// waiting on it after the stop, held the database a test opens next.
if online && matches!(host, Host::Daemon) {
let repair_core = core.clone();
let root_uid = core.primary_root_uid.clone();
if let Err(error) = std::thread::Builder::new()
@@ -621,30 +696,11 @@ pub fn mount(
// are delivered onto the async runtime; bridge them onto a sync channel so
// the loop below can react without blocking a worker thread. A bounded
// channel of 1 is enough — we only need to know that *a* stop arrived.
let (sig_tx, sig_rx) = std::sync::mpsc::sync_channel::<()>(1);
rt.spawn(async move {
let mut sigterm =
match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate()) {
Ok(s) => s,
Err(e) => {
warn!(error = %e, "install SIGTERM handler failed");
return;
}
};
let mut sigint =
match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::interrupt()) {
Ok(s) => s,
Err(e) => {
warn!(error = %e, "install SIGINT handler failed");
return;
}
};
tokio::select! {
_ = sigterm.recv() => info!("received SIGTERM"),
_ = sigint.recv() => info!("received SIGINT"),
}
let _ = sig_tx.try_send(());
});
let sig_rx = match host {
Host::Daemon => signal_stop(&rt),
#[cfg(test)]
Host::Simulation(rx) => rx,
};
// Wait for whichever happens first: a stop signal (→ we unmount ourselves
// via the lazy MNT_DETACH path, clean even mid-download), or the kernel
@@ -682,7 +738,7 @@ pub fn mount(
};
// Unmount every on-demand sync folder too, or the kernel mounts linger as
// stale and the next start fails with EBUSY (devices.md Phase 3).
// stale and the next start fails with EBUSY.
if let Some(worker) = restore_worker
&& worker.join().is_err()
{
@@ -694,6 +750,7 @@ pub fn mount(
warn!(id, error = %e, "unmount on-demand folder failed");
}
}
core.queue_unreleased_writes();
// Last, because the workers may legitimately be finishing an upload, and
// nothing they can still do is harmful once the mounts are down.
@@ -701,9 +758,39 @@ pub fn mount(
Ok(outcome)
}
/// How long teardown waits for the FUSE worker pool before proceeding without
/// it. Long enough for an in-flight block to finish, short enough that
/// `systemctl --user stop` still returns inside `TimeoutStopSec`.
/// A channel that receives once SIGTERM or SIGINT arrives.
fn signal_stop(rt: &tokio::runtime::Handle) -> std::sync::mpsc::Receiver<()> {
let (sig_tx, sig_rx) = std::sync::mpsc::sync_channel::<()>(1);
rt.spawn(async move {
let mut sigterm =
match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate()) {
Ok(s) => s,
Err(e) => {
warn!(error = %e, "install SIGTERM handler failed");
return;
}
};
let mut sigint =
match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::interrupt()) {
Ok(s) => s,
Err(e) => {
warn!(error = %e, "install SIGINT handler failed");
return;
}
};
tokio::select! {
_ = sigterm.recv() => info!("received SIGTERM"),
_ = sigint.recv() => info!("received SIGINT"),
}
let _ = sig_tx.try_send(());
});
sig_rx
}
/// How long teardown waits for the background workers and then the FUSE worker
/// pool before proceeding without them. Long enough for an in-flight block to
/// finish, short enough that `systemctl --user stop` still returns inside
/// `TimeoutStopSec`.
const WORKER_JOIN_DEADLINE: Duration = Duration::from_secs(10);
/// Signal every background loop, wake the ones that are blocked, and join them.
@@ -711,14 +798,15 @@ const WORKER_JOIN_DEADLINE: Duration = Duration::from_secs(10);
/// The mount used to return with its drain workers, sync engine, sweep, online
/// probe, indexer and control listener all still running on `Core` clones. The
/// process usually exited straight afterwards and hid it; an in-process remount
/// did not, and started a second full set on top of the first (bugs.md B44).
/// did not, and started a second full set on top of the first (`docs/BUGS.md` B44).
///
/// Two of the waits need waking rather than merely flagging: the drain workers
/// block on their own condvar (`wake_drain`), and the control listener blocks
/// inside `accept`, which only returns when a connection arrives — so it gets
/// one, from us, after the flag is already set. A join that takes a while is
/// expected and correct: a worker part-way through an upload finishes it rather
/// than abandoning the user's bytes mid-flight.
/// than abandoning the user's bytes mid-flight. Up to [`WORKER_JOIN_DEADLINE`]:
/// past it, systemd kills the daemon anyway, and a queued op is sent again.
fn stop_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::JoinHandle<()>>) {
// Say so before the joins: a teardown that takes a while must not be read as
// a missed watchdog ping and turned into a kill.
@@ -732,25 +820,71 @@ fn stop_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::Jo
// unlink cannot put it back to sleep.
let _ = std::os::unix::net::UnixStream::connect(control_socket);
let _ = std::fs::remove_file(control_socket);
for worker in workers {
if worker.join().is_err() {
warn!("a background worker panicked before shutdown");
}
let deadline = Instant::now() + WORKER_JOIN_DEADLINE;
let running = join_until(workers, deadline);
if !running.is_empty() {
warn!(
running = running.join(","),
"background workers did not finish in time; shutting down around them"
);
}
// The FUSE worker pool last, and *before* the caller drops the tokio
// runtime: a worker still inside a job when the runtime goes away panics on
// its next timer with "A Tokio 1.x context was found, but it is being
// shutdown" (seen 2026-09-18 09:58:35). Bounded, so a worker wedged on the
// network cannot hold the stop the user asked for.
core.workers.stop_and_join(WORKER_JOIN_DEADLINE);
core.workers
.stop_and_join(deadline.saturating_duration_since(Instant::now()));
debug!("background workers stopped");
}
/// Join `workers` until `deadline`, and name the ones still running then.
///
/// A stop used to wait for each one however long it took. The sync engine's
/// mirror pass waits on Drive with no deadline, and a stop that waited for it
/// was killed by systemd (`docs/BUGS.md` B165).
fn join_until(workers: Vec<std::thread::JoinHandle<()>>, deadline: Instant) -> Vec<String> {
let mut running = Vec::new();
for worker in workers {
while !worker.is_finished() && Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(10));
}
if !worker.is_finished() {
running.push(worker.thread().name().unwrap_or("unnamed").to_string());
} else if worker.join().is_err() {
warn!("a background worker panicked before shutdown");
}
}
running
}
#[cfg(test)]
mod tests {
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::{Duration, Instant};
use super::{SecondaryInsertRejection, SecondaryMountRegistry, teardown_session};
use super::{SecondaryInsertRejection, SecondaryMountRegistry, join_until, teardown_session};
#[test]
fn a_stop_names_the_workers_still_running_at_its_deadline() {
let (release, wait) = std::sync::mpsc::channel::<()>();
let done = std::thread::Builder::new()
.name("done".into())
.spawn(|| {})
.unwrap();
let stuck = std::thread::Builder::new()
.name("stuck".into())
.spawn(move || {
let _ = wait.recv();
})
.unwrap();
let running = join_until(
vec![done, stuck],
Instant::now() + Duration::from_millis(100),
);
assert_eq!(running, ["stuck"]);
drop(release);
}
#[test]
fn session_is_unroutable_before_teardown_starts() {
+315 -13
View File
@@ -18,8 +18,8 @@ use std::time::{Duration, Instant};
use base64::Engine as _;
use pdfs_core::control::{
FileThumbRequest, PhotoItem, PhotoKind, PhotoThumb, ThumbnailBuildStatus, is_raw_image_name,
is_thumbnail_image_name,
FileThumbRequest, MappingProgress, PhotoItem, PhotoKind, PhotoLocation, PhotoThumb, PlaceInfo,
ThumbnailBuildStatus, Topic, is_raw_image_name, is_thumbnail_image_name,
};
use pdfs_core::db::{self, StoredPhoto};
use pdfs_core::{CoreError, CoreResult};
@@ -43,6 +43,8 @@ const THUMB_QUALITY: u8 = 82;
/// thumbnails. Bounded: a screenful of 20 MB digicam JPEGs would otherwise
/// saturate the link and starve the rest of the daemon.
pub(crate) const THUMB_GEN_CONCURRENCY: usize = 4;
/// Shots the similar-photo pass fetches thumbnails for at a time.
const SIMILAR_CHUNK: usize = 64;
/// Images processed between recursive-build progress updates.
const THUMB_BUILD_CHUNK: usize = 16;
/// A recursive build may wait briefly for an opportunistic tile job that
@@ -81,6 +83,46 @@ fn ratio_of(bytes: &[u8]) -> Option<f64> {
(height > 0).then(|| f64::from(width) / f64::from(height))
}
/// The difference hash of each cached thumbnail in `paths`, keyed like the
/// input, or [`pdfs_core::similar::NO_HASH`] for one that can't be decoded or
/// shows no picture. Decoding is spread over the machine's cores.
fn similar_hashes(paths: Vec<(String, PathBuf)>) -> Vec<(String, u64)> {
fn hash(path: &Path) -> u64 {
image::ImageReader::open(path)
.ok()
.and_then(|reader| reader.with_guessed_format().ok())
.and_then(|reader| reader.decode().ok())
.and_then(|image| {
let luma = image.to_luma8();
pdfs_core::similar::difference_hash(
luma.width() as usize,
luma.height() as usize,
luma.as_raw(),
)
})
.unwrap_or(pdfs_core::similar::NO_HASH)
}
let threads = std::thread::available_parallelism().map_or(1, |n| n.get());
let per_thread = paths.len().div_ceil(threads).max(1);
std::thread::scope(|scope| {
let workers: Vec<_> = paths
.chunks(per_thread)
.map(|chunk| {
scope.spawn(move || {
chunk
.iter()
.map(|(uid, path)| (uid.clone(), hash(path)))
.collect::<Vec<_>>()
})
})
.collect();
workers
.into_iter()
.flat_map(|worker| worker.join().unwrap_or_default())
.collect()
})
}
/// A thumbnail the daemon made itself, and the aspect ratio of the photo it was
/// made from (free at that point — the full image had to be decoded anyway).
struct GeneratedThumb {
@@ -601,6 +643,16 @@ fn extract_raw_preview(
pub(crate) type TrashOutcome = (Vec<String>, Vec<(String, String)>);
/// Formats supported by the local thumbnail decoder and exposed by the GUI.
/// Clears the timeline refresh's progress when dropped, so an error or a
/// cancelled refresh doesn't leave a bar standing.
struct ProgressGuard<'a>(&'a parking_lot::Mutex<Option<(usize, usize)>>);
impl Drop for ProgressGuard<'_> {
fn drop(&mut self) {
*self.0.lock() = None;
}
}
/// What a timeline enrichment pass resolves for one photo, beyond what the
/// timeline listing already carries.
#[derive(Clone, Debug)]
@@ -609,6 +661,8 @@ struct PhotoMeta {
media_type: Option<String>,
favorite: bool,
content_hash: Option<String>,
/// Where the photo was taken, as its uploader recorded it.
location: Option<(f64, f64)>,
}
impl Core {
@@ -637,6 +691,7 @@ impl Core {
kind: Option<PhotoKind>,
range: Option<(i64, i64)>,
favorites: bool,
not_in_album: bool,
) -> CoreResult<Option<Vec<PhotoItem>>> {
let count = self.db.photos_count().map_err(CoreError::from)?;
if count == 0 {
@@ -653,10 +708,16 @@ impl Core {
} else if self.listing_stale(PHOTOS_SYNCED_MS, TIMELINE_TTL) {
self.spawn_timeline_refresh();
}
// The filter is only as good as the album contents on disk, and those
// are fetched lazily, per album opened. Fill in the missing ones before
// the first page; later pages reuse what that fetch stored.
if not_in_album && offset == 0 {
self.ensure_album_contents()?;
}
let page = self
.db
.photos_page(offset, limit, kind, range, favorites)
.photos_page(offset, limit, kind, range, favorites, not_in_album)
.map_err(CoreError::from)?;
Ok(Some(page.into_iter().map(|p| self.photo_item(p)).collect()))
}
@@ -722,6 +783,159 @@ impl Core {
Ok(())
}
/// The timeline's duplicate sets, as wire items (see
/// [`db::Db::photo_duplicates`]).
pub(crate) fn photo_duplicates(&self) -> CoreResult<Vec<Vec<PhotoItem>>> {
let sets = self.db.photo_duplicates().map_err(CoreError::from)?;
Ok(sets
.into_iter()
.map(|set| set.into_iter().map(|p| self.photo_item(p)).collect())
.collect())
}
/// The sets of shots that look alike, as wire items (see
/// [`db::Db::photo_similar`]), how far hashing has got while it runs, and
/// how many shots couldn't be compared once it's done. Starts a hashing
/// pass for the shots not hashed yet unless one is running.
pub(crate) fn photo_similar(
&self,
) -> CoreResult<(Vec<Vec<PhotoItem>>, Option<MappingProgress>, usize)> {
self.spawn_similar_hashing();
let sets = self.db.photo_similar().map_err(CoreError::from)?;
let (done, total) = self.db.similar_progress().map_err(CoreError::from)?;
let hashing = self
.similar_hashing
.load(Ordering::SeqCst)
.then_some(MappingProgress { done, total });
let uncompared = if hashing.is_some() { 0 } else { total - done };
let sets = sets
.into_iter()
.map(|set| set.into_iter().map(|p| self.photo_item(p)).collect())
.collect();
Ok((sets, hashing, uncompared))
}
fn spawn_similar_hashing(&self) {
if self.similar_hashing.swap(true, Ordering::SeqCst) {
return;
}
let core = self.clone();
let spawned = std::thread::Builder::new()
.name("pdfs-similar".into())
.spawn(move || {
if let Err(e) = core.hash_similar() {
warn!(error = %e, "similar-photo hashing failed");
}
core.similar_hashing.store(false, Ordering::SeqCst);
});
if let Err(e) = spawned {
warn!(error = %e, "could not start similar-photo hashing");
self.similar_hashing.store(false, Ordering::SeqCst);
}
}
/// Hash the look of every shot not hashed yet, a chunk at a time. Only
/// thumbnails the server has are used: a photo without one is skipped
/// rather than downloaded whole, and tried again on the next pass.
fn hash_similar(&self) -> CoreResult<()> {
let mut tried: HashSet<String> = HashSet::new();
loop {
if self.shutdown.is_stopping() {
return Ok(());
}
// Shots tried this pass without a thumbnail are still unhashed and
// come back first, so ask past them.
let chunk: Vec<NodeUid> = self
.db
.photos_unhashed(tried.len() + SIMILAR_CHUNK)
.map_err(CoreError::from)?
.into_iter()
.filter(|photo| tried.insert(photo.uid.clone()))
.take(SIMILAR_CHUNK)
.filter_map(|photo| parse_uid(&photo.uid))
.collect();
if chunk.is_empty() {
return Ok(());
}
let paths: Vec<(String, PathBuf)> = self
.photo_thumbs_with(&chunk, false)
.into_iter()
.filter_map(|thumb| Some((thumb.uid, PathBuf::from(thumb.path?))))
.collect();
for (uid, hash) in similar_hashes(paths) {
self.db
.photo_set_similar_hash(&uid, hash)
.map_err(CoreError::from)?;
}
}
}
/// The towns the photos were taken in, as wire items. A town the bundled
/// table no longer knows (the stored id came from an older table) is left
/// out rather than shown without a name.
pub(crate) fn photo_places(&self) -> CoreResult<Vec<PlaceInfo>> {
let places = self.db.photo_places().map_err(CoreError::from)?;
Ok(places
.into_iter()
.filter_map(|place| {
let city = pdfs_core::places::city(place.id)?;
Some(PlaceInfo {
id: place.id,
name: city.name.to_string(),
country: city.country.to_string(),
latitude: city.latitude,
longitude: city.longitude,
photo_count: place.count,
cover: self.photo_item(place.cover),
})
})
.collect())
}
/// A page of the photos taken in town `id`, as wire items.
pub(crate) fn place_photos(
&self,
id: u32,
offset: usize,
limit: usize,
) -> CoreResult<Vec<PhotoItem>> {
let photos = self
.db
.photos_at_place(id, offset, limit)
.map_err(CoreError::from)?;
Ok(photos.into_iter().map(|p| self.photo_item(p)).collect())
}
/// Every photo taken in town `id` with its own location, as wire items.
pub(crate) fn place_locations(&self, id: u32) -> CoreResult<Vec<PhotoLocation>> {
let photos = self
.db
.photo_locations_at_place(id)
.map_err(CoreError::from)?;
Ok(photos
.into_iter()
.map(|(photo, (latitude, longitude))| PhotoLocation {
photo: self.photo_item(photo),
latitude,
longitude,
})
.collect())
}
/// Photos taken on today's date in earlier years, as wire items. Served
/// from the stored timeline only: the strip is a nicety, not worth a wait
/// on the network, and the timeline's own load keeps that store fresh.
pub(crate) fn photos_on_this_day(&self, limit: usize) -> CoreResult<Vec<PhotoItem>> {
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map_or(0, |d| d.as_secs() as i64);
let photos = self
.db
.photos_on_this_day(now, limit)
.map_err(CoreError::from)?;
Ok(photos.into_iter().map(|p| self.photo_item(p)).collect())
}
/// Every file of one photo's group, in server order, as wire items. A photo
/// the timeline does not hold comes back empty, which the lightbox reads as
/// "nothing to switch between".
@@ -801,19 +1015,30 @@ impl Core {
/// A photo that is one file of a group takes its whole group with it. The
/// gallery shows a RAW+JPEG shot as one tile, so deleting that tile has to
/// delete the shot — leaving the RAW behind would put the photo back on the
/// Raw tab and nowhere else.
pub(crate) fn trash_photos(&self, uids: &[NodeUid]) -> CoreResult<TrashOutcome> {
/// Raw tab and nowhere else. `files_only` turns that off, for removing
/// one copy of a shot that is stored twice.
pub(crate) fn trash_photos(
&self,
uids: &[NodeUid],
files_only: bool,
) -> CoreResult<TrashOutcome> {
if uids.is_empty() {
return Ok((Vec::new(), Vec::new()));
}
let uids = &self.expand_photo_groups(uids);
let expanded;
let uids = if files_only {
uids
} else {
expanded = self.expand_photo_groups(uids);
&expanded
};
for uid in uids {
self.require_photo_writable(uid)
.map_err(|errno| self.errno_error(errno, "trash access"))?;
}
let outcomes = self
.rt
.block_on(self.client.trash_nodes(uids))
.block_on(self.drive.trash_nodes(uids))
.map_err(|e| CoreError::from_api(&e, "trash"))?;
let (trashed, failed) = pdfs_core::batch::split(outcomes);
@@ -855,6 +1080,13 @@ impl Core {
/// milliseconds where the raw costs a full download and a raw decode. The
/// tile still answers under the uid that was asked for.
pub(crate) fn photo_thumbs(&self, requested: &[NodeUid]) -> Vec<PhotoThumb> {
self.photo_thumbs_with(requested, true)
}
/// [`Core::photo_thumbs`], with local generation only when `generate` is
/// set. Without it, a photo the server has no thumbnail for is left
/// without one rather than downloaded whole.
fn photo_thumbs_with(&self, requested: &[NodeUid], generate: bool) -> Vec<PhotoThumb> {
let ttype = ThumbnailType::Thumbnail.as_i32();
// Only a raw can be stood in for, and that is one batched read; the
// per-shot group lookup is then paid for raws alone, not for a page of
@@ -955,7 +1187,7 @@ impl Core {
// that never generated one. Off the request path: a full-size photo
// takes far longer to fetch than the whole rest of the batch, and the
// thumbnails that *are* ready must not wait behind it.
if !missing.is_empty() {
if generate && !missing.is_empty() {
self.spawn_generate_thumbs(missing, &tags, &names, ThumbJob::Photos);
}
}
@@ -1690,7 +1922,14 @@ impl Core {
// can land in different chunks, and one of them may not resolve at all.
let mut main_of: HashMap<String, String> = HashMap::new();
let mut stopped = false;
for chunk in uids.chunks(TIMELINE_ENRICH_CHUNK) {
// Photos read before count as done, so the bar picks up where an
// interrupted pass left off. Cleared however the pass ends.
let skipped = items.len() - uids.len();
let progress = ProgressGuard(&self.timeline_progress);
if !uids.is_empty() {
*progress.0.lock() = Some((skipped, items.len()));
}
for (index, chunk) in uids.chunks(TIMELINE_ENRICH_CHUNK).enumerate() {
// Teardown cancels the runtime under this loop, and every chunk
// still in flight then fails its decrypt task — a burst of
// "skipping undecryptable photo ... task was cancelled" warnings
@@ -1717,6 +1956,11 @@ impl Core {
.is_some_and(|p| p.tags.contains(&PhotoTag::Favorite));
let key = node.uid.to_string();
let content_hash = node.photo.as_ref().and_then(|p| p.content_hash.clone());
let location = node
.photo
.as_ref()
.and_then(|p| p.location)
.map(|l| (l.latitude, l.longitude));
if let Some(photo) = node.photo.as_ref() {
if let Some(main) = photo.main_photo_uid.as_ref() {
main_of.insert(key.clone(), main.to_string());
@@ -1732,13 +1976,17 @@ impl Core {
media_type,
favorite,
content_hash,
location,
},
);
}
}
Err(e) => warn!(error = %e, "resolving photo metadata for a timeline chunk failed"),
}
let done = (skipped + (index + 1) * TIMELINE_ENRICH_CHUNK).min(items.len());
*progress.0.lock() = Some((done, items.len()));
}
drop(progress);
let rows: Vec<db::TimelineRow> = items
.iter()
@@ -1753,6 +2001,7 @@ impl Core {
media_type: m.media_type,
favorite: Some(m.favorite),
content_hash: m.content_hash,
location: m.location,
main_uid,
resolved_at: Some(resolved_at),
..db::TimelineRow::new(key, it.capture_time)
@@ -1792,13 +2041,25 @@ impl Core {
}
let core = self.clone();
self.rt.spawn(async move {
if let Err(e) = core.refresh_timeline().await {
warn!(error = %e, "background timeline refresh failed");
}
let refreshed = core.refresh_timeline().await;
core.timeline_refreshing.store(false, Ordering::SeqCst);
// Only a refresh that landed: a front-end re-reading after a failed
// one would find the timeline still stale and start another.
match refreshed {
Ok(_) => core.events.publish(&[Topic::Photos]),
Err(e) => warn!(error = %e, "background timeline refresh failed"),
}
});
}
/// How far the running timeline refresh has got reading photo metadata, or
/// `None` when no refresh is reading any.
pub(crate) fn timeline_progress(&self) -> Option<MappingProgress> {
self.timeline_progress
.lock()
.map(|(done, total)| MappingProgress { done, total })
}
/// Whether a timeline refresh is running. A front-end that asked for one
/// watches this to know when the new timeline is worth re-reading.
pub(crate) fn timeline_refreshing(&self) -> bool {
@@ -1833,10 +2094,11 @@ mod thumb_tests {
use super::{
RawTempFile, ScaleAttempt, THUMB_EDGE, UPLOAD_THUMB_MAX_BYTES, exiftool_binary,
fit_upload_thumb, is_video_name, ratio_of, scale_thumbnail, scale_thumbnail_with_exiftool,
thumbnail_build_may_start, upload_thumbnails_with, video_thumbnail,
similar_hashes, thumbnail_build_may_start, upload_thumbnails_with, video_thumbnail,
};
use pdfs_core::control::{ThumbnailBuildStatus, is_thumbnail_image_name};
use proton_drive_rs::ThumbnailType;
use std::collections::HashMap;
use std::ffi::OsStr;
use std::os::unix::fs::PermissionsExt as _;
use std::time::Duration;
@@ -2110,6 +2372,46 @@ mod thumb_tests {
assert!(thumbnail_build_may_start(&idle, "pictures/second").unwrap());
}
#[test]
fn thumbnails_hash_alike_across_sizes_and_undecodable_ones_get_no_hash() {
let tmp = scratch("similar-hashes");
std::fs::create_dir_all(&tmp).unwrap();
// One picture at two sizes: a diagonal gradient with a bright corner.
let picture = |width: u32, height: u32| {
let image = image::GrayImage::from_fn(width, height, |x, y| {
let (fx, fy) = (x * 4 / width, y * 4 / height);
image::Luma([if fx + fy == 0 {
250
} else {
(fx * 40 + fy * 20) as u8
}])
});
let mut bytes = Vec::new();
image::codecs::jpeg::JpegEncoder::new(&mut bytes)
.encode_image(&image)
.unwrap();
bytes
};
std::fs::write(tmp.join("big"), picture(512, 384)).unwrap();
std::fs::write(tmp.join("small"), picture(256, 192)).unwrap();
std::fs::write(tmp.join("torn"), b"not a picture").unwrap();
let hashes: HashMap<String, u64> = similar_hashes(
["big", "small", "torn", "gone"]
.iter()
.map(|name| (name.to_string(), tmp.join(name)))
.collect(),
)
.into_iter()
.collect();
let _ = std::fs::remove_dir_all(&tmp);
assert_eq!(hashes.len(), 4);
assert_ne!(hashes["big"], pdfs_core::similar::NO_HASH);
assert!(pdfs_core::similar::alike(hashes["big"], hashes["small"]));
assert_eq!(hashes["torn"], pdfs_core::similar::NO_HASH);
assert_eq!(hashes["gone"], pdfs_core::similar::NO_HASH);
}
#[test]
fn an_uploaded_photo_carries_its_thumbnail() {
let tmp = scratch("upload-photo");
+10 -10
View File
@@ -1,5 +1,5 @@
//! Reading and writing this machine's `profile.json` in its device root, and
//! the restore that rebuilds a machine from one (features.md 5.2 and 5.3).
//! the restore that rebuilds a machine from one.
//!
//! The document itself is defined in [`pdfs_core::profile`]; this is the half
//! that talks to Drive. Saving is best-effort and never blocks a user action:
@@ -101,7 +101,7 @@ impl Core {
match existing {
Some(uid) => self
.rt
.block_on(self.client.upload_new_revision_from(
.block_on(self.drive.upload_new_revision_from(
&uid,
Cursor::new(bytes),
len,
@@ -112,7 +112,7 @@ impl Core {
.map_err(|e| CoreError::from_api(&e, "upload profile revision"))?,
None => {
self.rt
.block_on(self.client.upload_file_replacing_draft_from(
.block_on(self.drive.upload_file_replacing_draft_from(
&dir_uid,
PROFILE_FILE_NAME,
"application/json",
@@ -182,14 +182,14 @@ impl Core {
) -> CoreResult<Option<NodeUid>> {
let uids = self
.rt
.block_on(self.client.enumerate_folder_children_node_uids(parent_uid))
.block_on(self.drive.enumerate_folder_children_node_uids(parent_uid))
.map_err(|e| CoreError::from_api(&e, "list device root"))?;
if uids.is_empty() {
return Ok(None);
}
let nodes = self
.rt
.block_on(self.client.enumerate_nodes(&uids))
.block_on(self.drive.enumerate_nodes(&uids))
.map_err(|e| CoreError::from_api(&e, "resolve device root children"))?;
Ok(nodes
.into_iter()
@@ -210,7 +210,7 @@ impl Core {
let uid = self
.rt
.block_on(
self.client
self.drive
.create_folder(root_uid, PROFILE_DIR_NAME, Some(now_secs())),
)
.map_err(|e| CoreError::from_api(&e, "create profile folder"))?;
@@ -241,12 +241,12 @@ impl Core {
};
let mut buf: Vec<u8> = Vec::new();
self.rt
.block_on(self.client.download_file_to(&uid, &mut buf))
.block_on(self.drive.download_file_to(&uid, &mut buf))
.map_err(|e| CoreError::from_api(&e, "download profile"))?;
Profile::parse(&buf).map(Some).map_err(CoreError::invalid)
}
// ---- restore (features.md 5.2) ----------------------------------------
// ---- restore ----------------------------------------
/// The folders under a device that could be synced here, each carrying a
/// proposed local path. `device` names another registered device by uid;
@@ -277,13 +277,13 @@ impl Core {
let uids = self
.rt
.block_on(self.client.enumerate_folder_children_node_uids(&root_uid))
.block_on(self.drive.enumerate_folder_children_node_uids(&root_uid))
.map_err(|e| CoreError::from_api(&e, "list device root"))?;
let nodes = if uids.is_empty() {
Vec::new()
} else {
self.rt
.block_on(self.client.enumerate_nodes(&uids))
.block_on(self.drive.enumerate_nodes(&uids))
.map_err(|e| CoreError::from_api(&e, "resolve device root children"))?
};
+247 -5
View File
@@ -1,12 +1,16 @@
//! The user's view of the pending-op queue: what has not reached the remote
//! yet, why, and a way to stop waiting out a backoff.
//!
//! Read-only apart from "retry now", which only moves a row's next-attempt time.
//! Nothing here drops a queued op: its staged blob is the only copy of the
//! user's write (see `docs/RECOVERY.md`).
//! Read-only apart from "retry now", which only moves a row's next-attempt time,
//! "export", which copies a staged blob out, and "discard". Only a discard the
//! user asks for drops a queued op: its staged blob is the only copy of the
//! user's write (see `docs/RECOVERY.md`), so it is the user's call, made after
//! an export or for a change with no content.
use super::*;
use pdfs_core::control::PendingOpInfo;
use std::io::Write;
use pdfs_core::control::{PendingOpInfo, SyncIssue};
use pdfs_core::db::{FAILING_ATTEMPTS, PARK_UNTIL};
impl Core {
@@ -16,10 +20,15 @@ impl Core {
.db
.pending_ops()
.map_err(|e| CoreError::internal(format!("listing the queue: {e}")))?;
let issues = self
.db
.op_issues()
.map_err(|e| CoreError::internal(format!("listing sync issues: {e}")))?;
Ok(ops
.into_iter()
.map(|op| {
let parked = op.next_attempt_at >= PARK_UNTIL;
let exportable = exportable(&op).is_some();
let path = self
.db
.node_path(&op.uid)
@@ -37,7 +46,9 @@ impl Core {
queued_at: op.created_at / 1000,
next_attempt_at: (!parked).then_some(op.next_attempt_at / 1000),
parked,
failing: op.attempts >= FAILING_ATTEMPTS,
failing: op.attempts >= FAILING_ATTEMPTS || issues.contains_key(&op.id),
issue: issues.get(&op.id).map(|issue| SyncIssue::parse(issue)),
exportable,
}
})
.collect())
@@ -57,4 +68,235 @@ impl Core {
}
Ok(moved)
}
/// Copy the content of queued op `id` to `dest`, which must not exist yet.
/// The op stays queued. Returns the bytes written.
pub(crate) fn export_pending_op(&self, id: i64, dest: &Path) -> CoreResult<u64> {
if !dest.is_absolute() {
return Err(CoreError::invalid(format!(
"destination must be an absolute path: {}",
dest.display()
)));
}
let ops = self
.db
.pending_ops()
.map_err(|e| CoreError::internal(format!("listing the queue: {e}")))?;
let op = ops
.iter()
.find(|op| op.id == id)
.ok_or_else(|| CoreError::not_found(format!("no queued change {id}")))?;
let blob = exportable(op).ok_or_else(|| {
CoreError::invalid(format!("queued change {id} carries no file content"))
})?;
let written = copy_to_new_file(blob, dest)?;
info!(id, dest = %dest.display(), written, "exported a queued change's content");
Ok(written)
}
/// Drop queued op `id` and undo the local change it stood for. Returns the
/// node's path.
///
/// A node only made here goes, as if it had been deleted before it was
/// sent. A node Drive has goes back to what Drive has: every change queued
/// on it goes, since each was made on top of the one before. What is queued
/// below it stays.
pub(crate) fn discard_pending_op(&self, id: i64) -> CoreResult<String> {
let ops = self
.db
.pending_ops()
.map_err(|e| CoreError::internal(format!("listing the queue: {e}")))?;
let op = ops
.into_iter()
.find(|op| op.id == id)
.ok_or_else(|| CoreError::not_found(format!("no queued change {id}")))?;
let uid = parse_node_uid(&op.uid)
.ok_or_else(|| CoreError::internal(format!("queued change {id} names {}", op.uid)))?;
let path = self
.db
.node_path(&op.uid)
.ok()
.flatten()
.filter(|path| !path.is_empty())
.or(op.name)
.unwrap_or_else(|| op.uid.clone());
if self.is_open_anywhere(&uid) {
return Err(CoreError::conflict(format!(
"{path} is open; close it first"
)));
}
if is_local_uid(&uid) {
self.discard_local_node(&uid, &path)?;
} else {
self.discard_remote_change(&uid, &path)?;
}
info!(id, %uid, path, "discarded a queued change");
Ok(path)
}
/// Forget a node made here, and the ops that would have sent it.
fn discard_local_node(&self, uid: &NodeUid, path: &str) -> CoreResult<()> {
// Its children's ops go with its own, and their content with them.
if self
.db
.has_children(uid)
.map_err(|e| CoreError::internal(format!("listing {path}: {e}")))?
{
return Err(CoreError::conflict(format!(
"{path} is not empty; discard what is in it first"
)));
}
self.discard_queued_ops(uid)
.map_err(|_| CoreError::internal(format!("dropping the changes to {path}")))?;
self.for_each_mount(|st, notify| {
background::forget_and_notify(st, notify, uid);
});
self.cache.evict(uid);
self.evict_reader(uid);
Ok(())
}
/// Put a node Drive has back as Drive has it.
fn discard_remote_change(&self, uid: &NodeUid, path: &str) -> CoreResult<()> {
// Asked first: the change is only dropped once what replaces it is known.
let node = self
.fetch_node_remote(uid)
.map_err(|e| CoreError::from_api(&e, "reading the node from Drive"))?
.filter(|node| !node.trashed);
self.cancel_upload(uid);
let blobs = self
.db
.discard_ops_on_node(&uid.to_string())
.map_err(|e| CoreError::internal(format!("dropping the changes to {path}: {e}")))?
.ok_or_else(|| {
CoreError::conflict(format!("{path} is being sent right now; try again"))
})?;
self.release_dropped_ops(uid, blobs);
self.hidden.lock().remove(uid);
match node {
Some(node) => self.for_each_mount(|st, notify| {
put_back(st, notify, node);
}),
None => self.for_each_mount(|st, notify| {
background::forget_and_notify(st, notify, uid);
}),
}
self.cache.evict(uid);
self.evict_reader(uid);
Ok(())
}
}
/// Show `node` as Drive has it, where Drive has it, under the inode it has.
fn put_back(st: &mut State, notify: &mut NotifyBatch, node: Node) {
let parent = node.parent_uid.as_ref().and_then(|parent| {
st.by_uid
.get(parent)
.copied()
.map(|ino| (ino, parent.clone()))
});
let here = st.by_uid.get(&node.uid).and_then(|ino| {
st.entries
.get(ino)
.map(|entry| (*ino, entry.parent, entry.node.name.clone()))
});
match (here, parent) {
(Some((ino, from, name)), Some((to, to_uid))) => {
st.relocate(ino, from, to, &to_uid, &node.name);
st.write_through(node.clone());
st.intern_mem(to, node);
notify.inval_entry(from, name);
notify.inval_inode(ino);
}
(here, parent) => {
if let Some((_, from, name)) = here {
st.forget_mem(&node.uid);
st.invalidate_listing(from);
notify.inval_entry(from, name);
}
if let Some((to, _)) = parent {
st.invalidate_listing(to);
notify.inval_entry(to, node.name.clone());
}
st.write_through(node);
}
}
}
/// Copy `blob` to `dest`, which must not exist: never replace something the
/// user already has.
fn copy_to_new_file(blob: &Path, dest: &Path) -> CoreResult<u64> {
let mut source = File::open(blob)
.map_err(|e| CoreError::internal(format!("open {}: {e}", blob.display())))?;
let mut out = std::fs::OpenOptions::new()
.write(true)
.create_new(true)
.open(dest)
.map_err(|e| match e.kind() {
std::io::ErrorKind::AlreadyExists => {
CoreError::invalid(format!("{} already exists", dest.display()))
}
_ => CoreError::internal(format!("create {}: {e}", dest.display())),
})?;
std::io::copy(&mut source, &mut out)
.and_then(|written| out.flush().and(out.sync_all()).map(|()| written))
.map_err(|e| CoreError::internal(format!("write {}: {e}", dest.display())))
}
/// The staged blob of an op that carries file content.
fn exportable(op: &PendingOp) -> Option<&Path> {
matches!(op.kind.as_str(), OP_CREATE | OP_REVISION)
.then_some(op.blob_path.as_deref())
.flatten()
.map(Path::new)
}
#[cfg(test)]
mod tests {
use super::*;
fn op(kind: &str, blob: Option<&str>) -> PendingOp {
PendingOp {
id: 1,
kind: kind.to_string(),
uid: "vol~a".to_string(),
parent_uid: None,
name: None,
blob_path: blob.map(str::to_string),
meta_json: None,
created_at: 0,
attempts: 0,
last_error: None,
next_attempt_at: 0,
}
}
#[test]
fn only_a_change_that_carries_file_content_can_be_exported() {
assert!(exportable(&op(OP_CREATE, Some("/staging/a"))).is_some());
assert!(exportable(&op(OP_REVISION, Some("/staging/a"))).is_some());
assert!(exportable(&op(OP_CREATE, None)).is_none());
assert!(exportable(&op(OP_RENAME, None)).is_none());
assert!(exportable(&op(OP_TRASH, None)).is_none());
}
#[test]
fn an_export_copies_the_bytes_and_never_replaces_a_file() {
let dir = std::env::temp_dir().join(format!("pdfs-export-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let blob = dir.join("blob");
std::fs::write(&blob, b"queued bytes").unwrap();
let dest = dir.join("copy.txt");
assert_eq!(copy_to_new_file(&blob, &dest).unwrap(), 12);
assert_eq!(std::fs::read(&dest).unwrap(), b"queued bytes");
std::fs::write(&dest, b"the user's own").unwrap();
assert!(copy_to_new_file(&blob, &dest).is_err());
assert_eq!(std::fs::read(&dest).unwrap(), b"the user's own");
// The staged blob is the only copy of the change: an export leaves it.
assert_eq!(std::fs::read(&blob).unwrap(), b"queued bytes");
let _ = std::fs::remove_dir_all(&dir);
}
}
+102 -41
View File
@@ -36,6 +36,21 @@ pub(super) const REPAIR_MAX: u64 = 512 * 1024 * 1024;
/// demand read in the SDK's flat block semaphore.
pub(super) const PREFETCH_BUDGET: usize = 8;
/// How long one read may wait on the network before it fails with `EIO`.
///
/// The SDK bounds each request, not a read: a block that times out is retried,
/// and a read can need several blocks, a block table and a repair. On a link
/// that stalled without dropping, one `read` held a FUSE worker for 34 minutes
/// and the application behind it with it (`docs/BUGS.md` B107). An error the
/// caller can retry beats a process stuck in `D` state.
pub(super) const READ_FETCH_TIMEOUT: Duration = Duration::from_secs(120);
/// How long a read of blocks nothing has cached waits for a lost link to come
/// back before failing. Long enough to ride out a Wi-Fi blip, short enough
/// that a program reading while the laptop is really offline gets its `EIO`
/// before the user gives up on it.
const READ_OFFLINE_WAIT: Duration = Duration::from_secs(30);
/// Cap on tracked sequential streams. Only reads in progress matter, so the map
/// is cleared wholesale rather than aged — losing the ramp costs one read.
const MAX_PREFETCH_STREAMS: usize = 256;
@@ -55,7 +70,7 @@ const MAX_OPEN_READERS: usize = 64;
/// `(mtime, size)` — the same validity pair the content cache uses (a new
/// revision bumps mtime). On a mismatch the reader is dropped and reopened.
pub(super) struct CachedReader {
reader: Arc<RevisionReader>,
reader: Arc<dyn RevisionRead>,
mtime: i64,
size: u64,
/// For LRU eviction.
@@ -86,7 +101,7 @@ pub(super) struct PendingOpen {
/// must not join this open — it would get a reader for the wrong bytes.
mtime: i64,
size: u64,
rx: tokio::sync::watch::Receiver<Option<Result<Arc<RevisionReader>, Errno>>>,
rx: tokio::sync::watch::Receiver<Option<Result<Arc<dyn RevisionRead>, Errno>>>,
}
/// Distinguishes `PendingOpen`s. Only uniqueness matters, so a plain counter
@@ -298,16 +313,16 @@ impl Core {
uid: &NodeUid,
mtime: i64,
fsize: u64,
) -> Result<Arc<RevisionReader>, Errno> {
) -> Result<Arc<dyn RevisionRead>, Errno> {
loop {
// Decide under the lock, act outside it — network I/O must never run
// under a std `Mutex`, and an await must never hold one.
enum Act {
Hit(Arc<RevisionReader>),
Join(tokio::sync::watch::Receiver<Option<Result<Arc<RevisionReader>, Errno>>>),
Hit(Arc<dyn RevisionRead>),
Join(tokio::sync::watch::Receiver<Option<Result<Arc<dyn RevisionRead>, Errno>>>),
Lead(
u64,
tokio::sync::watch::Sender<Option<Result<Arc<RevisionReader>, Errno>>>,
tokio::sync::watch::Sender<Option<Result<Arc<dyn RevisionRead>, Errno>>>,
),
}
let act = {
@@ -361,9 +376,10 @@ impl Core {
}
Act::Lead(id, tx) => {
debug!(%uid, mtime, fsize, "opening revision");
let result = match self.client.open_revision(uid).await {
Ok(reader) => Ok(Arc::new(reader)),
let result = match self.drive.open_revision(uid).await {
Ok(reader) => Ok(reader),
Err(e) => {
self.lost_link(&e, "open a revision");
warn!(%uid, error = %e, "open_revision failed");
Err(Errno::EIO)
}
@@ -448,7 +464,7 @@ impl Core {
/// it, and the [`BLOCK_SIZE`] assumption when it has not — which is every
/// first read, because learning it means opening a `RevisionReader`, and
/// opening one per read is precisely the per-file key derivation the block
/// cache exists to avoid (bugs.md B12, B85).
/// cache exists to avoid (`docs/BUGS.md` B12, B85).
///
/// Being wrong here is not a correctness problem: `read_at` plans over the
/// real sizes and clamps to what was asked for either way. It costs a
@@ -477,15 +493,20 @@ impl Core {
) -> Result<Vec<u8>, Errno> {
// A queued write has not reached the remote yet, so the remote's current
// revision is stale and the staged blob is the truth. Serve from it until
// the drain worker lands the upload (offline.md Phase 3).
// the drain worker lands the upload.
if let Some(pending) = self.pending.lock().get(uid).cloned() {
return self.read_pending(&pending, offset, len);
}
// A node created offline and never written has no blob and no remote: it
// is an empty file, and asking the API about a `local~` uid would only
// earn a 404 (offline.md Phase 3b).
// earn a 404. Unless its create landed since the
// read looked: the landing moved the queued write to the uid Drive
// gave it, and the row says which (`docs/BUGS.md` B179).
if is_local_uid(uid) {
return Ok(Vec::new());
return match self.landed_stand_in_uid(uid) {
Some(real) => self.read_range(&real, mtime, fsize, offset, len, cache_blocks),
None => Ok(Vec::new()),
};
}
self.read_range_remote(uid, mtime, fsize, offset, len, cache_blocks)
}
@@ -552,7 +573,7 @@ impl Core {
/// leader's claim is gone by then, so this cannot recurse.
async fn fetch_block(
&self,
reader: &Arc<RevisionReader>,
reader: &Arc<dyn RevisionRead>,
uid: &NodeUid,
mtime: i64,
fsize: u64,
@@ -615,10 +636,10 @@ impl Core {
///
/// A block that comes back shorter than the file's size says it must be is
/// never served, cached or promoted: it is repaired from the whole-file
/// download, or the read fails. See [`Core::repair_block`] (bugs.md B84).
/// download, or the read fails. See [`Core::repair_block`] (`docs/BUGS.md` B84).
async fn read_block(
&self,
reader: &Arc<RevisionReader>,
reader: &Arc<dyn RevisionRead>,
uid: &NodeUid,
mtime: i64,
fsize: u64,
@@ -630,7 +651,7 @@ impl Core {
// file's boundaries are wrong, and the failure is not confined to short
// reads: sizes that *overstate* a block shift every block after it and
// return full-length reads of the wrong bytes. So a disagreement of any
// kind takes the file off the range path entirely (bugs.md B84).
// kind takes the file off the range path entirely (`docs/BUGS.md` B84).
let bytes = if reader.size() != fsize {
warn!(
%uid, bidx = span.idx, fsize,
@@ -643,12 +664,13 @@ impl Core {
// Learned here rather than on a path of its own: this is where a
// reader is open anyway, and recording the real geometry is what
// lets the *next* read plan its blocks on the revision's own
// boundaries instead of the 4 MiB fallback (bugs.md B85).
// boundaries instead of the 4 MiB fallback (`docs/BUGS.md` B85).
if cache_blocks {
self.cache
.store_block_geometry(uid, mtime, fsize, reader.block_sizes());
}
let bytes = reader.read_at(span.start, span.len).await.map_err(|e| {
self.lost_link(&e, "read a block");
warn!(%uid, start = span.start, len = span.len, error = %e, "block read failed");
Errno::EIO
})?;
@@ -684,7 +706,7 @@ impl Core {
/// The range path derives every block boundary from the revision's recorded
/// block sizes. When those understate the file, reads come back short with
/// no error anywhere, and the caller hands userspace a truncated file that
/// looks like a clean EOF (bugs.md B84). `download_file_to` does not do that
/// looks like a clean EOF (`docs/BUGS.md` B84). `download_file_to` does not do that
/// arithmetic — it streams the block list — so it returns the whole file
/// where the range path cannot, which is why pinning was the workaround.
/// This does the same thing on the user's behalf, once, and the resulting
@@ -761,7 +783,7 @@ impl Core {
error!(%uid, %error, "creating a scratch file for a repair download failed");
Errno::EIO
})?;
let outcome = self.client.download_file_to(uid, &mut file).await;
let outcome = self.drive.download_file_to(uid, &mut file).await;
let stored = (|| {
outcome.map_err(|error| {
warn!(%uid, %error, "repair download failed");
@@ -896,7 +918,7 @@ impl Core {
let end = offset.saturating_add(len).min(fsize);
let mut out = Vec::with_capacity((end - offset) as usize);
// On the revision's own boundaries where they are known, and on the
// 4 MiB fallback where they are not (bugs.md B85).
// 4 MiB fallback where they are not (`docs/BUGS.md` B85).
let wanted = self.block_geometry(uid, mtime, fsize).spans(offset, end);
let (Some(&head), Some(&tail)) = (wanted.first(), wanted.last()) else {
return Ok(out);
@@ -939,28 +961,67 @@ impl Core {
}
if !misses.is_empty() {
let fetched = self.rt.block_on(async {
// Resolve the file's keys and block table once, then read every
// missing block through the shared reader. Previously each block
// called `download_range`, which redid that resolution per block.
let reader = self.revision_reader(uid, mtime, fsize).await?;
// One deadline for the whole read, including any wait for a lost
// link. A fetch that failed because the link went down is tried
// once more when it is back, so a blip is a slow read, not an `EIO`.
let deadline = Instant::now() + READ_FETCH_TIMEOUT;
let mut retried = false;
let fetched = loop {
if !self.is_online() {
let wait =
READ_OFFLINE_WAIT.min(deadline.saturating_duration_since(Instant::now()));
if !self.wait_online(wait) {
debug!(%uid, offset, len, "read of uncached blocks while offline; failing it");
return Err(Errno::EIO);
}
}
let fetch = async {
// Resolve the file's keys and block table once, then read every
// missing block through the shared reader. Previously each block
// called `download_range`, which redid that resolution per block.
let reader = self.revision_reader(uid, mtime, fsize).await?;
let mut set = tokio::task::JoinSet::new();
for &span in &misses {
let (core, reader, uid) = (self.clone(), reader.clone(), uid.clone());
set.spawn(async move {
core.fetch_block(&reader, &uid, mtime, fsize, span, cache_blocks)
.await
.map(|bytes| (span.idx, bytes))
});
let mut set = tokio::task::JoinSet::new();
for &span in &misses {
let (core, reader, uid) = (self.clone(), reader.clone(), uid.clone());
set.spawn(async move {
core.fetch_block(&reader, &uid, mtime, fsize, span, cache_blocks)
.await
.map(|bytes| (span.idx, bytes))
});
}
let mut out = Vec::with_capacity(misses.len());
while let Some(joined) = set.join_next().await {
// A join error means the task panicked; surface it as EIO.
out.push(joined.map_err(|_| Errno::EIO)??);
}
Ok::<_, Errno>(out)
};
// Dropping the fetch on the deadline aborts the block tasks with
// it; blocks that already landed stay cached for the retry. The
// timer is built inside the runtime: a FUSE worker has no reactor
// of its own, and building it out here panicked every remote read.
let budget = deadline.saturating_duration_since(Instant::now());
match self
.rt
.block_on(async { tokio::time::timeout(budget, fetch).await })
{
Ok(Ok(fetched)) => break fetched,
// The fetch saw the link go down; the top of the loop waits
// for it within what is left of the deadline.
Ok(Err(_)) if !retried && !self.is_online() => retried = true,
Ok(Err(e)) => return Err(e),
Err(_) => {
warn!(
%uid, offset, len, secs = READ_FETCH_TIMEOUT.as_secs(),
"read got no answer from the network in time; failing it"
);
self.evict_reader(uid);
self.mark_offline("read", &crate::link::no_answer(READ_FETCH_TIMEOUT));
return Err(Errno::EIO);
}
}
let mut out = Vec::with_capacity(misses.len());
while let Some(joined) = set.join_next().await {
// A join error means the task panicked; surface it as EIO.
out.push(joined.map_err(|_| Errno::EIO)??);
}
Ok::<_, Errno>(out)
})?;
};
for (bidx, bytes) in fetched {
blocks[(bidx - first) as usize] = Some(bytes);
}
@@ -977,7 +1038,7 @@ impl Core {
// — the cache validates it on read, the network path in `read_block`
// — so this refuses rather than quietly contributing fewer bytes
// than the range covers, which is what turned a bad block into a
// truncated file (bugs.md B84).
// truncated file (`docs/BUGS.md` B84).
if block.len() as u64 != span.len {
error!(
%uid, bidx = span.idx, len = block.len(), expected = span.len, fsize,
+1 -4
View File
@@ -248,10 +248,7 @@ impl Core {
let outcomes = self
.rt
.block_on(
self.client
.trash_nodes(std::slice::from_ref(&candidate.uid)),
)
.block_on(self.drive.trash_nodes(std::slice::from_ref(&candidate.uid)))
.map_err(|e| CoreError::from_api(&e, "trash the mis-dated original"))?;
if let Some((_, Err(e))) = outcomes.into_iter().next() {
return Err(CoreError::from_api(&e, "trash the mis-dated original"));
+611
View File
@@ -0,0 +1,611 @@
//! Moving a file or folder between two locations without moving its content.
//!
//! `~/ProtonDrive` and every on-demand synced folder are separate FUSE
//! sessions, and a mirror folder is a plain local directory. The kernel refuses
//! `rename(2)` between two mounts with `EXDEV` before the daemon hears of it, so
//! `mv` falls back to copy and delete: every file is downloaded, uploaded again,
//! and the original is trashed.
//!
//! On Proton Drive the same move is one metadata change, because My files and
//! this computer's device folders all live on the main volume. So a move
//! request that names two absolute paths is resolved here, against every
//! location this daemon knows, and done as a server-side move. The local side
//! is then brought level: a mount forgets the node and re-lists, and a mirror
//! folder either renames its copy along or drops it once Drive has it.
use std::collections::HashMap;
use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
use parking_lot::Mutex;
use pdfs_core::db::{StoredSyncEntry, StoredSyncFolder};
use pdfs_core::{CoreError, CoreResult};
use proton_drive_rs::NodeKind;
use proton_drive_rs::proton_sdk::ids::NodeUid;
use tracing::{info, warn};
use super::sync::mirror_subtree_unsynced;
use super::{Core, is_local_uid, parse_uid};
/// How long a move waits for a mirror folder's sync pass to finish before it
/// reports the folder busy. A pass can run for hours on a first sync, so the
/// move does not wait it out.
const MIRROR_LOCK_WAIT: Duration = Duration::from_secs(5);
/// Where one side of a move lives.
enum Place {
/// My files or an on-demand folder: a live FUSE mount. `core` is rooted at
/// that mount and `rel` is relative to its mountpoint.
Mounted { core: Box<Core>, rel: PathBuf },
/// A mirror folder: a local directory kept in step by the sync engine.
/// `rel` is `/`-separated, relative to the folder, as the baseline keys it.
Mirror {
folder: StoredSyncFolder,
rel: String,
},
}
impl Place {
fn mirror_id(&self) -> Option<i64> {
match self {
Place::Mirror { folder, .. } => Some(folder.id),
Place::Mounted { .. } => None,
}
}
fn is_root(&self) -> bool {
match self {
Place::Mounted { rel, .. } => rel.as_os_str().is_empty(),
Place::Mirror { rel, .. } => rel.is_empty(),
}
}
}
/// The mirror folder that holds `abs`, and `abs` relative to it. The deepest
/// folder wins, although folders are not expected to nest.
fn mirror_covering(
folders: Vec<StoredSyncFolder>,
abs: &Path,
) -> Option<(StoredSyncFolder, String)> {
let folder = folders
.into_iter()
.filter(|f| f.mode == "mirror" && abs.starts_with(&f.local_path))
.max_by_key(|f| Path::new(&f.local_path).components().count())?;
let rel = abs.strip_prefix(&folder.local_path).ok()?;
let parts = rel
.components()
.map(|c| match c {
Component::Normal(part) => part.to_str(),
_ => None,
})
.collect::<Option<Vec<_>>>()?;
Some((folder, parts.join("/")))
}
/// `name` inside the folder at `parent`, in baseline form.
fn join_rel(parent: &str, name: &str) -> String {
if parent.is_empty() {
name.to_string()
} else {
format!("{parent}/{name}")
}
}
/// Rename `from` to `to`, failing with `EEXIST` rather than replacing
/// whatever appeared at `to` since it was checked. A filesystem that cannot
/// promise that gets a plain rename after one more look.
fn rename_noreplace(from: &Path, to: &Path) -> std::io::Result<()> {
use std::ffi::CString;
use std::os::unix::ffi::OsStrExt;
let cstr = |p: &Path| {
CString::new(p.as_os_str().as_bytes())
.map_err(|_| std::io::Error::from_raw_os_error(libc::EINVAL))
};
let (c_from, c_to) = (cstr(from)?, cstr(to)?);
// SAFETY: both are valid NUL-terminated paths that outlive the call.
let rc = unsafe {
libc::syscall(
libc::SYS_renameat2,
libc::AT_FDCWD,
c_from.as_ptr(),
libc::AT_FDCWD,
c_to.as_ptr(),
libc::RENAME_NOREPLACE,
)
};
if rc == 0 {
return Ok(());
}
let err = std::io::Error::last_os_error();
match err.raw_os_error() {
Some(libc::EINVAL | libc::ENOSYS) => {
if std::fs::symlink_metadata(to).is_ok() {
return Err(std::io::Error::from_raw_os_error(libc::EEXIST));
}
std::fs::rename(from, to)
}
_ => Err(err),
}
}
/// Whether `uid` is `root` or lies under it, as far as the nodes table knows.
fn under(core: &Core, root: &NodeUid, uid: &str) -> bool {
let root = root.to_string();
uid == root || matches!(core.db.path_relative_to(&root, uid), Ok(Some(_)))
}
impl Core {
/// Move the file or folder at `src` into the folder at `dest_parent`. Both
/// are absolute local paths and may be in different locations: My files,
/// an on-demand folder, or a mirror folder. Returns the moved name.
///
/// Within one mount this is [`Core::move_to`]. Across locations it
/// is a server-side move followed by local bookkeeping; see the module
/// documentation. Anything the move could lose is refused up front.
pub(crate) fn move_between(&self, src: &Path, dest_parent: &Path) -> CoreResult<String> {
let source = self.locate(src)?;
let dest = self.locate(dest_parent)?;
if let (
Place::Mounted { core, rel },
Place::Mounted {
core: dest_core,
rel: dest_rel,
},
) = (&source, &dest)
&& Arc::ptr_eq(&core.root, &dest_core.root)
{
return core.move_to(rel, dest_rel);
}
if source.is_root() {
return Err(CoreError::invalid(format!(
"{} is the root of a synced location and cannot be moved",
src.display()
)));
}
if dest_parent.starts_with(src) {
return Err(CoreError::invalid("cannot move a folder into itself"));
}
let name = src
.file_name()
.and_then(|n| n.to_str())
.ok_or_else(|| CoreError::invalid(format!("{} has no usable name", src.display())))?
.to_string();
// Hold every mirror folder involved, lowest id first, so no sync pass
// sees the move half done and two moves cannot deadlock.
let mut ids: Vec<i64> = [source.mirror_id(), dest.mirror_id()]
.into_iter()
.flatten()
.collect();
ids.sort_unstable();
ids.dedup();
let locks: Vec<Arc<Mutex<()>>> = ids.iter().map(|&id| self.sync_lock(id)).collect();
let mut guards = Vec::with_capacity(locks.len());
for lock in &locks {
guards.push(lock.try_lock_for(MIRROR_LOCK_WAIT).ok_or_else(|| {
CoreError::conflict("a synced folder is busy syncing; try again when it is done")
})?);
}
let mut baselines: HashMap<i64, HashMap<String, StoredSyncEntry>> = HashMap::new();
for &id in &ids {
let baseline = self
.db
.sync_entries(id)
.map_err(|e| CoreError::internal(format!("db: {e}")))?;
baselines.insert(id, baseline);
}
let src_uid = self.source_uid(&source, &baselines)?;
let dest_uid = self.dest_uid(&dest, &baselines, &name)?;
if src_uid.volume_id != dest_uid.volume_id {
return Err(CoreError::invalid(
"the two locations are on different Proton Drive volumes; copy it instead",
));
}
if under(self, &src_uid, &dest_uid.to_string()) {
return Err(CoreError::invalid("cannot move a folder into itself"));
}
// Mirror to mirror renames the local copy along, so it needs nothing
// proven about it. Another filesystem makes that impossible, and the
// pair then drops the source copy like any other move out of a mirror.
let mut renamed = None;
if let (
Place::Mirror { folder, rel },
Place::Mirror {
folder: dest_folder,
rel: dest_rel,
},
) = (&source, &dest)
{
let from = src.to_path_buf();
let to = dest_parent.join(&name);
match rename_noreplace(&from, &to) {
Ok(()) => renamed = Some((from, to)),
Err(e) if e.raw_os_error() == Some(libc::EXDEV) => {
info!(from = %folder.local_path, to = %dest_folder.local_path,
"move: mirror folders are on different filesystems");
}
Err(e) if e.raw_os_error() == Some(libc::EEXIST) => {
return Err(CoreError::conflict(format!(
"{name} already exists in {}",
dest_parent.display()
)));
}
Err(e) => {
return Err(CoreError::internal(format!(
"could not move {} to {}: {e}",
rel,
join_rel(dest_rel, &name)
)));
}
}
}
if renamed.is_none()
&& let Place::Mirror { folder, rel } = &source
&& let Some(reason) =
mirror_subtree_unsynced(Path::new(&folder.local_path), rel, &baselines[&folder.id])
{
return Err(CoreError::conflict(format!(
"{name} is not fully synced yet ({reason}); wait for the sync to finish"
)));
}
if let Err(e) = self.move_remote(&src_uid, &dest_uid) {
if let Some((from, to)) = &renamed
&& let Err(back) = std::fs::rename(to, from)
{
warn!(from = %to.display(), to = %from.display(), error = %back,
"move: could not put the local copy back after the move failed");
}
return Err(CoreError::from_api(&e, "move"));
}
self.settle_source(
&source,
&src_uid,
renamed.is_some(),
&dest,
&dest_uid,
&name,
src,
);
self.settle_dest(&dest, &dest_uid, &name);
drop(guards);
if renamed.is_none()
&& let Place::Mirror { folder, .. } = &dest
{
// Nothing local exists yet; the folder's next pass downloads it.
self.sync_now(Some(folder.id));
}
Ok(name)
}
/// Which location `abs` is in. A mount is looked for first: a mirror folder
/// cannot live inside a FUSE mount, and a mount is what the kernel sees.
fn locate(&self, abs: &Path) -> CoreResult<Place> {
if let Some((core, rel)) = self.rooted_at(abs) {
if !rel
.components()
.all(|c| matches!(c, Component::Normal(_) | Component::CurDir))
{
return Err(CoreError::invalid(format!(
"{} escapes its location",
abs.display()
)));
}
return Ok(Place::Mounted {
core: Box::new(core),
rel,
});
}
let folders = self
.db
.sync_folder_list()
.map_err(|e| CoreError::internal(format!("db: {e}")))?;
mirror_covering(folders, abs)
.map(|(folder, rel)| Place::Mirror { folder, rel })
.ok_or_else(|| {
CoreError::invalid(format!(
"{} is not in a Proton Drive location",
abs.display()
))
})
}
/// The node being moved, once everything that must hold before it leaves
/// its location has been checked.
fn source_uid(
&self,
source: &Place,
baselines: &HashMap<i64, HashMap<String, StoredSyncEntry>>,
) -> CoreResult<NodeUid> {
match source {
Place::Mounted { core, rel } => {
let (ino, uid) = core.resolve(rel)?;
if is_local_uid(&uid) {
return Err(CoreError::conflict(
"it is not uploaded yet; wait for the upload to finish",
));
}
let old_parent = core.source_parent_uid(ino, rel)?;
for authority in [&uid, &old_parent] {
core.require_uid_writable(authority)
.map_err(|e| core.errno_error(e, "move access"))?;
}
if self.has_pending_work_under(&uid)? {
return Err(CoreError::conflict(
"it has changes that are not uploaded yet; wait for the upload to finish",
));
}
Ok(uid)
}
Place::Mirror { folder, rel } => baselines[&folder.id]
.get(rel)
.and_then(|entry| entry.remote_uid.as_deref())
.and_then(parse_uid)
.ok_or_else(|| {
CoreError::conflict(
"it is not on Proton Drive yet; wait for the sync to finish",
)
}),
}
}
/// The folder the node moves into. Refuses one that already holds `name`
/// on this disk; Drive refuses a clash on its side by itself.
fn dest_uid(
&self,
dest: &Place,
baselines: &HashMap<i64, HashMap<String, StoredSyncEntry>>,
name: &str,
) -> CoreResult<NodeUid> {
let uid = match dest {
Place::Mounted { core, rel } => {
let (ino, uid) = core.resolve(rel)?;
let is_folder = core
.state()
.entries
.get(&ino)
.is_some_and(|entry| matches!(entry.node.kind, NodeKind::Folder));
if !is_folder {
return Err(CoreError::invalid("the destination is not a folder"));
}
core.require_uid_writable(&uid)
.map_err(|e| core.errno_error(e, "move access"))?;
uid
}
Place::Mirror { folder, rel } => {
let local = Path::new(&folder.local_path).join(rel);
if !local.is_dir() {
return Err(CoreError::invalid("the destination is not a folder"));
}
if std::fs::symlink_metadata(local.join(name)).is_ok() {
return Err(CoreError::conflict(format!(
"{name} already exists in the destination"
)));
}
let uid = if rel.is_empty() {
Some(folder.remote_uid.as_str())
} else {
baselines[&folder.id]
.get(rel)
.and_then(|entry| entry.remote_uid.as_deref())
};
uid.and_then(parse_uid).ok_or_else(|| {
CoreError::conflict(
"the destination is not on Proton Drive yet; wait for the sync to finish",
)
})?
}
};
if is_local_uid(&uid) {
return Err(CoreError::conflict(
"the destination is not uploaded yet; wait for the upload to finish",
));
}
Ok(uid)
}
/// Whether the queue owes Drive anything for `uid` or a node under it, or a
/// file under it is open for writing. Moving such a node away would strand
/// the pending change against a path that no longer leads to it.
fn has_pending_work_under(&self, uid: &NodeUid) -> CoreResult<bool> {
let ops = self
.db
.pending_ops()
.map_err(|e| CoreError::internal(format!("db: {e}")))?;
if ops.iter().any(|op| {
under(self, uid, &op.uid)
|| op
.parent_uid
.as_deref()
.is_some_and(|p| under(self, uid, p))
}) {
return Ok(true);
}
// Collected first so no State lock is held across the nodes-table reads.
let mut writing = Vec::new();
self.for_each_state(|st| {
writing.extend(st.active_writes.values().map(|w| w.uid.to_string()));
});
Ok(writing.iter().any(|w| under(self, uid, w)))
}
/// Bring the source location level with a move Drive has made.
#[allow(clippy::too_many_arguments)]
fn settle_source(
&self,
source: &Place,
uid: &NodeUid,
renamed: bool,
dest: &Place,
dest_uid: &NodeUid,
name: &str,
src: &Path,
) {
match source {
Place::Mounted { core, rel } => {
// Moved everywhere, so no mount keeps serving it under the old
// path and its row stays (`docs/BUGS.md` B117). The content
// cache is keyed by uid and stays valid.
self.relocate_everywhere(uid, dest_uid, name);
core.invalidate_parent_listing(rel);
}
Place::Mirror { folder, rel } if renamed => {
let Place::Mirror {
folder: dest_folder,
rel: dest_rel,
} = dest
else {
unreachable!("a local rename needs a mirror destination");
};
if let Err(e) = self.db.sync_entries_move(
folder.id,
rel,
dest_folder.id,
&join_rel(dest_rel, name),
) {
// The next passes see a copy with no baseline on one side
// and a baseline with no copy on the other. That uploads
// the files again and trashes the moved node, but loses
// nothing.
warn!(error = ?e, "move: could not carry the sync baseline along");
}
}
Place::Mirror { folder, rel } => {
// Drive had all of it when the move was checked, so the local
// copy is only a stale duplicate now. Something written into it
// since would go with it, so look again first; if anything is
// new, the copy stays and the next pass uploads it as new.
let changed = match self.db.sync_entries(folder.id) {
Ok(baseline) => {
mirror_subtree_unsynced(Path::new(&folder.local_path), rel, &baseline)
}
Err(e) => Some(format!("db: {e}")),
};
if let Err(e) = self.db.sync_entries_remove_subtree(folder.id, rel) {
warn!(error = ?e, "move: could not drop the sync baseline");
}
if let Some(reason) = changed {
warn!(path = %src.display(), reason,
"move: the local copy changed during the move; keeping it");
return;
}
let removed = if src.is_dir() {
std::fs::remove_dir_all(src)
} else {
std::fs::remove_file(src)
};
if let Err(e) = removed {
warn!(path = %src.display(), error = %e,
"move: could not remove the local copy after the move");
}
}
}
}
/// Bring the destination location level with a move Drive has made.
fn settle_dest(&self, dest: &Place, uid: &NodeUid, name: &str) {
if let Place::Mounted { .. } = dest {
self.invalidate_children_of(uid);
// A lookup that missed before the move may be cached as absent.
self.for_each_mount(|st, notify| {
if let Some(&ino) = st.by_uid.get(uid) {
notify.inval_entry(ino, name.to_string());
}
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn folder(id: i64, local_path: &str, mode: &str) -> StoredSyncFolder {
StoredSyncFolder {
id,
local_path: local_path.to_string(),
remote_uid: format!("vol~root{id}"),
remote_share_id: String::new(),
mode: mode.to_string(),
pending_mode: None,
state: "idle".to_string(),
last_sync: 0,
}
}
#[test]
fn mirror_covering_finds_the_mirror_folder_and_the_path_inside_it() {
let folders = vec![folder(1, "/home/u/Docs", "mirror")];
let (found, rel) = mirror_covering(folders, Path::new("/home/u/Docs/a/b.txt")).unwrap();
assert_eq!(found.id, 1);
assert_eq!(rel, "a/b.txt");
}
#[test]
fn mirror_covering_gives_an_empty_path_for_the_folder_itself() {
let folders = vec![folder(1, "/home/u/Docs", "mirror")];
let (_, rel) = mirror_covering(folders, Path::new("/home/u/Docs")).unwrap();
assert_eq!(rel, "");
}
#[test]
fn mirror_covering_prefers_the_deepest_folder() {
let folders = vec![
folder(1, "/home/u/Docs", "mirror"),
folder(2, "/home/u/Docs/Work", "mirror"),
];
let (found, rel) = mirror_covering(folders, Path::new("/home/u/Docs/Work/x")).unwrap();
assert_eq!(found.id, 2);
assert_eq!(rel, "x");
}
#[test]
fn mirror_covering_skips_on_demand_folders_and_name_prefixed_siblings() {
let folders = || {
vec![
folder(1, "/home/u/Docs", "mirror"),
folder(2, "/home/u/Media", "ondemand"),
]
};
assert!(mirror_covering(folders(), Path::new("/home/u/Media/x")).is_none());
assert!(mirror_covering(folders(), Path::new("/home/u/Docs2/x")).is_none());
}
#[test]
fn mirror_covering_refuses_a_path_that_climbs_out() {
let folders = vec![folder(1, "/home/u/Docs", "mirror")];
assert!(mirror_covering(folders, Path::new("/home/u/Docs/../etc")).is_none());
}
#[test]
fn join_rel_names_a_child_of_the_folder_root_without_a_slash() {
assert_eq!(join_rel("", "a"), "a");
assert_eq!(join_rel("x/y", "a"), "x/y/a");
}
#[test]
fn a_local_rename_never_replaces_what_is_already_there() {
let dir = std::env::temp_dir().join(format!("pdfs-relocate-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let (a, b, c) = (dir.join("a"), dir.join("b"), dir.join("c"));
std::fs::write(&a, b"moved").unwrap();
std::fs::write(&b, b"kept").unwrap();
let err = rename_noreplace(&a, &b).unwrap_err();
assert_eq!(err.raw_os_error(), Some(libc::EEXIST));
assert_eq!(std::fs::read(&b).unwrap(), b"kept");
assert!(a.exists());
rename_noreplace(&a, &c).unwrap();
assert!(!a.exists());
assert_eq!(std::fs::read(&c).unwrap(), b"moved");
std::fs::remove_dir_all(&dir).unwrap();
}
}
+1 -1
View File
@@ -232,7 +232,7 @@ impl Core {
let mut out = std::io::BufWriter::new(file);
let written = match self
.rt
.block_on(self.client.download_revision_to(uid, revision_id, &mut out))
.block_on(self.drive.download_revision_to(uid, revision_id, &mut out))
.map_err(|e| CoreError::from_api(&e, "download revision"))
.and_then(|()| {
use std::io::Write as _;
+37 -28
View File
@@ -21,7 +21,7 @@ use proton_drive_rs::{MemberRole, NodeKind};
use proton_drive_rs::Node;
use super::{Core, ROOT_INO, node_size, parse_uid, public_link_info, role_from_str, role_to_str};
use super::{Core, node_size, parse_uid, public_link_info, role_from_str, role_to_str};
/// A node from a *shared* listing as a [`DirEntry`].
///
@@ -73,6 +73,9 @@ fn shared_entry_of(core: &Core, n: Node) -> DirEntry {
shared_by,
shared_at,
shared_by_unverified,
trashed_at: 0,
trashed_from: None,
issue: None,
}
}
@@ -461,7 +464,7 @@ impl Core {
}
let nodes = self
.rt
.block_on(self.client.enumerate_nodes(&uids))
.block_on(self.drive.enumerate_nodes(&uids))
.map_err(|e| CoreError::from_api(&e, "enumerate nodes"))?;
Ok(nodes
.into_iter()
@@ -479,14 +482,14 @@ impl Core {
parse_uid(uid).ok_or_else(|| CoreError::invalid(format!("invalid uid: {uid}")))?;
let child_uids = self
.rt
.block_on(self.client.enumerate_folder_children_node_uids(&uid))
.block_on(self.drive.enumerate_folder_children_node_uids(&uid))
.map_err(|e| CoreError::from_api(&e, "enumerate shared children"))?;
if child_uids.is_empty() {
return Ok(Vec::new());
}
let nodes = self
.rt
.block_on(self.client.enumerate_nodes(&child_uids))
.block_on(self.drive.enumerate_nodes(&child_uids))
.map_err(|e| CoreError::from_api(&e, "enumerate nodes"))?;
Ok(nodes
.into_iter()
@@ -507,7 +510,7 @@ impl Core {
parse_uid(uid).ok_or_else(|| CoreError::invalid(format!("invalid uid: {uid}")))?;
let node = self
.rt
.block_on(self.client.get_node(&uid))
.block_on(self.drive.get_node(&uid))
.map_err(|e| CoreError::from_api(&e, "get shared node"))?
.ok_or_else(|| CoreError::not_found("shared node not found"))?;
if !node.is_file() {
@@ -608,10 +611,10 @@ impl Core {
/// Proton products (not Drive-only). A remote round-trip; nothing here is
/// cached.
pub(crate) fn account_quota(&self) -> CoreResult<(i64, i64)> {
let q = self
.rt
.block_on(self.client.quota())
.map_err(|e| CoreError::from_api(&e, "account quota"))?;
let q = self.block_on_bounded(self.drive.quota()).map_err(|e| {
self.lost_link(&e, "account quota");
CoreError::from_api(&e, "account quota")
})?;
*self.quota.lock() = Some((std::time::Instant::now(), q.max_space, q.used_space));
Ok((q.max_space, q.used_space))
}
@@ -630,11 +633,22 @@ impl Core {
{
return Some((max_space, used_space));
}
match self.account_quota() {
let last = || self.quota.lock().map(|(_, max, used)| (max, used));
if !self.is_online()
|| self
.quota_refreshing
.swap(true, std::sync::atomic::Ordering::AcqRel)
{
return last();
}
let refreshed = self.account_quota();
self.quota_refreshing
.store(false, std::sync::atomic::Ordering::Release);
match refreshed {
Ok(q) => Some(q),
Err(error) => {
debug!(%error, "account quota refresh failed; serving the last known figures");
self.quota.lock().map(|(_, max, used)| (max, used))
last()
}
}
}
@@ -656,7 +670,7 @@ impl Core {
}
let nodes = self
.rt
.block_on(self.client.enumerate_nodes(&uids))
.block_on(self.drive.enumerate_nodes(&uids))
.map_err(|e| CoreError::from_api(&e, "enumerate nodes"))?;
let mut items = Vec::with_capacity(nodes.len());
for n in nodes {
@@ -701,19 +715,10 @@ impl Core {
/// been seen through the mount (e.g. shared but not browsed to this session) —
/// the caller then leaves the path empty.
pub(crate) fn rel_path_for_uid(&self, uid: &NodeUid) -> Option<String> {
// Searched across every mount: a node browsed through a sync folder is
// interned in that fork's inode space, so the primary state alone had no
// entry for it and the caller silently showed an empty path
// (`docs/BUGS.md` B74). The path is relative to whichever mount holds
// it, which is the one the user reached it through.
let mut path = None;
self.for_each_state(|st| {
if path.is_some() {
return;
}
path = walk_to_root(st, uid);
});
path
// Relative to whichever root the node sits under: a node browsed
// through a sync folder is under that fork's root, not My Files
// (`docs/BUGS.md` B74).
walk_to_root(&self.state(), uid)
}
}
@@ -758,14 +763,18 @@ impl StopSharingOutcome {
}
}
/// Walk one inode space from `uid` up to its root, building the path. `None` if
/// that mount has never interned the node, or if the chain breaks partway —
/// Walk the tree from `uid` up to the root it sits under, building the path.
/// `None` if the node has never been interned, or if the chain breaks partway —
/// a broken chain is a missing path, never a partial one.
fn walk_to_root(st: &super::State, uid: &NodeUid) -> Option<String> {
let mut ino = *st.by_uid.get(uid)?;
let mut parts = Vec::new();
while ino != ROOT_INO {
while !st.roots.contains(&ino) {
let entry = st.entries.get(&ino)?;
// An orphan is its own parent without being a root.
if entry.parent == ino {
return None;
}
parts.push(entry.node.name.clone());
ino = entry.parent;
}
+1 -1
View File
@@ -6,7 +6,7 @@
//! invisible; in an in-process remount — which is how the daemon recovers from a
//! mount going away underneath it — it leaks a whole generation of threads that
//! keep writing to the database and calling the API on behalf of a mount that no
//! longer exists (bugs.md B44).
//! longer exists (`docs/BUGS.md` B44).
//!
//! The primitive is deliberately small: a flag plus a condvar. Every loop that
//! would otherwise call `thread::sleep` calls [`Shutdown::sleep`] instead, which
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+9
View File
@@ -0,0 +1,9 @@
//! The simulation tests of `docs/TESTING.md`: a fake Drive with
//! the faults the real one has, and the tools to drive daemons against it.
pub(crate) mod daemon;
pub(crate) mod fake_drive;
pub(crate) mod model;
pub(crate) mod rng;
pub(crate) mod run;
pub(crate) mod watchdog;
+445
View File
@@ -0,0 +1,445 @@
//! The reference file system of the simulation (`docs/TESTING.md`,
//! "Simulation runs", property 1): what each syscall a simulated client makes should
//! answer, and the tree it should leave, on a POSIX file system.
//!
//! A [`Model`] covers one client's own folder; paths are relative to it. The
//! same type is used for the trees read back from a mount and from Drive.
use std::collections::BTreeMap;
use super::rng::Rng;
/// A tree by path: `None` is a folder, `Some` a file's content.
pub(crate) type Tree = BTreeMap<String, Option<Vec<u8>>>;
/// One file system call, or the few a program makes in a row.
#[derive(Clone, Debug)]
pub(crate) enum FsOp {
/// `open(O_CREAT | O_TRUNC)`, write `data`, close.
Create {
path: String,
data: Vec<u8>,
},
/// Open an existing file, write `data` at `offset` (past the end leaves
/// a hole), close.
Write {
path: String,
offset: u64,
data: Vec<u8>,
},
/// Open an existing file and `ftruncate` it to `len`.
Truncate {
path: String,
len: u64,
},
Mkdir {
path: String,
},
Rename {
from: String,
to: String,
},
Unlink {
path: String,
},
/// Open a file, unlink it, append `data`, read it all back, close.
UnlinkOpen {
path: String,
data: Vec<u8>,
},
Rmdir {
path: String,
},
}
impl FsOp {
/// The paths the op names, for the log.
pub(crate) fn describe(&self) -> String {
match self {
FsOp::Create { path, data } => format!("create {path} ({} bytes)", data.len()),
FsOp::Write { path, offset, data } => {
format!("write {path} at {offset} ({} bytes)", data.len())
}
FsOp::Truncate { path, len } => format!("truncate {path} to {len}"),
FsOp::Mkdir { path } => format!("mkdir {path}"),
FsOp::Rename { from, to } => format!("rename {from} -> {to}"),
FsOp::Unlink { path } => format!("unlink {path}"),
FsOp::UnlinkOpen { path, data } => {
format!("unlink {path} while open, then append {} bytes", data.len())
}
FsOp::Rmdir { path } => format!("rmdir {path}"),
}
}
}
/// The tree one client should have, and what its syscalls should answer.
#[derive(Clone, Debug, Default)]
pub(crate) struct Model {
tree: Tree,
}
/// Names are drawn from small pools, so ops collide: a create over an existing
/// file, a rename over another, a mkdir of a name that is taken.
const FILE_NAMES: [&str; 6] = ["a.txt", "b.txt", "c.bin", "d", "e.md", "f.txt"];
const DIR_NAMES: [&str; 3] = ["x", "y", "z"];
impl Model {
pub(crate) fn tree(&self) -> &Tree {
&self.tree
}
pub(crate) fn content(&self, path: &str) -> Option<&Vec<u8>> {
self.tree.get(path)?.as_ref()
}
/// What `op` should answer on a POSIX file system, applying it on success.
pub(crate) fn apply(&mut self, op: &FsOp) -> Result<(), i32> {
match op {
FsOp::Create { path, data } => {
self.check_parent(path)?;
match self.tree.get(path) {
Some(None) => Err(libc::EISDIR),
_ => {
self.tree.insert(path.clone(), Some(data.clone()));
Ok(())
}
}
}
FsOp::Write { path, offset, data } => {
let content = self.file_mut(path)?;
// Writing nothing changes nothing, even past the end.
if data.is_empty() {
return Ok(());
}
let end = *offset as usize + data.len();
if content.len() < end {
content.resize(end, 0);
}
content[*offset as usize..end].copy_from_slice(data);
Ok(())
}
FsOp::Truncate { path, len } => {
self.file_mut(path)?.resize(*len as usize, 0);
Ok(())
}
FsOp::Mkdir { path } => {
self.check_parent(path)?;
if self.tree.contains_key(path) {
return Err(libc::EEXIST);
}
self.tree.insert(path.clone(), None);
Ok(())
}
FsOp::Rename { from, to } => self.rename(from, to),
FsOp::Unlink { path } | FsOp::UnlinkOpen { path, .. } => {
self.check_parent(path)?;
match self.tree.get(path) {
None => Err(libc::ENOENT),
Some(None) => Err(libc::EISDIR),
Some(Some(_)) => {
self.tree.remove(path);
Ok(())
}
}
}
FsOp::Rmdir { path } => {
self.check_parent(path)?;
match self.tree.get(path) {
None => Err(libc::ENOENT),
Some(Some(_)) => Err(libc::ENOTDIR),
Some(None) if self.has_children(path) => Err(libc::ENOTEMPTY),
Some(None) => {
self.tree.remove(path);
Ok(())
}
}
}
}
}
fn rename(&mut self, from: &str, to: &str) -> Result<(), i32> {
self.check_parent(from)?;
let Some(moving) = self.tree.get(from).cloned() else {
return Err(libc::ENOENT);
};
self.check_parent(to)?;
if from == to {
return Ok(());
}
if is_below(to, from) {
return Err(libc::EINVAL);
}
match (&moving, self.tree.get(to)) {
(None, Some(Some(_))) => return Err(libc::ENOTDIR),
(Some(_), Some(None)) => return Err(libc::EISDIR),
(None, Some(None)) if self.has_children(to) => return Err(libc::ENOTEMPTY),
_ => {}
}
let below: Vec<(String, Option<Vec<u8>>)> = self
.tree
.iter()
.filter(|(path, _)| is_below(path, from))
.map(|(path, entry)| (path.clone(), entry.clone()))
.collect();
self.tree.remove(from);
self.tree.insert(to.to_owned(), moving);
for (path, entry) in below {
self.tree.remove(&path);
self.tree
.insert(format!("{to}{}", &path[from.len()..]), entry);
}
Ok(())
}
fn file_mut(&mut self, path: &str) -> Result<&mut Vec<u8>, i32> {
self.check_parent(path)?;
match self.tree.get_mut(path) {
None => Err(libc::ENOENT),
Some(None) => Err(libc::EISDIR),
Some(Some(content)) => Ok(content),
}
}
fn check_parent(&self, path: &str) -> Result<(), i32> {
let mut at = String::new();
let mut names = path.split('/').peekable();
while let Some(name) = names.next() {
if names.peek().is_none() {
break;
}
if !at.is_empty() {
at.push('/');
}
at.push_str(name);
match self.tree.get(&at) {
None => return Err(libc::ENOENT),
Some(Some(_)) => return Err(libc::ENOTDIR),
Some(None) => {}
}
}
Ok(())
}
fn has_children(&self, path: &str) -> bool {
self.tree.keys().any(|other| is_below(other, path))
}
pub(crate) fn files(&self) -> Vec<&String> {
self.tree
.iter()
.filter(|(_, entry)| entry.is_some())
.map(|(path, _)| path)
.collect()
}
/// Folders, the client's own root (`""`) first.
fn dirs(&self) -> Vec<String> {
std::iter::once(String::new())
.chain(
self.tree
.iter()
.filter(|(_, entry)| entry.is_none())
.map(|(path, _)| path.clone()),
)
.collect()
}
/// A random op, mostly one that should succeed.
pub(crate) fn next_op(&self, rng: &mut Rng) -> FsOp {
let dirs = self.dirs();
let files = self.files();
let in_dir = |rng: &mut Rng, names: &[&str]| {
let parent = rng.pick(&dirs).expect("the root is always there");
join(parent, rng.pick(names).expect("names"))
};
let a_file = |rng: &mut Rng| rng.pick(&files).map(|path| (*path).clone());
let roll = rng.below(100);
let op = match roll {
0..30 => Some(FsOp::Create {
path: in_dir(rng, &FILE_NAMES),
data: data(rng),
}),
30..45 => a_file(rng).map(|path| {
let len = self.content(&path).map_or(0, Vec::len) as u64;
FsOp::Write {
offset: rng.below(len + 8),
path,
data: data(rng),
}
}),
45..50 => a_file(rng).map(|path| FsOp::Truncate {
len: rng.below(32),
path,
}),
50..60 => Some(FsOp::Mkdir {
path: in_dir(rng, &DIR_NAMES),
}),
// A rename in place, over another file as often as not.
60..72 => a_file(rng).map(|from| FsOp::Rename {
to: join(parent_of(&from), rng.pick(&FILE_NAMES).expect("names")),
from,
}),
// A move to another folder, renamed or not.
72..82 => a_file(rng).map(|from| {
let name = if rng.chance(0.5) {
name_of(&from)
} else {
rng.pick(&FILE_NAMES).expect("names")
};
let parent = rng.pick(&dirs).expect("the root is always there");
FsOp::Rename {
to: join(parent, name),
from,
}
}),
// A folder renamed or moved, with what is in it.
82..86 => {
let from = rng.pick(&dirs[1..]).cloned();
from.map(|from| FsOp::Rename {
to: in_dir(rng, &DIR_NAMES),
from,
})
}
86..94 => a_file(rng).map(|path| FsOp::Unlink { path }),
94..97 => a_file(rng).map(|path| FsOp::UnlinkOpen {
path,
data: data(rng),
}),
_ => rng
.pick(&dirs[1..])
.cloned()
.map(|path| FsOp::Rmdir { path }),
};
op.unwrap_or_else(|| FsOp::Create {
path: in_dir(rng, &FILE_NAMES),
data: data(rng),
})
}
}
/// Content for a write: short and recognisable, now and then empty.
pub(crate) fn data(rng: &mut Rng) -> Vec<u8> {
let len = if rng.chance(0.1) {
0
} else {
rng.between(1, 48)
};
let tag = rng.next_u64();
(0..len).map(|i| b'a' + ((tag + i) % 26) as u8).collect()
}
pub(crate) fn join(parent: &str, name: &str) -> String {
if parent.is_empty() {
name.to_owned()
} else {
format!("{parent}/{name}")
}
}
pub(crate) fn parent_of(path: &str) -> &str {
path.rsplit_once('/').map_or("", |(parent, _)| parent)
}
pub(crate) fn name_of(path: &str) -> &str {
path.rsplit_once('/').map_or(path, |(_, name)| name)
}
fn is_below(path: &str, ancestor: &str) -> bool {
path.len() > ancestor.len() + 1
&& path.starts_with(ancestor)
&& path.as_bytes()[ancestor.len()] == b'/'
}
#[cfg(test)]
mod tests {
use super::*;
fn create(path: &str, data: &[u8]) -> FsOp {
FsOp::Create {
path: path.into(),
data: data.to_vec(),
}
}
#[test]
fn a_rename_over_a_file_replaces_it_and_a_folder_moves_with_its_contents() {
let mut model = Model::default();
model.apply(&FsOp::Mkdir { path: "x".into() }).unwrap();
model.apply(&create("x/a", b"1")).unwrap();
model.apply(&create("b", b"2")).unwrap();
model
.apply(&FsOp::Rename {
from: "b".into(),
to: "x/a".into(),
})
.unwrap();
assert_eq!(model.content("x/a"), Some(&b"2".to_vec()));
model
.apply(&FsOp::Rename {
from: "x".into(),
to: "y".into(),
})
.unwrap();
assert_eq!(model.tree().keys().collect::<Vec<_>>(), vec!["y", "y/a"]);
}
#[test]
fn calls_fail_the_way_posix_says() {
let mut model = Model::default();
model.apply(&FsOp::Mkdir { path: "x".into() }).unwrap();
model.apply(&create("x/a", b"1")).unwrap();
assert_eq!(
model.apply(&FsOp::Mkdir { path: "x".into() }),
Err(libc::EEXIST)
);
assert_eq!(model.apply(&create("x", b"")), Err(libc::EISDIR));
assert_eq!(model.apply(&create("x/a/b", b"")), Err(libc::ENOTDIR));
assert_eq!(model.apply(&create("q/a", b"")), Err(libc::ENOENT));
assert_eq!(
model.apply(&FsOp::Rmdir { path: "x".into() }),
Err(libc::ENOTEMPTY)
);
assert_eq!(
model.apply(&FsOp::Unlink { path: "x".into() }),
Err(libc::EISDIR)
);
let into_itself = FsOp::Rename {
from: "x".into(),
to: "x/y".into(),
};
assert_eq!(model.apply(&into_itself), Err(libc::EINVAL));
}
#[test]
fn a_write_past_the_end_leaves_a_hole() {
let mut model = Model::default();
model.apply(&create("a", b"ab")).unwrap();
let write = FsOp::Write {
path: "a".into(),
offset: 4,
data: b"z".to_vec(),
};
model.apply(&write).unwrap();
assert_eq!(model.content("a"), Some(&b"ab\0\0z".to_vec()));
let nothing = FsOp::Write {
path: "a".into(),
offset: 9,
data: Vec::new(),
};
model.apply(&nothing).unwrap();
assert_eq!(model.content("a"), Some(&b"ab\0\0z".to_vec()));
}
#[test]
fn generated_ops_mostly_succeed() {
let mut rng = Rng::new(9);
let mut model = Model::default();
let ok = (0..500)
.filter(|_| {
let op = model.next_op(&mut rng);
model.apply(&op).is_ok()
})
.count();
assert!(ok > 350, "{ok} of 500 succeeded");
}
}
+79
View File
@@ -0,0 +1,79 @@
//! A small seeded random number generator, so a failing simulation is a seed
//! that can be replayed. SplitMix64: fast, good enough for choosing operations
//! and faults, and no dependency.
#[derive(Clone, Debug)]
pub(crate) struct Rng(u64);
impl Rng {
pub(crate) fn new(seed: u64) -> Self {
Self(seed)
}
pub(crate) fn next_u64(&mut self) -> u64 {
self.0 = self.0.wrapping_add(0x9E37_79B9_7F4A_7C15);
let mut z = self.0;
z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9);
z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB);
z ^ (z >> 31)
}
/// A value in `0..n`; `0` when `n` is `0`.
pub(crate) fn below(&mut self, n: u64) -> u64 {
if n == 0 { 0 } else { self.next_u64() % n }
}
/// A value in `lo..=hi`.
pub(crate) fn between(&mut self, lo: u64, hi: u64) -> u64 {
lo + self.below(hi.saturating_sub(lo) + 1)
}
/// True with probability `p`.
pub(crate) fn chance(&mut self, p: f64) -> bool {
p > 0.0 && (self.next_u64() >> 11) as f64 / (1u64 << 53) as f64 <= p
}
pub(crate) fn pick<'a, T>(&mut self, items: &'a [T]) -> Option<&'a T> {
items.get(self.below(items.len() as u64) as usize)
}
/// A child generator, so one consumer's draws do not shift another's.
pub(crate) fn fork(&mut self) -> Self {
Self::new(self.next_u64())
}
}
#[cfg(test)]
mod tests {
use super::Rng;
#[test]
fn a_seed_replays_the_same_sequence() {
let (mut a, mut b) = (Rng::new(7), Rng::new(7));
for _ in 0..100 {
assert_eq!(a.next_u64(), b.next_u64());
}
assert_ne!(Rng::new(7).next_u64(), Rng::new(8).next_u64());
}
#[test]
fn bounded_draws_stay_in_range() {
let mut rng = Rng::new(1);
for _ in 0..1000 {
assert!(rng.below(5) < 5);
assert!((3..=9).contains(&rng.between(3, 9)));
}
assert_eq!(rng.below(0), 0);
assert!(!rng.chance(0.0));
assert!(rng.chance(1.0));
assert!([1, 2, 3].contains(rng.pick(&[1, 2, 3]).unwrap()));
assert_eq!(rng.pick::<u8>(&[]), None);
}
#[test]
fn a_fork_draws_apart_from_its_parent() {
let mut parent = Rng::new(3);
let mut child = parent.fork();
assert_ne!(parent.next_u64(), child.next_u64());
}
}
+766
View File
@@ -0,0 +1,766 @@
//! The simulation runs described in `docs/TESTING.md`.
//!
//! A run starts one or more daemons on a shared [`FakeDrive`], each owning a
//! folder `c<n>` that only it writes, so there is one writer per file. A seed
//! draws a sequence of steps: syscalls through a client's mount, the link of a
//! client going down or up, a client restarting, a pause, a settle (every
//! queue must drain within a budget), or a file deleted while the drain holds
//! a write to it. Every syscall is checked against the
//! client's [`Model`] as it happens. After the sequence every link comes up
//! and the run checks:
//!
//! 1. **No loss** and **POSIX**: Drive's copy of each client's folder is what
//! its model says.
//! 2. **No false conflicts**: no conflict copy anywhere, and the drain never
//! takes a client's own delete for another device's.
//! 3. **Convergence**: each mount shows Drive's tree.
//! 4. **Liveness**: no syscall took longer than its budget, settles drained in
//! theirs, and every queue drained once the links were up. A call that
//! never returns is caught by the [`Watchdog`], which keeps the stacks.
//!
//! A restart here is a stop and a start: the next run finds what the last
//! one wrote down, but in-flight work is let finish rather than cut. A kill
//! at an arbitrary point needs the daemon in a process of its own.
//!
//! Any of these fails the seed. No open bug is tolerated: the matchers that
//! counted the ones still open instead (B125 to B134) went once a run on
//! 812eb3d passed every profile without them.
//!
//! The runs are ignored by default because they mount FUSE; run them with
//! `cargo test -p pdfs-fuse --lib sim::run -- --ignored --test-threads=1`.
//! `PDFS_SIM_SEEDS=n` sets how many seeds each runs, `PDFS_SIM_SEED=s` replays
//! one, and `RUST_LOG=pdfs_fuse=debug` with `--nocapture` shows the daemons'
//! logs.
use std::fs::OpenOptions;
use std::io::{self, Read, Seek, SeekFrom, Write};
use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};
use super::daemon::{Daemon, scratch, take_logged, wait_until};
use super::fake_drive::{Entry, FakeDrive, Faults};
use super::model::{FsOp, Model, Tree, data, join};
use super::rng::Rng;
use super::watchdog::Watchdog;
/// What a run looks like.
pub(crate) struct Profile {
pub(crate) name: &'static str,
pub(crate) clients: usize,
pub(crate) steps: usize,
pub(crate) faults: fn() -> Faults,
/// Whether links go down and come back during the sequence.
pub(crate) link_flaps: bool,
pub(crate) restarts: bool,
/// Whether files are deleted while the drain holds a write to them.
pub(crate) deletes_mid_upload: bool,
/// How long a settle may take; `None` leaves settles out.
pub(crate) settle_budget: Option<Duration>,
/// How long one syscall may take.
pub(crate) syscall_budget: Duration,
/// How long the queues may take to drain at the end.
pub(crate) drain_budget: Duration,
}
/// How long the mounts may take to show Drive's tree at the end: one event
/// poll (`POLL_INTERVAL`) and change.
const CONVERGE_BUDGET: Duration = Duration::from_secs(30);
/// How long the drain may take to pick up a write: its debounce, and then
/// some.
const PICKUP_BUDGET: Duration = Duration::from_secs(8);
/// Steps shown before a failure. All of them are written to `steps.log` in
/// the state the failure keeps.
const LOG_TAIL: usize = 40;
/// How long past its own budget a step, or the checks after the last one, may
/// run before the watchdog takes it for hung. Longer than a restart may wait
/// for the last run and the mount.
const STALL: Duration = Duration::from_secs(120);
struct Client {
daemon: Option<Daemon>,
model: Model,
online: bool,
}
impl Client {
fn daemon(&self) -> &Daemon {
self.daemon
.as_ref()
.expect("a client always has a daemon between steps")
}
fn root(&self, index: usize) -> PathBuf {
self.daemon().mountpoint.join(owned_folder(index))
}
}
struct Run {
seed: u64,
profile: &'static Profile,
drive: std::sync::Arc<FakeDrive>,
clients: Vec<Client>,
log: Vec<String>,
started: Instant,
/// What the daemons logged at `INFO` and above so far.
logged: Vec<String>,
watchdog: Watchdog,
}
/// Run `profile` on `seed`. It answers a report of what went wrong, if
/// anything did.
pub(crate) fn run(seed: u64, profile: &'static Profile) -> Result<(), String> {
let dir = scratch(&format!("{}-{seed}", profile.name));
let mut rng = Rng::new(seed);
let drive = FakeDrive::new();
let mut run = Run {
seed,
profile,
drive: drive.clone(),
clients: Vec::new(),
log: Vec::new(),
started: Instant::now(),
logged: Vec::new(),
watchdog: Watchdog::new(&dir),
};
let outcome = run.execute(&dir, &mut rng);
run.watchdog.stop();
// A hang is what failed, whatever the aborted call answered.
let outcome = match run.watchdog.fired() {
Some(hung) => Err(format!("liveness: {}", hung.what)),
None => outcome,
};
for client in &mut run.clients {
if let Some(daemon) = client.daemon.take() {
let _ = daemon.stop();
}
}
match outcome {
Ok(()) => {
let _ = std::fs::remove_dir_all(&dir);
Ok(())
}
Err(failure) => Err(run.report(&failure, &dir)),
}
}
impl Run {
fn execute(&mut self, dir: &Path, rng: &mut Rng) -> Result<(), String> {
for index in 0..self.profile.clients {
let client = self.drive.client(rng.next_u64(), (self.profile.faults)());
let daemon = Daemon::start(&dir.join(format!("client{index}")), client)
.map_err(|error| format!("client {index} did not mount: {error}"))?;
let folder = daemon.mountpoint.join(owned_folder(index));
std::fs::create_dir(&folder)
.map_err(|error| format!("client {index}: mkdir {}: {error}", folder.display()))?;
self.clients.push(Client {
daemon: Some(daemon),
model: Model::default(),
online: true,
});
}
for step in 0..self.profile.steps {
let _watching = self.watchdog.arm(self.step_limit(), format!("step {step}"));
self.step(rng)?;
}
let _watching = self.watchdog.arm(
self.profile.drain_budget + STALL,
"the checks after the last step",
);
self.finish()
}
/// How long a step may run before the watchdog takes it for hung.
fn step_limit(&self) -> Duration {
let budget = self.profile.settle_budget.unwrap_or_default();
budget.max(self.profile.syscall_budget) + STALL
}
fn step(&mut self, rng: &mut Rng) -> Result<(), String> {
let index = rng.below(self.clients.len() as u64) as usize;
let roll = rng.below(100);
match roll {
0..4 if self.profile.link_flaps => {
let client = &mut self.clients[index];
client.online = !client.online;
client.daemon().client.set_online(client.online);
let state = if client.online { "up" } else { "down" };
self.note(format!("client {index}: link {state}"));
}
4..6 if self.profile.restarts => {
self.note(format!("client {index}: restart"));
let daemon = self.clients[index].daemon.take().expect("daemon");
let daemon = daemon
.restart()
.map_err(|error| format!("client {index} did not come back: {error}"))?;
self.clients[index].daemon = Some(daemon);
}
6..12 => {
let ms = rng.below(400);
self.note(format!("pause {ms} ms"));
std::thread::sleep(Duration::from_millis(ms));
}
12..15 => {
if let Some(budget) = self.profile.settle_budget
&& self.clients.iter().all(|client| client.online)
{
self.settle(budget)?;
}
}
15..25 if self.profile.deletes_mid_upload => self.delete_mid_upload(index, rng)?,
_ => {
let op = self.clients[index].model.next_op(rng);
self.syscall(index, op)?;
}
}
Ok(())
}
/// Wait until every queue is empty and Drive holds what the models say.
fn settle(&mut self, budget: Duration) -> Result<(), String> {
self.note("settle".into());
let start = Instant::now();
// What the drain logs meanwhile is what explains a known bug's damage.
if !wait_until(budget, || {
self.logged.extend(take_logged());
self.landed()
}) {
return Err(format!(
"liveness: the queues did not drain within {budget:?}\n{}",
self.difference()
));
}
self.note(format!("settled in {:?}", start.elapsed()));
Ok(())
}
/// Write a file Drive holds and delete it once the drain has picked the
/// write up. The drain's read of the node is held until the delete has
/// landed and a read shows it, so the drain finds the file trashed while
/// it holds a write that is no longer queued (B105). Half the time the
/// file is made again meanwhile, which queues a write under the op id the
/// delete freed (B133).
fn delete_mid_upload(&mut self, index: usize, rng: &mut Rng) -> Result<(), String> {
let file = rng
.pick(&self.clients[index].model.files())
.map(|path| (*path).clone());
let (Some(path), Some(budget), true) = (
file,
self.profile.settle_budget,
self.clients.iter().all(|client| client.online),
) else {
let op = self.clients[index].model.next_op(rng);
return self.syscall(index, op);
};
self.settle(budget)?;
let Some(uid) = self.drive.lookup(&join(&owned_folder(index), &path)) else {
return Err(format!(
"no loss: client {index}'s {path} is not on Drive after a settle"
));
};
let held = self.clients[index].daemon().client.hold_next_read(&uid);
self.note(format!("client {index}: hold the next read of {path}"));
let mut bytes = data(rng);
bytes.push(b'.');
self.syscall(
index,
FsOp::Write {
path: path.clone(),
offset: 0,
data: bytes,
},
)?;
if !wait_until(PICKUP_BUDGET, || held.reached()) {
self.note(format!("client {index}: nothing read {path} back"));
return Ok(());
}
self.syscall(index, FsOp::Unlink { path: path.clone() })?;
if rng.chance(0.5) {
self.syscall(
index,
FsOp::Create {
path: path.clone(),
data: data(rng),
},
)?;
}
let faults = (self.profile.faults)();
let latency = Duration::from_millis(faults.latency_ms.1);
std::thread::sleep(faults.listing_lag + latency);
drop(held);
self.note(format!("client {index}: the read of {path} goes on"));
// Nothing else is queued before the drain has answered.
std::thread::sleep(2 * latency + Duration::from_millis(50));
Ok(())
}
fn syscall(&mut self, index: usize, op: FsOp) -> Result<(), String> {
let client = &mut self.clients[index];
let root = client.root(index);
let before = client.model.content(target(&op)).cloned();
let expected = client.model.apply(&op);
self.logged.extend(take_logged());
let start = Instant::now();
let actual = perform(&root, &op, before.as_deref());
let took = start.elapsed();
self.note(format!(
"client {index}: {} = {} in {took:?}",
op.describe(),
outcome(&actual)
));
let actual = actual?;
self.logged.extend(take_logged());
if actual != expected {
return Err(format!(
"posix: client {index}: {} answered {}, expected {}",
op.describe(),
errno(actual),
errno(expected)
));
}
if took > self.profile.syscall_budget {
return Err(format!(
"liveness: client {index}: {} took {took:?}, over {:?}",
op.describe(),
self.profile.syscall_budget
));
}
Ok(())
}
fn finish(&mut self) -> Result<(), String> {
for client in &mut self.clients {
client.online = true;
client.daemon().client.set_online(true);
}
self.note("every link up; waiting for the queues".into());
if !wait_until(self.profile.drain_budget, || {
self.logged.extend(take_logged());
self.landed()
}) {
return Err(format!(
"no loss: Drive does not hold what the clients wrote, {:?} after the links came up\n{}",
self.profile.drain_budget,
self.difference()
));
}
// Each client's folder is its own, so a file of it gone from Drive was
// deleted by the client.
if let Some(line) = self.logged.iter().find(|line| is_own_delete_kept(line)) {
return Err(format!(
"no false conflicts: the drain took a client's own delete for another device's: {line}"
));
}
let drive = self.drive_tree();
let conflicts: Vec<String> = drive
.into_keys()
.filter(|path| is_conflict_copy(path))
.collect();
if !conflicts.is_empty() {
return Err(format!("no false conflicts: {conflicts:?}"));
}
for index in 0..self.clients.len() {
let mut last = Tree::new();
let converged = wait_until(CONVERGE_BUDGET, || {
last = read_tree(&self.clients[index].daemon().mountpoint);
last == self.drive_tree()
});
if !converged {
return Err(format!(
"convergence: client {index}'s mount differs from Drive\n{}",
diff(&last, &self.drive_tree(), "mount", "Drive")
));
}
}
Ok(())
}
/// Whether every client's queue is empty and Drive holds what its model
/// says.
fn landed(&self) -> bool {
let drive = self.drive_tree();
self.clients.iter().enumerate().all(|(index, client)| {
client
.daemon()
.pending()
.is_ok_and(|items| items.is_empty())
&& subtree(&drive, &owned_folder(index)) == *client.model.tree()
})
}
fn difference(&self) -> String {
let drive = self.drive_tree();
let mut out = String::new();
for (index, client) in self.clients.iter().enumerate() {
let pending = client.daemon().pending().unwrap_or_default();
for item in pending {
out.push_str(&format!(
" client {index} still owes: {} {} (attempts {}, last error {:?})\n",
item.kind, item.path, item.attempts, item.last_error
));
}
out.push_str(&diff(
client.model.tree(),
&subtree(&drive, &owned_folder(index)),
&format!("client {index}'s model"),
"Drive",
));
}
out
}
fn drive_tree(&self) -> Tree {
self.drive
.tree()
.into_iter()
.map(|(path, entry)| {
let content = match entry {
Entry::Folder => None,
Entry::File(content) => Some(content.as_ref().clone()),
};
(path, content)
})
.collect()
}
fn note(&mut self, line: String) {
let line = format!("{:>8.3}s {line}", self.started.elapsed().as_secs_f64());
self.log.push(line);
}
fn report(&self, failure: &str, dir: &Path) -> String {
let _ = std::fs::write(dir.join("steps.log"), self.log.join("\n") + "\n");
let tail = &self.log[self.log.len().saturating_sub(LOG_TAIL)..];
format!(
"{} seed {} failed: {failure}\nreplay with PDFS_SIM_SEED={}; state kept in {}\nlast steps:\n{}",
self.profile.name,
self.seed,
self.seed,
dir.display(),
tail.join("\n")
)
}
}
/// Make `op` under `root`. The outer error is a check that failed on the
/// way (the content an open file reads back), the inner one the errno.
fn perform(root: &Path, op: &FsOp, before: Option<&[u8]>) -> Result<Result<(), i32>, String> {
let at = |path: &str| root.join(path);
let result: io::Result<()> = match op {
FsOp::Create { path, data } => std::fs::write(at(path), data),
FsOp::Write { path, offset, data } => OpenOptions::new()
.write(true)
.open(at(path))
.and_then(|mut file| {
file.seek(SeekFrom::Start(*offset))?;
file.write_all(data)
}),
FsOp::Truncate { path, len } => OpenOptions::new()
.write(true)
.open(at(path))
.and_then(|file| file.set_len(*len)),
FsOp::Mkdir { path } => std::fs::create_dir(at(path)),
FsOp::Rename { from, to } => std::fs::rename(at(from), at(to)),
FsOp::Unlink { path } => std::fs::remove_file(at(path)),
FsOp::Rmdir { path } => std::fs::remove_dir(at(path)),
FsOp::UnlinkOpen { path, data } => {
let mut read = Vec::new();
let result = OpenOptions::new()
.read(true)
.write(true)
.open(at(path))
.and_then(|mut file| {
std::fs::remove_file(at(path))?;
file.seek(SeekFrom::End(0))?;
file.write_all(data)?;
file.seek(SeekFrom::Start(0))?;
file.read_to_end(&mut read).map(|_| ())
});
if result.is_ok() {
let expected = [before.unwrap_or_default(), data].concat();
if read != expected {
return Err(format!(
"posix: {} read back {:?}, expected {:?}",
op.describe(),
String::from_utf8_lossy(&read),
String::from_utf8_lossy(&expected)
));
}
}
result
}
};
Ok(result.map_err(|error| error.raw_os_error().unwrap_or(libc::EIO)))
}
fn target(op: &FsOp) -> &str {
match op {
FsOp::Create { path, .. }
| FsOp::Write { path, .. }
| FsOp::Truncate { path, .. }
| FsOp::Mkdir { path }
| FsOp::Unlink { path }
| FsOp::UnlinkOpen { path, .. }
| FsOp::Rmdir { path } => path,
FsOp::Rename { from, .. } => from,
}
}
/// The tree a mount shows, read through it, without the folders it makes up
/// that are not on Drive.
fn read_tree(mountpoint: &Path) -> Tree {
fn walk(dir: &Path, prefix: &str, tree: &mut Tree) {
let Ok(entries) = std::fs::read_dir(dir) else {
return;
};
for entry in entries.flatten() {
let name = entry.file_name().to_string_lossy().into_owned();
if prefix.is_empty() && name == crate::r#virtual::SHARED_WITH_ME_BASE {
continue;
}
let path = super::model::join(prefix, &name);
if entry.file_type().is_ok_and(|kind| kind.is_dir()) {
tree.insert(path.clone(), None);
walk(&entry.path(), &path, tree);
} else {
tree.insert(path, std::fs::read(entry.path()).ok());
}
}
}
let mut tree = Tree::new();
walk(mountpoint, "", &mut tree);
tree
}
/// The part of `tree` below `folder`, relative to it.
fn subtree(tree: &Tree, folder: &str) -> Tree {
let prefix = format!("{folder}/");
tree.iter()
.filter_map(|(path, entry)| Some((path.strip_prefix(&prefix)?.to_owned(), entry.clone())))
.collect()
}
fn owned_folder(index: usize) -> String {
format!("c{index}")
}
/// Whether `line` is the drain keeping a queued write as a conflict copy
/// because Drive no longer holds the file, or holds it in the trash.
fn is_own_delete_kept(line: &str) -> bool {
line.starts_with("queued write conflicts; keeping a conflict copy")
&& (line.contains(" reason=\"the file was trashed remotely\"")
|| line.contains(" reason=\"the file no longer exists remotely\""))
}
fn is_conflict_copy(path: &str) -> bool {
path.split('/')
.any(|name| name.contains("(sync-conflict") || name.starts_with("recovered-"))
}
fn diff(left: &Tree, right: &Tree, left_name: &str, right_name: &str) -> String {
let show = |entry: &Option<Vec<u8>>| match entry {
None => "folder".to_owned(),
Some(content) => format!("{:?}", String::from_utf8_lossy(content)),
};
let mut out = String::new();
for (path, entry) in left {
match right.get(path) {
None => out.push_str(&format!(
" {path}: only in {left_name} ({})\n",
show(entry)
)),
Some(other) if other != entry => out.push_str(&format!(
" {path}: {left_name} has {}, {right_name} has {}\n",
show(entry),
show(other)
)),
Some(_) => {}
}
}
for (path, entry) in right {
if !left.contains_key(path) {
out.push_str(&format!(
" {path}: only in {right_name} ({})\n",
show(entry)
));
}
}
out
}
fn errno(outcome: Result<(), i32>) -> String {
match outcome {
Ok(()) => "ok".into(),
Err(code) => io::Error::from_raw_os_error(code).to_string(),
}
}
fn outcome(actual: &Result<Result<(), i32>, String>) -> String {
match actual {
Ok(result) => errno(*result),
Err(check) => check.clone(),
}
}
/// Run `profile` over the seeds the environment asks for: `PDFS_SIM_SEED`
/// alone, or `PDFS_SIM_SEEDS` of them (default `default_seeds`) from 1.
pub(crate) fn run_seeds(profile: &'static Profile, default_seeds: u64) {
let seeds: Vec<u64> = match std::env::var("PDFS_SIM_SEED") {
Ok(seed) => vec![seed.parse().expect("PDFS_SIM_SEED is a number")],
Err(_) => {
let count = std::env::var("PDFS_SIM_SEEDS")
.ok()
.and_then(|count| count.parse().ok())
.unwrap_or(default_seeds);
(1..=count).collect()
}
};
let failures: Vec<String> = seeds
.into_iter()
.filter_map(|seed| run(seed, profile).err())
.collect();
assert!(failures.is_empty(), "\n{}", failures.join("\n\n"));
}
#[cfg(test)]
mod tests {
use super::*;
static ONE_CLIENT_LAN: Profile = Profile {
name: "one-client-lan",
clients: 1,
steps: 80,
faults: Faults::lan,
link_flaps: false,
restarts: false,
deletes_mid_upload: false,
settle_budget: Some(Duration::from_secs(4)),
syscall_budget: Duration::from_secs(2),
drain_budget: Duration::from_secs(20),
};
static ONE_CLIENT_DELETES: Profile = Profile {
name: "one-client-deletes",
clients: 1,
steps: 30,
faults: Faults::lan,
link_flaps: false,
restarts: false,
deletes_mid_upload: true,
settle_budget: Some(Duration::from_secs(4)),
syscall_budget: Duration::from_secs(2),
drain_budget: Duration::from_secs(20),
};
static ONE_CLIENT_ECHOES: Profile = Profile {
name: "one-client-echoes",
clients: 1,
steps: 60,
faults: Faults::echoes,
link_flaps: false,
restarts: false,
deletes_mid_upload: false,
settle_budget: Some(Duration::from_secs(6)),
syscall_budget: Duration::from_secs(2),
drain_budget: Duration::from_secs(20),
};
static ONE_CLIENT_FLAKY: Profile = Profile {
name: "one-client-flaky",
clients: 1,
steps: 80,
faults: Faults::wifi,
link_flaps: true,
restarts: true,
deletes_mid_upload: false,
settle_budget: None,
syscall_budget: Duration::from_secs(10),
drain_budget: Duration::from_secs(120),
};
/// Wi-Fi's latency, lag and lost replies without the outages: the queue
/// has to drain within a settle, not merely by the end.
static ONE_CLIENT_WIFI: Profile = Profile {
name: "one-client-wifi",
clients: 1,
steps: 80,
faults: Faults::wifi,
link_flaps: false,
restarts: false,
deletes_mid_upload: false,
settle_budget: Some(Duration::from_secs(30)),
syscall_budget: Duration::from_secs(10),
drain_budget: Duration::from_secs(120),
};
static THREE_CLIENTS: Profile = Profile {
name: "three-clients",
clients: 3,
steps: 90,
faults: Faults::lan,
link_flaps: true,
restarts: true,
deletes_mid_upload: false,
settle_budget: Some(Duration::from_secs(4)),
syscall_budget: Duration::from_secs(2),
drain_budget: Duration::from_secs(60),
};
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse --lib sim::run -- --ignored`"]
fn one_client_on_a_good_link() {
run_seeds(&ONE_CLIENT_LAN, 12);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse --lib sim::run -- --ignored`"]
fn one_client_deleting_files_as_they_upload() {
run_seeds(&ONE_CLIENT_DELETES, 3);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse --lib sim::run -- --ignored`"]
fn one_client_with_stale_hashes_and_reordered_echoes() {
run_seeds(&ONE_CLIENT_ECHOES, 6);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse --lib sim::run -- --ignored`"]
fn one_client_on_a_flaky_link() {
run_seeds(&ONE_CLIENT_FLAKY, 4);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse --lib sim::run -- --ignored`"]
fn one_client_on_a_slow_link() {
run_seeds(&ONE_CLIENT_WIFI, 4);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse --lib sim::run -- --ignored`"]
fn three_clients_one_writer_each() {
run_seeds(&THREE_CLIENTS, 3);
}
#[test]
fn conflict_copies_are_recognised_by_name() {
assert!(is_conflict_copy("c0/x/a (sync-conflict 1790000000).txt"));
assert!(is_conflict_copy("recovered-XrPUSW5W"));
assert!(!is_conflict_copy("c0/x/a.txt"));
}
#[test]
fn a_write_kept_because_its_file_is_gone_is_recognised() {
let kept = "queued write conflicts; keeping a conflict copy uid=simvol~L10 \
name=\"f.txt\" reason=\"the file was trashed remotely\"";
assert!(is_own_delete_kept(kept));
assert!(is_own_delete_kept(
&kept.replace("was trashed", "no longer exists")
));
assert!(!is_own_delete_kept(
&kept.replace("the file was trashed remotely", "revision changed")
));
}
}
+322
View File
@@ -0,0 +1,322 @@
//! Turns a call through a mount that never returns into a failed seed.
//!
//! A deadlock in a daemon leaves the run's own thread blocked in the kernel,
//! where no budget check is reached, and a CI job then waits for its timeout.
//! The watchdog runs beside the run. When what it was armed for outlives its
//! limit, it aborts the FUSE connections of the run's mounts, so the blocked
//! call fails and the run reports the hang, and writes where every thread was
//! to `stacks.txt`.
//!
//! The stacks are taken in the process: no tracer can stop a thread that
//! waits on a FUSE request, so `gdb` and `eu-stack` wait as long as the hang.
//! Each thread is sent [`stack_signal`] and captures its own backtrace in the
//! handler. A thread in the kernel takes the signal only once it leaves, so
//! the mounts are aborted after the others have answered, and its backtrace
//! is the call it was blocked in. Its kernel wait channel is recorded first.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread::JoinHandle;
use std::time::{Duration, Instant};
use parking_lot::Mutex;
use super::daemon::wait_until;
/// How long the threads that are not blocked in the kernel have to answer,
/// and the blocked ones once the mounts are aborted.
const ANSWER: Duration = Duration::from_secs(5);
pub(crate) struct Watchdog {
shared: Arc<Shared>,
thread: Option<JoinHandle<()>>,
}
struct Shared {
/// The run's directory: its mounts are below it, and the stacks go in it.
dir: PathBuf,
armed: Mutex<Option<Armed>>,
/// What hung, once something has.
fired: Mutex<Option<Hung>>,
stop: AtomicBool,
}
struct Armed {
what: String,
limit: Duration,
since: Instant,
}
/// What hung, and where every thread was then.
#[derive(Clone, Debug, PartialEq)]
pub(crate) struct Hung {
pub(crate) what: String,
pub(crate) stacks: String,
}
/// Disarms the watchdog when dropped.
pub(crate) struct Watching(Arc<Shared>);
impl Watchdog {
pub(crate) fn new(dir: &Path) -> Self {
let shared = Arc::new(Shared {
dir: dir.to_path_buf(),
armed: Mutex::new(None),
fired: Mutex::new(None),
stop: AtomicBool::new(false),
});
let thread = {
let shared = shared.clone();
std::thread::Builder::new()
.name("sim-watchdog".into())
.spawn(move || watch(&shared))
.expect("spawn the simulation watchdog")
};
Self {
shared,
thread: Some(thread),
}
}
/// Watch `what` until the guard is dropped, and abort the mounts if it is
/// still going after `limit`.
pub(crate) fn arm(&self, limit: Duration, what: impl Into<String>) -> Watching {
*self.shared.armed.lock() = Some(Armed {
what: what.into(),
limit,
since: Instant::now(),
});
Watching(self.shared.clone())
}
/// What hung, if something did. Call [`Watchdog::stop`] first, or the
/// stacks may still be on their way.
pub(crate) fn fired(&self) -> Option<Hung> {
self.shared.fired.lock().clone()
}
/// Stop watching, once the stacks of a hang that was caught are taken.
pub(crate) fn stop(&mut self) {
self.shared.stop.store(true, Ordering::Relaxed);
if let Some(thread) = self.thread.take() {
let _ = thread.join();
}
}
}
impl Drop for Watching {
fn drop(&mut self) {
*self.0.armed.lock() = None;
}
}
impl Drop for Watchdog {
fn drop(&mut self) {
self.stop();
}
}
fn watch(shared: &Shared) {
while !shared.stop.load(Ordering::Relaxed) {
std::thread::sleep(Duration::from_millis(200));
let hung = shared
.armed
.lock()
.as_ref()
.filter(|armed| armed.since.elapsed() > armed.limit)
.map(|armed| format!("{} has not returned after {:?}", armed.what, armed.limit));
let Some(hung) = hung else { continue };
*shared.armed.lock() = None;
let path = shared.dir.join("stacks.txt");
// Before anything is let go: the run reports the hang, not whatever
// the aborted call answers.
*shared.fired.lock() = Some(Hung {
what: format!("{hung}; the stacks are in {}", path.display()),
stacks: String::new(),
});
let stacks = take_stacks(|| abort_mounts_below(&shared.dir));
let what = match std::fs::write(&path, &stacks) {
Ok(()) => format!("{hung}; the stacks are in {}", path.display()),
Err(error) => format!("{hung}; the stacks could not be kept: {error}"),
};
*shared.fired.lock() = Some(Hung { what, stacks });
}
}
/// One thread of this process as the kernel sees it.
struct Task {
tid: i32,
name: String,
/// `D` while it waits in the kernel uninterruptibly.
state: String,
wchan: String,
}
/// Where every other thread of this process is, with `release` called once
/// those not blocked in the kernel have answered.
fn take_stacks(release: impl FnOnce()) -> String {
let own = gettid();
let tasks = tasks();
STACKS.lock().clear();
install_stack_handler();
for task in &tasks {
if task.tid != own {
// SAFETY: tgkill only sends a signal, whose handler is installed.
unsafe {
libc::syscall(
libc::SYS_tgkill,
std::process::id() as i32,
task.tid,
stack_signal(),
)
};
}
}
let answered = |blocked: bool| {
let stacks = STACKS.lock();
tasks
.iter()
.filter(|task| task.tid != own && (blocked || task.state != "D"))
.all(|task| stacks.contains_key(&task.tid))
};
wait_until(ANSWER, || answered(false));
release();
wait_until(ANSWER, || answered(true));
let stacks = std::mem::take(&mut *STACKS.lock());
let mut out = String::new();
for task in tasks.iter().filter(|task| task.tid != own) {
out.push_str(&format!(
"thread {} {:?}, state {}, waiting in {}\n",
task.tid, task.name, task.state, task.wchan
));
match stacks.get(&task.tid) {
Some(stack) => out.push_str(&format!("{stack}\n")),
None => out.push_str(" (no answer)\n\n"),
}
}
out
}
fn tasks() -> Vec<Task> {
let Ok(entries) = std::fs::read_dir("/proc/self/task") else {
return Vec::new();
};
let read = |dir: &Path, file: &str| {
std::fs::read_to_string(dir.join(file))
.unwrap_or_default()
.trim()
.to_owned()
};
entries
.flatten()
.filter_map(|entry| {
let tid = entry.file_name().to_str()?.parse().ok()?;
let dir = entry.path();
// The state follows the name in parentheses, which may hold spaces.
let stat = read(&dir, "stat");
let state = stat.rsplit_once(") ")?.1.split(' ').next()?.to_owned();
Some(Task {
tid,
name: read(&dir, "comm"),
state,
wchan: read(&dir, "wchan"),
})
})
.collect()
}
/// The backtraces the threads captured, by thread id.
static STACKS: Mutex<BTreeMap<i32, std::backtrace::Backtrace>> = Mutex::new(BTreeMap::new());
/// The signal that asks a thread for its stack: one of the real-time signals
/// that nothing in the daemon uses.
fn stack_signal() -> i32 {
libc::SIGRTMIN() + 7
}
fn install_stack_handler() {
static INSTALLED: std::sync::Once = std::sync::Once::new();
extern "C" fn handler(_: i32) {
// Not async-signal-safe: the capture allocates. The process is being
// taken down for a hang anyway, and a thread interrupted inside the
// allocator only fails to answer.
let stack = std::backtrace::Backtrace::force_capture();
STACKS.lock().insert(gettid(), stack);
}
INSTALLED.call_once(|| {
// SAFETY: a zeroed sigaction with a handler and SA_RESTART is valid,
// and the handler only touches `STACKS`.
unsafe {
let mut action: libc::sigaction = std::mem::zeroed();
action.sa_sigaction = handler as extern "C" fn(i32) as usize;
action.sa_flags = libc::SA_RESTART;
libc::sigemptyset(&mut action.sa_mask);
libc::sigaction(stack_signal(), &action, std::ptr::null_mut());
}
});
}
fn gettid() -> i32 {
// SAFETY: gettid has no arguments and cannot fail.
unsafe { libc::gettid() }
}
/// Abort the FUSE connection of every mount below `dir`. Its connection is
/// named by the mount's device minor, and the mount's owner may abort it.
fn abort_mounts_below(dir: &Path) {
let Ok(mountinfo) = std::fs::read_to_string("/proc/self/mountinfo") else {
return;
};
for line in mountinfo.lines() {
let fields: Vec<&str> = line.split(' ').collect();
let (Some(device), Some(mountpoint)) = (fields.get(2), fields.get(4)) else {
continue;
};
if !Path::new(mountpoint).starts_with(dir) {
continue;
}
if let Some((_, minor)) = device.split_once(':') {
let _ = std::fs::write(format!("/sys/fs/fuse/connections/{minor}/abort"), "1");
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::sim::daemon::scratch;
#[test]
#[ignore = "signals every thread of the test process: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_call_past_its_limit_is_reported_with_the_stacks() {
let dir = scratch("watchdog");
let mut watchdog = Watchdog::new(&dir);
{
let _watching = watchdog.arm(Duration::from_millis(100), "a nap");
std::thread::sleep(Duration::from_secs(1));
}
watchdog.stop();
let hung = watchdog.fired().expect("the watchdog fired");
assert!(
hung.what.starts_with("a nap has not returned after"),
"{hung:?}"
);
assert!(hung.stacks.contains("a_call_past_its_limit"), "{hung:?}");
let kept = std::fs::read_to_string(dir.join("stacks.txt")).unwrap_or_default();
assert_eq!(kept, hung.stacks);
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_call_within_its_limit_is_not() {
let dir = scratch("watchdog-quiet");
let watchdog = Watchdog::new(&dir);
drop(watchdog.arm(Duration::from_millis(100), "a quick call"));
std::thread::sleep(Duration::from_millis(500));
assert_eq!(watchdog.fired(), None);
drop(watchdog);
let _ = std::fs::remove_dir_all(&dir);
}
}
+461 -42
View File
@@ -128,7 +128,62 @@ pub(crate) struct WriteHandle {
pub(crate) open_count: usize,
}
/// A released write whose upload has not happened yet (offline.md Phase 3).
impl WriteHandle {
/// A handle on a file `create` just made: an empty base, so everything
/// written through it is authored.
///
/// The base is `node` as it actually exists — the empty file the server just
/// minted, or a queued create's placeholder — never the local clock or a
/// blank. `queue_revision` turns it into `StagedWrite::based_on` and the
/// drain compares the remote against that. A `now_secs()` mtime differs from
/// the server's stamp by the create round-trip. A missing revision id drops
/// the check to `(mtime, size)`, and moving or renaming the file advances
/// its mtime on the server. Either way the first write to a brand-new file
/// lands as a conflict copy of its own empty create (`docs/BUGS.md` B113).
pub(crate) fn created(ino: u64, node: &Node, file: File, path: PathBuf) -> Self {
Self {
ino,
uid: node.uid.clone(),
file: Arc::new(file),
path,
written: Intervals::default(),
len: 0,
base_size: 0,
base_mtime: node.modification_time,
base_revision_id: crate::node_revision_id(node),
dirty: false,
open_count: 0,
}
}
/// Write over `node` from now on: a revision this daemon has just put on the
/// server under the handle, by sealing an upload or by landing the queued
/// create the handle was opened on. The base the handle opened over is gone,
/// and keeping it would make the drain read our own revision as another
/// device's edit.
pub(crate) fn rebase_onto(&mut self, node: &Node) {
self.base_mtime = node.modification_time;
self.base_size = crate::node_size(node);
self.base_revision_id = crate::node_revision_id(node);
}
/// Account for `[offset, end)` having been written through this handle.
///
/// A write past the end leaves a hole that reads as zeros, exactly like a
/// grow through `setattr`, so the hole is authored too. Left unauthored, a
/// hole below the base size was refilled from the base at commit: a file
/// shrunk and then written past its new end got its old bytes back.
pub(crate) fn record_write(&mut self, offset: u64, end: u64) {
if offset > self.len {
self.written.add(self.len, offset);
}
self.written.add(offset, end);
self.len = self.len.max(end);
self.dirty = true;
}
}
/// A released write whose upload has not happened yet.
///
/// The bytes live in the content cache's staging dir and the intent lives in the
/// `pending_op` table; this pairs them in memory so a read can be served without
@@ -146,8 +201,29 @@ pub(crate) struct PendingRevision {
/// time so this module stays free of `fuser`.
pub(crate) type DirListing = Vec<(u64, bool, String)>;
/// Where [`State::next_ino`] starts: far above any local id a database hands
/// out, so a fallback inode never names a node that has a row.
pub(crate) const FALLBACK_INO: u64 = 1 << 48;
/// The inode of the node with local id `lid` ([`Db::lids_for`]). One above the
/// lid, so no node takes [`crate::ROOT_INO`], which each session uses for its
/// own root. The same in every mount and after a restart.
pub(crate) fn lid_ino(lid: i64) -> u64 {
lid.unsigned_abs() + crate::ROOT_INO
}
/// The local id of the node at inode `ino`, the inverse of [`lid_ino`]:
/// `None` for the root and a fallback inode.
pub(crate) fn ino_lid(ino: u64) -> Option<i64> {
(ino > crate::ROOT_INO && ino < FALLBACK_INO).then(|| (ino - crate::ROOT_INO) as i64)
}
/// Mutable inode bookkeeping, guarded by a mutex because fuser drives the
/// `Filesystem` trait through `&self`.
///
/// One tree for every mount of the daemon. Each FUSE session sees it from its
/// own root ([`State::roots`]) and calls that root inode 1; `ProtonFs`
/// translates at the kernel boundary.
pub(crate) struct State {
/// inode -> node metadata.
pub(crate) entries: HashMap<u64, Entry>,
@@ -156,6 +232,12 @@ pub(crate) struct State {
/// Cached directory listings: parent inode -> child inodes. Presence of a
/// key means the directory has been enumerated.
pub(crate) children: HashMap<u64, Vec<u64>>,
/// The inodes some session has been rooted at ([`State::install_root`]).
/// A root is its own parent. Kept after its session unmounts: the node is
/// still one of ours.
pub(crate) roots: HashSet<u64>,
/// The next inode for a node the database could not give a local id, from
/// a range lids never reach ([`FALLBACK_INO`]).
pub(crate) next_ino: u64,
/// Shared write state keyed by inode. Concurrent writers share the scratch file.
pub(crate) active_writes: HashMap<u64, WriteHandle>,
@@ -171,13 +253,13 @@ pub(crate) struct State {
/// two pages shifted every later entry, and the caller silently skipped or
/// repeated whatever crossed the page boundary. Each entry is
/// `(ino, is_dir, name)` — the kernel type is mapped at reply time so this
/// module stays free of `fuser` (bugs.md B43).
/// module stays free of `fuser` (`docs/BUGS.md` B43).
pub(crate) dir_snapshots: HashMap<u64, Arc<DirListing>>,
/// Resident inode attributes whose effective access changed during an
/// intern/root refresh. Core drains this set and notifies the matching
/// kernel only after releasing the State lock.
pub(crate) access_changes: HashSet<u64>,
/// Persisted shared-root authority, loaded once per inode space and updated
/// Persisted shared-root authority, loaded once and updated
/// on every explicit role/tombstone change. Descendant inheritance reads
/// this map instead of issuing one SQLite query per interned inode.
pub(crate) share_access: HashMap<NodeUid, Access>,
@@ -245,15 +327,6 @@ pub(crate) struct StateGuard<'a> {
db: &'a Arc<Db>,
}
/// Take a mount's inode lock through a [`StateGuard`], for the loops that walk
/// every live mount rather than this `Core`'s own.
pub(crate) fn lock_state<'a>(
state: &'a parking_lot::Mutex<State>,
db: &'a Arc<Db>,
) -> StateGuard<'a> {
StateGuard::new(state.lock(), db)
}
impl<'a> StateGuard<'a> {
pub(crate) fn new(guard: parking_lot::MutexGuard<'a, State>, db: &'a Arc<Db>) -> Self {
Self {
@@ -294,9 +367,6 @@ impl Drop for StateGuard<'_> {
impl State {
/// A fresh, empty inode space over `db`.
///
/// `next_ino` differs by caller: a real mount hands out [`crate::ROOT_INO`]
/// to its root and starts allocating above it, while a bare state under test
/// has no root and starts at 1.
/// Apply the write-throughs this state owes SQLite, here and now.
///
/// Production code never calls this: [`StateGuard`] releases the inode lock
@@ -323,12 +393,15 @@ impl State {
self.db.clone()
}
pub(crate) fn new(db: Arc<Db>, share_access: HashMap<NodeUid, Access>, next_ino: u64) -> Self {
/// A fresh, empty tree over `db`. A node's inode comes from its local id
/// ([`lid_ino`]).
pub(crate) fn new(db: Arc<Db>, share_access: HashMap<NodeUid, Access>) -> Self {
Self {
entries: HashMap::new(),
by_uid: HashMap::new(),
children: HashMap::new(),
next_ino,
roots: HashSet::new(),
next_ino: FALLBACK_INO,
active_writes: HashMap::new(),
handles: HashMap::new(),
next_fh: 1,
@@ -339,18 +412,55 @@ impl State {
outbox: Vec::new(),
}
}
/// Whether `uid` names an owned node currently reachable from this mount's
/// root. Resident entries can outlive their dentries for open-handle and
/// access-revocation semantics, so `by_uid` membership alone is not proof
/// that a control-socket operation may address the node.
pub(crate) fn owns_visible_uid(&self, uid: &NodeUid) -> bool {
let Some(root) = self.entries.get(&crate::ROOT_INO) else {
return false;
};
if uid.volume_id != root.uid.volume_id {
return false;
/// Make `root` the root of a session and return its inode, which comes
/// from its local id like any other node's.
///
/// The root is always owned. It must never be classified as a shared root,
/// even when the API returns membership data (which happens when the owner
/// shares the folder with someone else), so a stale share row a downgrade
/// may have left for it is dropped. A node already in the tree keeps its
/// inode and its lookups, and stops naming a parent: a session's `..` at
/// its root is the root.
pub(crate) fn install_root(&mut self, root: Node) -> u64 {
if let Err(e) = self.db.upsert_node(&root) {
warn!(uid = %root.uid, error = %e, "db upsert root failed");
}
self.share_access.remove(&root.uid);
let _ = self.db.delete_share_access(&root.uid);
let ino = match self.by_uid.get(&root.uid) {
Some(&ino) => ino,
None => self.new_inos(&[&root])[&root.uid],
};
if let Some(old) = self.entries.get(&ino).map(|e| e.parent)
&& old != ino
&& let Some(kids) = self.children.get_mut(&old)
{
kids.retain(|&k| k != ino);
}
self.by_uid.insert(root.uid.clone(), ino);
let lookup_count = self.entries.get(&ino).map_or(1, |e| e.lookup_count.max(1));
self.entries.insert(
ino,
Entry {
uid: root.uid.clone(),
parent: ino,
node: root,
access: Access::Owner,
lookup_count,
open_count: 0,
unlinked: false,
},
);
self.roots.insert(ino);
ino
}
/// Whether `uid` names an owned node currently reachable from one of
/// `roots`, the roots of the sessions that are mounted. Resident entries
/// can outlive their dentries for open-handle and access-revocation
/// semantics, so `by_uid` membership alone is not proof that a
/// control-socket operation may address the node.
pub(crate) fn owns_visible_uid(&self, uid: &NodeUid, roots: &[u64]) -> bool {
let Some(&target) = self.by_uid.get(uid) else {
return false;
};
@@ -363,7 +473,9 @@ impl State {
let Some(entry) = self.entries.get(&current) else {
return false;
};
if entry.uid.volume_id != root.uid.volume_id
// The node's root scopes it to its own volume, excluding foreign
// shared-with-me residents from uid-addressed sharing.
if entry.uid.volume_id != uid.volume_id
|| entry.unlinked
|| entry.node.trashed
|| entry.uid != entry.node.uid
@@ -371,7 +483,7 @@ impl State {
{
return false;
}
if current == crate::ROOT_INO {
if roots.contains(&current) {
return true;
}
@@ -413,15 +525,17 @@ impl State {
self.access_changes.drain().collect()
}
/// Whether `uid` lives on the volume this mount is rooted in.
/// Whether `uid` lives on a volume some mount is rooted in.
///
/// Used to decide how a node with no resolvable parent is classified:
/// our own volume fails open (it is owned content), a foreign volume
/// fails closed (it can only be there because of a share).
pub(crate) fn is_own_volume(&self, uid: &NodeUid) -> bool {
self.entries
.get(&crate::ROOT_INO)
.is_some_and(|root| root.uid.volume_id == uid.volume_id)
self.roots.iter().any(|ino| {
self.entries
.get(ino)
.is_some_and(|root| root.uid.volume_id == uid.volume_id)
})
}
fn stored_share_access(&self, uid: &NodeUid) -> Option<Access> {
@@ -770,11 +884,59 @@ impl State {
}
}
pub(crate) fn intern(&mut self, parent: u64, node: Node) -> u64 {
/// Write `node` through as it is, in order with the tree's other changes.
pub(crate) fn write_through(&mut self, node: Node) {
self.outbox.push(DbWrite::Upsert(vec![node]));
}
pub(crate) fn intern(&mut self, parent: u64, mut node: Node) -> u64 {
self.keep_revision_mtime(&mut node);
self.outbox.push(DbWrite::Upsert(vec![node.clone()]));
self.intern_mem(parent, node)
}
/// Keep what is already known of a file whose content has not changed.
///
/// Drive stamps a link's `ModifyTime` on a rename or a move as well, while
/// POSIX leaves a file's mtime alone there. The content cache is keyed by
/// it, too: taking the new stamp made the cached bytes unreachable, so the
/// next partial write could not fill its gaps (docs/BUGS.md B123). The
/// active revision id says whether the content is still the same, and so
/// whether the size and digest a listing left out are still the ones known.
fn keep_revision_mtime(&self, node: &mut Node) {
let Some(entry) = self
.by_uid
.get(&node.uid)
.and_then(|ino| self.entries.get(ino))
else {
return;
};
if let (
NodeKind::File {
active_revision_id: Some(now),
claimed_size,
content_sha1,
..
},
NodeKind::File {
active_revision_id: Some(was),
claimed_size: size_was,
content_sha1: sha1_was,
..
},
) = (&mut node.kind, &entry.node.kind)
&& now == was
{
node.modification_time = entry.node.modification_time;
if claimed_size.is_none() {
*claimed_size = *size_was;
}
if content_sha1.is_none() {
content_sha1.clone_from(sha1_was);
}
}
}
/// Allocate (or reuse) a stable inode for a node, updating the hot-cache maps
/// only. Every caller owes the DB a write-through — see the callers below;
/// the split exists so a batch can pay for one transaction instead of `n`.
@@ -784,11 +946,24 @@ impl State {
}
fn intern_mem_with_access(&mut self, parent: u64, node: Node, access: Access) -> u64 {
self.intern_mem_at(parent, node, access, None)
}
/// [`State::intern_mem_with_access`], with the inode a new entry takes when
/// the caller already has it ([`State::new_inos`]).
fn intern_mem_at(
&mut self,
parent: u64,
mut node: Node,
access: Access,
ino: Option<u64>,
) -> u64 {
if let Some(&ino) = self.by_uid.get(&node.uid) {
let changed = self
.entries
.get(&ino)
.is_some_and(|entry| entry.access != access);
self.keep_open_write_size(ino, &mut node);
if let Some(e) = self.entries.get_mut(&ino) {
e.node = node;
e.parent = parent;
@@ -801,8 +976,13 @@ impl State {
}
return ino;
}
let ino = self.next_ino;
self.next_ino += 1;
let ino = match ino {
Some(ino) => ino,
None => self
.new_inos(&[&node])
.remove(&node.uid)
.unwrap_or_default(),
};
self.by_uid.insert(node.uid.clone(), ino);
self.entries.insert(
ino,
@@ -819,6 +999,41 @@ impl State {
ino
}
/// The inodes for nodes that have no entry yet, from their local ids, in
/// one transaction. Should the database fail, they come from
/// [`State::next_ino`] instead, so the mount keeps working, with inodes
/// that last only this run. So do the inodes of virtual folders, which are
/// never stored.
fn new_inos(&mut self, nodes: &[&Node]) -> HashMap<NodeUid, u64> {
let (virtual_nodes, stored): (Vec<&Node>, Vec<&Node>) = nodes
.iter()
.partition(|node| crate::r#virtual::is_virtual_uid(&node.uid));
let mut inos = match self.db.lids_for(&stored) {
Ok(lids) => stored
.iter()
.zip(lids)
.map(|(node, lid)| (node.uid.clone(), lid_ino(lid)))
.collect(),
Err(error) => {
warn!(%error, "no local ids for new inodes; numbering them for this run");
self.run_inos(&stored)
}
};
inos.extend(self.run_inos(&virtual_nodes));
inos
}
fn run_inos(&mut self, nodes: &[&Node]) -> HashMap<NodeUid, u64> {
nodes
.iter()
.map(|node| {
let ino = self.next_ino;
self.next_ino += 1;
(node.uid.clone(), ino)
})
.collect()
}
/// Decrement lookup count for an inode and prune if lookup_count == 0 && open_count == 0 && unlinked.
pub(crate) fn forget_lookup(&mut self, ino: u64, nlookup: u64) -> Option<(u64, String)> {
if let Some(entry) = self.entries.get_mut(&ino) {
@@ -833,6 +1048,23 @@ impl State {
/// Allocate (or reuse) a stable inode for a node that came *from* the
/// database, which is why nothing is written back.
/// Carry the size and mtime of a file open for writing over onto `node`,
/// which is about to replace its entry.
///
/// The open handle is ahead of anything Drive can report until release
/// queues it. Taking the remote's size shrank a SQLite database to its empty
/// create while SQLite held it open, and the kernel then capped every read
/// at that size (docs/BUGS.md B109).
pub(crate) fn keep_open_write_size(&self, ino: u64, node: &mut Node) {
let (Some(aw), Some(entry)) = (self.active_writes.get(&ino), self.entries.get(&ino)) else {
return;
};
if let NodeKind::File { claimed_size, .. } = &mut node.kind {
*claimed_size = Some(aw.len as i64);
node.modification_time = entry.node.modification_time;
}
}
pub(crate) fn intern_from_db(&mut self, parent: u64, node: Node) -> u64 {
self.intern_mem(parent, node)
}
@@ -842,11 +1074,27 @@ impl State {
///
/// This is what keeps `ls` on a large folder quick: one commit for the
/// listing, rather than one autocommit — and one fsync — per child.
pub(crate) fn intern_batch(&mut self, parent: u64, nodes: Vec<Node>) -> Vec<u64> {
pub(crate) fn intern_batch(&mut self, parent: u64, mut nodes: Vec<Node>) -> Vec<u64> {
for node in &mut nodes {
self.keep_revision_mtime(node);
}
self.outbox.push(DbWrite::Upsert(nodes.clone()));
let fresh: Vec<&Node> = nodes
.iter()
.filter(|node| !self.by_uid.contains_key(&node.uid))
.collect();
let mut inos = if fresh.is_empty() {
HashMap::new()
} else {
self.new_inos(&fresh)
};
nodes
.into_iter()
.map(|node| self.intern_mem(parent, node))
.map(|node| {
let access = self.access_for_node(parent, &node);
let ino = inos.remove(&node.uid);
self.intern_mem_at(parent, node, access, ino)
})
.collect()
}
@@ -950,7 +1198,7 @@ impl State {
/// re-enumerate, which is the cheaper way to stay honest about what the
/// server did. A queued rename cannot: re-enumerating needs the network, and
/// the server has not been told yet in any case — so this *is* the tree's
/// new truth until the op drains (offline.md Phase 3b).
/// new truth until the op drains.
pub(crate) fn rename_in_place(
&mut self,
ino: u64,
@@ -1069,7 +1317,7 @@ impl State {
/// created offline, nothing at all), so this row is the only record that the
/// file is as long as the caller was told it is. Without it a restart serves
/// the stale size and the file reads as truncated — or empty — while its
/// bytes sit safely in staging (offline.md Phase 3).
/// bytes sit safely in staging.
/// Returns the node to persist rather than queueing it in the outbox, and is
/// the one mutation here whose write-through the caller must do itself:
/// failure has to be reported, not logged. Dropping the row silently means
@@ -1088,6 +1336,13 @@ mod tests {
use proton_drive_rs::ShareMembership;
use proton_drive_rs::proton_sdk::ids::{LinkId, ShareId, ShareMembershipId, VolumeId};
#[test]
fn an_inode_gives_back_the_lid_it_was_made_from() {
assert_eq!(ino_lid(lid_ino(7)), Some(7));
assert_eq!(ino_lid(crate::ROOT_INO), None);
assert_eq!(ino_lid(FALLBACK_INO + 3), None);
}
fn uid(link: &str) -> NodeUid {
NodeUid::new(VolumeId::from("vol"), LinkId::from(link))
}
@@ -1164,7 +1419,7 @@ mod tests {
let dir = TempDir::new();
let db = Db::open(&dir.0.join("cache.db")).unwrap();
let share_access = db.all_share_access().unwrap();
let st = State::new(Arc::new(db), share_access, 1);
let st = State::new(Arc::new(db), share_access);
(st, dir)
}
@@ -1182,6 +1437,36 @@ mod tests {
(st, dir, src, dst)
}
/// A node's inode comes from its row's local id, so it is the same in
/// every mount, after the hot cache drops it, and after a restart.
#[test]
fn a_node_keeps_its_inode_across_mounts_evictions_and_restarts() {
let (mut st, _dir) = state();
let inos = st.intern_batch(
0,
vec![
node("a", "root", "a.txt", false),
node("b", "root", "b.txt", false),
],
);
st.flush_outbox();
let db = st.db.clone();
assert_ne!(inos[0], inos[1]);
assert!(!inos.contains(&crate::ROOT_INO));
let mut other = State::new(db.clone(), HashMap::new());
assert_eq!(other.intern(0, node("b", "root", "b.txt", false)), inos[1]);
st.forget_mem(&uid("a"));
assert_eq!(st.intern(0, node("a", "root", "a.txt", false)), inos[0]);
let mut restarted = State::new(db, HashMap::new());
assert_eq!(
restarted.intern_batch(0, vec![node("a", "root", "a.txt", false)]),
[inos[0]]
);
}
/// Audit A5. Moving a file out of a folder leaves that folder's DB row
/// claiming a complete listing, so once the hot cache drops it, remote
/// changes under it stop being seen. `relocate` has to clear both sides.
@@ -1247,7 +1532,7 @@ mod tests {
let root = uid("root").to_string();
let state = parking_lot::Mutex::new(st);
{
let mut guard = lock_state(&state, &db);
let mut guard = StateGuard::new(state.lock(), &db);
guard.intern(0, node("root", "none", "My Files", true));
assert!(
db.node_by_uid(&root).unwrap().is_none(),
@@ -1490,6 +1775,7 @@ mod tests {
})
.collect();
st.next_ino = crate::ROOT_INO + nodes.len() as u64;
st.roots.insert(crate::ROOT_INO);
// Reverse order so entries are inserted before their parents, matching
// the hydration path's non-topological materialization.
for (i, node) in nodes.iter().enumerate().rev() {
@@ -1863,4 +2149,137 @@ mod tests {
let shared = st.intern(owned_root, shared);
assert_eq!(st.entries[&shared].access, Access::Viewer);
}
fn at_revision(mut node: Node, revision: &str, mtime: i64) -> Node {
node.modification_time = mtime;
if let NodeKind::File {
active_revision_id, ..
} = &mut node.kind
{
*active_revision_id = Some(revision.into());
}
node
}
#[test]
fn a_renamed_file_keeps_its_mtime_until_its_content_changes() {
let (mut st, _dir) = state();
let root = st.intern(0, node("root", "none", "My Files", true));
let mut file = node("file", "root", "a.txt", false);
if let NodeKind::File {
claimed_size,
total_size_on_storage,
..
} = &mut file.kind
{
*claimed_size = Some(7);
*total_size_on_storage = 519;
}
let ino = st.intern(root, at_revision(file.clone(), "r1", 100));
// Drive stamps the link on a rename; the content is still r1.
let mut renamed = at_revision(file.clone(), "r1", 160);
renamed.name = "b.txt".into();
st.intern_batch(root, vec![renamed]);
assert_eq!(st.entries[&ino].node.modification_time, 100);
assert_eq!(st.entries[&ino].node.name, "b.txt");
// A listing that leaves the size out still describes revision r1.
let mut bare = at_revision(file.clone(), "r1", 170);
assert_eq!(crate::node_size(&st.entries[&ino].node), 7);
if let NodeKind::File { claimed_size, .. } = &mut bare.kind {
*claimed_size = None;
}
st.intern(root, bare);
assert_eq!(st.entries[&ino].node.modification_time, 100);
assert_eq!(crate::node_size(&st.entries[&ino].node), 7);
st.intern(root, at_revision(file, "r2", 200));
assert_eq!(st.entries[&ino].node.modification_time, 200);
}
fn handle(len: u64, base_size: u64) -> WriteHandle {
let path = std::env::temp_dir().join(format!("pdfs-handle-test-{}", std::process::id()));
let file = File::create(&path).unwrap();
let _ = std::fs::remove_file(&path);
WriteHandle {
ino: 2,
uid: uid("file"),
file: Arc::new(file),
path,
written: Intervals::default(),
len,
base_size,
base_mtime: 0,
base_revision_id: None,
dirty: false,
open_count: 1,
}
}
/// A file created offline is written while its queued create lands. The
/// placeholder had no revision, so the handle takes the empty file the
/// create made as its base (docs/BUGS.md B113).
#[test]
fn a_handle_on_a_placeholder_is_based_on_the_create_that_landed() {
let mut h = handle(0, 0);
assert_eq!(h.base_revision_id, None);
let mut landed = node("real", "root", "f.mkv", false);
landed.modification_time = 160;
if let NodeKind::File {
active_revision_id, ..
} = &mut landed.kind
{
*active_revision_id = Some("empty".into());
}
h.rebase_onto(&landed);
assert_eq!(h.base_mtime, 160);
assert_eq!(h.base_size, 0);
assert_eq!(h.base_revision_id.as_deref(), Some("empty"));
}
#[test]
fn a_listing_does_not_shrink_a_file_open_for_writing() {
let (mut st, _dir) = state();
let ino = st.intern(0, node("file", "root", "workload.db", false));
let mut h = handle(16384, 0);
h.ino = ino;
st.active_writes.insert(ino, h);
st.set_size(ino, 16384);
st.touch_mtime(ino, 500);
// The remote still holds the empty file the create sealed.
st.intern(0, node("file", "root", "workload.db", false));
assert_eq!(crate::node_size(&st.entries[&ino].node), 16384);
assert_eq!(st.entries[&ino].node.modification_time, 500);
// Once the handle is gone the remote is authoritative again.
st.active_writes.remove(&ino);
st.intern(0, node("file", "root", "workload.db", false));
assert_eq!(crate::node_size(&st.entries[&ino].node), 0);
}
#[test]
fn a_write_past_the_end_authors_the_hole_it_leaves() {
// Opened over a 100-byte base, shrunk to 10, then written at 50: bytes
// 10..50 are zeros now, not the base's bytes 10..50.
let mut h = handle(100, 100);
h.written.clip(10);
h.len = 10;
h.record_write(50, 60);
assert_eq!(h.len, 60);
assert_eq!(
h.written.segments(0, 60),
vec![(0, 10, false), (10, 60, true)],
"only the head still comes from the base"
);
}
#[test]
fn a_write_inside_the_file_leaves_the_rest_to_the_base() {
let mut h = handle(100, 100);
h.record_write(40, 50);
assert_eq!(h.len, 100);
assert_eq!(
h.written.segments(0, 100),
vec![(0, 40, false), (40, 50, true), (50, 100, false)]
);
}
}
+5 -6
View File
@@ -68,7 +68,7 @@ impl Core {
/// docs for the identical/divergent policy.
pub(crate) fn run_conflict_sweep_loop(&self) {
// Both waits are interruptible, so teardown does not have to sit out a
// five-minute interval to join this thread (bugs.md B44).
// five-minute interval to join this thread (`docs/BUGS.md` B44).
if !self.shutdown.sleep(CONFLICT_SWEEP_WARMUP) {
return;
}
@@ -165,7 +165,7 @@ impl Core {
}
match self
.rt
.block_on(self.client.trash_nodes(std::slice::from_ref(uid)))
.block_on(self.drive.trash_nodes(std::slice::from_ref(uid)))
.and_then(batch::into_unit)
{
Ok(()) => {}
@@ -175,10 +175,9 @@ impl Core {
return;
}
}
// Every mount, not just the primary: a conflict copy inside a sync
// folder lives in that fork's inode space, and leaving it hooked up
// there keeps a node we have just trashed remotely visible and
// readable there (`docs/BUGS.md` B74).
// Leaving a conflict copy hooked up in the tree keeps a node we have
// just trashed remotely visible and readable in whichever mount shows
// it (`docs/BUGS.md` B74).
self.for_each_state(|st| {
st.forget_or_unlink(uid);
});
+295 -48
View File
@@ -1,4 +1,4 @@
//! Two-way folder sync engine (devices.md Phase 2).
//! Two-way folder sync engine.
//!
//! Each `mirror`-mode synced folder is kept mirrored between a local directory
//! and a folder under this machine's Proton Drive device. A local filesystem
@@ -73,6 +73,9 @@ enum Applied {
Downloaded,
/// A conflict copy was kept.
Conflict,
/// The local file to upload was deleted after the scan planned it. There is
/// nothing left to send, and the next pass sees the deletion (B108).
Vanished,
}
impl Core {
@@ -99,6 +102,9 @@ impl Core {
/// One reconcile pass over a mirror folder, updating its `state` column.
fn reconcile_pass(&self, folder: &StoredSyncFolder) {
if self.shutdown.is_stopping() {
return;
}
// Hold the folder's lock for the whole pass so a mode switch can't evict the
// local tree (and mount FUSE over it) while we walk and upload it.
let lock = self.sync_lock(folder.id);
@@ -145,8 +151,6 @@ impl Core {
self.push_pass(folder.id, &local_root, &remote_root, &rules)
} else {
match self.do_reconcile(folder.id, &local_root, &remote_root, &rules) {
Ok(outcome) => Ok(outcome),
Err(PassAbort::Failed(e)) => Err(e),
// The user asked for on-demand while this pass was running. Rather
// than make them wait out a walk-and-download whose results are
// about to be deleted, drop it here and do the only part that still
@@ -161,6 +165,7 @@ impl Core {
self.progress_begin(folder.id);
self.push_pass(folder.id, &local_root, &remote_root, &rules)
}
result => result,
}
};
self.progress_end(folder.id);
@@ -213,7 +218,15 @@ impl Core {
);
}
}
Err(e) => {
// A push pass is never interrupted, so only the stop gets here: the
// folder keeps the state it had, which the next start's pass updates.
Err(PassAbort::Stopped | PassAbort::Interrupted) => {
info!(id = folder.id, "sync: pass cut short by the stop");
let _ = self
.db
.sync_folder_set_state(folder.id, &current.state, current.last_sync);
}
Err(PassAbort::Failed(e)) => {
warn!(id = folder.id, error = %e, "sync: reconcile failed");
let _ = self
.db
@@ -291,7 +304,7 @@ impl Core {
local_root: &Path,
remote_root: &NodeUid,
rules: &IgnoreRules,
) -> Result<Outcome, String> {
) -> Result<Outcome, PassAbort> {
let baseline = self
.db
.sync_entries(folder_id)
@@ -367,7 +380,7 @@ impl Core {
HashMap::new()
} else {
self.rt
.block_on(self.client.enumerate_nodes_light(&check))
.block_on(self.drive.enumerate_nodes_light(&check))
.map_err(|e| format!("resolve changed nodes: {e}"))?
.into_iter()
.filter(|n| !n.trashed)
@@ -397,7 +410,7 @@ impl Core {
&mut remote_dirs,
std::mem::take(&mut batch),
&mut outcome,
);
)?;
batch_depth = depth;
}
let item = &local[rel];
@@ -495,7 +508,7 @@ impl Core {
&mut remote_dirs,
std::mem::take(&mut batch),
&mut outcome,
);
)?;
// Paths the baseline knows that are no longer on disk: the user deleted them
// locally, and the deletion has to reach Drive before the mount starts
@@ -522,7 +535,7 @@ impl Core {
};
if let Err(e) = self
.rt
.block_on(self.client.trash_nodes(&[uid]))
.block_on(self.drive.trash_nodes(&[uid]))
.and_then(batch::into_unit)
{
warn!(rel, error = %e, "sync: trash remote failed");
@@ -622,7 +635,7 @@ impl Core {
&mut remote_dirs,
std::mem::take(&mut batch),
&mut outcome,
);
)?;
batch_depth = depth;
// A depth boundary is the pass's natural checkpoint — everything
// queued has been applied, so stopping here leaves nothing half done.
@@ -773,7 +786,7 @@ impl Core {
let remote = r.expect("remote deletion requires a remote file");
if let Err(e) = self
.rt
.block_on(self.client.trash_nodes(std::slice::from_ref(&remote.uid)))
.block_on(self.drive.trash_nodes(std::slice::from_ref(&remote.uid)))
.and_then(batch::into_unit)
{
warn!(rel, error = %e, "sync: trash remote failed");
@@ -810,7 +823,7 @@ impl Core {
&mut remote_dirs,
std::mem::take(&mut batch),
&mut outcome,
);
)?;
// Deferred folder deletions, deepest first.
delete_local_dirs.sort_by_key(|p| std::cmp::Reverse(p.matches('/').count()));
@@ -836,7 +849,7 @@ impl Core {
for (rel, uid) in delete_remote_dirs {
if let Err(e) = self
.rt
.block_on(self.client.trash_nodes(&[uid]))
.block_on(self.drive.trash_nodes(&[uid]))
.and_then(batch::into_unit)
{
warn!(rel, error = %e, "sync: trash remote folder failed");
@@ -891,14 +904,14 @@ impl Core {
) -> Result<(), String> {
let uids = self
.rt
.block_on(self.client.enumerate_folder_children_node_uids(folder))
.block_on(self.drive.enumerate_folder_children_node_uids(folder))
.map_err(|e| format!("enumerate {folder}: {e}"))?;
if uids.is_empty() {
return Ok(());
}
let nodes = self
.rt
.block_on(self.client.enumerate_nodes_light(&uids))
.block_on(self.drive.enumerate_nodes_light(&uids))
.map_err(|e| format!("resolve nodes: {e}"))?;
for node in nodes {
if node.trashed || !node.is_folder() {
@@ -938,19 +951,22 @@ impl Core {
// is safe precisely because the walk has applied nothing — a *partial*
// remote map, on the other hand, would be read as "the remote deleted
// everything we haven't reached yet", so it must never reach the diff.
if self.shutdown.is_stopping() {
return Err(PassAbort::Stopped);
}
if self.ondemand_queued(folder_id) {
return Err(PassAbort::Interrupted);
}
let uids = self
.rt
.block_on(self.client.enumerate_folder_children_node_uids(folder))
.block_on(self.drive.enumerate_folder_children_node_uids(folder))
.map_err(|e| format!("enumerate {folder}: {e}"))?;
if uids.is_empty() {
return Ok(());
}
let nodes = self
.rt
.block_on(self.client.enumerate_nodes_light(&uids))
.block_on(self.drive.enumerate_nodes_light(&uids))
.map_err(|e| format!("resolve nodes: {e}"))?;
// Files whose recorded mtime no longer matches: their size has to be
@@ -968,7 +984,7 @@ impl Core {
HashMap::new()
} else {
self.rt
.block_on(self.client.enumerate_nodes(&stale))
.block_on(self.drive.enumerate_nodes(&stale))
.map_err(|e| format!("resolve nodes: {e}"))?
.iter()
.map(|n| (n.uid.to_string(), remote_file_sig(n).1))
@@ -1048,6 +1064,9 @@ impl Core {
/// `remote_dirs` (so deeper paths resolve their parent), and what each op did
/// is tallied into `outcome`. Driven by `block_on` from the (non-runtime) sync
/// engine thread, spawning the tasks onto the shared runtime.
///
/// Fails with [`PassAbort::Stopped`] once the daemon is stopping, after the
/// ops already sending have finished.
fn flush_batch(
&self,
folder_id: i64,
@@ -1055,9 +1074,9 @@ impl Core {
remote_dirs: &mut HashMap<String, NodeUid>,
batch: Vec<Pending>,
outcome: &mut Outcome,
) {
) -> Result<(), PassAbort> {
if batch.is_empty() {
return;
return Ok(());
}
self.progress_queued(folder_id, batch.len());
let core = self.clone();
@@ -1071,14 +1090,22 @@ impl Core {
let root = root.clone();
set.spawn(async move {
let _permit = sem.acquire_owned().await.expect("semaphore not closed");
core.apply_pending(folder_id, &root, op).await
// An op still waiting for its turn when the stop comes is
// not started; the ones already sending finish.
if core.shutdown.is_stopping() {
return None;
}
Some(core.apply_pending(folder_id, &root, op).await)
});
}
let mut out = Vec::new();
let mut join_errors = 0usize;
while let Some(joined) = set.join_next().await {
match joined {
Ok(result) => out.push(result),
Ok(Some(result)) => out.push(result),
Ok(None) => {}
// The runtime ends with the daemon and cancels what is left.
Err(e) if e.is_cancelled() => {}
Err(e) => {
join_errors += 1;
warn!(error = %e, "sync: task panicked");
@@ -1103,6 +1130,13 @@ impl Core {
}
}
}
// The pass goes no further: the next depth's paths may need a folder
// this batch did not get to make, and went on to fail one by one
// (`docs/BUGS.md` B169).
if self.shutdown.is_stopping() {
return Err(PassAbort::Stopped);
}
Ok(())
}
/// Apply one [`Pending`] op (async, so it can run concurrently in a batch),
@@ -1136,7 +1170,7 @@ impl Core {
match op {
Pending::CreateDir { rel, parent } => {
let uid = self
.client
.drive
.create_folder(parent, base_name(rel), Some(now_secs()))
.await
.map_err(|e| format!("create remote folder {rel}: {e}"))?;
@@ -1147,13 +1181,10 @@ impl Core {
Ok(Applied::Dir(rel.clone(), uid))
}
Pending::UploadNew { rel, parent } => {
self.upload_new(folder_id, local_root, rel, parent).await?;
Ok(Applied::Uploaded)
self.upload_new(folder_id, local_root, rel, parent).await
}
Pending::UploadRevision { rel, uid } => {
self.upload_revision(folder_id, local_root, rel, uid)
.await?;
Ok(Applied::Uploaded)
self.upload_revision(folder_id, local_root, rel, uid).await
}
Pending::Download {
rel,
@@ -1217,10 +1248,12 @@ impl Core {
local_root: &Path,
rel: &str,
parent: &NodeUid,
) -> Result<(), String> {
) -> Result<Applied, String> {
let path = local_root.join(rel_to_path(rel));
let name = base_name(rel);
let file = std::fs::File::open(&path).map_err(|e| format!("open {rel}: {e}"))?;
let Some(file) = open_to_upload(&path, rel)? else {
return Ok(Applied::Vanished);
};
let meta = file.metadata().map_err(|e| format!("stat {rel}: {e}"))?;
let mtime = system_mtime(&meta);
// Taken from the descriptor being streamed, so it names the content that
@@ -1238,7 +1271,7 @@ impl Core {
.begin(name, "", TransferDirection::Upload, meta.len()),
);
let uid = self
.client
.drive
// Mirror push: the local file is authoritative. If a prior attempt was
// interrupted mid-upload it left an unsealed draft of this name; recover
// it even across a daemon restart (which rotates our client uid), so the
@@ -1261,7 +1294,8 @@ impl Core {
self.invalidate_children_of(parent);
warn_if_torn(rel, &path, streamed);
self.record_file_baseline(folder_id, rel, Some(streamed), &uid)
.await
.await?;
Ok(Applied::Uploaded)
}
/// Upload a changed local file as a new revision of an existing remote node.
@@ -1271,9 +1305,11 @@ impl Core {
local_root: &Path,
rel: &str,
uid: &NodeUid,
) -> Result<(), String> {
) -> Result<Applied, String> {
let path = local_root.join(rel_to_path(rel));
let file = std::fs::File::open(&path).map_err(|e| format!("open {rel}: {e}"))?;
let Some(file) = open_to_upload(&path, rel)? else {
return Ok(Applied::Vanished);
};
let meta = file.metadata().map_err(|e| format!("stat {rel}: {e}"))?;
let mtime = system_mtime(&meta);
let streamed = LocalSig::from(&meta);
@@ -1289,13 +1325,14 @@ impl Core {
meta.len(),
),
);
self.client
self.drive
.upload_new_revision_from(uid, reader, meta.len() as i64, thumbnails, Some(mtime))
.await
.map_err(|e| format!("upload revision {rel}: {e}"))?;
warn_if_torn(rel, &path, streamed);
self.record_file_baseline(folder_id, rel, Some(streamed), uid)
.await
.await?;
Ok(Applied::Uploaded)
}
/// Download a remote file to its local path (atomically via a temp file),
@@ -1305,7 +1342,7 @@ impl Core {
/// rename below is the destructive step, and the download in between can run
/// for minutes, so the destination is re-examined right before it: a local
/// file that no longer matches the plan is set aside as a conflict copy
/// instead of being replaced (bugs.md B39).
/// instead of being replaced (`docs/BUGS.md` B39).
#[allow(clippy::too_many_arguments)]
async fn download_file(
&self,
@@ -1328,7 +1365,21 @@ impl Core {
.map(|d| d.as_nanos())
.unwrap_or(0)
));
{
// A file an on-demand mount already read, say one just moved here from
// it, is in the content cache under the same revision key.
let seeded = size > 0 && self.cache.copy_cached_to(uid, mtime, size as u64, &tmp);
if seeded {
std::fs::File::open(&tmp)
.and_then(|f| f.sync_all())
.map_err(|e| {
let _ = std::fs::remove_file(&tmp);
format!("sync tmp {rel}: {e}")
})?;
debug!(
rel,
"sync: placed from the content cache instead of downloading"
);
} else {
let out = std::fs::File::create(&tmp).map_err(|e| format!("create tmp {rel}: {e}"))?;
// Count the bytes as they land, so the download reports progress like
// an on-demand hydration does.
@@ -1339,7 +1390,7 @@ impl Core {
size.max(0) as u64,
);
let mut out = CountingWriter::new(out, &guard);
self.client
self.drive
.download_file_to(uid, &mut out)
.await
.map_err(|e| {
@@ -1390,7 +1441,7 @@ impl Core {
/// the pre-upload identity is what makes the next pass see the path as
/// locally changed and upload a whole, clean revision over it. Recording the
/// post-upload stat instead declared the torn revision synchronized and left
/// it as the file's content on Drive (bugs.md B40).
/// it as the file's content on Drive (`docs/BUGS.md` B40).
async fn record_file_baseline(
&self,
folder_id: i64,
@@ -1404,7 +1455,7 @@ impl Core {
let (lmtime, lsize) = (local.mtime, local.size);
// Re-fetch the node so the stored remote signature is exactly what a walk
// will report next time.
let (rmtime, rsize) = match self.client.enumerate_nodes(std::slice::from_ref(uid)).await {
let (rmtime, rsize) = match self.drive.enumerate_nodes(std::slice::from_ref(uid)).await {
Ok(nodes) => nodes
.first()
.map(remote_file_sig)
@@ -1431,13 +1482,30 @@ impl Core {
}
}
/// Open a local file for upload, or `None` when it is gone.
///
/// A scan plans uploads from a listing, and the file can be deleted before its
/// upload starts — a build or a test tree cleaned up mid-pass. That is not a
/// failure: the next pass sees the deletion. Counting it as one put the folder
/// in `error` for a file nobody wanted any more (B108).
fn open_to_upload(path: &Path, rel: &str) -> Result<Option<std::fs::File>, String> {
match std::fs::File::open(path) {
Ok(file) => Ok(Some(file)),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
debug!(rel, "sync: file deleted before its upload; skipping");
Ok(None)
}
Err(e) => Err(format!("open {rel}: {e}")),
}
}
/// Say so when a file changed under an upload that was streaming it.
///
/// Purely diagnostic — the correction is already in place, because the baseline
/// records the identity the upload *read* rather than the one on disk now, so
/// the next pass classifies the path as locally changed and replaces the torn
/// revision. Without the log there is nothing in the journal to explain the
/// extra revision (bugs.md B40).
/// extra revision (`docs/BUGS.md` B40).
fn warn_if_torn(rel: &str, path: &Path, streamed: LocalSig) {
let now = local_sig(path);
if !now.is_some_and(|now| now.same_content(&streamed)) {
@@ -1469,7 +1537,7 @@ fn local_sig(path: &Path) -> Option<LocalSig> {
/// transfer started, and a download of a large file can run for minutes. Without
/// this, a local edit completed inside that window was replaced by the remote
/// version with no copy of it kept anywhere — the one shape of loss a sync
/// engine must not have (bugs.md B39). Keeping the copy makes it re-upload as a
/// engine must not have (`docs/BUGS.md` B39). Keeping the copy makes it re-upload as a
/// new file on the next pass, which is the same resolution a both-sides-changed
/// conflict already gets.
///
@@ -1505,7 +1573,7 @@ fn keep_racing_local_edit(path: &Path, expected: Option<LocalSig>) -> std::io::R
/// exists, so the same removal was re-attempted and re-reported as an error on
/// every pass, forever. Every other error is still an error, and keeping the
/// baseline is exactly right there: the surviving file must not be mistaken for
/// a new local creation on the next pass and uploaded back (bugs.md B42).
/// a new local creation on the next pass and uploaded back (`docs/BUGS.md` B42).
fn removed_locally(result: std::io::Result<()>) -> std::io::Result<()> {
match result {
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
@@ -1602,6 +1670,76 @@ fn walk_local_tree(
Ok(())
}
/// Why the mirror copy at `rel` holds something Proton Drive does not, or
/// `None` when everything under it is on Drive exactly as it is on disk.
///
/// A move out of a mirror folder removes the local copy once Drive has moved
/// the node, so anything only this disk has would be lost with it: a file not
/// uploaded yet, one edited since, a deletion not carried up yet, or anything
/// the engine never uploads. That is why ignored names count here, and why the
/// walk is its own rather than [`walk_local_tree`], which skips symlinks.
pub(crate) fn mirror_subtree_unsynced(
root: &Path,
rel: &str,
baseline: &HashMap<String, StoredSyncEntry>,
) -> Option<String> {
let mut seen = HashSet::new();
if let Some(reason) = unsynced_local(root, rel, baseline, &mut seen) {
return Some(reason);
}
let prefix = format!("{rel}/");
baseline
.keys()
.filter(|r| *r == rel || r.starts_with(&prefix))
.find(|r| !seen.contains(r.as_str()))
.map(|r| format!("{r} was deleted here but not on Proton Drive yet"))
}
fn unsynced_local(
root: &Path,
rel: &str,
baseline: &HashMap<String, StoredSyncEntry>,
seen: &mut HashSet<String>,
) -> Option<String> {
let path = root.join(rel_to_path(rel));
let meta = match std::fs::symlink_metadata(&path) {
Ok(meta) => meta,
Err(e) => return Some(format!("cannot read {rel}: {e}")),
};
let Some(base) = baseline.get(rel) else {
return Some(format!("{rel} is not on Proton Drive yet"));
};
seen.insert(rel.to_string());
if meta.is_file() {
return (!LocalSig::from(&meta).same_content(&LocalSig::from(base)))
.then(|| format!("{rel} has changes that are not on Proton Drive yet"));
}
if !meta.is_dir() {
return Some(format!("{rel} is not a regular file or folder"));
}
let entries = match std::fs::read_dir(&path) {
Ok(entries) => entries,
Err(e) => return Some(format!("cannot read {rel}: {e}")),
};
for entry in entries {
let name = match entry {
Ok(entry) => entry.file_name(),
Err(e) => return Some(format!("cannot read {rel}: {e}")),
};
let Some(name) = name.to_str() else {
return Some(format!("{rel} holds a name that is not valid UTF-8"));
};
// A download the engine left half done; it is the engine's to clean up.
if name.contains(".pdfs-tmp-") {
continue;
}
if let Some(reason) = unsynced_local(root, &format!("{rel}/{name}"), baseline, seen) {
return Some(reason);
}
}
None
}
/// Publish a conflict copy without overwriting an earlier conflict bearing the
/// same timestamp. The original is removed only after the new copy is durable.
fn preserve_conflict_copy(path: &Path, stamp: i64) -> std::io::Result<PathBuf> {
@@ -1692,7 +1830,7 @@ fn system_mtime(meta: &std::fs::Metadata) -> i64 {
///
/// Whole seconds cannot distinguish an edit made in the same second as the last
/// sync from no edit at all, and if the size did not change either, that file
/// was never uploaded (bugs.md B25). `None` for a clock or filesystem that
/// was never uploaded (`docs/BUGS.md` B25). `None` for a clock or filesystem that
/// cannot answer, which falls the comparison back to seconds rather than
/// asserting an mtime of zero.
fn system_mtime_ns(meta: &std::fs::Metadata) -> Option<i64> {
@@ -1707,7 +1845,7 @@ mod tests {
/// A removal of something already gone has produced the state the sync
/// engine wanted, so its baseline is dropped rather than retried forever.
/// Anything else stays an error and keeps the baseline (bugs.md B42).
/// Anything else stays an error and keeps the baseline (`docs/BUGS.md` B42).
#[test]
fn an_already_absent_path_counts_as_removed() {
let root = std::env::temp_dir().join(format!("pdfs-b42-{}", std::process::id()));
@@ -1724,7 +1862,7 @@ mod tests {
}
/// The download guard: a destination that still matches the plan is
/// replaced silently, one that has moved on since is kept (bugs.md B39).
/// replaced silently, one that has moved on since is kept (`docs/BUGS.md` B39).
#[test]
fn a_download_keeps_a_local_edit_it_did_not_plan_for() {
let root = std::env::temp_dir().join(format!("pdfs-b39-{}", std::process::id()));
@@ -2039,7 +2177,7 @@ mod tests {
std::fs::remove_dir_all(&root).unwrap();
}
/// A restored folder (features.md 5.2) starts with an empty baseline against
/// A restored folder starts with an empty baseline against
/// an empty local directory and a full remote. That must reconcile as
/// "download everything" rather than tripping the wipe guard — otherwise
/// every restore would wedge its folder on the first pass.
@@ -2155,6 +2293,27 @@ mod tests {
assert_eq!(remote_sig(&e), None);
}
#[test]
fn a_file_deleted_before_its_upload_is_skipped_not_failed() {
let root = std::env::temp_dir().join(format!("pdfs-b108-{}", std::process::id()));
std::fs::create_dir_all(&root).unwrap();
let gone = root.join("gone.txt");
assert!(open_to_upload(&gone, "gone.txt").unwrap().is_none());
std::fs::write(&gone, b"x").unwrap();
assert!(open_to_upload(&gone, "gone.txt").unwrap().is_some());
// Other open failures still count as errors.
let err = open_to_upload(&gone.join("child"), "gone.txt/child");
assert!(err.unwrap_err().starts_with("open gone.txt/child:"));
let _ = std::fs::remove_dir_all(&root);
let mut o = Outcome::default();
o.record(&Applied::Vanished);
assert!(o.is_empty());
assert_eq!(o.errors, 0);
}
#[test]
fn outcome_summarises_only_what_it_moved() {
let mut o = Outcome::default();
@@ -2346,6 +2505,94 @@ mod tests {
);
}
/// A baseline row recording `path` as it is on disk right now.
fn synced_row(root: &Path, rel: &str) -> (String, StoredSyncEntry) {
let meta = std::fs::symlink_metadata(root.join(rel_to_path(rel))).unwrap();
let sig = LocalSig::from(&meta);
let entry = StoredSyncEntry {
rel_path: rel.to_string(),
remote_uid: Some(format!("vol~{rel}")),
local_mtime: sig.mtime,
local_mtime_ns: sig.mtime_ns,
local_size: sig.size,
remote_rev: None,
remote_hash: None,
};
(rel.to_string(), entry)
}
fn synced_subtree(name: &str) -> (PathBuf, HashMap<String, StoredSyncEntry>) {
let root = sync_test_dir(name);
std::fs::create_dir_all(root.join("dir/sub")).unwrap();
std::fs::write(root.join("dir/a.txt"), b"a").unwrap();
std::fs::write(root.join("dir/sub/b.txt"), b"b").unwrap();
let baseline = ["dir", "dir/sub", "dir/a.txt", "dir/sub/b.txt"]
.into_iter()
.map(|rel| synced_row(&root, rel))
.collect();
(root, baseline)
}
#[test]
fn a_fully_synced_mirror_subtree_may_leave() {
let (root, baseline) = synced_subtree("move-synced");
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert_eq!(reason, None);
}
#[test]
fn a_mirror_subtree_with_a_file_not_uploaded_may_not_leave() {
let (root, baseline) = synced_subtree("move-new-file");
std::fs::write(root.join("dir/sub/new.txt"), b"n").unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/sub/new.txt"));
}
#[test]
fn a_mirror_subtree_with_an_edit_not_uploaded_may_not_leave() {
let (root, baseline) = synced_subtree("move-edit");
std::fs::write(root.join("dir/a.txt"), b"longer now").unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/a.txt"));
}
#[test]
fn a_mirror_subtree_with_a_deletion_not_carried_up_may_not_leave() {
let (root, baseline) = synced_subtree("move-deleted");
std::fs::remove_file(root.join("dir/sub/b.txt")).unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/sub/b.txt"));
}
#[test]
fn a_mirror_subtree_holding_a_symlink_may_not_leave() {
let (root, baseline) = synced_subtree("move-symlink");
std::os::unix::fs::symlink("a.txt", root.join("dir/link")).unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/link"));
}
#[test]
fn a_name_prefixed_sibling_does_not_hold_a_mirror_subtree_back() {
let (root, mut baseline) = synced_subtree("move-sibling");
// Deleted here but not on Drive, yet outside the subtree being moved.
baseline.insert(
"dirt".to_string(),
StoredSyncEntry {
rel_path: "dirt".to_string(),
..baseline["dir/a.txt"].clone()
},
);
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert_eq!(reason, None);
}
/// The same rule one level up: if the sync root itself cannot be read, the
/// pass sees nothing, and "nothing" must never mean "delete everything".
#[test]
+1 -1
View File
@@ -33,7 +33,7 @@ pub(crate) enum SyncMsg {
/// Re-read the folder set and adjust the filesystem watches (after add/remove).
Rewatch,
/// The daemon is tearing down: finish the current pass and end the engine
/// thread, so the mount can join it (bugs.md B44).
/// thread, so the mount can join it (`docs/BUGS.md` B44).
Stop,
}
+7 -2
View File
@@ -20,7 +20,7 @@ impl LocalSig {
/// Nanoseconds decide it when both sides have them; otherwise the comparison
/// falls back to whole seconds, which is what a baseline row written before
/// schema 28 — or a filesystem that reports no sub-second time — can support
/// (bugs.md B25).
/// (`docs/BUGS.md` B25).
pub(super) fn same_content(&self, other: &Self) -> bool {
if self.size != other.size {
return false;
@@ -108,6 +108,7 @@ impl Outcome {
Applied::Uploaded => self.uploaded += 1,
Applied::Downloaded => self.downloaded += 1,
Applied::Conflict => self.conflicts += 1,
Applied::Vanished => {}
}
}
@@ -160,7 +161,7 @@ pub(super) enum Pending {
/// was there. A download can run for minutes, so the apply step revalidates
/// against it immediately before it renames over the destination: a writer
/// that finished an edit in the meantime must not be overwritten without a
/// conflict copy (bugs.md B39).
/// conflict copy (`docs/BUGS.md` B39).
Download {
rel: String,
uid: NodeUid,
@@ -226,6 +227,10 @@ pub(super) enum PassAbort {
Interrupted,
/// The pass could not establish its diff (a walk or the baseline load failed).
Failed(String),
/// The daemon is stopping. What the pass sent before the stop has landed or
/// will be cancelled with the runtime; nothing more is sent, and the next
/// start's pass picks up from the baseline.
Stopped,
}
impl From<String> for PassAbort {
+2 -2
View File
@@ -218,7 +218,7 @@ pub(super) fn plan_file(
}
let local_changed = local.is_some_and(|item| {
// Nanoseconds where both sides have them (bugs.md B25): whole seconds
// Nanoseconds where both sides have them (`docs/BUGS.md` B25): whole seconds
// could not see an edit made in the same second as the last sync that
// left the file the same length, so it was never uploaded.
baseline.is_none_or(|base| !LocalSig::from(item).same_content(&LocalSig::from(base)))
@@ -291,7 +291,7 @@ mod file_plan_tests {
/// Whole seconds cannot see an edit made inside the same second that leaves the
/// file the same length, so a mirror silently stopped uploading it. With a
/// nanosecond time on both sides it is an ordinary local change (bugs.md B25).
/// nanosecond time on both sides it is an ordinary local change (`docs/BUGS.md` B25).
#[test]
fn a_same_second_same_size_edit_is_still_a_local_change() {
let mut base = baseline(10, 20, 30, 40);
+1 -1
View File
@@ -530,7 +530,7 @@ fn open_archive(
/// The lowercase-hex SHA-1 of everything `reader` yields — the digest Proton's
/// photo duplicate detection is keyed on.
fn sha1_of(reader: &mut (dyn Read + Send)) -> CoreResult<String> {
pub(crate) fn sha1_of(reader: &mut (dyn Read + Send)) -> CoreResult<String> {
let mut hasher = Sha1::new();
let mut buffer = vec![0u8; 256 * 1024];
loop {
+4 -7
View File
@@ -73,7 +73,7 @@ async fn run_uploads(
.upload_thumbnails_async(t.path.clone(), t.name.clone())
.await;
match core
.client
.drive
.upload_file_from(
&t.parent_uid,
&t.name,
@@ -238,14 +238,11 @@ impl Core {
.map_err(|error| self.errno_error(error, "upload destination"))?;
let new_uid = self
.rt
.block_on(
self.client
.create_folder(parent_uid, name, Some(now_secs())),
)
.block_on(self.drive.create_folder(parent_uid, name, Some(now_secs())))
.map_err(|e| CoreError::from_api(&e, &format!("create folder {name}")))?;
let node = self
.fetch_node(&new_uid)
.map_err(|e| self.errno_error(e, "fetch node"))?;
.fetch_minted_node(&new_uid, name)
.ok_or_else(|| minted_but_unlisted(name))?;
let mut st = self.state();
let ino = st.intern(pino, node);
if let Some(kids) = st.children.get_mut(&pino)
+4
View File
@@ -35,5 +35,9 @@ adw = { package = "libadwaita", version = "0.9", features = ["v1_5"] }
async-channel = "2"
exif = { version = "0.6", package = "kamadak-exif" }
webkit6 = "0.6"
# Town and street names on the Places map, with font fallback for every script.
pangocairo = "0.22"
# Map tiles for the opt-in online map. The same build proton-sdk already pulls in.
reqwest = { version = "0.13", default-features = false, features = ["rustls", "webpki-roots"] }
# The system libintl (glibc) rather than a bundled gettext build.
gettext-rs = { version = "0.7", features = ["gettext-system"] }
+2
View File
@@ -26,5 +26,7 @@ fn main() {
println!("cargo:rerun-if-changed=resources/pdfs.gresource.xml");
println!("cargo:rerun-if-changed=resources/icons");
println!("cargo:rerun-if-changed=resources/style.css");
println!("cargo:rerun-if-changed=resources/proton-theme.css");
println!("cargo:rerun-if-changed=resources/prompt.css");
println!("cargo:rerun-if-changed=../../packaging/io.narl.proton-drive-linux.svg");
}
+393
View File
@@ -0,0 +1,393 @@
47.98,8.00 48.94,9.45 48.94,11.26
47.98,8.00 46.42,6.50 44.94,4.91 43.99,4.95 43.58,4.85 43.13,4.64 42.86,4.32 42.02,4.14 41.88,3.98
34.25,31.21 34.53,31.53 34.48,31.58
35.55,32.40 35.19,32.53 35.07,32.46 34.96,32.16 34.95,31.84 35.20,31.75 34.95,31.60 34.87,31.40 34.91,31.35 35.10,31.37 35.45,31.48
35.11,33.08 35.41,33.08 35.49,33.12 35.53,33.25 35.60,33.24
35.79,32.73 35.89,32.94 35.79,33.29 35.84,33.42
34.00,35.07 33.87,35.09 33.48,35.00 33.38,35.16 33.19,35.17 32.92,35.09 32.71,35.17
77.80,35.50 77.89,35.45 78.04,35.48 78.01,35.25 78.33,34.61 78.67,34.52 78.97,34.30 78.97,34.23 78.73,34.01 78.80,33.50 78.92,33.39
77.05,35.11 77.00,34.99 76.59,34.74 76.04,34.67 75.71,34.50 75.32,34.58
91.63,27.76 91.98,27.73 92.10,27.81 92.55,27.88 92.69,27.99 92.64,28.06 92.70,28.15 93.17,28.36 93.36,28.65 93.90,28.74 93.97,28.86 94.27,28.97 94.29,29.14 94.62,29.31 94.77,29.18 95.39,29.04 95.52,29.21 95.71,29.24 95.77,29.34 96.13,29.38 96.23,29.25 96.36,29.25 96.18,29.12 96.20,28.96 96.44,29.05 96.58,28.76 96.28,28.41 96.39,28.37 96.60,28.46 96.78,28.37 97.08,28.37 97.32,28.22
128.37,38.62 128.17,38.36 128.04,38.31 127.09,38.28 126.67,37.92 126.63,37.78
-8.68,27.66 -8.82,27.66 -8.75,27.19 -8.79,27.12 -9.41,27.09 -9.82,26.85 -10.03,26.91 -10.25,26.86 -10.76,27.02 -11.39,26.88 -11.32,26.74 -11.34,26.63 -11.51,26.47 -11.72,26.10 -12.06,25.99 -12.43,24.83 -12.99,24.47 -13.31,23.98 -13.89,23.69 -14.10,23.10 -14.22,22.31 -14.63,21.86 -14.61,21.75 -14.75,21.50 -14.97,21.44 -16.04,21.50 -17.00,21.42
35.92,4.62 35.76,4.81 35.80,5.16 35.75,5.34 35.47,5.42 35.33,5.36 35.27,5.49
-124.76,48.49 -123.86,48.26 -123.39,48.25 -123.14,48.40 -123.24,48.64 -123.01,48.79 -123.31,48.99
91.63,27.76 91.64,27.92 91.27,28.08 91.02,27.97 90.33,28.12
23.61,51.52 23.61,51.61 23.71,51.64 23.98,51.59 24.36,51.87 25.79,51.92 27.14,51.75 27.30,51.60 27.69,51.57 27.70,51.48 27.86,51.59 28.18,51.61 28.60,51.54 28.73,51.43 29.10,51.63 29.35,51.38 30.16,51.48 30.33,51.33 30.54,51.27 30.63,51.36 30.53,51.60 30.76,51.90 31.08,52.08 31.76,52.10
35.84,33.42 35.60,33.24
35.84,33.42 36.03,33.59 35.94,33.67 35.99,33.75 36.09,33.83 36.37,33.84 36.28,33.93 36.58,34.22 36.50,34.43 36.33,34.50 36.43,34.61 36.38,34.66 35.98,34.63
34.68,-12.01 34.76,-12.03 34.68,-12.12 34.68,-12.01
80.21,42.19 80.26,42.24 80.16,42.63 80.20,42.73 80.54,42.87 80.38,42.97 80.39,43.04 80.62,43.13 80.75,43.10 80.79,43.16 80.67,43.31 80.70,43.43 80.65,43.56 80.36,44.10 80.36,44.55 80.38,44.66 80.48,44.71 80.26,44.81 80.00,44.80 79.87,44.88 80.06,45.01 80.85,45.13 81.69,45.35 81.79,45.23 81.94,45.16 82.27,45.22 82.52,45.13 82.62,45.29 82.61,45.42 82.33,45.52 82.32,45.59 82.97,46.97 83.03,47.19 84.02,46.97 84.67,46.97 84.79,46.83 85.23,47.04 85.48,47.06 85.66,47.25 85.53,47.92 85.75,48.39 86.55,48.53 86.72,48.70 86.73,48.94 86.81,49.05 87.05,49.11 87.32,49.09
23.48,53.94 24.19,53.95 24.32,53.89 24.77,53.97 24.87,54.15 25.05,54.13 25.46,54.29 25.51,54.16 25.75,54.16 25.75,54.26 25.55,54.33 25.72,54.56 25.72,54.72 25.86,54.92 26.18,55.00 26.29,55.14 26.60,55.13 26.78,55.27 26.46,55.34 26.59,55.67
-53.37,-33.74 -53.53,-33.66 -53.53,-33.17 -53.13,-32.74 -53.16,-32.68 -53.60,-32.40 -53.76,-32.06 -54.22,-31.86 -54.59,-31.49 -54.90,-31.39 -55.04,-31.28 -55.17,-31.28 -55.56,-30.88 -55.63,-30.86 -55.81,-31.03
116.68,49.82 116.03,48.78 115.82,48.58 115.79,48.25 115.53,48.13 115.62,47.87 115.90,47.69 116.23,47.86 116.76,47.87 117.07,47.81 117.35,47.65 117.77,47.99 118.15,48.03 118.50,47.98 118.76,47.76 119.08,47.65 119.16,47.53 119.71,47.15 119.90,46.86 119.87,46.67 119.71,46.61 119.33,46.61 118.84,46.76 118.72,46.69 118.31,46.72 118.07,46.67 117.74,46.52 117.44,46.59 117.33,46.36 116.86,46.39 116.56,46.29 116.21,45.89 116.20,45.74 115.68,45.46 114.56,45.39 114.42,45.20 114.08,44.97 113.59,44.75 112.71,44.88 112.41,45.06 111.90,45.06 111.62,44.83 111.40,44.37 111.60,44.11 111.84,43.93 111.93,43.71 111.55,43.50 111.01,43.34 110.40,42.77 109.34,42.44 108.17,42.45 106.77,42.29 105.20,41.74 104.98,41.60 104.50,41.66 104.50,41.88 103.71,41.75 103.07,42.01 102.16,42.16 101.97,42.22 101.66,42.50 101.50,42.54 99.98,42.68 99.47,42.57 97.21,42.79 96.39,42.72 96.30,42.93 95.86,43.28 95.53,43.95 95.33,44.04 95.35,44.28 95.05,44.26 94.71,44.35 94.20,44.65 93.96,44.67 93.52,44.94 92.79,45.04 91.58,45.08 91.05,45.22 90.88,45.20 90.66,45.53 90.71,45.73 91.00,46.04 90.91,46.27 91.03,46.57 90.87,46.95 90.72,47.00 90.50,47.29 90.33,47.66 90.10,47.75 90.03,47.88 89.78,47.83 89.48,48.03 89.05,48.00 88.58,48.22 88.52,48.38 87.98,48.56 88.06,48.71 87.74,48.88 87.87,49.00 87.81,49.16
13.81,48.77 13.55,48.96 13.40,48.98 12.92,49.33 12.68,49.41 12.39,49.74 12.51,49.90 12.21,50.10 12.09,50.30 12.28,50.18 12.55,50.39 12.94,50.41 13.02,50.49 13.44,50.60 13.56,50.70 14.37,50.90 14.26,51.00 14.32,51.04 14.55,50.99 14.66,50.83 14.81,50.86
27.35,57.53 26.97,57.61 26.46,57.54 25.99,57.84 25.28,58.05 25.26,58.00 25.11,58.06 24.32,57.87
27.35,57.53 27.51,57.51 27.54,57.43 27.83,57.29 27.64,56.85 27.85,56.85 28.10,56.55 28.20,56.26 28.15,56.14
21.05,56.07 21.65,56.31 22.08,56.41 22.88,56.40 23.04,56.32 23.20,56.37 24.12,56.26 24.47,56.28 24.84,56.41 25.07,56.20 25.66,56.10 26.28,55.75 26.59,55.67
20.62,69.04 20.12,69.02 20.35,68.85 20.24,68.67 19.97,68.54 20.24,68.48 19.97,68.36 18.30,68.56 18.16,68.53 18.18,68.20 17.92,67.96 17.32,68.10 16.78,67.90 16.57,67.62 16.19,67.51 16.13,67.43 16.43,67.16 16.40,67.05 15.42,66.49 15.48,66.31 15.04,66.17 14.54,66.13 14.63,65.79 14.48,65.30 13.65,64.58 14.08,64.46 14.14,64.17 13.96,64.01 13.20,64.08 12.79,64.00 12.18,63.60 12.21,63.49 12.00,63.29 12.22,63.00 12.11,62.92 12.11,62.59 12.30,62.29 12.16,61.72 12.88,61.35 12.71,61.06 12.29,61.00 12.59,60.45 12.49,60.11 12.17,59.91 11.93,59.86 11.83,59.70 11.68,59.59 11.80,59.29 11.64,58.93 11.47,58.91 11.34,59.09 11.24,59.10 11.14,59.09 11.06,58.96 10.91,58.95
20.62,69.04 21.07,69.04 21.13,69.08 21.07,69.21 21.59,69.27 22.30,68.86 22.41,68.72 23.32,68.65 23.71,68.71 23.85,68.81 24.94,68.59 25.09,68.64 25.25,68.82 25.58,68.89 25.75,68.99 25.77,69.28 26.07,69.69 26.53,69.92 27.13,69.91 27.75,70.06 27.89,70.06 28.41,69.82 29.14,69.67 29.33,69.47 28.85,69.18 28.83,69.12 28.97,69.02
104.43,10.41 104.56,10.52 104.85,10.53 105.05,10.70 105.05,10.91 105.31,10.85 105.41,10.95 105.76,10.99 105.85,10.86 106.16,10.79 106.16,11.04 105.86,11.29 105.85,11.64 106.01,11.76 106.40,11.69 106.41,11.95 106.70,11.98 106.76,12.05 106.93,12.08 107.21,12.30 107.33,12.32 107.39,12.26 107.51,12.36 107.56,12.54 107.48,13.03 107.61,13.44 107.33,14.13 107.36,14.37 107.49,14.55 107.52,14.71
-69.51,-17.50 -69.31,-17.94 -69.09,-18.05 -69.15,-18.14 -68.97,-18.97 -68.46,-19.43 -68.70,-19.72 -68.56,-19.97 -68.60,-20.04 -68.76,-20.09 -68.69,-20.31 -68.76,-20.42 -68.48,-20.63 -68.56,-20.72 -68.56,-20.90 -68.44,-20.95 -68.20,-21.30 -68.19,-21.62 -68.08,-21.98 -67.99,-22.06 -67.88,-22.49 -67.88,-22.82 -67.58,-22.89 -67.19,-22.82
-69.51,-17.51 -69.85,-17.70 -69.80,-17.99 -69.93,-18.21 -70.18,-18.33 -70.42,-18.35
6.34,49.45 6.49,49.80 6.20,49.92 6.11,50.03 6.12,50.12
2.52,51.10 2.60,50.88 2.76,50.75 2.84,50.71 3.11,50.78 3.23,50.66 3.27,50.53 3.60,50.48 3.69,50.31 3.95,50.34 4.17,50.25 4.15,49.97 4.55,49.96 4.82,50.15 4.86,50.14 4.79,49.96 4.87,49.79 5.28,49.68 5.51,49.51 5.79,49.54
6.12,50.12 5.98,50.17 5.74,49.92 5.72,49.81 5.88,49.64 5.79,49.54
46.43,41.89 45.95,42.04 45.64,42.21 45.71,42.50 45.34,42.53 45.16,42.68 44.87,42.76 44.77,42.62 44.64,42.73 44.51,42.75 43.83,42.57 43.74,42.62 43.78,42.75 43.09,42.99 42.99,43.09 42.76,43.17 42.57,43.16 42.42,43.22 41.58,43.22 40.65,43.53 40.15,43.57 39.98,43.42
20.96,40.85 20.93,40.90 20.71,40.93 20.49,41.27 20.45,41.52 20.52,41.57 20.57,41.87
20.57,41.87 20.58,42.01 20.49,42.22 20.24,42.34 20.06,42.55
41.51,41.52 41.82,41.43 41.93,41.50 42.47,41.44 42.61,41.58 42.75,41.58 43.15,41.31 43.21,41.20 43.40,41.18 43.44,41.11
101.14,21.57 101.20,21.52 101.25,21.20 101.54,21.23 101.70,21.15 101.80,21.21 101.72,21.31 101.74,21.83 101.60,21.99 101.52,22.25 101.74,22.50 101.84,22.39 102.02,22.44 102.13,22.38
73.63,39.45 73.82,39.49 73.91,39.58 73.84,39.80 73.99,40.04 74.41,40.14 74.68,40.31 74.83,40.33 74.84,40.48 75.11,40.45 75.56,40.63 75.68,40.31 75.87,40.30 76.26,40.43 76.32,40.35 76.48,40.45 76.66,40.78 76.91,41.02 77.58,40.99 78.12,41.08 78.44,41.42 79.29,41.78 79.77,41.90 79.84,42.00 80.22,42.03 80.21,42.19
43.44,41.11 44.23,41.21 44.84,41.21 44.81,41.26 45.00,41.29
80.21,42.19 79.92,42.41 79.43,42.48 79.13,42.78 76.99,42.97 75.84,42.94 75.64,42.81 75.37,42.84 74.82,42.98 74.21,43.24 73.95,43.20 73.56,43.00 73.42,42.59 73.49,42.41 72.28,42.76 71.76,42.82 71.26,42.73 70.89,42.34 70.95,42.25
-56.48,1.94 -56.84,1.88 -57.12,2.01 -57.32,1.96 -57.55,1.73 -57.98,1.65 -58.03,1.52 -58.34,1.59 -58.40,1.48 -58.51,1.44 -58.51,1.28 -58.68,1.28 -58.82,1.20 -59.23,1.38 -59.54,1.70 -59.67,1.75 -59.67,1.84 -59.76,1.90 -59.76,2.27 -59.89,2.36 -59.99,2.69 -59.97,2.99 -59.83,3.35 -59.85,3.59 -59.68,3.70 -59.55,3.93 -59.74,4.23 -59.70,4.38 -59.83,4.48 -60.15,4.53 -60.03,4.74 -59.99,5.08 -60.14,5.24 -60.74,5.20
8.39,55.10 8.59,54.92 9.25,54.81 9.74,54.83
9.52,30.23 9.90,30.39 10.22,30.78 10.26,30.94 10.11,31.46 10.27,31.68 10.48,31.74 10.61,31.93 10.83,32.08 11.51,32.41 11.53,32.52 11.45,32.64 11.50,33.18
45.00,41.29 45.19,41.15 45.07,41.08 45.42,40.99 45.59,40.85 45.38,40.67 45.45,40.53 45.57,40.42 45.96,40.23 45.89,40.02 45.58,39.98 46.20,39.59 46.48,39.56 46.48,39.48 46.37,39.40 46.58,39.22 46.40,39.19 46.49,39.07 46.49,38.91
44.77,39.70 45.03,39.77 45.17,39.57 45.25,39.60 45.46,39.49 45.78,39.55 45.77,39.38 45.98,39.24 45.95,39.18 46.11,38.88
20.24,46.11 20.51,46.17 20.61,46.13 20.76,46.25 21.12,46.28 21.26,46.41 21.32,46.61 21.50,46.70 21.66,47.04 22.00,47.40 22.00,47.51 22.29,47.73 22.61,47.77 22.88,47.95
70.96,40.24 71.30,40.29 71.63,40.22 71.69,40.15 71.96,40.26 72.13,40.44 72.36,40.40 72.40,40.58 72.60,40.53 73.14,40.81 72.66,40.87 72.36,41.04 72.19,41.03 72.16,41.17 71.88,41.20 71.86,41.31 71.66,41.54 71.60,41.50 71.59,41.33 71.42,41.34 71.39,41.12 70.86,41.22 70.69,41.45 70.47,41.41 70.20,41.51 70.18,41.57 70.63,41.88 70.80,41.92 70.86,42.03 71.23,42.16 71.04,42.28 70.95,42.25
18.83,49.51 18.94,49.50 18.97,49.40 19.15,49.40 19.44,49.60 19.63,49.41 19.77,49.37 19.76,49.20 20.06,49.18 20.16,49.32 20.36,49.39 20.62,49.39 20.87,49.31 21.08,49.42 21.64,49.41 21.89,49.34 22.02,49.21 22.54,49.07
-7.22,55.09 -7.38,55.03 -7.55,54.77 -7.91,54.70 -7.75,54.59 -8.12,54.48 -8.12,54.41 -7.85,54.22 -7.61,54.14 -7.32,54.13 -7.16,54.24 -7.20,54.30 -7.01,54.41 -6.80,54.21 -6.65,54.16 -6.65,54.06 -6.22,54.09
18.83,49.51 18.81,49.61 18.59,49.76 18.56,49.88 18.03,50.04 17.87,49.97 17.63,50.12 17.59,50.16 17.74,50.23 17.70,50.31 17.42,50.25 16.88,50.43 16.99,50.24 16.64,50.10 16.21,50.42 16.42,50.57 16.28,50.66 16.01,50.61 15.73,50.74 15.35,50.81 15.26,50.96 14.99,51.01 14.98,50.89 14.81,50.86
70.96,40.24 70.95,40.19 70.64,40.08 70.62,40.00 70.52,39.95 70.45,40.05 69.97,40.20 69.53,40.10 69.48,39.92 69.31,39.97 69.28,39.92 69.23,39.76 69.30,39.52 70.50,39.59 70.80,39.39 71.00,39.41 71.07,39.49 71.47,39.60 71.50,39.48 71.73,39.42 71.73,39.31 71.78,39.28 72.04,39.35 72.23,39.21 72.36,39.34 72.56,39.38 73.11,39.36 73.63,39.45
20.00,39.71 20.25,39.68 20.31,39.80 20.38,39.80 20.31,39.98 20.66,40.12 20.81,40.45 20.95,40.49 21.03,40.62 20.96,40.85
33.20,-14.01 33.15,-13.94 32.99,-14.02 32.87,-13.82 32.77,-13.76 32.80,-13.69 32.67,-13.59 32.85,-13.46 32.97,-13.22 33.00,-12.90 32.95,-12.80 33.02,-12.63 33.40,-12.49 33.51,-12.35 33.34,-12.31 33.25,-12.11 33.30,-11.69 33.23,-11.42 33.38,-11.16 33.26,-10.89 33.46,-10.78 33.66,-10.55 33.53,-10.23 33.31,-10.04 33.35,-9.86 33.20,-9.63 33.00,-9.62 32.92,-9.41
22.13,48.41 22.14,48.57 22.30,48.69 22.54,49.07
-11.51,6.91 -11.27,7.23 -10.65,7.76 -10.57,8.07 -10.36,8.19 -10.28,8.49
24.15,8.67 24.22,8.61 24.21,8.37 24.29,8.29 24.85,8.14 25.20,7.81 25.25,7.72 25.18,7.56 25.28,7.43 25.89,7.06 26.36,6.64 26.31,6.46 26.51,6.07 27.14,5.72 27.33,5.19
42.92,11.00 43.25,11.50
42.92,11.00 42.56,11.08 41.96,10.94 41.80,10.98 41.79,11.69 42.38,12.47
29.40,-4.45 29.32,-4.90 29.61,-5.72 29.48,-6.03 29.54,-6.31 29.71,-6.62 30.21,-7.04 30.75,-8.19
42.38,12.47 42.48,12.51 42.70,12.38 42.87,12.62 43.12,12.71
13.81,48.77 14.05,48.60 14.69,48.60 14.79,48.75 14.92,48.77 14.97,48.98 15.07,49.00 15.83,48.86 16.06,48.75 16.37,48.74 16.54,48.80 16.88,48.70 16.95,48.60
42.36,37.11 42.77,37.37 42.94,37.32 43.09,37.37 43.68,37.23 44.11,37.30 44.19,37.25 44.20,37.05 44.28,36.98 44.61,37.18 44.77,37.14
16.09,46.86 16.25,46.97 16.45,47.01 16.49,47.12 16.42,47.22 16.44,47.40 16.62,47.45 16.68,47.54 16.64,47.61 16.42,47.67 16.59,47.75 16.79,47.68 17.07,47.71 17.03,47.84 17.15,48.01
7.02,45.93 7.13,45.88 7.54,45.98 7.79,45.92 7.99,46.02 8.13,46.16 8.10,46.27 8.42,46.45 8.46,46.25 8.64,46.11 8.82,46.08 8.78,46.00 8.95,45.83 9.02,45.85 9.00,46.01 9.25,46.29 9.26,46.48 9.43,46.48 9.53,46.31 9.94,46.36 10.04,46.24 10.13,46.24 10.04,46.48 10.09,46.60 10.43,46.55 10.40,46.67 10.45,46.86
10.45,46.86 10.35,46.98 10.13,46.85 9.88,46.94 9.85,47.01 9.58,47.06
44.77,37.14 44.79,37.29 44.57,37.44 44.55,37.66 44.59,37.71 44.21,37.91 44.45,38.33 44.30,38.39 44.27,38.84 44.17,38.93 44.18,39.14 44.08,39.22 44.02,39.38 44.39,39.42 44.46,39.67 44.59,39.77 44.82,39.65
23.48,53.94 23.45,54.14 23.37,54.20 22.89,54.39 22.77,54.36
30.75,-8.19 30.89,-8.47 31.03,-8.60 31.35,-8.61 31.53,-8.71 31.67,-8.91 31.92,-8.94 31.94,-9.05 32.43,-9.16 32.92,-9.41
7.20,53.28 7.19,53.00 7.03,52.65 6.71,52.62 6.75,52.46 6.97,52.44 7.04,52.38 6.98,52.21 6.72,52.08 6.80,51.97 6.52,51.85 6.36,51.82 6.17,51.88 5.95,51.80 6.20,51.45 6.08,51.22 6.13,51.15 5.86,51.03 6.05,50.90 5.99,50.75
9.58,47.06 9.61,47.11 9.53,47.27
-8.49,7.56 -8.43,7.60 -8.23,7.56 -8.01,8.08 -8.05,8.17 -8.26,8.25 -8.24,8.46 -8.05,8.50 -7.82,8.47 -7.74,8.38 -7.68,8.41 -7.72,8.64 -7.95,8.79 -7.94,8.98 -7.78,9.08 -7.92,9.19 -7.90,9.42 -8.03,9.40 -8.14,9.50 -8.16,9.97 -7.99,10.16
7.49,43.77 7.48,43.86 7.68,44.08 7.64,44.16 7.32,44.14 6.90,44.34 6.84,44.51 7.03,44.72 6.99,44.83 6.74,44.92 6.63,45.07 6.69,45.14 6.84,45.14 7.08,45.24 7.15,45.38 6.79,45.74 6.80,45.81 7.02,45.93
28.97,69.02 28.41,68.90 28.77,68.84 28.48,68.54 28.56,68.35 28.69,68.19 29.34,68.06 29.99,67.67 29.94,67.55 29.24,67.10 29.07,66.89 29.90,66.09 30.09,65.79 30.10,65.68 29.72,65.62 29.82,65.57 29.73,65.47 29.71,65.34 29.61,65.25 29.81,65.20 29.83,65.15 29.62,65.04 29.60,64.97 29.78,64.80 30.11,64.73 30.12,64.64 29.99,64.52 30.11,64.37 30.49,64.24 30.53,64.08 30.21,63.80 29.99,63.74 31.18,63.21 31.53,62.89 31.19,62.48 29.25,61.29 28.41,60.90 27.73,60.46
30.87,69.78 30.92,69.61 30.86,69.54 30.62,69.53 30.18,69.64 30.19,69.54 30.09,69.43 29.39,69.30 29.21,69.10 28.97,69.02
55.19,22.70 55.20,23.03 55.53,23.82 55.47,23.94 55.99,24.06 55.93,24.22 55.76,24.24 55.80,24.87 55.92,24.97 56.00,24.95 55.97,24.86 56.06,24.74 56.39,24.98
48.57,41.84 48.39,41.60 48.06,41.46 47.79,41.20 47.32,41.28 47.14,41.52 46.75,41.81 46.57,41.80 46.54,41.87 46.43,41.89
-54.62,2.33 -54.72,2.44 -54.88,2.45 -54.98,2.60 -55.39,2.44 -55.73,2.41 -55.96,2.52 -56.14,2.26 -55.92,2.04 -55.93,1.89 -56.02,1.84 -56.48,1.94
46.43,41.89 46.20,41.74 46.18,41.66 46.31,41.51 46.67,41.29 46.53,41.09 46.43,41.08 46.17,41.20 45.92,41.19 45.73,41.26 45.72,41.34 45.28,41.45 45.00,41.29
-11.39,12.40 -11.45,12.56 -11.39,12.94 -11.63,13.37 -11.76,13.39 -11.83,13.32 -12.05,13.63 -11.96,13.88 -12.02,13.97 -12.02,14.21 -12.23,14.46 -12.19,14.65 -12.28,14.81
-54.62,2.33 -54.13,2.12 -53.77,2.35 -53.51,2.25 -53.33,2.34 -53.23,2.20 -52.90,2.21 -52.58,2.53 -52.33,3.18 -51.99,3.70 -51.65,4.06
2.71,6.37 2.77,6.71 2.72,6.98 2.79,7.48 2.69,7.87 2.77,9.05 3.04,9.08 3.15,9.32 3.14,9.45 3.33,9.67 3.35,9.81 3.56,9.91 3.65,10.16 3.58,10.29 3.65,10.41 3.77,10.42 3.83,10.61 3.72,11.08 3.49,11.40 3.60,11.70
13.07,-4.63 12.80,-4.43 12.38,-4.62 12.31,-4.77 12.20,-4.78 12.02,-5.00
16.52,46.50 16.32,46.53 16.24,46.48 16.23,46.37 16.07,46.37 15.64,46.20 15.59,46.14 15.67,46.05 15.65,45.86 15.28,45.73 15.35,45.66 15.28,45.58 15.34,45.47 14.79,45.48 14.57,45.66 14.37,45.48 13.99,45.51 13.88,45.43 13.58,45.52
18.91,45.93 18.67,45.91 18.53,45.80 18.36,45.75 17.81,45.79 17.31,46.00 16.87,46.34 16.52,46.50
20.24,46.11 19.61,46.17 19.28,46.00 19.09,46.02 18.91,45.93
16.52,46.50 16.28,46.86 16.09,46.86
51.27,24.61 50.97,24.57 50.80,24.79
25.26,-17.79 25.24,-17.97 25.94,-18.94 25.95,-19.08 26.17,-19.54 26.68,-19.89 27.18,-20.10 27.26,-20.23 27.28,-20.48 27.68,-20.50 27.67,-21.06 28.01,-21.55 29.03,-21.80 29.04,-22.02 29.36,-22.19
77.80,35.50 77.45,35.48 76.77,35.66
28.01,41.97 27.53,41.92 27.24,42.09 27.01,42.06 26.58,41.95 26.51,41.83 26.36,41.80 26.32,41.72
102.93,11.71 102.74,12.09 102.76,12.43 102.50,12.67 102.46,13.02 102.33,13.29 102.34,13.56 102.55,13.59 102.54,13.66 102.91,14.14 103.20,14.33 103.60,14.42 103.98,14.36 104.78,14.43 104.97,14.37 105.00,14.25 105.07,14.23 105.18,14.35
88.89,27.32 88.74,27.18 88.86,26.96 89.15,26.82 89.33,26.85 89.55,26.80 89.61,26.72 90.12,26.75 90.35,26.89 90.74,26.77 91.29,26.79 91.43,26.87 91.67,26.80 92.05,26.87 92.07,26.98 91.99,27.10 92.08,27.29 91.99,27.45 91.74,27.44 91.59,27.56 91.63,27.76
104.07,1.28 104.06,1.40 103.82,1.46 103.70,1.44 103.60,1.21
100.12,6.44 100.18,6.67 100.26,6.68 100.35,6.55 100.75,6.46 100.87,6.25 101.05,6.24 101.08,5.96 100.98,5.77 101.11,5.64 101.26,5.79 101.56,5.91 101.68,5.78 101.87,5.83 102.10,6.24
5.99,50.75 6.24,50.60 6.17,50.55 6.20,50.50 6.34,50.45 6.36,50.32 6.18,50.23 6.12,50.12
-71.78,19.72 -71.71,19.49 -71.75,19.29 -71.65,19.16 -71.81,18.99 -71.74,18.73 -71.87,18.61 -72.00,18.60 -71.76,18.34 -71.77,18.04
14.06,13.08 13.61,13.70
46.53,29.10 46.91,29.54 47.15,30.00 47.33,30.08 47.64,30.10 47.98,29.98
88.11,27.87 88.14,27.95 88.58,28.09 88.80,28.01 88.85,27.87 88.75,27.52 88.89,27.32
-90.10,13.74 -90.05,13.90 -89.55,14.24 -89.57,14.39 -89.36,14.42
124.04,-9.34 124.09,-9.42 124.28,-9.43 124.41,-9.31 124.44,-9.19
124.92,-8.94 124.94,-9.05 125.10,-9.00 125.15,-9.04 125.10,-9.19 124.96,-9.21 125.07,-9.51
115.14,4.90 115.27,4.63 115.29,4.35 115.11,4.39 115.03,4.90
109.63,2.03 109.54,1.90 109.64,1.78 109.65,1.61 110.51,0.86 111.10,1.05 111.81,1.01 111.92,1.11 112.08,1.14 112.19,1.44 112.48,1.56 112.94,1.57 113.01,1.43 113.62,1.24 113.90,1.43 114.39,1.50 114.51,1.45 114.70,1.85 114.83,1.98 114.76,2.16 114.79,2.25 115.18,2.52 115.08,2.63 115.12,2.89 115.25,3.03 115.45,3.03 115.57,3.45 115.57,3.94 115.68,4.19 115.84,4.33 116.02,4.29 116.24,4.36 116.41,4.31 116.51,4.37 117.10,4.34 117.54,4.17 117.88,4.19
10.45,46.86 10.99,46.78 11.24,46.98 11.78,46.99 12.17,47.08 12.15,46.94 12.39,46.70 13.70,46.52
13.70,46.52 13.38,46.26 13.63,46.18 13.48,46.01 13.61,45.96 13.58,45.81 13.87,45.61 13.72,45.59
29.01,-2.72 29.10,-2.60 29.30,-2.67 29.39,-2.81 29.70,-2.79 29.87,-2.72 29.93,-2.34 30.12,-2.42 30.41,-2.31 30.55,-2.40
92.57,21.98 92.63,21.31 92.57,21.26 92.33,21.44 92.18,21.29 92.32,20.79
1.43,42.60 1.29,42.71 0.70,42.85 0.63,42.69 -0.04,42.69 -0.30,42.83 -0.59,42.80 -0.76,42.94 -1.18,43.02 -1.30,43.10 -1.46,43.05 -1.41,43.24 -1.71,43.31 -1.79,43.41
1.43,42.60 1.50,42.64 1.71,42.60 1.71,42.50
1.71,42.50 2.03,42.35 2.20,42.42 2.65,42.34 2.89,42.46 3.21,42.43
1.43,42.60 1.45,42.44 1.71,42.50
74.89,37.23 75.12,37.39 74.89,37.60 74.94,37.77 74.78,38.19 74.81,38.46 74.28,38.66 74.13,38.66 74.03,38.54 73.80,38.61 73.70,38.85 73.81,38.97 73.61,39.23 73.63,39.45
74.89,37.23 74.73,37.29 74.37,37.16 74.54,37.02
20.96,40.85 21.40,40.91 21.58,40.87 21.99,41.13 22.60,41.14 22.72,41.18 22.78,41.33 22.92,41.34
21.56,42.25 21.39,42.22 21.29,42.10 21.06,42.17 20.78,42.07 20.72,41.87 20.57,41.87
22.34,42.31 22.24,42.36 21.56,42.25
22.34,42.31 22.52,42.44 22.44,42.63 22.47,42.84 22.71,42.88 22.98,43.19 22.55,43.45 22.37,43.78 22.42,44.01 22.60,44.08 22.63,44.19 22.71,44.24
22.92,41.34 23.00,41.74 22.84,41.99 22.58,42.10 22.34,42.31
18.44,42.56 18.52,42.43
18.44,42.56 17.80,42.90 17.67,42.90
78.92,33.39 79.12,33.23
20.34,42.83 20.47,42.86 20.48,42.95 20.62,43.03 20.62,43.20 20.80,43.26 20.84,43.17 21.06,43.09 21.40,42.83 21.39,42.75 21.75,42.67 21.61,42.39 21.52,42.33 21.56,42.25
20.34,42.83 20.05,42.76 20.06,42.55
19.19,43.53 19.22,43.45 19.61,43.17 20.27,42.94 20.34,42.83
-89.36,14.42 -89.12,14.37 -88.71,14.03 -88.51,13.98 -88.48,13.85 -88.15,13.99 -87.99,13.88 -87.80,13.89 -87.72,13.81 -87.78,13.52 -87.74,13.45 -87.81,13.40
-87.34,12.98 -87.01,13.01 -86.92,13.22 -86.73,13.28 -86.76,13.75 -86.33,13.77 -86.04,14.05 -85.98,13.97 -85.73,13.86 -85.68,13.98 -85.18,14.34 -85.16,14.53 -85.04,14.61 -84.99,14.75 -84.86,14.81 -84.65,14.66 -84.45,14.64 -84.24,14.75 -84.15,14.72 -83.64,14.88 -83.54,14.98 -83.16,14.99
-75.28,-0.11 -75.78,0.09 -76.07,0.35 -76.27,0.44 -76.39,0.40 -76.43,0.26 -76.49,0.24 -76.83,0.25 -77.40,0.39 -77.47,0.64 -77.65,0.72 -77.70,0.84 -77.83,0.83 -78.18,0.97 -78.86,1.46
-57.60,3.37 -57.65,3.52 -57.83,3.68 -58.03,4.00 -58.05,4.17 -57.85,4.67 -57.92,4.82 -57.88,4.88 -57.71,4.99 -57.33,5.02 -57.21,5.20 -57.32,5.34 -57.19,5.55
-54.08,3.33 -53.99,3.59 -54.35,4.05 -54.48,4.84 -54.45,5.01 -54.16,5.36
-58.41,-33.94 -58.48,-33.65 -58.38,-33.14 -58.24,-33.11 -58.13,-32.98 -58.20,-32.47
-57.61,-30.19 -57.87,-30.59 -57.81,-30.86 -57.90,-30.98 -57.89,-31.20 -58.05,-31.49 -57.99,-31.58 -58.01,-31.68 -58.19,-31.92 -58.12,-32.25 -58.20,-32.47
-54.62,-25.58 -54.44,-25.62 -54.15,-25.52 -53.89,-25.67 -53.67,-26.29 -53.75,-26.75 -53.72,-26.88 -53.84,-27.12 -54.16,-27.25 -54.33,-27.42 -54.83,-27.55 -54.91,-27.71 -55.10,-27.87 -55.73,-28.20 -55.69,-28.38 -55.89,-28.37 -56.94,-29.59 -57.22,-29.78 -57.41,-30.03 -57.61,-30.19
20.62,69.04 20.90,68.98 20.92,68.91 22.00,68.52 22.85,68.37 23.10,68.26 23.18,68.14 23.32,68.13 23.64,67.95 23.50,67.88 23.54,67.61 23.45,67.46 23.73,67.42 23.77,67.33 23.63,67.23 23.64,67.13 23.99,66.81 23.67,66.38 23.72,66.22 23.91,66.15 24.16,65.81 24.19,65.61 24.12,65.51
4.23,51.39 4.17,51.31 3.90,51.21 3.58,51.29 3.43,51.25 3.35,51.38
5.99,50.75 5.75,50.76 5.64,50.84 5.75,50.95 5.80,51.15 5.48,51.29 5.21,51.28 5.03,51.47 4.85,51.40 4.76,51.49 4.63,51.42 4.50,51.47 4.38,51.43 4.37,51.36 4.23,51.39
6.34,49.45 6.53,49.39 6.74,49.16 6.89,49.21 7.04,49.11 7.45,49.15 7.61,49.06 8.13,48.97 8.14,48.89 7.84,48.64 7.62,48.16 7.53,47.67 7.62,47.59
9.52,47.52 9.18,47.67 8.88,47.66 8.57,47.78 8.40,47.69 8.57,47.65 8.45,47.60 7.62,47.59
9.53,47.27 9.63,47.47 9.52,47.52
9.58,47.06 9.49,47.06 9.53,47.27
7.02,45.93 6.77,46.17 6.82,46.28 6.76,46.42 6.43,46.43 6.23,46.33 6.27,46.25 6.20,46.19 6.01,46.14 5.97,46.21 6.10,46.28 6.12,46.38 6.07,46.46 6.16,46.61 6.41,46.76 6.46,46.95 6.67,47.03 7.00,47.32 6.90,47.39 6.97,47.45 7.14,47.49 7.34,47.43 7.62,47.59
-7.41,37.18 -7.50,37.59 -7.44,37.73 -7.19,38.01 -7.02,38.04 -6.96,38.19 -7.11,38.18 -7.34,38.46 -7.28,38.71 -7.05,38.91 -7.00,39.06 -7.17,39.14 -7.34,39.47 -7.54,39.66 -7.12,39.68 -6.98,39.80 -6.90,40.02 -7.03,40.17 -6.81,40.34 -6.84,40.78 -6.93,41.01 -6.21,41.53 -6.31,41.64 -6.54,41.67 -6.62,41.94 -7.15,41.98 -7.21,41.90 -7.40,41.83 -7.92,41.88 -8.15,41.81 -8.22,41.90 -8.13,42.02 -8.27,42.14 -8.78,41.94
13.81,48.77 13.79,48.59 13.72,48.54 13.49,48.58 13.37,48.36 12.76,48.11 12.95,47.89 12.90,47.72 13.05,47.66 13.01,47.48 12.81,47.54 12.77,47.64 12.69,47.67 12.48,47.64 12.21,47.72 12.19,47.62 11.72,47.58 11.04,47.39 10.87,47.52 10.44,47.55 10.37,47.37 10.18,47.28 10.20,47.36 10.07,47.39 9.97,47.51 9.75,47.58 9.52,47.52
16.09,46.86 15.98,46.80 15.96,46.68 15.77,46.71 15.44,46.63 14.89,46.61 14.55,46.40 13.70,46.52
18.44,42.56 18.55,42.64 18.47,42.78 18.46,43.00 18.62,43.03 18.67,43.23 18.85,43.35 19.03,43.29 18.95,43.53 19.19,43.53
19.19,43.53 19.30,43.59 19.45,43.56 19.50,43.64 19.25,43.97 19.58,44.01 19.12,44.36 19.35,44.88 18.84,44.88 18.66,45.08 18.00,45.14 17.81,45.08 17.65,45.16 17.50,45.12 17.13,45.17 16.92,45.28 16.79,45.20 16.53,45.22 16.29,45.01 16.03,45.19 15.79,45.18 15.74,44.77 16.10,44.52 16.30,44.12 17.08,43.52 17.27,43.45 17.29,43.31 17.65,43.01 17.59,42.94
19.01,44.87 19.10,44.97 19.06,45.14 19.14,45.20 19.40,45.21 19.00,45.40 19.06,45.52 18.92,45.60 18.95,45.66 18.84,45.84 18.91,45.93
16.95,48.60 17.14,48.84 17.76,48.89 18.09,49.07 18.16,49.26 18.60,49.49 18.83,49.51
17.15,48.01 16.87,48.39 16.95,48.60
17.15,48.01 17.32,47.99 17.76,47.77 18.72,47.79 18.79,48.00 19.47,48.11 19.63,48.22 19.95,48.15 20.33,48.30 20.49,48.53 21.45,48.55 21.72,48.35 22.13,48.41
14.81,50.86 15.02,51.25 14.91,51.46 14.72,51.52 14.74,51.63 14.60,51.83 14.75,52.08 14.55,52.36 14.62,52.53 14.13,52.88 14.41,53.22 14.21,53.95
22.88,47.95 22.77,48.11 22.58,48.13 22.35,48.26 22.25,48.41 22.13,48.41
28.59,43.74 28.22,43.77 28.05,43.82 27.88,43.99 27.74,43.96 27.43,44.02 27.09,44.17 26.22,44.01 25.82,43.77 25.50,43.67 23.22,43.87 22.92,43.83 22.87,43.95 23.03,44.08 22.71,44.24
20.24,46.11 20.71,45.74 20.77,45.75 20.79,45.47 21.49,45.15 21.35,45.01 21.53,44.90 21.36,44.83
22.92,41.34 23.64,41.39 24.01,41.46 24.06,41.53 24.49,41.56 24.77,41.36 24.99,41.36 25.25,41.24 25.92,41.31 26.14,41.39 26.07,41.67 26.11,41.73 26.41,41.70 26.58,41.60 26.62,41.40 26.33,41.24 26.33,40.95 26.04,40.73
20.06,42.55 19.79,42.48 19.73,42.63 19.65,42.63 19.28,42.17 19.36,42.07 19.34,41.87
26.62,48.26 26.79,48.26 26.98,48.16 27.61,47.34 28.07,46.98 28.24,46.64 28.24,46.45 28.10,45.97 28.16,45.65 28.07,45.60 28.21,45.45
22.88,47.95 23.20,48.08 23.41,47.99 24.18,47.91 24.58,47.93 24.89,47.72 24.98,47.72 25.46,47.91 26.16,47.99 26.31,48.20 26.62,48.26
22.54,49.07 22.81,49.02 22.85,49.06 22.71,49.17 22.73,49.30 22.65,49.54 22.71,49.61 23.71,50.38 23.97,50.41 24.09,50.53 23.98,50.79 24.10,50.87 23.66,51.31 23.61,51.52
20.90,55.29 21.39,55.28 22.07,55.06 22.57,55.06 22.82,54.87 22.68,54.56 22.77,54.36
31.76,52.10 31.58,52.31 31.62,52.55 31.52,52.70 31.56,52.76 31.26,53.02 31.42,53.20 31.67,53.20 31.85,53.11 32.14,53.09 32.70,53.34 32.71,53.42 32.47,53.55 32.45,53.69 32.20,53.78 31.75,53.81 31.83,54.03 31.40,54.20 31.18,54.45 31.07,54.49 31.15,54.63 30.80,54.78 30.83,54.91 30.98,55.05 30.81,55.31 30.90,55.40 30.91,55.57 30.23,55.85 29.94,55.85 29.48,55.68 29.35,55.78 29.38,55.94 29.03,56.02 28.79,55.94 28.56,56.09 28.28,56.06 28.15,56.14
31.76,52.10 32.12,52.05 32.28,52.11 32.44,52.31 32.81,52.25 33.15,52.34 33.74,52.34 33.92,52.25 34.11,51.98 34.40,51.78 34.38,51.72 34.12,51.68 34.27,51.34 34.21,51.26 34.71,51.17 35.06,51.20 35.16,51.06 35.31,51.04 35.44,50.73 35.41,50.54 35.59,50.37 35.67,50.35 35.89,50.44 36.12,50.41 36.31,50.28 36.50,50.28 36.62,50.21 36.76,50.29 37.42,50.41 37.70,50.11 38.05,49.92 38.26,50.05 38.92,49.82 39.17,49.86 39.30,49.74 39.46,49.73 39.78,49.57 40.08,49.58 40.11,49.25 39.89,49.06 39.69,49.01 39.75,48.91 40.00,48.82 39.79,48.81 39.64,48.59 39.84,48.54 39.89,48.36 39.85,48.30 39.96,48.27 39.78,47.89 39.66,47.84 38.90,47.86 38.64,47.67 38.37,47.61 38.29,47.56 38.20,47.32 38.28,47.28 38.20,47.18 38.21,47.09
43.44,41.11 43.63,40.93 43.72,40.72 43.57,40.48 43.71,40.17 43.67,40.13 44.01,40.01 44.29,40.04 44.78,39.68
87.82,49.16 88.12,49.26 88.19,49.45 88.83,49.45 88.90,49.54 89.01,49.47 89.11,49.50 89.24,49.63 89.40,49.61 89.65,49.72 89.64,49.90 89.98,49.98 90.05,50.09 90.66,50.22 91.02,50.42 91.42,50.47 91.80,50.69 92.19,50.70 92.35,50.86 92.63,50.69 92.78,50.78 92.94,50.78 93.10,50.60 94.25,50.56 94.35,50.22 94.61,50.02 94.93,50.04 95.11,49.94 95.52,49.91 95.79,50.01 95.94,49.96 96.07,50.00 96.32,49.90 96.99,49.88 97.21,49.73 97.36,49.74 97.59,49.91 98.00,50.01 98.25,50.30 98.28,50.53 98.03,50.64 97.84,51.05 98.04,51.45 98.22,51.51 98.35,51.72 98.64,51.80 98.89,52.12 99.72,51.87 99.92,51.76 100.47,51.73 101.38,51.45 102.11,51.35 102.22,50.83 102.32,50.72 102.29,50.59 102.68,50.39 103.16,50.29 103.30,50.20 103.63,50.14 104.08,50.15 104.47,50.31 105.38,50.47 106.22,50.30 106.71,50.31 107.23,49.99 107.92,49.95 107.97,49.65 108.61,49.32 109.24,49.33 110.71,49.14 111.34,49.36 112.08,49.42 112.49,49.53 112.81,49.52 113.06,49.62 113.16,49.80 113.57,50.01 114.30,50.27 114.74,50.23 115.43,49.90 115.72,49.88 116.22,50.01 116.55,49.92 116.68,49.82
116.68,49.82 117.25,49.62 117.87,49.51 118.45,49.84 119.26,50.07 119.35,50.28 119.16,50.41 119.45,50.70 120.07,51.60 120.51,51.85 120.75,52.10 120.67,52.30 120.70,52.49 120.66,52.57 120.17,52.60 120.07,52.63 120.04,52.72 120.70,53.17 120.99,53.28 122.34,53.49 123.61,53.55 124.81,53.13 125.08,53.20 125.65,53.04 125.73,52.89 125.87,52.87 126.05,52.74 126.02,52.61 126.34,52.36 126.51,51.93 126.70,51.70 126.71,51.57 126.81,51.51 126.92,51.10 127.31,50.71 127.34,50.35 127.59,50.21 127.49,49.98 127.55,49.80 127.71,49.67 128.00,49.57 128.70,49.60 128.82,49.46 129.07,49.37 129.50,49.39 130.20,48.89 130.55,48.86 130.62,48.77 130.55,48.60 130.80,48.34 130.71,48.13 130.73,48.02 130.92,47.84 130.96,47.71 132.48,47.71 132.71,47.95 133.14,48.11 133.47,48.10 133.84,48.27 134.29,48.37 134.67,48.25 134.67,48.15 134.57,48.02 134.75,47.72 134.54,47.49 134.29,47.41 134.17,47.30 134.20,47.13 133.87,46.50 133.86,46.25 133.70,46.14 133.65,45.96 133.51,45.88 133.44,45.60 133.19,45.49 133.11,45.13 132.94,45.03 131.98,45.24 131.85,45.33 131.45,44.98 130.98,44.84 131.26,44.07 131.17,43.70 131.18,43.51 131.26,43.38 131.07,42.90 130.58,42.81 130.42,42.73 130.58,42.62 130.53,42.54
130.53,42.54 130.25,42.74 130.24,42.89 129.90,43.00 129.70,42.45 129.52,42.38 129.31,42.41 129.20,42.22 128.92,42.04 128.05,41.99 128.03,41.90 128.29,41.61 128.15,41.39 128.01,41.45 127.69,41.44 127.18,41.53 126.95,41.77 126.72,41.72 126.60,41.64 126.49,41.36 125.99,40.90 125.73,40.87 125.42,40.66 124.89,40.46 124.39,40.10 124.36,40.00
130.53,42.54 130.69,42.30
70.96,40.24 70.65,40.20 70.37,40.38 70.38,40.45 70.75,40.72 70.40,41.04 70.29,40.89 69.71,40.66 69.41,40.80 69.36,40.77 69.31,40.63 69.21,40.57 69.30,40.33 69.22,40.29 69.27,40.20 68.95,40.22 68.63,40.17 68.97,40.09 68.80,40.05 68.87,39.91 68.64,39.84 68.59,39.63 68.46,39.54 67.72,39.62 67.43,39.47 67.36,39.22 67.65,39.13 67.69,38.99 68.04,38.98 68.13,38.93 68.05,38.67 68.09,38.47 68.35,38.21 68.29,38.03 67.81,37.49 67.76,37.17
35.89,35.92 36.15,35.83 36.20,35.94 36.35,36.00 36.38,36.17 36.64,36.23 36.54,36.46 36.66,36.80 36.94,36.76 37.07,36.65 37.44,36.64 38.19,36.90 38.44,36.86 38.77,36.69 39.36,36.68 40.02,36.83 40.71,37.10 41.52,37.09 42.06,37.21 42.20,37.30 42.31,37.23 42.36,37.11
35.45,31.48 35.40,31.23 35.44,31.13 35.14,30.42 35.14,30.14 34.97,29.56
35.55,32.40 35.56,31.77 35.45,31.48
35.61,32.68 35.55,32.40
88.11,27.87 87.14,27.84 86.61,28.10 86.52,27.96 86.41,27.93 86.14,28.11 86.08,28.08 86.06,27.93 85.99,27.91 85.68,28.28 85.12,28.32 85.09,28.37 85.16,28.59 84.80,28.56 84.23,28.91 84.10,29.22 83.94,29.28 83.58,29.18 83.16,29.61 82.85,29.68 82.22,30.06 82.04,30.33 81.64,30.39 81.42,30.34 81.18,30.04 81.11,30.04 81.01,30.16
97.32,28.22 97.34,27.94 96.88,27.59 96.90,27.44 97.10,27.12 96.95,27.13 96.73,27.33 96.19,27.26 95.46,26.76 95.13,26.60 95.05,26.35 95.13,26.04 95.02,25.91 94.95,25.70 94.58,25.32 94.55,25.22 94.71,25.05 94.29,24.32 94.13,23.88 93.68,24.01 93.49,23.97 93.33,24.06 93.41,23.68 93.39,23.34 93.35,23.08 93.31,23.03 93.16,23.03 93.08,22.72 93.15,22.23 93.04,22.18 92.96,22.00 92.85,22.01 92.69,22.13 92.57,21.98
100.12,20.32 99.89,20.42 99.72,20.33 99.46,20.36 99.49,20.15 99.34,20.08 99.07,20.10 98.92,19.77 98.37,19.69 98.02,19.75 97.82,19.46 97.80,19.13 97.71,18.93 97.75,18.59 97.52,18.50 97.37,18.52 97.45,18.36 97.63,18.29 97.74,17.94 97.71,17.80 98.44,16.98 98.48,16.73 98.66,16.33 98.84,16.42 98.89,16.35 98.82,16.18 98.59,16.05 98.56,15.37 98.19,15.20 98.20,14.98 98.25,14.81 98.57,14.36 98.93,14.05 99.14,13.72 99.18,13.23 99.12,13.03 99.22,12.74 99.41,12.55 99.61,11.78 99.19,11.11 98.76,10.66 98.70,10.19
103.09,18.14 103.09,18.14
103.09,18.14 103.26,18.28 103.29,18.41 103.95,18.32 104.43,17.70 104.74,17.46 104.82,17.30 104.74,16.88 104.82,16.47 105.05,16.16 105.41,15.99 105.40,15.83 105.64,15.66 105.49,15.26 105.55,14.93 105.50,14.59 105.42,14.47 105.18,14.35
100.12,20.32 100.14,20.25 100.32,20.39 100.52,20.18 100.54,20.09 100.40,19.76 100.51,19.55 100.74,19.51 100.97,19.61 101.21,19.55 101.20,19.33 101.29,18.98 101.05,18.44 101.14,18.14 100.91,17.58 100.96,17.54 101.17,17.50 101.69,17.89 101.77,18.03 101.95,18.08 102.10,18.21 102.55,17.97 102.66,17.82 102.81,17.95 103.05,18.03 103.09,18.14
101.14,21.57 101.08,21.47 100.70,21.25 100.54,20.99 100.52,20.92 100.62,20.86 100.25,20.73 100.12,20.32
97.32,28.22 97.54,28.51 97.60,28.52 98.06,28.19 98.30,27.55 98.45,27.66 98.65,27.57 98.74,26.79 98.69,26.19 98.56,26.07 98.66,25.86 98.47,25.79 98.33,25.59 98.14,25.57 98.01,25.29 97.82,25.25 97.71,25.03 97.74,24.87 97.58,24.77 97.53,24.49 97.67,24.38 97.71,24.23 97.56,23.91 97.69,23.90 98.21,24.11 98.58,24.07 98.84,24.12 98.68,23.91 98.83,23.62 98.80,23.52 98.88,23.38 98.86,23.19 99.42,23.07 99.51,22.96 99.39,22.83 99.34,22.50 99.21,22.28 99.19,22.13 99.92,22.03 99.94,21.76 100.10,21.66 100.15,21.48 100.60,21.47 101.08,21.76 101.13,21.74 101.14,21.57
102.13,22.38 102.58,21.90 102.66,21.68 102.74,21.68 102.82,21.81 102.95,21.68 102.85,21.27 103.10,20.89 103.64,20.70 104.10,20.95 104.58,20.65 104.37,20.44 104.62,20.37 104.70,20.21 104.85,20.20 104.93,20.02 104.59,19.62 104.03,19.68 104.06,19.48 103.89,19.30 104.72,18.80 105.15,18.65 105.11,18.41 105.33,18.19 105.46,18.15 105.69,17.74 106.50,16.95 106.55,16.65 106.66,16.49 106.74,16.45 106.85,16.52 106.93,16.35 107.40,16.04 107.39,15.95 107.17,15.80 107.65,15.26 107.48,14.98 107.52,14.71
27.40,5.11 27.07,5.20 26.82,5.06 25.53,5.31 25.40,5.26 25.25,5.02 25.07,4.97 24.77,4.93 24.32,4.99 23.42,4.66 23.12,4.74 22.86,4.72 22.71,4.59 22.42,4.13 21.69,4.28 21.54,4.24 20.56,4.46 20.23,4.83 19.81,5.09 19.50,5.13 19.07,4.89 18.83,4.52 18.59,4.35 18.61,3.48
30.55,-2.40 30.42,-2.64 30.47,-2.69 30.43,-2.87 30.78,-2.98 30.79,-3.27 30.63,-3.35 30.63,-3.42 30.43,-3.59 30.15,-4.09 29.95,-4.31 29.72,-4.46 29.40,-4.45
29.01,-2.72 29.02,-2.80 29.22,-3.05 29.21,-3.83 29.40,-4.45
29.58,-1.39 29.35,-1.52 29.20,-1.72 29.13,-1.86 29.13,-2.20 28.88,-2.40 28.89,-2.64 29.01,-2.72
35.27,5.49 34.98,5.86 34.71,6.66 34.06,7.23 33.90,7.51 33.67,7.67 33.23,7.76 33.00,7.90 33.28,8.44 33.95,8.44 34.07,8.55 34.08,9.46
22.86,10.92 22.49,11.00 22.04,10.82 21.77,10.64 21.71,10.57 21.68,10.29 21.53,10.21 21.40,10.00 21.26,9.97 20.77,9.41 20.34,9.13 20.07,9.13 19.67,9.02 19.15,9.02 18.96,8.94 18.89,8.84 19.11,8.66 18.56,8.05 17.65,7.98 16.78,7.55 16.55,7.87 16.40,7.77 16.38,7.68 15.96,7.51 15.48,7.52
13.29,2.16 13.22,2.26 11.56,2.30 11.35,2.30 11.33,2.17
11.13,-3.92 11.23,-3.69 11.50,-3.52 11.69,-3.68 11.88,-3.67 11.83,-3.53 11.93,-3.32 11.72,-3.18 11.76,-2.98 11.64,-2.86 11.54,-2.84 11.61,-2.34 11.73,-2.39 11.95,-2.34 12.06,-2.41 12.45,-2.33 12.43,-1.93 12.59,-1.83 12.79,-1.93 12.99,-2.31 13.46,-2.40 13.73,-2.14 13.88,-2.33 13.89,-2.47 13.99,-2.49 14.20,-2.35 14.16,-2.22 14.25,-2.00 14.38,-1.89 14.47,-0.57 14.21,-0.43 14.07,-0.27 13.86,-0.20 13.95,0.35 14.09,0.54 14.32,0.62 14.43,0.90 14.18,1.37 13.85,1.42 13.22,1.25 13.25,1.37 13.17,1.79 13.29,2.16
15.48,7.52 15.55,7.79 15.44,7.85 15.12,8.56 14.33,9.20 13.98,9.69 14.24,9.98 14.84,9.94 15.65,10.01 15.28,10.36 15.13,10.65 15.03,11.11 15.12,11.54 15.08,11.85 14.88,12.27 14.85,12.50 14.76,12.66 14.54,12.82 14.46,13.02 14.06,13.08
14.06,13.08 14.20,12.38 14.52,12.30 14.62,12.15 14.58,11.53 14.20,11.27 13.98,11.21 13.70,10.87 13.54,10.61 13.41,10.17 13.27,10.04 13.20,9.56 12.93,9.43 12.78,8.82 12.58,8.62 12.40,8.60 12.23,8.28 12.02,7.59 11.77,7.27 11.86,7.12 11.58,6.89 11.48,6.60 11.24,6.45 11.11,6.46 11.01,6.74 10.61,7.06 10.48,6.89 10.21,6.89 10.14,7.00 9.78,6.76 9.66,6.53 9.00,5.92 8.80,5.20 8.56,4.76
3.60,11.70 3.66,11.76 3.61,11.89 3.65,12.53 3.95,12.78 4.04,12.93 4.15,13.46 4.66,13.73 5.24,13.76 5.49,13.87 6.30,13.66 6.59,13.41 6.87,13.04 7.01,13.00 7.83,13.34 8.10,13.29 8.75,12.91 9.62,12.81 10.05,13.21 10.48,13.33 11.41,13.35 11.99,13.19 12.12,13.09 12.46,13.09 12.65,13.33 12.87,13.45 13.32,13.67 13.61,13.70
1.19,6.09 0.98,6.32 0.74,6.45 0.53,6.85 0.63,7.35 0.51,7.44 0.50,7.55 0.61,7.73 0.58,8.15 0.69,8.35 0.37,8.76 0.49,8.85 0.47,9.12 0.53,9.40 0.41,9.49 0.26,9.43 0.23,9.46 0.34,9.60 0.26,9.64 0.32,9.69 0.38,10.29 -0.09,10.67 0.01,11.02 -0.07,11.12
-2.70,9.48 -2.67,9.28 -2.75,9.05 -2.60,8.80 -2.51,8.21 -2.61,8.15 -2.61,8.05 -2.79,7.93 -2.99,7.20 -3.24,6.81 -3.24,6.65 -3.00,5.71 -2.96,5.64 -2.79,5.60 -2.76,5.36 -2.82,5.15 -3.11,5.09
-5.52,10.43 -5.46,10.77 -5.49,11.04 -5.35,11.13 -5.25,11.38 -5.29,11.83 -5.11,11.97 -4.70,12.08 -4.43,12.34 -4.42,12.58 -4.48,12.67 -4.23,12.79 -4.33,13.12 -4.15,13.31 -3.95,13.40 -3.53,13.18 -3.30,13.28 -3.25,13.66 -2.95,13.65 -2.87,13.95 -2.59,14.23 -2.46,14.27 -2.11,14.17 -1.97,14.46 -1.70,14.51 -1.05,14.82 -0.76,15.05 -0.54,15.08 -0.41,15.01 -0.24,15.06 0.22,14.91
-2.70,9.48 -2.82,9.43 -2.99,9.69 -3.22,9.90 -3.79,9.92 -4.27,9.74 -4.33,9.65 -4.63,9.71 -4.97,9.93 -5.10,10.24 -5.38,10.31 -5.52,10.43
-8.49,7.56 -8.30,6.98 -8.33,6.80 -8.60,6.51 -8.29,6.32 -7.89,6.23 -7.80,5.98 -7.45,5.84 -7.40,5.55 -7.43,5.32 -7.57,5.08 -7.59,4.82 -7.54,4.35
-8.49,7.56 -8.66,7.69 -8.89,7.26 -9.12,7.22 -9.26,7.38 -9.46,7.42 -9.37,7.70 -9.52,8.35 -9.68,8.48 -9.74,8.45 -9.78,8.54 -10.06,8.43 -10.15,8.52 -10.28,8.49
-5.52,10.43 -5.84,10.39 -6.03,10.19 -6.20,10.23 -6.26,10.72 -6.43,10.67 -6.42,10.56 -6.65,10.66 -6.69,10.35 -6.95,10.34 -7.02,10.14 -7.36,10.26 -7.50,10.44 -7.66,10.43 -7.81,10.24 -7.99,10.16
-15.04,10.94 -14.68,11.51 -14.27,11.66 -13.73,11.74 -13.74,12.01 -13.95,12.18 -13.71,12.31 -13.67,12.48 -13.73,12.67
-11.39,12.40 -11.50,12.20 -11.31,12.02 -10.93,12.21 -10.71,11.90 -10.47,12.14 -10.27,12.21 -9.75,12.03 -9.66,12.14 -9.36,12.26 -9.40,12.46 -9.22,12.48 -9.04,12.40 -8.82,11.92 -8.82,11.67 -8.40,11.37 -8.67,11.01 -8.47,11.05 -8.34,10.99 -8.27,10.49 -8.01,10.32 -7.99,10.16
107.52,14.71 107.41,14.56 107.29,14.59 106.94,14.33 106.78,14.34 106.50,14.58 106.35,14.45 106.23,14.48 106.17,14.37 105.98,14.34 106.12,14.05 106.07,13.92 105.90,13.92 105.74,14.08 105.53,14.16 105.35,14.11 105.18,14.35
114.06,4.60 114.22,4.48 114.32,4.26 114.42,4.26 114.57,4.05 114.65,4.04 114.78,4.17 114.84,4.39 114.75,4.72 115.03,4.90
-88.23,15.73 -88.96,15.15 -89.14,15.07 -89.22,14.87 -89.17,14.61 -89.36,14.42
-77.37,8.66 -77.48,8.50 -77.20,7.97 -77.35,7.84 -77.35,7.71 -77.54,7.57 -77.59,7.54 -77.76,7.70 -77.74,7.54 -77.90,7.23
-60.74,5.20 -61.39,5.94 -61.13,6.21 -61.20,6.59 -61.15,6.69 -60.87,6.79 -60.72,6.77 -60.35,7.00 -60.35,7.15 -60.58,7.16 -60.63,7.21 -60.61,7.32 -60.72,7.54 -60.51,7.81 -60.38,7.83 -60.03,8.05 -59.96,8.19 -59.85,8.25 -59.83,8.31 -60.02,8.55
-60.74,5.20 -60.67,5.16 -60.60,4.95 -61.00,4.54 -61.28,4.52 -61.55,4.29 -61.82,4.20 -62.15,4.10 -62.41,4.16 -62.71,4.02 -62.76,3.67 -62.86,3.59 -62.97,3.59 -63.34,3.94 -63.53,3.89 -64.02,3.93 -64.19,4.13 -64.58,4.14 -64.79,4.28 -64.82,4.23 -64.67,4.01 -64.22,3.59 -64.22,3.20 -64.01,2.67 -64.05,2.50 -63.39,2.41 -63.43,2.16 -64.01,1.93 -64.11,1.62 -64.21,1.53 -64.30,1.46 -64.49,1.45 -64.73,1.25 -65.10,1.11 -65.47,0.69 -65.56,0.69 -65.52,0.84 -65.68,0.98 -66.06,0.79 -66.35,0.77 -66.88,1.22
52.49,41.78 53.06,42.15 54.12,42.34 54.85,41.97 55.43,41.30 55.98,41.32
55.98,41.32 57.02,41.26 57.12,41.35 57.02,41.45 56.96,41.86 57.29,42.12 57.81,42.19 58.03,42.49 58.28,42.43 58.40,42.29 58.47,42.30 58.15,42.63 58.26,42.69 58.48,42.66 58.59,42.78 58.88,42.56 59.16,42.51 59.35,42.32 59.86,42.30 59.99,42.21 59.94,41.97 60.20,41.80 60.08,41.76 60.14,41.59 60.09,41.40 60.45,41.22 60.87,41.25 61.24,41.19 61.50,41.28 61.90,41.09 62.10,40.68 62.38,40.33 62.48,39.98 63.51,39.38 63.76,39.16 64.16,38.95 64.31,38.98 65.61,38.24 65.97,38.24 66.57,38.01 66.63,37.93 66.53,37.79 66.52,37.35
49.23,46.34 48.54,46.61 48.50,46.70 48.56,46.76 48.88,46.71 48.96,46.77 48.55,47.32 48.17,47.71 47.48,47.80 47.29,47.74 47.09,47.95 47.12,48.13 47.06,48.23 46.66,48.41 46.61,48.57 46.70,48.81 47.03,49.15 46.80,49.37 46.89,49.70 46.99,49.85 47.25,50.00 47.33,50.27 47.50,50.40 47.71,50.38 48.33,49.86 48.43,49.83 48.76,49.93 48.84,50.01 48.63,50.61 48.81,50.60 49.32,50.85 49.50,51.08 49.82,51.13 50.31,51.32 50.79,51.73 51.16,51.65 51.34,51.48 51.61,51.48 52.22,51.71 52.57,51.48 53.34,51.48 54.14,51.04 54.42,50.78 54.47,50.58 54.56,50.54 54.65,50.66 54.55,50.95 54.64,51.01 55.36,50.67 55.69,50.58 56.05,50.71 56.14,50.84 56.49,51.02 56.62,50.98 57.01,51.07 57.44,50.89 57.65,50.93 57.84,51.09 58.36,51.06 58.88,50.69 59.45,50.62 59.52,50.58 59.52,50.49 59.81,50.58 59.96,50.80 60.06,50.85 60.42,50.68 60.94,50.70 61.39,50.86 61.59,51.23 61.55,51.32 61.36,51.44 60.46,51.65 60.39,51.77 60.03,51.93 60.43,52.13 60.67,52.15 60.99,52.34 60.77,52.68 61.05,52.97 61.97,52.94 62.08,53.01 62.01,53.11 61.66,53.23 61.20,53.29 61.16,53.34 61.23,53.45 61.53,53.52 61.47,53.58 61.25,53.55 60.98,53.62 61.11,53.75 61.14,53.96 61.23,54.02 61.33,54.05 61.93,53.95 63.85,54.24 64.46,54.38 64.93,54.40 65.09,54.34 65.24,54.52 65.48,54.62 65.71,54.62 65.91,54.69 66.22,54.67 68.16,54.98 68.24,55.05 68.21,55.16 68.52,55.20 68.98,55.39 69.49,55.36 70.18,55.16 70.49,55.28 70.74,55.31 70.91,55.13 71.19,54.60 71.15,54.36 71.05,54.26 71.09,54.21 71.34,54.16 72.00,54.21 72.19,54.33 72.39,54.12 72.45,53.94 72.59,54.00 72.56,54.09 72.62,54.13 72.91,54.11 73.23,53.96 73.62,54.07 73.71,54.04 73.68,53.93 73.40,53.81 73.31,53.71 73.29,53.60 73.41,53.45 73.86,53.62 74.21,53.58 74.35,53.49 74.45,53.65 74.83,53.83 75.22,53.89 75.38,53.97 75.44,54.09 76.84,54.44 76.65,54.15 76.42,54.15 76.48,54.02 77.86,53.27 79.47,51.49 79.99,50.77 80.07,50.76 80.09,50.84 80.22,50.91 80.42,50.95 80.45,51.18 80.74,51.29 80.88,51.28 80.97,51.19 81.13,51.19 81.07,50.97 81.39,50.96 81.47,50.74 82.49,50.73 82.76,50.89 83.36,50.99 83.95,50.77 84.18,50.52 84.32,50.24 84.99,50.06 85.00,49.89 85.23,49.62 86.18,49.50 86.68,49.78 86.73,49.70 86.63,49.56 86.81,49.49 87.00,49.29 87.23,49.22 87.32,49.09
34.20,31.32 34.90,29.48
33.98,4.22 34.38,4.62
41.88,3.98 41.14,3.96 40.77,4.27 39.84,3.85 39.54,3.47 39.23,3.48 38.09,3.65 36.91,4.41 36.08,4.45 35.92,4.62
23.38,-17.64 21.42,-18.00 21.11,-17.96 20.75,-18.02 20.39,-17.89 18.96,-17.80 18.72,-17.70 18.40,-17.40 14.02,-17.41 13.48,-17.04 13.10,-16.97 12.55,-17.21 12.01,-17.17 11.74,-17.25
25.26,-17.79 25.64,-17.82 26.00,-17.97 26.14,-17.91 26.33,-17.93 26.78,-18.04 27.02,-17.96 27.93,-16.90 28.76,-16.53 28.83,-16.42 28.91,-15.99 29.49,-15.70 30.40,-15.64
23.38,-17.64 24.27,-17.48 24.73,-17.52 25.00,-17.57 25.26,-17.79
31.29,-22.40 32.43,-21.30 32.35,-21.14 32.48,-20.95 32.49,-20.66 32.67,-20.52 32.99,-19.98 33.01,-19.87 32.78,-19.39 32.85,-19.10 32.70,-18.94 32.72,-18.83 32.88,-18.73 32.99,-18.36 32.97,-17.25 32.88,-16.88 32.95,-16.71 32.24,-16.45 31.94,-16.43 31.69,-16.21 31.49,-16.18 31.24,-16.02 30.44,-16.00 30.40,-15.64
12.28,-6.00 12.42,-6.05 13.18,-5.86 16.43,-5.90 16.70,-6.16 16.74,-6.62 16.92,-6.93 16.98,-7.26 17.58,-8.10 18.01,-8.11 18.56,-7.94 18.94,-8.00 19.34,-7.97 19.53,-7.14 19.66,-7.04 19.88,-6.99 20.59,-6.92 20.54,-7.18 20.61,-7.28 21.78,-7.31 21.84,-7.42 21.80,-8.11 21.90,-8.34 21.91,-8.69 21.81,-9.47 22.20,-10.04 22.27,-10.26 22.31,-10.69 22.18,-10.89 22.23,-11.12 22.31,-11.20 22.56,-11.06 23.08,-11.09 23.46,-10.97 23.83,-11.01 23.97,-10.87
-16.56,13.59 -15.51,13.59 -15.43,13.73 -15.11,13.81 -14.94,13.79 -14.41,13.50 -13.98,13.54 -13.83,13.41 -13.85,13.34 -14.25,13.24 -15.15,13.56 -15.29,13.40 -15.81,13.33 -15.83,13.16 -16.65,13.15 -16.76,13.06
0.22,14.91 0.16,14.50 0.38,14.25 0.35,14.14 0.43,13.97 0.62,13.70 0.90,13.61 1.20,13.36 0.99,13.36 0.99,13.04 1.56,12.64 1.79,12.61 1.96,12.71 2.10,12.70 2.23,12.47 2.07,12.38 2.07,12.31 2.39,11.90
18.61,3.48 18.47,3.62 18.16,3.50 17.49,3.69 16.61,3.51 16.48,3.17 16.47,2.83 16.18,2.27
16.18,2.26 16.08,2.11 16.14,1.72 16.06,1.68 15.74,1.92 15.34,1.94 15.16,2.04 14.90,2.01 14.88,2.08 14.76,2.08 14.58,2.20 14.48,2.15 13.29,2.16
15.48,7.52 15.21,7.21 14.74,6.28 14.51,6.16 14.43,6.04 14.62,5.87 14.56,5.28 14.64,5.18 14.73,4.60 15.06,4.28 15.14,4.04 15.03,4.02 15.13,3.83 15.77,3.13 15.90,3.10 16.06,2.91 16.18,2.26
33.98,4.22 33.49,3.76 33.15,3.77 33.00,3.88 32.34,3.71 32.14,3.52 31.94,3.61 31.80,3.80 31.55,3.68 31.15,3.79 30.93,3.63 30.84,3.49
13.07,-4.63 12.83,-4.74 12.45,-5.07 12.52,-5.15 12.50,-5.70 12.21,-5.76
31.95,-25.96 31.99,-24.46 31.80,-23.89 31.55,-23.48 31.29,-22.40
33.20,-14.01 30.23,-14.99 30.40,-15.64
34.96,-11.58 34.62,-11.62 34.36,-12.16 34.56,-13.36 34.66,-13.49 34.91,-13.55 35.25,-13.90 35.85,-14.67 35.89,-14.89 35.81,-15.27 35.82,-15.68 35.76,-16.06 35.36,-16.16 35.17,-16.56 35.28,-16.81 35.27,-17.12 35.06,-17.08 35.08,-16.83 34.93,-16.76 34.42,-16.25 34.40,-16.08 34.25,-15.89 34.54,-15.30 34.51,-14.60 34.33,-14.41 33.64,-14.57 33.20,-14.01
29.58,-1.39 29.56,-0.98 29.72,0.10 29.93,0.50 29.94,0.82 30.18,0.97 30.32,1.19 30.48,1.24 31.25,2.04 31.27,2.15 31.18,2.27 30.96,2.40 30.73,2.46 30.85,2.89 30.75,3.04 30.91,3.41 30.84,3.49
36.52,14.26 36.43,15.13 36.52,15.25 36.91,16.30 36.89,16.62 36.98,16.80 37.01,17.06 37.41,17.06 37.55,17.32 37.78,17.46 38.25,17.58 38.61,18.01
42.38,12.47 42.13,12.77 41.95,12.88 41.77,13.18 41.36,13.50 40.82,14.11 40.14,14.46 39.90,14.44 39.53,14.54 39.20,14.48 39.02,14.63 38.81,14.48 38.43,14.43 38.22,14.65 37.88,14.85 37.57,14.15 37.26,14.45 37.02,14.27 36.81,14.32 36.52,14.26
27.35,57.53 27.40,57.67 27.54,57.80 27.78,57.87 27.67,57.93 27.50,58.22 27.53,58.44 27.43,58.79 27.76,59.05 27.90,59.28 28.15,59.37 28.01,59.48
74.31,32.81 74.66,32.76 74.69,32.49 74.99,32.46 75.30,32.32
-84.35,10.98 -84.70,11.05 -84.91,10.95 -85.58,11.19 -85.74,11.06
-82.88,8.07 -83.03,8.34 -82.86,8.45 -82.86,8.64 -82.92,8.74 -82.73,8.92 -82.94,9.06 -82.94,9.45 -82.80,9.59 -82.64,9.51 -82.56,9.58
-66.88,1.22 -67.21,2.39 -67.62,2.79 -67.86,2.79 -67.83,2.89 -67.31,3.42 -67.66,3.86 -67.86,4.51 -67.82,5.27 -67.64,5.56 -67.58,5.83 -67.47,5.93 -67.48,6.18 -67.86,6.29 -68.47,6.16 -68.94,6.20 -69.27,6.10 -69.43,6.12 -70.13,6.95 -70.74,7.09 -71.13,6.99 -72.01,7.03 -72.21,7.37 -72.39,7.42 -72.47,7.52 -72.45,7.97 -72.36,8.09 -72.39,8.29 -72.67,8.63 -72.80,9.11 -72.96,9.14 -73.06,9.26 -73.37,9.19 -73.01,9.79 -72.87,10.49 -72.69,10.84 -72.45,11.11 -72.25,11.20 -71.96,11.67 -71.32,11.86
-68.44,-52.36 -68.46,-52.29 -69.96,-52.01 -71.97,-51.96 -71.95,-51.88 -72.41,-51.54 -72.30,-51.30 -72.38,-51.10 -72.28,-50.91 -72.34,-50.68 -72.51,-50.61 -73.15,-50.74 -73.31,-50.36 -73.50,-50.13 -73.48,-49.77
-68.63,-52.65 -68.65,-54.92 -68.06,-54.88 -67.35,-54.93 -66.48,-55.11
22.77,54.36 19.60,54.46
48.44,28.54 47.67,28.53 47.43,28.99 46.53,29.10
44.77,37.14 44.88,36.80 45.02,36.70 45.05,36.47 45.24,36.36 45.36,36.02 45.56,35.98 45.78,35.82 46.27,35.77 46.00,35.61 45.98,35.48 46.11,35.32 46.13,35.13 45.92,35.03 45.68,34.80 45.64,34.57 45.50,34.58 45.44,34.42 45.54,34.22 45.40,33.97 45.74,33.60 45.88,33.61 45.87,33.49 46.02,33.42 46.15,33.23 46.08,33.09 46.11,32.96 46.38,32.93 47.12,32.47 47.37,32.42 47.51,32.15 47.83,31.79 47.68,31.40 47.68,31.00 48.01,30.99 48.01,30.47 48.33,30.29 48.43,30.04 48.55,29.96
61.26,35.62 61.25,35.87 61.15,35.98 61.21,36.19 61.12,36.64 60.34,36.64 60.06,36.96 59.45,37.25 59.30,37.51 58.82,37.68 58.26,37.67 57.98,37.83 57.35,37.97 57.31,38.13 57.19,38.22 56.44,38.25 56.32,38.19 56.27,38.08 55.58,38.10 55.38,38.05 54.90,37.78 54.70,37.47 54.19,37.33 53.91,37.34
140.97,-2.60 140.97,-6.35 140.86,-6.74 140.98,-6.91 140.98,-9.12
9.52,30.23 9.31,30.12 9.64,29.64 9.81,29.18 9.92,27.79 9.75,27.33 9.89,26.85 9.88,26.63 9.86,26.55 9.49,26.33 9.45,26.07 10.00,25.33 10.03,25.05 10.26,24.59 10.40,24.49 10.69,24.55 11.51,24.31 11.97,23.52
-13.73,12.67 -15.20,12.68 -15.57,12.49 -16.71,12.35
-11.39,12.40 -12.40,12.34 -12.93,12.53 -13.06,12.49 -13.08,12.63 -13.73,12.67
0.90,10.99 0.49,10.95 -0.07,11.12
-139.06,60.00 -139.19,60.08 -139.08,60.34 -139.68,60.33 -139.97,60.18 -140.45,60.30 -140.53,60.22 -141.00,60.30 -141.00,69.65
-130.62,54.71 -130.21,55.03 -130.02,55.28 -130.12,55.69 -130.03,55.89 -130.06,56.07 -130.41,56.12 -130.48,56.23 -130.74,56.34 -131.58,56.60 -131.82,56.59 -131.87,56.79 -132.10,56.86 -132.03,57.03 -132.34,57.08 -132.23,57.20 -133.42,58.34 -133.40,58.41 -133.82,58.71 -134.30,58.90 -134.44,59.09 -134.94,59.29 -135.07,59.44 -135.05,59.58 -135.48,59.79 -136.32,59.60 -136.25,59.53 -136.35,59.46 -136.47,59.46 -136.47,59.28 -136.58,59.15 -136.81,59.15 -137.44,58.90 -137.52,58.92 -137.48,58.99 -137.59,59.23 -138.63,59.78 -138.71,59.90 -139.06,60.00
-123.31,48.99 -122.69,48.99
-88.89,15.89 -89.24,15.89 -89.16,17.81
-75.28,-0.11 -75.48,-0.16 -75.63,-0.12 -75.26,-0.59 -75.25,-0.95 -75.41,-0.92 -75.57,-1.53 -76.09,-2.13 -76.68,-2.56 -77.86,-2.98 -78.18,-3.35 -78.16,-3.47 -78.23,-3.49 -78.35,-3.40 -78.42,-3.78 -78.68,-4.33 -78.69,-4.56 -78.91,-4.71 -78.92,-4.86 -79.03,-4.97 -79.33,-4.93 -79.50,-4.67 -79.52,-4.54 -79.64,-4.45 -79.80,-4.48 -80.14,-4.30 -80.38,-4.46 -80.48,-4.43 -80.35,-4.21 -80.49,-4.17 -80.49,-4.01 -80.30,-4.01 -80.18,-3.88 -80.32,-3.39
-69.97,-4.24 -70.34,-3.81 -70.53,-3.87 -70.74,-3.78 -70.07,-2.75 -70.10,-2.66 -70.97,-2.21 -71.40,-2.33 -71.75,-2.15 -71.98,-2.33 -72.22,-2.40 -72.40,-2.43 -72.63,-2.35 -72.94,-2.39 -73.15,-2.28 -73.13,-2.08 -73.20,-1.83 -73.50,-1.69 -73.52,-1.45 -73.66,-1.25 -74.25,-0.97 -74.42,-0.58 -74.80,-0.20 -74.95,-0.19 -75.14,-0.05 -75.28,-0.11
-66.88,1.22 -67.08,1.19 -67.12,1.70 -67.40,2.12 -67.56,2.07 -67.82,1.79 -67.94,1.75 -68.19,1.99 -68.26,1.85 -68.18,1.72 -69.32,1.72 -69.54,1.77 -69.85,1.71 -69.85,1.06 -69.31,1.05 -69.16,0.86 -69.15,0.66 -69.28,0.63 -69.47,0.73 -70.05,0.58 -70.07,-0.14 -69.92,-0.32 -69.63,-0.51 -69.61,-0.76 -69.45,-1.00 -69.40,-1.19 -69.97,-4.24
-69.58,-10.95 -68.69,-12.50 -68.76,-12.69 -68.98,-12.88 -68.98,-13.50 -69.07,-13.68 -68.87,-14.17 -69.00,-14.27 -69.01,-14.38 -69.36,-14.80 -69.37,-14.96 -69.17,-15.24 -69.42,-15.64 -69.22,-16.15 -68.84,-16.34 -69.03,-16.48 -69.02,-16.64 -69.20,-16.77 -69.44,-17.09 -69.62,-17.20 -69.51,-17.50
-69.58,-10.95 -68.85,-11.01 -68.73,-11.12 -68.62,-11.11 -68.31,-10.98 -68.07,-10.70 -67.72,-10.68 -67.42,-10.39 -67.11,-10.27 -66.58,-9.90 -65.92,-9.79 -65.56,-9.80 -65.40,-9.71 -65.31,-9.87 -65.30,-10.15 -65.45,-10.51 -65.32,-11.02 -65.39,-11.25 -65.18,-11.65 -65.19,-11.75 -65.09,-11.74 -64.99,-11.98 -64.51,-12.25 -64.42,-12.44 -63.94,-12.53 -63.69,-12.48 -63.35,-12.68 -63.07,-12.67 -63.02,-12.81 -62.77,-13.00 -62.12,-13.16 -61.79,-13.53 -61.08,-13.49 -60.51,-13.79 -60.40,-14.02 -60.47,-14.18 -60.30,-14.62 -60.27,-15.09 -60.58,-15.10 -60.24,-15.48 -60.18,-16.27 -58.54,-16.33 -58.35,-16.28 -58.35,-16.49 -58.48,-16.70 -58.40,-17.23 -57.99,-17.51 -57.83,-17.51 -57.59,-18.12 -57.50,-18.21 -57.57,-18.28 -57.78,-18.91 -57.73,-18.92 -57.72,-19.04 -57.80,-19.08 -58.13,-19.74 -57.86,-19.98 -58.16,-20.16
-69.97,-4.24 -70.05,-4.33 -70.24,-4.30 -70.40,-4.15 -70.80,-4.17 -70.97,-4.35 -71.84,-4.50 -72.35,-4.79 -72.70,-5.07 -72.89,-5.12 -72.98,-5.63 -73.24,-6.10 -73.14,-6.47 -73.76,-6.91 -73.80,-7.08 -73.72,-7.31 -73.96,-7.38 -74.00,-7.56 -73.72,-7.78 -73.78,-7.94 -73.64,-8.07 -73.55,-8.35 -73.36,-8.48 -73.30,-8.65 -72.97,-8.99 -72.97,-9.12 -73.21,-9.41 -72.81,-9.41 -72.38,-9.51 -72.17,-9.84 -72.14,-10.01 -71.24,-9.97 -70.54,-9.44 -70.64,-9.82 -70.64,-11.01 -70.53,-10.95 -70.29,-11.06 -69.96,-10.93 -69.58,-10.95
-62.65,-22.23 -62.28,-21.07 -62.28,-20.56 -61.92,-20.06 -61.76,-19.65 -60.01,-19.30 -59.09,-19.29 -58.18,-19.82 -58.16,-20.16
-17.05,20.81 -16.96,21.33 -13.02,21.33 -13.15,22.82 -13.03,23.00 -12.62,23.27 -12.02,23.47 -12.02,26.00 -8.68,26.00 -8.68,27.29
-8.68,27.29 -4.82,25.00
-8.68,27.66 -8.66,28.72 -7.69,29.35 -7.49,29.39 -7.16,29.61 -6.64,29.57 -6.52,29.66 -6.48,29.82 -6.00,29.83 -5.45,29.96 -5.18,30.17 -4.97,30.47 -4.32,30.70 -3.99,30.91 -3.67,30.96 -3.62,31.07 -3.83,31.20 -3.79,31.36 -3.83,31.66 -3.02,31.83 -2.86,32.07 -2.45,32.13 -1.28,32.09
-8.68,27.29 -8.68,27.66
-12.28,14.81 -12.86,15.24 -12.93,15.45 -13.21,15.62 -13.41,16.06 -13.51,16.14 -13.87,16.15 -14.09,16.42 -14.30,16.58 -14.53,16.66 -14.99,16.68 -15.77,16.49 -16.24,16.53 -16.44,16.20 -16.54,15.84
-4.82,25.00 -6.59,24.99 -5.63,16.57 -5.36,16.28 -5.51,15.50 -9.18,15.50 -9.34,15.53 -9.35,15.68 -9.43,15.62 -9.45,15.46 -9.94,15.37 -10.70,15.42 -10.95,15.15 -11.50,15.64 -11.76,15.43 -11.94,14.89 -12.10,14.75 -12.28,14.81
4.23,19.14 3.40,18.99 3.26,19.01 3.11,19.15 3.26,19.41 3.20,19.79 3.13,19.85 2.41,20.06 2.22,20.25 1.75,20.33 1.61,20.56 1.17,20.82 1.15,21.10 -4.82,25.00
4.23,19.14 5.84,19.48 7.48,20.87 11.97,23.52
9.52,30.23 9.04,32.07 8.33,32.54 8.11,33.06 7.73,33.27 7.50,33.83 7.51,34.08 7.75,34.25 7.84,34.41 8.12,34.56 8.25,34.73 8.28,34.98 8.39,35.20 8.25,35.80 8.35,36.37 8.21,36.52 8.44,36.76 8.60,36.83 8.58,36.94
23.98,19.50 15.98,23.45 14.98,23.00
24.98,22.00 24.98,20.00 23.98,20.00 23.98,19.50
25.15,31.65 24.85,31.33 24.96,30.68 24.70,30.20 24.98,29.18 24.98,22.00
22.86,10.92 22.92,11.34 22.59,11.58 22.58,11.99 22.47,12.07 22.35,12.66 22.23,12.71 21.93,12.68 21.83,12.79 21.99,13.11 22.23,13.33 22.11,13.80 22.17,13.91 22.54,14.16 22.38,14.55 22.67,14.72 22.76,15.00 22.93,15.16 22.93,15.53 23.11,15.70 23.97,15.72 23.98,19.50
24.98,22.00 31.21,21.99 31.40,22.20 31.49,22.15 31.43,22.00 36.87,22.00
41.88,3.98 41.34,3.20 40.96,2.81 40.98,-0.87 41.52,-1.57 41.53,-1.70
42.92,11.00 42.66,10.60 42.84,10.20 43.07,9.93 43.18,9.88 43.48,9.38 43.62,9.34 43.98,9.01 46.98,8.00 47.98,8.00
30.75,-8.19 28.90,-8.49 28.93,-8.59 28.87,-8.79 28.40,-9.22 28.60,-9.68 28.65,-10.55 28.36,-11.48 28.38,-11.57 28.48,-11.81 29.06,-12.35 29.49,-12.42 29.51,-12.23 29.80,-12.16 29.78,-13.44 29.65,-13.41 29.63,-13.30 29.55,-13.25 29.20,-13.40 29.01,-13.37 28.73,-12.93 28.55,-12.84 28.41,-12.52 27.57,-12.23 27.42,-11.94 27.24,-11.78 27.16,-11.58 27.05,-11.62 26.95,-11.90 26.82,-11.97 26.03,-11.89 25.51,-11.75 25.35,-11.62 25.29,-11.21 24.52,-11.44 24.38,-11.42 24.34,-11.37 24.40,-11.26 24.37,-11.13 24.14,-11.03 24.08,-10.89 23.97,-10.87
23.38,-17.64 22.15,-16.60 22.04,-16.26 21.98,-15.96 21.98,-13.00 23.96,-12.99 23.88,-12.80 23.99,-12.42 23.97,-11.64 24.05,-11.41 23.97,-10.87
33.90,-1.00 33.94,0.17 34.16,0.61 34.41,0.87 34.48,1.04 34.80,1.24 34.78,1.38 34.97,1.64 34.98,1.77 34.91,2.48 34.74,2.82 34.59,2.92 34.52,3.12 34.45,3.16 34.40,3.41 34.44,3.65 34.17,3.81 34.19,3.87 33.98,4.22
33.90,-1.00 37.64,-3.05 37.69,-3.25 37.62,-3.51 39.22,-4.69
11.33,2.17 9.98,2.17 9.80,2.30
11.33,2.17 11.34,1.00 10.03,1.00 9.91,0.96 9.76,1.07 9.59,1.03
25.26,-17.79 24.91,-17.82 24.53,-18.05 24.36,-17.98 24.24,-18.02 23.60,-18.46 23.30,-18.03 23.22,-18.00 20.97,-18.32 20.98,-21.96 19.98,-22.00 19.98,-24.75
19.98,-24.78 19.98,-28.45 19.54,-28.57 19.31,-28.73 19.25,-28.90 19.16,-28.94 18.84,-28.87 18.10,-28.87 17.45,-28.70 17.35,-28.50 17.36,-28.27 17.06,-28.03 16.88,-28.13 16.76,-28.45 16.45,-28.62
19.98,-24.75 19.98,-24.78
0.90,10.99 0.79,10.71 0.76,10.39 1.33,10.00 1.39,9.36 1.60,9.05 1.62,7.00 1.53,6.99 1.60,6.61 1.78,6.29 1.62,6.22
42.80,16.37 43.17,16.69 43.18,16.81 43.12,16.94 43.16,17.21 43.24,17.27 43.19,17.36 43.42,17.52 43.92,17.32 44.16,17.40 45.15,17.43 45.54,17.30 46.31,17.23 46.73,17.27 46.98,16.95 47.14,16.95 47.44,17.11 47.58,17.45 48.17,18.16 49.04,18.58 51.98,19.00
51.98,19.00 52.73,17.30 52.80,17.27 53.09,16.65
55.19,22.70 55.64,22.00 54.98,20.00 51.98,19.00
55.19,22.70 55.10,22.62 52.56,22.93 51.59,24.08 51.57,24.29
42.36,37.11 41.79,36.60 41.42,36.51 41.30,36.38 41.25,36.07 41.35,35.64 41.22,35.29 41.19,34.77 40.99,34.43 40.69,34.33 38.77,33.37
35.79,32.73 36.37,32.39 36.82,32.32 38.77,33.37
35.61,32.68 35.79,32.73
39.15,32.12 39.29,32.24 39.25,32.35 39.04,32.31 38.98,32.47 39.06,32.49 38.77,33.37
34.95,29.35 36.07,29.20 36.48,29.50 36.76,29.87 37.47,30.00 37.65,30.33 37.98,30.50 36.96,31.49 38.96,31.99 39.15,32.12
46.53,29.10 46.36,29.06 44.69,29.20 42.07,31.08 40.37,31.94 39.15,32.12
61.59,25.20 61.66,25.75 61.75,25.84 61.84,26.23 62.13,26.37 62.24,26.36 62.31,26.49 62.44,26.56 63.16,26.65 63.30,27.15 63.17,27.25 62.76,27.25 62.81,27.50 62.76,28.24 62.56,28.24 62.35,28.41 61.89,28.55 61.62,28.79 61.32,29.37 60.84,29.86
60.84,29.86 61.81,30.91 61.76,31.29 61.66,31.38 60.82,31.50 60.83,32.25 60.56,33.06 60.56,33.14 60.92,33.51 60.51,33.64 60.49,34.09 60.64,34.31 60.89,34.32 60.73,34.52 60.95,34.65 61.08,34.86 61.15,35.09 61.10,35.27 61.19,35.31 61.28,35.51 61.26,35.62
73.73,36.89 74.04,36.83 74.54,37.02
60.84,29.86 62.48,29.41 63.57,29.50 64.10,29.39 64.39,29.54 65.10,29.56 66.18,29.84 66.31,29.97 66.24,30.11 66.35,30.80 66.40,30.91 66.60,31.02 66.83,31.26 66.92,31.31 67.45,31.23 67.74,31.34 67.58,31.51 68.02,31.68 68.16,31.80 68.44,31.75 68.60,31.80 68.87,31.63 69.08,31.74 69.28,31.94 69.24,32.43 69.41,32.68 69.50,33.02 69.92,33.11 70.26,33.29 70.28,33.37 70.13,33.62 69.87,33.90 69.89,34.01 69.99,34.05 70.33,33.96 70.65,33.95 71.05,34.05 71.10,34.37 70.97,34.53 71.62,35.18 71.55,35.29 71.60,35.41 71.57,35.55 71.40,35.88 71.19,36.04 71.23,36.12 71.62,36.44 71.77,36.43 72.25,36.73 72.62,36.83 73.73,36.89
70.95,42.25 70.72,42.17 70.58,42.04 70.42,42.08 70.10,41.82 69.15,41.43 68.99,41.21 68.58,40.88 68.57,40.62 68.42,40.62 68.05,40.81 68.11,41.03 67.94,41.20 66.71,41.18 66.50,41.99 66.01,42.00 66.10,42.99 65.80,42.88 65.50,43.31 64.91,43.71 64.44,43.55 63.21,43.63 61.99,43.49 61.16,44.17 61.01,44.39 58.56,45.56 55.98,44.99 55.98,41.32
13.61,13.70 13.45,14.38 14.37,15.75 15.47,16.91 15.74,19.90 15.96,20.35 15.59,20.73 15.54,20.87 15.61,20.95 15.18,21.52 15.17,21.92 14.98,23.00
11.97,23.52 13.48,23.18 14.22,22.62 14.98,23.00
0.22,14.91 0.29,14.98 0.95,14.98 1.30,15.27 3.00,15.34 3.06,15.43 3.50,15.36 3.52,15.48 3.84,15.70 4.12,16.36 4.23,17.00 4.23,19.14
-0.07,11.12 -0.30,11.17 -0.63,10.93 -0.70,10.99 -1.04,11.01 -2.83,11.00 -2.91,10.59 -2.79,10.43 -2.78,9.75 -2.70,9.48
-13.29,9.05 -13.03,9.10 -12.96,9.26 -12.76,9.37 -12.50,9.86 -12.43,9.90 -12.28,9.93 -12.14,9.88 -11.92,9.92 -11.91,9.99 -11.27,10.00 -10.69,9.31 -10.75,9.10 -10.62,9.06 -10.61,8.87 -10.50,8.69 -10.71,8.34 -10.56,8.32 -10.39,8.48 -10.28,8.49
5.79,49.54 6.01,49.45 6.18,49.50 6.34,49.45
56.08,26.06 56.17,26.05 56.14,25.69 56.30,25.65
30.51,-1.07 30.36,-1.07 30.10,-1.37 29.93,-1.47 29.83,-1.34 29.58,-1.39
30.51,-1.07 30.51,-1.21 30.81,-1.56 30.88,-2.14 30.83,-2.34 30.55,-2.40
33.90,-1.00 30.81,-0.99 30.51,-1.07
34.59,-11.97 34.54,-12.02 34.62,-12.07 34.59,-11.97
87.32,49.09 87.82,49.16
45.50,40.66 45.48,40.61 45.55,40.62 45.50,40.66
44.99,41.09 45.00,41.02 44.99,41.09
12.44,43.98 12.40,43.93 12.49,43.90 12.50,43.99 12.44,43.98
70.57,41.02 70.65,40.94 70.57,41.02
-117.13,32.53 -114.72,32.72 -114.84,32.51 -111.04,31.32 -108.21,31.33 -108.21,31.78 -106.45,31.77 -106.15,31.45 -104.92,30.58 -104.68,30.13 -104.62,29.85 -104.40,29.57 -103.26,29.00 -103.09,29.04 -102.89,29.22 -102.73,29.64 -102.34,29.86 -101.99,29.80 -101.55,29.81 -101.38,29.74 -100.75,29.18 -100.33,28.50 -100.30,28.33 -99.75,27.73 -99.51,27.55 -99.46,27.06 -99.23,26.76 -99.11,26.45 -98.28,26.11 -97.80,26.04 -97.38,25.87 -97.15,25.96
-92.24,14.55 -92.16,14.69 -92.16,14.96 -92.07,15.07 -92.20,15.28 -91.74,16.07 -90.45,16.07 -90.42,16.39 -90.63,16.51 -90.71,16.71 -91.41,17.26 -90.99,17.25 -90.99,17.82 -89.16,17.81 -89.13,17.97 -89.05,18.00 -88.90,17.91 -88.81,17.97 -88.46,18.48 -88.04,18.42 -88.04,18.20 -87.83,18.18
-54.62,-25.58 -54.61,-25.43 -54.44,-25.12 -54.28,-24.31 -54.32,-24.13 -54.24,-24.05 -54.63,-23.81 -55.19,-24.02 -55.42,-23.95 -55.65,-22.89 -55.62,-22.67 -55.85,-22.31 -56.25,-22.26 -56.45,-22.08 -56.63,-22.23 -56.94,-22.27 -57.57,-22.18 -57.72,-22.10 -57.96,-22.11 -57.95,-21.49 -57.83,-21.00 -58.16,-20.16
-62.65,-22.23 -62.21,-22.61 -61.80,-23.18 -61.21,-23.56 -61.03,-23.76 -60.51,-23.96 -59.89,-24.09 -59.19,-24.56 -58.37,-24.96 -57.96,-25.05 -57.64,-25.33 -57.57,-25.53 -57.75,-25.70 -57.89,-26.01 -58.11,-26.18 -58.20,-26.38 -58.19,-26.63 -58.32,-26.86 -58.62,-27.13 -58.60,-27.31 -58.17,-27.27 -57.11,-27.47 -56.60,-27.47 -56.44,-27.55 -56.16,-27.32 -55.71,-27.41 -55.43,-27.01 -55.14,-26.93 -54.83,-26.65 -54.68,-26.31 -54.62,-25.58
-62.65,-22.23 -62.84,-22.00 -63.92,-22.03 -64.27,-22.60 -64.33,-22.83 -64.61,-22.23 -64.99,-22.11 -65.77,-22.10 -66.22,-21.80 -66.37,-22.11 -66.71,-22.22 -66.80,-22.41 -67.03,-22.55 -67.19,-22.82
26.62,48.26 26.85,48.39 27.23,48.37 27.55,48.48 27.82,48.42 28.09,48.26 28.29,48.24 28.46,48.09 28.53,48.15 28.77,48.12 28.92,47.95 29.13,47.96 29.21,47.78 29.13,47.49 29.54,47.27 29.51,47.13 29.57,46.96 29.88,46.83 29.94,46.72 29.92,46.54 30.13,46.42 29.84,46.35 29.71,46.45 29.61,46.40 29.30,46.47 29.20,46.38 29.19,46.52 28.96,46.46 28.93,46.36 29.01,46.18 28.95,46.05 28.74,45.94 28.73,45.85 28.49,45.67 28.50,45.52 28.21,45.45
67.76,37.17 67.96,36.97 68.07,36.95 68.91,37.33 69.30,37.12 69.41,37.21 69.40,37.40 69.49,37.55 69.82,37.61 70.04,37.55 70.19,37.58 70.25,37.66 70.21,37.92 70.62,38.33 70.88,38.46 71.26,38.31 71.33,38.17 71.28,37.92 71.58,37.91 71.43,37.13 71.53,36.85 71.67,36.70 71.80,36.69 72.36,36.98 72.66,37.03 72.90,37.27 73.38,37.46 73.72,37.42 73.65,37.24 74.17,37.33 74.35,37.42 74.66,37.39 74.89,37.23
66.52,37.35 66.83,37.37 67.07,37.33 67.20,37.24 67.52,37.27 67.76,37.17
61.26,35.62 61.34,35.63 61.62,35.43 61.98,35.44 62.31,35.17 62.46,35.25 62.69,35.26 63.06,35.45 63.17,35.68 63.13,35.85 64.01,36.01 64.51,36.34 64.82,37.13 65.09,37.24 65.55,37.25 65.64,37.47 65.77,37.57 66.11,37.41 66.52,37.35
92.57,21.98 92.49,22.69 92.36,22.93 92.25,23.68 91.93,23.69 91.94,23.50 91.75,23.29 91.75,23.05 91.62,22.98 91.51,23.03 91.44,23.20 91.37,23.20 91.37,23.07 91.32,23.10 91.16,23.66 91.23,23.92 91.37,24.09 91.88,24.20 91.95,24.36 92.10,24.41 92.23,24.88 92.44,24.85 92.47,24.94 92.05,25.17 90.44,25.16 89.83,25.29 89.82,25.94 89.67,26.21 89.59,26.19 89.55,26.01 89.37,26.01 89.19,26.11 89.02,26.41 88.92,26.38 88.97,26.25 88.83,26.25 88.68,26.29 88.62,26.43 88.42,26.57 88.35,26.50 88.44,26.37 88.15,26.09 88.08,25.89 88.50,25.54 88.77,25.49 88.95,25.26 88.82,25.18 88.46,25.19 88.31,24.88 88.15,24.91 88.02,24.63 88.15,24.49 88.29,24.48 88.72,24.27 88.70,24.00 88.57,23.67 88.74,23.44 88.72,23.25 88.93,23.19 88.85,23.04 89.05,22.09
2.39,11.90 2.37,12.22 2.81,12.38 3.60,11.70
18.61,3.48 18.62,3.30 18.49,2.92 18.21,2.41 18.07,2.01 18.06,1.53 17.90,1.12 17.89,0.23 17.72,-0.28 17.75,-0.55 17.28,-1.00 16.88,-1.23 16.54,-1.84 16.22,-2.18 16.22,-3.03 16.15,-3.46 15.99,-3.77 15.87,-3.93 15.60,-4.03 15.27,-4.31 14.71,-4.88 14.41,-4.83 14.37,-4.59 14.45,-4.45 14.36,-4.30 13.94,-4.48 13.72,-4.45 13.66,-4.72 13.41,-4.84 13.14,-4.60 13.07,-4.63
28.98,-30.00 29.10,-29.92 29.14,-29.70 29.29,-29.57 29.39,-29.27 29.30,-29.09 28.95,-28.88 28.63,-28.58 27.74,-28.94 27.29,-29.52 27.06,-29.63 27.39,-30.32 27.75,-30.60 28.06,-30.63 28.39,-30.15 28.65,-30.13 28.98,-30.00
29.36,-22.19 29.66,-22.15 30.19,-22.29 30.92,-22.29 31.29,-22.40
19.98,-24.78 20.43,-25.15 20.79,-25.92 20.82,-26.16 20.63,-26.44 20.69,-26.82 21.69,-26.84 21.79,-26.71 22.09,-26.58 22.22,-26.39 22.60,-26.13 22.88,-25.46 23.06,-25.31 23.39,-25.29 23.89,-25.60 24.19,-25.63 24.33,-25.74 24.75,-25.82 25.44,-25.71 25.58,-25.61 25.91,-24.75 26.40,-24.61 26.84,-24.24 26.99,-23.70 27.09,-23.58 27.76,-23.20 28.21,-22.69 28.38,-22.59 28.84,-22.48 29.13,-22.21 29.36,-22.19
34.96,-11.58 35.56,-11.60 35.79,-11.45 35.91,-11.45 36.31,-11.71 36.67,-11.68 36.98,-11.57 37.37,-11.71 37.72,-11.58 37.92,-11.29 38.18,-11.28 38.49,-11.41 38.79,-11.23 39.32,-11.12 39.99,-10.82 40.46,-10.46
-84.35,10.98 -84.17,10.78 -83.92,10.74 -83.71,10.79 -83.64,10.92
28.15,56.14 27.90,56.08 27.64,55.91 27.58,55.80 27.05,55.83 26.59,55.67
28.21,45.45 28.45,45.29 28.76,45.23 28.78,45.31 29.40,45.42 29.71,45.26
21.36,44.83 21.91,44.67 22.09,44.54 22.50,44.71 22.64,44.65 22.73,44.57 22.55,44.54 22.49,44.44 22.71,44.24
56.23,25.30 56.24,25.21 56.29,25.28 56.23,25.30
32.11,-26.84 32.89,-26.85
71.74,39.98 71.67,39.95 71.75,39.91 71.79,40.00 71.74,39.98
-122.69,48.99 -95.16,48.99 -95.16,49.37 -94.85,49.30 -94.80,49.00 -94.62,48.74 -94.06,48.66 -93.71,48.53 -93.38,48.62 -93.00,48.61 -92.50,48.44 -92.41,48.28 -92.17,48.34 -92.01,48.30 -91.52,48.06 -90.92,48.21 -90.74,48.10 -90.09,48.12 -89.90,48.00 -89.46,48.00 -88.38,48.30 -84.88,46.90 -84.78,46.64 -84.56,46.46 -84.15,46.54 -84.09,46.23 -83.98,46.08 -83.62,46.12 -83.47,45.99 -83.59,45.82 -82.55,45.35 -82.14,43.57 -82.41,43.07 -82.55,42.62 -83.07,42.30 -83.15,42.14 -83.14,41.98 -83.03,41.83 -82.69,41.68 -82.44,41.67 -81.28,42.21 -80.25,42.37 -79.04,42.80 -78.92,42.91 -79.03,43.02 -79.06,43.28 -79.17,43.47 -78.85,43.58 -76.82,43.63 -76.46,44.06 -76.15,44.30 -75.00,44.97 -71.52,45.01 -71.33,45.29 -71.13,45.26 -71.00,45.34 -70.87,45.27 -70.80,45.40 -70.69,45.43 -70.70,45.55 -70.42,45.74 -70.30,45.91 -70.25,46.25 -70.07,46.44 -70.01,46.71 -69.24,47.46 -69.05,47.43 -69.05,47.27 -68.94,47.21 -68.24,47.35 -67.81,47.08 -67.80,45.73 -67.43,45.60 -67.49,45.47 -67.43,45.38 -67.47,45.28 -67.37,45.17 -67.12,45.17
34.96,-11.58 34.89,-11.39 34.77,-11.34 34.61,-11.08 34.66,-10.71 34.58,-10.53 34.52,-10.03 34.32,-9.73 34.09,-9.54 34.00,-9.50 33.95,-9.66 33.89,-9.67
107.97,21.51 107.76,21.66 107.35,21.61 107.27,21.71 107.06,21.79 106.97,21.92 106.66,21.98 106.65,22.24 106.55,22.34 106.55,22.50 106.78,22.78 106.54,22.91 106.28,22.86 106.15,22.97 105.84,22.92 105.55,23.07 105.28,23.35 104.86,23.14 104.80,22.91 104.69,22.82 104.37,22.70 104.14,22.80 104.05,22.75 103.94,22.54 103.62,22.78 103.49,22.59 103.33,22.77 102.98,22.45 102.83,22.59 102.47,22.75 102.13,22.38
7.44,43.75 7.38,43.73
12.43,41.91 12.43,41.91
113.55,22.22 113.48,22.20
114.02,22.51 114.12,22.57 114.27,22.54
70.61,39.79 70.48,39.88 70.70,39.85 70.61,39.79
71.22,39.91 71.01,39.90 71.04,39.99 70.97,40.04 70.98,40.13 71.23,40.05 71.18,39.98 71.22,39.91
145.94,43.10 145.78,43.48 145.69,43.55 145.54,43.51 145.24,43.78 145.27,43.91 145.62,44.29 145.66,44.45 145.58,44.79
-1.28,32.09 -1.23,32.11 -1.24,32.34 -1.07,32.47 -1.45,32.78 -1.68,33.32 -1.63,33.57 -1.71,33.78 -1.71,34.18 -1.79,34.37 -1.73,34.47 -1.85,34.61 -1.80,34.75 -2.13,34.97 -2.22,35.10
29.00,9.61 29.47,9.77 29.60,9.92 29.61,10.07 30.00,10.28 30.76,9.73 31.22,9.80 31.79,10.38 31.93,10.66 32.42,11.09 32.34,11.31 32.34,11.72 32.07,12.01 32.74,12.01 32.72,12.22 33.20,12.22 33.07,11.59 33.17,10.85 33.13,10.76 33.38,10.65 33.91,10.18 33.96,9.86 33.87,9.51 34.08,9.46
24.15,8.67 24.21,8.77 24.53,8.89 24.67,9.39 24.78,9.53 24.79,9.81 24.96,9.99 25.02,10.24 25.10,10.31 25.86,10.41 25.92,10.17 26.55,9.53 26.66,9.48 27.07,9.61 27.88,9.60
27.88,9.60 28.05,9.33 28.84,9.33 28.84,9.46 29.00,9.61
-56.48,1.94 -56.70,2.04 -57.20,2.85 -57.30,3.38 -57.60,3.37
-54.62,2.33 -54.40,2.46 -54.20,2.82 -54.20,3.14 -54.08,3.33
33.89,-9.67 33.70,-9.60 33.42,-9.61 33.33,-9.52 32.92,-9.41
33.59,46.10 33.66,46.22 34.22,46.10 34.45,45.97 34.69,45.98 34.79,45.89 34.80,45.79 35.00,45.73
77.05,35.11 76.77,35.66
77.05,35.11 77.80,35.50
79.12,33.23 79.38,33.15 79.33,32.98 79.50,32.73 79.38,32.57 79.22,32.50
79.22,32.50 78.92,32.36 78.70,32.60 78.41,32.56 78.49,32.23
78.49,32.23 78.73,31.98 78.72,31.88
78.72,31.88 78.69,31.74 78.80,31.62 78.73,31.47 78.74,31.32 78.84,31.30
78.84,31.30 78.97,31.33 79.01,31.41 79.11,31.40 79.37,31.08
79.37,31.08 79.57,30.95 79.79,30.97 79.91,30.90
79.91,30.90 80.19,30.76 80.19,30.57
80.19,30.57 80.68,30.41 80.99,30.24 81.01,30.16
90.33,28.12 90.35,28.24 90.22,28.28 89.98,28.31 89.80,28.24
89.80,28.24 89.48,28.06 89.12,27.62
89.12,27.62 89.02,27.51
89.02,27.51 88.89,27.32
75.30,32.32 75.32,32.22 75.25,32.14 74.74,31.95 74.56,31.82 74.51,31.71 74.59,31.47 74.52,31.19 74.63,31.03 74.34,30.89 73.90,30.44 73.93,30.22 73.81,30.09 73.38,29.93 73.23,29.55 72.90,29.03 72.34,28.75 72.13,28.35 71.95,28.18 71.87,27.96 70.80,27.71 70.69,27.77 70.63,27.94 70.40,28.03 70.14,27.85 69.90,27.47 69.54,27.12 69.47,26.80 69.51,26.74 70.15,26.51 70.10,25.91 70.26,25.71 70.65,25.67 70.65,25.42 71.05,24.69 70.97,24.57 71.04,24.40 70.72,24.24 70.58,24.28 70.55,24.42 70.49,24.41 70.10,24.29 70.02,24.19 69.81,24.17 69.56,24.27 68.83,24.26 68.78,24.31 68.73,24.27 68.72,23.96 68.38,23.95 68.17,23.86
36.69,45.62 36.69,45.43 36.59,45.30 36.64,45.26 36.51,45.20 36.61,44.95
-55.81,-31.03 -56.00,-31.08 -56.00,-30.85
-56.00,-30.85 -56.18,-30.63 -56.83,-30.11 -57.12,-30.14 -57.21,-30.28 -57.55,-30.26 -57.61,-30.19
-73.48,-49.77 -73.58,-49.58 -73.46,-49.31 -73.14,-49.30
-73.14,-49.30 -73.15,-49.19 -73.03,-49.01 -72.65,-48.84 -72.58,-48.48 -72.35,-48.37 -72.29,-48.23 -72.33,-48.11 -72.51,-47.97 -72.52,-47.88 -72.35,-47.49 -72.04,-47.24 -71.90,-47.20 -71.96,-47.02 -71.94,-46.83 -71.70,-46.65 -71.76,-46.32 -71.88,-46.16 -71.63,-45.95 -71.75,-45.84 -71.75,-45.58 -71.51,-45.51 -71.35,-45.33 -71.35,-45.23 -71.60,-44.98 -72.04,-44.90 -72.06,-44.77 -71.26,-44.76 -71.16,-44.56 -71.21,-44.44 -71.82,-44.38 -71.81,-44.11 -71.68,-43.93 -71.79,-43.75 -71.74,-43.70 -71.75,-43.59 -71.90,-43.44 -71.90,-43.35 -71.75,-43.24 -71.78,-43.17 -72.10,-43.07 -72.14,-42.58 -72.05,-42.47 -72.11,-42.25 -72.03,-42.15 -71.86,-42.15 -71.75,-42.05 -71.91,-41.65 -71.87,-40.89 -71.94,-40.79 -71.80,-40.44 -71.70,-40.34 -71.82,-40.18 -71.66,-40.02 -71.64,-39.89 -71.72,-39.64 -71.54,-39.60 -71.42,-39.29 -71.40,-38.94 -70.95,-38.74 -70.85,-38.54 -70.97,-38.45 -71.03,-38.04 -71.17,-37.76 -71.13,-37.45 -71.20,-37.30 -71.12,-37.11 -71.19,-36.84 -71.06,-36.52 -70.75,-36.39 -70.72,-36.28 -70.40,-36.06 -70.42,-35.52 -70.56,-35.25 -70.39,-35.15 -70.29,-34.73 -70.05,-34.30 -69.85,-34.22 -69.89,-33.73 -69.80,-33.40 -69.82,-33.28 -70.02,-33.27 -70.08,-33.20 -70.09,-33.03 -70.02,-32.88 -70.12,-32.81 -70.17,-32.47 -70.32,-32.27 -70.36,-32.08 -70.25,-31.96 -70.45,-31.84 -70.59,-31.57 -70.52,-31.15 -70.39,-31.12 -70.31,-31.02 -70.35,-30.90 -70.15,-30.36 -69.96,-30.36 -69.84,-30.18 -69.96,-30.08 -69.93,-29.77 -70.03,-29.32 -69.83,-29.10 -69.66,-28.41 -69.17,-27.92 -68.85,-27.15 -68.71,-27.10 -68.59,-27.14 -68.32,-26.97 -68.32,-26.88 -68.59,-26.47 -68.58,-26.35 -68.41,-26.15 -68.59,-25.42 -68.50,-25.16 -68.38,-25.09 -68.56,-24.84 -68.56,-24.75 -68.25,-24.39 -67.36,-24.03 -67.01,-23.00 -67.19,-22.82
75.32,34.58 74.30,34.77 73.96,34.65 73.81,34.42 73.81,34.33 73.97,34.24 73.92,34.04 74.25,33.99 74.22,33.89 74.00,33.79 73.98,33.72 74.15,33.51 73.99,33.22 74.30,32.99 74.31,32.81
23.61,51.52 23.54,51.71 23.63,51.81 23.65,52.04 23.18,52.29 23.41,52.52 23.84,52.66 23.92,52.77 23.86,53.11 23.48,53.94
34.08,9.46 34.16,9.85 34.31,10.19 34.28,10.53 34.34,10.66 34.57,10.88 34.77,10.75 34.93,10.86 34.96,11.28 35.11,11.82 35.67,12.62 36.13,12.76 36.16,13.09 36.39,13.63 36.52,14.26
2.39,11.90 2.29,11.69 1.98,11.42 1.43,11.45 1.32,11.30 1.15,11.25 1.08,11.08 0.99,11.08 0.90,10.99
31.95,-25.96 31.87,-25.98 31.42,-25.75 31.21,-25.84 30.80,-26.41 30.79,-26.76 30.88,-26.79 31.06,-27.11 31.47,-27.30 31.96,-27.31 31.99,-26.82 32.11,-26.84
32.11,-26.84 32.06,-26.02 31.95,-25.96
76.77,35.66 76.56,35.77 76.55,35.89 76.15,35.83 76.07,35.98 75.91,36.05 75.97,36.38
27.33,5.19 27.79,4.64 28.19,4.35 28.43,4.32 28.73,4.50 29.22,4.39 29.47,4.61 29.68,4.59 30.19,3.98 30.51,3.84 30.59,3.62 30.76,3.62 30.84,3.49
22.86,10.92 23.31,10.39 23.65,9.82 23.62,9.34 23.47,9.11 23.55,8.94 23.54,8.82 23.68,8.73 24.15,8.67
46.11,38.88 46.49,38.91
44.78,39.68 45.14,39.25 45.48,39.01 46.11,38.88
46.49,38.91 47.48,39.50 47.77,39.65 48.00,39.68 48.32,39.40 48.10,39.24 48.29,39.02 48.05,38.93 48.00,38.85 48.59,38.41 48.87,38.44
81.01,30.16 80.85,30.14 80.40,29.73 80.25,29.42 80.23,29.19 80.08,28.99 80.07,28.83 80.42,28.61 80.59,28.65 81.85,27.87 81.99,27.91 82.45,27.67 82.68,27.67 82.73,27.52 83.29,27.37 83.45,27.47 83.83,27.38 84.09,27.49 84.61,27.30 84.69,27.04 85.13,26.86 85.24,26.75 85.57,26.84 85.70,26.78 85.79,26.60 86.01,26.65 86.70,26.44 87.02,26.56 87.09,26.43 87.29,26.36 87.41,26.42 87.85,26.44 88.00,26.38 88.16,26.72 87.98,27.13 88.15,27.75 88.11,27.87
75.97,36.38 75.84,36.65 75.67,36.74 75.42,36.74 75.35,36.91 74.54,37.02
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- A copy is on this computer: a check in an outlined circle. Adwaita 50
no longer ships emblem-ok-symbolic. -->
<path fill="#222222" fill-rule="evenodd" d="M 8,1 A 7,7 0 1 0 8,15 A 7,7 0 1 0 8,1 Z M 8,2.5 A 5.5,5.5 0 1 1 8,13.5 A 5.5,5.5 0 1 1 8,2.5 Z"/>
<path fill="#222222" d="M 4.1,8.6 L 5.2,7.5 L 7,9.3 L 10.8,5.4 L 11.9,6.5 L 7,11.5 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 450 B

@@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Show or hide the details pane: a window with its right-hand pane
filled in. Some themes draw sidebar-show-right-symbolic as a solid block. -->
<path fill="#222222" fill-rule="evenodd" d="M 3,2 H 13 A 2,2 0 0 1 15,4 V 12 A 2,2 0 0 1 13,14 H 3 A 2,2 0 0 1 1,12 V 4 A 2,2 0 0 1 3,2 Z M 3,3.5 A 0.5,0.5 0 0 0 2.5,4 V 12 A 0.5,0.5 0 0 0 3,12.5 H 13 A 0.5,0.5 0 0 0 13.5,12 V 4 A 0.5,0.5 0 0 0 13,3.5 Z"/>
<path fill="#222222" d="M 9.5,5 H 12.5 V 11 H 9.5 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 564 B

@@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- "Filters": a funnel, wide at the top and narrowing to a short spout.
No stock icon theme ships one under a standard name. -->
<path fill="#222222" d="M 1.5,2 H 14.5 L 9.5,8.5 V 13.5 L 6.5,15 V 8.5 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 309 B

@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- A share link: two open chain ends and the bar that joins them, laid
on the diagonal. Written out in plain coordinates: GTK's symbolic icon
renderer drops a transformed group. -->
<path fill="#222222" d="M 4.82,6.23 L 3.05,8 A 3.5,3.5 0 0 0 8,12.95 L 9.77,11.18 L 8.53,9.94 L 6.76,11.71 A 1.75,1.75 0 0 1 4.29,9.24 L 6.06,7.47 Z"/>
<path fill="#222222" d="M 6.23,4.82 L 8,3.05 A 3.5,3.5 0 0 1 12.95,8 L 11.18,9.77 L 9.94,8.53 L 11.71,6.76 A 1.75,1.75 0 0 0 9.24,4.29 L 7.47,6.06 Z"/>
<path fill="#222222" d="M 5.26,9.5 L 9.5,5.26 L 10.74,6.5 L 6.5,10.74 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 676 B

@@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Show on map: a folded map in three panels. No stock theme has a
map-symbolic. -->
<path fill="#222222" d="M 1,3.2 L 5,1.5 V 12.8 L 1,14.5 Z M 11,3.2 L 15,1.5 V 12.8 L 11,14.5 Z"/>
<path fill="#222222" opacity="0.55" d="M 6,1.5 L 10,3.2 V 14.5 L 6,12.8 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 366 B

@@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Main menu: three bars. Bundled because themes such as Papirus draw
open-menu-symbolic as three dots, the same as every row's "more" menu. -->
<path fill="#222222" d="M 2,2.5 H 14 A 1,1 0 0 1 14,4.5 H 2 A 1,1 0 0 1 2,2.5 Z M 2,7 H 14 A 1,1 0 0 1 14,9 H 2 A 1,1 0 0 1 2,7 Z M 2,11.5 H 14 A 1,1 0 0 1 14,13.5 H 2 A 1,1 0 0 1 2,11.5 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 441 B

@@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Shared with me: two people. system-users-symbolic survives only in
Adwaita's legacy folder. -->
<path fill="#222222" d="M 6,2.5 A 2.5,2.5 0 1 0 6,7.5 A 2.5,2.5 0 1 0 6,2.5 Z M 1.5,14.5 C 1.5,11 3.5,9 6,9 C 8.5,9 10.5,11 10.5,14.5 Z"/>
<path fill="#222222" d="M 11,2.4 A 2.1,2.1 0 1 0 11,6.6 A 2.1,2.1 0 1 0 11,2.4 Z M 9.8,8.4 C 10.2,8.3 10.6,8.25 11,8.25 C 13.3,8.25 15,10 15,13.5 H 11.8 C 11.7,11.5 11,9.7 9.8,8.4 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 529 B

@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Shared by me: one node passing to two others. Adwaita 50 no longer
ships emblem-shared-symbolic. -->
<path fill="#222222" d="M 12,1.25 A 2.25,2.25 0 1 0 12,5.75 A 2.25,2.25 0 1 0 12,1.25 Z M 4,5.75 A 2.25,2.25 0 1 0 4,10.25 A 2.25,2.25 0 1 0 4,5.75 Z"/>
<path fill="#222222" d="M 4.37,8.65 L 12.37,4.15 L 11.63,2.85 L 3.63,7.35 Z"/>
<path fill="#222222" d="M 12,10.25 A 2.25,2.25 0 1 0 12,14.75 A 2.25,2.25 0 1 0 12,10.25 Z"/>
<path fill="#222222" d="M 3.63,8.65 L 11.63,13.15 L 12.37,11.85 L 4.37,7.35 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 621 B

@@ -0,0 +1,10 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Syncing: two arrows chasing each other round a circle. Bundled because
Adwaita 50 no longer ships emblem-synchronizing-symbolic. The lower arrow
is the upper one turned half a circle, written out in plain coordinates:
GTK's symbolic icon renderer drops a transformed group. -->
<path fill="#222222" d="M 2,8 A 6,6 0 0 1 12.91,4.56 L 11.48,5.56 A 4.25,4.25 0 0 0 3.75,8 Z"/>
<path fill="#222222" d="M 14.23,3.64 L 14.03,7.68 L 10.17,6.48 Z"/>
<path fill="#222222" d="M 14,8 A 6,6 0 0 1 3.09,11.44 L 4.52,10.44 A 4.25,4.25 0 0 0 12.25,8 Z"/>
<path fill="#222222" d="M 1.77,12.36 L 1.97,8.32 L 5.83,9.52 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 733 B

@@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Up to date: a check knocked out of a filled circle. -->
<path fill="#222222" fill-rule="evenodd" d="M 8,1 A 7,7 0 1 0 8,15 A 7,7 0 1 0 8,1 Z M 4.1,8.6 L 5.2,7.5 L 7,9.3 L 10.8,5.4 L 11.9,6.5 L 7,11.5 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 305 B

@@ -0,0 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 16 16">
<!-- Sharing stopped: the share glyph with its last node crossed out. Not
a no-entry sign, which reads as an error. -->
<path fill="#222222" d="M 12,1.25 A 2.25,2.25 0 1 0 12,5.75 A 2.25,2.25 0 1 0 12,1.25 Z M 4,5.75 A 2.25,2.25 0 1 0 4,10.25 A 2.25,2.25 0 1 0 4,5.75 Z"/>
<path fill="#222222" d="M 4.37,8.65 L 12.37,4.15 L 11.63,2.85 L 3.63,7.35 Z"/>
<path fill="#222222" d="M 3.63,8.65 L 8.23,11.25 L 8.97,9.95 L 4.37,7.35 Z"/>
<path fill="#222222" d="M 9.28,11.02 L 13.48,15.22 L 14.72,13.98 L 10.52,9.78 Z"/>
<path fill="#222222" d="M 13.48,9.78 L 9.28,13.98 L 10.52,15.22 L 14.72,11.02 Z"/>
</svg>

After

Width:  |  Height:  |  Size: 706 B

@@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" width="48" height="48" viewBox="0 0 48 48">
<path fill="#6d4aff" d="M4 12a4 4 0 0 1 4-4h11.2a4 4 0 0 1 2.9 1.2L25 12h15a4 4 0 0 1 4 4v22a4 4 0 0 1-4 4H8a4 4 0 0 1-4-4z"/>
<path fill="#8a6eff" d="M4 19a3 3 0 0 1 3-3h34a3 3 0 0 1 3 3v19a4 4 0 0 1-4 4H8a4 4 0 0 1-4-4z"/>
<path fill="#ffffff" fill-opacity="0.18" d="M7 16h34a3 3 0 0 1 3 3v1a3 3 0 0 0-3-3H7a3 3 0 0 0-3 3v-1a3 3 0 0 1 3-3z"/>
</svg>

After

Width:  |  Height:  |  Size: 442 B

+18 -1
View File
@@ -7,16 +7,33 @@
<file preprocess="xml-stripblanks">scalable/actions/pdfs-build-thumbnails-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-folder-upload-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-upload-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-filter-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-offline-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-online-only-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-cached-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-details-pane-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-link-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-map-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-menu-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-people-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-share-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-sync-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-synced-symbolic.svg</file>
<file preprocess="xml-stripblanks">scalable/actions/pdfs-unshare-symbolic.svg</file>
<!-- The Proton theme's folder: a full-colour icon from the icon theme
can't be recoloured by CSS. -->
<file preprocess="xml-stripblanks">scalable/places/pdfs-proton-folder.svg</file>
<!-- The app icon, from the one file packaging installs, so the in-app brand
matches the launcher and resolves even when nothing is installed. -->
<file alias="scalable/apps/io.narl.proton-drive-linux.svg" preprocess="xml-stripblanks">../../../packaging/io.narl.proton-drive-linux.svg</file>
</gresource>
<!-- The app stylesheet, loaded once at startup. Colours come from the
libadwaita palette so the app follows the system accent, light/dark and
high contrast; the Proton purple accent is a separate, opt-in provider. -->
high contrast. The Proton theme is a separate, opt-in provider. -->
<gresource prefix="/de/nils/protondrivelinux">
<file>style.css</file>
<file>proton-theme.css</file>
<!-- The search launcher's own sheet, over the same palette. -->
<file>prompt.css</file>
</gresource>
</gresources>
+224
View File
@@ -0,0 +1,224 @@
/* The search launcher (pdfs-prompt). Colours come from the libadwaita
palette, so the launcher follows the system accent, light/dark and high
contrast like the app; the opt-in Proton theme recolours both alike. */
window.launcher-window {
background: transparent;
}
.launcher-card {
background-color: var(--view-bg-color);
color: var(--view-fg-color);
border: 1px solid var(--border-color);
border-radius: 14px;
box-shadow: 0 16px 48px rgb(0 0 0 / 35%);
}
.search-bar {
padding: 14px 18px;
border-bottom: 1px solid var(--border-color);
}
.search-icon {
-gtk-icon-size: 20px;
color: color-mix(in srgb, currentColor 55%, transparent);
}
.search-entry {
background: none;
border: none;
box-shadow: none;
outline: none;
padding: 2px 0;
font-size: 1.15rem;
}
.key-hint {
padding: 2px 8px;
border-radius: 6px;
background-color: color-mix(in srgb, currentColor 8%, transparent);
font-size: 0.75rem;
opacity: 0.55;
}
.chip-row {
padding: 10px 14px;
border-bottom: 1px solid var(--border-color);
}
.chip {
padding: 4px 14px;
min-height: 0;
border-radius: 999px;
background: none;
border: 1px solid var(--border-color);
box-shadow: none;
font-size: 0.86rem;
font-weight: 500;
}
.chip:hover {
background-color: color-mix(in srgb, currentColor 6%, transparent);
}
.chip:checked {
background-color: var(--accent-bg-color);
color: var(--accent-fg-color);
border-color: var(--accent-bg-color);
}
/* --- results ------------------------------------------------------------ */
.results {
padding: 2px 8px 10px 8px;
}
.section-header {
padding: 12px 10px 6px 10px;
}
.section-title {
font-size: 0.78rem;
font-weight: 700;
letter-spacing: 0.06em;
text-transform: uppercase;
opacity: 0.55;
}
.section-icon {
-gtk-icon-size: 14px;
opacity: 0.55;
}
.result-list {
background: none;
}
.result-row {
padding: 6px 10px;
border-radius: 10px;
}
.result-row:hover {
background-color: color-mix(in srgb, currentColor 5%, transparent);
}
/* A soft tint rather than a solid accent bar: the text keeps its colour, so
the highlighted match stays readable on the selected row. */
.result-row:selected,
.result-row:selected:hover {
background-color: color-mix(in srgb, var(--accent-bg-color) 18%, transparent);
color: inherit;
}
.result-name {
font-weight: 500;
}
.result-location,
.result-meta {
font-size: 0.8rem;
opacity: 0.55;
}
.result-row:selected .result-location,
.result-row:selected .result-meta {
opacity: 0.7;
}
.thumb {
border-radius: 6px;
}
.result-badge,
.preview-badge {
-gtk-icon-size: 14px;
}
.badge-pinned {
color: var(--success-color);
}
.badge-cached {
color: var(--accent-color);
}
.badge-cloud,
.badge-local {
opacity: 0.55;
}
/* --- preview ------------------------------------------------------------ */
.preview {
padding: 16px;
background-color: color-mix(in srgb, currentColor 3%, transparent);
}
.preview-thumb {
margin-bottom: 12px;
}
.preview-thumb .thumb {
border-radius: 10px;
}
.preview-name {
font-weight: 700;
font-size: 1.05rem;
}
.preview-kind {
font-size: 0.85rem;
opacity: 0.6;
margin-bottom: 12px;
}
.preview-key {
font-size: 0.8rem;
opacity: 0.55;
}
.preview-value {
font-size: 0.85rem;
}
.preview-actions {
margin-top: 12px;
}
.preview-action {
padding: 6px 8px;
border-radius: 8px;
}
.preview-shortcut {
font-size: 0.75rem;
opacity: 0.6;
}
.preview-feedback {
margin-top: 6px;
font-size: 0.8rem;
color: var(--accent-color);
}
.preview-empty {
opacity: 0.2;
}
/* --- footer ------------------------------------------------------------- */
.footer {
padding: 8px 16px;
border-top: 1px solid var(--border-color);
}
.footer-text {
font-size: 0.78rem;
opacity: 0.5;
}
statuspage.compact > scrolledwindow > viewport > box {
margin: 24px 12px;
}
@@ -0,0 +1,72 @@
/* Proton theme — the Proton web apps' Snow (light) and Carbon (dark) palettes.
*
* Loaded by main.rs after libadwaita's own stylesheet, which it re-applies on
* top of the user's GTK theme so that theme's hard-coded colours can't show
* through. Only libadwaita's colour variables are set here; every widget
* derives its look from them. Values from ProtonMail/WebClients,
* packages/colors/themes/src/{snow,carbon}/standard-base.css.
*/
:root {
--accent-bg-color: #6d4aff;
--accent-fg-color: #ffffff;
--destructive-bg-color: #dc3251;
--destructive-fg-color: #ffffff;
--error-bg-color: #dc3251;
--error-fg-color: #ffffff;
--success-bg-color: #1ea885;
--success-fg-color: #ffffff;
--warning-bg-color: #ff9900;
--warning-fg-color: rgb(0 0 0 / 80%);
--window-bg-color: #ffffff;
--window-fg-color: #0c0c14;
--view-bg-color: #ffffff;
--view-fg-color: #0c0c14;
--headerbar-bg-color: #ffffff;
--headerbar-fg-color: #0c0c14;
--headerbar-backdrop-color: #ffffff;
--sidebar-bg-color: #f5f4f2;
--sidebar-fg-color: #0c0c14;
--sidebar-backdrop-color: #f8f8f6;
--secondary-sidebar-bg-color: #f8f8f6;
--secondary-sidebar-fg-color: #0c0c14;
--secondary-sidebar-backdrop-color: #f8f8f6;
--card-bg-color: #f5f4f2;
--card-fg-color: #0c0c14;
--dialog-bg-color: #ffffff;
--dialog-fg-color: #0c0c14;
--popover-bg-color: #ffffff;
--popover-fg-color: #0c0c14;
--thumbnail-bg-color: #f5f4f2;
--thumbnail-fg-color: #0c0c14;
}
@media (prefers-color-scheme: dark) {
:root {
--destructive-bg-color: #f5385a;
--error-bg-color: #f5385a;
--window-bg-color: #1e1c26;
--window-fg-color: #ffffff;
--view-bg-color: #1e1c26;
--view-fg-color: #ffffff;
--headerbar-bg-color: #1e1c26;
--headerbar-fg-color: #ffffff;
--headerbar-backdrop-color: #1e1c26;
--sidebar-bg-color: #16141c;
--sidebar-fg-color: #ffffff;
--sidebar-backdrop-color: #16141c;
--secondary-sidebar-bg-color: #1a1821;
--secondary-sidebar-fg-color: #ffffff;
--secondary-sidebar-backdrop-color: #1a1821;
--card-bg-color: #292733;
--card-fg-color: #ffffff;
--dialog-bg-color: #292733;
--dialog-fg-color: #ffffff;
--popover-bg-color: #292733;
--popover-fg-color: #ffffff;
--thumbnail-bg-color: #292733;
--thumbnail-fg-color: #ffffff;
}
}
+51 -12
View File
@@ -4,7 +4,8 @@
* var(--success-color), ...) so the app follows the system accent, light/dark
* and high contrast. The one exception is the photo viewer, which always sits
* on a near-black backdrop and so uses fixed light-on-dark colours. The Proton
* purple accent is opt-in and lives in its own provider (see main.rs).
* theme is opt-in and lives in its own provider (see main.rs and
* proton-theme.css).
*/
/* Brand */
@@ -18,6 +19,22 @@
.sidebar-status image.error { color: var(--error-color); }
.sidebar-quota progressbar trough, .sidebar-quota progressbar progress { min-height: 4px; }
.sidebar-account { padding: 4px 4px 4px 8px; }
/* A count of what waits on a page, at the end of its sidebar row */
.sidebar-badge {
min-width: 10px;
padding: 0 7px;
border-radius: 999px;
background: alpha(currentColor, 0.15);
font-size: 0.8em;
font-weight: 700;
}
/* Sync page status card: the state is the headline, the detail explains it */
.sync-card { padding: 18px 24px; }
.sync-card-icon { padding: 6px; }
.sync-card-icon.success { color: var(--success-color); }
.sync-card-icon.warning { color: var(--warning-color); }
.sync-card-icon.error { color: var(--error-color); }
/* File browser */
.file-grid { padding: 6px; }
@@ -26,13 +43,12 @@
.file-tile:selected, .file-tile:hover:selected { background: color-mix(in srgb, var(--accent-bg-color) 20%, transparent); }
.file-thumbnail { border-radius: 7px; }
.file-badge { -gtk-icon-shadow: 0 1px 2px rgba(0, 0, 0, 0.5); }
.badge-pinned { color: var(--yellow-3); }
.badge-pinned { color: var(--success-color); }
.badge-cached { color: var(--success-color); }
.badge-cloud { color: color-mix(in srgb, currentColor 55%, transparent); }
.badge-issue { color: var(--error-color); }
.browser-statusbar { background-color: alpha(currentColor, 0.025); }
scale.browser-status-meter trough, progressbar.browser-status-meter trough { min-width: 104px; }
scale.browser-status-meter trough { min-height: 6px; }
progressbar.browser-status-meter trough, progressbar.browser-status-meter progress { min-height: 6px; }
scale.browser-status-meter trough { min-width: 104px; min-height: 6px; }
.bulk-bar { padding: 6px 8px; border-radius: 12px; background: color-mix(in srgb, var(--accent-bg-color) 12%, transparent); }
.dropzone { border: 2px dashed alpha(currentColor, 0.25); border-radius: 16px; padding: 28px 18px; transition: background 160ms ease, border-color 160ms ease; }
.dropzone-active { border-color: var(--accent-bg-color); background: color-mix(in srgb, var(--accent-bg-color) 10%, transparent); }
@@ -51,19 +67,19 @@ progressbar.browser-status-meter trough, progressbar.browser-status-meter progre
.viewer-nav-btn { background-color: rgba(0, 0, 0, 0.45); color: white; margin: 20px; min-width: 44px; min-height: 44px; border-radius: 50%; opacity: 0.7; transition: opacity 150ms ease, background-color 150ms ease; }
.viewer-nav-btn:hover { background-color: rgba(0, 0, 0, 0.85); color: white; opacity: 1; }
.viewer-nav-btn:disabled { opacity: 0; }
.photo-scrubber { min-width: 20px; padding: 0 2px; }
.photo-scrubber value { font-weight: 600; background-color: var(--popover-bg-color); border-radius: 6px; padding: 2px 8px; }
.photo-scrubber { background: linear-gradient(to left, alpha(var(--window-bg-color), 0.92) 40%, alpha(var(--window-bg-color), 0)); }
.photo-scrubber-knob { background-color: var(--accent-bg-color); border-radius: 2px; }
.photo-scrubber-bubble { font-weight: 600; background-color: var(--popover-bg-color); color: var(--popover-fg-color); border-radius: 999px; padding: 4px 12px; box-shadow: 0 1px 4px rgba(0, 0, 0, 0.3); }
.viewer-spinner { color: white; }
.viewer-status { color: white; font-size: 1rem; background-color: rgba(0, 0, 0, 0.75); padding: 12px 24px; border-radius: 12px; }
.viewer-info-panel { background-color: var(--window-bg-color); border-left: 1px solid alpha(currentColor, 0.12); }
/* Photos grid */
.gallery-day { font-size: 0.82rem; font-weight: 700; letter-spacing: 0.04em; text-transform: uppercase; color: alpha(currentColor, 0.65); padding: 4px 2px 8px 2px; }
.photo-tile { padding: 0; margin: 0; min-height: 0; min-width: 0; border-radius: 6px; background: none; box-shadow: none; transition: box-shadow 180ms ease; }
.photo-tile:hover { box-shadow: 0 8px 22px rgba(0, 0, 0, 0.45); }
.gallery-day { font-size: 0.82rem; font-weight: 700; letter-spacing: 0.04em; text-transform: uppercase; color: alpha(currentColor, 0.65); padding: 2px 2px 6px 2px; }
.photo-tile { padding: 0; margin: 0; min-height: 0; min-width: 0; border: none; border-radius: 4px; background: none; box-shadow: none; }
.photo-tile:focus { outline: 2px solid var(--accent-color); outline-offset: -2px; }
.photo-thumb { transition: transform 220ms ease; }
.photo-tile:hover .photo-thumb { transform: scale(1.06); }
.photo-tile:hover .photo-thumb { transform: scale(1.02); }
.photo-check { color: white; background: rgba(0, 0, 0, 0.5); border-radius: 999px; padding: 3px; }
.photo-check-on { background: var(--accent-bg-color); color: var(--accent-fg-color); }
.photo-tile-selected { outline: 3px solid var(--accent-bg-color); outline-offset: -3px; }
@@ -71,14 +87,37 @@ progressbar.browser-status-meter trough, progressbar.browser-status-meter progre
.photo-caption { font-size: 0.78rem; color: white; text-shadow: 0 1px 3px rgba(0, 0, 0, 0.9); padding: 22px 10px 6px 10px; opacity: 0; transition: opacity 180ms ease; }
.photo-tile:hover .photo-caption { opacity: 1; background: linear-gradient(to top, rgba(0, 0, 0, 0.55), rgba(0, 0, 0, 0)); }
.photo-video-badge { color: white; background: rgba(0, 0, 0, 0.45); border-radius: 999px; padding: 8px; min-width: 20px; min-height: 20px; box-shadow: 0 2px 8px rgba(0, 0, 0, 0.5); transition: background 160ms ease; }
.photo-video-badge-small { padding: 3px; min-width: 0; min-height: 0; }
.photo-tile:hover .photo-video-badge { background: color-mix(in srgb, var(--accent-bg-color) 85%, transparent); }
.photo-group-badge { font-size: 0.68rem; font-weight: 700; letter-spacing: 0.04em; color: white; background: rgba(0, 0, 0, 0.55); border-radius: 6px; padding: 1px 5px; margin: 6px; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.9); }
.memory-card { padding: 0; border-radius: 12px; }
.memory-cover { border-radius: 12px; background: alpha(currentColor, 0.06); }
.memory-label { font-weight: 700; color: white; text-shadow: 0 1px 3px rgba(0, 0, 0, 0.9); padding: 22px 10px 6px 10px; background: linear-gradient(to top, rgba(0, 0, 0, 0.6), rgba(0, 0, 0, 0)); }
.album-card { padding: 0; border-radius: 16px; transition: background 180ms ease; }
.album-card:hover { background: alpha(currentColor, 0.07); }
.album-cover { border-radius: 16px; background: alpha(currentColor, 0.06); box-shadow: 0 4px 16px rgba(0, 0, 0, 0.32); transition: box-shadow 180ms ease; }
.album-card:hover .album-cover { box-shadow: 0 10px 26px rgba(0, 0, 0, 0.5); }
.album-card:hover .photo-thumb { transform: scale(1.05); }
.view-switch button { padding-left: 18px; padding-right: 18px; }
.view-switch button, .view-switch toggle { padding-left: 18px; padding-right: 18px; }
/* Shared with Me: what a share lets me do, as a tinted pill beside the name. */
.role-pill { padding: 2px 10px; border-radius: 999px; font-weight: 600; color: var(--accent-color); background-color: color-mix(in srgb, var(--accent-bg-color) 16%, transparent); }
.role-pill.warning-pill { color: var(--warning-color); background-color: color-mix(in srgb, var(--warning-bg-color) 20%, transparent); }
/* Activity: the feed sits on the window background as one card per row, under
* day headings, like a boxed list with its rows set apart. */
listview.activity-feed { background: none; color: inherit; padding: 0 12px 18px 12px; }
listview.activity-feed > row { padding: 0; margin-bottom: 6px; background: none; }
listview.activity-feed > header { padding: 18px 6px 6px 6px; background: none; }
listview.activity-feed row.card { border-radius: 12px; }
/* Photos: how many timeline filters are on, as a small badge on the Filters button. */
.filter-count { min-width: 18px; padding: 0 5px; border-radius: 999px; font-size: 0.8em; font-weight: 700; color: var(--accent-fg-color); background-color: var(--accent-bg-color); }
/* The Places map: sea and land come in the map's own colours; markers (drawn
* in the area's own colour) in the accent. */
.places-map-frame { border-radius: 12px; }
.places-map { color: var(--accent-bg-color); }
/* Credit on the Places street map, in the corner as on every web map. */
.places-spot-photo { padding: 0; }
.places-spot-photo picture { border-radius: 6px; overflow: hidden; }
.places-map-credit { padding: 2px 8px; border-top-left-radius: 8px; color: var(--window-fg-color); background: color-mix(in srgb, var(--window-bg-color) 80%, transparent); }
+139
View File
@@ -0,0 +1,139 @@
//! Dates and times as the app shows them, so every page words a moment the
//! same way.
//!
//! - [`relative`] for "when did this happen" in lists: "Just now", "5 min
//! ago", "Yesterday, 14:05", "Tuesday", then a date.
//! - [`short_date`] for date columns: "Sep 23", with the year only when it is
//! not this year.
//! - [`full`] for tooltips: the weekday, date, year and time.
//!
//! The formats are `strftime` patterns that go through gettext, so they follow
//! the language picked in Preferences, not only the system locale.
use gtk4::glib;
use crate::{gettext, gettext_f, ngettext_f};
/// Seconds in a day, for the "within the last week" checks.
const DAY: i64 = 86_400;
/// A past moment in epoch seconds, relative to now. Empty when the time can't
/// be read.
pub(crate) fn relative(secs: i64) -> String {
match (
glib::DateTime::from_unix_local(secs),
glib::DateTime::now_local(),
) {
(Ok(at), Ok(now)) => relative_to(&at, &now),
_ => String::new(),
}
}
/// [`relative`] for a fixed `now`, so it can be tested.
pub(crate) fn relative_to(at: &glib::DateTime, now: &glib::DateTime) -> String {
let ago = now.to_unix() - at.to_unix();
// A clock a little ahead of ours still reads as "just now".
if (-60..60).contains(&ago) {
return gettext("Just now");
}
if (60..3600).contains(&ago) {
let minutes = (ago / 60) as u64;
return ngettext_f("{n} min ago", "{n} min ago", minutes, &[]);
}
// Translators: strftime format for a time of day, such as "14:05".
let time = || format(at, &gettext("%H:%M"));
if at.ymd() == now.ymd() {
return time();
}
if now.add_days(-1).is_ok_and(|y| y.ymd() == at.ymd()) {
// Translators: {time} is a time of day, such as "14:05".
return gettext_f("Yesterday, {time}", &[("time", &time())]);
}
if (0..6 * DAY).contains(&ago) {
return format(at, "%A");
}
short_date_to(at, now)
}
/// A date in epoch seconds, without the time. Empty when the time can't be
/// read.
pub(crate) fn short_date(secs: i64) -> String {
match (
glib::DateTime::from_unix_local(secs),
glib::DateTime::now_local(),
) {
(Ok(at), Ok(now)) => short_date_to(&at, &now),
_ => String::new(),
}
}
/// [`short_date`] for a fixed `now`, so it can be tested.
pub(crate) fn short_date_to(at: &glib::DateTime, now: &glib::DateTime) -> String {
let pattern = if at.year() == now.year() {
// Translators: strftime format for a date in this year, such as "Sep 23".
gettext("%b %-d")
} else {
// Translators: strftime format for a date in another year, such as "Sep 23, 2025".
gettext("%b %-d, %Y")
};
format(at, &pattern)
}
/// The whole moment in epoch seconds, for a tooltip. Empty when the time
/// can't be read.
pub(crate) fn full(secs: i64) -> String {
glib::DateTime::from_unix_local(secs)
.map(|at| {
// Translators: strftime format for a full date and time, such as
// "Wednesday, September 23, 2026, 14:05".
format(&at, &gettext("%A, %B %-d, %Y, %H:%M"))
})
.unwrap_or_default()
}
fn format(at: &glib::DateTime, pattern: &str) -> String {
at.format(pattern)
.map(|s| s.trim().to_string())
.unwrap_or_default()
}
#[cfg(test)]
mod tests {
use super::*;
fn at(y: i32, m: i32, d: i32, h: i32, min: i32) -> glib::DateTime {
glib::DateTime::from_local(y, m, d, h, min, 0.0).unwrap()
}
#[test]
fn relative_times_get_coarser_with_age() {
let now = at(2026, 9, 23, 15, 0);
assert_eq!(relative_to(&at(2026, 9, 23, 14, 59), &now), "1 min ago");
assert_eq!(relative_to(&at(2026, 9, 23, 14, 55), &now), "5 min ago");
assert_eq!(relative_to(&at(2026, 9, 23, 9, 5), &now), "09:05");
assert_eq!(
relative_to(&at(2026, 9, 22, 14, 5), &now),
"Yesterday, 14:05"
);
assert_eq!(relative_to(&at(2026, 9, 20, 10, 0), &now), "Sunday");
assert_eq!(relative_to(&at(2026, 9, 1, 10, 0), &now), "Sep 1");
assert_eq!(relative_to(&at(2024, 3, 5, 10, 0), &now), "Mar 5, 2024");
}
#[test]
fn a_moment_ago_or_slightly_ahead_is_just_now() {
let now = at(2026, 9, 23, 15, 0);
assert_eq!(relative_to(&now, &now), "Just now");
assert_eq!(
relative_to(&at(2026, 9, 23, 15, 0), &now.add_seconds(30.0).unwrap()),
"Just now"
);
}
#[test]
fn short_dates_show_the_year_only_when_it_differs() {
let now = at(2026, 9, 23, 15, 0);
assert_eq!(short_date_to(&at(2026, 1, 2, 8, 0), &now), "Jan 2");
assert_eq!(short_date_to(&at(2025, 12, 31, 8, 0), &now), "Dec 31, 2025");
}
}
+179
View File
@@ -0,0 +1,179 @@
//! The window's side of the daemon's event feed ([`Request::Subscribe`]).
//!
//! The window used to ask the daemon for its status every two seconds and let
//! pages re-read themselves on timers. Now one thread follows the feed and hands
//! each event to the main loop: a status or transfers snapshot is painted as it
//! comes, and a change reloads the page on screen if the page shows it. A
//! window nobody can see keeps its changes for later and catches up when it is
//! focused again.
use std::collections::BTreeSet;
use pdfs_core::control::{Event, Feed, follow};
use crate::*;
/// How long the conflict list waits after a change before it is read again.
/// Listing walks every node the daemon knows, and a sync pass can report
/// changes many times a second.
const CONFLICTS_DEBOUNCE: Duration = Duration::from_secs(3);
/// How long the import's state waits after its progress moved before it is
/// read again.
const TAKEOUT_DEBOUNCE: Duration = Duration::from_secs(2);
/// How old the invitations may be when the window is focused. Another person
/// inviting this account sends the daemon no event, so focus is when they are
/// looked for.
const INVITATIONS_ON_FOCUS: Duration = Duration::from_secs(60);
/// What the feed keeps between events.
#[derive(Default)]
struct FeedState {
/// What changed while the window was out of sight.
deferred: RefCell<BTreeSet<Topic>>,
/// Whether a conflict read is already waiting out [`CONFLICTS_DEBOUNCE`].
conflicts_pending: Rc<Cell<bool>>,
/// Whether an import read is already waiting out [`TAKEOUT_DEBOUNCE`].
takeout_pending: Rc<Cell<bool>>,
}
/// Follow the daemon for as long as `window` is open.
pub(crate) fn follow_daemon(ui: &Rc<Ui>, window: &adw::ApplicationWindow) {
let (tx, rx) = async_channel::bounded(64);
let socket = ui.dirs.control_socket();
let spawned = std::thread::Builder::new()
.name("pdfs-feed".into())
.spawn(move || follow(&socket, |feed| tx.send_blocking(feed).is_ok()));
if let Err(e) = spawned {
tracing::error!("cannot start the event feed: {e}");
return;
}
let state = Rc::new(FeedState::default());
let ui_feed = ui.clone();
let state_feed = state.clone();
let window_feed = window.downgrade();
let rx_feed = rx.clone();
glib::spawn_future_local(async move {
while let Ok(feed) = rx_feed.recv().await {
let hidden = window_feed.upgrade().is_none_or(|w| window_hidden(&w));
on_feed(&ui_feed, &state_feed, feed, hidden);
}
});
// Catch up as soon as the user comes back to the window.
let ui_focus = ui.clone();
window.connect_is_active_notify(move |window| {
if !window.is_active() {
return;
}
let topics: Vec<Topic> = state.deferred.take().into_iter().collect();
if !topics.is_empty() {
follow_changes(&ui_focus, &state, &topics);
}
refresh_quota(&ui_focus);
refresh_invitations_badge(&ui_focus, INVITATIONS_ON_FOCUS);
});
// Closing the channel ends the loop above, and the thread with the next
// event it tries to hand over.
window.connect_close_request(move |_| {
rx.close();
glib::Propagation::Proceed
});
}
fn on_feed(ui: &Rc<Ui>, state: &Rc<FeedState>, feed: Feed, hidden: bool) {
match feed {
// The daemon follows up with its whole state, so there is nothing to
// ask for here.
Feed::Connected | Feed::Event(Event::Heartbeat) => {}
Feed::Event(Event::Status(status)) => {
paint_status(ui, *status);
// Uploads are what move the quota, and they move the status too.
refresh_quota(ui);
}
// Painted even out of sight: the end of a batch is what sends the
// "Sync complete" notification.
Feed::Event(Event::Transfers { items, jobs }) => {
repaint_transfers(ui, &items, &jobs);
follow_takeout(ui, state, &jobs);
}
Feed::Event(Event::Changed { topics }) if hidden => {
state.deferred.borrow_mut().extend(topics);
}
Feed::Event(Event::Changed { topics }) => follow_changes(ui, state, &topics),
// Ask once, which paints "not connected" or "not responding", and clear
// progress that can no longer move.
Feed::Disconnected => {
refresh_status(ui);
repaint_transfers(ui, &[], &[]);
}
}
}
/// Reload what `topics` says changed: the sidebar badges wherever the user
/// is, and the page on screen if it shows one of them.
fn follow_changes(ui: &Rc<Ui>, state: &FeedState, topics: &[Topic]) {
let changed = |topic| topics.contains(&topic);
if changed(Topic::Conflicts) {
let ui = ui.clone();
debounce(&state.conflicts_pending, CONFLICTS_DEBOUNCE, move || {
refresh_conflicts(&ui);
if activity_visible(&ui) {
refresh_activity_conflicts(&ui, true);
}
});
}
if changed(Topic::Shares) {
refresh_invitations_badge(ui, Duration::ZERO);
}
match ui.stack.visible_child_name().as_deref() {
Some("browser") if changed(Topic::Files) => load_browser(ui),
// The timeline is left alone: reloading it would throw away the place
// the user scrolled to. F5 reloads it on request.
Some("gallery")
if changed(Topic::Photos)
&& ui.gallery.content.visible_child_name().as_deref() == Some("albums") =>
{
load_albums(ui)
}
Some("trash") if changed(Topic::Trash) => load_trash(ui),
Some("shared") if changed(Topic::Shares) => load_shared(ui),
Some("sharedbyme") if changed(Topic::Shares) || changed(Topic::Files) => {
load_shared_by_me(ui)
}
Some("devices") if changed(Topic::Devices) => load_devices(ui),
Some("locations") => follow_sync_view(ui, topics),
_ => {}
}
}
/// Read the import's state again while its progress moves, and once more after
/// it stops, which is how a finished import is noticed. Nothing is read while
/// no import runs and the page is elsewhere.
fn follow_takeout(ui: &Rc<Ui>, state: &FeedState, jobs: &[JobItem]) {
let importing = jobs.iter().any(|job| job.title == IMPORT_JOB);
if importing
|| ui.takeout.running.get()
|| ui.stack.visible_child_name().as_deref() == Some("takeout")
{
let ui = ui.clone();
debounce(&state.takeout_pending, TAKEOUT_DEBOUNCE, move || {
refresh_takeout(&ui)
});
}
}
/// Run `f` once `delay` has passed, unless a run is already waiting. The run
/// sees everything that happened up to it, so nothing is missed by skipping.
fn debounce(pending: &Rc<Cell<bool>>, delay: Duration, f: impl FnOnce() + 'static) {
if pending.replace(true) {
return;
}
let pending = pending.clone();
glib::timeout_add_local_once(delay, move || {
pending.set(false);
f();
});
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,714 @@
//! Activity entries as the feed words them.
//!
//! The daemon logs each entry in plain English for its own log and the CLI:
//! a target ("report.pdf", "3 item(s)", "sync conflict in Docs") and a detail
//! ("to /Documents", "was notes.txt", "2 uploaded, 1 deleted"). This module
//! reads the shapes the daemon writes and says them again in the user's
//! language. A shape it does not know, which is mostly an error message,
//! shows as the daemon wrote it.
use crate::*;
/// What a feed row says about one entry.
#[derive(Clone, Debug, PartialEq)]
pub(crate) struct Described {
pub(crate) title: String,
/// Subtitle parts, shown joined by " · ".
pub(crate) details: Vec<String>,
}
/// The title and subtitle parts for `entry`.
pub(crate) fn describe(entry: &ActivityEntry) -> Described {
let title = entry_title(entry);
let mut details = Vec::new();
if entry.kind == ActivityKind::Conflict
&& let Some(stamp) = conflict_stamp(&entry.target)
{
let when = dates::relative(stamp);
if !when.is_empty() {
// Translators: {time} is when a conflicting copy was made, such as
// "Sep 21" or "5 min ago".
details.push(gettext_f("Made {time}", &[("time", &when)]));
}
}
if let Some(detail) = detail_label(entry) {
details.insert(0, detail);
}
Described { title, details }
}
/// The row title: what happened to what, one sentence per case so a
/// translator can place the name where their language needs it.
fn entry_title(entry: &ActivityEntry) -> String {
use ActivityKind::*;
let target = entry.target.as_str();
let ok = entry.ok;
// Entries whose target is a count or a fixed phrase rather than a name.
match (entry.kind, target) {
(Upload, "Google Photos import") => {
return if ok {
gettext("Imported from Google Photos")
} else {
gettext("Couldn't import from Google Photos")
};
}
(Upload, "bulk upload") => return gettext("Couldn't upload files"),
(Upload, "add sync folder") => return gettext("Couldn't add a sync folder"),
(Download, "restore folders") => return gettext("Couldn't restore folders"),
(Unshare, "shared item") => return gettext("Left a shared item"),
_ => {}
}
if matches!(entry.kind, Restore | DeleteForever | EmptyTrash)
&& let Some(n) = leading_count(target, &["item(s)", "item", "items"])
{
return match entry.kind {
Restore => ngettext_f("Restored {n} item", "Restored {n} items", n, &[]),
DeleteForever => ngettext_f(
"Deleted {n} item forever",
"Deleted {n} items forever",
n,
&[],
),
_ => ngettext_f(
"Emptied the Trash ({n} item)",
"Emptied the Trash ({n} items)",
n,
&[],
),
};
}
if entry.kind == Download
&& let Some(rest) = target.strip_prefix("restored ")
&& let Some(n) = leading_count(rest, &["folder(s)"])
{
return ngettext_f("Restored {n} folder", "Restored {n} folders", n, &[]);
}
if entry.kind == Sync
&& let Some(title) = sync_title(target)
{
return title;
}
let Some(name) = target_name(target) else {
return match entry.kind {
Share => gettext("Shared an item"),
PublicLink => gettext("Created a link to an item"),
Unshare if entry.detail == "link removed" => gettext("Removed a link to an item"),
Unshare => gettext("Stopped sharing an item"),
_ => activity_title(entry.kind, ok, target),
};
};
if entry.kind == Unshare && entry.detail == "link removed" {
// Translators: {name} is a file or folder name.
return gettext_f("Removed the link to {name}", &[("name", &name)]);
}
if entry.kind == Conflict
&& let Some(base) = conflict_base(&name)
{
// Translators: {name} is the file the conflicting copy was made from.
return gettext_f("Conflicting copy of {name}", &[("name", &base)]);
}
activity_title(entry.kind, ok, &name)
}
/// The title of a sync pass or mode switch, whose target is a phrase.
fn sync_title(target: &str) -> Option<String> {
if let Some(name) = target.strip_prefix("sync conflict in ") {
// Translators: {name} is a synced folder's name.
return Some(gettext_f(
"Conflicting changes in {name}",
&[("name", name)],
));
}
if let Some(name) = target.strip_prefix("sync incomplete for ") {
// Translators: {name} is a synced folder's name.
return Some(gettext_f(
"Some items in {name} didn't sync",
&[("name", name)],
));
}
if let Some(name) = target.strip_prefix("sync failed for ") {
// Translators: {name} is a synced folder's name.
return Some(gettext_f("Couldn't sync {name}", &[("name", name)]));
}
if let Some(name) = target.strip_prefix("couldn't switch ") {
// Translators: {name} is a synced folder's name.
return Some(gettext_f(
"Couldn't change how {name} syncs",
&[("name", name)],
));
}
if let Some(path) = target.strip_suffix(" is now on-demand") {
let name = file_name(path);
// Translators: {name} is a folder's name.
return Some(gettext_f("{name} is now online only", &[("name", name)]));
}
if let Some(path) = target.strip_suffix(" is mirroring again; downloading") {
let name = file_name(path);
// Translators: {name} is a folder's name.
return Some(gettext_f("{name} is synced again", &[("name", name)]));
}
if target.starts_with("already ") {
return Some(gettext("Sync mode was already set"));
}
None
}
/// The row title for an activity on a named item.
pub(crate) fn activity_title(kind: ActivityKind, ok: bool, name: &str) -> String {
let args = [("name", name)];
match (kind, ok) {
// Translators: {name} is a file or folder name.
(ActivityKind::Upload, true) => gettext_f("Uploaded {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Upload, false) => gettext_f("Couldn't upload {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Download, true) => gettext_f("Downloaded {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Download, false) => gettext_f("Couldn't download {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Sync, true) => gettext_f("Synced {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Sync, false) => gettext_f("Couldn't sync {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Rename, true) => gettext_f("Renamed {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Rename, false) => gettext_f("Couldn't rename {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Move, true) => gettext_f("Moved {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Move, false) => gettext_f("Couldn't move {name}", &args),
// Translators: {name} is a folder name.
(ActivityKind::CreateFolder, true) => gettext_f("Created folder {name}", &args),
// Translators: {name} is a folder name.
(ActivityKind::CreateFolder, false) => gettext_f("Couldn't create folder {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Trash, true) => gettext_f("Moved {name} to the Trash", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Trash, false) => gettext_f("Couldn't move {name} to the Trash", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Restore, true) => gettext_f("Restored {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Restore, false) => gettext_f("Couldn't restore {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::DeleteForever, true) => gettext_f("Deleted {name} forever", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::DeleteForever, false) => gettext_f("Couldn't delete {name}", &args),
(ActivityKind::EmptyTrash, true) => gettext("Emptied the Trash"),
(ActivityKind::EmptyTrash, false) => gettext("Couldn't empty the Trash"),
// Translators: {name} is a file or folder name.
(ActivityKind::Share, true) => gettext_f("Shared {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Share, false) => gettext_f("Couldn't share {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::PublicLink, true) => gettext_f("Created a link to {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::PublicLink, false) => gettext_f("Couldn't create a link to {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Unshare, true) => gettext_f("Stopped sharing {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Unshare, false) => gettext_f("Couldn't stop sharing {name}", &args),
// Translators: {name} is the name of the conflicting copy.
(ActivityKind::Conflict, _) => gettext_f("Conflicting copy {name}", &args),
}
}
/// The subtitle's first part: the daemon's detail in the user's language, or
/// `None` when it only repeats the title.
fn detail_label(entry: &ActivityEntry) -> Option<String> {
let detail = entry.detail.trim();
let name = |s: &str| file_name(s).to_string();
match detail {
""
| "uploaded"
| "created"
| "trashed"
| "trashed from the mount"
| "stopped sharing"
| "left"
| "link created"
| "link removed"
| "access removed"
| "removed" => {
return None;
}
"removed on Drive" => return Some(gettext("Removed on Proton Drive")),
"removed locally" => return Some(gettext("Removed on this computer")),
"replaced by a rename from the mount" => {
return Some(gettext("Replaced by a renamed file"));
}
"destination folder no longer exists; the file was left in place" => {
return Some(gettext(
"The destination folder no longer exists, so the file stayed where it was",
));
}
_ => {}
}
if entry.kind == ActivityKind::Upload
&& entry.target == "Google Photos import"
&& entry.ok
&& let Some(summary) = import_summary(detail)
{
return Some(summary);
}
if entry.kind == ActivityKind::Sync
&& let Some(summary) = sync_summary(detail)
{
return Some(summary);
}
if let Some(dest) = detail.strip_prefix("to ") {
// Translators: {name} is the folder an item was moved into.
return Some(gettext_f("To {name}", &[("name", &folder_name(dest))]));
}
if let Some(old) = detail.strip_prefix("was ") {
// Translators: {name} is an item's previous name.
return Some(gettext_f("Previously {name}", &[("name", &name(old))]));
}
if detail.starts_with("restored version ") {
return Some(gettext("Restored an earlier version"));
}
if detail.starts_with("deleted version ") {
return Some(gettext("Deleted an earlier version"));
}
if let Some(rest) = detail.strip_prefix("saved version ")
&& let Some((_, path)) = rest.split_once(" to ")
{
// Translators: {path} is where an earlier version was saved.
return Some(gettext_f(
"Saved an earlier version to {path}",
&[("path", &tilde_path(path))],
));
}
if let Some(copy) = detail.strip_prefix("kept both; the copy is now ") {
// Translators: {name} is the new name of the conflicting copy.
return Some(gettext_f(
"Kept both; the copy is now {name}",
&[("name", copy)],
));
}
if let Some(base) = detail
.strip_prefix("kept ")
.and_then(|rest| rest.strip_suffix("; the copy is in Trash"))
{
// Translators: {name} is the file that was kept.
return Some(gettext_f(
"Kept {name}; the copy is in the Trash",
&[("name", base)],
));
}
if detail.starts_with("kept the conflict copy ") {
return Some(gettext("Kept the conflicting copy"));
}
if let Some(base) = detail.strip_prefix("kept ") {
// Translators: {name} is the file that was kept.
return Some(gettext_f("Kept {name}", &[("name", base)]));
}
if let Some(base) = detail.strip_prefix("auto-removed duplicate of ") {
// Translators: {name} is the file the removed copy was identical to.
return Some(gettext_f(
"Same as {name}, so it was removed",
&[("name", base)],
));
}
if let Some(alt) = detail.strip_prefix("destination already had that name; moved as ") {
// Translators: {name} is the name the item was given instead.
return Some(gettext_f(
"The name was taken, so it was moved as {name}",
&[("name", alt)],
));
}
if let Some(alt) = detail.strip_prefix("name was taken remotely; renamed to ") {
// Translators: {name} is the name the item was given instead.
return Some(gettext_f(
"The name was taken on Proton Drive, so it was renamed to {name}",
&[("name", alt)],
));
}
if let Some(alt) = detail.strip_prefix("name was taken remotely; created as ") {
// Translators: {name} is the name the item was given instead.
return Some(gettext_f(
"The name was taken on Proton Drive, so it was created as {name}",
&[("name", alt)],
));
}
if let Some(alt) =
detail.strip_prefix("its folder was trashed remotely; created in the root as ")
{
// Translators: {name} is the name the item was given.
return Some(gettext_f(
"Its folder was moved to the Trash on Proton Drive, so it was created at the top as {name}",
&[("name", alt)],
));
}
if let Some((reason, copy)) = detail.split_once("; local changes uploaded as ") {
let args = [("name", copy)];
return Some(match edit_conflict(reason) {
// Translators: {name} is the copy the changes made on this
// computer were uploaded as.
EditConflict::Gone => gettext_f(
"It no longer exists on Proton Drive, so your changes were uploaded as {name}",
&args,
),
// Translators: {name} is the copy the changes made on this
// computer were uploaded as.
EditConflict::Trashed => gettext_f(
"It was moved to the Trash on Proton Drive, so your changes were uploaded as {name}",
&args,
),
// Translators: {name} is the copy the changes made on this
// computer were uploaded as.
EditConflict::Changed => gettext_f(
"It was changed on Proton Drive, so your changes were uploaded as {name}",
&args,
),
// Translators: {name} is the copy the changes made on this
// computer were uploaded as.
EditConflict::Other => gettext_f("Your changes were uploaded as {name}", &args),
});
}
if let Some((reason, path)) = detail.split_once("; local changes kept at ") {
let path = tilde_path(path);
let args = [("path", path.as_str())];
return Some(match edit_conflict(reason) {
// Translators: {path} is the file on this computer that holds the
// changes which could not be uploaded.
EditConflict::Gone => gettext_f(
"It no longer exists on Proton Drive, so your changes were kept in {path}",
&args,
),
// Translators: {path} is the file on this computer that holds the
// changes which could not be uploaded.
EditConflict::Trashed => gettext_f(
"It was moved to the Trash on Proton Drive, so your changes were kept in {path}",
&args,
),
// Translators: {path} is the file on this computer that holds the
// changes which could not be uploaded.
EditConflict::Changed => gettext_f(
"It was changed on Proton Drive, so your changes were kept in {path}",
&args,
),
// Translators: {path} is the file on this computer that holds the
// changes which could not be uploaded.
EditConflict::Other => gettext_f("Your changes were kept in {path}", &args),
});
}
if let Some(base) = detail.strip_prefix("differs from ") {
// Translators: {name} is the file the conflicting copy was made from.
return Some(gettext_f("Differs from {name}", &[("name", base)]));
}
if let Some(base) = detail
.strip_prefix("no original ")
.and_then(|rest| rest.strip_suffix(" to reconcile against"))
{
// Translators: {name} is the file the conflicting copy was made from.
return Some(gettext_f("{name} no longer exists", &[("name", base)]));
}
if let Some(base) = detail
.strip_prefix("identical to ")
.and_then(|rest| rest.split_once(';').map(|(base, _)| base))
{
// Translators: {name} is the file the conflicting copy was made from.
return Some(gettext_f("Same as {name}", &[("name", base)]));
}
if let Some((n, role)) = detail
.split_once(" recipient(s) as ")
.and_then(|(n, role)| Some((n.parse::<u64>().ok()?, role)))
{
return Some(match role {
"viewer" => ngettext_f("{n} person can view", "{n} people can view", n, &[]),
"editor" => ngettext_f("{n} person can edit", "{n} people can edit", n, &[]),
_ => ngettext_f("{n} person invited", "{n} people invited", n, &[]),
});
}
if let Some(n) = leading_count(detail, &["file(s) kept as conflict copies"]) {
return Some(ngettext_f(
"{n} file kept as a conflicting copy",
"{n} files kept as conflicting copies",
n,
&[],
));
}
if let Some(n) = leading_count(detail, &["item(s) failed; will retry"]) {
return Some(ngettext_f(
"{n} item failed and will be tried again",
"{n} items failed and will be tried again",
n,
&[],
));
}
Some(capitalize(detail))
}
/// Why a queued edit could not land on its file, from the reason the daemon
/// gives before "; local changes uploaded as" or "; local changes kept at".
enum EditConflict {
Gone,
Trashed,
Changed,
Other,
}
fn edit_conflict(reason: &str) -> EditConflict {
match reason {
"the file no longer exists remotely" => EditConflict::Gone,
"the file was trashed remotely" => EditConflict::Trashed,
r if r.starts_with("the remote revision changed under the queued write") => {
EditConflict::Changed
}
_ => EditConflict::Other,
}
}
/// A sync pass summary such as "2 uploaded, 1 folder(s) created", as a
/// list of counts. `None` when a part is not one the daemon writes.
fn sync_summary(detail: &str) -> Option<String> {
let mut parts = Vec::new();
for part in detail.split(", ") {
let (n, label) = part.split_once(' ')?;
let n = n.parse::<u64>().ok()?;
parts.push(match label {
// Translators: part of a list summing up a sync pass.
"uploaded" => ngettext_f("{n} uploaded", "{n} uploaded", n, &[]),
// Translators: part of a list summing up a sync pass.
"downloaded" => ngettext_f("{n} downloaded", "{n} downloaded", n, &[]),
// Translators: part of a list summing up a sync pass.
"folder(s) created" => ngettext_f("{n} folder created", "{n} folders created", n, &[]),
// Translators: part of a list summing up a sync pass.
"deleted" => ngettext_f("{n} deleted", "{n} deleted", n, &[]),
// Translators: part of a list summing up a sync pass.
"conflicted" => ngettext_f("{n} in conflict", "{n} in conflict", n, &[]),
// Translators: part of a list summing up a sync pass; the files are
// still open in another app.
"deferred (open for write)" => ngettext_f("{n} still open", "{n} still open", n, &[]),
// Translators: part of a list summing up a sync pass.
"failed" => ngettext_f("{n} failed", "{n} failed", n, &[]),
_ => return None,
});
}
Some(parts.join(", "))
}
/// A Google Photos import summary such as "40 uploaded · 2 already there".
fn import_summary(detail: &str) -> Option<String> {
let mut parts = Vec::new();
for part in detail.split(" · ") {
if part == "cancelled" {
parts.push(gettext("Cancelled"));
continue;
}
let (n, label) = part.split_once(' ')?;
let n = n.parse::<u64>().ok()?;
parts.push(match label {
"uploaded" => ngettext_f("{n} uploaded", "{n} uploaded", n, &[]),
// Translators: photos an import skipped because they were already
// in Proton Drive.
"already there" => ngettext_f("{n} already there", "{n} already there", n, &[]),
"albums" => ngettext_f("{n} album", "{n} albums", n, &[]),
"failed" => ngettext_f("{n} failed", "{n} failed", n, &[]),
_ => return None,
});
}
Some(parts.join(" · "))
}
/// The number in front of `text` when the rest is one of `units`, as in
/// "3 item(s)".
fn leading_count(text: &str, units: &[&str]) -> Option<u64> {
let (n, unit) = text.split_once(' ')?;
units.contains(&unit).then_some(())?;
n.parse().ok()
}
/// The name to show for a target: the last part of a path, or `None` when
/// the daemon logged a node id because it had no path.
fn target_name(target: &str) -> Option<String> {
let looks_like_uid =
target.len() >= 40 && target.matches('~').count() == 1 && !target.contains(['/', ' ', '.']);
if target.is_empty() || looks_like_uid {
return None;
}
Some(file_name(target).to_string())
}
/// The folder a destination path names, with the root read as "My files".
fn folder_name(dest: &str) -> String {
let name = file_name(dest.trim_end_matches('/'));
if name.is_empty() {
gettext("My files")
} else {
name.to_string()
}
}
/// The file a `name (sync-conflict <stamp>).ext` copy was made from.
pub(crate) fn conflict_base(name: &str) -> Option<String> {
let (start, close) = conflict_marker(name)?;
Some(format!("{}{}", &name[..start], &name[close + 1..]))
}
/// When a conflicting copy was made, from the stamp in its name.
fn conflict_stamp(name: &str) -> Option<i64> {
let (start, close) = conflict_marker(name)?;
let inner = &name[start + MARKER.len()..close];
inner.split('-').next()?.parse().ok()
}
const MARKER: &str = " (sync-conflict ";
/// Where the ` (sync-conflict <stamp>[-<n>])` marker starts and where its
/// closing parenthesis is. The same shape the daemon reads back.
fn conflict_marker(name: &str) -> Option<(usize, usize)> {
let start = name.find(MARKER)?;
let after = start + MARKER.len();
let close = after + name[after..].find(')')?;
let inner = &name[after..close];
let mut parts = inner.splitn(2, '-');
let digits = |s: &str| !s.is_empty() && s.bytes().all(|b| b.is_ascii_digit());
if !digits(parts.next()?) || parts.next().is_some_and(|s| !digits(s)) {
return None;
}
Some((start, close))
}
#[cfg(test)]
mod tests {
use super::*;
fn entry(kind: ActivityKind, target: &str, detail: &str, ok: bool) -> ActivityEntry {
ActivityEntry {
time: 0,
kind,
target: target.into(),
detail: detail.into(),
ok,
}
}
#[test]
fn details_that_repeat_the_title_are_dropped() {
let d = describe(&entry(ActivityKind::Upload, "a.txt", "uploaded", true));
assert_eq!(d.title, "Uploaded a.txt");
assert!(d.details.is_empty());
}
#[test]
fn paths_and_counts_read_as_names_and_numbers() {
let d = describe(&entry(
ActivityKind::Move,
"/mnt/a.txt",
"to /mnt/Docs",
true,
));
assert_eq!(d.title, "Moved a.txt");
assert_eq!(d.details, vec!["To Docs"]);
let d = describe(&entry(ActivityKind::Restore, "3 item(s)", "", true));
assert_eq!(d.title, "Restored 3 items");
let d = describe(&entry(ActivityKind::EmptyTrash, "1 item", "", true));
assert_eq!(d.title, "Emptied the Trash (1 item)");
}
#[test]
fn failures_say_what_could_not_happen() {
let d = describe(&entry(
ActivityKind::Upload,
"a.txt",
"quota exceeded",
false,
));
assert_eq!(d.title, "Couldn't upload a.txt");
assert_eq!(d.details, vec!["Quota exceeded"]);
}
#[test]
fn sync_passes_are_summed_up() {
let d = describe(&entry(
ActivityKind::Sync,
"Docs",
"2 uploaded, 1 folder(s) created",
true,
));
assert_eq!(d.title, "Synced Docs");
assert_eq!(d.details, vec!["2 uploaded, 1 folder created"]);
let d = describe(&entry(
ActivityKind::Sync,
"sync incomplete for Docs",
"2 item(s) failed; will retry",
false,
));
assert_eq!(d.title, "Some items in Docs didn't sync");
assert_eq!(d.details, vec!["2 items failed and will be tried again"]);
}
#[test]
fn a_conflicting_copy_names_its_original() {
let d = describe(&entry(
ActivityKind::Conflict,
"notes (sync-conflict 1700000000).txt",
"differs from notes.txt",
false,
));
assert_eq!(d.title, "Conflicting copy of notes.txt");
assert_eq!(d.details[0], "Differs from notes.txt");
assert_eq!(d.details.len(), 2);
assert_eq!(conflict_base("a (sync-conflict 12-2)"), Some("a".into()));
assert_eq!(conflict_base("a (sync-conflict x).txt"), None);
}
#[test]
fn node_ids_are_not_shown_as_names() {
let uid = format!("{}~{}", "A".repeat(30), "B".repeat(30));
let d = describe(&entry(
ActivityKind::Share,
&uid,
"1 recipient(s) as viewer",
true,
));
assert_eq!(d.title, "Shared an item");
assert_eq!(d.details, vec!["1 person can view"]);
}
#[test]
fn an_edit_kept_as_a_copy_says_why_in_words() {
let d = describe(&entry(
ActivityKind::Upload,
"notes.txt",
"the remote revision changed under the queued write \
(based on revision r1, remote now at r2); \
local changes uploaded as notes (sync-conflict 1700000000).txt",
false,
));
assert_eq!(
d.details,
vec![
"It was changed on Proton Drive, so your changes were uploaded as \
notes (sync-conflict 1700000000).txt"
]
);
let d = describe(&entry(
ActivityKind::Upload,
"notes.txt",
"the file was trashed remotely; local changes kept at /var/staging/7",
false,
));
assert_eq!(
d.details,
vec![
"It was moved to the Trash on Proton Drive, so your changes were kept in \
/var/staging/7"
]
);
}
#[test]
fn unknown_details_show_as_written() {
let d = describe(&entry(
ActivityKind::Download,
"a.txt",
"network timeout",
false,
));
assert_eq!(d.details, vec!["Network timeout"]);
}
}
+35 -17
View File
@@ -24,6 +24,7 @@ pub(crate) fn show_albums(ui: &Rc<Ui>) {
/// position. The timeline's filters do not apply to albums, so they go.
fn show_album_grid(ui: &Rc<Ui>) {
close_album(ui);
close_place(ui);
ui.gallery.filters.set_visible(false);
ui.gallery.content.set_visible_child_name("albums");
}
@@ -82,7 +83,7 @@ pub(crate) fn load_albums(ui: &Rc<Ui>) {
&ui,
"network-offline-symbolic",
&gettext("Not connected"),
&gettext("The Proton Drive mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
),
}
});
@@ -285,7 +286,7 @@ pub(crate) fn prompt_new_album(ui: &Rc<Ui>, photos: Vec<String>) {
_ => toast_error(
&ui,
&gettext("Couldn't create the album"),
&gettext("The mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
),
}
});
@@ -350,7 +351,11 @@ fn run_album_request(ui: &Rc<Ui>, request: Request, done: String, failed: String
Ok(Ok(Response::Error { message, kind })) => {
toast_failure(&ui, &failed, &message, kind)
}
_ => toast_error(&ui, &failed, &gettext("The mount service didn't respond.")),
_ => toast_error(
&ui,
&failed,
&gettext("The Proton Drive service didn't respond."),
),
}
});
}
@@ -379,7 +384,7 @@ pub(crate) fn prompt_add_to_album(ui: &Rc<Ui>, photos: Vec<String>) {
toast_error(
&ui,
&gettext("Couldn't load albums"),
&gettext("The mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
);
return;
}
@@ -468,6 +473,16 @@ fn add_to_album(ui: &Rc<Ui>, album: String, photos: Vec<String>) {
),
);
}
// Filed photos no longer belong in a timeline of unfiled ones.
// The daemon's filter drops them too, so the paging offset stays
// in step with what is left.
if ui.gallery.not_in_album.get() && ui.gallery.album.borrow().is_none() {
ui.gallery
.selected
.borrow_mut()
.retain(|uid| !changed.contains(uid));
remove_photos(&ui, &changed);
}
reload_album_grid(&ui);
}
Ok(Ok(Response::Error { message, kind })) => {
@@ -476,7 +491,7 @@ fn add_to_album(ui: &Rc<Ui>, album: String, photos: Vec<String>) {
_ => toast_error(
&ui,
&gettext("Couldn't add to the album"),
&gettext("The mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
),
}
});
@@ -548,7 +563,7 @@ pub(crate) fn remove_from_album(ui: &Rc<Ui>, album: &AlbumInfo, photos: Vec<Stri
toast_error(
&ui,
&gettext("Couldn't remove from the album"),
&gettext("The mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
)
}
}
@@ -572,7 +587,7 @@ fn album_subtitle(album: &AlbumInfo) -> String {
/// Queue an album cover for the gallery's own thumbnail machinery: same batching,
/// same texture cache, same decode pacing as a timeline tile.
fn want_cover(ui: &Rc<Ui>, uid: String, picture: &gtk4::Picture) {
pub(crate) fn want_cover(ui: &Rc<Ui>, uid: String, picture: &gtk4::Picture) {
if let Some(texture) = ui.gallery.photo_tex.borrow().get(&uid) {
picture.set_paintable(Some(texture));
return;
@@ -603,7 +618,7 @@ pub(crate) fn open_album(ui: &Rc<Ui>, album: AlbumInfo) {
// would silently do something else. The Photos/Albums switcher goes too: an
// album is a level below it, and back is the way out.
ui.gallery.filters.set_visible(false);
ui.gallery.view_switch.set_visible(false);
ui.gallery.view_switch.widget().set_visible(false);
ui.gallery.upload.set_visible(false);
ui.gallery.back.set_visible(true);
// A stale kind/date filter would otherwise be sent with the album request.
@@ -621,7 +636,7 @@ pub(crate) fn close_album(ui: &Rc<Ui>) {
return;
}
ui.gallery.title.set_title(&gettext("Photos"));
ui.gallery.view_switch.set_visible(true);
ui.gallery.view_switch.widget().set_visible(true);
ui.gallery.upload.set_visible(true);
ui.gallery.back.set_visible(false);
}
@@ -638,20 +653,19 @@ fn albums_status(ui: &Rc<Ui>, icon: &str, title: &str, description: &str) {
/// whole content area; back leaves an open album for the grid it came from.
pub(crate) fn wire_albums(ui: &Rc<Ui>) {
let ui_albums = ui.clone();
ui.gallery.albums_btn.clone().connect_toggled(move |btn| {
if btn.is_active() {
ui.gallery.view_switch.connect_changed(move |view| {
if view == VIEW_ALBUMS {
show_albums(&ui_albums);
}
});
// Only the button being switched *on* acts: the group fires `toggled` for
// the one going off too, and acting on both would load twice.
let ui_photos = ui.clone();
ui.gallery.photos_btn.clone().connect_toggled(move |btn| {
if !btn.is_active() {
ui.gallery.view_switch.connect_changed(move |view| {
if view != VIEW_TIMELINE {
return;
}
close_album(&ui_photos);
close_place(&ui_photos);
ui_photos.gallery.filters.set_visible(true);
ui_photos.gallery.content.set_visible_child_name("timeline");
// The timeline comes back as it was left unless an album took over the
@@ -664,10 +678,14 @@ pub(crate) fn wire_albums(ui: &Rc<Ui>) {
}
});
// Back returns to the grid the album was opened from, scrolled where it
// was, instead of reloading it.
// Back returns to the grid the album or place was opened from, scrolled
// where it was, instead of reloading it.
let ui_back = ui.clone();
ui.gallery.back.clone().connect_clicked(move |_| {
if ui_back.gallery.place.borrow().is_some() {
show_place_grid(&ui_back);
return;
}
show_album_grid(&ui_back);
let count = ui_back.gallery.album_list.borrow().len();
ui_back.gallery.title.set_subtitle(&ngettext_f(
File diff suppressed because it is too large. Load diff
+506 -276
View File
@@ -1,85 +1,115 @@
use crate::*;
pub(crate) struct DevicesState {
// Devices page: two sections (synced folders + other devices), rebuilt
// wholesale on each load.
// Computers page: one list of the account's computers, this one first, and
// a read-only browser over another computer's backup.
/// "list", "browse", or "status" while the page loads or can't be read.
pub(crate) content: gtk4::Stack,
pub(crate) status: adw::StatusPage,
pub(crate) retry: gtk4::Button,
pub(crate) group: adw::PreferencesGroup,
/// Rows added last time, removed before the next paint (adw groups have no
/// clear-all).
pub(crate) rows: RefCell<Vec<gtk4::Widget>>,
pub(crate) sync_group: adw::PreferencesGroup,
/// The single "This computer" summary row. The per-folder rows moved to the
/// Locations page, which owns every local path.
pub(crate) sync_rows: RefCell<Vec<gtk4::Widget>>,
/// "Rename" action in the "This computer" header. Insensitive until the
/// device list identifies this machine's own device.
pub(crate) rename_this: gtk4::Button,
/// This machine's own device `(uid, name)`, from the last device list. The
/// current device is filtered out of the "Other computers" rows, so this is
/// where its rename target lives. `None` until identified.
pub(crate) this_device: RefCell<Option<(String, String)>>,
pub(crate) inflight: Cell<bool>,
pub(crate) loaded_at: Cell<Option<Instant>>,
/// The browse view's files.
pub(crate) files: FileList,
/// The browse view: "list", or "empty".
pub(crate) files_content: gtk4::Stack,
/// The breadcrumb trail from "Computers" down to the current folder.
pub(crate) crumb: gtk4::Box,
/// Grid or list for the browse view; hidden on the device list.
pub(crate) layout: gtk4::Button,
/// Where the browse view is, as `(uid, name)`: the computer first, then each
/// folder below it. Empty = the device list. A backup has no path in the
/// mount, so descending is uid-addressed and the stack is the breadcrumb.
pub(crate) nav: RefCell<Vec<(String, String)>>,
/// The [`DevicesState::nav`] the page on screen was painted for, or `None`
/// before the first paint.
pub(crate) listed: RefCell<Option<Vec<(String, String)>>>,
/// Runs the loads; the rows stay up while one runs.
pub(crate) loader: Rc<Loader>,
}
impl DevicesState {
pub(crate) fn new(widgets: &DevicesWidgets) -> Self {
DevicesState {
content: widgets.content.clone(),
status: widgets.status.clone(),
retry: widgets.retry.clone(),
group: widgets.group.clone(),
rows: RefCell::new(Vec::new()),
files: widgets.files.clone(),
files_content: widgets.files_content.clone(),
crumb: widgets.crumb.clone(),
layout: widgets.layout.clone(),
nav: RefCell::new(Vec::new()),
listed: RefCell::new(None),
loader: Loader::new(&widgets.content),
}
}
}
/// Widgets the Devices page's load/repaint touch.
pub(crate) struct DevicesWidgets {
pub(crate) content: gtk4::Stack,
pub(crate) status: adw::StatusPage,
/// "This computer" — this machine's device identity and a pointer to its
/// folders on the Locations page.
pub(crate) sync_group: adw::PreferencesGroup,
/// "Rename" in the "This computer" header — renames this machine's device.
pub(crate) rename_this: gtk4::Button,
/// "Other computers" — the account's *other* registered devices. This
/// machine's own device is deliberately not among them; see
/// [`repaint_devices`].
/// Every computer on the account, this one first.
pub(crate) group: adw::PreferencesGroup,
pub(crate) files: FileList,
pub(crate) files_content: gtk4::Stack,
/// Goes up one folder, and from a computer's top level back to the list.
pub(crate) back: gtk4::Button,
pub(crate) crumb: gtk4::Box,
pub(crate) layout: gtk4::Button,
pub(crate) retry: gtk4::Button,
pub(crate) refresh: gtk4::Button,
}
/// The Computers page: a "This computer" section naming the device this machine
/// backs up to (renamable from its header), plus an "Other computers" section
/// listing the account's other registered devices. The synced folders
/// themselves live on the Sync page.
/// The Computers page: every computer backing up to the account, with this one
/// first. This computer's row leads to its folders on the Sync page; another
/// computer's row opens its backup read-only, in the same list as My files.
pub(crate) fn build_devices_page() -> (gtk4::Widget, DevicesWidgets) {
let refresh = refresh_button();
// This page is about device *identity* — which computer this is, which other
// computers back up to the account, and how to adopt one. The folders
// themselves are local paths, so they live on the Sync page; keeping a
// second copy of that list here would be two places to change the same mode.
let sync_group = adw::PreferencesGroup::builder()
.title(gettext("This computer"))
.description(gettext("The computer this machine backs up as."))
.build();
// Rename control for *this* machine's device, in the section header. The
// current device is filtered out of "Other computers", so this is the only
// place it can be renamed; insensitive until the device list identifies it.
let rename_this = gtk4::Button::builder()
.icon_name("document-edit-symbolic")
.tooltip_text(gettext("Rename this computer"))
.valign(gtk4::Align::Center)
.sensitive(false)
.build();
rename_this.add_css_class("flat");
sync_group.set_header_suffix(Some(&rename_this));
let group = adw::PreferencesGroup::builder()
.title(gettext("Other computers"))
.description(gettext("Other computers backing up to this account."))
.build();
let groups = gtk4::Box::new(gtk4::Orientation::Vertical, 18);
groups.append(&sync_group);
groups.append(&group);
let clamp = adw::Clamp::builder().child(&groups).build();
let group = adw::PreferencesGroup::new();
let clamp = adw::Clamp::builder().child(&group).build();
let scroll = gtk4::ScrolledWindow::builder()
.vexpand(true)
.child(&clamp)
.build();
// Browse: a path bar over the list, as on Shared with me.
let files = FileList::new();
files.show_list(true);
let layout = layout_button(&files);
layout.set_visible(false);
let back = gtk4::Button::builder()
.icon_name("go-previous-symbolic")
.tooltip_text(gettext("Back"))
.build();
back.add_css_class("flat");
let crumb = gtk4::Box::new(gtk4::Orientation::Horizontal, 2);
crumb.set_valign(gtk4::Align::Center);
let crumb_scroll = gtk4::ScrolledWindow::builder()
.hscrollbar_policy(gtk4::PolicyType::External)
.vscrollbar_policy(gtk4::PolicyType::Never)
.hexpand(true)
.child(&crumb)
.build();
let path_bar = gtk4::Box::new(gtk4::Orientation::Horizontal, 8);
path_bar.append(&back);
path_bar.append(&crumb_scroll);
let files_empty = adw::StatusPage::builder()
.icon_name("folder-open-symbolic")
.title(gettext("This folder is empty"))
.description(gettext("Nothing in this folder has been backed up."))
.vexpand(true)
.build();
files_empty.add_css_class("compact");
let files_content = gtk4::Stack::new();
files_content.add_named(&files.views, Some("list"));
files_content.add_named(&files_empty, Some("empty"));
let browse = gtk4::Box::new(gtk4::Orientation::Vertical, 12);
browse.append(&path_bar);
browse.append(&files_content);
let retry = gtk4::Button::builder()
.label(gettext("Retry"))
.halign(gtk4::Align::Center)
@@ -98,6 +128,7 @@ pub(crate) fn build_devices_page() -> (gtk4::Widget, DevicesWidgets) {
content.set_vexpand(true);
content.set_transition_type(gtk4::StackTransitionType::Crossfade);
content.add_named(&scroll, Some("list"));
content.add_named(&browse, Some("browse"));
content.add_named(&status, Some("status"));
let inner = gtk4::Box::new(gtk4::Orientation::Vertical, 12);
@@ -107,35 +138,66 @@ pub(crate) fn build_devices_page() -> (gtk4::Widget, DevicesWidgets) {
inner.set_margin_end(18);
inner.append(&content);
let (frame, header, _) = page_frame(&gettext("Computers"), &inner);
header.pack_end(&refresh);
header.pack_end(&layout);
(
frame.upcast(),
DevicesWidgets {
content,
status,
sync_group,
rename_this,
group,
files,
files_content,
back,
crumb,
layout,
retry,
refresh,
},
)
}
/// Install the Devices page's retry button and the Rename action.
pub(crate) fn wire_devices(ui: &Rc<Ui>, retry: &gtk4::Button) {
/// Install the browse view's file list and the page's buttons.
pub(crate) fn wire_devices(ui: &Rc<Ui>, widgets: &DevicesWidgets) {
let files = &ui.devices.files;
files.wire(
ui,
FileListBehavior {
activate: open_device_entry,
entry_menu: device_entry_menu,
bulk_menu: device_bulk_menu,
background_menu: device_background_menu,
badges: false,
drag_and_drop: false,
},
);
files
.column_view
.append_column(&text_column(&pgettext("column", "Modified"), |e| {
if e.modified > 0 {
dates::short_date(e.modified)
} else {
"—".to_string()
}
}));
files
.column_view
.append_column(&text_column(&pgettext("column", "Size"), |e| {
if e.is_dir {
"—".to_string()
} else {
human_bytes(e.size)
}
}));
let ui_back = ui.clone();
widgets.back.connect_clicked(move |_| {
ui_back.devices.nav.borrow_mut().pop();
load_devices(&ui_back);
});
let ui_retry = ui.clone();
retry.connect_clicked(move |_| {
service::restart();
load_devices(&ui_retry);
});
let ui_ren = ui.clone();
ui.devices.rename_this.connect_clicked(move |_| {
if let Some((uid, name)) = ui_ren.devices.this_device.borrow().clone() {
prompt_rename_device(&ui_ren, &uid, &name);
}
});
widgets
.retry
.connect_clicked(move |_| restart_service_then(&ui_retry, load_devices));
}
/// Show a status page in place of the Devices list.
@@ -147,21 +209,17 @@ pub(crate) fn devices_status(ui: &Rc<Ui>, icon: &str, title: &str, description:
ui.devices.content.set_visible_child_name("status");
}
/// Fetch this machine's synced folders and the account's other devices, then
/// repaint both sections. The two requests are chained so a single unreachable
/// daemon collapses the whole page to a status view.
/// Load the view [`DevicesState::nav`] points at: the device list, or a folder
/// of another computer's backup.
pub(crate) fn load_devices(ui: &Rc<Ui>) {
if ui.devices.inflight.get() {
cancel_file_thumbnails(ui);
let nav = ui.devices.nav.borrow().clone();
ui.devices.layout.set_visible(!nav.is_empty());
if !nav.is_empty() {
load_device_folder(ui, &nav);
return;
}
ui.devices.inflight.set(true);
devices_status(
ui,
"computer-symbolic",
&gettext("Loading…"),
&gettext("Reading your computers."),
false,
);
let ticket = begin_devices_load(ui, "computer-symbolic", gettext("Reading your computers."));
ui.busy_begin();
// The two lists are independent and the daemon serves requests concurrently,
// so fire both up front and collect them, rather than paying two round trips
@@ -172,213 +230,361 @@ pub(crate) fn load_devices(ui: &Rc<Ui>) {
glib::spawn_future_local(async move {
let sync = rx.recv().await;
let devices_reply = rx2.recv().await;
ui.busy_end();
if !ticket.is_current() {
return;
}
let folders = match sync {
Ok(Ok(Response::SyncFolders { items })) => items,
// The daemon answered but not as expected — treat as empty and carry
// on to the devices list rather than failing the whole page.
Ok(Ok(_)) => Vec::new(),
Ok(Err(_)) | Err(_) => {
ui.busy_end();
ui.devices.inflight.set(false);
ui.devices.loaded_at.set(None);
devices_unreachable(&ui);
return;
}
};
// Daemon reachable: show the list and paint the "This computer" section.
ui.devices.content.set_visible_child_name("list");
repaint_this_computer(&ui, &folders);
let devices = match devices_reply {
Ok(Ok(Response::Devices { items })) => items,
_ => Vec::new(),
};
ui.busy_end();
ui.devices.inflight.set(false);
repaint_devices(&ui, &devices);
ui.devices.loaded_at.set(Some(Instant::now()));
ui.devices.content.set_visible_child_name("list");
repaint_devices(&ui, &devices, &folders);
repaint_device_crumb(&ui, &[]);
*ui.devices.listed.borrow_mut() = Some(Vec::new());
});
}
/// List one folder of another computer's backup. The computer's own top level
/// is its restorable folders; below that, any folder lists by uid.
fn load_device_folder(ui: &Rc<Ui>, nav: &[(String, String)]) {
let ticket = begin_devices_load(ui, "folder-symbolic", gettext("Reading this folder."));
ui.busy_begin();
let (uid, _) = nav.last().cloned().unwrap_or_default();
let request = if nav.len() == 1 {
Request::ListDeviceRestorableFolders { device: uid }
} else {
Request::ListSharedFolder { uid }
};
let rx = spawn_request(ui.dirs.control_socket(), request);
let ui = ui.clone();
let nav = nav.to_vec();
glib::spawn_future_local(async move {
let result = rx.recv().await;
ui.busy_end();
if !ticket.is_current() {
return;
}
let entries = match result {
Ok(Ok(Response::Entries { entries })) => entries,
Ok(Ok(Response::RestorableFolders { items })) => {
items.into_iter().map(restorable_entry).collect()
}
Ok(Ok(Response::Error { message, kind })) => {
// The folder or the computer is gone: fall back a level rather
// than stranding the page on a dead uid.
ui.devices.nav.borrow_mut().pop();
toast_failure(&ui, &gettext("Couldn't open folder"), &message, kind);
load_devices(&ui);
return;
}
_ => {
devices_unreachable(&ui);
return;
}
};
ui.devices.content.set_visible_child_name("browse");
repaint_device_crumb(&ui, &nav);
*ui.devices.listed.borrow_mut() = Some(nav);
if entries.is_empty() {
ui.devices.files.model.remove_all();
ui.devices.files_content.set_visible_child_name("empty");
} else {
ui.devices.files_content.set_visible_child_name("list");
replace_items(&ui.devices.files.model, &entries);
}
});
}
/// A computer's top-level folder as a browsable entry.
fn restorable_entry(folder: RestorableFolder) -> DirEntry {
DirEntry {
name: folder.name,
is_dir: true,
uid: folder.remote_uid,
..DirEntry::default()
}
}
/// Start a load of the view [`DevicesState::nav`] points at. A reload of the
/// view on screen keeps it up and usable; moving elsewhere greys it out until
/// the new view arrives.
fn begin_devices_load(ui: &Rc<Ui>, icon: &'static str, description: String) -> LoadTicket {
let ui_p = ui.clone();
let placeholder = move || {
devices_status(&ui_p, icon, &gettext("Loading…"), &description, false);
};
if *ui.devices.listed.borrow() == Some(ui.devices.nav.borrow().clone()) {
ui.devices.loader.refresh(placeholder)
} else {
ui.devices.listed.borrow_mut().take();
ui.devices.loader.replace(placeholder)
}
}
/// The daemon didn't answer the Devices page.
pub(crate) fn devices_unreachable(ui: &Rc<Ui>) {
if service::is_failed() || !service::is_active() {
devices_status(
ui,
"network-offline-symbolic",
&gettext("Not connected"),
&gettext("The Proton Drive mount service isn't running."),
true,
);
return;
}
devices_status(
ui,
"folder-remote-symbolic",
&gettext("Connecting…"),
&gettext("Waiting for the Proton Drive mount service to come up."),
false,
);
let ui = ui.clone();
glib::timeout_add_local_once(CONNECT_RETRY_INTERVAL, move || {
if ui.stack.visible_child_name().as_deref() == Some("devices") {
load_devices(&ui);
}
});
service_unreachable(ui, "devices", devices_status, load_devices);
}
/// Rebuild the "Other computers" section from a fresh listing. Empty is a normal
/// state (this machine may be the only device), so it shows a placeholder row
/// rather than collapsing the page.
/// Rebuild the device list: this computer first, then the others.
///
/// This machine's own device is filtered out. Deleting a device deletes its root
/// folder — every file backed up from it — and for *this* machine that would also
/// pull the ground out from under the synced folders listed directly above, so it
/// is not offered as a peer of "remove some other laptop". Removing this
/// computer's backup means removing its folders, each of which asks about its
/// cloud copy on its own terms.
pub(crate) fn repaint_devices(ui: &Rc<Ui>, devices: &[DeviceInfo]) {
/// This computer's row offers no Remove. Deleting a device deletes its root
/// folder — every file backed up from it — and for this machine that would also
/// pull the ground out from under its synced folders, so it is not offered as a
/// peer of "remove some other laptop". Removing this computer's backup means
/// removing its folders, each of which asks about its cloud copy on its own
/// terms.
pub(crate) fn repaint_devices(ui: &Rc<Ui>, devices: &[DeviceInfo], folders: &[SyncFolderInfo]) {
for row in ui.devices.rows.borrow_mut().drain(..) {
ui.devices.group.remove(&row);
}
// Identify this machine's own device so the "This computer" header's Rename
// action has a target. It never appears in the rows below, so this is the
// only place its name/uid is captured.
match devices.iter().find(|d| d.this_device) {
Some(me) => {
*ui.devices.this_device.borrow_mut() = Some((me.uid.clone(), me.name.clone()));
ui.devices
.sync_group
// Translators: {name} is the name of this computer's device.
.set_description(Some(&gettext_f(
"Backing up as “{name}”.",
&[("name", &me.name)],
)));
ui.devices.rename_this.set_sensitive(true);
ui.devices
.rename_this
// Translators: {name} is the name of this computer's device.
.set_tooltip_text(Some(&gettext_f(
"Rename this computer ({name})",
&[("name", &me.name)],
)));
}
None => {
*ui.devices.this_device.borrow_mut() = None;
ui.devices.rename_this.set_sensitive(false);
}
}
let mut rows: Vec<gtk4::Widget> = Vec::new();
let me = devices.iter().find(|d| d.this_device);
rows.push(this_computer_row(ui, me, folders).upcast());
let others: Vec<&DeviceInfo> = devices.iter().filter(|d| !d.this_device).collect();
for dev in &others {
rows.push(other_computer_row(ui, dev).upcast());
}
if others.is_empty() {
let row = adw::ActionRow::builder()
.title(gettext("No other computers"))
.subtitle(gettext("Desktop apps syncing to this account appear here."))
.subtitle(gettext(
"Install Proton Drive on another computer to see it here.",
))
.build();
row.add_prefix(&gtk4::Image::from_icon_name("computer-symbolic"));
ui.devices.group.add(&row);
*ui.devices.rows.borrow_mut() = vec![row.upcast()];
return;
}
let mut rows: Vec<gtk4::Widget> = Vec::new();
for dev in others {
let row = adw::ActionRow::builder()
.title(&dev.name)
.subtitle(device_subtitle(dev))
.build();
row.add_prefix(&gtk4::Image::from_icon_name("computer-symbolic"));
// Adoption is how a reinstalled or renamed machine re-attaches to the
// device it used to be, instead of registering a duplicate. It only
// makes sense on *another* computer's row, which is the only place this
// loop paints.
let (ui_ren, uid_ren, name_ren) = (ui.clone(), dev.uid.clone(), dev.name.clone());
let (ui_ad, uid_ad, name_ad) = (ui.clone(), dev.uid.clone(), dev.name.clone());
let (ui_rm, uid_rm, name_rm) = (ui.clone(), dev.uid.clone(), dev.name.clone());
let (ui_rs, uid_rs, name_rs) = (ui.clone(), dev.uid.clone(), dev.name.clone());
row.add_suffix(&more_menu_button(vec![
(
&gettext("Restore to This Computer…"),
Box::new(move || {
prompt_restore_folders(&ui_rs, Some((uid_rs.clone(), name_rs.clone())))
}),
),
(
&gettext("Rename…"),
Box::new(move || prompt_rename_device(&ui_ren, &uid_ren, &name_ren)),
),
(
&gettext("Use This Computer's Identity…"),
Box::new(move || prompt_adopt_device(&ui_ad, &uid_ad, &name_ad)),
),
(
&gettext("Remove Computer…"),
Box::new(move || prompt_remove_device(&ui_rm, &uid_rm, &name_rm)),
),
]));
ui.devices.group.add(&row);
let get = gtk4::LinkButton::with_label(PROTON_DRIVE_DOWNLOAD, &gettext("Get Proton Drive"));
get.set_valign(gtk4::Align::Center);
row.add_suffix(&get);
rows.push(row.upcast());
}
for row in &rows {
ui.devices.group.add(row);
}
*ui.devices.rows.borrow_mut() = rows;
}
/// A device row's subtitle: its platform, and when it last synced. A device that
/// has never synced says so — an unexplained missing date reads as a bug, and
/// "never" is the fact that tells the user this computer isn't backing anything up.
pub(crate) fn device_subtitle(dev: &DeviceInfo) -> String {
match dev.last_sync {
// Translators: {platform} is the device's platform (such as "Linux"), {time} a date and time.
Some(secs) if secs > 0 => gettext_f(
"{platform} · last synced {time}",
&[
("platform", &dev.device_type),
("time", &activity_time(secs)),
],
),
// Translators: {platform} is the device's platform, such as "Linux".
_ => gettext_f(
"{platform} · never synced",
&[("platform", &dev.device_type)],
),
/// Where the Proton Drive apps for other systems are.
const PROTON_DRIVE_DOWNLOAD: &str = "https://proton.me/drive/download";
/// This computer's row: its device name with a "This computer" pill, and how
/// many folders it backs up. It leads to those folders on the Sync page, which
/// owns every local path. `me` is `None` until the daemon has registered this
/// machine as a device.
fn this_computer_row(
ui: &Rc<Ui>,
me: Option<&DeviceInfo>,
folders: &[SyncFolderInfo],
) -> adw::ActionRow {
let row = adw::ActionRow::builder()
.title(me.map_or_else(|| gettext("This computer"), |d| d.name.clone()))
.subtitle(this_computer_subtitle(folders))
.activatable(true)
.build();
row.add_prefix(&gtk4::Image::from_icon_name("computer-symbolic"));
if me.is_some() {
let pill = gtk4::Label::new(Some(&gettext("This computer")));
pill.add_css_class("role-pill");
pill.add_css_class("caption");
pill.set_valign(gtk4::Align::Center);
row.add_suffix(&pill);
}
let mut menu = ActionMenu::new();
if let Some(me) = me {
let (ui_c, uid, name) = (ui.clone(), me.uid.clone(), me.name.clone());
menu.item(&gettext("Rename…"), move || {
prompt_rename_device(&ui_c, &uid, &name)
});
}
let ui_c = ui.clone();
menu.item(&gettext("Restore Folders…"), move || {
prompt_restore_folders(&ui_c, None)
});
row.add_suffix(&menu.button());
row.add_suffix(&gtk4::Image::from_icon_name("go-next-symbolic"));
let ui_c = ui.clone();
row.connect_activated(move |_| {
ui_c.locations.views.set_visible_child_name("folders");
ui_c.stack.set_visible_child_name("locations");
});
row
}
/// Another computer's row. Opening it browses that computer's backup.
fn other_computer_row(ui: &Rc<Ui>, dev: &DeviceInfo) -> adw::ActionRow {
let row = adw::ActionRow::builder()
.title(&dev.name)
.subtitle(device_subtitle(dev))
.activatable(true)
.build();
row.add_prefix(&gtk4::Image::from_icon_name("computer-symbolic"));
let (uid, name) = (dev.uid.clone(), dev.name.clone());
let mut menu = ActionMenu::new();
let (ui_c, uid_c, name_c) = (ui.clone(), uid.clone(), name.clone());
menu.item(&gettext("Browse Files"), move || {
browse_device(&ui_c, &uid_c, &name_c)
});
let (ui_c, uid_c, name_c) = (ui.clone(), uid.clone(), name.clone());
menu.item(&gettext("Restore to This Computer…"), move || {
prompt_restore_folders(&ui_c, Some((uid_c.clone(), name_c.clone())))
});
menu.section();
let (ui_c, uid_c, name_c) = (ui.clone(), uid.clone(), name.clone());
menu.item(&gettext("Rename…"), move || {
prompt_rename_device(&ui_c, &uid_c, &name_c)
});
// Adoption is how a reinstalled or renamed machine re-attaches to the device
// it used to be, instead of registering a duplicate. It only makes sense on
// another computer's row.
let (ui_c, uid_c, name_c) = (ui.clone(), uid.clone(), name.clone());
menu.item(&gettext("Continue This Backup Here…"), move || {
prompt_adopt_device(&ui_c, &uid_c, &name_c)
});
menu.section();
let (ui_c, uid_c, name_c) = (ui.clone(), uid.clone(), name.clone());
menu.item(&gettext("Remove…"), move || {
prompt_remove_device(&ui_c, &uid_c, &name_c)
});
row.add_suffix(&menu.button());
row.add_suffix(&gtk4::Image::from_icon_name("go-next-symbolic"));
let ui_c = ui.clone();
row.connect_activated(move |_| browse_device(&ui_c, &uid, &name));
row
}
/// Open another computer's backup at its top level.
fn browse_device(ui: &Rc<Ui>, uid: &str, name: &str) {
*ui.devices.nav.borrow_mut() = vec![(uid.to_string(), name.to_string())];
load_devices(ui);
}
/// Rebuild the breadcrumb trail for `nav`: "Computers", then the computer and
/// each folder down to the current one, which is a plain heading.
fn repaint_device_crumb(ui: &Rc<Ui>, nav: &[(String, String)]) {
let crumb = &ui.devices.crumb;
while let Some(child) = crumb.first_child() {
crumb.remove(&child);
}
crumb.append(&device_crumb_node(
ui,
&gettext("Computers"),
0,
nav.is_empty(),
));
for (i, (_, name)) in nav.iter().enumerate() {
let sep = gtk4::Label::new(Some("›"));
sep.add_css_class("dim-label");
crumb.append(&sep);
crumb.append(&device_crumb_node(ui, name, i + 1, i + 1 == nav.len()));
}
}
/// Rebuild the "This computer" section: one row naming the device this machine
/// backs up to, and how many folders it carries.
///
/// The per-folder rows (mode switch, sync now, remove) live on the Locations
/// page. They are local paths, and Locations is the one page that owns those;
/// two lists of the same folders would be two places to flip the same switch.
pub(crate) fn repaint_this_computer(ui: &Rc<Ui>, folders: &[SyncFolderInfo]) {
for row in ui.devices.sync_rows.borrow_mut().drain(..) {
ui.devices.sync_group.remove(&row);
/// One breadcrumb segment: a heading for the current folder, or a flat button
/// that goes back up to `depth` levels below the device list.
fn device_crumb_node(ui: &Rc<Ui>, label: &str, depth: usize, current: bool) -> gtk4::Widget {
if current {
let l = gtk4::Label::builder()
.label(label)
.ellipsize(gtk4::pango::EllipsizeMode::Start)
.build();
l.add_css_class("heading");
return l.upcast();
}
let button = gtk4::Button::builder().label(label).build();
button.add_css_class("flat");
let ui = ui.clone();
button.connect_clicked(move |_| {
ui.devices.nav.borrow_mut().truncate(depth);
load_devices(&ui);
});
button.upcast()
}
/// Folders open in place; files download and open with the default app.
fn open_device_entry(ui: &Rc<Ui>, entry: &DirEntry) {
if entry.is_dir {
ui.devices
.nav
.borrow_mut()
.push((entry.uid.clone(), entry.name.clone()));
load_devices(ui);
} else {
open_shared_file(ui, &entry.uid, &entry.name);
}
}
fn device_entry_menu(ui: &Rc<Ui>, entry: &DirEntry) -> ActionMenu {
let mut menu = ActionMenu::new();
let (ui_c, entry_c) = (ui.clone(), entry.clone());
menu.item(&pgettext("verb", "Open"), move || {
open_device_entry(&ui_c, &entry_c)
});
menu
}
fn device_bulk_menu(_ui: &Rc<Ui>, entries: Vec<DirEntry>) -> ActionMenu {
let mut menu = ActionMenu::new();
menu.labelled_section(&ngettext_f(
"{n} selected",
"{n} selected",
entries.len() as u64,
&[],
));
menu
}
fn device_background_menu(ui: &Rc<Ui>) -> ActionMenu {
let mut menu = ActionMenu::new();
let ui_c = ui.clone();
menu.item(&gettext("Select All"), move || {
ui_c.devices.files.selection.select_all();
});
let ui_c = ui.clone();
menu.item(&gettext("Refresh"), move || reload_current_page(&ui_c));
menu
}
/// A device row's subtitle: its platform, and when it last backed up. A device
/// that never did says so — an unexplained missing date reads as a bug, and
/// "no backups yet" is the fact that tells the user this computer isn't backing
/// anything up.
pub(crate) fn device_subtitle(dev: &DeviceInfo) -> String {
let platform = platform_label(&dev.device_type);
match dev.last_sync {
// Translators: {platform} is the device's platform (such as "Linux"), {time} a relative time such as "5 minutes ago".
Some(secs) if secs > 0 => gettext_f(
"{platform} · last backup {time}",
&[("platform", platform), ("time", &dates::relative(secs))],
),
// Translators: {platform} is the device's platform, such as "Linux".
_ => gettext_f("{platform} · no backups yet", &[("platform", platform)]),
}
}
/// The platform as its maker spells it. The API's names are enum variants.
fn platform_label(device_type: &str) -> &str {
match device_type {
"MacOs" => "macOS",
other => other,
}
let row = adw::ActionRow::builder()
.title(gettext("This computer"))
.subtitle(this_computer_subtitle(folders))
.build();
row.add_prefix(&gtk4::Image::from_icon_name("computer-symbolic"));
let manage = gtk4::Button::builder()
.label(gettext("Open Sync"))
.tooltip_text(gettext("Manage this computer's folders and mountpoint"))
.valign(gtk4::Align::Center)
.build();
manage.add_css_class("flat");
let ui_go = ui.clone();
manage.connect_clicked(move |_| ui_go.stack.set_visible_child_name("locations"));
row.add_suffix(&manage);
let ui_rs = ui.clone();
row.add_suffix(&more_menu_button(vec![(
&gettext("Restore Folders…"),
Box::new(move || prompt_restore_folders(&ui_rs, None)),
)]));
ui.devices.sync_group.add(&row);
*ui.devices.sync_rows.borrow_mut() = vec![row.upcast()];
}
/// How many folders this machine backs up, and whether any of them needs
/// attention — the one fact worth surfacing away from the folder list itself.
pub(crate) fn this_computer_subtitle(folders: &[SyncFolderInfo]) -> String {
if folders.is_empty() {
return gettext("No folders backed up yet — add one on the Sync page.");
return gettext("No folders backed up yet");
}
let count = ngettext_f(
"{n} folder backed up",
@@ -391,11 +597,7 @@ pub(crate) fn this_computer_subtitle(folders: &[SyncFolderInfo]) -> String {
.filter(|f| f.state == "error" || f.state == "conflict")
.count();
match attention {
// Translators: {folders} is "N folders backed up".
0 => gettext_f(
"{folders} · manage them on the Sync page",
&[("folders", &count)],
),
0 => count,
// Translators: {folders} is "N folders backed up"; {n} counts the folders with a problem.
n => ngettext_f(
"{folders} · {n} needs attention",
@@ -489,7 +691,7 @@ pub(crate) fn prompt_add_sync_folder(ui: &Rc<Ui>) {
_ => toast_error(
&ui,
&gettext("Couldn't add folder"),
&gettext("The mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
),
}
});
@@ -529,7 +731,7 @@ pub(crate) fn prompt_restore_folders(ui: &Rc<Ui>, device: Option<(String, String
_ => toast_error(
&ui,
&gettext("Couldn't list folders to restore"),
&gettext("The mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
),
}
});
@@ -577,7 +779,7 @@ fn show_restore_picker(
.child(&group)
.build();
let dialog = adw::AlertDialog::builder()
.heading(gettext("Restore folders"))
.heading(gettext("Restore Folders"))
.body(restore_picker_body(
device.as_ref().map(|(_, n)| n.as_str()),
))
@@ -668,7 +870,7 @@ pub(crate) fn set_sync_folder_mode(ui: &Rc<Ui>, id: i64, mode: &'static str) {
Ok(Ok(Response::Ok { .. })) => toast(
&ui,
&if mode == "ondemand" {
gettext("Folder will switch to on-demand")
gettext("Folder will switch to online only")
} else {
gettext("Folder will download and stay synced")
},
@@ -679,7 +881,7 @@ pub(crate) fn set_sync_folder_mode(ui: &Rc<Ui>, id: i64, mode: &'static str) {
_ => toast_error(
&ui,
&gettext("Couldn't change mode"),
&gettext("The mount service didn't respond."),
&gettext("The Proton Drive service didn't respond."),
),
}
reload_sync_pages(&ui);
@@ -710,7 +912,7 @@ pub(crate) fn prompt_remove_sync_folder(ui: &Rc<Ui>, id: i64, path: &str, ondema
let body = if ondemand {
// Translators: {path} is a local folder path.
gettext_f(
"Stop syncing “{path}”?\n\nThis folder is on-demand: its files live in Proton Drive, not on this disk, so the folder will be empty once it is unmounted. To keep a local copy, cancel, turn off On-demand, and wait for the download to finish before removing it.",
"Stop syncing “{path}”?\n\nThis folder is online only: its files live in Proton Drive, not on this disk, so the folder will be empty once it stops syncing. To keep a local copy, cancel, switch the folder to Synced, and wait for the download to finish before removing it.",
&[("path", path)],
)
} else {
@@ -721,7 +923,7 @@ pub(crate) fn prompt_remove_sync_folder(ui: &Rc<Ui>, id: i64, path: &str, ondema
)
};
let dialog = adw::AlertDialog::builder()
.heading(gettext("Stop syncing folder"))
.heading(gettext("Stop Syncing Folder?"))
.body(body)
.build();
let group = adw::PreferencesGroup::new();
@@ -818,7 +1020,7 @@ pub(crate) fn prompt_remove_device(ui: &Rc<Ui>, uid: &str, name: &str) {
let group = adw::PreferencesGroup::new();
group.add(&confirm);
let dialog = adw::AlertDialog::builder()
.heading(gettext("Remove Computer"))
.heading(gettext("Remove Computer?"))
// Translators: {name} is the name of the computer being removed.
.body(gettext_f(
"Remove “{name}” from this account?\n\nEverything it backed up to Proton Drive is deleted along with it. The files on that computer itself are not touched — but this cannot be undone from here.",
@@ -852,7 +1054,8 @@ pub(crate) fn prompt_remove_device(ui: &Rc<Ui>, uid: &str, name: &str) {
dialog.present(win.as_ref());
}
/// Confirm, then adopt another device as this machine's identity.
/// Confirm, then continue another computer's backup on this machine (adopt its
/// device).
///
/// Worth a confirmation rather than a plain click: adoption re-points this
/// machine's syncing at another computer's device folder, and the folders
@@ -862,20 +1065,22 @@ pub(crate) fn prompt_remove_device(ui: &Rc<Ui>, uid: &str, name: &str) {
pub(crate) fn prompt_adopt_device(ui: &Rc<Ui>, uid: &str, name: &str) {
let win = ui_window(ui);
let dialog = adw::AlertDialog::builder()
.heading(gettext("Use this computer's identity"))
// Translators: {name} is the name of the other computer whose identity is adopted.
.heading(gettext("Continue This Backup Here?"))
// Translators: {name} is the name of the other computer whose backup this one takes over.
.body(gettext_f(
"Treat this machine as “{name}”?\n\nNew synced folders are created under that computer in Proton Drive, and this machine keeps that identity even if its hostname changes. Folders already synced here are not moved.\n\nUse “Restore folders” afterwards to bring back what “{name}” was syncing.",
"This computer takes over the backup of “{name}”, for example after a reinstall.\n\nNew synced folders go under “{name}” in Proton Drive, even if this computer's name changes. Folders already synced here are not moved. Use “Restore to This Computer” afterwards to bring back what “{name}” was syncing.",
&[("name", name)],
))
.build();
dialog.add_response("cancel", &gettext("Cancel"));
dialog.add_response("adopt", &gettext("Use identity"));
dialog.add_response("adopt", &gettext("Continue Here"));
dialog.set_response_appearance("adopt", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("cancel"));
dialog.set_close_response("cancel");
let ui = ui.clone();
let uid = uid.to_string();
// Translators: {name} is the name of the computer whose backup this one took over.
let done = gettext_f("Now backing up as “{name}”", &[("name", name)]);
dialog.connect_response(None, move |_, resp| {
if resp == "adopt" {
run_devices_mutation(
@@ -883,8 +1088,8 @@ pub(crate) fn prompt_adopt_device(ui: &Rc<Ui>, uid: &str, name: &str) {
Request::AdoptDevice {
uid: Some(uid.clone()),
},
&gettext("Identity adopted"),
&gettext("Couldn't adopt that computer"),
&done,
&gettext("Couldn't continue that backup here"),
);
}
});
@@ -909,7 +1114,11 @@ pub(crate) fn run_devices_mutation(ui: &Rc<Ui>, req: Request, done: &str, failed
Ok(Ok(Response::Error { message, kind })) => {
toast_failure(&ui, &failed, &message, kind)
}
_ => toast_error(&ui, &failed, &gettext("The mount service didn't respond.")),
_ => toast_error(
&ui,
&failed,
&gettext("The Proton Drive service didn't respond."),
),
}
});
}
@@ -933,20 +1142,17 @@ mod tests {
}
#[test]
fn a_machine_with_no_folders_is_told_where_to_add_one() {
assert_eq!(
this_computer_subtitle(&[]),
"No folders backed up yet — add one on the Sync page."
);
fn a_machine_with_no_folders_says_so() {
assert_eq!(this_computer_subtitle(&[]), "No folders backed up yet");
}
#[test]
fn folders_needing_attention_are_counted_not_hidden() {
// The per-folder rows moved to Locations, so this row is the only place
// the Computers page can surface that something is wrong.
// The per-folder rows live on the Sync page, so this row is the only
// place the Computers page can surface that something is wrong.
assert_eq!(
this_computer_subtitle(&[folder("idle"), folder("idle")]),
"2 folders backed up · manage them on the Sync page"
"2 folders backed up"
);
assert_eq!(
this_computer_subtitle(&[folder("idle"), folder("error")]),
@@ -964,4 +1170,28 @@ mod tests {
assert!(restore_picker_body(None).contains("this computer"));
assert!(nothing_to_restore(Some("laptop")).contains("“laptop”"));
}
fn device(device_type: &str, last_sync: Option<i64>) -> DeviceInfo {
DeviceInfo {
uid: "dev".into(),
name: "laptop".into(),
device_type: device_type.into(),
last_sync,
this_device: false,
adopted: false,
}
}
#[test]
fn a_computer_that_never_backed_up_says_no_backups_yet() {
assert_eq!(
device_subtitle(&device("Linux", None)),
"Linux · no backups yet"
);
assert_eq!(
device_subtitle(&device("Linux", Some(0))),
"Linux · no backups yet"
);
assert!(device_subtitle(&device("MacOs", Some(1))).starts_with("macOS · last backup "));
}
}
+473
View File
@@ -0,0 +1,473 @@
use crate::*;
/// Side of a copy's thumbnail in the duplicates dialog, in px.
const COPY_SIZE: i32 = 150;
/// Each set's copies with their Keep toggles.
type SetToggles = Vec<Vec<(PhotoItem, gtk4::ToggleButton)>>;
/// How often the dialog asks again while the daemon hashes thumbnails.
const HASHING_POLL: std::time::Duration = std::time::Duration::from_secs(1);
/// What the finder looks for.
#[derive(Clone, Copy, PartialEq, Eq)]
enum Finder {
/// Byte-identical copies, going by the server's content hash.
Exact,
/// Shots that look alike, going by their thumbnails.
Similar,
}
/// The dialog's parts that a search fills in.
struct FinderView {
ui: Rc<Ui>,
status: adw::StatusPage,
progress: gtk4::ProgressBar,
note: gtk4::Label,
stack: gtk4::Stack,
trash: gtk4::Button,
toolbar: adw::ToolbarView,
/// Bumped by every new search and when the dialog closes, so a reply or a
/// poll for an older one is dropped.
generation: Cell<u32>,
/// The Trash button's handler for the review on show.
clicked: RefCell<Option<glib::SignalHandlerId>>,
}
/// Open the duplicate finder: every set of photos stored more than once, side
/// by side, with the copy worth keeping ticked. The copies left unticked go to
/// Trash — one file each, never the rest of their shot. The Similar switch
/// looks for shots that look alike instead.
pub(crate) fn show_duplicates(ui: &Rc<Ui>) {
let progress = gtk4::ProgressBar::builder()
.show_text(true)
.width_request(280)
.halign(gtk4::Align::Center)
.visible(false)
.build();
let note = gtk4::Label::builder()
.wrap(true)
.justify(gtk4::Justification::Center)
.visible(false)
.build();
note.add_css_class("dim-label");
let status_extra = gtk4::Box::new(gtk4::Orientation::Vertical, 12);
status_extra.append(&progress);
status_extra.append(&note);
let status = adw::StatusPage::builder()
.icon_name("edit-copy-symbolic")
.vexpand(true)
.child(&status_extra)
.build();
status.add_css_class("compact");
let stack = gtk4::Stack::new();
stack.add_named(&status, Some("status"));
let trash = gtk4::Button::builder()
.halign(gtk4::Align::End)
.sensitive(false)
.build();
trash.add_css_class("destructive-action");
trash.add_css_class("pill");
let bottom = gtk4::Box::new(gtk4::Orientation::Horizontal, 0);
bottom.add_css_class("toolbar");
bottom.set_halign(gtk4::Align::End);
bottom.append(&trash);
// Exact first, then Similar: the index is the finder.
let switch = ToggleGroup::new(&[
(
&pgettext("duplicates", "Exact"),
Some(&gettext("Photos stored more than once")),
),
(
&pgettext("duplicates", "Similar"),
Some(&gettext("Photos that look alike")),
),
]);
let header = adw::HeaderBar::new();
header.set_title_widget(Some(&switch.widget()));
let toolbar = adw::ToolbarView::new();
toolbar.add_top_bar(&header);
toolbar.set_content(Some(&stack));
toolbar.add_bottom_bar(&bottom);
toolbar.set_reveal_bottom_bars(false);
let dialog = adw::Dialog::builder()
.title(gettext("Duplicates"))
.content_width(760)
.content_height(640)
.child(&toolbar)
.build();
dialog.present(ui_window(ui).as_ref());
let view = Rc::new(FinderView {
ui: ui.clone(),
status,
progress,
note,
stack,
trash,
toolbar,
generation: Cell::new(0),
clicked: RefCell::new(None),
});
let view_closed = view.clone();
dialog.connect_closed(move |_| {
view_closed
.generation
.set(view_closed.generation.get().wrapping_add(1));
});
let view_switch = view.clone();
switch.connect_changed(move |index| {
let finder = if index == 1 {
Finder::Similar
} else {
Finder::Exact
};
search(&view_switch, finder);
});
search(&view, Finder::Exact);
}
/// Drop what the dialog shows and look again, for `finder`.
fn search(view: &Rc<FinderView>, finder: Finder) {
let generation = view.generation.get().wrapping_add(1);
view.generation.set(generation);
if let Some(review) = view.stack.child_by_name("review") {
view.stack.remove(&review);
}
if let Some(handler) = view.clicked.take() {
view.trash.disconnect(handler);
}
view.toolbar.set_reveal_bottom_bars(false);
view.status.set_icon_name(Some("edit-copy-symbolic"));
view.status.set_title(&match finder {
Finder::Exact => gettext("Looking for duplicates…"),
Finder::Similar => gettext("Looking for similar photos…"),
});
view.status.set_description(None);
view.progress.set_visible(false);
view.note.set_visible(false);
view.stack.set_visible_child_name("status");
ask(view, finder, generation);
}
/// Ask the daemon for `finder`'s sets and show them, or, while it is still
/// hashing thumbnails, how far it has got — and ask again in a moment.
fn ask(view: &Rc<FinderView>, finder: Finder, generation: u32) {
let request = match finder {
Finder::Exact => Request::PhotoDuplicates,
Finder::Similar => Request::PhotoSimilar,
};
let rx = spawn_request(view.ui.dirs.control_socket(), request);
let view = view.clone();
glib::spawn_future_local(async move {
let reply = rx.recv().await;
if view.generation.get() != generation {
return;
}
let failed = match finder {
Finder::Exact => gettext("Couldn't look for duplicates"),
Finder::Similar => gettext("Couldn't look for similar photos"),
};
let (sets, uncompared) = match reply {
Ok(Ok(Response::PhotoDuplicates { sets })) => (sets, 0),
Ok(Ok(Response::PhotoSimilar {
hashing: Some(MappingProgress { done, total }),
..
})) => {
view.status.set_title(&gettext("Comparing photos…"));
view.status.set_description(Some(&gettext(
"Photos are compared by their thumbnails. The first time takes a while.",
)));
view.progress.set_fraction(if total == 0 {
0.0
} else {
done as f64 / total as f64
});
view.progress.set_text(Some(&gettext_f(
// Translators: {done} and {total} are numbers of photos.
"{done} of {total}",
&[("done", &thousands(done)), ("total", &thousands(total))],
)));
view.progress.set_visible(true);
glib::timeout_add_local_once(HASHING_POLL, move || {
if view.generation.get() == generation {
ask(&view, finder, generation);
}
});
return;
}
Ok(Ok(Response::PhotoSimilar {
sets,
hashing: None,
uncompared,
})) => (sets, uncompared),
Ok(Ok(Response::Error { message, .. })) => {
view.status.set_icon_name(Some("dialog-warning-symbolic"));
view.status.set_title(&failed);
view.status.set_description(Some(&message));
view.progress.set_visible(false);
return;
}
_ => {
view.status.set_icon_name(Some("dialog-warning-symbolic"));
view.status.set_title(&failed);
view.status
.set_description(Some(&gettext("The Proton Drive service didn't respond.")));
view.progress.set_visible(false);
return;
}
};
view.progress.set_visible(false);
if sets.is_empty() {
view.status.set_icon_name(Some("pdfs-synced-symbolic"));
match finder {
Finder::Exact => {
view.status.set_title(&gettext("No duplicates"));
view.status.set_description(Some(&gettext(
"Every photo in your library is stored once.",
)));
}
Finder::Similar => {
view.status.set_title(&gettext("No similar photos"));
view.status.set_description(Some(&gettext(
"No two photos in your library look alike.",
)));
}
}
if uncompared > 0 {
view.note.set_label(&uncompared_note(uncompared));
view.note.set_visible(true);
}
return;
}
let (review, handler) = duplicate_review(&view.ui, sets, &view.trash, finder, uncompared);
view.clicked.replace(Some(handler));
view.stack.add_named(&review, Some("review"));
view.stack.set_visible_child_name("review");
view.toolbar.set_reveal_bottom_bars(true);
schedule_thumbs(&view.ui);
});
}
/// Says how many photos the similar finder left out for want of a thumbnail.
fn uncompared_note(n: usize) -> String {
ngettext_f(
"{n} photo couldn't be compared because it has no thumbnail yet.",
"{n} photos couldn't be compared because they have no thumbnail yet.",
n as u64,
&[],
)
}
/// The review list: one section per set, each photo a thumbnail with a Keep
/// toggle. Wires `trash` to move every photo not kept to Trash, and returns
/// that handler so a new search can take it off again.
///
/// Exact copies start with only the first of each set kept. Similar photos
/// all start kept: photos that look alike can still be different shots.
fn duplicate_review(
ui: &Rc<Ui>,
sets: Vec<Vec<PhotoItem>>,
trash: &gtk4::Button,
finder: Finder,
uncompared: usize,
) -> (gtk4::Widget, glib::SignalHandlerId) {
let intro = gtk4::Label::builder()
.label(match finder {
Finder::Exact => gettext(
"These photos are stored more than once. The copies you don't keep move to Trash, where you can restore them.",
),
Finder::Similar => gettext(
"These photos look alike. They can be copies of one photo or different shots of one moment, so all of them are kept until you untick them. The photos you don't keep move to Trash, where you can restore them.",
),
})
.wrap(true)
.xalign(0.0)
.build();
intro.add_css_class("dim-label");
let content = gtk4::Box::new(gtk4::Orientation::Vertical, 18);
content.set_margin_top(12);
content.set_margin_bottom(12);
content.set_margin_start(18);
content.set_margin_end(18);
content.append(&intro);
if uncompared > 0 {
let note = gtk4::Label::builder()
.label(uncompared_note(uncompared))
.wrap(true)
.xalign(0.0)
.build();
note.add_css_class("dim-label");
content.append(&note);
}
// Every copy's toggle, by set, so the button can count what goes.
let toggles: Rc<RefCell<SetToggles>> = Rc::new(RefCell::new(Vec::new()));
for set in sets {
let Some(first) = set.first() else {
continue;
};
let date = section_heading(first.capture_time, Grouping::Day);
let title = match finder {
// Translators: heading of one set in the duplicates dialog; {date} is the day the photo was taken.
Finder::Exact => ngettext_f(
"{date}, {n} copy",
"{date}, {n} copies",
set.len() as u64,
&[("date", &date)],
),
// Translators: heading of one set of photos that look alike; {date} is the day the first was taken.
Finder::Similar => ngettext_f(
"{date}, {n} photo",
"{date}, {n} photos",
set.len() as u64,
&[("date", &date)],
),
};
let heading = gtk4::Label::builder().label(title).xalign(0.0).build();
heading.add_css_class("heading");
let row = gtk4::Box::new(gtk4::Orientation::Horizontal, 12);
let mut set_toggles = Vec::new();
for (index, photo) in set.into_iter().enumerate() {
let (card, keep) = copy_card(ui, &photo, index == 0 || finder == Finder::Similar);
row.append(&card);
set_toggles.push((photo, keep));
}
let row_scroll = gtk4::ScrolledWindow::builder()
.vscrollbar_policy(gtk4::PolicyType::Never)
.propagate_natural_height(true)
.child(&row)
.build();
let section = gtk4::Box::new(gtk4::Orientation::Vertical, 6);
section.append(&heading);
section.append(&row_scroll);
content.append(&section);
toggles.borrow_mut().push(set_toggles);
}
let sync = {
let toggles = toggles.clone();
let trash = trash.clone();
move || {
let n = doomed(&toggles.borrow()).len();
trash.set_label(&match finder {
Finder::Exact => ngettext_f(
"Move {n} copy to Trash",
"Move {n} copies to Trash",
n as u64,
&[],
),
Finder::Similar => ngettext_f(
"Move {n} photo to Trash",
"Move {n} photos to Trash",
n as u64,
&[],
),
});
trash.set_sensitive(n > 0);
}
};
sync();
for set in toggles.borrow().iter() {
for (_, keep) in set {
let toggles = toggles.clone();
let sync = sync.clone();
keep.connect_toggled(move |keep| {
// A set always keeps a copy: unticking the last one would trash
// the photo itself, which is not what removing duplicates means.
if !keep.is_active()
&& toggles.borrow().iter().any(|set| {
set.iter().any(|(_, t)| t == keep)
&& set.iter().all(|(_, t)| !t.is_active())
})
{
keep.set_active(true);
return;
}
sync();
});
}
}
let ui_trash = ui.clone();
let handler = trash.connect_clicked(move |button| {
let uids = doomed(&toggles.borrow());
if uids.is_empty() {
return;
}
// An exact copy is one file of its shot; a similar photo stands for
// its whole shot, raw and all.
trash_photos(&ui_trash, uids, finder == Finder::Exact);
if let Some(dialog) = button
.ancestor(adw::Dialog::static_type())
.and_downcast::<adw::Dialog>()
{
dialog.close();
}
});
let scroll = gtk4::ScrolledWindow::builder()
.hscrollbar_policy(gtk4::PolicyType::Never)
.vexpand(true)
.child(&content)
.build();
(scroll.upcast(), handler)
}
/// The uids of every copy not kept.
fn doomed(toggles: &SetToggles) -> Vec<String> {
toggles
.iter()
.flatten()
.filter(|(_, keep)| !keep.is_active())
.map(|(photo, _)| photo.uid.clone())
.collect()
}
/// One copy: its thumbnail, its file name, and whether to keep it.
fn copy_card(ui: &Rc<Ui>, photo: &PhotoItem, keep: bool) -> (gtk4::Box, gtk4::ToggleButton) {
let picture = gtk4::Picture::builder()
.content_fit(gtk4::ContentFit::Cover)
.can_shrink(true)
.build();
let placeholder = gtk4::Image::builder()
.icon_name("image-x-generic-symbolic")
.pixel_size(24)
.build();
placeholder.add_css_class("photo-placeholder");
let overlay = gtk4::Overlay::new();
overlay.set_size_request(COPY_SIZE, COPY_SIZE);
overlay.set_child(Some(&placeholder));
overlay.add_overlay(&picture);
overlay.set_overflow(gtk4::Overflow::Hidden);
overlay.add_css_class("memory-cover");
want_thumb(ui, photo, &picture);
let name = photo.name.clone().unwrap_or_default();
let label = gtk4::Label::builder()
.label(&name)
.tooltip_text(&name)
.ellipsize(gtk4::pango::EllipsizeMode::Middle)
.max_width_chars(18)
.build();
label.add_css_class("caption");
let toggle = gtk4::ToggleButton::builder()
.label(gettext("Keep"))
.active(keep)
.build();
toggle.add_css_class("pill");
let card = gtk4::Box::new(gtk4::Orientation::Vertical, 6);
card.set_size_request(COPY_SIZE, -1);
card.append(&overlay);
card.append(&label);
card.append(&toggle);
(card, toggle)
}
File diff suppressed because it is too large. Load diff
Loaded 100 of 189 files, more files were not shown because too many files have changed in this diff. Show more