Compare commits

..
Author SHA1 Message Date
nvrl d1b7535696 Merge branch 'fix/restore-outcome-and-walk-gaps' into fix/sim-flakes-and-open-race
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:44 +01:00
nvrl ba414c4446 Merge branch 'fix/diagnose-coverage' into fix/restore-outcome-and-walk-gaps
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:41 +01:00
nvrl 5b7f17bdaf Merge branch 'fix/db-errors-not-offline' into fix/diagnose-coverage
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:38 +01:00
nvrl 73068f5868 Merge branch 'fix/mount-failure-surfaced' into fix/db-errors-not-offline
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:36 +01:00
nvrl b093010b64 Merge branch 'fix/never-synced-label' into fix/mount-failure-surfaced
# Conflicts:
#	docs/BUGS.md
#	po/de.po
#	po/en_GB.po
#	po/es.po
#	po/fr.po
#	po/it.po
#	po/nl.po
#	po/pdfs.pot
#	po/pl.po
#	po/pt_BR.po
2026-10-31 19:26:34 +01:00
nvrl ab8fe0e61d Merge branch 'fix/sync-scheduling-watches' into fix/never-synced-label
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:28 +01:00
nvrl b8295373f7 Merge remote-tracking branch 'origin/docs/bugs-issue-audit' into fix/sync-scheduling-watches
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:21:21 +01:00
nvrl 5bc0b5c58a Merge pull request #56 from narrrl/fix/sync-engine-containment
fix(fuse,core,cli): keep a panicking sync pass from ending the engine
2026-10-31 19:20:04 +01:00
nvrl 73af39ee65 Merge remote-tracking branch 'origin/docs/bugs-issue-audit' into fix/sync-engine-containment
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:11:41 +01:00
nvrl 48aaad23ce Merge pull request #55 from narrrl/fix/journald-priority
fix(cli): give journald each log line's priority
2026-10-31 19:09:37 +01:00
nvrl 53543c91f9 docs(bugs): record the slow-link conflict copy seen in CI (B275) 2026-10-31 13:16:47 +01:00
nvrl fccb3d9561 fix(fuse): retry a write open whose queued revision landed under the copy
A write open copies the queued revision's blob into its scratch file.
When the upload landed between the sample and the copy, the drain had
already deleted the blob, and the open failed with EIO (B274, seen once
in CI). The drain drops the pending entry before the blob, so a failed
copy whose entry moved on now goes around again, as the re-check after
the copy already does.

Two simulations raced the daemon rather than it:
- a_file_deleted_after_a_restart_leaves_drive_with_what_its_create_sent
  let the create's reply through before the kernel's release attached
  the second bytes. That attach resets the attempts, so the retry
  landed before the test could see the failure. It now waits for the
  bytes to be attached.
- a_file_deleted_offline_while_open_still_reads_and_takes_writes read
  the cache as the queue emptied, which is a moment before the
  eviction. It now waits for the cache to empty.
2026-10-31 12:58:11 +01:00
nvrl 6f2b4bb80a fix(fuse,cli): keep a pass going past a failed listing, report skipped restores
A listing that failed anywhere in the remote walk ended the whole pass,
so a first sync of a large folder on a flaky link walked the tree again
and again and never downloaded anything (B271, issue #46). A failed
listing below the root now records that folder as a gap: nothing under
it is deleted, the rest of the pass applies, and the report names it.

A restore that skipped every folder was logged as a successful activity
(B269). The daemon now gets the restored and skipped folders back, logs
any skip as a failed activity with the reasons, and sends the Rewatch
for folders already restored when the database insert fails partway.
`pdfs devices join` points to `pdfs activity` for the outcome.
2026-10-31 11:11:01 +01:00
nvrl a617b2b410 fix(cli): have diagnose report failing writes, a pause and folders in error
diagnose dropped most of the Status reply and never looked at the synced
folders, so it said "No problems found" while a folder would not sync
(B268, issue #53). It now warns about writes that keep failing, a write
a day old, a pause, folders in error or conflict, and a control socket
nothing listens on.
2026-10-31 10:57:34 +01:00
nvrl bdbfc0bfd4 fix(core): read database errors handed to from_api as internal, not offline
from_api took every error that was not a ProtonError for a transport
failure. Pin, unpin, the searches and node loads hand it pdfs_core
errors from the cache and the database, so a failure there said
"You're offline" (B266, the rest of B205). The source chain now
decides, and only an error with neither type reads as offline.
2026-10-31 10:53:25 +01:00
nvrl e9577f953f fix(cli,core,fuse,gui): say why the mount keeps failing, and back off
A mount that failed on every try retried every 5 s forever, and the
reason was only in the journal: the app and the tray said Connecting...
for days (B263, issue #40). The retry now doubles to a 2-minute ceiling,
and the last failure is kept in mount-failure.json for the app, the
tray, pdfs diagnose and systemctl status.

The early returns in mount left the workers they had started running
beside the next attempt (B264). They are now stopped on every error
path, without telling systemd the daemon is stopping.
2026-10-31 10:49:01 +01:00
nvrl 437d14b0f5 fix(gui): say a folder is waiting for its first sync, not up to date
A new sync folder row starts with state idle, so the app called a folder
up to date before any of it reached Drive (B261, issue #53). A mirror
folder that has never synced now reads "waiting for first sync", and
the This computer row counts such folders. All catalogs translated.
2026-10-31 10:30:52 +01:00
nvrl b40ca797d9 fix(fuse): run a burst's named folders first and retry failed watches
A poll in the same burst as a new folder's Reconcile(id) took its place,
so the new folder waited behind a pass over every other one (B260).

A recursive watch that failed partway left its watches registered, was
never retried, and its events ran passes over every folder (B262). It
is now unwatched and retried every 10 minutes, symlinks are not followed,
and watcher errors are logged. A folder removed and added again in one
burst keeps its watch under the new id (B272).
2026-10-31 10:24:32 +01:00
nvrl c64bb23e3c fix(fuse,core,cli): keep a panicking sync pass from ending the engine
One thread runs every synced folder's passes. A panic in a pass ended
it, and every later add, sync-now, file event and poll went nowhere
without a word until a restart, which fits issue #53 (B259).

Each pass now runs under catch_unwind; a panic sets the folder to error
and the next poll tries again. Diagnostics reports the engine's state,
the supervisor warns when it has ended or a pass has gone quiet, and
first passes log at info.
2026-10-31 10:14:23 +01:00
nvrl 8d4a005aa2 fix(cli): give journald each log line's priority
journald stored every daemon line at info, so journalctl -p warning,
which TROUBLESHOOTING.md asks reporters for, was always empty (B258).
When fd 1 is the journal stream, prefix lines with <N> and drop colours.
2026-10-31 09:54:07 +01:00
nvrl 9460c93b50 docs(bugs): record the audit of issues #53, #46, #40 and #38
B258 to B273: why a stuck sync, a failing mount and a silent restore
are invisible, plus the sync, mount and restore bugs behind them.
B165, B202, B205 and B210 point at the entries that finish them.
2026-10-31 09:47:54 +01:00
Nils Pukropp bd50ba5959 chore(aur): update packages to 3.6.0 2026-10-27 13:33:37 +00:00
nvrl 27201914ee Merge pull request #51 from narrrl/fix/emptied-folder-direct-switch
fix(fuse): let an emptied mirror folder switch to on-demand at once
2026-10-27 12:05:50 +01:00
nvrl 2775f0681d test(acceptance): count a purge of a listed trash entry that then vanishes
The reaper found an abandoned root in the trash and sent the delete. The
reply did not confirm it, and the next listing no longer had the node,
but the retry loop only counted deletes it sent itself. It waited for a
node that was already gone, reported it as never trashed, and the
leftover check stopped the run before any case ran.

Callers that just found the node in the trash now say so, and the error
of the first delete is kept for the warning.
2026-10-27 01:22:07 +01:00
nvrl 97277d41dd fix(fuse): let an emptied mirror folder switch to on-demand at once
A switch asked for before any pass saw the folder empty found it still
idle, so the check for what only this computer has ran at once and
refused it for good: every synced path read as "deleted here but not on
Proton Drive yet". The account run hit it in the B255 case.

A local copy gone in full is the wipe guard's case, which the push pass
of a queued switch settles without trashing anything on Drive. The
switch now returns NotNow for it, so the request is queued and that pass
runs first. A partial deletion is still refused as before.
2026-10-27 01:22:07 +01:00
nvrl d49738bf0b Merge pull request #50 from narrrl/test/size-batch-clock-before-spawn
test(fuse): start the size batch clock before the worker
2026-10-27 01:03:49 +01:00
nvrl 48945b9c97 Merge pull request #49 from narrrl/test/b256-case-own-folder
test(acceptance): give the B256 case a copy folder of its own
2026-10-27 01:03:40 +01:00
nvrl b860ac3b49 test(fuse): start the size batch clock before the worker
The worker thread's sleeps could begin before the test read the clock, so the elapsed time came out a few milliseconds under the 90 ms bound on a busy runner. Reading the clock first makes the bound hold.
2026-10-27 01:02:29 +01:00
nvrl 42f3a18177 test(acceptance): give the B256 case a copy folder of its own
Every case of a run shares one root, and the pdfs copy case before it
had already made "copies" there, so the B256 case failed on its mkdir
with EEXIST before it checked anything.
2026-10-27 00:56:35 +01:00
nvrl 6f504d2e83 Merge pull request #48 from narrrl/test/acceptance-new-features
3.6.0: acceptance cases for the new features, --json for members/du/public-link
2026-10-27 00:44:30 +01:00
nvrl 8c1c37a928 test(acceptance): cover copy, folder size, link updates and sync recovery
New cases for what 3.6.0 adds:

- copy_file_range over several calls, into the middle of a file, past
  its end, at the source's end, and from a file whose writes are not
  released yet; on a live mount also read back from Drive.
- pdfs copy of a tree and of a file beside itself, read back as listed
  and after a refresh, with nothing queued; refused while a file has
  writes Drive lacks, and never the version before a save (B256).
- pdfs du counts every item below a folder, trash included.
- pdfs public-link update keeps the URL and the options it was not
  asked to change, and refuses a link id that is not the folder's.
- In a sync folder of their own: Sync From Scratch changes nothing that
  already matched, and an emptied mirror folder still switches to
  on-demand with every file intact on Drive (B255).

docs/TESTING.md gains a checklist for the new screens in the app. The
CHANGELOG moves these into 3.6.0, which is not tagged yet.
2026-10-27 00:42:44 +01:00
nvrl e4d2340bbf feat(cli): --json for members, du and public-link create/update
Scripts, the acceptance suite among them, can now read a link's id,
role, password flag and expiry, and a folder's size and item count,
instead of parsing the human-readable lines.
2026-10-27 00:42:44 +01:00
nvrl c1b72e455e test(fuse): time the short-outage reconnect from the mount coming back
The probe that sees the link return runs 2 s after the drop, then 6 s.
A loaded run could turn the link back on after the first probe, and the
next one came after the test's 3 s were up. The test now checks that the
drain takes the mount offline within 2 s, which is the B189 fix itself,
and counts the 3 s from the mount reporting it is back online.
2026-10-27 00:32:49 +01:00
nvrl 2e3b7748e6 Merge pull request #47 from narrrl/feat/server-copy
Release 3.6.0: server copy, link settings, folder size, sync fixes (#46)
2026-10-26 22:47:47 +01:00
nvrl 7f4f11ef18 chore(release): 3.6.0 2026-10-26 22:45:36 +01:00
nvrl cc9f7c6d8a Merge remote-tracking branch 'origin/main' into feat/server-copy
# Conflicts:
#	Cargo.lock
#	Cargo.toml
2026-10-26 22:41:40 +01:00
nvrl 9add185c1f Merge pull request #43 from narrrl/feat/diagnose-keys
feat(cli,core): trace the keys that open My Files in pdfs diagnose
2026-10-26 22:35:35 +01:00
nvrl 0e7ffe1261 fix(gui,cli): count what a folder's first scan has checked (#46)
The first pass over a synced folder has no baseline to estimate from, so
its scan showed only "checking for changes…" in the app and "scanning…"
in `pdfs sync list`. For a large folder that scan reads every item on
Proton Drive before anything downloads, which takes many minutes and
looked stuck. Both now show how many items the scan has checked so far.
2026-10-26 22:28:46 +01:00
nvrl a5acae41b3 fix(sync,gui,cli): get a synced folder unstuck when its local copy is gone (#46)
A mirror folder whose files were all gone from this computer stayed in
error for good. The wipe guard rightly refuses to trash the whole folder
on Drive, but nothing got the folder out of that state: a queued switch
to on-demand waits for a pass that ends idle, and its push pass hit the
same guard. Only removing the folder from sync cleared the baseline that
made the empty folder read as "delete everything".

- The push pass for a switch to on-demand forgets the baseline rows of a
  local copy that is gone in full instead of trashing them, so the
  switch goes ahead and the mount shows what Drive has.
- New Request::ResetSyncFolder, `pdfs sync reset ID` and "Sync From
  Scratch…" in a synced folder's menu clear a mirror folder's baseline
  (and a queued switch to on-demand, whose push pass would upload a
  duplicate of every file). The next pass compares both sides as on the
  first sync: one-sided files are copied across, files that differ keep
  both versions, nothing is deleted.
- The wipe guards' messages name the reset; restore logs why it skipped
  a folder.

Two simulations cover it. The on-demand one failed in exactly the
reported state (mirror, pending ondemand, error) before the fix.
All 8 catalogs translated.
2026-10-26 17:45:22 +01:00
nvrl ea10a43216 fix(fuse): a copy right after a save waits for the edit (B256)
close(2) returns before the kernel sends release, and release takes the
write handle out of the open writes a few milliseconds before the
revision is in the queue. A `pdfs copy` checked in that gap found the
edit nowhere and copied the version Drive had. A folder copy looked only
at the queue, so it also missed files below it still open with unsaved
writes.

release now keeps the inode in State::releasing until the revision is
queued. A copy refuses a file or folder with unsaved writes, or a
release still queueing, at it or below it, and a move checks the same
set.
2026-10-26 17:45:14 +01:00
nvrl 917e7268f9 build(deps): proton-sdk and proton-drive-rs 0.10.0
0.10.0 adds copy_node, get_folder_size, update_public_link and a link's custom password.
2026-10-26 17:03:33 +01:00
nvrl 9089c8d13c feat(fuse): answer copy_file_range from bytes already on this machine
A copy inside the mount went through the kernel's read/write loop, one
FUSE round trip per page, because copy_file_range answered ENOSYS. The
daemon now copies a range whose source bytes are local (an open write
handle, a queued write, the cache) into the destination's scratch file,
the same way write stages them. A range it would have to download gets
EOPNOTSUPP, which makes the kernel copy that one call itself and keeps
the operation enabled. The copy still uploads: Drive cannot copy into a
file cp has already created.

In the simulation, a warm cp of a 256 MiB file went from about
100 MiB/s to about 450 MiB/s (debug build).
2026-10-26 16:57:06 +01:00
nvrl 071a11ef65 Merge remote-tracking branch 'origin/main' into feat/diagnose-keys
# Conflicts:
#	docs/CHANGELOG.md
#	docs/TROUBLESHOOTING.md
2026-10-26 16:45:36 +01:00
nvrl 4edcff3f87 feat(sharing,gui,cli): change public links and show folder sizes
The share dialog gets Link settings under an existing link: role, password
and expiry apply at once and keep the URL. The daemon fills in the options a
change keeps from the link as it stands, and refuses to keep a password it
cannot read rather than dropping it. pdfs public-link update does the same
from a terminal, and pdfs members shows the expiry.

Details shows a folder's size and item count as Proton Drive counts it,
trash included, and pdfs du prints it. Both are refused offline and for a
folder that is not on Drive yet.

Needs proton-drive-rs with update_public_link, get_folder_size and
PublicLink::custom_password.
2026-10-25 22:18:11 +01:00
nvrl ee59e338b6 feat(gui): make a copy and copy to another folder in My files
The entry menu gains Make a Copy, which copies an item next to itself, and
Copy To…, which reuses the move folder picker limited to My files; Copy
To… is also in the selection menu. Both send Request::Copy, so the copy is
made on Proton Drive.

The move and copy reports share one request loop. Docs describe the new
actions, and the changelog notes that saved public links now show their
names once the SDK decrypts them.
2026-10-25 21:41:21 +01:00
nvrl c1d9538669 feat(cli,fuse,core): copy files and folders on Proton Drive
`pdfs copy PATH... NEW_PARENT` sends a new control request, Copy, which
copies a file with its content or a folder with its subtree on Proton
Drive through proton-drive-rs's copy_node. Nothing is downloaded or
uploaded. A name already taken in the destination gets the next free
" (n)" suffix through get_available_name.

A copy cannot wait in the queue, because Drive copies what it holds.
So it is refused offline, and while the mount still owes Drive part of
the node: a local placeholder, a queued op, unsaved writes, or, for a
folder, any queued op at all. Local-first does not change this.

DriveApi gains copy_node and get_available_name, and the fake drive
copies subtrees, so a FUSE sim test covers the copy, the suffix, the
folder subtree and both refusals. The activity feed logs a new Copy
kind, translated in every catalog.

Needs the proton-drive-rs release that adds copy_node; until then, build
with a [patch.crates-io] pointing at proton-sdk-rs.
2026-10-25 21:10:50 +01:00
nvrl a73ed34421 Merge pull request #45 from narrrl/docs/wiki
docs: publish docs/ to the GitHub wiki and reorganise the user guide
2026-10-17 14:06:46 +02:00
nvrl e065586d7e build(deps): proton-sdk and proton-drive-rs 0.9.2
0.9.2 adds the key report pdfs diagnose uses.
2026-10-17 12:23:19 +02:00
nvrl fd80c34152 feat(cli,core): trace the keys that open My Files in pdfs diagnose
When the session is stored and unlocked but the daemon does not answer,
pdfs diagnose asks Proton for a key report and prints a Keys section:
each account key, each address key and the account key its token is
for, and the address key the My Files share is encrypted to. It holds
key IDs only, so it can be posted in an issue.

Refs #40
2026-10-17 12:13:27 +02:00
65 changed files with 10403 additions and 4034 deletions

No files matched your search

Generated
+8 -8
View File
@@ -3474,7 +3474,7 @@ checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4"
[[package]]
name = "pdfs-cli"
version = "3.5.0"
version = "3.6.0"
dependencies = [
"anyhow",
"clap",
@@ -3490,7 +3490,7 @@ dependencies = [
[[package]]
name = "pdfs-core"
version = "3.5.0"
version = "3.6.0"
dependencies = [
"async-trait",
"base64 0.23.1",
@@ -3517,7 +3517,7 @@ dependencies = [
[[package]]
name = "pdfs-fuse"
version = "3.5.0"
version = "3.6.0"
dependencies = [
"async-trait",
"base64 0.23.1",
@@ -3539,7 +3539,7 @@ dependencies = [
[[package]]
name = "pdfs-gui"
version = "3.5.0"
version = "3.6.0"
dependencies = [
"anyhow",
"async-channel",
@@ -3779,9 +3779,9 @@ dependencies = [
[[package]]
name = "proton-drive-rs"
version = "0.9.1"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e6b87f78711e8fc72737c8eb24ebda156079308d79ab37ca2b1b32de6812880"
checksum = "8f70318394b2815b009c52f804d75747086db93a338ceb6eae62541f2b25d5bf"
dependencies = [
"async-trait",
"base64 0.23.1",
@@ -3806,9 +3806,9 @@ dependencies = [
[[package]]
name = "proton-sdk"
version = "0.9.1"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19f1936755c603b98de1c82a5b343c6232e9fea8adaa77139f7cb8fcddcda697"
checksum = "4196392032f56b3bbaac0bb1f24eea62fa7e2a0af72004de8d08e518d14720fb"
dependencies = [
"aes-gcm 0.11.1",
"async-trait",
+3 -3
View File
@@ -3,7 +3,7 @@ resolver = "2"
members = ["crates/pdfs-core", "crates/pdfs-fuse", "crates/pdfs-cli", "crates/pdfs-gui"]
[workspace.package]
version = "3.5.0"
version = "3.6.0"
edition = "2024"
license = "MIT"
authors = ["Nils Pukropp <contact@narl.io>"]
@@ -15,8 +15,8 @@ keywords = ["proton", "drive", "fuse", "sync", "linux"]
categories = ["filesystem", "command-line-utilities"]
[workspace.dependencies]
proton-sdk = { version = "0.9.1" }
proton-drive-rs = { version = "0.9.1" }
proton-sdk = { version = "0.10.0" }
proton-drive-rs = { version = "0.10.0" }
# Internal crates.
pdfs-core = { path = "crates/pdfs-core" }
+122
View File
@@ -0,0 +1,122 @@
//! Log priorities for journald.
//!
//! journald files each line a service writes at the unit's default priority,
//! info, unless the line opens with a syslog `<N>` prefix. Without one every
//! WARN and ERROR the daemon logs is stored as info, and the
//! `journalctl -p warning` that the troubleshooting guide asks for shows
//! nothing (`docs/BUGS.md` B258).
use std::fmt;
use std::os::unix::fs::MetadataExt;
use tracing::{Event, Level, Subscriber};
use tracing_subscriber::fmt::format::Writer;
use tracing_subscriber::fmt::{FmtContext, FormatEvent, FormatFields};
use tracing_subscriber::registry::LookupSpan;
/// Whether standard output is the journal stream systemd connected this
/// process to.
///
/// `JOURNAL_STREAM` alone does not say so: a terminal started from a user unit
/// inherits it, and every `pdfs` run in that terminal would print the
/// prefixes. systemd sets it to the stream's `device:inode`, so it is compared
/// against what fd 1 actually is.
pub(crate) fn stdout_is_journal() -> bool {
let Some(stream) = std::env::var_os("JOURNAL_STREAM") else {
return false;
};
let Ok(stdout) = std::fs::metadata("/proc/self/fd/1") else {
return false;
};
stream
.to_str()
.is_some_and(|value| names_stream(value, stdout.dev(), stdout.ino()))
}
/// Whether a `JOURNAL_STREAM` value names the file `dev`:`ino`.
fn names_stream(value: &str, dev: u64, ino: u64) -> bool {
value
.split_once(':')
.is_some_and(|(d, i)| d.parse() == Ok(dev) && i.parse() == Ok(ino))
}
/// The syslog priority journald should file a `level` line at.
fn priority(level: &Level) -> u8 {
match *level {
Level::ERROR => 3,
Level::WARN => 4,
Level::INFO => 6,
Level::DEBUG | Level::TRACE => 7,
}
}
/// An event format that opens each line with its `<N>` priority, then writes
/// the line as `F` would.
pub(crate) struct Prioritized<F>(pub(crate) F);
impl<S, N, F> FormatEvent<S, N> for Prioritized<F>
where
S: Subscriber + for<'a> LookupSpan<'a>,
N: for<'a> FormatFields<'a> + 'static,
F: FormatEvent<S, N>,
{
fn format_event(
&self,
ctx: &FmtContext<'_, S, N>,
mut writer: Writer<'_>,
event: &Event<'_>,
) -> fmt::Result {
write!(writer, "<{}>", priority(event.metadata().level()))?;
self.0.format_event(ctx, writer, event)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io;
use std::sync::{Arc, Mutex};
#[derive(Clone)]
struct Buf(Arc<Mutex<Vec<u8>>>);
impl io::Write for Buf {
fn write(&mut self, bytes: &[u8]) -> io::Result<usize> {
self.0.lock().unwrap().extend_from_slice(bytes);
Ok(bytes.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
#[test]
fn each_line_opens_with_the_priority_of_its_level() {
let buf = Buf(Arc::new(Mutex::new(Vec::new())));
let writer = buf.clone();
let subscriber = tracing_subscriber::fmt()
.with_max_level(Level::DEBUG)
.with_ansi(false)
.with_writer(move || writer.clone())
.event_format(Prioritized(tracing_subscriber::fmt::format()))
.finish();
tracing::subscriber::with_default(subscriber, || {
tracing::error!("e");
tracing::warn!("w");
tracing::info!("i");
tracing::debug!("d");
});
let out = String::from_utf8(buf.0.lock().unwrap().clone()).unwrap();
let prefixes: Vec<&str> = out.lines().map(|line| &line[..3]).collect();
assert_eq!(prefixes, ["<3>", "<4>", "<6>", "<7>"]);
}
#[test]
fn journal_stream_must_name_the_same_file() {
assert!(names_stream("8:12345", 8, 12345));
assert!(!names_stream("8:12345", 8, 54321));
assert!(!names_stream("9:12345", 8, 12345));
assert!(!names_stream("8", 8, 12345));
assert!(!names_stream("", 8, 12345));
}
}
+491 -25
View File
@@ -13,13 +13,15 @@ use pdfs_core::auth;
use pdfs_core::cache::ContentCache;
use pdfs_core::config::AppDirs;
use pdfs_core::control::{
ConflictKeep, ErrorKind, MountKind, RefreshScope, Request as CtlRequest,
ConflictKeep, ErrorKind, LinkChange, MountKind, RefreshScope, Request as CtlRequest,
Response as CtlResponse, RestoreItem as CtlRestoreItem, ShareEntryKind, SyncIssue, SyncPhase,
daemon_outdated, pending_summary,
};
use pdfs_core::db::Db;
use pdfs_core::service;
mod journal;
/// `println!` for this binary, shadowing std's, which panics when stdout is
/// closed: `pdfs ls | head` ended in a "Broken pipe" panic once `head` had its
/// lines. This one hands the failure to [`stdout_written`] instead.
@@ -356,6 +358,26 @@ enum Command {
#[arg(required = true, num_args = 2.., value_name = "PATH")]
paths: Vec<PathBuf>,
},
/// Copy files or folders into another folder in My files via the running
/// daemon.
///
/// The copy happens on Proton Drive, so nothing is downloaded or uploaded,
/// unlike `cp` inside the mount. A name already taken in the destination
/// gets a " (1)" suffix. The last path is the destination folder.
Copy {
/// Files/folders to copy, then the destination folder: inside My files,
/// or relative to the mountpoint.
#[arg(required = true, num_args = 2.., value_name = "PATH")]
paths: Vec<PathBuf>,
},
/// Show how much a folder holds on Proton Drive.
///
/// Proton Drive adds it up, so nothing is downloaded. The size is the
/// encrypted bytes on storage and includes items in the trash.
Du {
/// Folder path, inside My files or relative to the mountpoint.
path: PathBuf,
},
/// Trash a file or folder via the running daemon.
Rm {
/// File/folder path, inside My files or an on-demand synced folder, or
@@ -689,6 +711,13 @@ enum SyncCmd {
/// Optional synced-folder id; omit to reconcile every folder.
id: Option<i64>,
},
/// Forget what a synced folder synced before and compare it with Proton
/// Drive afresh: what one side has alone is copied to the other, and a file
/// that differs keeps both versions. Nothing is deleted.
Reset {
/// Synced-folder id (from `sync list`).
id: i64,
},
/// Pause syncing: queued uploads and mirror folders wait, reads keep working.
Pause {
/// End the pause by itself after this long, e.g. `30m`, `1h`, `2d`.
@@ -777,6 +806,29 @@ enum PublicLinkCmd {
#[arg(long)]
expires: Option<i64>,
},
/// Change a node's public link. The URL stays the same; an option not
/// given is kept.
Update {
/// File/folder path, inside the mountpoint or relative to it.
path: PathBuf,
/// Public-link id, from `pdfs members`.
id: String,
/// New role: viewer or editor.
#[arg(long)]
role: Option<String>,
/// New custom password protecting the link.
#[arg(long, conflicts_with = "no_password")]
password: Option<String>,
/// Remove the custom password.
#[arg(long)]
no_password: bool,
/// New expiry, Unix epoch seconds.
#[arg(long, conflicts_with = "no_expiry")]
expires: Option<i64>,
/// Remove the expiry.
#[arg(long)]
no_expiry: bool,
},
/// Remove a node's public link.
Remove {
/// File/folder path, inside the mountpoint or relative to it.
@@ -829,15 +881,22 @@ fn cli_error(kind: ErrorKind, message: &str) -> String {
}
fn main() -> Result<()> {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
// `pgp` logs a benign WARN when it re-serializes a Proton secret-key
// packet with a longer (but equivalent) length encoding than the
// original; drop it to error so normal runs stay quiet.
.unwrap_or_else(|_| "info,pgp=error".into()),
)
.init();
let filter = tracing_subscriber::EnvFilter::try_from_default_env()
// `pgp` logs a benign WARN when it re-serializes a Proton secret-key
// packet with a longer (but equivalent) length encoding than the
// original; drop it to error so normal runs stay quiet.
.unwrap_or_else(|_| "info,pgp=error".into());
if journal::stdout_is_journal() {
// journalctl colours lines by the priority the prefix gives them, so
// the escape codes would only be noise in the stored text.
tracing_subscriber::fmt()
.with_env_filter(filter)
.with_ansi(false)
.event_format(journal::Prioritized(tracing_subscriber::fmt::format()))
.init();
} else {
tracing_subscriber::fmt().with_env_filter(filter).init();
}
let cli = Cli::parse();
JSON_OUTPUT.store(cli.json, std::sync::atomic::Ordering::Relaxed);
@@ -919,6 +978,11 @@ fn main() -> Result<()> {
let new_parent = paths.pop().expect("clap requires at least two paths");
cmd_move(paths, new_parent)
}
Command::Copy { mut paths } => {
let new_parent = paths.pop().expect("clap requires at least two paths");
cmd_copy(paths, new_parent)
}
Command::Du { path } => cmd_du(path),
Command::Rm { path } => cmd_rm(path),
Command::Mkdir { parent, name } => cmd_mkdir(parent, name),
Command::Upload { sources, parent } => cmd_upload(sources, parent),
@@ -1088,7 +1152,14 @@ fn cmd_sync(action: SyncCmd) -> Result<()> {
// A pass in flight says what it is doing, indented under it.
if let Some(p) = &f.progress {
match p.phase {
SyncPhase::Scanning if p.total == 0 => println!(" scanning…"),
SyncPhase::Scanning if p.total == 0 && p.done == 0 => {
println!(" scanning…")
}
// A first pass has no estimate yet; the count alone
// shows a long first scan is moving.
SyncPhase::Scanning if p.total == 0 => {
println!(" scanning: {} checked", p.done)
}
SyncPhase::Scanning => {
println!(
" scanning: {} of {}",
@@ -1117,6 +1188,7 @@ fn cmd_sync(action: SyncCmd) -> Result<()> {
})?)?
}
SyncCmd::Now { id } => ok_or_bail(control_request(CtlRequest::SyncNow { id })?)?,
SyncCmd::Reset { id } => ok_or_bail(control_request(CtlRequest::ResetSyncFolder { id })?)?,
SyncCmd::Pause {
folder: Some(id), ..
} => ok_or_bail(control_request(CtlRequest::SetSyncFolderPaused {
@@ -1395,7 +1467,11 @@ fn cmd_sync_restore(yes: bool, device: Option<String>) -> Result<()> {
items: chosen,
},
None => CtlRequest::RestoreSyncFolders { items: chosen },
})?)
})?)?;
// The daemon acks before it works, so the outcome, and any folder it had
// to skip, only shows up later.
println!("Run `pdfs activity` to see which folders were restored.");
Ok(())
}
/// Expand a leading `~` in a hand-typed path. The daemon requires an absolute
@@ -1493,9 +1569,13 @@ fn cmd_versions(action: VersionCmd) -> Result<()> {
}
fn cmd_members(path: PathBuf) -> Result<()> {
match control_request(CtlRequest::ListShare {
let response = control_request(CtlRequest::ListShare {
path: path_arg(&path)?,
})? {
})?;
if emit_json(&response)? {
return Ok(());
}
match response {
CtlResponse::Share { entries, link } => {
if entries.is_empty() {
println!("No members or pending invitations.");
@@ -1512,7 +1592,11 @@ fn cmd_members(path: PathBuf) -> Result<()> {
Some(l) => {
let url = l.url.as_deref().unwrap_or("(url hidden)");
let pw = if l.has_password { " +password" } else { "" };
println!("public link ({}{pw}): {url} [{}]", l.role, l.id);
let expiry = l
.expires
.map(|at| format!(", expires {}", format_epoch_utc(at)))
.unwrap_or_default();
println!("public link ({}{pw}{expiry}): {url} [{}]", l.role, l.id);
}
None => println!("public link: none"),
}
@@ -1558,6 +1642,29 @@ fn cmd_public_link(action: PublicLinkCmd) -> Result<()> {
password,
expires,
})? {
response if emit_json(&response)? => {}
CtlResponse::PublicLink { link } => {
println!("{}", link.url.as_deref().unwrap_or("(no url returned)"));
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
},
PublicLinkCmd::Update {
path,
id,
role,
password,
no_password,
expires,
no_expiry,
} => match control_request(CtlRequest::UpdatePublicLink {
path: path_arg(&path)?,
id,
role,
password: link_change(password, no_password),
expires: link_change(expires, no_expiry),
})? {
response if emit_json(&response)? => {}
CtlResponse::PublicLink { link } => {
println!("{}", link.url.as_deref().unwrap_or("(no url returned)"));
}
@@ -1574,6 +1681,15 @@ fn cmd_public_link(action: PublicLinkCmd) -> Result<()> {
Ok(())
}
/// A `--x VALUE` / `--no-x` flag pair as one link option change.
fn link_change<T>(value: Option<T>, remove: bool) -> LinkChange<T> {
match (value, remove) {
(Some(value), _) => LinkChange::Set(value),
(None, true) => LinkChange::Remove,
(None, false) => LinkChange::Keep,
}
}
fn cmd_shared() -> Result<()> {
match control_request(CtlRequest::ListSharedByMe)? {
CtlResponse::SharedByMe { items } if items.is_empty() => {
@@ -1765,6 +1881,7 @@ fn activity_verb(kind: pdfs_core::control::ActivityKind) -> &'static str {
Sync => "sync",
Rename => "rename",
Move => "move",
Copy => "copy",
CreateFolder => "mkdir",
Trash => "trash",
Restore => "restore",
@@ -2271,12 +2388,21 @@ where
/// an external unmount triggers a remount; errors back off and retry. This is
/// the entry point for the systemd user service.
fn cmd_daemon(mountpoint: Option<PathBuf>) -> Result<()> {
let state_dir = AppDirs::new()?.state_dir();
// A failure an earlier daemon recorded is not this one's.
pdfs_core::mountfail::clear(&state_dir);
let mut failure: Option<pdfs_core::mountfail::MountFailure> = None;
loop {
// Wait until a session is stored. The GUI enables this service on login,
// but the service may also start at boot before the user has logged in.
wait_for_session(auth::ready, pdfs_fuse::idle_sleep)?;
match mount_once(mountpoint.clone()) {
let outcome = mount_once(mountpoint.clone());
if outcome.is_ok() {
// The mount came up, so the next failure starts a new run of them.
failure = None;
}
match outcome {
Ok(pdfs_fuse::MountOutcome::Shutdown) => {
tracing::info!("daemon stopping");
return Ok(());
@@ -2291,13 +2417,51 @@ fn cmd_daemon(mountpoint: Option<PathBuf>) -> Result<()> {
Err(e) => {
// `{:#}` so the whole anyhow chain lands in the log — the top
// context alone ("mount failed") never says *why*.
tracing::error!(error = format!("{e:#}"), "mount failed; retrying in 5s");
pdfs_fuse::idle_sleep(std::time::Duration::from_secs(5));
let error = format!("{e:#}");
let now = unix_now();
let attempts = failure.as_ref().map_or(1, |f| f.attempts.saturating_add(1));
let delay = mount_retry_delay(attempts);
tracing::error!(
error,
attempts,
retry_secs = delay.as_secs(),
"mount failed; retrying"
);
let record = pdfs_core::mountfail::MountFailure {
error: error.clone(),
since: failure.as_ref().map_or(now, |f| f.since),
attempts,
next_attempt: now + delay.as_secs() as i64,
};
if let Err(e) = pdfs_core::mountfail::record(&state_dir, &record) {
tracing::warn!(error = %e, "could not record the mount failure");
}
pdfs_fuse::notify_status(&format!("Mount failing: {error}"));
failure = Some(record);
pdfs_fuse::idle_sleep(delay);
}
}
}
}
/// How long to wait before mount attempt `attempts + 1`, after `attempts`
/// failures in a row.
///
/// Doubles from 5 s to a 2-minute ceiling. A flat 5 s retry sent about four
/// requests to Proton per attempt, tens of thousands a day from an account
/// whose mount fails on a key, which no retry fixes (`docs/BUGS.md` B263). The
/// ceiling stays at minutes because a cold start without a network fails the
/// same way, and must mount soon after the network returns.
fn mount_retry_delay(attempts: u32) -> std::time::Duration {
const BASE_SECS: u64 = 5;
const MAX_SECS: u64 = 120;
let factor = 1u64
.checked_shl(attempts.saturating_sub(1))
.unwrap_or(u64::MAX);
std::time::Duration::from_secs(BASE_SECS.saturating_mul(factor).min(MAX_SECS))
}
/// One mount run: resume the session, mount, and block until the mount ends
/// (clean shutdown or external unmount). Returns the [`MountOutcome`] so the
/// daemon loop can decide whether to exit or remount.
@@ -2920,6 +3084,56 @@ fn cmd_move(sources: Vec<PathBuf>, new_parent: PathBuf) -> Result<()> {
Ok(())
}
fn cmd_copy(sources: Vec<PathBuf>, new_parent: PathBuf) -> Result<()> {
let new_parent = path_arg(&new_parent)?;
let mut failed = 0;
for source in &sources {
let outcome = path_arg(source).and_then(|path| {
match control_request(CtlRequest::Copy {
path,
new_parent: new_parent.clone(),
})? {
CtlResponse::Ok { message } => Ok(message),
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
});
match outcome {
Ok(message) => println!("{message}"),
Err(e) if sources.len() == 1 => return Err(e),
Err(e) => {
eprintln!("{}: {e:#}", source.display());
failed += 1;
}
}
}
if failed > 0 {
bail!("{failed} of {} could not be copied", sources.len());
}
Ok(())
}
fn cmd_du(path: PathBuf) -> Result<()> {
let response = control_request(CtlRequest::FolderSize {
path: path_arg(&path)?,
})?;
if emit_json(&response)? {
return Ok(());
}
match response {
CtlResponse::FolderSize { size, items } => {
let noun = if items == 1 { "item" } else { "items" };
println!(
"{} in {items} {noun} (on Proton Drive, trash included)",
human_bytes(size)
);
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
Ok(())
}
fn cmd_rm(path: PathBuf) -> Result<()> {
match control_request(CtlRequest::Delete {
path: path_arg(&path)?,
@@ -2942,6 +3156,24 @@ fn human_secs(secs: u64) -> String {
}
}
/// The sync engine's line in `pdfs diagnostics`: whether it runs, and which
/// folder's pass it is on.
fn sync_engine_line(sync: &pdfs_core::control::SyncEngineState) -> String {
let mut line = match (sync.running, sync.pass_folder) {
(false, _) => "STOPPED; synced folders do not sync until the daemon restarts".to_string(),
(true, None) => "idle".to_string(),
(true, Some(folder)) => format!(
"pass on folder {folder} for {}, last progress {} ago",
human_secs(sync.pass_age_secs),
human_secs(sync.pass_idle_secs),
),
};
if sync.panics > 0 {
line.push_str(&format!("; {} pass(es) panicked", sync.panics));
}
line
}
fn cmd_diagnostics() -> Result<()> {
let response = control_request(CtlRequest::Diagnostics)?;
if emit_json(&response)? {
@@ -2977,6 +3209,9 @@ fn cmd_diagnostics() -> Result<()> {
human_secs(worker.age_secs)
);
}
if let Some(sync) = &d.sync {
println!("sync engine: {}", sync_engine_line(sync));
}
if d.inflight.is_empty() {
println!("in-flight requests: none");
} else {
@@ -3215,6 +3450,79 @@ fn cmd_sync_limit(up: Option<u64>, down: Option<u64>) -> Result<()> {
})?)
}
/// How long the oldest write may wait to upload before `diagnose` calls it
/// out. Under a day it is a slow link or a big file; past it, something is
/// holding the queue.
const STAGED_STALE_SECS: u64 = 86_400;
/// `diagnose` findings for the queue and the pause, from the `Status` reply.
/// `paused_for` is how long a timed pause still runs, in seconds.
fn sync_health(
failing_ops: u64,
failing_error: Option<&str>,
staged_bytes: u64,
staged_oldest_secs: u64,
paused: bool,
paused_for: Option<i64>,
) -> Vec<(DiagnoseLevel, &'static str, String)> {
let mut findings = Vec::new();
if failing_ops > 0 {
findings.push((
DiagnoseLevel::Warn,
" failing writes",
match failing_error {
Some(error) => format!("{failing_ops} keep failing; the last error: {error}"),
None => format!("{failing_ops} keep failing"),
},
));
}
if staged_bytes > 0 && staged_oldest_secs >= STAGED_STALE_SECS {
findings.push((
DiagnoseLevel::Warn,
" oldest unsent write",
format!(
"{} old, {} only on this computer",
human_secs(staged_oldest_secs),
human_bytes(staged_bytes)
),
));
}
if paused {
findings.push((
DiagnoseLevel::Warn,
" sync",
match paused_for {
Some(left) if left > 0 => {
format!("paused for another {}", human_secs(left as u64))
}
_ => "paused until resumed (`pdfs sync resume`)".to_string(),
},
));
}
findings
}
/// `diagnose` findings for the synced folders that need someone to look.
fn folder_health(
folders: &[pdfs_core::control::SyncFolderInfo],
) -> Vec<(DiagnoseLevel, String, String)> {
folders
.iter()
.filter_map(|folder| {
let what = match folder.state.as_str() {
"error" => "sync error; `pdfs sync list` and the activity feed say more",
"conflict" => "conflict copies to look at",
_ => return None,
};
Some((
DiagnoseLevel::Warn,
format!(" synced folder [{}]", folder.id),
format!("{}: {what}", folder.local_path),
))
})
.collect()
}
fn unix_now() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
@@ -3519,10 +3827,13 @@ fn cmd_diagnose() -> Result<()> {
);
println!("\nAccount");
let mut keys_unlocked = false;
match pdfs_core::auth::load() {
Ok(session) => {
report.finding(DiagnoseLevel::Ok, " keyring session", session.username);
match key_state_name() {
let key_state = key_state_name();
keys_unlocked = key_state != "locked";
match key_state {
"locked" => report.finding(
DiagnoseLevel::Fail,
" account keys",
@@ -3548,20 +3859,26 @@ fn cmd_diagnose() -> Result<()> {
let socket_exists = socket.exists();
let socket_kind_ok =
std::fs::symlink_metadata(&socket).is_ok_and(|m| m.file_type().is_socket());
// A stale socket file with no listener is the common failure, so the reply
// is what decides this rather than the file's existence.
let status = control_request(CtlRequest::Status);
report.finding(
if socket_exists && !socket_kind_ok {
DiagnoseLevel::Fail
} else if socket_exists {
} else if socket_exists && status.is_ok() {
DiagnoseLevel::Ok
} else {
DiagnoseLevel::Warn
},
" control socket",
socket.display().to_string(),
if socket_exists && socket_kind_ok && status.is_err() {
format!("{} (left over; nothing listens on it)", socket.display())
} else {
socket.display().to_string()
},
);
// A stale socket file with no listener is the common failure, so the reply
// is what decides this rather than the file's existence.
match control_request(CtlRequest::Status) {
let mut daemon_responding = false;
match status {
Ok(CtlResponse::Status {
mountpoint,
online,
@@ -3569,9 +3886,16 @@ fn cmd_diagnose() -> Result<()> {
tokens_unsaved,
pending_uploads,
pending_changes,
failing_ops,
failing_error,
staged_bytes,
staged_oldest_secs,
paused,
paused_until,
version,
..
}) => {
daemon_responding = true;
report.finding(DiagnoseLevel::Ok, " daemon responding", "");
if daemon_outdated(&version) {
report.finding(
@@ -3642,6 +3966,27 @@ fn cmd_diagnose() -> Result<()> {
format!("{pending} waiting to upload")
},
);
// The rest of the daemon's state that the report used to drop:
// writes that keep failing, a pause, a folder in error. Each of
// them stops changes from reaching Drive while everything above
// reads as fine (`docs/BUGS.md` B268).
for (level, check, detail) in sync_health(
failing_ops,
failing_error.as_deref(),
staged_bytes,
staged_oldest_secs,
paused,
paused_until.map(|until| until - unix_now()),
) {
report.finding(level, check, detail);
}
if let Ok(CtlResponse::SyncFolders { items }) =
control_request(CtlRequest::ListSyncFolders)
{
for (level, check, detail) in folder_health(&items) {
report.finding(level, &check, detail);
}
}
match await_integrity_check() {
Ok(CtlResponse::CacheReport {
integrity_problems,
@@ -3698,7 +4043,23 @@ fn cmd_diagnose() -> Result<()> {
" daemon responding",
format!("stopped or unreachable: {e:#}"),
);
println!(" (start it with `systemctl --user start proton-drive.service`)");
match pdfs_core::mountfail::read(&dirs.state_dir()) {
// A daemon whose mount keeps failing never opens the socket,
// so this is the only place its reason shows (B263).
Some(failure) => report.finding(
DiagnoseLevel::Fail,
" mount",
format!(
"failing for {}, {} attempt(s): {}",
human_secs(unix_now().saturating_sub(failure.since).max(0) as u64),
failure.attempts,
failure.error
),
),
None => println!(
" (start it with `systemctl --user start proton-drive.service`)"
),
}
}
}
@@ -3713,6 +4074,14 @@ fn cmd_diagnose() -> Result<()> {
mountpoint.display().to_string(),
);
// The daemon opens its control socket only once My Files is mounted, so a
// daemon that answers has the keys it needs. One that does not may be stuck
// on a key that does not open (#40); only then is Proton asked.
if keys_unlocked && !daemon_responding {
println!("\nKeys");
diagnose_keys(&mut report);
}
if report.failures == 0 {
if report.warnings == 0 {
println!("\nNo problems found.");
@@ -3732,6 +4101,43 @@ fn cmd_diagnose() -> Result<()> {
}
}
/// Ask Proton which account and address keys open, and which of them the My
/// Files share needs. The findings hold key IDs and no secrets.
fn diagnose_keys(report: &mut DiagnoseReport) {
use pdfs_core::keycheck::{self, Level};
let checked = tokio::runtime::Runtime::new()
.map_err(anyhow::Error::from)
.and_then(|rt| {
rt.block_on(async {
tokio::time::timeout(std::time::Duration::from_secs(30), keycheck::run())
.await
.map_err(|_| anyhow!("Proton did not answer within 30 seconds"))?
.map_err(anyhow::Error::from)
})
});
let (keys, share) = match checked {
Ok(checked) => checked,
Err(e) => {
report.finding(DiagnoseLevel::Warn, " key check", format!("{e:#}"));
return;
}
};
let findings = keycheck::findings(&keys, &share);
let any_failed = findings.iter().any(|f| f.level != Level::Ok);
for finding in findings {
let level = match finding.level {
Level::Ok => DiagnoseLevel::Ok,
Level::Warn => DiagnoseLevel::Warn,
Level::Fail => DiagnoseLevel::Fail,
};
report.finding(level, &format!(" {}", finding.label), finding.detail);
}
if any_failed {
println!(" (key IDs are not secret; include this section when you report it)");
}
}
/// Send one request to the running mount daemon's control socket and read its
/// reply. Errors if no daemon is listening.
fn control_request(req: CtlRequest) -> Result<CtlResponse> {
@@ -3778,6 +4184,50 @@ mod pause_tests {
mod diagnose_tests {
use super::*;
/// A healthy queue says nothing; writes that keep failing, an old unsent
/// write and a pause each say so (`docs/BUGS.md` B268).
#[test]
fn a_stuck_queue_or_a_pause_is_reported() {
assert!(sync_health(0, None, 0, 0, false, None).is_empty());
assert!(sync_health(0, None, 4096, STAGED_STALE_SECS - 1, false, None).is_empty());
let checks: Vec<&str> = sync_health(
2,
Some("upload a.txt: quota"),
4096,
STAGED_STALE_SECS,
true,
Some(600),
)
.into_iter()
.map(|(level, check, _)| {
assert!(matches!(level, DiagnoseLevel::Warn));
check.trim()
})
.collect();
assert_eq!(checks, ["failing writes", "oldest unsent write", "sync"]);
}
/// Issue #53: a folder that would not sync, and `diagnose` found nothing.
#[test]
fn a_synced_folder_in_error_is_reported() {
let folder = |id: i64, state: &str| pdfs_core::control::SyncFolderInfo {
id,
local_path: format!("/home/u/f{id}"),
remote_uid: "vol~link".into(),
mode: "mirror".into(),
state: state.into(),
last_sync: 0,
pending_mode: None,
progress: None,
paused: false,
};
let findings =
folder_health(&[folder(1, "idle"), folder(2, "error"), folder(3, "syncing")]);
assert_eq!(findings.len(), 1);
assert_eq!(findings[0].1.trim(), "synced folder [2]");
assert!(findings[0].2.starts_with("/home/u/f2: sync error"));
}
#[test]
fn finding_severity_is_counted_centrally() {
let mut report = DiagnoseReport::default();
@@ -3813,6 +4263,15 @@ mod diagnose_tests {
#[cfg(test)]
mod login_wait_tests {
use super::*;
/// The mount retry backs off from 5 s to a ceiling of minutes, rather than
/// asking Proton every 5 s forever (`docs/BUGS.md` B263).
#[test]
fn a_failing_mount_backs_off_to_a_ceiling() {
let secs: Vec<u64> = (1..=8).map(|n| mount_retry_delay(n).as_secs()).collect();
assert_eq!(secs, [5, 10, 20, 40, 80, 120, 120, 120]);
assert_eq!(mount_retry_delay(u32::MAX).as_secs(), 120);
}
use std::time::Duration;
#[test]
@@ -3906,6 +4365,13 @@ mod login_wait_tests {
assert_eq!(slept, 0);
}
#[test]
fn a_link_flag_pair_keeps_removes_or_sets() {
assert_eq!(link_change::<i64>(None, false), LinkChange::Keep);
assert_eq!(link_change::<i64>(None, true), LinkChange::Remove);
assert_eq!(link_change(Some(5), false), LinkChange::Set(5));
}
#[test]
fn epoch_seconds_format_as_utc_calendar_dates() {
assert_eq!(format_epoch_utc(0), "1970-01-01");
+130 -2
View File
@@ -344,6 +344,17 @@ pub enum Request {
/// or a mirror synced folder), and the move is still made on Proton Drive
/// without transferring content. Replies with [`Response::Ok`].
Move { path: String, new_parent: String },
/// Copy a file or folder into a folder on Proton Drive, without
/// downloading or uploading it. Both `path` and `new_parent` are
/// mountpoint-relative, or absolute paths inside My files. A name already
/// taken in `new_parent` gets the next free ` (n)` suffix. Replies with
/// [`Response::Ok`].
Copy { path: String, new_parent: String },
/// Ask Proton Drive how much the folder at mountpoint-relative `path`
/// holds. Replies with [`Response::FolderSize`].
FolderSize { path: String },
/// The by-uid twin of [`Request::FolderSize`].
FolderSizeByUid { uid: String },
/// Trash a file or folder. `path` is mountpoint-relative. Replies with
/// [`Response::Ok`].
Delete { path: String },
@@ -469,6 +480,11 @@ pub enum Request {
/// Pause or resume one mirror folder's reconcile, independent of the
/// global pause. Persisted until resumed. Replies with [`Response::Ok`].
SetSyncFolderPaused { id: i64, paused: bool },
/// Forget what a mirror folder synced before, so its next pass compares
/// both sides afresh: what is on one side only is copied to the other, and
/// a file that differs keeps both versions. Nothing is deleted. Replies
/// with [`Response::Ok`].
ResetSyncFolder { id: i64 },
/// Force a reconcile pass: one folder by id, or all when `id` is `None`.
/// Replies with [`Response::Ok`].
SyncNow { id: Option<i64> },
@@ -596,6 +612,24 @@ pub enum Request {
RemovePublicLink { path: String, id: String },
/// The by-uid twin of [`Request::RemovePublicLink`].
RemovePublicLinkByUid { uid: String, id: String },
/// Change the public link `id` on the node at `path`, keeping its URL.
/// `role` is "viewer" or "editor", or `None` to keep the current one.
/// Replies with [`Response::PublicLink`].
UpdatePublicLink {
path: String,
id: String,
role: Option<String>,
password: LinkChange<String>,
expires: LinkChange<i64>,
},
/// The by-uid twin of [`Request::UpdatePublicLink`].
UpdatePublicLinkByUid {
uid: String,
id: String,
role: Option<String>,
password: LinkChange<String>,
expires: LinkChange<i64>,
},
/// Stop sharing the node at `path`: remove its public link, every pending
/// invitation and every member. Each removal is attempted even if an
/// earlier one fails. Replies with [`Response::Ok`], or an error naming
@@ -1075,6 +1109,18 @@ pub struct PublicLinkInfo {
pub created: i64,
}
/// What [`Request::UpdatePublicLink`] does with one option of a link: its
/// custom password or its expiry.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub enum LinkChange<T> {
/// Leave the option as it is.
Keep,
/// Remove the option: no custom password, or no expiry.
Remove,
/// Set the option to this value.
Set(T),
}
/// One invitation addressed to me (in [`Response::Invitations`]).
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct InvitationInfo {
@@ -1169,6 +1215,7 @@ pub enum ActivityKind {
Sync,
Rename,
Move,
Copy,
CreateFolder,
Trash,
Restore,
@@ -1933,6 +1980,29 @@ pub struct Diagnostics {
pub control_handler_limit: usize,
/// Queued mutations not yet drained.
pub pending_ops: u64,
/// The folder-sync engine. `None` from a daemon that predates the field.
#[serde(default)]
pub sync: Option<SyncEngineState>,
}
/// What the folder-sync engine is doing, for [`Diagnostics`].
///
/// The engine is one thread that runs every synced folder's passes in turn.
/// Dead or stuck, it looked exactly like an idle one: workers idle, nothing
/// queued, nothing in the journal (`docs/BUGS.md` B259).
#[derive(Serialize, Deserialize, Debug, Clone, Default)]
pub struct SyncEngineState {
/// Whether the engine thread is running. Once it has ended, nothing syncs
/// until the daemon restarts.
pub running: bool,
/// The synced folder whose pass is running, if one is.
pub pass_folder: Option<i64>,
/// How long that pass has been running.
pub pass_age_secs: u64,
/// How long since that pass last reported progress.
pub pass_idle_secs: u64,
/// Passes that ended in a panic since the daemon started.
pub panics: u64,
}
/// The daemon's reply to a [`Request`].
@@ -2185,9 +2255,14 @@ pub enum Response {
entries: Vec<ShareEntry>,
link: Option<PublicLinkInfo>,
},
/// A freshly created public link, carrying the shareable URL (reply to
/// [`Request::CreatePublicLink`]).
/// A freshly created or changed public link, carrying the shareable URL
/// (reply to [`Request::CreatePublicLink`] and
/// [`Request::UpdatePublicLink`]).
PublicLink { link: PublicLinkInfo },
/// What a folder holds on Proton Drive (reply to [`Request::FolderSize`]):
/// the bytes on storage and the number of items below it, folders and
/// trashed items included.
FolderSize { size: u64, items: u64 },
/// Invitations addressed to me (reply to [`Request::ListInvitations`]).
Invitations { items: Vec<InvitationInfo> },
/// Saved public links (reply to [`Request::ListBookmarks`]).
@@ -2687,6 +2762,10 @@ mod tests {
path: "a/b.txt".into(),
new_parent: "d".into(),
},
Request::Copy {
path: "a/b.txt".into(),
new_parent: "d".into(),
},
Request::Delete {
path: "a/b.txt".into(),
},
@@ -3158,6 +3237,7 @@ mod tests {
id: 3,
paused: true,
},
Request::ResetSyncFolder { id: 3 },
Request::SetBandwidthLimits {
upload: 1_000_000,
download: 0,
@@ -3247,6 +3327,24 @@ mod tests {
uid: "vol~link".into(),
id: "url-1".into(),
},
Request::UpdatePublicLink {
path: "a/b".into(),
id: "url-1".into(),
role: Some("editor".into()),
password: LinkChange::Set("pw".into()),
expires: LinkChange::Remove,
},
Request::UpdatePublicLinkByUid {
uid: "vol~link".into(),
id: "url-1".into(),
role: None,
password: LinkChange::Keep,
expires: LinkChange::Set(1_700_000_000),
},
Request::FolderSize { path: "a".into() },
Request::FolderSizeByUid {
uid: "vol~link".into(),
},
Request::ListSharedByMe,
Request::ListActivity { limit: 100 },
Request::ListSharedWithMe,
@@ -3621,6 +3719,16 @@ mod tests {
},
"RemovePublicLinkByUid",
),
(
Request::UpdatePublicLinkByUid {
uid: "vol~link".into(),
id: "link".into(),
role: None,
password: LinkChange::Keep,
expires: LinkChange::Keep,
},
"UpdatePublicLinkByUid",
),
(
Request::StopSharingByUid {
uid: "vol~link".into(),
@@ -3646,6 +3754,7 @@ mod tests {
| Request::RemoveShareEntryByUid { uid, .. }
| Request::CreatePublicLinkByUid { uid, .. }
| Request::RemovePublicLinkByUid { uid, .. }
| Request::UpdatePublicLinkByUid { uid, .. }
| Request::StopSharingByUid { uid } => uid,
other => panic!("decoded into the wrong dispatch variant: {other:?}"),
};
@@ -3654,6 +3763,25 @@ mod tests {
}
}
#[test]
fn a_link_update_names_what_it_keeps_removes_and_sets() {
let request = Request::UpdatePublicLink {
path: "a/b".into(),
id: "url-1".into(),
role: None,
password: LinkChange::Keep,
expires: LinkChange::Set(1_700_000_000),
};
assert_eq!(
serde_json::to_string(&request).unwrap(),
r#"{"UpdatePublicLink":{"path":"a/b","id":"url-1","role":null,"password":"Keep","expires":{"Set":1700000000}}}"#
);
assert_eq!(
serde_json::to_string(&LinkChange::<String>::Remove).unwrap(),
r#""Remove""#
);
}
#[test]
fn open_file_accepts_a_request_from_a_client_predating_the_uid_field() {
let legacy = r#"{"OpenFile":{"path":"Documents/ONBOARDING.md"}}"#;
+52 -16
View File
@@ -160,13 +160,14 @@ impl CoreError {
/// telling the user their connection is down beats telling them the
/// server refused. The SDK's other failures say what went wrong instead
/// (see [`proton_kind`]).
///
/// Callers also hand it this crate's [`Error`] from the cache and the
/// database, which is not a transport error at all. Taking those for
/// "offline" told the user their connection was down when a pin failed on
/// the database (`docs/BUGS.md` B266), so the source chain is searched for
/// either type before falling back.
pub fn from_api(e: &(dyn std::error::Error + 'static), what: &str) -> Self {
let kind = match e.downcast_ref::<proton_sdk::ProtonError>() {
Some(proton) => proton_kind(proton),
// Not the SDK's at all: a timeout around the call, or the socket.
None => ErrorKind::Offline,
};
Self::new(kind, format!("{what}: {e}"))
Self::new(api_kind_of(e), format!("{what}: {e}"))
}
/// Prefix the message while keeping the classification.
@@ -208,19 +209,40 @@ impl From<serde_json::Error> for CoreError {
impl From<Error> for CoreError {
fn from(e: Error) -> Self {
let kind = match &e {
Error::NotLoggedIn
| Error::ReloginRequired
| Error::Locked
| Error::WrongMailboxPassword
| Error::HumanVerificationRequired(_) => ErrorKind::Denied,
Error::Proton(proton) => proton_kind(proton),
_ => ErrorKind::Internal,
};
Self::new(kind, e.to_string())
Self::new(core_kind(&e), e.to_string())
}
}
/// What this crate's [`Error`] means to a front-end.
fn core_kind(e: &Error) -> ErrorKind {
match e {
Error::NotLoggedIn
| Error::ReloginRequired
| Error::Locked
| Error::WrongMailboxPassword
| Error::HumanVerificationRequired(_) => ErrorKind::Denied,
Error::Proton(proton) => proton_kind(proton),
_ => ErrorKind::Internal,
}
}
/// The kind for [`CoreError::from_api`]: the first SDK or crate error in `e`'s
/// source chain decides it.
fn api_kind_of(e: &(dyn std::error::Error + 'static)) -> ErrorKind {
let mut cause = Some(e);
while let Some(err) = cause {
if let Some(proton) = err.downcast_ref::<proton_sdk::ProtonError>() {
return proton_kind(proton);
}
if let Some(core) = err.downcast_ref::<Error>() {
return core_kind(core);
}
cause = err.source();
}
// Not the SDK's at all: a timeout around the call, or the socket.
ErrorKind::Offline
}
/// What a failed SDK call means to a front-end.
///
/// Only a transport failure is the connection. The rest happened on this
@@ -414,6 +436,20 @@ mod tests {
assert_ne!(e.kind, ErrorKind::Offline);
}
/// A pin or a search that fails on the database is not the connection, so
/// it must not say "You're offline" (`docs/BUGS.md` B266). An SDK error
/// wrapped in this crate's type classifies as the SDK error does.
#[test]
fn a_database_error_handed_to_from_api_is_not_offline() {
let db = Error::Other("database is locked".into());
assert_eq!(CoreError::from_api(&db, "pin").kind, ErrorKind::Internal);
let wrapped = Error::Proton(api(C::DoesNotExist));
assert_eq!(
CoreError::from_api(&wrapped, "load node").kind,
ErrorKind::NotFound
);
}
/// `Error` is the plumbing type; converting it must classify as sharply as
/// the direct path, or the two disagree about the same failure.
#[test]
+294
View File
@@ -0,0 +1,294 @@
//! Trace the key chain that opens My Files, for `pdfs diagnose`.
//!
//! The daemon only says "missing key" when the My Files share does not open
//! (#40). This walks the chain the other way and names the link that breaks:
//! the share's passphrase is encrypted to an address key, whose token is
//! encrypted to an account key, which opens with the stored passphrase. Every
//! finding holds key IDs and flags only, never key material, so the output is
//! safe to post in an issue.
use proton_sdk::account::{AddressKeyReport, KeyReport, UserKeyReport};
use proton_sdk::ids::AddressId;
use crate::auth;
use crate::error::Result;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Level {
Ok,
Warn,
Fail,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Finding {
pub level: Level,
pub label: String,
pub detail: String,
}
/// The address and recipients of the My Files share passphrase, or why they
/// could not be read.
pub type ShareRecipients = std::result::Result<(AddressId, Vec<Option<String>>), String>;
/// Resume the stored session and ask Proton for the account's keys and the
/// My Files share. Fails when there is no session, the keys are locked, or
/// Proton cannot be reached; reading the share is allowed to fail on its own.
///
/// The session's tokens are the daemon's too. A refresh here spends the
/// refresh token the daemon holds, which it recovers from by resuming the
/// stored session (see [`auth::stored_session_is_newer`]).
pub async fn run() -> Result<(KeyReport, ShareRecipients)> {
let (client, _session) = auth::resume_client().await?;
let report = client.account().key_report().await?;
let share = client
.my_files_share_recipients()
.await
.map_err(|e| e.to_string());
Ok((report, share))
}
/// One finding per key, then one for the My Files share.
pub fn findings(report: &KeyReport, share: &ShareRecipients) -> Vec<Finding> {
let mut findings = Vec::new();
for key in &report.user_keys {
findings.push(user_key_finding(key));
}
for address in &report.addresses {
for key in &address.keys {
findings.push(address_key_finding(report, &address.email, key));
}
}
findings.push(share_finding(report, share));
findings
}
fn user_key_finding(key: &UserKeyReport) -> Finding {
let label = format!("account key {}", short(key.id.as_str()));
let (level, detail) = match (key.active, key.unlocks) {
(true, true) => (Level::Ok, "opens".to_string()),
(true, false) => (
Level::Fail,
format!(
"does not open: {}",
key.error.as_deref().unwrap_or("unknown")
),
),
(false, true) => (Level::Ok, "inactive, opens with your password".to_string()),
(false, false) => (Level::Ok, "inactive, does not open".to_string()),
};
Finding {
level,
label,
detail: with_primary(key.primary, detail),
}
}
fn address_key_finding(report: &KeyReport, email: &str, key: &AddressKeyReport) -> Finding {
let label = format!("{email} key {}", short(key.id.as_str()));
let (level, detail) = if !key.active {
(Level::Ok, "inactive, not used".to_string())
} else if key.unlocks {
(Level::Ok, "opens".to_string())
} else {
// A non-primary key matters only for what was encrypted to it; the
// share finding says whether My Files is.
let level = if key.primary {
Level::Fail
} else {
Level::Warn
};
(level, format!("does not open: {}", why_closed(report, key)))
};
Finding {
level,
label,
detail: with_primary(key.primary, detail),
}
}
/// Why an active address key does not open, named by the account key its
/// token was meant for when the token has one.
fn why_closed(report: &KeyReport, key: &AddressKeyReport) -> String {
let error = key.error.as_deref().unwrap_or("unknown");
let Some(recipients) = &key.token_recipients else {
return error.to_string();
};
let named: Vec<&str> = recipients.iter().flatten().map(String::as_str).collect();
if let Some(user_key) = named.iter().find_map(|r| report.user_key_named(r)) {
let id = short(user_key.id.as_str());
return match (user_key.active, user_key.unlocks) {
(false, true) => {
format!(
"its token is for inactive account key {id}, which opens with your password"
)
}
(false, false) => {
format!("its token is for inactive account key {id}, which does not open either")
}
(true, false) => format!("its token is for account key {id}, which does not open"),
// The token opened; the key itself refused its passphrase.
(true, true) => error.to_string(),
};
}
match named.first() {
Some(recipient) => {
format!("its token is for key {recipient}, which is none of your account keys")
}
None if recipients.is_empty() => error.to_string(),
None => "its token does not name the key it is for".to_string(),
}
}
fn share_finding(report: &KeyReport, share: &ShareRecipients) -> Finding {
let label = "My Files share".to_string();
let (level, detail) = match share {
Err(e) => (Level::Warn, format!("could not be read: {e}")),
Ok((address_id, recipients)) => {
let named: Vec<&str> = recipients.iter().flatten().map(String::as_str).collect();
match named.iter().find_map(|r| report.address_key_named(r)) {
Some((address, key)) => {
let name = format!("{} key {}", address.email, short(key.id.as_str()));
if key.unlocks {
(Level::Ok, format!("encrypted to {name}, which opens"))
} else {
(
Level::Fail,
format!("encrypted to {name}, which does not open"),
)
}
}
None => (
Level::Fail,
match named.first() {
Some(recipient) => format!(
"encrypted to key {recipient}, which is none of the keys of address {address_id}"
),
None => "its passphrase does not name the key it is for".to_string(),
},
),
}
}
};
Finding {
level,
label,
detail,
}
}
fn with_primary(primary: bool, detail: String) -> String {
if primary {
format!("primary, {detail}")
} else {
detail
}
}
/// The start of a Proton key ID: enough to tell keys apart and to find them
/// in the daemon log, which prints the whole ID.
fn short(id: &str) -> String {
match id.char_indices().nth(8) {
Some((end, _)) => format!("{}…", &id[..end]),
None => id.to_string(),
}
}
#[cfg(test)]
mod tests {
use super::*;
use proton_sdk::account::AddressReport;
use proton_sdk::ids::{AddressKeyId, UserKeyId};
fn user_key(id: &str, active: bool, unlocks: bool, pgp: &str) -> UserKeyReport {
UserKeyReport {
id: UserKeyId::new(id),
primary: active,
active,
unlocks,
error: (!unlocks).then(|| "unlock: wrong passphrase".to_string()),
key_ids: vec![pgp.to_string()],
}
}
fn address_key(id: &str, primary: bool, token_for: &str, pgp: &str) -> AddressKeyReport {
AddressKeyReport {
id: AddressKeyId::new(id),
primary,
active: true,
token_recipients: Some(vec![Some(token_for.to_string())]),
unlocks: token_for == "u1",
error: (token_for != "u1").then(|| "token: failed to decrypt: missing key".to_string()),
key_ids: vec![pgp.to_string()],
}
}
/// The account in #40, as far as its log shows: the primary address key
/// opens, an older one does not, and My Files is encrypted to the older one.
fn report(stranded_token_for: &str) -> KeyReport {
KeyReport {
user_keys: vec![
user_key("UserKeyOne-123", true, true, "u1"),
user_key("UserKeyTwo-456", false, true, "u2"),
],
addresses: vec![AddressReport {
id: AddressId::new("address-1"),
email: "user@proton.me".to_string(),
keys: vec![
address_key("AddrKeyNew-789", true, "u1", "a1"),
address_key("AddrKeyOld-000", false, stranded_token_for, "a2"),
],
}],
}
}
#[test]
fn a_stranded_address_key_is_traced_to_the_inactive_account_key() {
let report = report("u2");
let share = Ok((AddressId::new("address-1"), vec![Some("a2".to_string())]));
let findings = findings(&report, &share);
assert_eq!(findings[0].detail, "primary, opens");
assert_eq!(findings[1].detail, "inactive, opens with your password");
assert_eq!(findings[2].level, Level::Ok);
assert_eq!(findings[3].level, Level::Warn);
assert_eq!(findings[3].label, "user@proton.me key AddrKeyO…");
assert_eq!(
findings[3].detail,
"does not open: its token is for inactive account key UserKeyT…, which opens with your password"
);
assert_eq!(findings[4].level, Level::Fail);
assert_eq!(
findings[4].detail,
"encrypted to user@proton.me key AddrKeyO…, which does not open"
);
}
#[test]
fn a_token_for_a_key_the_account_does_not_list_says_so() {
let report = report("ffff");
let detail = &findings(&report, &Err("offline".to_string()))[3].detail;
assert_eq!(
detail,
"does not open: its token is for key ffff, which is none of your account keys"
);
}
#[test]
fn a_share_that_opens_and_one_that_cannot_be_read() {
let report = report("u2");
let open = Ok((AddressId::new("address-1"), vec![Some("a1".to_string())]));
let last = findings(&report, &open).pop().unwrap();
assert_eq!(last.level, Level::Ok);
assert_eq!(
last.detail,
"encrypted to user@proton.me key AddrKeyN…, which opens"
);
let last = findings(&report, &Err("offline".to_string()))
.pop()
.unwrap();
assert_eq!(last.level, Level::Warn);
assert_eq!(last.detail, "could not be read: offline");
}
}
+2
View File
@@ -12,8 +12,10 @@ pub mod db;
pub mod error;
pub mod exiftime;
pub mod kernelkey;
pub mod keycheck;
pub mod localindex;
pub mod menu;
pub mod mountfail;
pub mod mounts;
pub mod opener;
pub mod places;
+75
View File
@@ -0,0 +1,75 @@
//! Why the daemon's mount keeps failing, for the front ends to read.
//!
//! The control socket opens only once a mount has succeeded, so a daemon whose
//! mount fails on every try had no way to say why. The app and the tray showed
//! "Connecting…" for days while the reason sat in the journal (`docs/BUGS.md`
//! B263). The daemon now writes its last failure to a small file in the state
//! directory, next to the control socket, and removes it once a mount is up.
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
/// The file's name inside the state directory.
const FILE_NAME: &str = "mount-failure.json";
/// The last failed mount attempt.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
pub struct MountFailure {
/// The error, in English, as the journal has it.
pub error: String,
/// Unix seconds of the first failure in this run of failures.
pub since: i64,
/// Failed attempts in a row.
pub attempts: u32,
/// Unix seconds of the next attempt.
pub next_attempt: i64,
}
/// Where the failure is kept for the state directory `state_dir`.
pub fn path(state_dir: &Path) -> PathBuf {
state_dir.join(FILE_NAME)
}
/// Record `failure`, replacing the one before it. Written to a temporary file
/// and renamed, so a reader never sees half of it.
pub fn record(state_dir: &Path, failure: &MountFailure) -> std::io::Result<()> {
let json = serde_json::to_vec(failure).map_err(std::io::Error::other)?;
let tmp = state_dir.join(format!("{FILE_NAME}.tmp"));
std::fs::write(&tmp, json)?;
std::fs::rename(&tmp, path(state_dir))
}
/// Forget the failure: a mount is up, or a new daemon has started.
pub fn clear(state_dir: &Path) {
let _ = std::fs::remove_file(path(state_dir));
}
/// The recorded failure, if there is one and it can be read.
pub fn read(state_dir: &Path) -> Option<MountFailure> {
let bytes = std::fs::read(path(state_dir)).ok()?;
serde_json::from_slice(&bytes).ok()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_recorded_failure_reads_back_until_cleared() {
let dir = std::env::temp_dir().join(format!("pdfs-mountfail-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
assert_eq!(read(&dir), None);
let failure = MountFailure {
error: "fetch My Files root: missing key".into(),
since: 1_700_000_000,
attempts: 3,
next_attempt: 1_700_000_040,
};
record(&dir, &failure).unwrap();
assert_eq!(read(&dir), Some(failure));
clear(&dir);
assert_eq!(read(&dir), None);
let _ = std::fs::remove_dir_all(&dir);
}
}
+86 -2
View File
@@ -29,6 +29,17 @@ use super::{Core, count_noun, human_bytes, human_duration, parse_uid};
const MAX_CONTROL_REQUEST_BYTES: u64 = 1024 * 1024;
const CONTROL_READ_TIMEOUT: Duration = Duration::from_secs(10);
/// Answer a [`CtlRequest::FolderSize`].
fn folder_size_reply(size: CoreResult<proton_drive_rs::FolderSize>) -> CtlResponse {
match size {
Ok(size) => CtlResponse::FolderSize {
size: size.size,
items: size.descendants,
},
Err(e) => CtlResponse::error(e),
}
}
/// Answer a [`CtlRequest::StopSharing`]: log it, and turn a partial removal
/// into an error that names what is still shared.
fn stop_sharing_reply(
@@ -377,6 +388,7 @@ fn touched_topics(request: &CtlRequest) -> &'static [Topic] {
| R::RemoveSyncFolder { .. }
| R::SetSyncFolderMode { .. }
| R::SetSyncFolderPaused { .. }
| R::ResetSyncFolder { .. }
| R::RestoreSyncFolders { .. }
| R::RestoreDeviceFolders { .. } => &[Topic::Locations, Topic::Devices],
R::RetryPendingOp { .. } => &[Topic::Queue],
@@ -392,6 +404,8 @@ fn touched_topics(request: &CtlRequest) -> &'static [Topic] {
| R::CreatePublicLinkByUid { .. }
| R::RemovePublicLink { .. }
| R::RemovePublicLinkByUid { .. }
| R::UpdatePublicLink { .. }
| R::UpdatePublicLinkByUid { .. }
| R::StopSharing { .. }
| R::StopSharingByUid { .. } => &[Topic::Shares, Topic::Files],
R::LeaveShared { .. }
@@ -1063,6 +1077,36 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
(Err(e), _) | (_, Err(e)) => CtlResponse::error(e),
}
}
Ok(CtlRequest::Copy { path, new_parent }) => {
match (
rel_to_mount(mountpoint, &path),
rel_to_mount(mountpoint, &new_parent),
) {
(Ok(rel), Ok(parent_rel)) => match core.copy_to(&rel, &parent_rel) {
Ok(name) => {
let dest = if new_parent.is_empty() {
"My files".to_string()
} else {
new_parent.clone()
};
core.log_activity(ActivityKind::Copy, &name, format!("to {dest}"), true);
CtlResponse::Ok {
message: format!("copied to {name}"),
}
}
Err(e) => {
core.log_activity(ActivityKind::Copy, &path, &e, false);
CtlResponse::error(e)
}
},
(Err(e), _) | (_, Err(e)) => CtlResponse::error(e),
}
}
Ok(CtlRequest::FolderSize { path }) => match rel_to_mount(mountpoint, &path) {
Ok(rel) => folder_size_reply(core.folder_size(&rel)),
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::FolderSizeByUid { uid }) => folder_size_reply(core.folder_size_by_uid(&uid)),
Ok(CtlRequest::Delete { path }) => match route_node(core, mountpoint, &path) {
Ok((core, rel)) => match core.delete(&rel) {
Ok(name) => {
@@ -1389,6 +1433,13 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
Err(e) => CtlResponse::error(e),
}
}
Ok(CtlRequest::ResetSyncFolder { id }) => match core.reset_sync_folder(id) {
Ok(message) => {
core.log_activity(ActivityKind::Sync, &message, "", true);
CtlResponse::Ok { message }
}
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::SyncNow { id }) => {
core.sync_now(id);
CtlResponse::Ok {
@@ -1475,9 +1526,17 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
let core = core.clone();
std::thread::spawn(move || {
match core.restore_sync_folders(&items, device.as_deref()) {
Ok(message) => {
Ok(outcome) => {
core.touch_profile();
core.log_activity(ActivityKind::Download, &message, "", true);
if !outcome.complete() {
warn!(skipped = ?outcome.skipped, "restore skipped folders");
}
core.log_activity(
ActivityKind::Download,
outcome.summary(),
outcome.skipped.join("; "),
outcome.complete(),
);
}
Err(e) => {
warn!(error = %e, "restore failed");
@@ -1630,6 +1689,31 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
CtlResponse::error(e)
}
},
Ok(CtlRequest::UpdatePublicLink {
path,
id,
role,
password,
expires,
}) => match rel_to_mount(mountpoint, &path) {
Ok(rel) => {
match core.update_public_link(&rel, &id, role.as_deref(), password, expires) {
Ok(link) => CtlResponse::PublicLink { link },
Err(e) => CtlResponse::error(e),
}
}
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::UpdatePublicLinkByUid {
uid,
id,
role,
password,
expires,
}) => match core.update_public_link_by_uid(&uid, &id, role.as_deref(), password, expires) {
Ok(link) => CtlResponse::PublicLink { link },
Err(e) => CtlResponse::error(e),
},
Ok(CtlRequest::RemovePublicLink { path, id }) => match rel_to_mount(mountpoint, &path) {
Ok(rel) => match core.remove_public_link(&rel, &id) {
Ok(()) => {
+53 -3
View File
@@ -989,9 +989,15 @@ impl Core {
.db
.sync_entries(id)
.map_err(|e| SwitchBlocked::Failed(format!("db: {e:?}")))?;
if let Some(reason) =
sync::mirror_tree_unsynced(&local, &baseline, &self.ignore_rules(&local))
{
let rules = self.ignore_rules(&local);
// Emptied before any pass saw it, the folder is still `idle`, and
// the check below would refuse the switch for good. The push pass
// of a queued switch forgets those paths instead (`docs/BUGS.md`
// B255).
if sync::mirror_tree_gone(&local, &baseline, &rules) {
return Err(SwitchBlocked::NotNow);
}
if let Some(reason) = sync::mirror_tree_unsynced(&local, &baseline, &rules) {
return Err(SwitchBlocked::Failed(format!(
"going online-only would delete what is only on this computer: {reason}"
)));
@@ -1080,6 +1086,50 @@ impl Core {
}
}
/// Forget what a mirror folder synced before, so its next pass compares the
/// two sides as on the first sync: what one side has alone is copied to the
/// other, and a file that differs is kept twice as a conflict. Nothing is
/// deleted on either side.
///
/// This is the way out for a folder whose passes refuse to run because its
/// whole local copy, or its whole copy on Drive, is gone (#46). The baseline
/// is what makes that read as "delete everything", and it is what this
/// drops. A queued switch to on-demand is dropped with it: the push pass
/// that switch runs reads an unrecorded local file as new and would upload
/// a second copy of every file Drive already has.
pub(crate) fn reset_sync_folder(&self, id: i64) -> CoreResult<String> {
let lock = self.sync_lock(id);
let Some(_guard) = lock.try_lock() else {
return Err(CoreError::conflict(
"this folder is syncing; try again when the pass ends",
));
};
let folder = self
.db
.sync_folder_get(id)
.map_err(|e| CoreError::internal(format!("db: {e:?}")))?
.ok_or_else(|| CoreError::not_found(format!("no synced folder with id {id}")))?;
if folder.mode != "mirror" {
return Err(CoreError::invalid(
"an online-only folder keeps nothing on this computer to sync from scratch",
));
}
self.db
.sync_entries_clear(id)
.map_err(|e| CoreError::internal(format!("db: {e:?}")))?;
if folder.pending_mode.as_deref() == Some("ondemand") {
self.db
.sync_folder_set_pending_mode(id, None)
.map_err(|e| CoreError::internal(format!("db: {e:?}")))?;
}
let _ = self.sync_tx.send(sync::SyncMsg::Reconcile(id));
info!(id, path = %folder.local_path, "sync folder reset");
Ok(format!(
"{} will sync from scratch",
base_name(&folder.local_path)
))
}
/// Spawn a secondary FUSE session rooted at `root` over `local` from a
/// [`Core::fork_state`] sibling. Clears any stale kernel mount first (a crashed run can leave
/// one, which would fail the fresh mount with EBUSY).
+1
View File
@@ -163,6 +163,7 @@ impl Core {
control_handlers: super::control::active_handlers(),
control_handler_limit: super::control::handler_limit(),
pending_ops: self.pending_ops_quickly(),
sync: Some(self.sync_engine.report()),
}
}
+36 -2
View File
@@ -28,8 +28,8 @@ use proton_drive_rs::proton_sdk::account::Quota;
use proton_drive_rs::proton_sdk::error::Result;
use proton_drive_rs::proton_sdk::ids::{DriveEventId, NodeUid};
use proton_drive_rs::{
Device, DeviceType, DriveEvent, DriveEventScopeId, Node, NodeMoveItem, ProtonDriveClient,
RevisionReader, Thumbnail,
AvailableNameTarget, Device, DeviceType, DriveEvent, DriveEventScopeId, Node, NodeMoveItem,
ProtonDriveClient, RevisionReader, Thumbnail,
};
/// One outcome per node of a batch request; see [`ProtonDriveClient::trash_nodes`].
@@ -138,6 +138,22 @@ pub(crate) trait DriveApi: Send + Sync {
async fn move_node(&self, uid: &NodeUid, new_parent: &NodeUid) -> Result<()>;
/// See [`ProtonDriveClient::copy_node`].
async fn copy_node(
&self,
uid: &NodeUid,
new_parent: &NodeUid,
name: Option<&str>,
) -> Result<NodeUid>;
/// See [`ProtonDriveClient::get_available_name`].
async fn get_available_name(
&self,
parent_uid: &NodeUid,
name: &str,
target: AvailableNameTarget,
) -> Result<String>;
fn move_nodes_streaming<'a>(
&'a self,
items: Vec<NodeMoveItem>,
@@ -448,6 +464,24 @@ impl DriveApi for ProtonDriveClient {
ProtonDriveClient::move_node(self, uid, new_parent).await
}
async fn copy_node(
&self,
uid: &NodeUid,
new_parent: &NodeUid,
name: Option<&str>,
) -> Result<NodeUid> {
ProtonDriveClient::copy_node(self, uid, new_parent, name).await
}
async fn get_available_name(
&self,
parent_uid: &NodeUid,
name: &str,
target: AvailableNameTarget,
) -> Result<String> {
ProtonDriveClient::get_available_name(self, parent_uid, name, target).await
}
fn move_nodes_streaming<'a>(
&'a self,
items: Vec<NodeMoveItem>,
+117 -64
View File
@@ -430,58 +430,10 @@ impl Filesystem for ProtonFs {
reply.error(error);
return;
}
let fh = fh.0;
// Stage the bytes straight into the scratch file (no base download): only
// the untouched remainder is pulled from the remote, and only at commit.
let file = {
let st = self.core.state();
match st.handles.get(&fh).and_then(|&i| st.active_writes.get(&i)) {
Some(aw) => aw.file.clone(),
None => {
reply.error(Errno::EBADF);
return;
}
}
};
if let Err(e) = file.write_all_at(data, offset) {
error!(ino = ino.0, fh, error = %e, "scratch write failed");
// A full local disk is not a broken file: answer `ENOSPC` so the
// application can say so (and so `cp`/editors take their own
// out-of-space path), and free what the cache is holding — the
// scratch file the next write goes to shares that filesystem.
if e.raw_os_error() == Some(libc::ENOSPC) {
self.core.cache.emergency_evict();
reply.error(Errno::ENOSPC);
} else {
reply.error(Errno::EIO);
}
return;
match self.stage_write(ino.0, fh.0, offset, data) {
Ok(()) => reply.written(data.len() as u32),
Err(error) => reply.error(error),
}
let new_len = {
let mut st = self.core.state();
let Some(aw) = st
.handles
.get(&fh)
.copied()
.and_then(|i| st.active_writes.get_mut(&i))
else {
reply.error(Errno::EBADF);
return;
};
aw.record_write(offset, offset + data.len() as u64);
let len = aw.len;
st.set_size(ino.0, len);
len
};
debug!(
ino = ino.0,
fh,
offset,
len = data.len(),
new_len,
"staged write"
);
reply.written(data.len() as u32);
}
#[allow(clippy::too_many_arguments)]
@@ -719,7 +671,7 @@ impl Filesystem for ProtonFs {
reply: ReplyEmpty,
) {
let ino = self.tree(ino);
let (handle, unlinked_uid) = {
let (handle, unlinked_uid, releasing) = {
let mut st = self.core.state();
let unlinked_uid = release_unlinked_entry(&mut st, ino.0);
let ino = st.handles.remove(&fh.0);
@@ -732,7 +684,11 @@ impl Filesystem for ProtonFs {
None
}
});
(h, unlinked_uid)
let releasing = ino.filter(|_| h.is_some());
if let Some(ino) = releasing {
st.releasing.insert(ino);
}
(h, unlinked_uid, releasing)
};
// Deliberately **not** handed to a worker, unlike the other slow
// handlers here. The kernel does not wait for a `release` reply before
@@ -749,6 +705,9 @@ impl Filesystem for ProtonFs {
// finish it (audit F1). What is left is a rename, two fsyncs, a database
// insert, and at most a bounded copy out of the local cache.
self.finish_release(handle, unlinked_uid, reply);
if let Some(ino) = releasing {
self.core.state().releasing.remove(&ino);
}
}
fn unlink(&self, _req: &Request, parent: INodeNo, name: &OsStr, reply: ReplyEmpty) {
@@ -1016,20 +975,59 @@ impl Filesystem for ProtonFs {
fn copy_file_range(
&self,
_req: &Request,
_ino_in: INodeNo,
ino_in: INodeNo,
_fh_in: FileHandle,
_offset_in: u64,
_ino_out: INodeNo,
_fh_out: FileHandle,
_offset_out: u64,
_len: u64,
_flags: CopyFileRangeFlags,
offset_in: u64,
ino_out: INodeNo,
fh_out: FileHandle,
offset_out: u64,
len: u64,
flags: CopyFileRangeFlags,
reply: ReplyWrite,
) {
// Also must stay ENOSYS: there is no server-side copy, and this is the
// errno that sends coreutils (and the kernel) down the read/write path
// instead of failing the copy outright.
reply.error(Errno::ENOSYS);
// Copied here when the source bytes are on this machine, as `write`
// would stage them, minus a FUSE round trip per page: `cp` of a cached
// 256 MiB file ran at about 100 MiB/s through read and write. Anything
// that would have to be downloaded gets `EOPNOTSUPP`, which makes the
// kernel copy that one call through read and write itself; `ENOSYS`
// would turn the op off for the whole mount. There is no server-side
// copy into a file `cp` has already created, so the copy still uploads.
//
// On the dispatch loop, like `write`, so it keeps its order with the
// reads and writes of the same handle. [`COPY_RANGE_MAX`] bounds how
// long one call holds the loop.
let (ino_in, ino_out) = (self.tree(ino_in), self.tree(ino_out));
if !flags.is_empty() || ino_in == ino_out {
reply.error(Errno::EOPNOTSUPP);
return;
}
if let Err(error) = self.core.require_writable(ino_out.0) {
reply.error(error);
return;
}
let bytes = match self
.core
.read_local(ino_in.0, offset_in, len.min(COPY_RANGE_MAX))
{
Ok(Some(bytes)) => bytes,
Ok(None) => {
debug!(ino_in = ino_in.0, offset_in, "copy range not local");
reply.error(Errno::EOPNOTSUPP);
return;
}
Err(error) => {
reply.error(error);
return;
}
};
if bytes.is_empty() {
reply.written(0);
return;
}
match self.stage_write(ino_out.0, fh_out.0, offset_out, &bytes) {
Ok(()) => reply.written(bytes.len() as u32),
Err(error) => reply.error(error),
}
}
}
@@ -1057,6 +1055,11 @@ pub struct ProtonFs {
/// worker busy indefinitely.
const OPEN_BASE_ATTEMPTS: u32 = 3;
/// The most one `copy_file_range` call copies; the caller loops for the rest.
/// A local copy of this much takes milliseconds, which is how long it holds
/// the dispatch loop.
const COPY_RANGE_MAX: u64 = 16 * 1024 * 1024;
/// One directory's entries as `readdir` reports them: `.` and `..` first, then
/// the interned children in `children` order.
///
@@ -1431,6 +1434,14 @@ impl ProtonFs {
// (`ContentCache::clone_or_copy`).
if let Err(e) = ContentCache::clone_or_copy(&pending.path, &path) {
let _ = std::fs::remove_file(&path);
// The drain lets go of the pending entry before it
// discards the blob, so a blob that went away under the
// copy is a revision that landed or was replaced: the
// same race the re-check below catches, only earlier.
if self.core.pending_blob(&uid).as_ref() != Some(&pending.path) {
warn!(%uid, attempt, "queued revision went away while opening for write; retrying");
continue;
}
error!(%uid, source = %pending.path.display(), error = %e,
"copy queued revision into write scratch failed");
break 'install Err(Errno::EIO);
@@ -1641,6 +1652,48 @@ impl ProtonFs {
}
/// The body of [`Filesystem::create`], run off the dispatch loop.
/// Stage `data` at `offset` of the write handle `fh` on `ino`, straight into
/// its scratch file (no base download): only the untouched remainder is
/// pulled from the remote, and only at commit.
fn stage_write(&self, ino: u64, fh: u64, offset: u64, data: &[u8]) -> Result<(), Errno> {
let file = {
let st = self.core.state();
match st.handles.get(&fh).and_then(|&i| st.active_writes.get(&i)) {
Some(aw) => aw.file.clone(),
None => return Err(Errno::EBADF),
}
};
if let Err(e) = file.write_all_at(data, offset) {
error!(ino, fh, error = %e, "scratch write failed");
// A full local disk is not a broken file: answer `ENOSPC` so the
// application can say so (and so `cp`/editors take their own
// out-of-space path), and free what the cache is holding — the
// scratch file the next write goes to shares that filesystem.
if e.raw_os_error() == Some(libc::ENOSPC) {
self.core.cache.emergency_evict();
return Err(Errno::ENOSPC);
}
return Err(Errno::EIO);
}
let new_len = {
let mut st = self.core.state();
let Some(aw) = st
.handles
.get(&fh)
.copied()
.and_then(|i| st.active_writes.get_mut(&i))
else {
return Err(Errno::EBADF);
};
aw.record_write(offset, offset + data.len() as u64);
let len = aw.len;
st.set_size(ino, len);
len
};
debug!(ino, fh, offset, len = data.len(), new_len, "staged write");
Ok(())
}
fn serve_create(&self, parent: u64, name: &str, reply: ReplyCreate) {
if let Err(error) = self.core.require_writable(parent) {
reply.error(error);
+136 -4
View File
@@ -74,8 +74,8 @@ use proton_drive_rs::proton_sdk::api::ResponseCode;
use proton_drive_rs::proton_sdk::error::ProtonError;
use proton_drive_rs::proton_sdk::ids::{DriveEventId, LinkId, NodeUid, ShareId, VolumeId};
use proton_drive_rs::{
DriveEvent, DriveEventScopeId, MemberRole, Node, NodeKind, NodeMoveItem, ProtonDriveClient,
ProtonPhotosClient, SharedWithMeItem, ThumbnailType,
AvailableNameTarget, DriveEvent, DriveEventScopeId, MemberRole, Node, NodeKind, NodeMoveItem,
ProtonDriveClient, ProtonPhotosClient, SharedWithMeItem, ThumbnailType,
};
mod albums;
@@ -124,7 +124,7 @@ use mount::{
};
use reads::{BlockFlight, BlockRing, PREFETCH_BUDGET, Prefetch, ReaderSlot, STREAM_BYPASS_MIN};
use state::{Entry, Intervals, PendingRevision, State, StateGuard, WriteHandle};
pub use systemd::idle_sleep;
pub use systemd::{idle_sleep, status as notify_status};
use tracing::{debug, error, info, warn};
use transfers::{CountingWriter, JobGuard, OwnedCountingReader, TransferRegistry};
use r#virtual::{
@@ -728,6 +728,8 @@ struct Core {
/// can say what a pass is doing rather than just "syncing". An entry exists
/// only while that folder's reconcile pass is running.
sync_progress: Arc<Mutex<HashMap<i64, SyncProgress>>>,
/// Whether the folder-sync engine is alive, and which pass it is on.
sync_engine: Arc<sync::EngineHealth>,
/// Channel to the folder-sync engine: nudges it to
/// reconcile a folder, reconcile everything, or re-scan its watch set.
sync_tx: std::sync::mpsc::Sender<sync::SyncMsg>,
@@ -5206,6 +5208,133 @@ impl Core {
Ok(name)
}
/// Copy a file or folder into `new_parent_rel` on Proton Drive, which copies
/// the content there: nothing is downloaded or uploaded. Both paths are
/// mountpoint-relative. A name already taken in the folder gets the next
/// free ` (n)` suffix. Returns the copy's name.
///
/// Drive copies what it holds, so the copy is refused offline and while the
/// mount still owes Drive part of it: a node or folder only queued locally,
/// a file with a queued upload or unsaved writes, or — for a folder, whose
/// queued changes could sit anywhere below it — any queued op at all.
fn copy_to(&self, rel: &Path, new_parent_rel: &Path) -> CoreResult<String> {
let (ino, uid) = self.resolve(rel)?;
let (pino, new_parent_uid) = self
.resolve_path(new_parent_rel)
.map_err(|e| self.errno_error(e, "resolve new parent"))?;
self.ensure_children(pino)
.map_err(|e| self.errno_error(e, "enumerate"))?;
self.require_uid_writable(&new_parent_uid)
.map_err(|error| self.errno_error(error, "copy access"))?;
let (name, is_folder, unsaved) = {
let st = self.state();
let entry = st
.entries
.get(&ino)
.ok_or_else(|| CoreError::not_found(format!("{} is gone", rel.display())))?;
// Unsaved writes, or a release still queueing them, to the file or,
// for a folder, to any file below it.
let unsaved = st
.active_writes
.iter()
.filter(|(_, w)| w.dirty)
.map(|(&open, _)| open)
.chain(st.releasing.iter().copied())
.any(|open| relocate::ino_is_under(&st, open, ino));
(entry.node.name.clone(), entry.node.is_folder(), unsaved)
};
// Unlike a move, a copy cannot wait in the queue: it needs the node as
// Drive has it, so it goes now or not at all, local-first or not.
if !self.is_online() {
return Err(CoreError::offline());
}
let owed = unsaved
|| is_local_uid(&uid)
|| is_local_uid(&new_parent_uid)
|| self.db.has_any_op(&uid.to_string()).unwrap_or(true)
|| (is_folder
&& self
.db
.pending_op_counts()
.map(|counts| counts.uploads + counts.changes > 0)
.unwrap_or(true));
if owed {
return Err(CoreError::conflict(format!(
"{name} has changes that have not reached Proton Drive yet; \
copy it once they have synced"
)));
}
let target = if is_folder {
AvailableNameTarget::Folder
} else {
AvailableNameTarget::File
};
let copy_name = self
.rt
.block_on(
self.drive
.get_available_name(&new_parent_uid, &name, target),
)
.map_err(|e| CoreError::from_api(&e, "copy"))?;
let new_uid = self
.rt
.block_on(
self.drive
.copy_node(&uid, &new_parent_uid, Some(&copy_name)),
)
.map_err(|e| CoreError::from_api(&e, "copy"))?;
let node = self
.fetch_minted_node(&new_uid, &copy_name)
.ok_or_else(|| minted_but_unlisted(&copy_name))?;
let mut st = self.state();
let copy_ino = st.intern(pino, node);
if let Some(kids) = st.children.get_mut(&pino)
&& !kids.contains(&copy_ino)
{
kids.push(copy_ino);
}
drop(st);
self.flush_access_changes();
Ok(copy_name)
}
/// What the folder at `rel` holds on Proton Drive: bytes on storage and the
/// number of items below it. Drive adds it up, so nothing is listed here;
/// trashed items count, and the bytes are encrypted ones.
pub(crate) fn folder_size(&self, rel: &Path) -> CoreResult<proton_drive_rs::FolderSize> {
let (ino, uid) = self.resolve(rel)?;
let is_folder = self
.state()
.entries
.get(&ino)
.is_some_and(|entry| entry.node.is_folder());
if !is_folder {
return Err(CoreError::invalid(format!(
"{} is not a folder",
rel.display()
)));
}
self.folder_size_for_uid(&uid)
}
/// [`Core::folder_size`] for a folder addressed through any daemon location.
pub(crate) fn folder_size_by_uid(&self, uid: &str) -> CoreResult<proton_drive_rs::FolderSize> {
let uid = self.resolve_anywhere(uid)?;
self.folder_size_for_uid(&uid)
}
fn folder_size_for_uid(&self, uid: &NodeUid) -> CoreResult<proton_drive_rs::FolderSize> {
if is_local_uid(uid) {
return Err(CoreError::conflict("the folder is not on Proton Drive yet"));
}
if !self.is_online() {
return Err(CoreError::offline());
}
self.rt
.block_on(self.client.get_folder_size(uid))
.map_err(|e| CoreError::from_api(&e, "folder size"))
}
/// Trash a file or folder. `rel` is mountpoint-relative. Forgets the node,
/// evicts any cached content, and invalidates the parent listing.
fn delete(&self, rel: &Path) -> CoreResult<String> {
@@ -5862,6 +5991,7 @@ impl Core {
if let Some(p) = self.sync_progress.lock().get_mut(&folder_id) {
f(p);
}
self.sync_engine.moved();
}
/// Set the number of items the scan expects to check, from the size of the
@@ -7185,13 +7315,15 @@ mod size_upgrade_tests {
fn a_listing_waits_for_the_whole_size_batch_but_not_forever() {
let slot = Arc::new(SizeUpgrade::default());
let worker = slot.clone();
// Read before the spawn: the worker's sleeps can start before this
// thread runs again, which made the bound below fail on a busy runner.
let started = Instant::now();
let t = std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(30));
worker.chunk_done();
std::thread::sleep(Duration::from_millis(60));
worker.finish();
});
let started = Instant::now();
slot.wait_finished(started + Duration::from_secs(5));
// Past the first chunk, up to the end of the batch.
assert!(started.elapsed() >= Duration::from_millis(90));
+72 -28
View File
@@ -507,6 +507,7 @@ pub(crate) fn mount_with(
events: Arc::new(Default::default()),
indexing: Arc::new(AtomicBool::new(false)),
sync_progress: Arc::new(Mutex::new(HashMap::new())),
sync_engine: Arc::default(),
sync_tx,
mounts: Arc::new(Mutex::new(SecondaryMountRegistry::default())),
sync_locks: Arc::new(Mutex::new(HashMap::new())),
@@ -540,12 +541,11 @@ pub(crate) fn mount_with(
// and sweeps all operating on a mount that no longer exists (`docs/BUGS.md` B44).
let mut workers: Vec<std::thread::JoinHandle<()>> = Vec::new();
for worker in 0..DRAIN_WORKERS {
let core = core.clone();
workers.push(
std::thread::Builder::new()
.name(format!("pdfs-drain-{worker}"))
.spawn(move || core.run_pending_drain(worker == 0))?,
);
let drain_core = core.clone();
let spawned = std::thread::Builder::new()
.name(format!("pdfs-drain-{worker}"))
.spawn(move || drain_core.run_pending_drain(worker == 0));
keep_or_abandon(&core, control_socket, &mut workers, spawned)?;
}
// Start the folder-sync engine. It watches every mirror folder, polls the
@@ -565,24 +565,22 @@ pub(crate) fn mount_with(
info!("conflict sweep disabled by configuration");
} else {
info!(mode = ?sweep_mode, "conflict sweep enabled");
let core = core.clone();
workers.push(
std::thread::Builder::new()
.name("pdfs-conflict-sweep".into())
.spawn(move || core.run_conflict_sweep_loop())?,
);
let sweep_core = core.clone();
let spawned = std::thread::Builder::new()
.name("pdfs-conflict-sweep".into())
.spawn(move || sweep_core.run_conflict_sweep_loop());
keep_or_abandon(&core, control_socket, &mut workers, spawned)?;
}
// Watch the link for the life of the mount: a mount from the cache waits for
// the network to come back, and an online one can lose it at any moment and
// must notice it coming back just the same.
{
let core = core.clone();
workers.push(
std::thread::Builder::new()
.name("pdfs-online-probe".into())
.spawn(move || core.run_online_probe())?,
);
let probe_core = core.clone();
let spawned = std::thread::Builder::new()
.name("pdfs-online-probe".into())
.spawn(move || probe_core.run_online_probe());
keep_or_abandon(&core, control_socket, &mut workers, spawned)?;
}
// Keep the launcher prompt's "This computer" index fresh. Its own thread:
@@ -593,11 +591,10 @@ pub(crate) fn mount_with(
let transfers = core.transfers.clone();
let mountpoint = mountpoint.to_path_buf();
let shutdown = core.shutdown.clone();
workers.push(
std::thread::Builder::new()
.name("pdfs-localindex".into())
.spawn(move || run_local_index(db, indexing, transfers, mountpoint, shutdown))?,
);
let spawned = std::thread::Builder::new()
.name("pdfs-localindex".into())
.spawn(move || run_local_index(db, indexing, transfers, mountpoint, shutdown));
keep_or_abandon(&core, control_socket, &mut workers, spawned)?;
}
// Bind the control socket before the FUSE session takes over the thread. A
@@ -606,12 +603,18 @@ pub(crate) fn mount_with(
let old_umask = unsafe { libc::umask(0o77) };
let listener_res = UnixListener::bind(control_socket);
unsafe { libc::umask(old_umask) };
let listener = listener_res?;
let listener = match listener_res {
Ok(listener) => listener,
Err(error) => {
abandon_workers(&core, control_socket, workers);
return Err(error);
}
};
// Owner-only before anything can connect: a peer on this socket commands the
// daemon's authenticated session without a credential of its own (B6).
if let Err(e) = pdfs_core::config::restrict_socket(control_socket) {
error!(error = %e, "cannot restrict control socket permissions; refusing to serve");
let _ = std::fs::remove_file(control_socket);
abandon_workers(&core, control_socket, workers);
return Err(std::io::Error::other(format!(
"control socket permissions: {e}"
)));
@@ -622,7 +625,7 @@ pub(crate) fn mount_with(
let bg = match spawn_session(&core, mountpoint, root) {
Ok(session) => session,
Err(error) => {
let _ = std::fs::remove_file(control_socket);
abandon_workers(&core, control_socket, workers);
return Err(error);
}
};
@@ -642,7 +645,7 @@ pub(crate) fn mount_with(
Err(error) => {
// A failure here abandons the mount, so the background threads
// started above have to be told to stop with it.
stop_workers(&core, control_socket, workers);
abandon_workers(&core, control_socket, workers);
let secondaries = core.mounts.lock().close_and_drain();
for (id, mount) in secondaries {
if let Err(unmount_error) = mount.teardown() {
@@ -692,8 +695,12 @@ pub(crate) fn mount_with(
}
}
// The mount is live and the control socket answers: this is the moment the
// unit is actually usable, which is what `Type=notify` reports.
// unit is actually usable, which is what `Type=notify` reports. Whatever
// failure an earlier attempt recorded for the front ends is over.
systemd::ready();
if let Some(state_dir) = control_socket.parent() {
pdfs_core::mountfail::clear(state_dir);
}
// Re-establish on-demand mounts only after the primary session is live.
// Keep the worker so shutdown can close publication, wait for any in-flight
@@ -859,6 +866,43 @@ fn stop_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::Jo
// Say so before the joins: a teardown that takes a while must not be read as
// a missed watchdog ping and turned into a kill.
systemd::stopping();
halt_workers(core, control_socket, workers);
}
/// Stop the threads a mount attempt started before it failed.
///
/// The early returns used to leave them running: every retry started another
/// set beside the last one's, and their `Core` kept the database open, so the
/// next attempt could fail on its lock instead of the real error
/// (`docs/BUGS.md` B264). Unlike [`stop_workers`] this does not tell systemd
/// the daemon is stopping, because it is about to try again, and a manager
/// told so would wait for the process to exit and then kill it.
fn abandon_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::JoinHandle<()>>) {
halt_workers(core, control_socket, workers);
}
/// Keep a worker the mount just spawned, or, if the spawn failed, stop the
/// ones already running and hand the error back.
fn keep_or_abandon(
core: &Core,
control_socket: &Path,
workers: &mut Vec<std::thread::JoinHandle<()>>,
spawned: std::io::Result<std::thread::JoinHandle<()>>,
) -> std::io::Result<()> {
match spawned {
Ok(handle) => {
workers.push(handle);
Ok(())
}
Err(error) => {
abandon_workers(core, control_socket, std::mem::take(workers));
Err(error)
}
}
}
/// Signal every worker to stop, then join them up to [`WORKER_JOIN_DEADLINE`].
fn halt_workers(core: &Core, control_socket: &Path, workers: Vec<std::thread::JoinHandle<()>>) {
core.shutdown.stop();
core.wake_drain();
let _ = core.sync_tx.send(sync::SyncMsg::Stop);
+45 -9
View File
@@ -388,7 +388,7 @@ impl Core {
&self,
items: &[pdfs_core::control::RestoreItem],
device: Option<&str>,
) -> CoreResult<String> {
) -> CoreResult<RestoreOutcome> {
if items.is_empty() {
return Err(CoreError::invalid("nothing to restore"));
}
@@ -434,10 +434,19 @@ impl Core {
continue;
}
};
let id = self
.db
.sync_folder_add(&local.to_string_lossy(), &item.remote_uid, &device.share_id)
.map_err(|e| CoreError::internal(format!("db: {e:?}")))?;
let id = match self.db.sync_folder_add(
&local.to_string_lossy(),
&item.remote_uid,
&device.share_id,
) {
Ok(id) => id,
Err(e) => {
// The folders restored before this one still need their
// watches.
let _ = self.sync_tx.send(sync::SyncMsg::Rewatch);
return Err(CoreError::internal(format!("db: {e:?}")));
}
};
// `ondemand` is applied through the normal request path, which
// mounts the FUSE session and handles the folder being busy — the
// restore does not get its own copy of that logic.
@@ -450,12 +459,39 @@ impl Core {
}
let _ = self.sync_tx.send(sync::SyncMsg::Rewatch);
let mut message = format!("restored {restored} folder(s)");
if !skipped.is_empty() {
message.push_str(&format!("; skipped: {}", skipped.join(", ")));
// The reply is gone once the app has shown it; the log keeps why.
for reason in &skipped {
info!(reason, "restore skipped a folder");
}
info!(restored, skipped = skipped.len(), "restore complete");
Ok(message)
Ok(RestoreOutcome { restored, skipped })
}
}
/// What a restore did: how many folders it attached, and why it left out the
/// others.
pub(crate) struct RestoreOutcome {
pub(crate) restored: usize,
pub(crate) skipped: Vec<String>,
}
impl RestoreOutcome {
/// The activity entry's headline.
pub(crate) fn summary(&self) -> String {
match (self.restored, self.skipped.len()) {
(n, 0) => format!("restored {n} folder(s)"),
(0, k) => format!("restored no folders; skipped {k}"),
(n, k) => format!("restored {n} folder(s); skipped {k}"),
}
}
/// Whether the restore did all it was asked. A skipped folder is a
/// failure the user has to see: the reply went out as "restoring folders"
/// before any work, and a restore that skipped every folder used to be
/// logged as a success, so nothing anywhere said that nothing would
/// download (`docs/BUGS.md` B269).
pub(crate) fn complete(&self) -> bool {
self.skipped.is_empty()
}
}
+86
View File
@@ -1101,6 +1101,92 @@ impl Core {
})
}
/// Bytes `[offset, offset + size)` of file `ino` as `read` would serve
/// them, but only from this machine: an open write handle, a queued write,
/// or the caches. `Ok(None)` the moment any of it would have to be
/// downloaded, and for a size still provisional (B100), which only the
/// network settles. Short or empty past the end of the file.
///
/// `copy_file_range` answers with this and leaves anything else to the
/// kernel's read/write copy.
pub(super) fn read_local(
&self,
ino: u64,
offset: u64,
size: u64,
) -> Result<Option<Vec<u8>>, Errno> {
let handle = self.state().active_writes.get(&ino).map(|h| {
(
h.file.clone(),
h.len,
h.uid.clone(),
h.base_mtime,
h.base_size,
h.written.clone(),
)
});
if let Some((file, len, uid, base_mtime, base_size, written)) = handle {
return self.serve_open_read_local(
&file, len, &uid, base_mtime, base_size, &written, offset, size,
);
}
let (uid, mtime, fsize, provisional) = {
let st = self.state();
match st.entries.get(&ino) {
Some(e) if e.node.is_file() => (
e.uid.clone(),
e.node.modification_time,
node_size(&e.node),
matches!(
&e.node.kind,
NodeKind::File {
claimed_size: None,
..
}
),
),
Some(_) => return Err(Errno::EISDIR),
None => return Err(Errno::ENOENT),
}
};
let uid = match is_local_uid(&uid) {
true => self.landed_stand_in_uid(&uid).unwrap_or(uid),
false => uid,
};
if let Some(pending) = self.pending.lock().get(&uid).cloned() {
let m = &pending.meta;
let file = File::open(&pending.path).map_err(|e| {
error!(%uid, path = %pending.path.display(), error = %e, "open staged blob failed");
Errno::EIO
})?;
let mut written = Intervals::default();
for &(s, e) in &m.authored {
written.add(s, e);
}
return self.serve_open_read_local(
&Arc::new(file),
m.len,
&uid,
m.base_mtime,
m.base_size,
&written,
offset,
size,
);
}
// Created here and never written: an empty file.
if is_local_uid(&uid) {
return Ok(Some(Vec::new()));
}
if provisional {
return Ok(None);
}
if offset >= fsize || size == 0 {
return Ok(Some(Vec::new()));
}
Ok(self.cached_range(&uid, mtime, fsize, offset, size.min(fsize - offset)))
}
/// The largest amount [`Core::fill_gaps_cached`] will copy out of the local
/// caches while a `release(2)` is in progress.
///
+2 -1
View File
@@ -133,7 +133,7 @@ fn rename_noreplace(from: &Path, to: &Path) -> std::io::Result<()> {
/// Whether inode `ino` is `root` or lies under it in the tree. Bounded, so a
/// parent cycle in corrupt data cannot hang the caller.
fn ino_is_under(st: &State, mut ino: u64, root: u64) -> bool {
pub(crate) fn ino_is_under(st: &State, mut ino: u64, root: u64) -> bool {
for _ in 0..MAX_TREE_DEPTH {
if ino == root {
return true;
@@ -456,6 +456,7 @@ impl Core {
writing = st
.active_writes
.keys()
.chain(&st.releasing)
.any(|&open| ino_is_under(st, open, ino));
});
Ok(writing)
+85 -1
View File
@@ -9,7 +9,8 @@
use std::path::Path;
use pdfs_core::control::{
BookmarkInfo, DirEntry, InvitationInfo, PublicLinkInfo, ShareEntry, ShareEntryKind, SharedItem,
BookmarkInfo, DirEntry, InvitationInfo, LinkChange, PublicLinkInfo, ShareEntry, ShareEntryKind,
SharedItem,
};
use pdfs_core::{CoreError, CoreResult};
use tracing::debug;
@@ -367,6 +368,89 @@ impl Core {
Ok(public_link_info(link))
}
/// Change the public link `id` on the node at `rel`, keeping its URL.
pub(crate) fn update_public_link(
&self,
rel: &Path,
id: &str,
role: Option<&str>,
password: LinkChange<String>,
expires: LinkChange<i64>,
) -> CoreResult<PublicLinkInfo> {
let (_ino, uid) = self.resolve(rel)?;
self.update_public_link_for_uid(&uid, id, role, password, expires)
}
/// [`Core::update_public_link`] for a node addressed through any daemon
/// location.
pub(crate) fn update_public_link_by_uid(
&self,
uid: &str,
id: &str,
role: Option<&str>,
password: LinkChange<String>,
expires: LinkChange<i64>,
) -> CoreResult<PublicLinkInfo> {
let uid = self.resolve_anywhere(uid)?;
self.update_public_link_for_uid(&uid, id, role, password, expires)
}
/// The SDK sets all three options on every update, so each `Keep` is
/// filled in from the link as it stands.
fn update_public_link_for_uid(
&self,
uid: &NodeUid,
id: &str,
role: Option<&str>,
password: LinkChange<String>,
expires: LinkChange<i64>,
) -> CoreResult<PublicLinkInfo> {
let link = self
.rt
.block_on(self.client.get_public_link(uid))
.map_err(|e| CoreError::from_api(&e, "get public link"))?
.filter(|l| l.public_link_id == id)
.ok_or_else(|| CoreError::not_found("public link not found"))?;
let role = match role {
Some(role) => role_from_str(role)?,
None => link.role,
};
let unchanged = role == link.role
&& matches!(password, LinkChange::Keep)
&& matches!(expires, LinkChange::Keep);
if unchanged {
return Ok(public_link_info(link));
}
let password = match password {
LinkChange::Keep if link.has_custom_password => {
// Sending `None` here would quietly remove the password.
Some(link.custom_password.clone().ok_or_else(|| {
CoreError::conflict(
"the link's password could not be read; set a new one or remove it",
)
})?)
}
LinkChange::Keep | LinkChange::Remove => None,
LinkChange::Set(password) if password.is_empty() => {
return Err(CoreError::invalid("the new password is empty"));
}
LinkChange::Set(password) => Some(password),
};
let expires = match expires {
LinkChange::Keep => link.expiration_time,
LinkChange::Remove => None,
LinkChange::Set(at) => Some(at),
};
let link = self
.rt
.block_on(
self.client
.update_public_link(uid, role, password.as_deref(), expires),
)
.map_err(|e| CoreError::from_api(&e, "update public link"))?;
Ok(public_link_info(link))
}
/// Remove the public link `id` from the node at `rel`.
pub(crate) fn remove_public_link(&self, rel: &Path, id: &str) -> CoreResult<()> {
let (_ino, uid) = self.resolve(rel)?;
+440 -8
View File
@@ -130,6 +130,15 @@ impl Daemon {
}
}
/// Whether the mount believes Proton is reachable, as `pdfs status` shows
/// it.
pub(crate) fn online(&self) -> io::Result<bool> {
match self.request(&Request::Status)? {
Response::Status { online, .. } => Ok(online),
other => Err(io::Error::other(format!("unexpected answer {other:?}"))),
}
}
/// Send one control request, as the `pdfs` CLI does, and fail on a
/// refusal.
pub(crate) fn request(&self, request: &Request) -> io::Result<Response> {
@@ -583,14 +592,22 @@ mod tests {
daemon.pending(),
drive.tree()
);
// The bytes kept for the handle went once the trash landed with it closed.
let cached: Vec<_> = std::fs::read_dir(dir.join("cache"))
.unwrap()
.filter_map(|e| e.ok())
.filter(|e| e.file_type().is_ok_and(|t| t.is_file()))
.map(|e| e.file_name())
.collect();
assert!(cached.is_empty(), "{cached:?}");
// The bytes kept for the handle went once the trash landed with it
// closed. The queue empties as the trash's row goes, a moment before
// the eviction, so this waits too.
let cached = || -> Vec<_> {
std::fs::read_dir(dir.join("cache"))
.unwrap()
.filter_map(|e| e.ok())
.filter(|e| e.file_type().is_ok_and(|t| t.is_file()))
.map(|e| e.file_name())
.collect()
};
assert!(
wait_until(Duration::from_secs(30), || cached().is_empty()),
"{:?}",
cached()
);
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
@@ -648,7 +665,19 @@ mod tests {
.pending()
.is_ok_and(|items| items.iter().any(|item| item.attempts > 0))
}));
// The drain records the failure just before it takes the mount
// offline. Left online, only the event poll would, much later.
assert!(wait_until(Duration::from_secs(2), || daemon
.online()
.is_ok_and(|online| !online)));
daemon.client.set_online(true);
// The probe that sees the link return runs 2 s after the drop, then
// 6 s, and a loaded run can reach this line after the first one. So
// the 3 s below count from the mount being back, not from the link:
// an op left on its backoff waits 4 s or more after its failure.
assert!(wait_until(Duration::from_secs(30), || daemon
.online()
.unwrap_or(false)));
assert!(
wait_until(Duration::from_secs(3), || {
daemon.pending().is_ok_and(|items| items.is_empty())
@@ -1977,6 +2006,21 @@ mod tests {
daemon.write_new(&path, b"first").unwrap();
assert!(wait_until(Duration::from_secs(30), || held.reached()));
std::fs::write(&path, b"second").unwrap();
// The kernel sends the release after `close` returns. Attaching the
// bytes resets the create's attempts, so a release that came after the
// lost reply had it retried at once, and the retry landed before
// anything here saw it fail. The bytes it replaced go only once they
// are attached.
let staged = dir.join("cache").join("staging");
assert!(wait_until(Duration::from_secs(30), || {
let blobs: Vec<Vec<u8>> = std::fs::read_dir(&staged)
.into_iter()
.flatten()
.filter_map(|e| e.ok())
.filter_map(|e| std::fs::read(e.path()).ok())
.collect();
blobs.iter().any(|b| b == b"second") && !blobs.iter().any(|b| b == b"first")
}));
drop(held);
assert!(wait_until(Duration::from_secs(30), || {
daemon
@@ -2214,6 +2258,75 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_pdfs_copy_copies_on_drive_and_waits_for_unsynced_changes() {
let drive = FakeDrive::new();
let dir = scratch("pdfs-copy");
let daemon = Daemon::start(&dir, drive.client(1, Faults::lan())).unwrap();
let mount = &daemon.mountpoint;
std::fs::create_dir_all(mount.join("src/inner")).unwrap();
std::fs::create_dir_all(mount.join("dest")).unwrap();
std::fs::write(mount.join("src/a.txt"), b"alpha").unwrap();
std::fs::write(mount.join("src/inner/b.txt"), b"beta").unwrap();
let file = |bytes: &[u8]| Some(Entry::File(Arc::new(bytes.to_vec())));
assert!(wait_until(Duration::from_secs(30), || {
drive.tree().get("src/inner/b.txt").cloned() == file(b"beta")
&& daemon.pending().is_ok_and(|ops| ops.is_empty())
}));
let path = |rel: &str| mount.join(rel).to_string_lossy().into_owned();
let copy = |from: &str, to: &str| {
daemon.request(&Request::Copy {
path: path(from),
new_parent: path(to),
})
};
let uploads = drive.tree().len();
copy("src/a.txt", "dest").unwrap();
// A name already taken gets a suffix rather than an error.
copy("src/a.txt", "dest").unwrap();
copy("src", "dest").unwrap();
assert_eq!(std::fs::read(mount.join("dest/a.txt")).unwrap(), b"alpha");
assert_eq!(
std::fs::read(mount.join("dest/a (1).txt")).unwrap(),
b"alpha"
);
assert_eq!(
std::fs::read(mount.join("dest/src/inner/b.txt")).unwrap(),
b"beta"
);
let tree = drive.tree();
assert_eq!(tree.get("dest/src/a.txt").cloned(), file(b"alpha"));
assert_eq!(
tree.len(),
uploads + 6,
"{:?}",
tree.keys().collect::<Vec<_>>()
);
// Drive would copy what it has, not the bytes still queued.
let paused = |paused| Request::SetSyncPaused {
paused,
until: None,
};
daemon.request(&paused(true)).unwrap();
std::fs::write(mount.join("src/a.txt"), b"alpha, edited").unwrap();
match copy("src/a.txt", "dest") {
Err(e) => assert!(e.to_string().contains("not reached Proton Drive"), "{e}"),
Ok(answer) => panic!("the copy went ahead: {answer:?}"),
}
match copy("src", "dest") {
Err(e) => assert!(e.to_string().contains("not reached Proton Drive"), "{e}"),
Ok(answer) => panic!("the folder copy went ahead: {answer:?}"),
}
daemon.request(&paused(false)).unwrap();
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_pdfs_rename_after_a_queued_rename_is_not_undone() {
@@ -2364,6 +2477,152 @@ mod tests {
eprintln!("wifi: syscalls {wifi_syscalls:?}, drain {wifi_drain:?}");
}
/// `copy_file_range` inside the mount is answered from the bytes the daemon
/// holds, and left to the kernel for bytes it would have to download. Every
/// source kind has to come out byte for byte: cached, queued, open with
/// unsaved writes, and on Drive only.
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_copy_inside_the_mount_has_the_source_bytes() {
use std::os::fd::AsRawFd as _;
use std::os::unix::fs::FileExt as _;
fn copy_range(from: &std::fs::File, at: u64, to: &std::fs::File, to_at: u64, len: usize) {
let (mut at, mut to_at) = (at as i64, to_at as i64);
let mut left = len;
while left > 0 {
// SAFETY: both descriptors are open; the offsets are ours.
let n = unsafe {
libc::copy_file_range(
from.as_raw_fd(),
&mut at,
to.as_raw_fd(),
&mut to_at,
left,
0,
)
};
assert!(n > 0, "copy_file_range: {}", io::Error::last_os_error());
left -= n as usize;
}
}
let pattern = |len: usize, seed: u8| -> Vec<u8> {
(0..len).map(|i| (i % 253) as u8 ^ seed).collect()
};
let drive = FakeDrive::new();
// On Drive before the mount knows it, so nothing of it is cached.
let remote = pattern(9 << 20, 1);
drive.device().write("remote.bin", &remote);
let dir = scratch("copy-range");
let daemon = Daemon::start(&dir, drive.client(1, Faults::lan())).unwrap();
let mount = &daemon.mountpoint;
// Downloaded through the kernel's own copy, then served from the cache.
std::fs::copy(mount.join("remote.bin"), mount.join("remote-1.bin")).unwrap();
assert!(std::fs::read(mount.join("remote-1.bin")).unwrap() == remote);
std::fs::copy(mount.join("remote.bin"), mount.join("remote-2.bin")).unwrap();
assert!(std::fs::read(mount.join("remote-2.bin")).unwrap() == remote);
// Queued and not on Drive yet: the staged blob is the content.
let paused = |paused| Request::SetSyncPaused {
paused,
until: None,
};
daemon.request(&paused(true)).unwrap();
let queued = pattern(20 << 20, 2);
std::fs::write(mount.join("queued.bin"), &queued).unwrap();
std::fs::copy(mount.join("queued.bin"), mount.join("queued-copy.bin")).unwrap();
assert!(std::fs::read(mount.join("queued-copy.bin")).unwrap() == queued);
// Open with writes not saved yet: the copy sees them, as a read would.
let open = std::fs::OpenOptions::new()
.read(true)
.write(true)
.open(mount.join("queued.bin"))
.unwrap();
open.write_all_at(b"UNSAVED", 100).unwrap();
let target = std::fs::File::create(mount.join("unsaved-copy.bin")).unwrap();
copy_range(&open, 96, &target, 0, 16);
drop(target);
drop(open);
let mut expected = queued[96..112].to_vec();
expected[4..11].copy_from_slice(b"UNSAVED");
assert_eq!(
std::fs::read(mount.join("unsaved-copy.bin")).unwrap(),
expected
);
// A range into the middle of a file keeps the bytes around it.
std::fs::write(mount.join("middle.bin"), b"0123456789").unwrap();
let source = std::fs::File::open(mount.join("remote.bin")).unwrap();
let target = std::fs::OpenOptions::new()
.write(true)
.open(mount.join("middle.bin"))
.unwrap();
copy_range(&source, 5, &target, 3, 4);
drop(target);
let mut expected = b"0123456789".to_vec();
expected[3..7].copy_from_slice(&remote[5..9]);
assert_eq!(std::fs::read(mount.join("middle.bin")).unwrap(), expected);
daemon.request(&paused(false)).unwrap();
let mut edited = queued.clone();
edited[100..107].copy_from_slice(b"UNSAVED");
let file = |bytes: &[u8]| Some(Entry::File(Arc::new(bytes.to_vec())));
assert!(wait_until(Duration::from_secs(120), || {
let tree = drive.tree();
daemon.pending().is_ok_and(|ops| ops.is_empty())
&& tree.get("queued.bin").cloned() == file(&edited)
&& tree.get("queued-copy.bin").cloned() == file(&queued)
&& tree.get("remote-2.bin").cloned() == file(&remote)
&& tree.get("middle.bin").cloned() == file(&expected)
}));
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
/// How long `cp` inside the mount takes for a file the daemon already
/// holds, cold and from the page cache. It decides whether the daemon
/// should answer `copy_file_range` itself rather than leave the kernel to
/// copy through read and write. Runs when `PDFS_SIM_MEASURE` is set.
#[test]
#[ignore = "mounts FUSE: run with `PDFS_SIM_MEASURE=1 cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_copy_inside_the_mount_measures_its_speed() {
if std::env::var_os("PDFS_SIM_MEASURE").is_none() {
eprintln!("skipped: set PDFS_SIM_MEASURE to measure the copy");
return;
}
const SIZE: usize = 256 << 20;
let drive = FakeDrive::new();
let dir = scratch("copy-speed");
let daemon = Daemon::start(&dir, drive.client(1, Faults::lan())).unwrap();
let mount = &daemon.mountpoint;
let bytes: Vec<u8> = (0..SIZE).map(|i| (i % 251) as u8).collect();
std::fs::write(mount.join("big.bin"), &bytes).unwrap();
assert!(wait_until(Duration::from_secs(120), || {
daemon.pending().is_ok_and(|ops| ops.is_empty())
}));
for (round, name) in ["copy-1.bin", "copy-2.bin", "copy-3.bin"]
.into_iter()
.enumerate()
{
let start = Instant::now();
let status = std::process::Command::new("cp")
.arg(mount.join("big.bin"))
.arg(mount.join(name))
.status()
.unwrap();
let took = start.elapsed();
assert!(status.success());
let mib_per_s = (SIZE >> 20) as f64 / took.as_secs_f64();
eprintln!("cp round {round}: {took:?}, {mib_per_s:.0} MiB/s");
assert!(std::fs::read(mount.join(name)).unwrap() == bytes);
}
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn half_of_a_wide_folder_unlinked_at_once_leaves_the_other_half() {
@@ -2658,4 +2917,177 @@ mod tests {
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
/// The synced folder `id` as the daemon lists it.
fn listed_folder(daemon: &Daemon, id: i64) -> Option<control::SyncFolderInfo> {
match daemon.request(&Request::ListSyncFolders) {
Ok(Response::SyncFolders { items }) => items.into_iter().find(|f| f.id == id),
_ => None,
}
}
/// A mirror folder holding `a.txt` and `sub/b.txt`, synced to Drive, whose
/// local copy is then deleted in full while the folder is paused, so no
/// pass sees half of it. Waits for the pass that refuses to run (#46).
fn folder_whose_local_copy_is_gone(daemon: &Daemon, drive: &FakeDrive, local: &Path) -> i64 {
let id = sync_folder(daemon, local, "mirror");
std::fs::write(local.join("a.txt"), b"alpha").unwrap();
std::fs::create_dir(local.join("sub")).unwrap();
std::fs::write(local.join("sub/b.txt"), b"beta").unwrap();
daemon.request(&Request::SyncNow { id: Some(id) }).unwrap();
let on_drive = |name: &str, content: &[u8]| {
drive.tree().iter().any(|(path, entry)| {
path.ends_with(name) && *entry == Entry::File(Arc::new(content.to_vec()))
})
};
assert!(
wait_until(Duration::from_secs(30), || on_drive("a.txt", b"alpha")
&& on_drive("sub/b.txt", b"beta")
&& listed_folder(daemon, id).is_some_and(|f| f.state == "idle")),
"the folder synced: {:?} {:?}",
listed_folder(daemon, id),
drive.tree().keys().collect::<Vec<_>>()
);
daemon
.request(&Request::SetSyncFolderPaused { id, paused: true })
.unwrap();
std::fs::remove_file(local.join("a.txt")).unwrap();
std::fs::remove_dir_all(local.join("sub")).unwrap();
daemon
.request(&Request::SetSyncFolderPaused { id, paused: false })
.unwrap();
daemon.request(&Request::SyncNow { id: Some(id) }).unwrap();
assert!(
wait_until(Duration::from_secs(30), || listed_folder(daemon, id)
.is_some_and(|f| f.state == "error")),
"the pass refused to trash everything: {:?}",
listed_folder(daemon, id)
);
assert!(
on_drive("a.txt", b"alpha") && on_drive("sub/b.txt", b"beta"),
"nothing was trashed on Drive: {:?}",
drive.tree().keys().collect::<Vec<_>>()
);
id
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_folder_whose_local_copy_is_gone_downloads_again_after_a_reset() {
let drive = FakeDrive::new();
let dir = scratch("reset-gone-local-copy");
let daemon = Daemon::start(&dir, drive.client(1, Faults::lan())).unwrap();
let local = dir.join("docs");
let id = folder_whose_local_copy_is_gone(&daemon, &drive, &local);
daemon.request(&Request::ResetSyncFolder { id }).unwrap();
let has = |rel: &str, content: &[u8]| {
std::fs::read(local.join(rel)).is_ok_and(|bytes| bytes == content)
};
assert!(
wait_until(Duration::from_secs(30), || has("a.txt", b"alpha")
&& has("sub/b.txt", b"beta")
&& listed_folder(&daemon, id).is_some_and(|f| f.state == "idle")),
"the reset downloaded the folder again: {:?}",
listed_folder(&daemon, id)
);
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_folder_whose_local_copy_is_gone_still_goes_on_demand() {
let drive = FakeDrive::new();
let dir = scratch("ondemand-gone-local-copy");
let daemon = Daemon::start(&dir, drive.client(1, Faults::lan())).unwrap();
let local = dir.join("docs");
let id = folder_whose_local_copy_is_gone(&daemon, &drive, &local);
daemon
.request(&Request::SetSyncFolderMode {
id,
mode: "ondemand".into(),
})
.unwrap();
assert!(
wait_until(Duration::from_secs(30), || listed_folder(&daemon, id)
.is_some_and(|f| f.mode == "ondemand" && f.pending_mode.is_none())
&& is_mounted(&local)),
"the queued switch landed: {:?}",
listed_folder(&daemon, id)
);
assert_eq!(std::fs::read(local.join("a.txt")).unwrap(), b"alpha");
assert_eq!(std::fs::read(local.join("sub/b.txt")).unwrap(), b"beta");
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
/// Emptied before any pass saw it, the folder is still `idle`, and the
/// switch was refused for good: every path read as "deleted here but not on
/// Proton Drive yet" (B255, found by the account run).
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_folder_emptied_before_any_pass_still_goes_on_demand() {
let drive = FakeDrive::new();
let dir = scratch("ondemand-emptied-idle");
let daemon = Daemon::start(&dir, drive.client(1, Faults::lan())).unwrap();
let local = dir.join("docs");
let id = sync_folder(&daemon, &local, "mirror");
std::fs::create_dir(local.join("sub")).unwrap();
std::fs::write(local.join("sub/a.txt"), b"alpha").unwrap();
std::fs::write(local.join("sub/b.txt"), b"beta").unwrap();
daemon.request(&Request::SyncNow { id: Some(id) }).unwrap();
let on_drive = |name: &str, content: &[u8]| {
drive.tree().iter().any(|(path, entry)| {
path.ends_with(name) && *entry == Entry::File(Arc::new(content.to_vec()))
})
};
assert!(
wait_until(Duration::from_secs(30), || on_drive("sub/a.txt", b"alpha")
&& on_drive("sub/b.txt", b"beta")
&& listed_folder(&daemon, id).is_some_and(|f| f.state == "idle")),
"the folder synced: {:?}",
listed_folder(&daemon, id)
);
// In one rename, so no pass sees half of it, and asked at once, well
// inside the watcher's debounce, so no pass sees it empty either.
std::fs::rename(local.join("sub"), dir.join("moved-away")).unwrap();
let asked = daemon.request(&Request::SetSyncFolderMode {
id,
mode: "ondemand".into(),
});
assert!(
matches!(asked, Ok(Response::Ok { .. })),
"the switch was refused: {asked:?}"
);
assert!(
listed_folder(&daemon, id)
.is_some_and(|f| f.pending_mode.as_deref() == Some("ondemand")),
"the switch waits for a pass: {:?}",
listed_folder(&daemon, id)
);
assert!(
wait_until(Duration::from_secs(30), || listed_folder(&daemon, id)
.is_some_and(|f| f.mode == "ondemand" && f.pending_mode.is_none())
&& is_mounted(&local)),
"the queued switch landed: {:?}",
listed_folder(&daemon, id)
);
assert_eq!(std::fs::read(local.join("sub/a.txt")).unwrap(), b"alpha");
assert_eq!(std::fs::read(local.join("sub/b.txt")).unwrap(), b"beta");
assert!(
on_drive("sub/a.txt", b"alpha") && on_drive("sub/b.txt", b"beta"),
"nothing was trashed on Drive: {:?}",
drive.tree().keys().collect::<Vec<_>>()
);
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
}
+90 -2
View File
@@ -61,8 +61,8 @@ use proton_drive_rs::proton_sdk::ids::{
DeviceUid, DriveEventId, LinkId, NodeUid, ShareId, VolumeId,
};
use proton_drive_rs::{
Device as Computer, DeviceType, DriveEvent, DriveEventScopeId, Node, NodeKind, NodeMoveItem,
RevisionState, Thumbnail,
AvailableNameTarget, Device as Computer, DeviceType, DriveEvent, DriveEventScopeId, Node,
NodeKind, NodeMoveItem, RevisionState, Thumbnail,
};
use sha1::{Digest as _, Sha1};
@@ -185,6 +185,39 @@ impl Server {
}
}
/// Copy `uid` and, for a folder, everything live below it under `parent` as
/// `name`, the way Drive's copy endpoint does. Returns the copy's uid.
fn copy_subtree(&mut self, uid: &NodeUid, parent: &NodeUid, name: String) -> NodeUid {
let source = self
.current(uid)
.cloned()
.expect("the caller checked the source");
let copy = self.mint_uid();
let now = now_secs();
self.put(
&copy,
Some(Remote {
parent: Some(parent.clone()),
name,
created: now,
..source
}),
);
let children: Vec<(NodeUid, String)> = self
.nodes
.keys()
.filter_map(|child| {
let remote = self.current(child)?;
(remote.parent.as_ref() == Some(uid) && !remote.trashed)
.then(|| (child.clone(), remote.name.clone()))
})
.collect();
for (child, child_name) in children {
self.copy_subtree(&child, &copy, child_name);
}
copy
}
fn mint_uid(&mut self) -> NodeUid {
self.next_link += 1;
uid(&format!("L{}", self.next_link))
@@ -1490,6 +1523,61 @@ impl DriveApi for FakeClient {
.await
}
async fn copy_node(
&self,
uid: &NodeUid,
new_parent: &NodeUid,
name: Option<&str>,
) -> Result<NodeUid> {
self.request(true, |server| {
if !server.is_folder(new_parent) {
return Err(api(ResponseCode::DoesNotExist, "parent folder not found"));
}
let source = server
.current(uid)
.filter(|remote| !remote.trashed)
.cloned()
.ok_or_else(|| api(ResponseCode::DoesNotExist, "node not found"))?;
if server.is_below(new_parent, uid) {
return Err(api(
ResponseCode::InvalidRequirements,
"cannot copy a folder into itself",
));
}
let name = name.map_or(source.name.clone(), str::to_owned);
if server.live_child_named(new_parent, &name).is_some() {
return Err(api(
ResponseCode::AlreadyExists,
"a file or folder with that name already exists",
));
}
Ok(server.copy_subtree(uid, new_parent, name))
})
.await
}
async fn get_available_name(
&self,
parent_uid: &NodeUid,
name: &str,
target: AvailableNameTarget,
) -> Result<String> {
self.request(false, |server| {
let (stem, extension) = match (target, name.rsplit_once('.')) {
(AvailableNameTarget::File, Some((stem, ext))) if !stem.is_empty() => {
(stem, format!(".{ext}"))
}
_ => (name, String::new()),
};
let free = std::iter::once(name.to_owned())
.chain((1..).map(|n| format!("{stem} ({n}){extension}")))
.find(|candidate| server.live_child_named(parent_uid, candidate).is_none())
.expect("an unbounded sequence has a free name");
Ok(free)
})
.await
}
fn move_nodes_streaming<'a>(
&'a self,
items: Vec<NodeMoveItem>,
+6
View File
@@ -266,6 +266,11 @@ pub(crate) struct State {
pub(crate) next_ino: u64,
/// Shared write state keyed by inode. Concurrent writers share the scratch file.
pub(crate) active_writes: HashMap<u64, WriteHandle>,
/// Inodes whose last write handle `release` has taken out of
/// [`State::active_writes`] and is still queueing. `close(2)` returns
/// before that, and in between the edit is in neither the handles nor the
/// queue, so a check for changes not on Drive yet looks here too.
pub(crate) releasing: HashSet<u64>,
/// Maps file handle id (fh) to inode (ino). Read-only opens use fh 0 and
/// have no entry here.
pub(crate) handles: HashMap<u64, u64>,
@@ -428,6 +433,7 @@ impl State {
roots: HashSet::new(),
next_ino: FALLBACK_INO,
active_writes: HashMap::new(),
releasing: HashSet::new(),
handles: HashMap::new(),
next_fh: 1,
dir_snapshots: HashMap::new(),
+67 -1
View File
@@ -36,6 +36,11 @@ const TICK: Duration = Duration::from_secs(15);
/// would train everyone to ignore the real one.
const STALL_AFTER: Duration = Duration::from_secs(120);
/// How long a sync pass may go without reporting progress before it is called
/// out. Much longer than [`STALL_AFTER`]: one large file in a pass reports
/// nothing until its whole transfer is done.
const SYNC_STALL_AFTER: Duration = Duration::from_secs(900);
/// How often resident size goes into the log.
const RSS_EVERY: Duration = Duration::from_secs(300);
@@ -137,6 +142,26 @@ fn report_stalls(report: &pdfs_core::control::Diagnostics, last_completed: (u64,
"the transfer lane has not completed a job since the last check"
);
}
// The engine is one thread for every synced folder, so a dead or stuck one
// stops all of them while the worker pool looks idle (B259).
if let Some(sync) = &report.sync {
if !sync.running {
stalled = true;
warn!(
"the sync engine is not running; synced folders do not sync until the daemon restarts"
);
} else if let Some(folder) = sync.pass_folder
&& sync.pass_idle_secs >= SYNC_STALL_AFTER.as_secs()
{
stalled = true;
warn!(
folder,
age_secs = sync.pass_age_secs,
idle_secs = sync.pass_idle_secs,
"a sync pass has not reported progress for a long time"
);
}
}
stalled
}
@@ -169,7 +194,7 @@ fn probe(control_socket: &Path) -> bool {
#[cfg(test)]
mod tests {
use super::*;
use pdfs_core::control::{Diagnostics, InflightRequest, WorkerState};
use pdfs_core::control::{Diagnostics, InflightRequest, SyncEngineState, WorkerState};
fn busy_worker(age_secs: u64) -> WorkerState {
WorkerState {
@@ -226,4 +251,45 @@ mod tests {
assert!(!report_stalls(&report, (0, 30)));
assert!(report_stalls(&report, (0, 40)));
}
fn with_sync(sync: SyncEngineState) -> Diagnostics {
Diagnostics {
sync: Some(sync),
..Default::default()
}
}
/// A sync engine that has ended stops every synced folder while the
/// workers look idle, which is what issue #53 looked like (B259).
#[test]
fn a_stopped_sync_engine_is_a_stall() {
assert!(report_stalls(
&with_sync(SyncEngineState::default()),
(0, 0)
));
}
/// A pass that keeps reporting progress is working, however long it runs;
/// one that has gone quiet past the threshold is not.
#[test]
fn only_a_quiet_sync_pass_is_a_stall() {
let busy = SyncEngineState {
running: true,
pass_folder: Some(3),
pass_age_secs: SYNC_STALL_AFTER.as_secs() * 4,
pass_idle_secs: 5,
panics: 0,
};
assert!(!report_stalls(&with_sync(busy.clone()), (0, 0)));
let quiet = SyncEngineState {
pass_idle_secs: SYNC_STALL_AFTER.as_secs(),
..busy
};
assert!(report_stalls(&with_sync(quiet), (0, 0)));
let idle = SyncEngineState {
running: true,
..Default::default()
};
assert!(!report_stalls(&with_sync(idle), (0, 0)));
}
}
+214 -8
View File
@@ -49,7 +49,9 @@ mod local_scan;
mod model;
mod planner;
pub(crate) use engine::SYNC_CONCURRENCY;
pub(crate) use engine::{SyncMsg, spawn};
pub(crate) use engine::{EngineHealth, SyncMsg, spawn};
#[cfg(test)]
use engine::{Running, burst_order, keep_wanted};
#[cfg(test)]
use engine::{classify, event_target, settle_with};
#[cfg(test)]
@@ -67,6 +69,11 @@ use planner::{
plan_file, rel_to_path, remote_sig, remote_subtree_changed, revision_moved, tracked_baseline,
unchanged_remote_size,
};
/// A pass that took at least this long is logged at info even when it changed
/// nothing, so the journal says where a slow engine spends its time.
const SLOW_PASS: Duration = Duration::from_secs(30);
/// What a [`Pending`] op did, so the reconcile can update shared state on the
/// engine thread (never inside a task).
enum Applied {
@@ -154,6 +161,14 @@ impl Core {
// The cost is one small file read against a pass that is about to walk the
// whole tree.
let rules = self.ignore_rules(&local_root);
// A first pass is the one a user waits on after adding or restoring a
// folder, so it is said at info; the poll's passes stay at debug.
if folder.last_sync == 0 {
info!(id = folder.id, path = %folder.local_path, "sync: first pass starting");
} else {
debug!(id = folder.id, "sync: pass starting");
}
let started = Instant::now();
self.progress_begin(folder.id);
let result = if push_only {
self.push_pass(folder.id, &local_root, &remote_root, &rules)
@@ -195,6 +210,18 @@ impl Core {
"idle"
};
let _ = self.db.sync_folder_set_state(folder.id, state, now_secs());
let elapsed = started.elapsed();
if folder.last_sync == 0 || !outcome.is_empty() || elapsed >= SLOW_PASS {
info!(
id = folder.id,
state,
elapsed_ms = elapsed.as_millis() as u64,
summary = %outcome.summary(),
"sync: pass finished"
);
} else {
debug!(id = folder.id, state, "sync: pass finished");
}
self.report_unscanned(folder.id, name, &outcome.unscanned);
// Summarise a pass that actually moved something. A poll over an
// unchanged folder does nothing and says nothing — otherwise the
@@ -369,7 +396,20 @@ impl Core {
// guard on every pass — wedging the folder's sync permanently — the
// moment a rule covers the whole tree.
let tracked = tracked_baseline(&baseline, rules, &unscanned);
guard_local_wipe(&tracked, &local)?;
// A local copy that is gone in its entirety is not trashed on Drive,
// for the reason `guard_local_wipe` gives. A full pass stops there, but
// this one exists for a switch to on-demand, which needs nothing local
// and leaves Drive as it is: refusing the pass would only keep the
// folder out of `idle`, wedging the switch for good (#46). So the pass
// forgets those paths instead of trashing them, and the mount then
// shows everything Drive still has.
let local_gone = match guard_local_wipe(&tracked, &local) {
Ok(()) => false,
Err(reason) => {
warn!(id = folder_id, %reason, "sync: going on-demand without trashing anything on Drive");
true
}
};
// The remote folder uids come from the baseline instead of a walk: every
// directory this device has synced recorded its uid there when it was
@@ -579,8 +619,27 @@ impl Core {
// showing the folder's remote contents — otherwise everything they deleted
// comes back the moment the switch lands. Shallowest first, skipping anything
// under a folder already trashed (its children went with it).
let deletions = local_deletions(&tracked, &local);
if local_gone {
// Forgotten instead of trashed. A row left behind would read to
// the switch as a deletion not on Drive yet and withdraw it, and
// to a later full pass as a reason to stop again. Forgotten, a full
// pass downloads the paths again.
let mut forgotten: HashSet<&str> = HashSet::new();
for rel in &deletions {
let mut parent = parent_rel(rel);
while !parent.is_empty() && !forgotten.contains(parent) {
parent = parent_rel(parent);
}
if parent.is_empty() {
let _ = self.db.sync_entries_remove_subtree(folder_id, rel);
forgotten.insert(rel);
}
}
return Ok(outcome);
}
let mut trashed: Vec<String> = Vec::new();
for rel in &local_deletions(&tracked, &local) {
for rel in &deletions {
if trashed
.iter()
.any(|dir| rel.starts_with(&format!("{dir}/")))
@@ -1103,14 +1162,22 @@ impl Core {
let uids = self
.rt
.block_on(self.drive.enumerate_folder_children_node_uids(folder))
.map_err(|e| format!("enumerate {folder}: {e}"))?;
.map_err(|e| format!("enumerate {folder}: {e}"));
let uids = match unlisted_gap(walk, prefix, uids)? {
Some(uids) => uids,
None => return Ok(()),
};
if uids.is_empty() {
return Ok(());
}
let nodes = self
.rt
.block_on(self.drive.enumerate_nodes_light(&uids))
.map_err(|e| format!("resolve nodes: {e}"))?;
.map_err(|e| format!("resolve nodes: {e}"));
let nodes = match unlisted_gap(walk, prefix, nodes)? {
Some(nodes) => nodes,
None => return Ok(()),
};
// The SDK leaves out a node it cannot decrypt, and says so only in its
// log. Its name went with it, so all this can record is the folder.
if nodes.len() < uids.len() {
@@ -1154,10 +1221,14 @@ impl Core {
let sized: HashMap<String, (i64, Option<String>)> = if stale.is_empty() {
HashMap::new()
} else {
self.rt
let full = self
.rt
.block_on(self.drive.enumerate_nodes(&stale))
.map_err(|e| format!("resolve nodes: {e}"))?
.iter()
.map_err(|e| format!("resolve nodes: {e}"));
let Some(full) = unlisted_gap(walk, prefix, full)? else {
return Ok(());
};
full.iter()
.map(|n| {
(
n.uid.to_string(),
@@ -1975,6 +2046,34 @@ struct RemoteWalk<'a> {
gaps: &'a mut RemoteGaps,
}
/// A listing call's result in [`Core::walk_remote`]. A failure below the root
/// records `prefix` as unlisted and returns `None`, so the walk goes on
/// without that folder. A failure at the root still fails the pass, since
/// nothing would be left to compare.
///
/// One listing error, after the SDK's own retries, used to fail the whole
/// walk. Nothing is applied before the walk ends, so on a link that drops a
/// request now and then a large first sync started over from nothing on
/// every pass and never got past it (`docs/BUGS.md` B271).
fn unlisted_gap<T>(
walk: &mut RemoteWalk<'_>,
prefix: &str,
result: Result<T, String>,
) -> Result<Option<T>, PassAbort> {
match result {
Ok(value) => Ok(Some(value)),
Err(error) if prefix.is_empty() => Err(error.into()),
Err(error) => {
warn!(
folder = prefix,
error, "sync: a folder on Drive could not be listed; leaving it out of this pass"
);
walk.gaps.unlisted(prefix);
Ok(None)
}
}
}
/// Whether the file at `path` is no longer what the scan saw as `planned`.
/// A file that is gone has nothing left to lose, so it has not changed.
fn changed_since_scan(path: &Path, planned: &LocalItem) -> bool {
@@ -2221,6 +2320,27 @@ pub(crate) fn mirror_tree_unsynced(
.map(|r| format!("{r} was deleted here but not on Proton Drive yet"))
}
/// Whether a mirror folder's local copy is gone in full: the folder is still
/// there, but not one tracked path of its baseline is.
///
/// [`mirror_tree_unsynced`] reads that as deletions not carried up yet, which
/// would refuse the switch to on-demand for good. It is the wipe guard's case
/// instead, which the push pass a queued switch runs settles without trashing
/// anything on Drive (#46), so the switch waits for that pass.
pub(crate) fn mirror_tree_gone(
root: &Path,
baseline: &HashMap<String, StoredSyncEntry>,
rules: &IgnoreRules,
) -> bool {
let tracked = filter_baseline(baseline, rules);
root.is_dir()
&& !tracked.is_empty()
&& tracked.keys().all(|rel| {
std::fs::symlink_metadata(root.join(rel_to_path(rel)))
.is_err_and(|e| e.kind() == std::io::ErrorKind::NotFound)
})
}
/// The walk behind both checks. `rules`, when given, only names an ignored
/// path as such in the reason; it is still something only this disk has.
fn unsynced_local(
@@ -2385,6 +2505,72 @@ fn system_mtime_ns(meta: &std::fs::Metadata) -> Option<i64> {
mod tests {
use super::*;
/// A panic in a pass ends that pass and is counted; the engine is not
/// left believing the pass still runs (`docs/BUGS.md` B259).
#[test]
fn a_panicking_pass_is_contained_and_counted() {
let health = EngineHealth::default();
assert!(!health.contain(7, || panic!("a pass that panics")));
let report = health.report();
assert_eq!(report.panics, 1);
assert_eq!(report.pass_folder, None);
assert!(health.contain(7, || {}));
assert_eq!(health.report().panics, 1);
}
/// Folders named by id run before the rest of a `ReconcileAll` in the
/// same burst, so a folder just added does not wait behind a pass over
/// every other folder (`docs/BUGS.md` B260).
#[test]
fn named_folders_run_before_the_rest_of_a_burst() {
let ids: HashSet<i64> = [9, 4].into();
assert_eq!(burst_order(&ids, &[1, 4, 6, 9]), [4, 9, 1, 6]);
assert_eq!(burst_order(&HashSet::new(), &[1, 4]), [1, 4]);
// A folder the list no longer has still gets the pass it asked for.
assert_eq!(burst_order(&[12].into(), &[1]), [12, 1]);
}
/// A folder removed and added again keeps its watch under the new id;
/// a folder that is gone loses its watch (`docs/BUGS.md` B272).
#[test]
fn a_re_added_folder_keeps_its_watch_under_the_new_id() {
let mut have = vec![(PathBuf::from("/a"), 1), (PathBuf::from("/b"), 2)];
let dropped = keep_wanted(&mut have, &[(PathBuf::from("/a"), 7)]);
assert_eq!(have, [(PathBuf::from("/a"), 7)]);
assert_eq!(dropped, [PathBuf::from("/b")]);
}
/// While a pass runs, the report names its folder, which is what tells a
/// long pass apart from an idle engine.
#[test]
fn the_running_pass_is_reported() {
let health = EngineHealth::default();
let mut seen = None;
health.contain(7, || seen = health.report().pass_folder);
assert_eq!(seen, Some(7));
}
/// The engine reads as running only while its thread lives, including
/// when the thread ends in a panic.
#[test]
fn the_engine_reads_as_stopped_once_its_thread_ends() {
let health = Arc::new(EngineHealth::default());
let thread_health = health.clone();
let (started_tx, started_rx) = std::sync::mpsc::channel();
let (end_tx, end_rx) = std::sync::mpsc::channel::<()>();
let thread = std::thread::spawn(move || {
let _running = Running::start(&thread_health);
started_tx.send(()).unwrap();
let _ = end_rx.recv();
panic!("the engine thread panics");
});
started_rx.recv().unwrap();
assert!(health.report().running);
end_tx.send(()).unwrap();
assert!(thread.join().is_err());
assert!(!health.report().running);
}
/// A removal of something already gone has produced the state the sync
/// engine wanted, so its baseline is dropped rather than retried forever.
/// Anything else stays an error and keeps the baseline (`docs/BUGS.md` B42).
@@ -3506,6 +3692,26 @@ mod tests {
assert!(reason.unwrap().contains("link"));
}
/// Emptied before any pass saw it, the folder waits for the push pass that
/// forgets those paths, instead of being refused for good (B255).
#[test]
fn only_a_local_copy_gone_in_full_counts_as_gone() {
let (root, baseline) = synced_subtree("evict-gone");
let rules = rules_for(&[]);
let whole = mirror_tree_gone(&root, &baseline, &rules);
std::fs::remove_dir_all(root.join("dir/sub")).unwrap();
let partly = mirror_tree_gone(&root, &baseline, &rules);
std::fs::remove_dir_all(root.join("dir")).unwrap();
let emptied = mirror_tree_gone(&root, &baseline, &rules);
let none_synced = mirror_tree_gone(&root, &HashMap::new(), &rules);
std::fs::remove_dir_all(&root).unwrap();
let no_root = mirror_tree_gone(&root, &baseline, &rules);
assert!(!whole && !partly, "a deletion with survivors is a real one");
assert!(emptied);
assert!(!none_synced, "an empty baseline has nothing to lose");
assert!(!no_root, "a missing folder is the pass's error to report");
}
/// The same rule one level up: if the sync root itself cannot be read, the
/// pass sees nothing, and "nothing" must never mean "delete everything".
#[test]
+253 -72
View File
@@ -1,5 +1,11 @@
//! Filesystem watcher, polling, and debounce control plane.
use std::panic::AssertUnwindSafe;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use pdfs_core::control::SyncEngineState;
use tracing::error;
use super::*;
/// How long the watcher must go quiet before a burst counts as finished.
@@ -23,6 +29,9 @@ const POLL_INTERVAL: Duration = Duration::from_secs(120);
/// unusable done one-at-a-time; this bounds concurrency like the bulk-upload
/// engine does.
pub(crate) const SYNC_CONCURRENCY: usize = 8;
/// How long after a failed watch the engine tries the watches again. Each try
/// walks the folder's whole tree, so not on every burst.
const REWATCH_RETRY: Duration = Duration::from_secs(600);
/// A message to the sync engine's control thread.
pub(crate) enum SyncMsg {
@@ -53,32 +62,159 @@ pub(crate) fn spawn(core: Core, rx: Receiver<SyncMsg>) -> Option<std::thread::Jo
}
}
/// Whether the engine is alive and which pass it is on, for
/// [`Core::diagnostics`] and the supervisor.
///
/// A panic in a pass used to end the engine thread. Every later add, sync-now,
/// file event and poll then went into a channel nobody read, the folder stayed
/// `syncing`, and nothing said so until a restart (`docs/BUGS.md` B259).
#[derive(Default)]
pub(crate) struct EngineHealth {
running: AtomicBool,
pass: Mutex<Option<PassClock>>,
panics: AtomicU64,
}
#[derive(Clone, Copy)]
struct PassClock {
folder: i64,
started: Instant,
moved: Instant,
}
impl EngineHealth {
/// Note that the running pass reported progress.
pub(crate) fn moved(&self) {
if let Some(pass) = self.pass.lock().as_mut() {
pass.moved = Instant::now();
}
}
/// The engine's state now. `try_lock`, like the rest of the diagnostics:
/// a report must not wait on the thing it reports about.
pub(crate) fn report(&self) -> SyncEngineState {
let pass = self.pass.try_lock().and_then(|pass| *pass);
SyncEngineState {
running: self.running.load(Ordering::Relaxed),
pass_folder: pass.map(|p| p.folder),
pass_age_secs: pass.map_or(0, |p| p.started.elapsed().as_secs()),
pass_idle_secs: pass.map_or(0, |p| p.moved.elapsed().as_secs()),
panics: self.panics.load(Ordering::Relaxed),
}
}
/// Run `pass` over `folder` as the current pass. Returns `false` when it
/// panicked: the panic ends that pass, not the engine.
pub(super) fn contain(&self, folder: i64, pass: impl FnOnce()) -> bool {
let now = Instant::now();
*self.pass.lock() = Some(PassClock {
folder,
started: now,
moved: now,
});
let finished = std::panic::catch_unwind(AssertUnwindSafe(pass)).is_ok();
*self.pass.lock() = None;
if !finished {
self.panics.fetch_add(1, Ordering::Relaxed);
}
finished
}
}
/// Marks the engine running for as long as it lives.
pub(super) struct Running<'a>(&'a EngineHealth);
impl<'a> Running<'a> {
pub(super) fn start(health: &'a EngineHealth) -> Self {
health.running.store(true, Ordering::Relaxed);
Self(health)
}
}
impl Drop for Running<'_> {
fn drop(&mut self) {
self.0.running.store(false, Ordering::Relaxed);
if std::thread::panicking() {
error!("sync engine stopped by a panic; nothing syncs until the daemon restarts");
}
}
}
/// One folder's pass, contained: a panic in it sets the folder to `error` and
/// leaves the engine running for the next one.
fn run_pass(core: &Core, folder: &StoredSyncFolder) {
if core
.sync_engine
.contain(folder.id, || core.reconcile_folder(folder))
{
return;
}
core.progress_end(folder.id);
let _ = core
.db
.sync_folder_set_state(folder.id, "error", folder.last_sync);
error!(
id = folder.id,
path = %folder.local_path,
"sync: pass panicked; the folder is set to error and tried again on the next poll"
);
core.log_activity(
ActivityKind::Sync,
format!("sync failed for {}", base_name(&folder.local_path)),
"an internal error stopped the pass; it is tried again on the next poll",
false,
);
}
/// The pass a `Reconcile(id)` asked for. A folder removed since, or one the
/// database cannot read, is said so rather than dropped in silence.
fn run_pass_by_id(core: &Core, id: i64) {
match core.db.sync_folder_get(id) {
Ok(Some(folder)) => run_pass(core, &folder),
Ok(None) => debug!(id, "sync: folder removed before its pass; skipping"),
Err(e) => warn!(id, error = ?e, "sync: cannot read folder; skipping its pass"),
}
}
/// The engine control loop. Single-threaded so reconcile passes never overlap.
fn engine_loop(core: Core, rx: Receiver<SyncMsg>) {
let health = core.sync_engine.clone();
let _running = Running::start(&health);
// Paths the watcher currently covers, mapped to their folder id, so an event
// path can be resolved back to the folder it belongs to.
let watched: Arc<Mutex<Vec<(PathBuf, i64)>>> = Arc::new(Mutex::new(Vec::new()));
let tx_events = core.sync_tx.clone();
let watched_events = watched.clone();
let mut watcher =
match notify::recommended_watcher(move |res: notify::Result<notify::Event>| {
let Ok(event) = res else { return };
// Ignore pure metadata/access noise; content changes are what matter.
if matches!(event.kind, notify::EventKind::Access(_)) {
return;
}
let msg = event_target(&watched_events.lock(), &event.paths);
let _ = tx_events.send(msg);
}) {
Ok(w) => w,
let handler = move |res: notify::Result<notify::Event>| {
let event = match res {
Ok(event) => event,
// Running out of inotify watches for a new subfolder, say: that
// subfolder's changes are then found by the poll only (B262).
Err(e) => {
warn!(error = %e, "filesystem watcher unavailable; sync is poll-only");
// Still run the loop so polling and explicit SyncNow work.
poll_only_loop(&core, rx);
warn!(error = %e, "sync: filesystem watcher error");
return;
}
};
// Ignore pure metadata/access noise; content changes are what matter.
if matches!(event.kind, notify::EventKind::Access(_)) {
return;
}
let msg = event_target(&watched_events.lock(), &event.paths);
let _ = tx_events.send(msg);
};
// The local walk skips symlinks, so following them here would only watch
// trees outside the folder, and run passes over it for changes made there.
let config = notify::Config::default().with_follow_symlinks(false);
let mut watcher = match notify::RecommendedWatcher::new(handler, config) {
Ok(w) => w,
Err(e) => {
warn!(error = %e, "filesystem watcher unavailable; sync is poll-only");
// Still run the loop so polling and explicit SyncNow work.
poll_only_loop(&core, rx);
return;
}
};
// Periodic remote poll.
{
@@ -96,8 +232,9 @@ fn engine_loop(core: Core, rx: Receiver<SyncMsg>) {
.ok();
}
rewatch(&core, &mut watcher, &watched);
reconcile_all(&core);
let mut unwatched = rewatch(&core, &mut watcher, &watched);
let mut last_rewatch = Instant::now();
run_burst(&core, &HashSet::new(), true);
while let Ok(msg) = rx.recv() {
let mut ids: HashSet<i64> = HashSet::new();
@@ -110,21 +247,16 @@ fn engine_loop(core: Core, rx: Receiver<SyncMsg>) {
return;
}
// A folder whose watch failed is tried again now and then, not only
// when the folder set changes.
if unwatched && last_rewatch.elapsed() >= REWATCH_RETRY {
do_rewatch = true;
}
if do_rewatch {
rewatch(&core, &mut watcher, &watched);
}
if all {
reconcile_all(&core);
} else {
for id in ids {
if core.sync_paused() {
break;
}
if let Ok(Some(folder)) = core.db.sync_folder_get(id) {
core.reconcile_folder(&folder);
}
}
unwatched = rewatch(&core, &mut watcher, &watched);
last_rewatch = Instant::now();
}
run_burst(&core, &ids, all);
}
}
@@ -141,7 +273,7 @@ fn poll_only_loop(core: &Core, rx: Receiver<SyncMsg>) {
}
});
}
reconcile_all(core);
run_burst(core, &HashSet::new(), true);
while let Ok(msg) = rx.recv() {
let mut ids: HashSet<i64> = HashSet::new();
let mut all = false;
@@ -152,21 +284,59 @@ fn poll_only_loop(core: &Core, rx: Receiver<SyncMsg>) {
info!("sync engine stopping");
return;
}
if all {
reconcile_all(core);
} else {
for id in ids {
if core.sync_paused() {
break;
}
if let Ok(Some(folder)) = core.db.sync_folder_get(id) {
core.reconcile_folder(&folder);
}
run_burst(core, &ids, all);
}
}
/// Run the passes one burst asked for: the folders named by id, then, when the
/// burst also holds a `ReconcileAll`, every other folder.
///
/// The named folders go first. A poll's `ReconcileAll` in the same burst used
/// to take their place, and a folder just added, which has the highest id,
/// waited behind a full pass over every other folder (`docs/BUGS.md` B260).
///
/// Skipped while sync is paused, and cut short when a pause lands between
/// passes: the resume sends a fresh [`SyncMsg::ReconcileAll`], which walks
/// everything the watcher reported meanwhile.
fn run_burst(core: &Core, ids: &HashSet<i64>, all: bool) {
if core.sync_paused() {
debug!("sync: paused, skipping reconcile");
return;
}
let folders = if all {
match core.db.sync_folder_list() {
Ok(folders) => folders,
Err(e) => {
warn!(error = ?e, "sync: cannot list folders");
Vec::new()
}
}
} else {
Vec::new()
};
let listed: Vec<i64> = folders.iter().map(|f| f.id).collect();
for id in burst_order(ids, &listed) {
if core.sync_paused() {
break;
}
// A folder named by id is read again: it may have been added or
// changed after the list was read.
match folders.iter().find(|f| f.id == id) {
Some(folder) if !ids.contains(&id) => run_pass(core, folder),
_ => run_pass_by_id(core, id),
}
}
}
/// The order of a burst's passes: the folders named by id, lowest first, then
/// the rest of `listed` in its own order.
pub(super) fn burst_order(ids: &HashSet<i64>, listed: &[i64]) -> Vec<i64> {
let mut order: Vec<i64> = ids.iter().copied().collect();
order.sort_unstable();
order.extend(listed.iter().copied().filter(|id| !ids.contains(id)));
order
}
/// Absorb the rest of an event burst, returning once the watcher has been quiet
/// for `quiet` or `cap` has elapsed since the first event — whichever comes
/// first. The caller has already classified the event that opened the burst.
@@ -238,26 +408,6 @@ pub(super) fn classify(
true
}
/// Reconcile every mirror folder in turn. Skipped while sync is paused: the
/// resume sends a fresh [`SyncMsg::ReconcileAll`], which walks everything the
/// watcher reported meanwhile.
fn reconcile_all(core: &Core) {
if core.sync_paused() {
debug!("sync: paused, skipping reconcile");
return;
}
let folders = match core.db.sync_folder_list() {
Ok(f) => f,
Err(e) => {
warn!(error = ?e, "sync: cannot list folders");
return;
}
};
for folder in folders {
core.reconcile_folder(&folder);
}
}
/// The reconcile a watcher event asks for: the one mirror folder all its paths
/// lie in, so an edit in one folder does not walk every other one too. An event
/// that spans folders, or that no watched folder claims (a watch list a moment
@@ -278,7 +428,8 @@ pub(super) fn event_target(watched: &[(PathBuf, i64)], paths: &[PathBuf]) -> Syn
}
/// Bring the filesystem watches in line with the current mirror-folder set:
/// watch newly-added folders, drop removed ones.
/// watch newly-added folders, drop removed ones. Returns whether a folder is
/// left unwatched because its watch failed.
///
/// The watch calls run on a copy, outside the lock: the watcher's event
/// callback takes the same lock, and a backend that hands a watch request to
@@ -287,7 +438,7 @@ fn rewatch(
core: &Core,
watcher: &mut notify::RecommendedWatcher,
watched: &Mutex<Vec<(PathBuf, i64)>>,
) {
) -> bool {
let want: Vec<(PathBuf, i64)> = match core.db.sync_folder_list() {
Ok(f) => f
.into_iter()
@@ -296,30 +447,60 @@ fn rewatch(
.collect(),
Err(e) => {
warn!(error = ?e, "sync: cannot list folders to watch");
return;
return false;
}
};
let mut have = watched.lock().clone();
// Drop watches no longer wanted.
have.retain(|(path, _)| {
if want.iter().any(|(p, _)| p == path) {
true
} else {
let _ = watcher.unwatch(path);
false
}
});
for path in keep_wanted(&mut have, &want) {
let _ = watcher.unwatch(&path);
}
// Add newly wanted watches.
let mut failed = false;
for (path, id) in &want {
if have.iter().any(|(p, _)| p == path) {
continue;
}
match watcher.watch(path, RecursiveMode::Recursive) {
Ok(()) => have.push((path.clone(), *id)),
Err(e) => warn!(path = %path.display(), error = %e, "sync: watch failed"),
Err(e) => {
// A recursive watch that fails partway keeps the watches it had
// added. Their events would match no folder and run a pass over
// every one, and they hold watches the next folder needs (B262).
let _ = watcher.unwatch(path);
failed = true;
warn!(
path = %path.display(),
error = %e,
"sync: watch failed; the poll finds this folder's changes until a retry \
works. A low fs.inotify.max_user_watches is the usual cause"
);
}
}
}
*watched.lock() = have;
failed
}
/// Keep the watches in `have` that `want` still asks for, and return the paths
/// of the others, to unwatch.
///
/// A path is matched alone, but takes the id `want` gives it: a folder removed
/// and added again in one burst keeps its watch, and its events go to the new
/// row rather than to the removed one (`docs/BUGS.md` B272).
pub(super) fn keep_wanted(have: &mut Vec<(PathBuf, i64)>, want: &[(PathBuf, i64)]) -> Vec<PathBuf> {
let mut dropped = Vec::new();
have.retain_mut(|(path, id)| match want.iter().find(|(p, _)| p == path) {
Some((_, wanted)) => {
*id = *wanted;
true
}
None => {
dropped.push(path.clone());
false
}
});
dropped
}
// ---- reconcile ------------------------------------------------------------
+43 -4
View File
@@ -192,7 +192,8 @@ pub(super) fn guard_local_wipe<B, L>(
if !baseline.is_empty() && baseline.keys().all(|rel| !local.contains_key(rel)) {
return Err(format!(
"every one of the {} synced paths is missing locally; refusing to trash \
them on Drive. Check that the folder is mounted and readable.",
them on Drive. Check that the folder is mounted and readable; to download \
them again, sync the folder from scratch.",
baseline.len()
));
}
@@ -213,7 +214,8 @@ pub(super) fn guard_remote_wipe<B, R>(
if !baseline.is_empty() && baseline.keys().all(|rel| !remote.contains_key(rel)) {
return Err(format!(
"every one of the {} synced paths is missing on Proton Drive; refusing to \
delete them here. If they were deleted on purpose, remove the folder from sync.",
delete them here. If they were deleted on purpose, remove the folder from sync; \
to upload them again, sync the folder from scratch.",
baseline.len()
));
}
@@ -234,6 +236,9 @@ pub(super) struct RemoteGaps {
short: HashSet<String>,
/// Paths seen, but whose entry could not be read.
unread: HashSet<String>,
/// Folders whose listing failed outright, after the SDK's retries. Nothing
/// under one is known, so all of it is left out of the pass (B271).
unlisted: HashSet<String>,
}
impl RemoteGaps {
@@ -247,8 +252,13 @@ impl RemoteGaps {
self.unread.insert(rel);
}
/// The listing of `folder` failed, so nothing under it was walked.
pub(super) fn unlisted(&mut self, folder: &str) {
self.unlisted.insert(folder.to_string());
}
pub(super) fn is_empty(&self) -> bool {
self.short.is_empty() && self.unread.is_empty()
self.short.is_empty() && self.unread.is_empty() && self.unlisted.is_empty()
}
/// Whether `rel` may be missing from `remote` only because the walk could
@@ -270,7 +280,7 @@ impl RemoteGaps {
let mut at = parent_rel(rel);
loop {
if remote_dirs.contains_key(at) {
return self.short.contains(at);
return self.short.contains(at) || self.unlisted.contains(at);
}
if at.is_empty() {
return false;
@@ -293,6 +303,9 @@ impl RemoteGaps {
.iter()
.map(|rel| format!("{rel} (cannot be read on Proton Drive)")),
)
.chain(self.unlisted.iter().map(|rel| {
format!("{rel} (could not be listed on Proton Drive; tried again next pass)")
}))
.collect();
lines.sort_unstable();
lines
@@ -842,6 +855,32 @@ mod file_plan_tests {
assert!(gaps.hides("top.txt", &remote, &dirs));
}
/// A folder whose listing failed was walked no further: nothing under it
/// reads as deleted on Drive, and a sibling that listed still does
/// (`docs/BUGS.md` B271).
#[test]
fn a_folder_that_failed_to_list_hides_everything_under_it() {
let remote = HashMap::from([
("docs".to_string(), folder()),
("pics".to_string(), folder()),
]);
let dirs = HashMap::from([
(String::new(), uid("root")),
("docs".to_string(), uid("docs")),
("pics".to_string(), uid("pics")),
]);
let mut gaps = RemoteGaps::default();
gaps.unlisted("docs");
assert!(gaps.hides("docs/a.txt", &remote, &dirs));
assert!(gaps.hides("docs/sub/deep.txt", &remote, &dirs));
assert!(!gaps.hides("docs", &remote, &dirs));
assert!(!gaps.hides("pics/gone.jpg", &remote, &dirs));
assert_eq!(
gaps.report(),
["docs (could not be listed on Proton Drive; tried again next pass)"]
);
}
#[test]
fn a_file_whose_size_could_not_be_read_is_hidden() {
let remote: HashMap<String, RemoteItem> = HashMap::new();
+9 -1
View File
@@ -64,11 +64,19 @@ fn send_abstract(socket: &UnixDatagram, name: &str, message: &str) -> bool {
/// so `systemctl --user start` returns when the mount actually works rather than
/// when the process exists.
pub(crate) fn ready() {
if notify("READY=1\n") {
if notify("READY=1\nSTATUS=Mounted\n") {
debug!("notified systemd: ready");
}
}
/// Set the one-line status `systemctl --user status` shows for the unit, such
/// as why the mount keeps failing (`docs/BUGS.md` B263).
pub fn status(text: &str) {
// A newline would end the `STATUS=` line and start another variable.
let line = text.replace('\n', " ");
notify(&format!("STATUS={line}\n"));
}
/// Tell the manager the daemon is still healthy.
pub(crate) fn watchdog_ping() {
let _ = notify("WATCHDOG=1\n");
+34 -12
View File
@@ -75,11 +75,11 @@ use pdfs_core::config::{AppDirs, FileSort, FilesView, WindowState};
use pdfs_core::control::{
ActivityEntry, ActivityKind, AlbumInfo, BookmarkInfo, ConflictInfo, ConflictKeep, DeviceInfo,
DirEntry, ErrorKind, ImportSummary, InvitationInfo, JobItem, MappingProgress, PendingOpInfo,
PhotoItem, PhotoKind, PhotoMonth, PlaceInfo, PublicLinkInfo, RefreshScope, Request, Response,
RestorableFolder, RestoreItem, RevisionInfo, SearchHit, ShareEntry, ShareEntryKind, SharedItem,
SyncFolderInfo, SyncIssue, SyncPhase, SyncProgress, ThumbnailBuildStatus, Topic,
TransferDirection, TransferItem, send,
DirEntry, ErrorKind, ImportSummary, InvitationInfo, JobItem, LinkChange, MappingProgress,
PendingOpInfo, PhotoItem, PhotoKind, PhotoMonth, PlaceInfo, PublicLinkInfo, RefreshScope,
Request, Response, RestorableFolder, RestoreItem, RevisionInfo, SearchHit, ShareEntry,
ShareEntryKind, SharedItem, SyncFolderInfo, SyncIssue, SyncPhase, SyncProgress,
ThumbnailBuildStatus, Topic, TransferDirection, TransferItem, send,
};
use pdfs_core::mounts::{MountAccess, MountKind, MountMode, MountSpec};
@@ -1797,13 +1797,35 @@ pub(crate) fn service_unreachable(
);
return;
}
status(
&ui,
"network-idle-symbolic",
&gettext("Connecting…"),
&gettext("Waiting for the Proton Drive service to start."),
false,
);
// Running but not answering may be a mount that fails on every try,
// which never opens the socket. Its reason is in a file the daemon
// keeps for this (B263); "Connecting…" alone looked like a slow start
// for as long as it lasted.
let state_dir = ui.dirs.state_dir();
let failure = gio::spawn_blocking(move || pdfs_core::mountfail::read(&state_dir))
.await
.ok()
.flatten();
match failure {
Some(failure) => status(
&ui,
"dialog-warning-symbolic",
&gettext("Can't open your Drive"),
// Translators: {error} is an error message, in English.
&gettext_f(
"The Proton Drive service keeps failing to start and tries again on its own. The last error was: {error}",
&[("error", &failure.error)],
),
true,
),
None => status(
&ui,
"network-idle-symbolic",
&gettext("Connecting…"),
&gettext("Waiting for the Proton Drive service to start."),
false,
),
}
glib::timeout_add_local_once(CONNECT_RETRY_INTERVAL, move || {
if ui.stack.visible_child_name().as_deref() == Some(page) {
reload(&ui);
@@ -1043,6 +1043,7 @@ pub(crate) fn activity_icon(kind: ActivityKind) -> &'static str {
ActivityKind::Sync => "pdfs-sync-symbolic",
ActivityKind::Rename => "document-edit-symbolic",
ActivityKind::Move => "go-jump-symbolic",
ActivityKind::Copy => "edit-copy-symbolic",
ActivityKind::CreateFolder => "folder-new-symbolic",
ActivityKind::Trash => "user-trash-symbolic",
ActivityKind::DeleteForever | ActivityKind::EmptyTrash => "edit-delete-symbolic",
@@ -179,6 +179,10 @@ pub(crate) fn activity_title(kind: ActivityKind, ok: bool, name: &str) -> String
(ActivityKind::Move, true) => gettext_f("Moved {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Move, false) => gettext_f("Couldn't move {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Copy, true) => gettext_f("Copied {name}", &args),
// Translators: {name} is a file or folder name.
(ActivityKind::Copy, false) => gettext_f("Couldn't copy {name}", &args),
// Translators: {name} is a folder name.
(ActivityKind::CreateFolder, true) => gettext_f("Created folder {name}", &args),
// Translators: {name} is a folder name.
+200 -46
View File
@@ -904,6 +904,12 @@ pub(crate) fn entry_context_menu(ui: &Rc<Ui>, entry: &DirEntry) -> ActionMenu {
menu.item(&gettext("Move To…"), move || {
prompt_move(&ui_c, vec![entry_c.clone()])
});
let (ui_c, entry_c) = (ui.clone(), entry.clone());
menu.item(&gettext("Make a Copy"), move || make_copy(&ui_c, &entry_c));
let (ui_c, entry_c) = (ui.clone(), entry.clone());
menu.item(&gettext("Copy To…"), move || {
prompt_copy(&ui_c, vec![entry_c.clone()])
});
if !entry.is_dir {
let (ui_c, entry_c) = (ui.clone(), entry.clone());
menu.item(&gettext("Versions…"), move || {
@@ -938,6 +944,10 @@ pub(crate) fn bulk_context_menu(ui: &Rc<Ui>, entries: Vec<DirEntry>) -> ActionMe
menu.item(&gettext("Move To…"), move || {
prompt_move(&ui_c, batch.clone())
});
let (ui_c, batch) = (ui.clone(), entries.clone());
menu.item(&gettext("Copy To…"), move || {
prompt_copy(&ui_c, batch.clone())
});
menu.labelled_section(&ngettext_f(
"{n} selected",
"{n} selected",
@@ -1940,17 +1950,63 @@ pub(crate) fn prompt_rename(ui: &Rc<Ui>, entry: &DirEntry) {
/// still made on Proton Drive, so nothing is downloaded or uploaded again; it
/// is sent with absolute paths, which is how the daemon tells locations apart.
pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
prompt_destination(ui, entries, PickerKind::Move);
}
/// Ask where to copy `entries`, then copy them there on Proton Drive. Only
/// My files is offered: a copy is made on the server, inside the mount.
pub(crate) fn prompt_copy(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
prompt_destination(ui, entries, PickerKind::Copy);
}
/// Copy one entry into the folder it is in. The copy gets a free name, such
/// as "report (1).pdf".
pub(crate) fn make_copy(ui: &Rc<Ui>, entry: &DirEntry) {
let source = entry_rel(ui, entry);
let parent = source
.rfind('/')
.map(|i| source[..i].to_string())
.unwrap_or_default();
run_bulk_copy(ui, vec![source], parent);
}
/// What the folder picker chooses a destination for.
#[derive(Clone, Copy, Debug, PartialEq)]
pub(crate) enum PickerKind {
Move,
Copy,
}
fn prompt_destination(ui: &Rc<Ui>, entries: Vec<DirEntry>, kind: PickerKind) {
if entries.is_empty() {
return;
}
let sources: Vec<String> = entries.iter().map(|e| entry_rel(ui, e)).collect();
let heading = match entries.as_slice() {
[one] => gettext_f("Move “{name}”", &[("name", &one.name)]),
many => ngettext_f("Move {n} item", "Move {n} items", many.len() as u64, &[]),
let (heading, body, confirm) = match (kind, entries.as_slice()) {
(PickerKind::Move, [one]) => (
gettext_f("Move “{name}”", &[("name", &one.name)]),
gettext("Choose the folder to move into."),
gettext("Move Here"),
),
(PickerKind::Move, many) => (
ngettext_f("Move {n} item", "Move {n} items", many.len() as u64, &[]),
gettext("Choose the folder to move into."),
gettext("Move Here"),
),
(PickerKind::Copy, [one]) => (
gettext_f("Copy “{name}”", &[("name", &one.name)]),
gettext("Choose the folder to copy into."),
gettext("Copy Here"),
),
(PickerKind::Copy, many) => (
ngettext_f("Copy {n} item", "Copy {n} items", many.len() as u64, &[]),
gettext("Choose the folder to copy into."),
gettext("Copy Here"),
),
};
let dialog = adw::AlertDialog::builder()
.heading(heading)
.body(gettext("Choose the folder to move into."))
.body(body)
.build();
let up = gtk4::Button::builder()
@@ -1981,7 +2037,7 @@ pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
body.append(&scroll);
dialog.set_extra_child(Some(&body));
dialog.add_response("cancel", &gettext("Cancel"));
dialog.add_response("move", &gettext("Move Here"));
dialog.add_response("move", &confirm);
dialog.set_response_appearance("move", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("move"));
dialog.set_close_response("cancel");
@@ -1994,6 +2050,7 @@ pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
list,
up: up.clone(),
sources: sources.clone(),
kind,
folder: RefCell::new(MoveTarget::Locations),
generation: Cell::new(0),
});
@@ -2013,6 +2070,10 @@ pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
}
match picker_done.folder.borrow().clone() {
MoveTarget::Locations => {}
MoveTarget::Drive(folder) if kind == PickerKind::Copy => {
run_bulk_copy(&ui, sources.clone(), folder)
}
MoveTarget::Synced { .. } if kind == PickerKind::Copy => {}
MoveTarget::Drive(folder) => run_bulk_move(&ui, sources.clone(), folder),
MoveTarget::Synced { path, .. } => {
let absolute = sources
@@ -2088,8 +2149,9 @@ struct MovePicker {
location: gtk4::Label,
list: gtk4::ListBox,
up: gtk4::Button,
/// What is being moved, mountpoint-relative.
/// What is being moved or copied, mountpoint-relative.
sources: Vec<String>,
kind: PickerKind,
/// The folder on show, and the destination if the user confirms now.
folder: RefCell<MoveTarget>,
/// Drops a listing that arrives after the user went elsewhere.
@@ -2110,11 +2172,13 @@ impl MovePicker {
self.location.set_tooltip_text(Some(&tooltip));
self.up
.set_sensitive(!matches!(folder, MoveTarget::Locations));
let allowed = match &folder {
MoveTarget::Locations => false,
MoveTarget::Drive(rel) => move_target_allowed(&self.sources, rel),
let allowed = match (&folder, self.kind) {
(MoveTarget::Locations, _) => false,
(MoveTarget::Drive(rel), PickerKind::Move) => move_target_allowed(&self.sources, rel),
(MoveTarget::Drive(rel), PickerKind::Copy) => copy_target_allowed(&self.sources, rel),
// What is moved is in My files, so it cannot be in a synced folder.
MoveTarget::Synced { .. } => true,
(MoveTarget::Synced { .. }, PickerKind::Move) => true,
(MoveTarget::Synced { .. }, PickerKind::Copy) => false,
};
self.dialog.set_response_enabled("move", allowed);
*self.folder.borrow_mut() = folder.clone();
@@ -2180,6 +2244,10 @@ impl MovePicker {
/// My files and every synced folder, as rows of the top level.
async fn locations(&self) -> Option<Vec<PickerRow>> {
let mut rows = vec![(gettext("My files"), None, MoveTarget::Drive(String::new()))];
// A copy is made on the server, so it can only land in My files.
if self.kind == PickerKind::Copy {
return Some(rows);
}
let rx = spawn_request(self.ui.dirs.control_socket(), Request::ListSyncFolders);
let Ok(Ok(Response::SyncFolders { mut items })) = rx.recv().await else {
// My files alone is still somewhere to move to.
@@ -2286,11 +2354,16 @@ pub(crate) fn move_target_allowed(sources: &[String], target: &str) -> bool {
.map(|i| path[..i].to_string())
.unwrap_or_default()
};
let inside = sources
.iter()
.any(|src| target == src || target.starts_with(&format!("{src}/")));
let all_here = sources.iter().all(|src| parent(src) == target);
!inside && !all_here
copy_target_allowed(sources, target) && !all_here
}
/// Whether `target` is somewhere `sources` can be copied to: anywhere but into
/// one of themselves. The folder they are in is fine; the copy gets a free name.
pub(crate) fn copy_target_allowed(sources: &[String], target: &str) -> bool {
!sources
.iter()
.any(|src| target == src || target.starts_with(&format!("{src}/")))
}
/// Move each of `sources` into `target`, one request at a time, then report
@@ -2309,38 +2382,14 @@ pub(crate) fn run_bulk_move(ui: &Rc<Ui>, sources: Vec<String>, target: String) {
clear_selection(ui);
let ui = ui.clone();
glib::spawn_future_local(async move {
let mut done = 0usize;
let mut failure: Option<(String, ErrorKind)> = None;
for path in sources {
let rx = spawn_request(
socket.clone(),
Request::Move {
path,
new_parent: target.clone(),
},
);
match rx.recv().await {
Ok(Ok(Response::Ok { .. })) => done += 1,
Ok(Ok(Response::Error { message, kind })) => {
failure.get_or_insert((message, kind));
}
_ => {
failure.get_or_insert_with(|| {
(
gettext("The Proton Drive service didn't respond."),
ErrorKind::Internal,
)
});
}
}
}
let (done, failure) = send_each(&socket, sources, |path| Request::Move {
path,
new_parent: target.clone(),
})
.await;
ui.busy_end();
reload_listing(&ui);
let place = match target.rsplit('/').next() {
// Translators: a folder name in quotes, filled in as {place} in "Moved to {place}".
Some(name) if !name.is_empty() => gettext_f("“{name}”", &[("name", name)]),
_ => "Proton Drive".to_string(),
};
let place = quoted_place(&target);
match (done, failure) {
(0, Some((message, kind))) => {
toast_failure(&ui, &gettext("Couldn't move"), &message, kind)
@@ -2373,6 +2422,101 @@ pub(crate) fn run_bulk_move(ui: &Rc<Ui>, sources: Vec<String>, target: String) {
});
}
/// Copy each of `sources` into `target` on Proton Drive, one request at a
/// time, then report once for the lot.
pub(crate) fn run_bulk_copy(ui: &Rc<Ui>, sources: Vec<String>, target: String) {
if !*ui.mounted.borrow() {
toast_error(
ui,
&gettext("Couldn't copy"),
&gettext("Proton Drive isn't connected."),
);
return;
}
let socket = ui.dirs.control_socket();
ui.busy_begin();
clear_selection(ui);
let ui = ui.clone();
glib::spawn_future_local(async move {
let (done, failure) = send_each(&socket, sources, |path| Request::Copy {
path,
new_parent: target.clone(),
})
.await;
ui.busy_end();
reload_listing(&ui);
let place = quoted_place(&target);
match (done, failure) {
(0, Some((message, kind))) => {
toast_failure(&ui, &gettext("Couldn't copy"), &message, kind)
}
(0, None) => {}
(n, Some((message, _))) => toast_error(
&ui,
// Translators: {place} is a quoted folder name or "Proton Drive".
&ngettext_f(
"Copied {n} item to {place}, but not all",
"Copied {n} items to {place}, but not all",
n as u64,
&[("place", &place)],
),
&message,
),
// Translators: {place} is a quoted folder name or "Proton Drive".
(1, None) => toast(&ui, &gettext_f("Copied to {place}", &[("place", &place)])),
(n, None) => toast(
&ui,
// Translators: {place} is a quoted folder name or "Proton Drive".
&ngettext_f(
"Copied {n} item to {place}",
"Copied {n} items to {place}",
n as u64,
&[("place", &place)],
),
),
}
});
}
/// Send one request per source, in order, and count how many succeeded. The
/// first failure is kept for the report.
async fn send_each(
socket: &Path,
sources: Vec<String>,
request: impl Fn(String) -> Request,
) -> (usize, Option<(String, ErrorKind)>) {
let mut done = 0usize;
let mut failure: Option<(String, ErrorKind)> = None;
for path in sources {
let rx = spawn_request(socket.to_path_buf(), request(path));
match rx.recv().await {
Ok(Ok(Response::Ok { .. })) => done += 1,
Ok(Ok(Response::Error { message, kind })) => {
failure.get_or_insert((message, kind));
}
_ => {
failure.get_or_insert_with(|| {
(
gettext("The Proton Drive service didn't respond."),
ErrorKind::Internal,
)
});
}
}
}
(done, failure)
}
/// The destination folder for a report: its name in quotes, or "Proton Drive"
/// for My files.
fn quoted_place(target: &str) -> String {
match target.rsplit('/').next() {
// Translators: a folder name in quotes, filled in as {place} in "Moved to {place}".
Some(name) if !name.is_empty() => gettext_f("“{name}”", &[("name", name)]),
_ => "Proton Drive".to_string(),
}
}
/// Move one entry to Trash, through the batch path so it gets the same Undo.
pub(crate) fn trash_entry(ui: &Rc<Ui>, entry: &DirEntry) {
run_bulk_delete(ui, vec![entry.clone()]);
@@ -2978,8 +3122,8 @@ fn listing_summary(files: usize, folders: usize, file_bytes: u64) -> String {
#[cfg(test)]
mod tests {
use super::{
MoveTarget, badge_for, hit_location, listing_summary, move_target_allowed,
show_thumbnail_build_progress, sort_entries,
MoveTarget, badge_for, copy_target_allowed, hit_location, listing_summary,
move_target_allowed, show_thumbnail_build_progress, sort_entries,
};
use pdfs_core::config::{FileSort, FilesView};
use pdfs_core::control::{DirEntry, SyncIssue, ThumbnailBuildStatus};
@@ -3067,6 +3211,16 @@ mod tests {
assert!(move_target_allowed(&mixed, ""));
}
#[test]
fn copy_refuses_its_own_subtree_but_not_the_folder_it_is_in() {
let sources = ["Docs/Work".to_string()];
assert!(!copy_target_allowed(&sources, "Docs/Work"));
assert!(!copy_target_allowed(&sources, "Docs/Work/Old"));
assert!(copy_target_allowed(&sources, "Docs"));
assert!(copy_target_allowed(&sources, ""));
assert!(copy_target_allowed(&sources, "Docs/Workshop"));
}
#[test]
fn listing_summary_matches_dolphin_order_and_wording() {
assert_eq!(listing_summary(0, 0, 0), "0 folders, 0 files");
+94 -6
View File
@@ -597,6 +597,19 @@ pub(crate) fn this_computer_subtitle(folders: &[SyncFolderInfo]) -> String {
folders.len() as u64,
&[],
);
// A folder whose first pass has not run is not backed up yet, whatever its
// state column says (`docs/BUGS.md` B261).
let waiting = folders.iter().filter(|f| never_synced(f)).count();
let count = match waiting {
0 => count,
// Translators: {folders} is "N folders backed up"; {n} counts the folders whose first sync has not run yet.
n => ngettext_f(
"{folders} · {n} waiting for its first sync",
"{folders} · {n} waiting for their first sync",
n as u64,
&[("folders", &count)],
),
};
let attention = folders
.iter()
.filter(|f| f.state == "error" || f.state == "conflict")
@@ -613,12 +626,21 @@ pub(crate) fn this_computer_subtitle(folders: &[SyncFolderInfo]) -> String {
}
}
/// Human label for a synced folder's `state` column.
pub(crate) fn sync_state_label(state: &str) -> String {
/// Whether a synced folder is a mirror whose first pass has not finished.
/// Its `state` starts out `idle`, which would read as "up to date".
fn never_synced(folder: &SyncFolderInfo) -> bool {
folder.mode == "mirror" && folder.last_sync == 0 && folder.state == "idle"
}
/// Human label for a synced folder's `state` column. `never_synced` is a
/// mirror folder whose first pass has not finished: its `state` starts out
/// `idle`, and "up to date" would say the opposite of the truth (B261).
pub(crate) fn sync_state_label(state: &str, never_synced: bool) -> String {
match state {
"syncing" => gettext("syncing…"),
"error" => gettext("sync error"),
"conflict" => gettext("needs attention"),
_ if never_synced => gettext("waiting for first sync"),
_ => gettext("up to date"),
}
}
@@ -629,9 +651,17 @@ pub(crate) fn sync_state_label(state: &str) -> String {
/// — so both read "12 of 40" rather than a percentage that could go backwards.
pub(crate) fn sync_progress_label(p: &SyncProgress) -> String {
match p.phase {
// Before the first pass finishes there is no estimate, so the count would
// be "checked 12 of 12" — worse than saying nothing.
SyncPhase::Scanning if p.total == 0 => gettext("checking for changes…"),
SyncPhase::Scanning if p.total == 0 && p.done == 0 => gettext("checking for changes…"),
// Before the first pass finishes there is no estimate, so "12 of 12" would
// say nothing. The count alone still shows a first scan of a large folder
// moving, which can take many minutes before anything downloads.
// Translators: {n} counts the folders and files checked so far.
SyncPhase::Scanning if p.total == 0 => ngettext_f(
"checking for changes — {n} item checked",
"checking for changes — {n} items checked",
p.done as u64,
&[],
),
// Translators: {done} and {total} count folders and files checked so far.
SyncPhase::Scanning => gettext_f(
"checking for changes — {done} of {total}",
@@ -1266,7 +1296,7 @@ mod tests {
remote_uid: "vol~link".into(),
mode: "mirror".into(),
state: state.into(),
last_sync: 0,
last_sync: 1_700_000_000,
pending_mode: None,
progress: None,
paused: false,
@@ -1278,6 +1308,33 @@ mod tests {
assert_eq!(this_computer_subtitle(&[]), "No folders backed up yet");
}
/// A folder added a moment ago is not backed up yet, and the row must not
/// say it is (`docs/BUGS.md` B261). An on-demand folder has no first sync.
#[test]
fn a_folder_before_its_first_sync_is_not_counted_as_backed_up() {
let new = SyncFolderInfo {
last_sync: 0,
..folder("idle")
};
assert_eq!(
this_computer_subtitle(&[folder("idle"), new.clone()]),
"2 folders backed up · 1 waiting for its first sync"
);
let failed = SyncFolderInfo {
state: "error".into(),
..new.clone()
};
assert_eq!(
this_computer_subtitle(&[new.clone(), failed]),
"2 folders backed up · 1 waiting for its first sync · 1 needs attention"
);
let on_demand = SyncFolderInfo {
mode: "ondemand".into(),
..new
};
assert_eq!(this_computer_subtitle(&[on_demand]), "1 folder backed up");
}
#[test]
fn folders_needing_attention_are_counted_not_hidden() {
// The per-folder rows live on the Sync page, so this row is the only
@@ -1296,6 +1353,37 @@ mod tests {
);
}
fn scanning(done: usize, total: usize) -> SyncProgress {
SyncProgress {
phase: SyncPhase::Scanning,
done,
total,
current: String::new(),
}
}
#[test]
fn a_first_scan_counts_what_it_checked() {
// A first sync of a large folder scans for many minutes before anything
// downloads; without a count it reads as stuck (#46).
assert_eq!(
sync_progress_label(&scanning(0, 0)),
"checking for changes…"
);
assert_eq!(
sync_progress_label(&scanning(1, 0)),
"checking for changes — 1 item checked"
);
assert_eq!(
sync_progress_label(&scanning(1234, 0)),
"checking for changes — 1234 items checked"
);
assert_eq!(
sync_progress_label(&scanning(12, 40)),
"checking for changes — 12 of 40"
);
}
#[test]
fn the_restore_picker_names_the_computer_it_restores_from() {
assert!(restore_picker_body(Some("laptop")).contains("“laptop”"));
+73 -2
View File
@@ -1506,8 +1506,13 @@ fn add_device_items(ui: &Rc<Ui>, menu: &mut ActionMenu, spec: &MountSpec, id: i6
} else {
gettext("Pause Syncing")
};
let ui_pause = ui.clone();
menu.item(&label, move || set_folder_paused(&ui_pause, id, !paused));
let ui = ui.clone();
menu.item(&label, move || set_folder_paused(&ui, id, !paused));
let path = spec.local_path.clone();
menu.item(&gettext("Sync From Scratch…"), move || {
confirm_reset_sync_folder(&ui, id, &path)
});
menu.section();
}
let ui = ui.clone();
@@ -1520,6 +1525,51 @@ fn add_device_items(ui: &Rc<Ui>, menu: &mut ActionMenu, spec: &MountSpec, id: i6
});
}
/// Ask before syncing a folder from scratch. Nothing is deleted, but files
/// that differ come back as conflict copies, so it is not a click to make by
/// accident.
fn confirm_reset_sync_folder(ui: &Rc<Ui>, id: i64, path: &str) {
let dialog = adw::AlertDialog::builder()
.heading(gettext("Sync From Scratch?"))
// Translators: {path} is a local folder path.
.body(gettext_f("This computer forgets what it synced in {path} before and compares the folder with Proton Drive again. Nothing is deleted: a file on one side only is copied to the other, and a file that differs keeps both versions.", &[("path", path)]))
.build();
dialog.add_response("cancel", &gettext("Cancel"));
dialog.add_response("reset", &gettext("Sync From Scratch"));
dialog.set_response_appearance("reset", adw::ResponseAppearance::Suggested);
dialog.set_default_response(Some("cancel"));
dialog.set_close_response("cancel");
let win = ui_window(ui);
let ui = ui.clone();
dialog.connect_response(None, move |_, response| {
if response == "reset" {
reset_sync_folder(&ui, id);
}
});
dialog.present(win.as_ref());
}
/// Forget a folder's sync records, then repaint so its row shows the new pass.
fn reset_sync_folder(ui: &Rc<Ui>, id: i64) {
let rx = spawn_request(ui.dirs.control_socket(), Request::ResetSyncFolder { id });
let ui = ui.clone();
glib::spawn_future_local(async move {
let failed = gettext("Couldn't sync the folder from scratch");
match rx.recv().await {
Ok(Ok(Response::Ok { .. })) => toast(&ui, &gettext("Syncing folder…")),
Ok(Ok(Response::Error { message, kind })) => {
toast_failure(&ui, &failed, &message, kind)
}
_ => toast_error(
&ui,
&failed,
&gettext("The Proton Drive service didn't respond."),
),
}
refresh_locations(&ui);
});
}
/// Ask before making a synced folder online only: the switch frees disk space
/// by removing the local copies, which is not something to do by accident.
fn confirm_online_only(ui: &Rc<Ui>, id: i64, path: &str) {
@@ -1679,7 +1729,10 @@ pub(crate) fn location_subtitle(spec: &MountSpec) -> String {
parts.push(match &spec.progress {
Some(p) => sync_progress_label(p),
None if spec.paused => pgettext("state", "paused"),
None => sync_state_label(&spec.state).to_string(),
None => sync_state_label(
&spec.state,
spec.mode == MountMode::Mirror && spec.last_sync == 0,
),
});
}
// Only the locations that are supposed to have a session report its absence:
@@ -1725,7 +1778,25 @@ mod tests {
MountMode::Mirror,
false,
);
assert_eq!(location_subtitle(&spec), "Synced · waiting for first sync");
}
/// A folder whose first pass has not run is not up to date, though its
/// state column starts out `idle` (`docs/BUGS.md` B261).
#[test]
fn a_folder_reads_up_to_date_only_after_its_first_sync() {
let mut spec = spec(
MountKind::Device { sync_folder_id: 7 },
MountMode::Mirror,
false,
);
assert_eq!(location_subtitle(&spec), "Synced · waiting for first sync");
spec.last_sync = 1_700_000_000;
assert_eq!(location_subtitle(&spec), "Synced · up to date");
// A folder that failed before it ever synced says so.
spec.last_sync = 0;
spec.state = "error".into();
assert_eq!(location_subtitle(&spec), "Synced · sync error");
}
#[test]
+50 -7
View File
@@ -368,12 +368,19 @@ pub(crate) fn show_details(ui: &Rc<Ui>, entry: &DirEntry) {
} else {
gettext("File")
});
d.size_row.set_subtitle(&if entry.is_dir {
"—".to_string()
// A folder's size is Proton Drive's count, which takes a request; it lands
// in `load_details_extras`.
if entry.is_dir {
d.size_row.set_title(&gettext("Size on Proton Drive"));
d.size_row.set_subtitle("—");
d.size_row.set_tooltip_text(Some(&gettext(
"Counted by Proton Drive, including items in the trash.",
)));
} else {
human_bytes(entry.size)
});
d.size_row.set_visible(!entry.is_dir);
d.size_row.set_title(&pgettext("column", "Size"));
d.size_row.set_subtitle(&human_bytes(entry.size));
d.size_row.set_tooltip_text(None);
}
d.modified_row
.set_subtitle(&dates::short_date(entry.modified));
let rel = entry_rel(ui, entry);
@@ -426,8 +433,8 @@ pub(crate) fn show_details(ui: &Rc<Ui>, entry: &DirEntry) {
d.pages.set_visible_child_name("details");
}
/// Fill the Sharing and Versions rows, which need the network. The answers
/// land only if the pane still shows the same entry.
/// Fill the Sharing and Versions rows, and a folder's size, which need the
/// network. The answers land only if the pane still shows the same entry.
fn load_details_extras(ui: &Rc<Ui>, entry: &DirEntry) {
let by_uid = entry.path.is_empty() && !entry.uid.is_empty();
let rel = entry_rel(ui, entry);
@@ -453,6 +460,24 @@ fn load_details_extras(ui: &Rc<Ui>, entry: &DirEntry) {
});
if entry.is_dir {
let req = if by_uid {
Request::FolderSizeByUid {
uid: entry.uid.clone(),
}
} else {
Request::FolderSize { path: rel }
};
let rx = spawn_request(ui.dirs.control_socket(), req);
let (ui_c, uid) = (ui.clone(), entry.uid.clone());
glib::spawn_future_local(async move {
let text = match rx.recv().await {
Ok(Ok(Response::FolderSize { size, items })) => folder_size_summary(size, items),
_ => gettext("Couldn't load the size"),
};
if details_showing(&ui_c, &uid) {
ui_c.details.details.size_row.set_subtitle(&text);
}
});
return;
}
let req = if by_uid {
@@ -499,6 +524,18 @@ pub(crate) fn sharing_summary(people: usize, link: bool) -> String {
}
}
/// "1.2 GiB · 340 items": what a folder holds, as Proton Drive counts it.
pub(crate) fn folder_size_summary(size: u64, items: u64) -> String {
// Translators: {size} is a size such as "1.2 GiB"; {n} counts the files
// and folders inside a folder.
ngettext_f(
"{size} · {n} item",
"{size} · {n} items",
items,
&[("size", &human_bytes(size))],
)
}
/// "3 versions · latest 2 Sep 2026".
pub(crate) fn versions_summary(items: &[RevisionInfo]) -> String {
let latest = items.iter().map(|r| r.created).max();
@@ -547,6 +584,12 @@ pub(crate) fn selected_entry(ui: &Rc<Ui>) -> Option<DirEntry> {
mod tests {
use super::*;
#[test]
fn folder_size_summary_reads_size_and_item_count() {
assert_eq!(folder_size_summary(0, 0), "0 bytes · 0 items");
assert_eq!(folder_size_summary(2048, 1), "2.0 KiB · 1 item");
}
#[test]
fn sharing_summary_names_people_and_link() {
assert_eq!(sharing_summary(0, false), "Only you");
+233 -19
View File
@@ -51,6 +51,48 @@ pub(crate) fn link_expiry_at(idx: u32, now: i64) -> Option<i64> {
.map(|days| now + days * 86_400)
}
/// The labels of the Expires dropdown on an existing link. When the link has
/// an expiry, the first entry is that expiry, so the dropdown shows what is
/// set and choosing it again changes nothing.
pub(crate) fn link_expiry_choices(current: Option<i64>, now: i64) -> Vec<String> {
let current = current.map(|at| capitalize(&link_expiry_label(at, now)));
current
.into_iter()
.chain(LINK_EXPIRY.iter().map(|(name, _)| gettext(name)))
.collect()
}
/// What choosing entry `idx` of [`link_expiry_choices`] does to the expiry.
pub(crate) fn link_expiry_change(idx: u32, current: Option<i64>, now: i64) -> LinkChange<i64> {
let idx = match (current, idx) {
(Some(_), 0) => return LinkChange::Keep,
(Some(_), idx) => idx - 1,
(None, idx) => idx,
};
match link_expiry_at(idx, now) {
Some(at) => LinkChange::Set(at),
None if current.is_some() => LinkChange::Remove,
None => LinkChange::Keep,
}
}
/// A row with a title and a dropdown of `labels` as its suffix.
fn dropdown_row(title: &str, labels: &[String], selected: u32) -> (adw::ActionRow, gtk4::DropDown) {
let drop = gtk4::DropDown::builder()
.model(&string_list(labels))
.selected(selected)
.valign(gtk4::Align::Center)
.build();
let row = adw::ActionRow::builder().title(title).build();
row.add_suffix(&drop);
(row, drop)
}
/// The roles a public link can grant, in dropdown order.
fn link_roles() -> [String; 2] {
[pgettext("role", "Viewer"), pgettext("role", "Editor")]
}
/// A loose shape check for an email address: one `@` with something before
/// it and a dotted domain after it. The server does the real validation; this
/// only catches typos before the Invite button is pressed.
@@ -237,6 +279,31 @@ impl ShareTarget {
}
}
fn update_link(
&self,
id: String,
role: Option<String>,
password: LinkChange<String>,
expires: LinkChange<i64>,
) -> Request {
match self {
ShareTarget::Path(path) => Request::UpdatePublicLink {
path: path.clone(),
id,
role,
password,
expires,
},
ShareTarget::Uid(uid) => Request::UpdatePublicLinkByUid {
uid: uid.clone(),
id,
role,
password,
expires,
},
}
}
fn remove_link(&self, id: String) -> Request {
match self {
ShareTarget::Path(path) => Request::RemovePublicLink {
@@ -585,8 +652,8 @@ pub(crate) fn repaint_share_people(state: &Rc<ShareDialog>, entries: &[ShareEntr
*state.people_rows.borrow_mut() = rows;
}
/// Rebuild the public-link section: an existing link (copy / remove) or a
/// create control.
/// Rebuild the public-link section: an existing link (copy / remove, and its
/// settings) or a create control.
pub(crate) fn repaint_share_link(state: &Rc<ShareDialog>, link: Option<&PublicLinkInfo>) {
for row in state.link_rows.borrow_mut().drain(..) {
state.link_group.remove(&row);
@@ -670,30 +737,19 @@ pub(crate) fn repaint_share_link(state: &Rc<ShareDialog>, link: Option<&PublicLi
state.link_group.add(&row);
rows.push(row.upcast());
let settings = link_settings_row(state, link);
state.link_group.add(&settings);
rows.push(settings.upcast());
}
None => {
let role_model = string_list(&[pgettext("role", "Viewer"), pgettext("role", "Editor")]);
let role_drop = gtk4::DropDown::builder()
.model(&role_model)
.selected(0)
.valign(gtk4::Align::Center)
.build();
let role_row = adw::ActionRow::builder()
.title(gettext("Link role"))
.build();
role_row.add_suffix(&role_drop);
let (role_row, role_drop) = dropdown_row(&gettext("Link role"), &link_roles(), 0);
let pw_row = adw::PasswordEntryRow::builder()
.title(gettext("Password (optional)"))
.build();
let expiry_names: Vec<String> =
LINK_EXPIRY.iter().map(|(name, _)| gettext(name)).collect();
let expiry_drop = gtk4::DropDown::builder()
.model(&string_list(&expiry_names))
.selected(0)
.valign(gtk4::Align::Center)
.build();
let expiry_row = adw::ActionRow::builder().title(gettext("Expires")).build();
expiry_row.add_suffix(&expiry_drop);
let (expiry_row, expiry_drop) = dropdown_row(&gettext("Expires"), &expiry_names, 0);
let create = group_button(&gettext("Create Public Link"));
// Enter in the password field creates the link.
@@ -734,6 +790,142 @@ pub(crate) fn repaint_share_link(state: &Rc<ShareDialog>, link: Option<&PublicLi
*state.link_rows.borrow_mut() = rows;
}
/// The settings of an existing link, collapsed under one row. Each control
/// applies at once, the way a member's role does. While a change is on its way
/// the whole row is insensitive: the daemon fills in what a change keeps from
/// the link as it stands, so two changes in flight could undo each other.
fn link_settings_row(state: &Rc<ShareDialog>, link: &PublicLinkInfo) -> adw::ExpanderRow {
let now = glib::real_time() / 1_000_000;
let expander = adw::ExpanderRow::builder()
.title(gettext("Link settings"))
.subtitle(gettext(
"Change who can edit, the password or the expiry. The link stays the same.",
))
.build();
let selected_role = u32::from(link.role == "editor");
let (role_row, role_drop) = dropdown_row(&gettext("Link role"), &link_roles(), selected_role);
let (state_w, expander_w, id) = (Rc::downgrade(state), expander.clone(), link.id.clone());
role_drop.connect_selected_notify(move |d| {
let Some(state) = state_w.upgrade() else {
return;
};
let role = if d.selected() == 1 {
"editor"
} else {
"viewer"
};
let request = state.target.update_link(
id.clone(),
Some(role.into()),
LinkChange::Keep,
LinkChange::Keep,
);
share_dialog_update_link(&state, request, &expander_w);
});
expander.add_row(&role_row);
let current = link.expires;
let (expiry_row, expiry_drop) =
dropdown_row(&gettext("Expires"), &link_expiry_choices(current, now), 0);
let (state_w, expander_w, id) = (Rc::downgrade(state), expander.clone(), link.id.clone());
expiry_drop.connect_selected_notify(move |d| {
let Some(state) = state_w.upgrade() else {
return;
};
let expires = link_expiry_change(d.selected(), current, glib::real_time() / 1_000_000);
if expires == LinkChange::Keep {
return;
}
let request = state
.target
.update_link(id.clone(), None, LinkChange::Keep, expires);
share_dialog_update_link(&state, request, &expander_w);
});
expander.add_row(&expiry_row);
let pw_row = adw::PasswordEntryRow::builder()
.title(if link.has_password {
gettext("New password")
} else {
gettext("Add a password")
})
.show_apply_button(true)
.build();
let (state_w, expander_w, id) = (Rc::downgrade(state), expander.clone(), link.id.clone());
pw_row.connect_apply(move |row| {
let Some(state) = state_w.upgrade() else {
return;
};
let password = row.text().to_string();
if password.is_empty() {
return;
}
let request = state.target.update_link(
id.clone(),
None,
LinkChange::Set(password),
LinkChange::Keep,
);
share_dialog_update_link(&state, request, &expander_w);
});
expander.add_row(&pw_row);
if link.has_password {
let remove = gtk4::Button::builder()
.label(gettext("Remove"))
.valign(gtk4::Align::Center)
.build();
remove.add_css_class("flat");
let row = adw::ActionRow::builder()
.title(gettext("Remove the password"))
.subtitle(gettext(
"Anyone with the link can open it without a password.",
))
.build();
row.add_suffix(&remove);
let (state_w, expander_w, id) = (Rc::downgrade(state), expander.clone(), link.id.clone());
remove.connect_clicked(move |_| {
let Some(state) = state_w.upgrade() else {
return;
};
let request =
state
.target
.update_link(id.clone(), None, LinkChange::Remove, LinkChange::Keep);
share_dialog_update_link(&state, request, &expander_w);
});
expander.add_row(&row);
}
expander
}
/// Send a link change, then reload the dialog: on success to show the link as
/// it now is, on failure to put the controls back to what the link still is.
fn share_dialog_update_link(
state: &Rc<ShareDialog>,
request: Request,
settings: &adw::ExpanderRow,
) {
settings.set_sensitive(false);
let rx = spawn_request(state.ui.dirs.control_socket(), request);
let state = state.clone();
glib::spawn_future_local(async move {
match rx.recv().await {
Ok(Ok(Response::PublicLink { .. })) => toast(&state.ui, &gettext("Link updated")),
Ok(Ok(Response::Error { message, .. })) => {
toast_error(&state.ui, &gettext("Couldn't change the link"), &message)
}
_ => toast_error(
&state.ui,
&gettext("Couldn't change the link"),
&gettext("The Proton Drive service didn't respond."),
),
}
share_dialog_reload(&state);
});
}
/// Create a public link, then reload the dialog so the copy/remove controls
/// replace the create form (and the freshly minted URL is shown).
pub(crate) fn share_dialog_create_link(
@@ -863,6 +1055,28 @@ mod tests {
assert_eq!(link_expiry_at(99, 0), None);
}
#[test]
fn choosing_an_expiry_keeps_removes_or_sets_it() {
let now = 1_000;
// No expiry yet: the choices are just Never and the lifetimes.
assert_eq!(link_expiry_choices(None, now).len(), LINK_EXPIRY.len());
assert_eq!(link_expiry_change(0, None, now), LinkChange::Keep);
assert_eq!(
link_expiry_change(2, None, now),
LinkChange::Set(now + 7 * 86_400)
);
// An expiry set: it comes first, then Never and the lifetimes.
let at = Some(2_000_000_000);
assert_eq!(link_expiry_choices(at, now).len(), LINK_EXPIRY.len() + 1);
assert_eq!(link_expiry_change(0, at, now), LinkChange::Keep);
assert_eq!(link_expiry_change(1, at, now), LinkChange::Remove);
assert_eq!(
link_expiry_change(2, at, now),
LinkChange::Set(now + 86_400)
);
}
#[test]
fn past_expiry_reads_expired() {
assert_eq!(link_expiry_label(10, 20), "expired");
+18
View File
@@ -194,6 +194,24 @@ fn poll_state(socket: &Path, default_mountpoint: &Path) -> DriveState {
),
Phase::Attention,
),
// A daemon whose mount fails on every try never opens the
// socket; the app says why (B263).
Ok(s)
if socket
.parent()
.and_then(pdfs_core::mountfail::read)
.is_some() =>
{
(
// Translators: {username} is the Proton account name. The
// service keeps failing to start; the app shows the error.
gettext_f(
"Signed in as {username} — can't open your Drive",
&[("username", &s.username)],
),
Phase::Attention,
)
}
Ok(s) => (
// Translators: {username} is the Proton account name.
gettext_f(
+583
View File
@@ -68,6 +68,11 @@ original wording.
| Entry | Title | What is left |
|---|---|---|
| [B275](#b275--on-a-slow-link-a-files-last-write-can-land-only-in-a-conflict-copy) | On a slow link, a file's last write can land only in a conflict copy | Cause not found; seen once in CI |
| [B273](#b273--a-restore-proposes-another-computers-paths-when-the-two-share-a-hostname) | A restore proposes another computer's paths when the two share a hostname | Not fixed |
| [B270](#b270--one-file-left-in-a-mirror-folder-clears-the-wipe-guard-for-all-the-others) | One file left in a mirror folder clears the wipe guard for all the others | Not fixed |
| [B267](#b267--a-key-failure-at-a-start-from-a-warm-cache-shows-as-offline-for-good) | A key failure at a start from a warm cache shows as offline for good | Not fixed |
| [B265](#b265--address-keys-whose-token-is-for-an-inactive-account-key-never-open) | Address keys whose token is for an inactive account key never open | Needs an SDK change; not reproduced |
| [B134](#b134--a-file-deleted-online-while-the-drain-checks-its-write-is-kept-as-a-conflict-copy) | A file deleted online while the drain checks its write is kept as a conflict copy | Only with `local_first` off |
| [B91](#b91--the-daemon-hung-overnight-and-nothing-could-say-what-it-was-waiting-on) | The daemon hung overnight, and nothing could say what it was waiting on | The cause is not diagnosed; the daemon now reports where it is stuck and is restarted |
| [B67](#b67--fresh-state-restore-can-silently-select-a-new-device-high-22) | Fresh-state restore can silently select a new device (HIGH-22) | An ambiguous device after a loss of local state is not asked about |
@@ -80,6 +85,11 @@ original wording.
| Entry | Title | What is left |
|---|---|---|
| [B269](#b269--a-restore-that-skipped-every-folder-reports-success) | A restore that skipped every folder reports success | The app shows the result only in the activity feed; no offer to replace or reset the folder in the way |
| [B268](#b268--pdfs-diagnose-says-no-problems-found-while-sync-is-stuck) | `pdfs diagnose` says "No problems found" while sync is stuck | The sync engine's state is not checked |
| [B263](#b263--a-mount-that-can-never-succeed-retries-every-5-s-and-tells-only-the-journal) | A mount that can never succeed retries every 5 s and tells only the journal | Errors no retry fixes back off like network errors |
| [B262](#b262--watching-a-large-synced-folder-holds-up-the-sync-engine-and-a-failed-watch-is-never-retried) | Watching a large synced folder holds up the sync engine, and a failed watch is never retried | The first pass still waits for the watch; a failed watch is not shown in the app |
| [B259](#b259--the-sync-engine-can-die-or-hang-and-nothing-shows-it-until-a-restart) | The sync engine can die or hang, and nothing shows it until a restart | No pass deadline; an ended engine is reported, not restarted; the trigger in #53 is not known |
| [B199](#b199--a-stopped-share-stays-on-the-shared-by-me-page-for-minutes) | A stopped share stays on the Shared by me page for minutes | The share itself is not deleted; the SDK has no call for it |
| [B88](#b88--pdfs-pin-rename-move-delete-and-sharing-still-cannot-name-a-path-under-a-secondary-mount) | `pdfs pin`, `rename`, `move`, `delete` and sharing still cannot name a path under a secondary mount | `pin`, `unpin`, opening a file, thumbnails, uploads, sharing and versions |
| [B72](#b72--pdfs-trash-never-returns) | `pdfs trash` never returns | The slowness of the first refresh is unmeasured |
@@ -100,6 +110,16 @@ a live account since.
| Entry | Title |
|---|---|
| [B274](#b274--opening-a-file-for-writing-fails-with-eio-while-its-queued-revision-lands) | Opening a file for writing fails with EIO while its queued revision lands |
| [B271](#b271--one-failed-listing-makes-a-large-first-sync-start-over-from-nothing) | One failed listing makes a large first sync start over from nothing |
| [B266](#b266--pin-unpin-search-and-node-loads-report-a-database-error-as-offline) | Pin, unpin, search and node loads report a database error as "offline" |
| [B264](#b264--a-mount-that-fails-after-starting-its-workers-leaves-them-running) | A mount that fails after starting its workers leaves them running |
| [B261](#b261--a-folder-that-has-never-synced-reads-up-to-date) | A folder that has never synced reads "up to date" |
| [B272](#b272--after-a-folder-is-removed-and-added-again-its-file-events-go-to-the-old-id) | After a folder is removed and added again, its file events go to the old id |
| [B260](#b260--the-poll-pushes-a-new-folders-first-sync-behind-every-other-folder) | The poll pushes a new folder's first sync behind every other folder |
| [B258](#b258--journalctl--p-warning-shows-nothing-whatever-the-daemon-logged) | `journalctl -p warning` shows nothing, whatever the daemon logged |
| [B256](#b256--pdfs-copy-right-after-a-save-copies-the-old-version) | `pdfs copy` right after a save copies the old version |
| [B255](#b255--a-synced-folder-whose-files-are-all-gone-here-is-stuck-for-good) | A synced folder whose files are all gone here is stuck for good |
| [B254](#b254--an-edit-in-the-same-second-as-a-new-upload-never-arrives) | An edit in the same second as a new upload never arrives |
| [B253](#b253--the-app-shows-the-sign-in-page-when-the-keyring-is-locked) | The app shows the sign-in page when the keyring is locked |
| [B252](#b252--computers-sharing-a-device-overwrite-each-others-profile) | Computers sharing a device overwrite each other's profile |
@@ -357,6 +377,556 @@ a live account since.
Newest first.
### B275 — On a slow link, a file's last write can land only in a conflict copy
**Status:** Open
**Found:** 2026-10-31, CI on #60: `one_client_on_a_slow_link`, profile `one-client-wifi` seed 2,
failed with `Drive does not hold what the clients wrote, 120s after the links came up`. The
same seed passed 4 times locally, and #60 changes no daemon logic, so the failure depends on
timing.
**Where:** not located; the drain's conflict check in `crates/pdfs-fuse/src/drain.rs` is the
first suspect
**Repro:** Only the CI run above. The last steps created `x/z/y/a.txt` (37 bytes), wrote 42
bytes into it at offset 20, and brought every link up. Drive then held the file at its first
37 bytes, and the full content was in `x/z/y/a (sync-conflict …).txt`. No other client wrote
to it.
**Cause.** Not known. Nothing was lost, since the conflict copy holds the bytes, but the file
conflicted with its own create. B183 and B184 found the same symptom after a lost create
reply.
**Fix.** Not done. Needs a reproduction: run the seed in a loop under load and keep the state
directory of a failing run.
---
### B274 — Opening a file for writing fails with EIO while its queued revision lands
**Status:** Fixed, not verified live
**Found:** 2026-10-31, CI: the simulation
`a_file_whose_upload_was_not_read_back_takes_two_writes_at_once` failed once with
`Input/output error` on a write, after `copy queued revision into write scratch failed … No
such file or directory`. It did not fail in 54 local runs.
**Where:** the write open in `crates/pdfs-fuse/src/filesystem.rs`
**Repro:** Write a file, and open it for writing again at the moment its queued revision
finishes uploading. The open fails with EIO, and the program sees its save fail.
**Cause.** The open samples the queued revision, then copies its blob into the new handle's
scratch file. The drain lets go of the pending entry and then deletes the blob when the upload
lands. A blob deleted between the sample and the copy failed the open, although the re-check
just after the copy already retries a revision that changed during it.
**Fix.** A failed copy checks whether the pending entry still names that blob. When it does
not, the revision landed or was replaced, and the open goes around again with the base the
file has now, as the re-check does. Since the drain drops the entry before the blob, a blob
that is gone and still named is a real failure and still returns EIO.
**Test:** none deterministic; the simulation above covers the window.
---
### B273 — A restore proposes another computer's paths when the two share a hostname
**Status:** Open
**Found:** 2026-10-31, code audit after issue #46.
**Where:** `saved_folder` in `crates/pdfs-core/src/profile.rs`
**Repro:** Two computers with the same hostname, for example both left at a distribution's
default `archlinux`, sync the same device. Restore a folder on one of them. The proposed local
path and mode come from whichever of the two profiles is found first, not from this computer's.
**Cause.** `saved_folder` prefers the profile whose `hostname` matches. The machine id that
B252 added tells computers apart, but this lookup does not use it.
**Fix.** Not done. Match on the machine id first, then fall back to the hostname for profiles
written before it existed.
---
### B272 — After a folder is removed and added again, its file events go to the old id
**Status:** Fixed, not verified live
**Found:** 2026-10-31, code audit after issue #53.
**Where:** `rewatch` in `crates/pdfs-fuse/src/sync/engine.rs`
**Repro:** Remove a synced folder and add the same path again before the engine handles the
first `Rewatch`, so both land in one burst. Edits in the folder are not synced until the
two-minute poll, and the same goes for every later edit until the daemon restarts.
**Cause.** `rewatch` matches the watches it has against the ones it wants by path alone. The
path is still wanted, so the old entry stays with the removed folder's id. Its events become
`Reconcile(old id)`, `sync_folder_get` finds no row, and the request is dropped.
**Fix.** `keep_wanted` keeps a watch whose path is still wanted and gives it the id the
folder list has now.
**Test:** `a_re_added_folder_keeps_its_watch_under_the_new_id` in `pdfs-fuse`.
---
### B271 — One failed listing makes a large first sync start over from nothing
**Status:** Fixed, not verified live
**Found:** 2026-10-31, code audit after issue #46: a folder of about 19,900 items restored on a
second computer sat at `scanning…` with nothing downloaded, and the journal showed
`RequestTimeout (http 408)`.
**Where:** `walk_remote` in `crates/pdfs-fuse/src/sync.rs`
**Repro:** Restore or add a mirror folder with tens of thousands of items on Drive, on a link
that drops a request now and then. The pass walks the whole remote tree before it downloads
anything. When one folder's listing fails after the SDK's retries, the pass fails, and the
next one walks the whole tree again. On a slow enough link it never gets past the walk.
**Cause.** Each `enumerate_folder_children_node_uids`, `enumerate_nodes_light` and
`enumerate_nodes` call in the walk ends the whole walk through `?`. Nothing is applied until
the walk is done. Files that did download keep their baseline rows, so only the walk is lost,
but on a first sync that is all of it. Progress is also counted only after a folder's full
listing, so a large folder shows `scanning…` with 0 checked for a long time.
**Fix.** A listing that fails below the sync root records that folder as a gap in
`RemoteGaps`, next to the folders whose listing came back short. The planner deletes nothing
under a gap, the rest of the pass still applies, and the pass report names the folder as
tried again next pass. A failed listing of the root itself still fails the pass, since there is
nothing left to apply. The progress count inside one large folder is unchanged.
**Test:** `a_folder_that_failed_to_list_hides_everything_under_it` in `sync::planner`.
---
### B270 — One file left in a mirror folder clears the wipe guard for all the others
**Status:** Open
**Found:** 2026-10-31, code audit after issue #46, whose reporter emptied a synced folder
several times.
**Where:** `guard_local_wipe` and `guard_remote_wipe` in `crates/pdfs-fuse/src/sync/planner.rs`
**Repro:** Sync a mirror folder with many files. Make all of them but one unavailable here: a
half-mounted disk, a restored backup that brought back one file, or a script that emptied the
folder and wrote one file back. The next pass trashes every other file on Drive. On a device
that two computers sync, the other computer then deletes its copies too.
**Cause.** The guard refuses a pass only when every baseline path is missing. One surviving
path is enough to clear it, so it guards against an empty folder and nothing short of that.
The remote guard has the same shape.
**Fix.** Not done. Also refuse when most of a large baseline is missing at once, and leave the
choice to the user: delete on the other side, or sync from scratch.
---
### B269 — A restore that skipped every folder reports success
**Status:** Partly fixed
**Found:** 2026-10-31, code audit after issue #46: the journal showed `restore complete
restored=0 skipped=1` several times, and the user saw nothing.
**Where:** the `RestoreSyncFolders` and `RestoreDeviceFolders` handler in
`crates/pdfs-fuse/src/control.rs`; `restore_sync_folders` in `crates/pdfs-fuse/src/profile.rs`
**Repro:** Restore a folder from another computer to a path that is already a sync root here,
for example a folder left over from an earlier attempt. `pdfs devices join` prints `restoring
folders`, and the app shows the same. Nothing downloads, and no error appears anywhere.
**Cause.** The daemon acks before it works, because a restore outlasts the socket timeout. The
result, `restored 0 folder(s); skipped: …`, is then logged as a successful activity, so the
reason is only in the activity text and the journal. When the database insert fails partway,
the `?` also skips the `Rewatch` for the folders already restored.
**Fix.** `restore_sync_folders` returns what it restored and what it skipped, with the reason
for each skip. The daemon logs a restore that skipped any folder as a failed activity, with the
reasons as its detail, and warns in the journal. A database error partway now sends the
`Rewatch` for the folders already restored before it returns. `pdfs devices join` points to
`pdfs activity` for the result.
Left: the app shows the result only in the activity feed, and nothing offers to replace or reset
the folder in the way.
---
### B268 — `pdfs diagnose` says "No problems found" while sync is stuck
**Status:** Partly fixed. The sync engine's own state is not checked yet.
**Found:** 2026-10-31, issue #53: a folder that never synced, and `diagnose` found nothing.
**Where:** `diagnose` in `crates/pdfs-cli/src/main.rs`
**Repro:** Leave a synced folder in `error`, or have queued writes that keep failing, or pause
sync. `pdfs diagnose` reports every check `[ok]` and exits 0.
**Cause.** The `Status` reply is matched with `..`, which drops `failing_ops`, `failing_error`,
`parked_uploads`, `staged_oldest_secs` and `paused`. `diagnose` never asks for the synced
folders or for `Diagnostics`, so a folder in error and a stalled worker are not seen. A
control socket file with no daemon behind it is also reported `[ok]`; only the next line says
the daemon does not answer.
**Fix.** `diagnose` warns about writes that keep failing (with the last error), a write that
has waited a day or more to upload, a pause, and each synced folder in `error` or `conflict`
(`sync_health` and `folder_health` in `crates/pdfs-cli/src/main.rs`). A control socket file
that nothing listens on is a warning, not `[ok]`. Parked uploads are left out: a file still
wearing a download's temporary name is normal. Not done: the sync engine's state, which
`Diagnostics` reports once B259's change is in.
**Test:** `a_stuck_queue_or_a_pause_is_reported` and `a_synced_folder_in_error_is_reported` in
`pdfs-cli`. On 2026-10-31 the new `diagnose` against a healthy 3.6.0 daemon still reported
"No problems found".
---
### B267 — A key failure at a start from a warm cache shows as offline for good
**Status:** Open
**Found:** 2026-10-31, code audit after issue #40. B202 fixed the same path for a refused
session only.
**Where:** `fetch_or_recall_root` in `crates/pdfs-fuse/src/mount.rs`; `run_online_probe` in
`crates/pdfs-fuse/src/link.rs`
**Repro:** Sign in once so the cache holds the My Files root. Then start the daemon in a state
where the root cannot be decrypted, for example the address-key failure of B265. The mount
comes up "offline", the app says you are offline, and it stays so while the network is fine.
**Cause.** Any error from `get_my_files_folder` other than a revoked session becomes
`Start::Offline` when a cached root exists. The online probe then retries the same call and
logs its failures at debug level, so the journal at the default level says nothing.
**Fix.** Not done. Use `Start::Offline` only for an error `is_network_error` accepts, and
report the others as their own state through `Status`.
---
### B266 — Pin, unpin, search and node loads report a database error as "offline"
**Status:** Fixed, not verified live
**Found:** 2026-10-31, code audit after issue #38. B205 fixed the photo upload path only.
**Where:** `CoreError::from_api` in `crates/pdfs-core/src/error.rs`; its callers in
`crates/pdfs-fuse/src/lib.rs` (`pin`, `unpin`, the searches, `load node`)
**Repro:** Pin a folder while the cache database is locked or full. The app says "You're
offline" and drops the reason.
**Cause.** `from_api` classifies any error that is not a `ProtonError` as `Offline`, because
it was written for SDK calls, where that means a transport error. These callers pass it
`pdfs_core::Error` values from the cache and the database. A `ProtonError` wrapped in
`pdfs_core::Error::Proton` also misses the downcast, since the source chain is not walked.
**Fix.** `from_api` walks the source chain. The first `ProtonError` or `pdfs_core::Error` in it
decides the kind, the same way `From<Error>` does, and only an error with neither, a transport
error, reads as offline. That covers every caller at once, the database ones included.
**Test:** `a_database_error_handed_to_from_api_is_not_offline` in `pdfs-core`.
---
### B265 — Address keys whose token is for an inactive account key never open
**Status:** Open
**Found:** 2026-10-31, code audit after issue #40: a Duo account whose mount failed on every
start with `fetch My Files root: cryptography error: failed to decrypt: missing key`, after three
`failed to decrypt address key token` warnings. The web app worked. B210's fix does not apply,
because the account has an active volume.
**Where:** `user_keys` and `decrypt_address` in `proton-sdk` (`crates/proton-sdk/src/account.rs`
in `proton-sdk-rs`)
**Repro:** Not reproduced. An account whose address-key tokens are encrypted to an account key
that is no longer active cannot open those address keys, and so cannot open the My Files share.
**Cause.** `user_keys` skips every inactive account key, even one the password opens, and the
address-key tokens are decrypted only with what it returns. `keycheck.rs` in this repository
already reports "token is for inactive account key", so the case is known. Separately,
`addresses()` fails as a whole through `?` when one address has no usable primary key, which
takes the working addresses down with it.
**Fix.** Not done, and it belongs in the SDK. Use every account key that opens for decrypting
tokens, and only the active ones for signing. Skip and log an address that cannot be opened.
---
### B264 — A mount that fails after starting its workers leaves them running
**Status:** Fixed, not verified live
**Found:** 2026-10-31, code audit after issue #40.
**Where:** `run_mount` in `crates/pdfs-fuse/src/mount.rs`
**Repro:** Make the mount fail after the workers start: a busy mountpoint, or a control socket
whose permissions cannot be set. The daemon retries every 5 s, and the drain, sync, sweep,
probe and index threads of the failed attempt keep running beside the next one.
**Cause.** The early returns after the workers are spawned (the `?` on the socket bind, the
permission check, a `spawn_session` error, and a `?` on a later thread spawn) skip
`stop_workers`. The threads hold a `Core`, and the `Core` holds the database. If that keeps
the `flock` held, every later attempt fails with "another daemon is using cache.db" instead
of the real error. That part is inferred from the code and not observed.
**Fix.** Every early return in `mount` after the first worker starts stops the workers
(`abandon_workers`, `keep_or_abandon` in `crates/pdfs-fuse/src/mount.rs`). Unlike a normal stop
it does not send `STOPPING=1`, because the daemon is about to retry and systemd would
otherwise wait for it to exit and then kill it. The control-thread failure path, which already
stopped its workers, no longer sends it either.
**Test:** none of its own; the simulations (`sim::daemon`, 55 passed) cover the normal start
and stop.
---
### B263 — A mount that can never succeed retries every 5 s and tells only the journal
**Status:** Partly fixed. Errors that no retry fixes are not told apart from network errors yet.
**Found:** 2026-10-31, issue #40: the app showed "Connecting… waiting for the Proton Drive
service" for days, `pdfs diagnose` only reported the control socket missing, and the reporter
uninstalled.
**Where:** the mount loop in `crates/pdfs-cli/src/main.rs`; `run_mount` in
`crates/pdfs-fuse/src/mount.rs`
**Repro:** Start the daemon with a session whose keys cannot open My Files (B265). The journal
shows `mount failed; retrying in 5s` forever. The app and the tray wait for the service, and
nothing says why.
**Cause.** Every mount error gets the same 5 s retry, whether it is a network error or one that
no retry fixes, such as a key or a 4xx answer. Each attempt builds a new client and sends about
four requests, so a broken account sends tens of thousands a day, which may also feed
Proton's sign-in blocks (#39). The control socket is bound only after the mount succeeds, and
the error is written nowhere a front end can read it.
**Fix.**
- The retry doubles from 5 s to a 2-minute ceiling (`mount_retry_delay` in
`crates/pdfs-cli/src/main.rs`), about 720 attempts a day instead of 17,000. The ceiling stays
at minutes because a cold start without a network fails the same way and has to mount soon
after the network returns.
- Each failure is written to `mount-failure.json` in the state directory
(`crates/pdfs-core/src/mountfail.rs`), with the error, since when it fails and the attempt
count. A mount that comes up removes it, and so does a new daemon.
- The app shows "Can't open your Drive" with the error and a Retry button instead of
"Connecting…". The tray says "Signed in as … — can't open your Drive". `pdfs diagnose` reports
`[FAIL] mount: failing for …`. `systemctl --user status` shows the error as the unit's status.
- Not done: key, `KeysLocked` and 4xx errors still retry like network errors, only less often.
**Test:** `a_failing_mount_backs_off_to_a_ceiling` in `pdfs-cli`;
`a_recorded_failure_reads_back_until_cleared` in `pdfs-core`. On 2026-10-31 a second
`pdfs daemon` beside the running one failed on the database lock three times in 18 s (at 0, 5
and 15 s) and recorded `attempts: 3` with the lock error.
---
### B262 — Watching a large synced folder holds up the sync engine, and a failed watch is never retried
**Status:** Partly fixed. A watch is still set up before the first pass, and a failed watch
is not shown in the folder's state.
**Found:** 2026-10-31, code audit after issue #53.
**Where:** `rewatch` and `engine_loop` in `crates/pdfs-fuse/src/sync/engine.rs`
**Repro:** Add a synced folder with many subfolders, or one that holds a symlink to a large
tree, on a system near `fs.inotify.max_user_watches`. The new folder's first pass waits until
the recursive watch is set up. When the watch fails, the journal has one `sync: watch failed`
line, the folder is never watched, and later changes are found only by the two-minute poll.
**Cause.**
- `rewatch` runs on the engine thread before the reconcile, and notify's recursive watch walks
the whole tree, at startup for every folder.
- notify follows symlinks by default, and the local walk does not.
- When `watch` fails partway, the watches added before the failure stay registered, but the
folder is not recorded as watched. Their events match no folder and become `ReconcileAll`,
and they use up watches the next folder needs.
- Errors the watcher reports later are dropped in its callback.
**Fix.**
- A failed watch is unwatched, which removes the watches it had added under the folder, and
is tried again every 10 minutes. The warning names `fs.inotify.max_user_watches`.
- The watcher no longer follows symlinks.
- Errors the watcher reports are logged.
- Not done: the watch still runs before the folder's first pass. Watching first is what keeps
a change made during the pass from being missed, so the order stays; the cost is one walk
of a new folder's tree. A failed watch is not shown in the folder's state yet.
**Test:** none for the watch failure itself; it needs the inotify limit reached.
---
### B261 — A folder that has never synced reads "up to date"
**Status:** Fixed, not verified live
**Found:** 2026-10-31, issue #53: "In Computers I add the folder. It appears as synced."
**Where:** `sync_folder` in `crates/pdfs-core/src/db/migrations.rs`; `sync_state_label` in
`crates/pdfs-gui/src/app/pages/devices.rs`; the location rows in
`crates/pdfs-gui/src/app/pages/locations.rs`
**Repro:** Add a synced folder. Until its first pass starts, the app shows it as up to date.
The same happens after a switch from on-demand to mirror, after **Sync From Scratch…**, and for
every folder while sync is paused for the whole account.
**Cause.** The `state` column defaults to `idle`, and every state the label does not know reads
"up to date". `last_sync` is 0 for a folder that has never synced, but the rows do not look at
it, and a test pins the label for that case. The rows only show a folder's own pause, not the
account-wide one.
**Fix.** A mirror folder whose `last_sync` is 0 and whose state is `idle` reads "waiting for
first sync" on its row. The This computer row on the Computers page counts such folders, as in
"2 folders backed up · 1 waiting for its first sync". An on-demand folder has no first sync,
so it is left out. The account-wide pause stays where it already is, the pause card at the top
of the page; repeating it on every row would only add noise.
**Test:** `a_folder_reads_up_to_date_only_after_its_first_sync` and
`a_folder_before_its_first_sync_is_not_counted_as_backed_up` in `pdfs-gui`; the pinned test now
expects the new label.
---
### B260 — The poll pushes a new folder's first sync behind every other folder
**Status:** Fixed, not verified live
**Found:** 2026-10-31, code audit after issue #53.
**Where:** `engine_loop` in `crates/pdfs-fuse/src/sync/engine.rs`
**Repro:** Sync a few large mirror folders whose passes take longer than two minutes, then add
another folder. Its first pass waits until every other folder has been walked again, each
time.
**Cause.** The poll sends `ReconcileAll` every two minutes whether or not a pass is still
running. When a burst holds both a `ReconcileAll` and a folder's `Reconcile(id)`, the loop runs
only `reconcile_all`, which goes in id order. A new folder has the highest id, so it always
goes last.
**Fix.** `run_burst` runs the folders a burst names by id first, lowest id first, and then
every other folder when the burst also holds a `ReconcileAll`. A named folder is read again
before its pass. A pause that lands between passes now stops the rest of the burst.
**Test:** `named_folders_run_before_the_rest_of_a_burst` in `pdfs-fuse`.
---
### B259 — The sync engine can die or hang, and nothing shows it until a restart
**Status:** Partly fixed. A hung pass is reported but not ended, and a panic outside a pass
still stops the engine; it is reported, not restarted.
**Found:** 2026-10-31, issue #53: a newly added folder did not sync until the reporter restarted
the app. All FUSE workers were idle, nothing was queued, and the warning-level journal was
empty (B258).
**Where:** `spawn` and `engine_loop` in `crates/pdfs-fuse/src/sync/engine.rs`;
`reconcile_pass` in `crates/pdfs-fuse/src/sync.rs`; `crates/pdfs-fuse/src/diagnostics.rs`
**Repro:** Not reproduced. Make one pass panic, or block in a call that does not return. Every
later add, `pdfs sync now`, file event and poll is dropped without a word. The folder that was
syncing stays `syncing` with its last progress. A restart runs a pass over every folder, which
is why it helps.
**Cause.**
- The engine is one thread, and nothing catches a panic in a pass. When it unwinds, the
receiver is gone and every `let _ = sync_tx.send(..)` fails silently. The poll thread stops
at its first failed send.
- The thread's `JoinHandle` is checked only at shutdown.
- A pass has no overall deadline. Each request is bounded, but a pass is not; B165 dealt with
the stop only.
- Nothing logs a pass starting, or ending without changes. `Diagnostics` covers the FUSE
workers and the request queue, not this thread, so "workers idle, nothing queued" says
nothing about sync.
- A database error when the engine reads a folder drops that folder's request without a log.
**Fix.**
- Each pass runs under `catch_unwind` (`EngineHealth::contain` in
`crates/pdfs-fuse/src/sync/engine.rs`). A panic ends that pass only: its progress is cleared,
the folder is set to `error`, the activity feed says so, and the next poll tries again.
- `Diagnostics` carries the engine's state: running or not, the folder of the running pass, how
long it has run and how long since it last reported progress, and how many passes panicked.
`pdfs diagnostics` prints it as `sync engine: …`.
- The supervisor warns when the engine thread has ended, and when a pass has reported no
progress for 15 minutes.
- A first pass logs its start and end at info, as does any pass that changed something or
took 30 s or more. A request for a folder the database cannot read is logged.
- Not done: a deadline for a pass, and restarting an engine thread that has ended. The
channel's receiver ends with the thread, so a restart needs a new channel in every `Core`
clone.
**Test:** `a_panicking_pass_is_contained_and_counted`, `the_running_pass_is_reported` and
`the_engine_reads_as_stopped_once_its_thread_ends` in `pdfs-fuse`;
`a_stopped_sync_engine_is_a_stall` and `only_a_quiet_sync_pass_is_a_stall` in the supervisor.
---
### B258 — `journalctl -p warning` shows nothing, whatever the daemon logged
**Status:** Fixed, not verified live
**Found:** 2026-10-31, issue #53: the reporter ran `journalctl --user -u proton-drive.service -b
-p warning`, as `docs/TROUBLESHOOTING.md` tells them to, and got `-- No entries --`.
**Where:** `main` in `crates/pdfs-cli/src/main.rs`; `packaging/proton-drive.service`
**Repro:** Make the daemon log a warning, then run the command above. It prints no entries.
Without `-p` the warning is there.
**Cause.** The daemon logs with plain `tracing_subscriber::fmt()` to standard output. journald
stores every line of a service's output at the unit's default priority, info, unless the line
starts with a `<N>` prefix. So every `WARN` and `ERROR` line is stored as info, and the
troubleshooting guide's filter hides all of them.
**Fix.** When standard output is the journal stream (`JOURNAL_STREAM` names the same
device and inode as fd 1), each line starts with the `<N>` prefix for its level, and ANSI colours
are off. A terminal that merely inherited `JOURNAL_STREAM` keeps the plain format
(`crates/pdfs-cli/src/journal.rs`).
**Test:** `each_line_opens_with_the_priority_of_its_level` and
`journal_stream_must_name_the_same_file` in `pdfs-cli`. On 2026-10-31, `pdfs diagnose` run with
`RUST_LOG=debug` under `systemd-run --user -p StandardOutput=journal` stored its debug lines at
`PRIORITY=7` and its report at 6.
---
### B256 — `pdfs copy` right after a save copies the old version
**Status:** Fixed, not verified live
**Found:** 2026-10-26, the simulation `a_pdfs_copy_copies_on_drive_and_waits_for_unsynced_changes`
failing on most runs. The feature was not released yet.
**Where:** `release` in `crates/pdfs-fuse/src/filesystem.rs`; `copy_to` in
`crates/pdfs-fuse/src/lib.rs`; `has_pending_work_under` in `crates/pdfs-fuse/src/relocate.rs`
**Repro:** Save a file in the mount and copy it, or its folder, with `pdfs copy` straight away.
The copy goes ahead and Drive copies the version before the save.
**Cause.** `close(2)` returns before the kernel sends `release`. `release` takes the write handle
out of the open writes and then queues the revision, which takes a few milliseconds. A copy
checked in between found the edit in neither place. A folder copy looked only at the queue, so
it also missed a file still open with unsaved writes.
**Fix.** `release` keeps the inode in `State::releasing` until the revision is queued. A copy
refuses a file or folder with unsaved writes, or a release still queueing, at it or below it.
A move checks the same set.
**Test:** The simulation above passed 6 of 6 runs after the fix.
---
### B255 — A synced folder whose files are all gone here is stuck for good
**Status:** Fixed, not verified live
**Found:** 2026-10-26, issue #46: a folder restored from another computer never downloaded.
**Where:** `push_pass` in `crates/pdfs-fuse/src/sync.rs`; `reset_sync_folder` in
`crates/pdfs-fuse/src/devices.rs`
**Repro:** Sync a folder in mirror mode, then delete all of its files on this computer while
the folder is paused. Resume it. Every pass stops with `every one of the N synced paths is
missing locally`, the folder stays in error, and a switch to online only waits for good.
**Cause.** The wipe guard is right to stop a pass that would trash the whole folder on Drive.
But nothing got the folder out of that state. A switch to on-demand waits for a pass that ends
`idle`, and its push pass hit the same guard. The baseline that made the empty folder read as
"delete everything" could only be cleared by removing the folder from sync. How the reporter's
folder lost its files is not known yet.
**Fix.** The push pass for a switch to on-demand forgets the baseline rows of a local copy that
is gone in full, instead of trashing them on Drive, so the switch goes ahead and the mount shows
what Drive has. A switch asked for before any pass saw the folder empty found it still `idle`,
and the check for what only this computer has refused it at once: every path read as "deleted
here but not on Proton Drive yet". The account run found that. Such a switch now waits for the
same push pass (`mirror_tree_gone` in `crates/pdfs-fuse/src/sync.rs`, checked in
`apply_sync_folder_mode`). **Sync From Scratch…** in the app and `pdfs sync reset ID` clear a mirror
folder's baseline, so the next pass compares both sides as on the first sync: one-sided files
are copied across, files that differ keep both versions, and nothing is deleted. Restore now
logs why it skipped a folder.
**Test:** The simulations `a_folder_whose_local_copy_is_gone_still_goes_on_demand` (which
failed in exactly the reported state before the fix),
`a_folder_emptied_before_any_pass_still_goes_on_demand` (which failed with the account run's
error) and `a_folder_whose_local_copy_is_gone_downloads_again_after_a_reset`; the acceptance case
"an emptied mirror folder switches to on-demand".
---
### B254 — An edit in the same second as a new upload never arrives
**Status:** Fixed, not verified live
@@ -1461,6 +2031,8 @@ what it holds. What was synced in it is still deleted.
### B210 — After a password reset, the mount retries "missing key" forever
**Status:** Fixed, not verified live
**See also:** B265. The account in issue #40 has an active volume, so this fix does not run for
it; its "missing key" comes from address keys that do not open.
**Found:** 2026-10-15, issue #40: on a fresh 3.4.0 install the sign-in worked, but the daemon
logged `mount failed: fetch My Files root: cryptography error: failed to decrypt: missing key`
every 5 seconds. The app stayed on "Connecting…" because the daemon never got far enough to
@@ -1606,6 +2178,7 @@ front-end is the stale one, and restarting the service would not help.
### B205 — A photo upload the SDK refuses says "You're offline"
**Status:** Fixed, not verified live
**See also:** B266, the same misclassification on the pin, unpin, search and node-load paths.
**Found:** 2026-10-11, issue #38: uploading an image from the Photos page showed "You're offline"
while other uploads went through. Each `upload_photo` failed about 5 µs after it started, with no
request sent, and the journal gave no reason.
@@ -1690,6 +2263,7 @@ taken as this session's, so the daemon keeps writing its later rotations.
### B202 — A refused session shows as offline forever, and nothing says to sign in again
**Status:** Fixed, not verified live
**See also:** B267. Other errors at a start from a warm cache still mount as offline.
**Found:** 2026-10-09, issue #35: the tray and `pdfs diagnose` said "offline, 3 queued", and
diagnose also said "keyring session: ok", while every call in the log failed with
`InvalidRefreshToken (http 400)`. `pdfs sync now` and restarting the service changed nothing.
@@ -2631,6 +3205,8 @@ fix).
### B165 — A stop waits on a stuck thread until systemd kills the daemon
**Status:** Fixed (unverified).
**See also:** B259. A pass still has no deadline while the daemon runs, and a stuck or dead
sync engine is not reported.
**Found:** 2026-10-04, by the account run on 2.10.0 (cleanup after on-demand pairing 2). The
daemon was stopped at 15:53:04 and logged "control socket closing", but never "sync engine
stopping" or "daemon stopping". systemd killed it 15 s later (`TimeoutStopSec`). The run's
@@ -5877,6 +6453,13 @@ Only the three namespace ops were strictly required (the other four silence
themselves once the kernel caches `ENOSYS`), but implementing all seven keeps the
refusal explicit and documented at the call site rather than inherited.
`copy_file_range` has since been implemented: the daemon copies a range whose bytes are on this
machine (an open write handle, a queued write, the cache) into the destination's scratch file,
and answers `EOPNOTSUPP` for anything it would have to download. Unlike `ENOSYS`, that errno makes
the kernel copy that one call through read and write and keeps asking the daemon next time.
A warm `cp` of a 256 MiB file inside the mount went from about 100 MiB/s to about 450 MiB/s
(sim, debug build; `a_copy_inside_the_mount_measures_its_speed`).
**Verified:** full live acceptance run after the fix — 19 pass / 1 skip, the
capability diff identical to the pre-fix run, and **0** `Not Implemented` lines
in the journal across the whole run (was 17, 10 of them `link`).
+56
View File
@@ -11,7 +11,15 @@ scratch (user data in `staging/` and `recovery/` is never touched by this).
## [Unreleased]
## [3.6.0] — 2026-10-27
Schema: **44**, unchanged.
### Added
- **`pdfs diagnose` names the key that keeps My Files closed.** When you are signed in and the
service does not answer, a new **Keys** section asks Proton which account and address keys
open, which account key each address key depends on, and which address key the My Files share
is encrypted to. It shows key IDs only, no secrets (#40).
- **A COPR repository for Fedora**, so `dnf upgrade` brings each new release:
`sudo dnf copr enable narl-rs/proton-drive-linux`. Every release also ships the vendored crates
as `proton-drive-linux-vendor-<version>.tar.xz`, which the COPR build uses offline (#42).
@@ -19,8 +27,33 @@ scratch (user data in `staging/` and `recovery/` is never touched by this).
the app asks GitHub once a day and shows a banner with a **Download** button when a newer
version is out. It is off by default, and not offered for packages that a package manager
updates (COPR, the AUR, Nix) (#42).
- **Copy files and folders on Proton Drive**, without downloading or uploading them. In My files,
right-click an item and choose **Make a Copy** to copy it next to itself, or **Copy To…** to pick
another folder; **Copy To…** also works on a selection. From a terminal:
`pdfs copy ~/ProtonDrive/report.pdf ~/ProtonDrive/Archive`. A folder is copied with everything
in it, and a name already taken gets a " (1)" suffix. A file or folder with changes that have
not synced yet is refused until they have.
- **Change a public link without getting a new one.** In **Share…**, open **Link settings** under
the link to switch between Viewer and Editor, set, change or remove the password, and change or
remove the expiry. The address stays the same, so people who already have the link keep it.
From a terminal: `pdfs public-link update PATH ID [--role] [--password | --no-password]
[--expires EPOCH | --no-expiry]`. `pdfs members` now also shows when a link expires.
- **Folder sizes.** **Details** shows how much a folder holds and how many items are in it, as
Proton Drive counts it, including items in the trash. From a terminal: `pdfs du PATH`.
- **Sync a folder from scratch.** In a synced folder's menu on **Sync → Folders**, **Sync From
Scratch…** forgets what was synced before and compares the folder with Proton Drive again. A
file on one side only is copied to the other, and a file that differs keeps both versions.
Nothing is deleted. From the terminal: `pdfs sync reset ID`.
- **`--json` for `pdfs members`, `pdfs du` and `pdfs public-link create` / `update`**, so scripts
can read a link's id, role, password flag and expiry, and a folder's size and item count.
### Changed
- **Copying a file inside the mount is faster** when the computer already has the file, for
example with `cp` or a file manager: the copy is made on this computer, about four times
faster for a 256 MiB file, instead of going through the mount page by page. A file that is
only on Proton Drive is copied as before. The copy is then uploaded as usual.
- **Reorganised documentation, now also on the
[wiki](https://github.com/narrrl/proton-drive-linux/wiki).** A new
[Getting started](https://github.com/narrrl/proton-drive-linux/wiki/Getting-Started) page and
@@ -28,6 +61,29 @@ scratch (user data in `staging/` and `recovery/` is never touched by this).
pages for files, synced folders, Photos, sharing, computers, and the launcher and tray.
Installation puts the COPR repository first for Fedora and explains updating and uninstalling
for each kind of package. Each release publishes `docs/` to the wiki.
- **Acceptance cases for the 3.6.0 features:** `copy_file_range` over several calls and at
offsets, `pdfs copy` (including B256), `pdfs du`, `pdfs public-link update`, Sync From Scratch,
and the switch of an emptied mirror folder to on-demand (B255). `docs/TESTING.md` lists what to
check by hand in the app.
### Fixed
- **A synced folder whose files were all gone from this computer no longer gets stuck** (#46).
The sync still refuses to delete everything on Proton Drive, but a switch to online only now
goes ahead and shows what Proton Drive has, instead of waiting for good. This includes a
switch asked for right after the files went, which was refused with "… was deleted here but
not on Proton Drive yet". To download the files again instead, sync the folder from scratch.
Restoring folders also logs why it skipped a folder.
- **The first sync of a folder shows how far its scan got** (#46). Before anything downloads,
the first pass reads every item on Proton Drive, which takes many minutes for a large folder.
The folder showed only "checking for changes…" all that time, and `pdfs sync list` only
`scanning…`, which looked stuck. Both now count the items checked so far.
- **Saved public links show the name of what they point at**, in the app and in `pdfs bookmarks`,
instead of "Shared link". For a link with a custom password, the address no longer has the custom
password stuck to its end, so it matches the link the web shows and the page asks for the
password.
## [3.5.0] — 2026-10-16
+9 -4
View File
@@ -58,6 +58,8 @@ browser, complete it, then press Enter.
| `pdfs mkdir PARENT NAME` | Create a folder. |
| `pdfs rename PATH NEW_NAME` | Rename a file or folder. |
| `pdfs move PATH... NEW_PARENT` | Move one or more files or folders into another folder. The two may be in different locations (My files, an on-demand or a mirrored synced folder); the move happens on Proton Drive, without downloading or uploading anything. |
| `pdfs copy PATH... NEW_PARENT` | Copy one or more files or folders into another folder in My files. The copy happens on Proton Drive, without downloading or uploading anything; a name already taken gets a " (1)" suffix. Changes that have not synced yet must sync first. |
| `pdfs du PATH` | Show how much a folder holds on Proton Drive and how many items are in it. The size includes items in the trash. |
| `pdfs rm PATH` | Move to the Trash. |
| `pdfs upload SOURCES... [-t FOLDER]` | Upload local files and folders (recursively) in the background. Follow with `pdfs transfers`. |
| `pdfs refresh [TARGET] [--full]` | Forget a cached listing so the next read fetches it again. `TARGET` is a folder, `trash` or `photos`; `--full` with `photos` re-reads every photo's metadata. |
@@ -82,6 +84,7 @@ commands are for scripts and for cases where the mount is not available.
| `pdfs sync rm ID [--delete-remote]` | Stop syncing. Local files stay; `--delete-remote` also removes the copy on Proton Drive. |
| `pdfs sync mode ID mirror\|ondemand` | Switch between a full local copy (`mirror`) and online only (`ondemand`). |
| `pdfs sync now [ID]` | Start a sync pass now, for all folders or one. |
| `pdfs sync reset ID` | Forget what a synced folder synced before and compare it with Proton Drive again. A file on one side only is copied to the other, and a file that differs keeps both versions. Nothing is deleted. |
| `pdfs sync pause [--for 1h] [--folder ID]` | Pause uploads and synced folders. Reading files keeps working. `--for` ends the pause by itself (`30m`, `1h`, `2d`). |
| `pdfs sync resume [--folder ID]` | Resume after a pause. |
| `pdfs sync limit [--up RATE] [--down RATE]` | Cap bandwidth, for example `500K` or `2M` per second. `0` removes a cap. Without options, prints the current caps. |
@@ -133,10 +136,11 @@ commands are for scripts and for cases where the mount is not available.
| Command | Description |
|---|---|
| `pdfs share PATH EMAILS... [--role viewer\|editor\|admin] [--message TEXT]` | Invite people to a file or folder. |
| `pdfs members PATH` | List members, pending invitations and the public link. |
| `pdfs members PATH` | List members, pending invitations and the public link, with the link's id and expiry. |
| `pdfs share-role PATH ID KIND ROLE` | Change a member's or invitation's role. |
| `pdfs unshare PATH [ID KIND]` | Stop sharing, or remove one member or invitation. |
| `pdfs public-link create PATH [--role] [--password] [--expires EPOCH]` | Create a public link. `--expires` takes Unix seconds. |
| `pdfs public-link update PATH ID [--role] [--password TEXT \| --no-password] [--expires EPOCH \| --no-expiry]` | Change a public link's role, password or expiry. The URL stays the same, and an option you leave out is kept. |
| `pdfs public-link remove PATH ID` | Remove a public link. |
| `pdfs shared` | List what you share. |
| `pdfs shared-with-me [UID]` | List what others share with you, or the contents of one shared folder. |
@@ -175,7 +179,7 @@ commands are for scripts and for cases where the mount is not available.
| Command | Description |
|---|---|
| `pdfs diagnose` | Check the installation and print a report. Works without the service. Exits non-zero if a check fails. |
| `pdfs diagnose` | Check the installation and print a report. Works without the service. When the service does not answer, also asks Proton which of your keys open. Exits non-zero if a check fails. |
| `pdfs diagnostics` | What a running service is doing now: worker threads, queues, requests in flight, memory. For a service that has stopped answering. |
| `pdfs cache inspect [--deep]` | Database size, reclaimable space, row counts, cache use against the limit. `--deep` also runs SQLite's integrity check (slow). |
| `pdfs cache vacuum` | Compact the database. Needs free space for a second copy of it while it runs. |
@@ -186,8 +190,9 @@ commands are for scripts and for cases where the mount is not available.
## Scripting with `--json`
The global `--json` flag makes query commands print JSON: `status`, `ls`, `pins`, `sync list`,
`devices list`, `locations`, `transfers`, `activity`, `trash` and `cache inspect`. Commands that change
something keep their human-readable output; check their exit code.
`devices list`, `locations`, `transfers`, `activity`, `trash`, `cache inspect`, `members` and `du`.
`public-link create` and `public-link update` print the link they made or changed. Other commands
that change something keep their human-readable output; check their exit code.
```bash
pdfs --json status | jq -r '.mount.mountpoint'
+1 -1
View File
@@ -102,7 +102,7 @@ for l in $(cat po/LINGUAS); do msgfmt --check --statistics -o /dev/null po/$l.po
| Tool | Use |
|---|---|
| `RUST_LOG` | Log filter for every binary, for example `RUST_LOG=pdfs_fuse::sync=trace,info` |
| `pdfs diagnose` | Static checks: directories, FUSE, keyring, socket, database integrity. Works without the daemon |
| `pdfs diagnose` | Static checks: directories, FUSE, keyring, socket, database integrity. Works without the daemon. When the daemon does not answer, also traces which account and address keys open (`pdfs_core::keycheck`) |
| `pdfs diagnostics` | Live state of a running daemon: each worker's current job and its age, queue depth, control requests in flight, memory |
| `pdfs cache inspect --deep` | Database size, row counts and SQLite `integrity_check` |
| `pdfs --json …` | Raw protocol payloads for query commands |
+8 -1
View File
@@ -59,7 +59,14 @@ folder off instead.
The **My files** page shows the same files without going through the folder. It has a grid and
a list view, sorting, thumbnails (including camera RAW previews), multi-select, and drag and drop
to upload. **Details** (Alt+Enter) shows the size, dates, sharing and version history of the
selected item.
selected item. For a folder, the size and the number of items in it come from Proton Drive and
include items in the trash; `pdfs du PATH` prints the same.
Right-click an item for more. **Make a Copy** copies it into the same folder, and **Copy To…**
copies it, or a whole selection, into another folder in My files. The copy is made on Proton
Drive, so nothing is downloaded or uploaded, and a name already taken gets a " (1)" suffix. An
item with changes that have not reached Proton Drive yet can be copied once they have. From a
terminal: `pdfs copy PATH... NEW_PARENT`.
## Trash
+1 -1
View File
@@ -102,7 +102,7 @@ That URL follows `main`. To stay on releases, pin the input to a release tag ins
the tag to update:
```nix
inputs.proton-drive-linux.url = "github:narrrl/proton-drive-linux/v3.5.0";
inputs.proton-drive-linux.url = "github:narrrl/proton-drive-linux/v3.6.0";
```
Each release tag is built with Nix before the release is published.
+8 -1
View File
@@ -33,6 +33,12 @@ In **Share…**, create a public link. Anyone with the link can open the item. Y
link with a password and let it expire on a date. From a terminal, use `pdfs public-link create`
and `pdfs public-link remove`.
To change a link later, open **Link settings** under it. You can switch it between Viewer and
Editor, set, change or remove its password, and change or remove its expiry. The address stays
the same, so everyone who has the link keeps it. From a terminal, use `pdfs public-link update`;
`pdfs members` shows the link's id. A link made long ago by an older Proton app may not be
changeable; remove it and create a new one instead.
## See what you share
**Shared by me** lists everything you share. Filter it by links, by people, or by expired links.
@@ -47,6 +53,7 @@ a viewer are read-only.
- **Invitations** wait in a banner at the top of the page until you accept or reject them. From a
terminal: `pdfs invitations list`, then `accept ID` or `reject ID`.
- **Saved public links** have their own view. `pdfs bookmarks` manages them.
- **Saved public links** have their own view, listed by the name of what they point at.
`pdfs bookmarks` manages them.
- **Leave…** a share you no longer need, or run `pdfs leave UID` with the ID from
`pdfs shared-with-me`.
+8
View File
@@ -76,6 +76,14 @@ again when they move into a synced one.
Proton Drive. To delete the copy on Proton Drive as well, tick **Also delete from Proton Drive**,
or add `--delete-remote`.
## Sync a folder from scratch
**Sync From Scratch…** in the folder's menu, or `pdfs sync reset ID`, makes this computer forget
what it synced in the folder before and compare it with Proton Drive again, as on the first sync.
A file on one side only is copied to the other, and a file that differs keeps both versions.
Nothing is deleted. Use it when a folder's files went missing on this computer and the sync
refuses to go on (see [Troubleshooting](TROUBLESHOOTING.md#a-synced-folder-stays-in-error-and-nothing-downloads)).
## Excluding files with `.pdfsignore`
Synced folders skip build output, dependency folders and editor leftovers. Two sets of rules
+31 -1
View File
@@ -116,7 +116,10 @@ for something a network filesystem cannot do.
the move cases in every pairing of on-demand and mirror. Before a folder switches to on-demand,
a new file is written into it; once mounted, the file must be listed with its bytes. In every
pairing each folder must still hold the file written into it at setup.
4. Removes everything it made, locally and on Drive, and reports `cleanup`. A run that leaves
4. Registers a third sync folder for the recovery cases (`account` in `--list`): Sync From Scratch
must change nothing that already matched, and a mirror folder whose local files are all gone
must still switch to on-demand, with every file intact on Drive (B255).
5. Removes everything it made, locally and on Drive, and reports `cleanup`. A run that leaves
anything behind fails.
Cleanup finds remote folders by uid, never by name, so nothing of yours is touched. It removes
@@ -199,6 +202,15 @@ Besides the syscall contract:
both directions, several sources in one `pdfs move`, and a tree through My files and out again.
The refusals must move nothing and lose nothing: a name already taken, a folder into itself, a
mirrored source holding something Drive lacks, a file still open for writing.
- **`copy_file_range`.** A 17 MiB copy that takes several calls, copies into the middle of a file
and past its end, a short copy at the source's end, and a copy from a file whose writes are not
released yet. On a live mount the results are also read back from Drive.
- **Copies, sizes and links on Drive** (My files only). `pdfs copy` of a tree and of a file beside
itself, read back as listed and after `pdfs refresh`, with nothing queued; the refusal while a
file has writes Drive lacks, and never the version before a save (B256). `pdfs du` counts every
item below a folder and keeps counting one moved to the trash. `pdfs public-link update` keeps
the URL and every option it was not asked to change, and refuses a link id that is not the
folder's.
- **The CLI.** `pdfs` into a closed pipe exits 0 with nothing on stderr; `pdfs mkdir`, `rename`
and `rm` by local path, and their refusals.
@@ -263,3 +275,21 @@ typo it is. A release run leaves it unset.
What no automated run covers yet is listed under
[Verification still owed](ROADMAP.md#verification-still-owed).
### By hand in the app
The suite drives the CLI, not the app. Before a release that changes one of these screens, check
it by hand on a test account:
1. **Make a Copy** on a file in My files: the copy appears beside it with ` (1)`, and opens with
the same content. **Copy To…** a folder into another folder: the whole tree arrives.
2. Make a Copy on a file that is still open in an editor with unsaved changes: the app refuses,
and no copy appears.
3. **Details** of a folder: **Size on Proton Drive** shows a size and an item count, and its
tooltip says items in the trash are counted. Details of a file still shows **Size**.
4. **Share → Link settings** on a link with a password: change **Link role**. The link in the
dialog stays the same, and **Remove the password** is still offered. Then remove the password
and pick an expiry under **Expires**: the link row shows when it expires.
5. **Sync From Scratch…** on a mirror folder: after the pass, the folder holds the same files and
no conflict copies.
6. Each of the above in a second language (`LANGUAGE=de pdfs-app`): no English left.
+23 -1
View File
@@ -20,6 +20,11 @@ information listed under [Reporting a problem](#reporting-a-problem).
Checks the installation and prints a report. It does not need the service, because the state
worth diagnosing is often the one where the service will not start.
When you are signed in but the service does not answer, it also asks Proton which of your keys
open, under **Keys**: each account key, each address key, and the address key your My Files
share is encrypted to. A key that does not open is named with the key it depends on. The section
shows key IDs, which are not secret, so you can post it in an issue.
```console
$ pdfs diagnose
Paths
@@ -146,7 +151,8 @@ session before it shows your files.
Your account keys cannot open your Drive, which happens after a password reset: Proton locks the
files that were encrypted with the old keys. Since 3.4.1 the client then starts a new, empty My
files. Your old files are still there, locked: open [drive.proton.me](https://drive.proton.me) and
restore them with your previous password.
restore them with your previous password. If your files are not locked and the error stays, run
`pdfs diagnose` and post its **Keys** section in an issue.
### Everything in the folder hangs
@@ -226,6 +232,22 @@ Check that:
`pdfs sync now` starts a sync pass at once.
### A synced folder stays in error and nothing downloads
The log says `every one of the N synced paths is missing locally; refusing to trash them on
Drive`. The folder synced before, and now none of its files are on this computer. The sync
stops so that it does not delete them on Proton Drive too. Check that the folder is the right
one and that its disk is mounted. To get the files back, choose **Sync From Scratch…** in the
folder's menu on **Sync → Folders**, or run:
```bash
pdfs sync list # find the folder's ID
pdfs sync reset 1
```
This downloads what Proton Drive has and uploads what only this computer has. Nothing is deleted.
If you deleted the files on purpose, delete them on Proton Drive too, or stop syncing the folder.
### Files named `(sync-conflict …)` appeared
The file changed in two places at once. Nothing was lost. See
+1 -1
View File
@@ -9,7 +9,7 @@
# Build + install: cd packaging && makepkg -fi
pkgname=proton-drive-linux
pkgver=3.5.0
pkgver=3.6.0
pkgrel=1
pkgdesc="Unofficial Proton Drive client for Linux: files-on-demand FUSE mount, CLI, GTK4 app + tray"
arch=('x86_64')
@@ -1,6 +1,6 @@
pkgbase = proton-drive-for-linux-bin
pkgdesc = Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries)
pkgver = 3.5.0
pkgver = 3.6.0
pkgrel = 1
url = https://proton-drive.narl.io
install = proton-drive.install
@@ -22,21 +22,21 @@ pkgbase = proton-drive-for-linux-bin
optdepends = xdg-utils: "Open folder" action in the tray menu
optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf)
optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding
provides = proton-drive-for-linux=3.5.0
provides = proton-drive-linux=3.5.0
provides = proton-drive-for-linux=3.6.0
provides = proton-drive-linux=3.6.0
provides = pdfs
conflicts = proton-drive-linux
conflicts = proton-drive-for-linux
conflicts = proton-drive-for-linux-git
options = !strip
options = !debug
source = proton-drive-linux-3.5.0-x86_64.tar.gz::https://github.com/narrrl/proton-drive-linux/releases/download/v3.5.0/proton-drive-linux-3.5.0-x86_64.tar.gz
source = io.narl.proton-drive-linux-3.5.0.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.5.0/packaging/io.narl.proton-drive-linux.desktop
source = io.narl.proton-drive-linux-tray-3.5.0.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.5.0/packaging/io.narl.proton-drive-linux-tray.desktop
source = io.narl.proton-drive-linux-3.5.0.svg::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.5.0/packaging/io.narl.proton-drive-linux.svg
source = proton-drive-3.5.0.service::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.5.0/packaging/proton-drive.service
source = LICENSE-3.5.0::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.5.0/LICENSE
sha256sums = 902f60de5f471cb15c84d0ecb29cb4644a0ac52ce36d2be54c16b4befeba8de0
source = proton-drive-linux-3.6.0-x86_64.tar.gz::https://github.com/narrrl/proton-drive-linux/releases/download/v3.6.0/proton-drive-linux-3.6.0-x86_64.tar.gz
source = io.narl.proton-drive-linux-3.6.0.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.6.0/packaging/io.narl.proton-drive-linux.desktop
source = io.narl.proton-drive-linux-tray-3.6.0.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.6.0/packaging/io.narl.proton-drive-linux-tray.desktop
source = io.narl.proton-drive-linux-3.6.0.svg::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.6.0/packaging/io.narl.proton-drive-linux.svg
source = proton-drive-3.6.0.service::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.6.0/packaging/proton-drive.service
source = LICENSE-3.6.0::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v3.6.0/LICENSE
sha256sums = 8e9b76b0df238af1c8978c33b4c27c0c548e1faa81b368d0d271ed564d0a76c7
sha256sums = d190d6771fff0b975271fb62fd3fb0fa8bbf17d7278c10853c18557855b0c123
sha256sums = 1d95e250370220e3bb6aead6ce34d1d6cc759f42c4803b75f1667e78ff47c6c0
sha256sums = e598a93c2715c3a9fdc7fb937902083d4a1135e8fbdfe6fe7e37f0383ebbd828
@@ -1,7 +1,7 @@
# Maintainer: Nils Pukropp <contact@narl.io>
pkgname=proton-drive-for-linux-bin
pkgver=3.5.0
pkgver=3.6.0
pkgrel=1
pkgdesc="Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries)"
arch=('x86_64')
@@ -36,7 +36,7 @@ source=("proton-drive-linux-$pkgver-x86_64.tar.gz::$_repo/releases/download/$_ta
"io.narl.proton-drive-linux-$pkgver.svg::$_raw/packaging/io.narl.proton-drive-linux.svg"
"proton-drive-$pkgver.service::$_raw/packaging/proton-drive.service"
"LICENSE-$pkgver::$_raw/LICENSE")
sha256sums=('902f60de5f471cb15c84d0ecb29cb4644a0ac52ce36d2be54c16b4befeba8de0'
sha256sums=('8e9b76b0df238af1c8978c33b4c27c0c548e1faa81b368d0d271ed564d0a76c7'
'd190d6771fff0b975271fb62fd3fb0fa8bbf17d7278c10853c18557855b0c123'
'1d95e250370220e3bb6aead6ce34d1d6cc759f42c4803b75f1667e78ff47c6c0'
'e598a93c2715c3a9fdc7fb937902083d4a1135e8fbdfe6fe7e37f0383ebbd828'
@@ -1,6 +1,6 @@
pkgbase = proton-drive-for-linux-git
pkgdesc = Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (git)
pkgver = 3.5.0.r0.geb25c16
pkgver = 3.6.0.r0.g2720191
pkgrel = 1
url = https://proton-drive.narl.io
install = proton-drive.install
@@ -25,8 +25,8 @@ pkgbase = proton-drive-for-linux-git
optdepends = xdg-utils: "Open folder" action in the tray menu
optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf)
optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding
provides = proton-drive-for-linux=3.5.0.r0.geb25c16
provides = proton-drive-linux=3.5.0.r0.geb25c16
provides = proton-drive-for-linux=3.6.0.r0.g2720191
provides = proton-drive-linux=3.6.0.r0.g2720191
provides = pdfs
conflicts = proton-drive-linux
conflicts = proton-drive-for-linux
@@ -1,7 +1,7 @@
# Maintainer: Nils Pukropp <contact@narl.io>
pkgname=proton-drive-for-linux-git
pkgver=3.5.0.r0.geb25c16
pkgver=3.6.0.r0.g2720191
pkgrel=1
pkgdesc="Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (git)"
arch=('x86_64')
@@ -1,6 +1,6 @@
pkgbase = proton-drive-for-linux
pkgdesc = Unofficial Proton Drive client: files-on-demand FUSE mount, CLI, GTK4 app and tray
pkgver = 3.5.0
pkgver = 3.6.0
pkgrel = 1
url = https://proton-drive.narl.io
install = proton-drive.install
@@ -24,13 +24,13 @@ pkgbase = proton-drive-for-linux
optdepends = xdg-utils: "Open folder" action in the tray menu
optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf)
optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding
provides = proton-drive-linux=3.5.0
provides = proton-drive-linux=3.6.0
provides = pdfs
conflicts = proton-drive-linux
conflicts = proton-drive-for-linux-bin
conflicts = proton-drive-for-linux-git
options = !lto
source = proton-drive-for-linux-3.5.0.tar.gz::https://github.com/narrrl/proton-drive-linux/archive/refs/tags/v3.5.0.tar.gz
sha256sums = 5eb0e001ec2301ffb8b2b8429e907fab32810d6eba541bf978a0ea97867d8bc9
source = proton-drive-for-linux-3.6.0.tar.gz::https://github.com/narrrl/proton-drive-linux/archive/refs/tags/v3.6.0.tar.gz
sha256sums = d9b16b1027816a379d6a11c6396d779f751a6f94becaa16b17439420f561d13d
pkgname = proton-drive-for-linux
@@ -1,7 +1,7 @@
# Maintainer: Nils Pukropp <contact@narl.io>
pkgname=proton-drive-for-linux
pkgver=3.5.0
pkgver=3.6.0
pkgrel=1
pkgdesc="Unofficial Proton Drive client: files-on-demand FUSE mount, CLI, GTK4 app and tray"
arch=('x86_64')
@@ -26,7 +26,7 @@ options=('!lto')
# Stops the running daemon before removal, so the mount stops mirroring deletes.
install=proton-drive.install
source=("$pkgname-$pkgver.tar.gz::$_repo/archive/refs/tags/v$pkgver.tar.gz")
sha256sums=('5eb0e001ec2301ffb8b2b8429e907fab32810d6eba541bf978a0ea97867d8bc9')
sha256sums=('d9b16b1027816a379d6a11c6396d779f751a6f94becaa16b17439420f561d13d')
_srcname="proton-drive-linux-$pkgver"
+1 -1
View File
@@ -10,7 +10,7 @@
# Match Arch PKGBUILD `!lto` — LTO has broken some GTK/Rust links in practice.
%global _lto_cflags %{nil}
%{!?version: %global version 3.5.0}
%{!?version: %global version 3.6.0}
%global forge https://github.com/narrrl/proton-drive-linux
# Files come from the checkout, or from the unpacked tag archive.
%global src %{?git_dir}%{!?git_dir:.}
+602 -403
View File
File diff suppressed because it is too large. Load diff
+600 -403
View File
File diff suppressed because it is too large. Load diff
+601 -403
View File
File diff suppressed because it is too large. Load diff
+602 -403
View File
File diff suppressed because it is too large. Load diff
+602 -403
View File
File diff suppressed because it is too large. Load diff
+602 -403
View File
File diff suppressed because it is too large. Load diff
+602 -411
View File
File diff suppressed because it is too large. Load diff
+608 -403
View File
File diff suppressed because it is too large. Load diff
+601 -403
View File
File diff suppressed because it is too large. Load diff
+506 -9
View File
@@ -566,6 +566,357 @@ def test_mmap_and_copy_paths(ctx: Context) -> None:
)
def _copy_range(ctx: Context, key: str, src: int, dst: int, count: int, offset_src: int, offset_dst: int) -> int | None:
"""`copy_file_range` until `count` bytes moved or the source ran out.
Returns how many bytes moved, or None when the first call was refused
cleanly. The mount copies at most 16 MiB a call, and `cp` loops the same
way, so a large copy reaches it as several calls with growing offsets.
"""
moved = 0
first = ctx.probe(
key,
lambda: os.copy_file_range(src, dst, count, offset_src, offset_dst),
allow=CLEAN_UNSUPPORTED | {errno.EXDEV},
)
if not first.ok:
return None
step = first.value
calls = 1
while step and moved + step < count:
moved += step
step = os.copy_file_range(src, dst, count - moved, offset_src + moved, offset_dst + moved)
calls += 1
ctx.note(f"{key}.calls", calls)
return moved + step
def test_copy_file_range_ranges(ctx: Context) -> None:
"""`copy_file_range` beyond the first call, at offsets, and from an open file.
The mount answers the call from bytes this machine already has: the write
handle's scratch, a pending upload, or the cache. Each answer is a write
into the destination, so it has to land exactly where the offsets say,
fill a gap past the end with zeros, and survive the upload.
"""
root = ctx.root
big = pattern(17 * 1024 * 1024 + 4099, "cfr-big")
source = root / "cfr-source.bin"
write_durable(source, big)
def copy(key: str, src_path: Path, dst_path: Path, count: int, offset_src: int = 0, offset_dst: int = 0) -> int | None:
src = os.open(src_path, os.O_RDONLY)
dst = os.open(dst_path, os.O_CREAT | os.O_WRONLY, 0o600)
try:
moved = _copy_range(ctx, key, src, dst, count, offset_src, offset_dst)
os.fsync(dst)
finally:
os.close(src)
os.close(dst)
return moved
expected: dict[Path, bytes] = {}
whole = root / "cfr-whole.bin"
moved = copy("whole", source, whole, len(big))
if moved is not None:
check(moved == len(big), f"whole: copy_file_range moved {moved} bytes, expected {len(big)}")
expected[whole] = big
# Into the middle of a file that exists, and past the end of another.
middle = root / "cfr-middle.bin"
base = pattern(300_000, "cfr-base")
write_durable(middle, base)
moved = copy("middle", source, middle, 100_000, offset_src=5_000_000, offset_dst=123_457)
if moved is not None:
check(moved == 100_000, f"middle: copy_file_range moved {moved} bytes, expected {100_000}")
expected[middle] = base[:123_457] + big[5_000_000:5_100_000] + base[223_457:]
gap = root / "cfr-gap.bin"
write_durable(gap, b"head")
moved = copy("gap", source, gap, 70_001, offset_src=1, offset_dst=200_000)
if moved is not None:
check(moved == 70_001, f"gap: copy_file_range moved {moved} bytes, expected {70_001}")
expected[gap] = b"head" + bytes(200_000 - 4) + big[1:70_002]
# Past the source's end the copy is short, and at it nothing moves.
tail = root / "cfr-tail.bin"
moved = copy("tail", source, tail, 10_000, offset_src=len(big) - 99)
if moved is not None:
check(moved == 99, f"tail: copy_file_range moved {moved} bytes, expected {99}")
expected[tail] = big[-99:]
# From a file whose last write has not been released yet.
unsaved = root / "cfr-unsaved.bin"
fresh = pattern(600_001, "cfr-unsaved")
writer = os.open(unsaved, os.O_CREAT | os.O_RDWR | os.O_TRUNC, 0o600)
try:
write_all(writer, fresh)
moved = copy("unsaved", unsaved, root / "cfr-from-unsaved.bin", len(fresh))
finally:
os.close(writer)
if moved is not None:
check(moved == len(fresh), f"unsaved: copy_file_range moved {moved} bytes, expected {len(fresh)}")
expected[root / "cfr-from-unsaved.bin"] = fresh
def verify(when: str) -> None:
for path, data in expected.items():
size = os.lstat(path).st_size
check(size == len(data), f"{path.name} {when}: {size} bytes, expected {len(data)}")
check_bytes(read(path), data, f"{path.name} {when}")
verify("right after the copy")
if not ctx.is_live or ctx.daemon is None:
return
ctx.daemon.wait_for_queue()
verify("after the upload")
# A server-side copy has a new uid, so nothing of it is cached: reading
# it reads what Drive holds.
if _my_files_root(ctx) is None:
return
landed = root / "cfr-landed"
landed.mkdir()
ctx.daemon.wait_for_queue()
for path in expected:
_pdfs_copy(ctx.daemon, path, landed)
for path, data in expected.items():
check_bytes(read(landed / path.name), data, f"{path.name} as Drive holds it")
def _my_files_root(ctx: Context) -> Path | None:
"""The My files mount the case runs in, or None anywhere else.
`pdfs copy`, `pdfs du` and public links address a path inside My files;
synced folders have no path of their own there.
"""
if ctx.daemon is None:
return None
mountpoint = (ctx.daemon.status().get("mount") or {}).get("mountpoint")
if not mountpoint:
return None
mountpoint = Path(mountpoint).resolve()
return mountpoint if mountpoint in ctx.root.resolve().parents else None
def _require_my_files(ctx: Context) -> Daemon:
daemon = ctx.require_daemon()
if _my_files_root(ctx) is None:
raise Skip("needs a root inside My files")
return daemon
COPIED_TO = re.compile(r"^copied to (.+)$")
def _pdfs_copy(daemon: Daemon, source: Path, dest: Path) -> str:
"""`pdfs copy`, returning the name the copy got."""
message = daemon.command("copy", str(source), str(dest)).strip()
match = COPIED_TO.match(message)
check(match is not None, f"pdfs copy {source.name}: unexpected reply {message!r}")
return match[1]
def _pdfs_copy_refused(daemon: Daemon, source: Path, dest: Path) -> str | None:
"""`pdfs copy` expected to be refused for changes Drive lacks: the refusal,
or None if the copy went ahead (and returned its name in the log)."""
try:
daemon.command("copy", str(source), str(dest))
except RuntimeError as error:
detail = str(error)
check("have not reached Proton Drive" in detail, f"pdfs copy {source.name}: unexpected refusal: {detail}")
return detail
return None
def test_server_copy(ctx: Context) -> None:
"""`pdfs copy` makes an exact copy on Drive and uploads nothing.
A tree and a lone file are copied; the file into its own folder, where
its name is taken and the copy gets the next free one. The copy is read
back as listed and again after its listing is dropped, and nothing of it
may wait in the queue: Drive made it, so there is nothing to send.
"""
daemon = _require_my_files(ctx)
root = ctx.root
name = f"tree-{uuid.uuid4().hex[:8]}"
files = _move_tree(root, name)
dest = root / "copies"
dest.mkdir()
daemon.wait_for_queue()
check(_pdfs_copy(daemon, root / name, dest) == name, f"the copy of {name} was renamed")
note = root / name / "note.txt"
twin = _pdfs_copy(daemon, note, note.parent)
check(twin != note.name, f"the copy of {note.name} beside it kept the name {twin!r}")
check((note.parent / twin).is_file(), f"{twin} is not listed beside {note.name}")
queued = [
item
for item in daemon.queue()
if root.name in (parts := Path(item.get("path", "")).parts) and ("copies" in parts or twin in parts)
]
check(not queued, f"a server copy queued work: {_describe_uploads(queued)}")
def verify(when: str) -> None:
for relative, data in files.items():
size = os.lstat(dest / relative).st_size
check(size == len(data), f"{relative} {when}: {size} bytes, expected {len(data)}")
check_bytes(read(dest / relative), data, f"{relative} {when}")
check((dest / name / "empty-dir").is_dir(), f"the empty folder is missing {when}")
check_bytes(read(note.parent / twin), files[Path(name) / "note.txt"], f"{twin} {when}")
verify("right after the copy")
folders = sorted({dest / relative.parent for relative in files} | {dest / name, note.parent})
for folder in folders:
daemon.command("refresh", str(folder))
verify("after dropping the cached listing")
_unchanged_at(root, files, "after copying it")
def _unchanged_at(root: Path, files: dict[Path, bytes], what: str) -> None:
for relative, data in files.items():
check_bytes(read(root / relative), data, f"{relative} {what}")
def test_server_copy_never_copies_an_old_version(ctx: Context) -> None:
"""`pdfs copy` of a file being written is refused, and never copies the
version before the last save (B256).
Drive copies what it holds. A file open with unsaved writes, or saved but
not yet uploaded, holds more here than there: the copy has to wait.
"""
daemon = _require_my_files(ctx)
root = ctx.root
path = root / "being-written.txt"
write_durable(path, b"first version")
# Its own folder: every case of a run shares the root, and "pdfs copy"
# already made "copies" there.
dest = root / "b256-copies"
dest.mkdir()
daemon.wait_for_queue()
fd = os.open(path, os.O_WRONLY | os.O_TRUNC)
try:
write_all(fd, b"second version, still open")
check(
_pdfs_copy_refused(daemon, path, dest) is not None,
"pdfs copy went ahead while the file had unsaved writes",
)
check(not (dest / path.name).exists(), "a refused copy left a file behind")
finally:
os.close(fd)
# Right after the close: refused while the upload is owed, then the new
# bytes. Never the first version.
saved = b"third version, saved"
write_durable(path, saved)
deadline = time.monotonic() + daemon.timeout
refusals = 0
while _pdfs_copy_refused(daemon, path, dest) is not None:
refusals += 1
check(time.monotonic() < deadline, f"pdfs copy was still refused after {daemon.timeout}s")
time.sleep(1)
ctx.note("refusals before the upload landed", refusals)
check_bytes(read(dest / path.name), saved, "the copy made right after a save")
def test_folder_size(ctx: Context) -> None:
"""`pdfs du` counts exactly what is below a folder, trash included.
Drive adds the size up from encrypted bytes, so it is at least the sum of
the files. A file moved to the trash still counts, as `pdfs du` says.
"""
daemon = _require_my_files(ctx)
root = ctx.root
name = f"sized-{uuid.uuid4().hex[:8]}"
files = _move_tree(root, name)
folder = root / name
items = sum(1 for _ in folder.rglob("*"))
daemon.wait_for_queue()
def size() -> dict:
return json.loads(daemon.command("du", str(folder), json_output=True))
# Drive may add a new tree up a moment after it lands.
deadline = time.monotonic() + daemon.timeout
while (answer := size())["items"] != items and time.monotonic() < deadline:
time.sleep(2)
check(answer["items"] == items, f"pdfs du counts {answer['items']} items, expected {items}")
total = sum(len(data) for data in files.values())
check(answer["size"] >= total, f"pdfs du reports {answer['size']} bytes for {total} bytes of files")
ctx.info(f"{answer['size']} bytes on Drive for {total} bytes in {items} items")
os.unlink(folder / "note.txt")
daemon.wait_for_queue()
check(size()["items"] == items, "a file moved to the trash stopped counting in pdfs du")
def _public_link(daemon: Daemon, path: Path) -> dict | None:
return json.loads(daemon.command("members", str(path), json_output=True)).get("link")
def _link_token(url: str | None) -> str:
"""The part of a public link's URL that names the link, without the
password fragment."""
check(bool(url), "the link has no URL")
return url.split("#", 1)[0]
def test_public_link_update(ctx: Context) -> None:
"""`pdfs public-link update` changes a link and keeps its URL.
Each option changed alone leaves the others as they were: a role change
keeps the password, removing the password keeps the expiry. A link id
that is not the folder's link is refused.
"""
daemon = _require_my_files(ctx)
folder = ctx.root / "linked"
folder.mkdir()
write_durable(folder / "inside.txt", b"behind a public link")
daemon.wait_for_queue()
password = f"pw-{uuid.uuid4().hex[:12]}"
created = json.loads(
daemon.command("public-link", "create", str(folder), "--password", password, json_output=True)
)["link"]
link_id, token = created["id"], _link_token(created["url"])
try:
check(created["has_password"], "the new link has no password")
def update(*args: str) -> dict:
link = json.loads(
daemon.command("public-link", "update", str(folder), link_id, *args, json_output=True)
)["link"]
check(link["id"] == link_id, f"the update replaced the link: {link['id']} for {link_id}")
if link.get("url"):
check(_link_token(link["url"]) == token, "the update changed the link's URL")
listed = _public_link(daemon, folder)
check(listed is not None and listed["id"] == link_id, f"the link is gone after the update: {listed}")
return listed
listed = update("--role", "editor")
check(listed["role"] == "editor", f"the role is {listed['role']} after changing it to editor")
check(listed["has_password"], "changing the role removed the password")
expires = int(time.time()) + 7 * 24 * 3600
listed = update("--no-password", "--expires", str(expires))
check(not listed["has_password"], "--no-password kept the password")
check(listed["expires"] == expires, f"the expiry is {listed['expires']}, expected {expires}")
check(listed["role"] == "editor", "removing the password changed the role")
listed = update("--no-expiry")
check(listed["expires"] is None, f"--no-expiry kept the expiry {listed['expires']}")
try:
daemon.command("public-link", "update", str(folder), f"not-{link_id}", "--role", "viewer")
except RuntimeError as error:
check("not found" in str(error), f"a wrong link id was refused for another reason: {error}")
else:
check(False, "an update with a wrong link id went ahead")
check(_public_link(daemon, folder)["role"] == "editor", "a refused update changed the link")
finally:
with contextlib.suppress(Exception):
daemon.command("public-link", "remove", str(folder), link_id)
def test_namespace_and_errors(ctx: Context) -> None:
root = ctx.root
a, b = root / "dir-a", root / "dir-b"
@@ -2625,6 +2976,7 @@ TESTS = [
Case("positioned and vectored I/O", test_positioned_and_vectored_io),
Case("truncate, growth, sparse ranges, and EOF", test_resize_and_sparse_io),
Case("mmap and zero-copy copy paths", test_mmap_and_copy_paths),
Case("copy_file_range across calls, at offsets, and from an open file", test_copy_file_range_ranges),
Case("namespace operations and POSIX errors", test_namespace_and_errors),
Case("open-handle lifetime across unlink and rename", test_open_lifetime),
Case("names, lookup, stat, and enumeration", test_names_and_enumeration),
@@ -2670,6 +3022,10 @@ TESTS = [
Case("regression B116: trashing a folder drops the creates queued in its subfolders", test_regression_b116_trashed_folder_drops_deeper_creates, (LIVE,)),
Case("regression B117: a moved file's queued upload keeps its path", test_regression_b117_moved_upload_keeps_its_path, (LIVE,)),
Case("regression B119: an uploaded file stays cached", test_regression_b119_uploaded_file_stays_cached, (LIVE,)),
Case("pdfs copy: a copy on Drive reads back exactly and uploads nothing", test_server_copy, (LIVE,)),
Case("regression B256: pdfs copy never copies the version before a save", test_server_copy_never_copies_an_old_version, (LIVE,)),
Case("pdfs du counts what is below a folder, trash included", test_folder_size, (LIVE,)),
Case("pdfs public-link update keeps the URL and the options not changed", test_public_link_update, (LIVE,)),
Case("durability across a daemon restart", test_durability_across_restart, (LIVE,)),
]
@@ -2942,7 +3298,7 @@ def select_cases(kind: str, pool: list[Case] | None = None) -> list[Case]:
# typo worth failing on.
for needle in needles:
check(
any(needle in case.name.lower() for case in TESTS + MOVE_CASES),
any(needle in case.name.lower() for case in TESTS + MOVE_CASES + RECOVERY_CASES),
f"PDFS_ACCEPTANCE_ONLY: {needle!r} matched no tests",
)
return [case for case in cases if any(needle in case.name.lower() for needle in needles)]
@@ -3658,30 +4014,36 @@ class Janitor:
return [
f"{name} ({entry['uid']}) is still in the trash"
for entry in matches
if not self.purge(entry["uid"], name)
if not self.purge(entry["uid"], name, listed=True)
]
def purge(self, uid: str, name: str) -> bool:
def purge(self, uid: str, name: str, listed: bool = False) -> bool:
"""Permanently delete one trashed node the run owns, by uid.
By uid and never by name: a name can match something of the user's, a
uid cannot. The node has to show up in the trash first, which is also
what proves the trash step before it worked, and it has to be gone from
a fresh listing afterwards.
a fresh listing afterwards. `listed` says the caller just found it in
the trash.
The server's own answer comes first, though: it only deletes a node
that is in the trash, and it answers per uid. A listing is a whole-trash
enumeration, which on a large trash can time out on every attempt and
would otherwise keep a folder the server would delete on request.
"""
with contextlib.suppress(Exception):
last_error = ""
try:
reply = self.daemon.command("delete-forever", uid, stdin="yes\n")
if re.search(r"\bpermanently deleted 1 item", reply):
print(f"[cleanup] permanently deleted {name}")
return True
except Exception as error: # noqa: BLE001 - retried, then reported
last_error = f": {error}"
deadline = time.monotonic() + self.daemon.timeout
attempted = False
last_error = ""
# A node seen in the trash before that delete and gone after it was
# deleted by it, whatever it answered. Counting it as never sent kept
# waiting for a node that was already gone, and failed the run.
attempted = listed
while True:
try:
if self.daemon.trashed(uid):
@@ -3793,7 +4155,7 @@ class Janitor:
for entry in trashed:
if STALE_ROOT.match(entry["name"]) and entry["name"] not in live_names:
print(f"[reap] permanently deleting an abandoned root from the trash: {entry['name']}")
if not orphan.purge(entry["uid"], entry["name"]):
if not orphan.purge(entry["uid"], entry["name"], listed=True):
leftovers.append(f"{entry['name']} ({entry['uid']}) is still in the trash")
return leftovers
@@ -3894,7 +4256,7 @@ class ManagedSyncPair:
raise RuntimeError(
f"cannot find {self.pdfs!r}; set PDFS_ACCEPTANCE_PDFS to the CLI binary"
)
if self.paths[0] == self.paths[1]:
if len(set(self.paths)) != len(self.paths):
raise ValueError("managed live paths must be different directories")
deadline = time.monotonic() + self.timeout
last_error = None
@@ -4537,6 +4899,138 @@ def run_move_contract(
return run
# --------------------------------------------------------------------------
# Getting a synced folder unstuck
# --------------------------------------------------------------------------
class RecoveryContext:
"""One mirror folder of the run's own, for cases that reset or empty it.
The mode matrix needs its folders' sentinels intact, so these cases get a
folder nobody else checks.
"""
def __init__(self, pair: ManagedSyncPair, daemon: Daemon) -> None:
self.pair = pair
self.daemon = daemon
self.root = pair.paths[0]
self.id = pair.ids[self.root]
self.infos: list[str] = []
def folder(self) -> dict:
return next(item for item in self.pair.folders() if Path(item["local_path"]).resolve() == self.root)
def tree(self) -> tuple[str, dict[Path, bytes]]:
"""A fresh tree in the folder, with one file named after it, so that a
trash entry of it cannot be the user's."""
name = f"kept-{uuid.uuid4().hex[:8]}"
files = _move_tree(self.root, name)
files[Path(name) / f"{name}.txt"] = b"named after its tree " + name.encode()
write_durable(self.root / name / f"{name}.txt", files[Path(name) / f"{name}.txt"])
return name, files
def trashed(self, names: set[str]) -> list[str]:
self.daemon.command("refresh", "trash")
return sorted(entry["name"] for entry in self.daemon.trash() if entry["name"] in names)
def test_sync_reset_changes_nothing(rc: RecoveryContext) -> None:
"""Sync From Scratch compares the folder with Drive again and changes
nothing that already matched: no deletes, no conflict copies.
`pdfs sync reset` forgets the folder's record of what it synced. The next
pass then sees every file on both sides with no history, which is the
case that must copy what is missing and leave matching files alone.
"""
name, files = rc.tree()
rc.pair.force_sync(rc.root)
rc.daemon.command("sync", "reset", str(rc.id))
# Added after the reset, so the pass after it has to upload it.
later = Path(name) / "added-after-the-reset.txt"
files[later] = b"only here when the pass starts " + name.encode()
write_durable(rc.root / later, files[later])
rc.pair.force_sync(rc.root)
rc.pair.force_sync(rc.root)
_unchanged_at(rc.root, files, "after Sync From Scratch")
copies = [str(path) for path in rc.root.rglob("*") if "(sync-conflict" in path.name]
check(not copies, f"Sync From Scratch made conflict copies: {copies}")
trashed = rc.trashed({name, f"{name}.txt"})
check(not trashed, f"Sync From Scratch trashed {trashed}")
state = rc.folder()
check(state["state"] == "idle", f"the folder is {state['state']} after Sync From Scratch")
def test_emptied_folder_switches_to_ondemand(rc: RecoveryContext) -> None:
"""A mirror folder whose local files are all gone switches to online only,
and Drive keeps every file (B255, issue #46).
The wipe guard rightly refuses to pass the local loss on to Drive, but it
also blocked the push pass a queued switch runs first, so the switch never
landed and the folder stayed in error. Through the mount afterwards, every
file has to read back as it was.
"""
names: set[str] = set()
files: dict[Path, bytes] = {}
for _ in range(2):
name, tree = rc.tree()
names |= {name, f"{name}.txt"}
files.update(tree)
rc.pair.force_sync(rc.root)
for child in list(rc.root.iterdir()):
if child.is_dir() and not child.is_symlink():
shutil.rmtree(child)
else:
child.unlink()
rc.daemon.command("sync", "mode", str(rc.id), "ondemand")
# The folder may pass through error while the guard holds the deletions
# back; the switch has to land anyway.
deadline = time.monotonic() + rc.pair.timeout
while True:
state = rc.folder()
if state["mode"] == "ondemand" and state.get("pending_mode") is None and is_mountpoint(rc.root):
break
check(time.monotonic() < deadline, f"the switch to on-demand did not land: {state}")
time.sleep(2)
for relative, data in files.items():
data_read = rc.pair.through_outage(lambda relative=relative: read(rc.root / relative), f"reading {relative}", rc.infos)
check_bytes(data_read, data, f"{relative} on Drive after the local copy was gone")
sentinel = rc.root / "pdfs-mode-preservation.bin"
check_bytes(read(sentinel), rc.pair.sentinels[rc.root], f"{sentinel.name} on Drive")
trashed = rc.trashed(names)
check(not trashed, f"the switch trashed {trashed} on Drive")
RECOVERY_CASES = [
Case("Sync From Scratch changes nothing that already matched", test_sync_reset_changes_nothing, (LIVE,)),
Case("regression B255: an emptied mirror folder switches to on-demand", test_emptied_folder_switches_to_ondemand, (LIVE,)),
]
def run_sync_recovery(janitor: Janitor, args) -> None:
"""The recovery cases, in a sync folder of their own."""
cases = select_cases(LIVE, RECOVERY_CASES)
if not cases:
return
timeout = int(os.environ.get("PDFS_ACCEPTANCE_SYNC_TIMEOUT", "180"))
path = janitor.new_sync_folder()
pair = ManagedSyncPair([path], timeout)
daemon = Daemon.discover(timeout)
check(daemon is not None, "the daemon stopped answering")
run = Run("sync folder recovery", LIVE, path)
REPORT.append(run)
announce(run, {"folder": f"mirror, {path}", "cases": describe_cases(LIVE, cases, RECOVERY_CASES)})
started = time.monotonic()
pair.create()
context = RecoveryContext(pair, daemon)
run_cases(run, cases, lambda _case: context, max(args.timeout, 8 * timeout), args.fail_fast)
run.wall = time.monotonic() - started
CONSOLE.fact("result", tally(run.counts(), run.wall))
def is_mountpoint(path: Path) -> bool:
"""True for a distinct mount, including FUSE mounts over an existing dir."""
return os.path.ismount(path)
@@ -4694,6 +5188,8 @@ def main() -> int:
print(f"{','.join(case.kinds):<18} {case.name}")
for case in MOVE_CASES:
print(f"{'managed':<18} {case.name}")
for case in RECOVERY_CASES:
print(f"{'account':<18} {case.name}")
return 0
install_interrupt_handlers()
@@ -4814,6 +5310,7 @@ def run_account(reference: Run, args) -> None:
)
paths = [janitor.new_sync_folder(), janitor.new_sync_folder()]
run_managed_matrix(paths, reference, args, janitor)
run_sync_recovery(janitor, args)
finally:
started = time.monotonic()
leftovers = janitor.cleanup()