Compare commits

...
Author SHA1 Message Date
nvrl d2b5916b99 Merge remote-tracking branch 'origin/main' into fix/create-landing-counted 2026-11-01 11:28:03 +01:00
nvrl 4626ce17a7 Merge pull request #69 from narrrl/dependabot/cargo/cargo-f3dd2e6b0d
build(deps): bump the cargo group across 1 directory with 2 updates
2026-11-01 11:27:45 +01:00
nvrl 630f316fe6 Merge pull request #65 from narrrl/fix/clean-stop-runtime
fix(cli): leave the runtime running on a stop so stragglers cannot panic
2026-11-01 11:27:41 +01:00
nvrl 801681f747 Merge remote-tracking branch 'origin/main' into fix/create-landing-counted
# Conflicts:
#	docs/BUGS.md
2026-11-01 11:14:15 +01:00
nvrl e7382f2c7e test(core): name zip's options type the way zip 9 accepts 2026-11-01 11:12:17 +01:00
dependabot[bot] 7046f0f3ef build(deps): bump the cargo group across 1 directory with 2 updates
Bumps the cargo group with 2 updates in the / directory: [zip](https://github.com/zip-rs/zip2) and [rustls](https://github.com/rustls/rustls).


Updates `zip` from 8.6.0 to 9.0.5
- [Release notes](https://github.com/zip-rs/zip2/releases)
- [Changelog](https://github.com/zip-rs/zip2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zip-rs/zip2/compare/v8.6.0...v9.0.5)

Updates `rustls` from 0.23.45 to 0.23.46
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.23.45...v/0.23.46)

---
updated-dependencies:
- dependency-name: zip
  dependency-version: 9.0.5
  dependency-type: direct:production
  dependency-group: cargo
- dependency-name: rustls
  dependency-version: 0.23.46
  dependency-type: indirect
  dependency-group: cargo
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-31 23:39:08 +00:00
nvrl 37e8dae61b Merge pull request #54 from narrrl/docs/bugs-issue-audit
docs(bugs): record the audit of issues #53, #46, #40 and #38
2026-10-31 19:44:18 +01:00
nvrl c52f2098f2 Merge pull request #64 from narrrl/chore/sdk-0.10.1
chore(deps): bump proton-sdk and proton-drive-rs to 0.10.1
2026-10-31 19:40:34 +01:00
nvrl f4a2674b9c Merge pull request #63 from narrrl/fix/sim-flakes-and-open-race
fix(fuse): retry a write open whose queued revision landed under the copy
2026-10-31 19:40:24 +01:00
nvrl 347861bbc1 Merge pull request #62 from narrrl/fix/restore-outcome-and-walk-gaps
fix(fuse,cli): keep a pass going past a failed listing, report skipped restores
2026-10-31 19:40:14 +01:00
nvrl c57763b07f Merge pull request #61 from narrrl/fix/diagnose-coverage
fix(cli): have diagnose report failing writes, a pause and folders in error
2026-10-31 19:40:04 +01:00
nvrl 846dfd8d14 Merge pull request #60 from narrrl/fix/db-errors-not-offline
fix(core): read database errors handed to from_api as internal, not offline
2026-10-31 19:39:53 +01:00
nvrl 2ae99de844 Merge pull request #59 from narrrl/fix/mount-failure-surfaced
fix(cli,core,fuse,gui): say why the mount keeps failing, and back off
2026-10-31 19:39:44 +01:00
nvrl 69ff216e7d Merge pull request #58 from narrrl/fix/never-synced-label
fix(gui): say a folder is waiting for its first sync, not up to date
2026-10-31 19:39:34 +01:00
nvrl b3ea564c6b docs(bugs): keep the index newest first 2026-10-31 19:29:29 +01:00
nvrl 463c97a14d Merge branch 'chore/sdk-0.10.1' into fix/clean-stop-runtime
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:29:05 +01:00
nvrl 40aa8016ea Merge branch 'fix/sim-flakes-and-open-race' into chore/sdk-0.10.1
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:47 +01:00
nvrl d1b7535696 Merge branch 'fix/restore-outcome-and-walk-gaps' into fix/sim-flakes-and-open-race
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:44 +01:00
nvrl ba414c4446 Merge branch 'fix/diagnose-coverage' into fix/restore-outcome-and-walk-gaps
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:41 +01:00
nvrl 5b7f17bdaf Merge branch 'fix/db-errors-not-offline' into fix/diagnose-coverage
# Conflicts:
#	docs/BUGS.md
2026-10-31 19:26:38 +01:00
nvrl 176112a9cb fix(cli): leave the runtime running on a stop so stragglers cannot panic
A stop shut the tokio runtime down after joining the workers it owns.
Threads it never joins, such as a size lookup or a profile backup that
a request started during the stop, then panicked on their next timer
with "A Tokio 1.x context was found, but it is being shutdown". The
process exits right after a stop, so the runtime is now left for the
exit to end. A mount that is retried still shuts its runtime down with
the 2 s limit.

The acceptance journal check now counts panics, and no longer counts
the ENOENT reply errors a stop causes by aborting its FUSE connections.

docs/BUGS.md B276.
2026-10-31 19:09:23 +01:00
nvrl 7957e1ed59 chore(deps): bump proton-sdk and proton-drive-rs to 0.10.1
0.10.1 decrypts address key tokens with every account key that opens,
not only the active ones, and leaves out an address whose keys do not
open instead of failing the account. That is the likely cause of issue
#40, a mount that retried "missing key" forever (B265).
2026-10-31 13:48:01 +01:00
nvrl 53543c91f9 docs(bugs): record the slow-link conflict copy seen in CI (B275) 2026-10-31 13:16:47 +01:00
nvrl fccb3d9561 fix(fuse): retry a write open whose queued revision landed under the copy
A write open copies the queued revision's blob into its scratch file.
When the upload landed between the sample and the copy, the drain had
already deleted the blob, and the open failed with EIO (B274, seen once
in CI). The drain drops the pending entry before the blob, so a failed
copy whose entry moved on now goes around again, as the re-check after
the copy already does.

Two simulations raced the daemon rather than it:
- a_file_deleted_after_a_restart_leaves_drive_with_what_its_create_sent
  let the create's reply through before the kernel's release attached
  the second bytes. That attach resets the attempts, so the retry
  landed before the test could see the failure. It now waits for the
  bytes to be attached.
- a_file_deleted_offline_while_open_still_reads_and_takes_writes read
  the cache as the queue emptied, which is a moment before the
  eviction. It now waits for the cache to empty.
2026-10-31 12:58:11 +01:00
nvrl 6f2b4bb80a fix(fuse,cli): keep a pass going past a failed listing, report skipped restores
A listing that failed anywhere in the remote walk ended the whole pass,
so a first sync of a large folder on a flaky link walked the tree again
and again and never downloaded anything (B271, issue #46). A failed
listing below the root now records that folder as a gap: nothing under
it is deleted, the rest of the pass applies, and the report names it.

A restore that skipped every folder was logged as a successful activity
(B269). The daemon now gets the restored and skipped folders back, logs
any skip as a failed activity with the reasons, and sends the Rewatch
for folders already restored when the database insert fails partway.
`pdfs devices join` points to `pdfs activity` for the outcome.
2026-10-31 11:11:01 +01:00
nvrl a617b2b410 fix(cli): have diagnose report failing writes, a pause and folders in error
diagnose dropped most of the Status reply and never looked at the synced
folders, so it said "No problems found" while a folder would not sync
(B268, issue #53). It now warns about writes that keep failing, a write
a day old, a pause, folders in error or conflict, and a control socket
nothing listens on.
2026-10-31 10:57:34 +01:00
nvrl 639ca359fa fix(fuse,core): count a new file's landing until its bytes are cached
A create that carries a file's bytes retired its op, then cached the
bytes and gave the tree the real uid. Nothing counted that window, so
`pdfs status` read empty while it ran, which B119 promises means the
file reads without the network.

A delete in that window found the placeholder uid in the tree and
trashed the node by its real uid. `queue_trash` asked the tree whether
that uid was open, found no entry, and dropped the cached bytes the open
handle reads. If the cache store ran at the same moment, its tag was
written after the eviction and never removed. CI caught that as a
stray .meta file.

The create's landing is now counted like an upload's, from the op's
retirement until the tree has the real uid. The delete and the replace
pass whether they found the file open. Opening the cache removes a tag
whose bytes are gone, which also covers a crash between an eviction's
two removals. The simulations wait for `Daemon::drained`, which counts
the landing, before they go offline.

B255 and B256 move to fixed: the account run checked both.
2026-10-27 14:44:34 +01:00
17 changed files with 770 additions and 86 deletions

No files matched your search

Generated
+14 -11
View File
@@ -3779,9 +3779,9 @@ dependencies = [
[[package]] [[package]]
name = "proton-drive-rs" name = "proton-drive-rs"
version = "0.10.0" version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f70318394b2815b009c52f804d75747086db93a338ceb6eae62541f2b25d5bf" checksum = "a8fa2a88130883010b32035e208aaa36fb275f6d7d78ed44a1069b1d33d24481"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"base64 0.23.1", "base64 0.23.1",
@@ -3806,9 +3806,9 @@ dependencies = [
[[package]] [[package]]
name = "proton-sdk" name = "proton-sdk"
version = "0.10.0" version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4196392032f56b3bbaac0bb1f24eea62fa7e2a0af72004de8d08e518d14720fb" checksum = "af0bf8d50fa829d1a65b09c6855b606578d02734d4089874c971d1063c23c353"
dependencies = [ dependencies = [
"aes-gcm 0.11.1", "aes-gcm 0.11.1",
"async-trait", "async-trait",
@@ -4380,9 +4380,9 @@ dependencies = [
[[package]] [[package]]
name = "rustls" name = "rustls"
version = "0.23.45" version = "0.23.46"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d41d731c7d2f962d1ccc364cec258de3c0e93b38c2fb3ba97ac74513048d634" checksum = "48e13bd8c0e9365c43cfa5c9e8f9ad49d3c8444926c9aac819e0e4dc503c8fdf"
dependencies = [ dependencies = [
"aws-lc-rs", "aws-lc-rs",
"once_cell", "once_cell",
@@ -4933,7 +4933,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [ dependencies = [
"fastrand", "fastrand",
"getrandom 0.4.3", "getrandom 0.3.4",
"once_cell", "once_cell",
"rustix", "rustix",
"windows-sys 0.61.2", "windows-sys 0.61.2",
@@ -5249,9 +5249,9 @@ dependencies = [
[[package]] [[package]]
name = "typed-path" name = "typed-path"
version = "0.12.3" version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e" checksum = "9ce898823d8cd0f028662d2f349694b1af0a5cc9be83f1d213a00d3147445e1e"
[[package]] [[package]]
name = "typenum" name = "typenum"
@@ -6113,15 +6113,18 @@ dependencies = [
[[package]] [[package]]
name = "zip" name = "zip"
version = "8.6.0" version = "9.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b" checksum = "3f5b58784007ff6e5f280ec82c8596686a3b9b5dd5836c63288d4479f84728b9"
dependencies = [ dependencies = [
"crc32fast", "crc32fast",
"flate2", "flate2",
"getrandom 0.4.3",
"indexmap", "indexmap",
"memchr", "memchr",
"nix",
"typed-path", "typed-path",
"windows-sys 0.61.2",
"zopfli", "zopfli",
] ]
+3 -3
View File
@@ -15,8 +15,8 @@ keywords = ["proton", "drive", "fuse", "sync", "linux"]
categories = ["filesystem", "command-line-utilities"] categories = ["filesystem", "command-line-utilities"]
[workspace.dependencies] [workspace.dependencies]
proton-sdk = { version = "0.10.0" } proton-sdk = { version = "0.10.1" }
proton-drive-rs = { version = "0.10.0" } proton-drive-rs = { version = "0.10.1" }
# Internal crates. # Internal crates.
pdfs-core = { path = "crates/pdfs-core" } pdfs-core = { path = "crates/pdfs-core" }
@@ -41,7 +41,7 @@ ignore = "0.4"
sha2 = "0.11" sha2 = "0.11"
# Google Takeout exports are zip sets; the importer reads their central # Google Takeout exports are zip sets; the importer reads their central
# directories and streams entries out without extracting to disk first. # directories and streams entries out without extracting to disk first.
zip = { version = "8", default-features = false, features = ["deflate"] } zip = { version = "9", default-features = false, features = ["deflate"] }
# Proton's photo duplicate detection is keyed on the plaintext SHA-1, so an # Proton's photo duplicate detection is keyed on the plaintext SHA-1, so an
# import has to speak SHA-1 whatever its age. # import has to speak SHA-1 whatever its age.
sha1 = "0.11" sha1 = "0.11"
+219 -9
View File
@@ -1467,7 +1467,11 @@ fn cmd_sync_restore(yes: bool, device: Option<String>) -> Result<()> {
items: chosen, items: chosen,
}, },
None => CtlRequest::RestoreSyncFolders { items: chosen }, None => CtlRequest::RestoreSyncFolders { items: chosen },
})?) })?)?;
// The daemon acks before it works, so the outcome, and any folder it had
// to skip, only shows up later.
println!("Run `pdfs activity` to see which folders were restored.");
Ok(())
} }
/// Expand a leading `~` in a hand-typed path. The daemon requires an absolute /// Expand a leading `~` in a hand-typed path. The daemon requires an absolute
@@ -2523,14 +2527,30 @@ fn mount_once(mountpoint: Option<PathBuf>) -> Result<pdfs_fuse::MountOutcome> {
"could not save the refreshed session tokens; the next start may need `pdfs login`" "could not save the refreshed session tokens; the next start may need `pdfs login`"
); );
} }
// Dropping the runtime waits for every blocking task, however long; a call release_runtime(rt, matches!(result, Ok(pdfs_fuse::MountOutcome::Shutdown)));
// still stuck on Drive would hold the stop until systemd kills it (B165).
rt.shutdown_timeout(std::time::Duration::from_secs(2));
let outcome = result.context("mount failed")?; let outcome = result.context("mount failed")?;
Ok(outcome) Ok(outcome)
} }
/// Let go of a mount's runtime once the mount has ended.
///
/// On a stop the process exits right after, so the runtime is left running
/// for it to end with. Shutting it down first broke every thread the mount
/// does not join, such as a size lookup or a profile backup a request started
/// during the stop: its next call to Drive panicked with "A Tokio 1.x context
/// was found, but it is being shutdown" (`docs/BUGS.md` B276). A mount that
/// is about to be tried again does shut it down, because a new runtime
/// replaces it. Dropping it would wait for every blocking task, however long;
/// a call still stuck on Drive would hold the retry (B165).
fn release_runtime(rt: tokio::runtime::Runtime, exiting: bool) {
if exiting {
std::mem::forget(rt);
} else {
rt.shutdown_timeout(std::time::Duration::from_secs(2));
}
}
fn cmd_pin(path: PathBuf) -> Result<()> { fn cmd_pin(path: PathBuf) -> Result<()> {
match control_request(CtlRequest::Pin { match control_request(CtlRequest::Pin {
path: path_arg(&path)?, path: path_arg(&path)?,
@@ -3446,6 +3466,79 @@ fn cmd_sync_limit(up: Option<u64>, down: Option<u64>) -> Result<()> {
})?) })?)
} }
/// How long the oldest write may wait to upload before `diagnose` calls it
/// out. Under a day it is a slow link or a big file; past it, something is
/// holding the queue.
const STAGED_STALE_SECS: u64 = 86_400;
/// `diagnose` findings for the queue and the pause, from the `Status` reply.
/// `paused_for` is how long a timed pause still runs, in seconds.
fn sync_health(
failing_ops: u64,
failing_error: Option<&str>,
staged_bytes: u64,
staged_oldest_secs: u64,
paused: bool,
paused_for: Option<i64>,
) -> Vec<(DiagnoseLevel, &'static str, String)> {
let mut findings = Vec::new();
if failing_ops > 0 {
findings.push((
DiagnoseLevel::Warn,
" failing writes",
match failing_error {
Some(error) => format!("{failing_ops} keep failing; the last error: {error}"),
None => format!("{failing_ops} keep failing"),
},
));
}
if staged_bytes > 0 && staged_oldest_secs >= STAGED_STALE_SECS {
findings.push((
DiagnoseLevel::Warn,
" oldest unsent write",
format!(
"{} old, {} only on this computer",
human_secs(staged_oldest_secs),
human_bytes(staged_bytes)
),
));
}
if paused {
findings.push((
DiagnoseLevel::Warn,
" sync",
match paused_for {
Some(left) if left > 0 => {
format!("paused for another {}", human_secs(left as u64))
}
_ => "paused until resumed (`pdfs sync resume`)".to_string(),
},
));
}
findings
}
/// `diagnose` findings for the synced folders that need someone to look.
fn folder_health(
folders: &[pdfs_core::control::SyncFolderInfo],
) -> Vec<(DiagnoseLevel, String, String)> {
folders
.iter()
.filter_map(|folder| {
let what = match folder.state.as_str() {
"error" => "sync error; `pdfs sync list` and the activity feed say more",
"conflict" => "conflict copies to look at",
_ => return None,
};
Some((
DiagnoseLevel::Warn,
format!(" synced folder [{}]", folder.id),
format!("{}: {what}", folder.local_path),
))
})
.collect()
}
fn unix_now() -> i64 { fn unix_now() -> i64 {
std::time::SystemTime::now() std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH) .duration_since(std::time::UNIX_EPOCH)
@@ -3782,21 +3875,26 @@ fn cmd_diagnose() -> Result<()> {
let socket_exists = socket.exists(); let socket_exists = socket.exists();
let socket_kind_ok = let socket_kind_ok =
std::fs::symlink_metadata(&socket).is_ok_and(|m| m.file_type().is_socket()); std::fs::symlink_metadata(&socket).is_ok_and(|m| m.file_type().is_socket());
// A stale socket file with no listener is the common failure, so the reply
// is what decides this rather than the file's existence.
let status = control_request(CtlRequest::Status);
report.finding( report.finding(
if socket_exists && !socket_kind_ok { if socket_exists && !socket_kind_ok {
DiagnoseLevel::Fail DiagnoseLevel::Fail
} else if socket_exists { } else if socket_exists && status.is_ok() {
DiagnoseLevel::Ok DiagnoseLevel::Ok
} else { } else {
DiagnoseLevel::Warn DiagnoseLevel::Warn
}, },
" control socket", " control socket",
socket.display().to_string(), if socket_exists && socket_kind_ok && status.is_err() {
format!("{} (left over; nothing listens on it)", socket.display())
} else {
socket.display().to_string()
},
); );
// A stale socket file with no listener is the common failure, so the reply
// is what decides this rather than the file's existence.
let mut daemon_responding = false; let mut daemon_responding = false;
match control_request(CtlRequest::Status) { match status {
Ok(CtlResponse::Status { Ok(CtlResponse::Status {
mountpoint, mountpoint,
online, online,
@@ -3804,6 +3902,12 @@ fn cmd_diagnose() -> Result<()> {
tokens_unsaved, tokens_unsaved,
pending_uploads, pending_uploads,
pending_changes, pending_changes,
failing_ops,
failing_error,
staged_bytes,
staged_oldest_secs,
paused,
paused_until,
version, version,
.. ..
}) => { }) => {
@@ -3878,6 +3982,27 @@ fn cmd_diagnose() -> Result<()> {
format!("{pending} waiting to upload") format!("{pending} waiting to upload")
}, },
); );
// The rest of the daemon's state that the report used to drop:
// writes that keep failing, a pause, a folder in error. Each of
// them stops changes from reaching Drive while everything above
// reads as fine (`docs/BUGS.md` B268).
for (level, check, detail) in sync_health(
failing_ops,
failing_error.as_deref(),
staged_bytes,
staged_oldest_secs,
paused,
paused_until.map(|until| until - unix_now()),
) {
report.finding(level, check, detail);
}
if let Ok(CtlResponse::SyncFolders { items }) =
control_request(CtlRequest::ListSyncFolders)
{
for (level, check, detail) in folder_health(&items) {
report.finding(level, &check, detail);
}
}
match await_integrity_check() { match await_integrity_check() {
Ok(CtlResponse::CacheReport { Ok(CtlResponse::CacheReport {
integrity_problems, integrity_problems,
@@ -4037,6 +4162,47 @@ fn control_request(req: CtlRequest) -> Result<CtlResponse> {
.with_context(|| format!("talk to mount daemon at {}", socket.display())) .with_context(|| format!("talk to mount daemon at {}", socket.display()))
} }
#[cfg(test)]
mod release_runtime_tests {
use super::*;
use std::time::Duration;
/// A thread the mount never joined still reaches Drive after a stop. It
/// used to panic on its first timer once the runtime was shut down.
#[test]
fn a_thread_left_running_at_a_stop_can_still_use_the_runtime() {
let rt = tokio::runtime::Runtime::new().unwrap();
let handle = rt.handle().clone();
let (released_tx, released_rx) = std::sync::mpsc::channel();
let straggler = std::thread::spawn(move || {
released_rx.recv().unwrap();
handle.block_on(async {
tokio::time::sleep(Duration::from_millis(1)).await;
7
})
});
release_runtime(rt, true);
released_tx.send(()).unwrap();
assert_eq!(straggler.join().expect("the thread must not panic"), 7);
}
#[test]
fn a_mount_tried_again_shuts_its_runtime_down() {
let rt = tokio::runtime::Runtime::new().unwrap();
let (done_tx, done_rx) = std::sync::mpsc::channel::<()>();
rt.spawn(async move {
let _done = done_tx;
std::future::pending::<()>().await;
});
release_runtime(rt, false);
// The task is dropped with the runtime, and its sender with it.
assert_eq!(
done_rx.recv_timeout(Duration::from_secs(5)),
Err(std::sync::mpsc::RecvTimeoutError::Disconnected)
);
}
}
#[cfg(test)] #[cfg(test)]
mod pause_tests { mod pause_tests {
use super::*; use super::*;
@@ -4075,6 +4241,50 @@ mod pause_tests {
mod diagnose_tests { mod diagnose_tests {
use super::*; use super::*;
/// A healthy queue says nothing; writes that keep failing, an old unsent
/// write and a pause each say so (`docs/BUGS.md` B268).
#[test]
fn a_stuck_queue_or_a_pause_is_reported() {
assert!(sync_health(0, None, 0, 0, false, None).is_empty());
assert!(sync_health(0, None, 4096, STAGED_STALE_SECS - 1, false, None).is_empty());
let checks: Vec<&str> = sync_health(
2,
Some("upload a.txt: quota"),
4096,
STAGED_STALE_SECS,
true,
Some(600),
)
.into_iter()
.map(|(level, check, _)| {
assert!(matches!(level, DiagnoseLevel::Warn));
check.trim()
})
.collect();
assert_eq!(checks, ["failing writes", "oldest unsent write", "sync"]);
}
/// Issue #53: a folder that would not sync, and `diagnose` found nothing.
#[test]
fn a_synced_folder_in_error_is_reported() {
let folder = |id: i64, state: &str| pdfs_core::control::SyncFolderInfo {
id,
local_path: format!("/home/u/f{id}"),
remote_uid: "vol~link".into(),
mode: "mirror".into(),
state: state.into(),
last_sync: 0,
pending_mode: None,
progress: None,
paused: false,
};
let findings =
folder_health(&[folder(1, "idle"), folder(2, "error"), folder(3, "syncing")]);
assert_eq!(findings.len(), 1);
assert_eq!(findings[0].1.trim(), "synced folder [2]");
assert!(findings[0].2.starts_with("/home/u/f2: sync error"));
}
#[test] #[test]
fn finding_severity_is_counted_centrally() { fn finding_severity_is_counted_centrally() {
let mut report = DiagnoseReport::default(); let mut report = DiagnoseReport::default();
+33 -1
View File
@@ -621,10 +621,20 @@ impl ContentCache {
for entry in rd.flatten() { for entry in rd.flatten() {
let name = entry.file_name(); let name = entry.file_name();
let Some(name) = name.to_str() else { continue }; let Some(name) = name.to_str() else { continue };
// A `.meta` tags the blob named like it without the suffix. One
// whose blob is gone is what an eviction leaves between its two
// removals, by a crash or by a store of the same uid landing in
// between, and nothing else would ever remove it.
if let Some(blob) = name.strip_suffix(".meta") {
if !dir.join(blob).exists() {
let _ = std::fs::remove_file(entry.path());
}
continue;
}
// `.geom` is a block-geometry sidecar (see `GeometryMeta`), tens // `.geom` is a block-geometry sidecar (see `GeometryMeta`), tens
// of bytes and not a cached block; like `.meta` it is neither // of bytes and not a cached block; like `.meta` it is neither
// budgeted nor evictable on its own. // budgeted nor evictable on its own.
if name.ends_with(".meta") || name.ends_with(".geom") { if name.ends_with(".geom") {
continue; continue;
} }
// A `.tmp` is a staging file from a store that never completed // A `.tmp` is a staging file from a store that never completed
@@ -2250,6 +2260,28 @@ mod tests {
assert!(total <= 8, "budget enforced after reconcile, total={total}"); assert!(total <= 8, "budget enforced after reconcile, total={total}");
} }
/// An eviction removes the blob, then its tag. A crash in between, or a
/// store of the same uid landing in between, left the tag on its own, and
/// nothing removed it.
#[test]
fn a_tag_without_its_blob_is_swept_at_open() {
let dir = TempDir::new();
let content = dir.path().join("content");
let pins = dir.path().join("pins.json");
let db = Arc::new(Db::open_in_memory().unwrap());
{
let c = ContentCache::open(content.clone(), pins.clone(), 0, db.clone()).unwrap();
c.store(&uid("kept"), 1, 4, b"kept").unwrap();
c.store(&uid("gone"), 1, 4, b"gone").unwrap();
std::fs::remove_file(c.blob_path(&uid("gone"))).unwrap();
}
let c = ContentCache::open(content, pins, 0, db).unwrap();
assert!(!c.meta_path(&uid("gone")).exists());
assert!(c.meta_path(&uid("kept")).exists());
assert_eq!(c.read_range(&uid("kept"), 1, 4, 0, 4).unwrap(), b"kept");
}
#[test] #[test]
fn stale_metadata_is_a_miss() { fn stale_metadata_is_a_miss() {
let (c, _d) = cache(); let (c, _d) = cache();
+2 -2
View File
@@ -1332,8 +1332,8 @@ mod tests {
)); ));
let file = std::fs::File::create(&path).unwrap(); let file = std::fs::File::create(&path).unwrap();
let mut zip = zip::ZipWriter::new(file); let mut zip = zip::ZipWriter::new(file);
let options: zip::write::FileOptions<'_, ()> = let options = zip::write::SimpleFileOptions::default()
zip::write::FileOptions::default().compression_method(zip::CompressionMethod::Stored); .compression_method(zip::CompressionMethod::Stored);
for (name, body) in files { for (name, body) in files {
zip.start_file(*name, options).unwrap(); zip.start_file(*name, options).unwrap();
zip.write_all(body).unwrap(); zip.write_all(body).unwrap();
+10 -2
View File
@@ -1526,9 +1526,17 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
let core = core.clone(); let core = core.clone();
std::thread::spawn(move || { std::thread::spawn(move || {
match core.restore_sync_folders(&items, device.as_deref()) { match core.restore_sync_folders(&items, device.as_deref()) {
Ok(message) => { Ok(outcome) => {
core.touch_profile(); core.touch_profile();
core.log_activity(ActivityKind::Download, &message, "", true); if !outcome.complete() {
warn!(skipped = ?outcome.skipped, "restore skipped folders");
}
core.log_activity(
ActivityKind::Download,
outcome.summary(),
outcome.skipped.join("; "),
outcome.complete(),
);
} }
Err(e) => { Err(e) => {
warn!(error = %e, "restore failed"); warn!(error = %e, "restore failed");
+7
View File
@@ -1281,6 +1281,13 @@ impl Core {
sent: (&sent.0, &sent.1), sent: (&sent.0, &sent.1),
landed: (&home.to_string(), &name), landed: (&home.to_string(), &name),
}; };
// Counted from the op's retirement until the bytes are cached and the
// tree has the real uid, as an upload is. Uncounted, the queue read
// empty while that still ran: a delete in between took an open file
// for closed and dropped what its handle reads, and a script that
// waited for an empty queue read the file before it was cached
// (`docs/BUGS.md` B257).
let _landing = LandingUpload::begin(&self.landing_uploads);
let newer = match self.retire_create(op, uploaded, &landing, &local, &real)? { let newer = match self.retire_create(op, uploaded, &landing, &local, &real)? {
CreateRetired::Withdrawn { trash } => { CreateRetired::Withdrawn { trash } => {
return self.trash_withdrawn_create(op, trash, &local, &real, &name); return self.trash_withdrawn_create(op, trash, &local, &real, &name);
+9 -1
View File
@@ -1434,6 +1434,14 @@ impl ProtonFs {
// (`ContentCache::clone_or_copy`). // (`ContentCache::clone_or_copy`).
if let Err(e) = ContentCache::clone_or_copy(&pending.path, &path) { if let Err(e) = ContentCache::clone_or_copy(&pending.path, &path) {
let _ = std::fs::remove_file(&path); let _ = std::fs::remove_file(&path);
// The drain lets go of the pending entry before it
// discards the blob, so a blob that went away under the
// copy is a revision that landed or was replaced: the
// same race the re-check below catches, only earlier.
if self.core.pending_blob(&uid).as_ref() != Some(&pending.path) {
warn!(%uid, attempt, "queued revision went away while opening for write; retrying");
continue;
}
error!(%uid, source = %pending.path.display(), error = %e, error!(%uid, source = %pending.path.display(), error = %e,
"copy queued revision into write scratch failed"); "copy queued revision into write scratch failed");
break 'install Err(Errno::EIO); break 'install Err(Errno::EIO);
@@ -2368,7 +2376,7 @@ impl ProtonFs {
// regardless — the file is gone from their point of view the moment the // regardless — the file is gone from their point of view the moment the
// command returns. The same when the network goes // command returns. The same when the network goes
// away under the remote call below. // away under the remote call below.
let queue_it = |reply: ReplyEmpty| match self.core.queue_trash(&uid, name) { let queue_it = |reply: ReplyEmpty| match self.core.queue_trash(&uid, name, open_now) {
Ok(()) => reply.ok(), Ok(()) => reply.ok(),
Err(e) => reply.error(e), Err(e) => reply.error(e),
}; };
+14 -6
View File
@@ -3999,7 +3999,12 @@ impl Core {
/// the cached ones, as when the trash is sent at once: a partial write /// the cached ones, as when the trash is sent at once: a partial write
/// fills its gaps from them, offline too (`docs/BUGS.md` B186). The final /// fills its gaps from them, offline too (`docs/BUGS.md` B186). The final
/// close or the trash landing, whichever comes last, lets go of them. /// close or the trash landing, whichever comes last, lets go of them.
fn queue_trash(&self, uid: &NodeUid, name: &str) -> Result<(), Errno> { ///
/// `open` is whether the caller found the file open. A create that landed
/// under a delete is trashed by its real uid before the tree has it, so
/// asking the tree alone took an open file for closed (`docs/BUGS.md`
/// B257).
fn queue_trash(&self, uid: &NodeUid, name: &str, open: bool) -> Result<(), Errno> {
self.require_uid_writable(uid)?; self.require_uid_writable(uid)?;
// The bytes below are about to be discarded, and a drain worker may be // The bytes below are about to be discarded, and a drain worker may be
// reading them onto the wire right now. Stop it first: with several // reading them onto the wire right now. Stop it first: with several
@@ -4014,7 +4019,7 @@ impl Core {
error!(%uid, error = %e, "queueing trash failed"); error!(%uid, error = %e, "queueing trash failed");
Errno::EIO Errno::EIO
})?; })?;
let open_now = self.is_open_anywhere(uid); let open_now = open || self.is_open_anywhere(uid);
if open_now { if open_now {
let read_from = self.pending_blob(uid); let read_from = self.pending_blob(uid);
self.release_dropped_below( self.release_dropped_below(
@@ -5383,9 +5388,12 @@ impl Core {
/// `unlink` under an open handle. Its queued ops go now, or the drain would /// `unlink` under an open handle. Its queued ops go now, or the drain would
/// upload to the node after it was trashed (`docs/BUGS.md` B106). /// upload to the node after it was trashed (`docs/BUGS.md` B106).
fn remove_replaced(&self, ino: u64, uid: &NodeUid, name: &str) -> Result<(), Errno> { fn remove_replaced(&self, ino: u64, uid: &NodeUid, name: &str) -> Result<(), Errno> {
// Asked under the uid the tree has now: a landed create's real uid is
// not in the tree yet (`docs/BUGS.md` B257).
let open = self.is_open_anywhere(uid);
let landed; let landed;
let uid = if is_local_uid(uid) { let uid = if is_local_uid(uid) {
if self.is_open_anywhere(uid) { if open {
self.withdraw_queued_ops(uid)?; self.withdraw_queued_ops(uid)?;
} else { } else {
self.discard_queued_ops(uid)?; self.discard_queued_ops(uid)?;
@@ -5410,7 +5418,7 @@ impl Core {
// Local-first or offline, or the network goes away under the call: // Local-first or offline, or the network goes away under the call:
// queue the trash. The rename that follows queues as well, behind it. // queue the trash. The rename that follows queues as well, behind it.
if !self.sends_inline() { if !self.sends_inline() {
return self.queue_trash(uid, name); return self.queue_trash(uid, name, open);
} }
self.require_uid_writable(uid)?; self.require_uid_writable(uid)?;
if let Err(e) = self if let Err(e) = self
@@ -5418,7 +5426,7 @@ impl Core {
.and_then(batch::into_unit) .and_then(batch::into_unit)
{ {
if self.lost_link(&e, "trash a replaced node") { if self.lost_link(&e, "trash a replaced node") {
return self.queue_trash(uid, name); return self.queue_trash(uid, name, open);
} }
error!(%uid, name, error = %e, "trashing the node a rename replaces failed"); error!(%uid, name, error = %e, "trashing the node a rename replaces failed");
self.log_activity(ActivityKind::Trash, name, e.to_string(), false); self.log_activity(ActivityKind::Trash, name, e.to_string(), false);
@@ -9258,7 +9266,7 @@ mod tests {
assert!(!replaced.contains("st.forget(uid)")); assert!(!replaced.contains("st.forget(uid)"));
assert_before( assert_before(
replaced, replaced,
"if self.is_open_anywhere(uid)", "let open = self.is_open_anywhere(uid)",
"self.discard_queued_ops(uid)", "self.discard_queued_ops(uid)",
); );
assert_before(replaced, "if open_now", "self.cache.evict(uid)"); assert_before(replaced, "if open_now", "self.cache.evict(uid)");
+42 -10
View File
@@ -388,7 +388,7 @@ impl Core {
&self, &self,
items: &[pdfs_core::control::RestoreItem], items: &[pdfs_core::control::RestoreItem],
device: Option<&str>, device: Option<&str>,
) -> CoreResult<String> { ) -> CoreResult<RestoreOutcome> {
if items.is_empty() { if items.is_empty() {
return Err(CoreError::invalid("nothing to restore")); return Err(CoreError::invalid("nothing to restore"));
} }
@@ -434,10 +434,19 @@ impl Core {
continue; continue;
} }
}; };
let id = self let id = match self.db.sync_folder_add(
.db &local.to_string_lossy(),
.sync_folder_add(&local.to_string_lossy(), &item.remote_uid, &device.share_id) &item.remote_uid,
.map_err(|e| CoreError::internal(format!("db: {e:?}")))?; &device.share_id,
) {
Ok(id) => id,
Err(e) => {
// The folders restored before this one still need their
// watches.
let _ = self.sync_tx.send(sync::SyncMsg::Rewatch);
return Err(CoreError::internal(format!("db: {e:?}")));
}
};
// `ondemand` is applied through the normal request path, which // `ondemand` is applied through the normal request path, which
// mounts the FUSE session and handles the folder being busy — the // mounts the FUSE session and handles the folder being busy — the
// restore does not get its own copy of that logic. // restore does not get its own copy of that logic.
@@ -454,12 +463,35 @@ impl Core {
for reason in &skipped { for reason in &skipped {
info!(reason, "restore skipped a folder"); info!(reason, "restore skipped a folder");
} }
let mut message = format!("restored {restored} folder(s)");
if !skipped.is_empty() {
message.push_str(&format!("; skipped: {}", skipped.join(", ")));
}
info!(restored, skipped = skipped.len(), "restore complete"); info!(restored, skipped = skipped.len(), "restore complete");
Ok(message) Ok(RestoreOutcome { restored, skipped })
}
}
/// What a restore did: how many folders it attached, and why it left out the
/// others.
pub(crate) struct RestoreOutcome {
pub(crate) restored: usize,
pub(crate) skipped: Vec<String>,
}
impl RestoreOutcome {
/// The activity entry's headline.
pub(crate) fn summary(&self) -> String {
match (self.restored, self.skipped.len()) {
(n, 0) => format!("restored {n} folder(s)"),
(0, k) => format!("restored no folders; skipped {k}"),
(n, k) => format!("restored {n} folder(s); skipped {k}"),
}
}
/// Whether the restore did all it was asked. A skipped folder is a
/// failure the user has to see: the reply went out as "restoring folders"
/// before any work, and a restore that skipped every folder used to be
/// logged as a success, so nothing anywhere said that nothing would
/// download (`docs/BUGS.md` B269).
pub(crate) fn complete(&self) -> bool {
self.skipped.is_empty()
} }
} }
+94 -6
View File
@@ -130,6 +130,22 @@ impl Daemon {
} }
} }
/// Whether nothing is queued and no upload is still landing, as `pdfs
/// status` counts it. An empty [`Self::pending`] is not enough before a
/// read that needs the cache: the op goes before the uploaded bytes are
/// cached, so a test that went offline at once raced the drain.
pub(crate) fn drained(&self) -> bool {
self.pending().is_ok_and(|items| items.is_empty())
&& matches!(
self.request(&Request::Status),
Ok(Response::Status {
pending_uploads: 0,
pending_changes: 0,
..
})
)
}
/// Whether the mount believes Proton is reachable, as `pdfs status` shows /// Whether the mount believes Proton is reachable, as `pdfs status` shows
/// it. /// it.
pub(crate) fn online(&self) -> io::Result<bool> { pub(crate) fn online(&self) -> io::Result<bool> {
@@ -565,7 +581,7 @@ mod tests {
std::fs::write(&path, b"0123456789").unwrap(); std::fs::write(&path, b"0123456789").unwrap();
assert!(wait_until(Duration::from_secs(30), || { assert!(wait_until(Duration::from_secs(30), || {
daemon.pending().is_ok_and(|items| items.is_empty()) daemon.drained()
&& drive.tree().get("f.txt") == Some(&Entry::File(Arc::new(b"0123456789".to_vec()))) && drive.tree().get("f.txt") == Some(&Entry::File(Arc::new(b"0123456789".to_vec())))
})); }));
daemon.client.set_online(false); daemon.client.set_online(false);
@@ -592,14 +608,22 @@ mod tests {
daemon.pending(), daemon.pending(),
drive.tree() drive.tree()
); );
// The bytes kept for the handle went once the trash landed with it closed. // The bytes kept for the handle went once the trash landed with it
let cached: Vec<_> = std::fs::read_dir(dir.join("cache")) // closed. The queue empties as the trash's row goes, a moment before
// the eviction, so this waits too.
let cached = || -> Vec<_> {
std::fs::read_dir(dir.join("cache"))
.unwrap() .unwrap()
.filter_map(|e| e.ok()) .filter_map(|e| e.ok())
.filter(|e| e.file_type().is_ok_and(|t| t.is_file())) .filter(|e| e.file_type().is_ok_and(|t| t.is_file()))
.map(|e| e.file_name()) .map(|e| e.file_name())
.collect(); .collect()
assert!(cached.is_empty(), "{cached:?}"); };
assert!(
wait_until(Duration::from_secs(30), || cached().is_empty()),
"{:?}",
cached()
);
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown))); assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir); let _ = std::fs::remove_dir_all(&dir);
@@ -684,6 +708,55 @@ mod tests {
let _ = std::fs::remove_dir_all(&dir); let _ = std::fs::remove_dir_all(&dir);
} }
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_new_file_deleted_while_open_as_its_create_lands_still_reads() {
// The create's op was retired and the tree still had the placeholder.
// The queue read empty, and the delete trashed the node by its real
// uid, which the tree did not have yet: it took the open file for
// closed and dropped the bytes its handle reads (B257).
use std::io::{Read, Seek, SeekFrom, Write};
let drive = FakeDrive::new();
let dir = scratch("delete-open-as-create-lands");
let daemon = Daemon::start(&dir, drive.client(1, Faults::lan())).unwrap();
let path = daemon.mountpoint.join("f.txt");
let landing = daemon.client.hold_next_landing_of_any();
std::fs::write(&path, b"0123456789").unwrap();
assert!(wait_until(Duration::from_secs(30), || landing.reached()));
assert!(!daemon.drained(), "the landing create is counted");
let mut file = std::fs::OpenOptions::new()
.read(true)
.write(true)
.open(&path)
.unwrap();
drop(landing);
assert!(wait_until(Duration::from_secs(30), || daemon.drained()));
daemon.client.set_online(false);
std::fs::remove_file(&path).unwrap();
file.seek(SeekFrom::End(0)).unwrap();
file.write_all(b"ab").unwrap();
let mut read = Vec::new();
file.seek(SeekFrom::Start(0)).unwrap();
file.read_to_end(&mut read).unwrap();
assert_eq!(read, b"0123456789ab");
drop(file);
daemon.client.set_online(true);
assert!(
wait_until(Duration::from_secs(30), || {
daemon.drained() && drive.tree().is_empty()
}),
"{:?} {:?}",
daemon.pending(),
drive.tree()
);
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
#[test] #[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"] #[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_restart_does_not_bring_back_the_write_of_a_file_deleted_while_open() { fn a_restart_does_not_bring_back_the_write_of_a_file_deleted_while_open() {
@@ -698,7 +771,7 @@ mod tests {
std::fs::write(&path, b"0123456789").unwrap(); std::fs::write(&path, b"0123456789").unwrap();
assert!(wait_until(Duration::from_secs(30), || { assert!(wait_until(Duration::from_secs(30), || {
daemon.pending().is_ok_and(|items| items.is_empty()) daemon.drained()
&& drive.tree().get("f.txt") == Some(&Entry::File(Arc::new(b"0123456789".to_vec()))) && drive.tree().get("f.txt") == Some(&Entry::File(Arc::new(b"0123456789".to_vec())))
})); }));
daemon.client.set_online(false); daemon.client.set_online(false);
@@ -1998,6 +2071,21 @@ mod tests {
daemon.write_new(&path, b"first").unwrap(); daemon.write_new(&path, b"first").unwrap();
assert!(wait_until(Duration::from_secs(30), || held.reached())); assert!(wait_until(Duration::from_secs(30), || held.reached()));
std::fs::write(&path, b"second").unwrap(); std::fs::write(&path, b"second").unwrap();
// The kernel sends the release after `close` returns. Attaching the
// bytes resets the create's attempts, so a release that came after the
// lost reply had it retried at once, and the retry landed before
// anything here saw it fail. The bytes it replaced go only once they
// are attached.
let staged = dir.join("cache").join("staging");
assert!(wait_until(Duration::from_secs(30), || {
let blobs: Vec<Vec<u8>> = std::fs::read_dir(&staged)
.into_iter()
.flatten()
.filter_map(|e| e.ok())
.filter_map(|e| std::fs::read(e.path()).ok())
.collect();
blobs.iter().any(|b| b == b"second") && !blobs.iter().any(|b| b == b"first")
}));
drop(held); drop(held);
assert!(wait_until(Duration::from_secs(30), || { assert!(wait_until(Duration::from_secs(30), || {
daemon daemon
+45 -5
View File
@@ -1162,14 +1162,22 @@ impl Core {
let uids = self let uids = self
.rt .rt
.block_on(self.drive.enumerate_folder_children_node_uids(folder)) .block_on(self.drive.enumerate_folder_children_node_uids(folder))
.map_err(|e| format!("enumerate {folder}: {e}"))?; .map_err(|e| format!("enumerate {folder}: {e}"));
let uids = match unlisted_gap(walk, prefix, uids)? {
Some(uids) => uids,
None => return Ok(()),
};
if uids.is_empty() { if uids.is_empty() {
return Ok(()); return Ok(());
} }
let nodes = self let nodes = self
.rt .rt
.block_on(self.drive.enumerate_nodes_light(&uids)) .block_on(self.drive.enumerate_nodes_light(&uids))
.map_err(|e| format!("resolve nodes: {e}"))?; .map_err(|e| format!("resolve nodes: {e}"));
let nodes = match unlisted_gap(walk, prefix, nodes)? {
Some(nodes) => nodes,
None => return Ok(()),
};
// The SDK leaves out a node it cannot decrypt, and says so only in its // The SDK leaves out a node it cannot decrypt, and says so only in its
// log. Its name went with it, so all this can record is the folder. // log. Its name went with it, so all this can record is the folder.
if nodes.len() < uids.len() { if nodes.len() < uids.len() {
@@ -1213,10 +1221,14 @@ impl Core {
let sized: HashMap<String, (i64, Option<String>)> = if stale.is_empty() { let sized: HashMap<String, (i64, Option<String>)> = if stale.is_empty() {
HashMap::new() HashMap::new()
} else { } else {
self.rt let full = self
.rt
.block_on(self.drive.enumerate_nodes(&stale)) .block_on(self.drive.enumerate_nodes(&stale))
.map_err(|e| format!("resolve nodes: {e}"))? .map_err(|e| format!("resolve nodes: {e}"));
.iter() let Some(full) = unlisted_gap(walk, prefix, full)? else {
return Ok(());
};
full.iter()
.map(|n| { .map(|n| {
( (
n.uid.to_string(), n.uid.to_string(),
@@ -2034,6 +2046,34 @@ struct RemoteWalk<'a> {
gaps: &'a mut RemoteGaps, gaps: &'a mut RemoteGaps,
} }
/// A listing call's result in [`Core::walk_remote`]. A failure below the root
/// records `prefix` as unlisted and returns `None`, so the walk goes on
/// without that folder. A failure at the root still fails the pass, since
/// nothing would be left to compare.
///
/// One listing error, after the SDK's own retries, used to fail the whole
/// walk. Nothing is applied before the walk ends, so on a link that drops a
/// request now and then a large first sync started over from nothing on
/// every pass and never got past it (`docs/BUGS.md` B271).
fn unlisted_gap<T>(
walk: &mut RemoteWalk<'_>,
prefix: &str,
result: Result<T, String>,
) -> Result<Option<T>, PassAbort> {
match result {
Ok(value) => Ok(Some(value)),
Err(error) if prefix.is_empty() => Err(error.into()),
Err(error) => {
warn!(
folder = prefix,
error, "sync: a folder on Drive could not be listed; leaving it out of this pass"
);
walk.gaps.unlisted(prefix);
Ok(None)
}
}
}
/// Whether the file at `path` is no longer what the scan saw as `planned`. /// Whether the file at `path` is no longer what the scan saw as `planned`.
/// A file that is gone has nothing left to lose, so it has not changed. /// A file that is gone has nothing left to lose, so it has not changed.
fn changed_since_scan(path: &Path, planned: &LocalItem) -> bool { fn changed_since_scan(path: &Path, planned: &LocalItem) -> bool {
+39 -2
View File
@@ -236,6 +236,9 @@ pub(super) struct RemoteGaps {
short: HashSet<String>, short: HashSet<String>,
/// Paths seen, but whose entry could not be read. /// Paths seen, but whose entry could not be read.
unread: HashSet<String>, unread: HashSet<String>,
/// Folders whose listing failed outright, after the SDK's retries. Nothing
/// under one is known, so all of it is left out of the pass (B271).
unlisted: HashSet<String>,
} }
impl RemoteGaps { impl RemoteGaps {
@@ -249,8 +252,13 @@ impl RemoteGaps {
self.unread.insert(rel); self.unread.insert(rel);
} }
/// The listing of `folder` failed, so nothing under it was walked.
pub(super) fn unlisted(&mut self, folder: &str) {
self.unlisted.insert(folder.to_string());
}
pub(super) fn is_empty(&self) -> bool { pub(super) fn is_empty(&self) -> bool {
self.short.is_empty() && self.unread.is_empty() self.short.is_empty() && self.unread.is_empty() && self.unlisted.is_empty()
} }
/// Whether `rel` may be missing from `remote` only because the walk could /// Whether `rel` may be missing from `remote` only because the walk could
@@ -272,7 +280,7 @@ impl RemoteGaps {
let mut at = parent_rel(rel); let mut at = parent_rel(rel);
loop { loop {
if remote_dirs.contains_key(at) { if remote_dirs.contains_key(at) {
return self.short.contains(at); return self.short.contains(at) || self.unlisted.contains(at);
} }
if at.is_empty() { if at.is_empty() {
return false; return false;
@@ -295,6 +303,9 @@ impl RemoteGaps {
.iter() .iter()
.map(|rel| format!("{rel} (cannot be read on Proton Drive)")), .map(|rel| format!("{rel} (cannot be read on Proton Drive)")),
) )
.chain(self.unlisted.iter().map(|rel| {
format!("{rel} (could not be listed on Proton Drive; tried again next pass)")
}))
.collect(); .collect();
lines.sort_unstable(); lines.sort_unstable();
lines lines
@@ -844,6 +855,32 @@ mod file_plan_tests {
assert!(gaps.hides("top.txt", &remote, &dirs)); assert!(gaps.hides("top.txt", &remote, &dirs));
} }
/// A folder whose listing failed was walked no further: nothing under it
/// reads as deleted on Drive, and a sibling that listed still does
/// (`docs/BUGS.md` B271).
#[test]
fn a_folder_that_failed_to_list_hides_everything_under_it() {
let remote = HashMap::from([
("docs".to_string(), folder()),
("pics".to_string(), folder()),
]);
let dirs = HashMap::from([
(String::new(), uid("root")),
("docs".to_string(), uid("docs")),
("pics".to_string(), uid("pics")),
]);
let mut gaps = RemoteGaps::default();
gaps.unlisted("docs");
assert!(gaps.hides("docs/a.txt", &remote, &dirs));
assert!(gaps.hides("docs/sub/deep.txt", &remote, &dirs));
assert!(!gaps.hides("docs", &remote, &dirs));
assert!(!gaps.hides("pics/gone.jpg", &remote, &dirs));
assert_eq!(
gaps.report(),
["docs (could not be listed on Proton Drive; tried again next pass)"]
);
}
#[test] #[test]
fn a_file_whose_size_could_not_be_read_is_hidden() { fn a_file_whose_size_could_not_be_read_is_hidden() {
let remote: HashMap<String, RemoteItem> = HashMap::new(); let remote: HashMap<String, RemoteItem> = HashMap::new();
+200 -22
View File
@@ -68,13 +68,10 @@ original wording.
| Entry | Title | What is left | | Entry | Title | What is left |
|---|---|---| |---|---|---|
| [B275](#b275--on-a-slow-link-a-files-last-write-can-land-only-in-a-conflict-copy) | On a slow link, a file's last write can land only in a conflict copy | Cause not found; seen once in CI |
| [B273](#b273--a-restore-proposes-another-computers-paths-when-the-two-share-a-hostname) | A restore proposes another computer's paths when the two share a hostname | Not fixed | | [B273](#b273--a-restore-proposes-another-computers-paths-when-the-two-share-a-hostname) | A restore proposes another computer's paths when the two share a hostname | Not fixed |
| [B271](#b271--one-failed-listing-makes-a-large-first-sync-start-over-from-nothing) | One failed listing makes a large first sync start over from nothing | Not fixed |
| [B270](#b270--one-file-left-in-a-mirror-folder-clears-the-wipe-guard-for-all-the-others) | One file left in a mirror folder clears the wipe guard for all the others | Not fixed | | [B270](#b270--one-file-left-in-a-mirror-folder-clears-the-wipe-guard-for-all-the-others) | One file left in a mirror folder clears the wipe guard for all the others | Not fixed |
| [B269](#b269--a-restore-that-skipped-every-folder-reports-success) | A restore that skipped every folder reports success | Not fixed |
| [B268](#b268--pdfs-diagnose-says-no-problems-found-while-sync-is-stuck) | `pdfs diagnose` says "No problems found" while sync is stuck | Not fixed |
| [B267](#b267--a-key-failure-at-a-start-from-a-warm-cache-shows-as-offline-for-good) | A key failure at a start from a warm cache shows as offline for good | Not fixed | | [B267](#b267--a-key-failure-at-a-start-from-a-warm-cache-shows-as-offline-for-good) | A key failure at a start from a warm cache shows as offline for good | Not fixed |
| [B265](#b265--address-keys-whose-token-is-for-an-inactive-account-key-never-open) | Address keys whose token is for an inactive account key never open | Needs an SDK change; not reproduced |
| [B134](#b134--a-file-deleted-online-while-the-drain-checks-its-write-is-kept-as-a-conflict-copy) | A file deleted online while the drain checks its write is kept as a conflict copy | Only with `local_first` off | | [B134](#b134--a-file-deleted-online-while-the-drain-checks-its-write-is-kept-as-a-conflict-copy) | A file deleted online while the drain checks its write is kept as a conflict copy | Only with `local_first` off |
| [B91](#b91--the-daemon-hung-overnight-and-nothing-could-say-what-it-was-waiting-on) | The daemon hung overnight, and nothing could say what it was waiting on | The cause is not diagnosed; the daemon now reports where it is stuck and is restarted | | [B91](#b91--the-daemon-hung-overnight-and-nothing-could-say-what-it-was-waiting-on) | The daemon hung overnight, and nothing could say what it was waiting on | The cause is not diagnosed; the daemon now reports where it is stuck and is restarted |
| [B67](#b67--fresh-state-restore-can-silently-select-a-new-device-high-22) | Fresh-state restore can silently select a new device (HIGH-22) | An ambiguous device after a loss of local state is not asked about | | [B67](#b67--fresh-state-restore-can-silently-select-a-new-device-high-22) | Fresh-state restore can silently select a new device (HIGH-22) | An ambiguous device after a loss of local state is not asked about |
@@ -87,6 +84,8 @@ original wording.
| Entry | Title | What is left | | Entry | Title | What is left |
|---|---|---| |---|---|---|
| [B269](#b269--a-restore-that-skipped-every-folder-reports-success) | A restore that skipped every folder reports success | The app shows the result only in the activity feed; no offer to replace or reset the folder in the way |
| [B268](#b268--pdfs-diagnose-says-no-problems-found-while-sync-is-stuck) | `pdfs diagnose` says "No problems found" while sync is stuck | The sync engine's state is not checked |
| [B263](#b263--a-mount-that-can-never-succeed-retries-every-5-s-and-tells-only-the-journal) | A mount that can never succeed retries every 5 s and tells only the journal | Errors no retry fixes back off like network errors | | [B263](#b263--a-mount-that-can-never-succeed-retries-every-5-s-and-tells-only-the-journal) | A mount that can never succeed retries every 5 s and tells only the journal | Errors no retry fixes back off like network errors |
| [B262](#b262--watching-a-large-synced-folder-holds-up-the-sync-engine-and-a-failed-watch-is-never-retried) | Watching a large synced folder holds up the sync engine, and a failed watch is never retried | The first pass still waits for the watch; a failed watch is not shown in the app | | [B262](#b262--watching-a-large-synced-folder-holds-up-the-sync-engine-and-a-failed-watch-is-never-retried) | Watching a large synced folder holds up the sync engine, and a failed watch is never retried | The first pass still waits for the watch; a failed watch is not shown in the app |
| [B259](#b259--the-sync-engine-can-die-or-hang-and-nothing-shows-it-until-a-restart) | The sync engine can die or hang, and nothing shows it until a restart | No pass deadline; an ended engine is reported, not restarted; the trigger in #53 is not known | | [B259](#b259--the-sync-engine-can-die-or-hang-and-nothing-shows-it-until-a-restart) | The sync engine can die or hang, and nothing shows it until a restart | No pass deadline; an ended engine is reported, not restarted; the trigger in #53 is not known |
@@ -110,14 +109,17 @@ a live account since.
| Entry | Title | | Entry | Title |
|---|---| |---|---|
| [B276](#b276--a-stop-can-end-with-a-panic-in-a-thread-the-stop-never-waited-for) | A stop can end with a panic in a thread the stop never waited for |
| [B274](#b274--opening-a-file-for-writing-fails-with-eio-while-its-queued-revision-lands) | Opening a file for writing fails with EIO while its queued revision lands |
| [B272](#b272--after-a-folder-is-removed-and-added-again-its-file-events-go-to-the-old-id) | After a folder is removed and added again, its file events go to the old id |
| [B271](#b271--one-failed-listing-makes-a-large-first-sync-start-over-from-nothing) | One failed listing makes a large first sync start over from nothing |
| [B266](#b266--pin-unpin-search-and-node-loads-report-a-database-error-as-offline) | Pin, unpin, search and node loads report a database error as "offline" | | [B266](#b266--pin-unpin-search-and-node-loads-report-a-database-error-as-offline) | Pin, unpin, search and node loads report a database error as "offline" |
| [B265](#b265--address-keys-whose-token-is-for-an-inactive-account-key-never-open) | Address keys whose token is for an inactive account key never open |
| [B264](#b264--a-mount-that-fails-after-starting-its-workers-leaves-them-running) | A mount that fails after starting its workers leaves them running | | [B264](#b264--a-mount-that-fails-after-starting-its-workers-leaves-them-running) | A mount that fails after starting its workers leaves them running |
| [B261](#b261--a-folder-that-has-never-synced-reads-up-to-date) | A folder that has never synced reads "up to date" | | [B261](#b261--a-folder-that-has-never-synced-reads-up-to-date) | A folder that has never synced reads "up to date" |
| [B272](#b272--after-a-folder-is-removed-and-added-again-its-file-events-go-to-the-old-id) | After a folder is removed and added again, its file events go to the old id |
| [B260](#b260--the-poll-pushes-a-new-folders-first-sync-behind-every-other-folder) | The poll pushes a new folder's first sync behind every other folder | | [B260](#b260--the-poll-pushes-a-new-folders-first-sync-behind-every-other-folder) | The poll pushes a new folder's first sync behind every other folder |
| [B258](#b258--journalctl--p-warning-shows-nothing-whatever-the-daemon-logged) | `journalctl -p warning` shows nothing, whatever the daemon logged | | [B258](#b258--journalctl--p-warning-shows-nothing-whatever-the-daemon-logged) | `journalctl -p warning` shows nothing, whatever the daemon logged |
| [B256](#b256--pdfs-copy-right-after-a-save-copies-the-old-version) | `pdfs copy` right after a save copies the old version | | [B257](#b257--deleting-a-new-file-while-it-is-open-can-drop-the-bytes-its-handle-reads) | Deleting a new file while it is open can drop the bytes its handle reads |
| [B255](#b255--a-synced-folder-whose-files-are-all-gone-here-is-stuck-for-good) | A synced folder whose files are all gone here is stuck for good |
| [B254](#b254--an-edit-in-the-same-second-as-a-new-upload-never-arrives) | An edit in the same second as a new upload never arrives | | [B254](#b254--an-edit-in-the-same-second-as-a-new-upload-never-arrives) | An edit in the same second as a new upload never arrives |
| [B253](#b253--the-app-shows-the-sign-in-page-when-the-keyring-is-locked) | The app shows the sign-in page when the keyring is locked | | [B253](#b253--the-app-shows-the-sign-in-page-when-the-keyring-is-locked) | The app shows the sign-in page when the keyring is locked |
| [B252](#b252--computers-sharing-a-device-overwrite-each-others-profile) | Computers sharing a device overwrite each other's profile | | [B252](#b252--computers-sharing-a-device-overwrite-each-others-profile) | Computers sharing a device overwrite each other's profile |
@@ -292,6 +294,8 @@ a live account since.
| Entry | Title | | Entry | Title |
|---|---| |---|---|
| [B256](#b256--pdfs-copy-right-after-a-save-copies-the-old-version) | `pdfs copy` right after a save copies the old version |
| [B255](#b255--a-synced-folder-whose-files-are-all-gone-here-is-stuck-for-good) | A synced folder whose files are all gone here is stuck for good |
| [B194](#b194--on-a-fresh-install-the-tray-exits-and-the-takeout-page-cannot-remember-its-archives) | On a fresh install the tray exits and the Takeout page cannot remember its archives | | [B194](#b194--on-a-fresh-install-the-tray-exits-and-the-takeout-page-cannot-remember-its-archives) | On a fresh install the tray exits and the Takeout page cannot remember its archives |
| [B192](#b192--a-new-file-closed-empty-reaches-drive-only-half-a-minute-later) | A new file closed empty reaches Drive only half a minute later | | [B192](#b192--a-new-file-closed-empty-reaches-drive-only-half-a-minute-later) | A new file closed empty reaches Drive only half a minute later |
| [B181](#b181--the-app-shows-a-queued-edit-kept-as-a-conflict-copy-in-english) | The app shows a queued edit kept as a conflict copy in English | | [B181](#b181--the-app-shows-a-queued-edit-kept-as-a-conflict-copy-in-english) | The app shows a queued edit kept as a conflict copy in English |
@@ -375,6 +379,115 @@ a live account since.
Newest first. Newest first.
### B276 — A stop can end with a panic in a thread the stop never waited for
**Status:** Fixed, not verified live
**Found:** 2026-10-31, by an account run of `scripts/fuse-acceptance.sh --journal-check
--durability` against the audit branches. Every case passed, and the run still failed. During
the restart in the durability case on My files, the journal had three
`ERROR fuser::reply: Failed to send FUSE reply: No such file or directory (os error 2)` lines,
and after them a panic that the check did not report:
```
thread '<unnamed>' (1536340) panicked at .../tokio-1.53.1/src/runtime/time/entry.rs:539:9:
A Tokio 1.x context was found, but it is being shutdown.
```
The same panic is in the journal of a 3.6.0 stop on 2026-10-28, twice. B91 first saw it in a
meta worker.
**Where:** `mount_once` and `release_runtime` in `crates/pdfs-cli/src/main.rs`;
`JournalWatch` and `journal_problems` in `scripts/fuse-acceptance.py`.
**Cause.** A stop joins the drain workers, the sync engine and the FUSE worker pool, and then
`mount_once` shut the tokio runtime down. Other threads also use the runtime and are never
joined: a size lookup for a listing, a profile backup, a control request, a database vacuum.
A request still in progress when the stop began can start one of them. The thread id here is
nine below the next daemon's pid, so this thread started during the stop. Once the runtime is
shut down, that thread's next timer panics. Every call to Drive has a timer, because each one is
bounded.
The three `fuser::reply` lines are part of a correct stop. The stop aborts each mount's FUSE
connection so the unmount cannot hang. A request that was still running replies after that,
and the kernel answers a reply on an aborted connection with `ENOENT`.
The journal check matched only the `ERROR` token. A panic goes to stderr with no tracing level,
so the check did not see it.
**Fix.**
- On a stop, `release_runtime` leaves the runtime running for the process to end with, instead
of shutting it down. The process exits right after, so a thread the stop did not wait for
can finish its call or be ended by the exit, but it cannot panic. A mount that is about to
be tried again still shuts its runtime down with the 2 s limit from B165, because a new
runtime replaces it.
- The journal check counts `panicked at` lines.
- The journal check ignores the `ENOENT` reply error between the connection abort and the end
of that stop ("daemon stopping", or "control socket listening" for a mount that comes back).
At any other time the error still counts.
Making the bounded call helpers refuse new calls once the stop flag is set was rejected. The
stop sets that flag before it waits up to 10 s for the drain workers, so the refusal would cut
off the second half of an upload the stop is waiting for.
**Test:** `a_thread_left_running_at_a_stop_can_still_use_the_runtime` and
`a_mount_tried_again_shuts_its_runtime_down` in `crates/pdfs-cli/src/main.rs`. With the old
shutdown, the first one fails with the same panic at `entry.rs:539`. The new journal
filter was run on the journal of the run that found this: the three reply errors are no longer
counted, and the panic is.
---
### B275 — On a slow link, a file's last write can land only in a conflict copy
**Status:** Open
**Found:** 2026-10-31, CI on #60: `one_client_on_a_slow_link`, profile `one-client-wifi` seed 2,
failed with `Drive does not hold what the clients wrote, 120s after the links came up`. The
same seed passed 4 times locally, and #60 changes no daemon logic, so the failure depends on
timing.
**Where:** not located; the drain's conflict check in `crates/pdfs-fuse/src/drain.rs` is the
first suspect
**Repro:** Only the CI run above. The last steps created `x/z/y/a.txt` (37 bytes), wrote 42
bytes into it at offset 20, and brought every link up. Drive then held the file at its first
37 bytes, and the full content was in `x/z/y/a (sync-conflict …).txt`. No other client wrote
to it.
**Cause.** Not known. Nothing was lost, since the conflict copy holds the bytes, but the file
conflicted with its own create. B183 and B184 found the same symptom after a lost create
reply.
**Fix.** Not done. Needs a reproduction: run the seed in a loop under load and keep the state
directory of a failing run.
---
### B274 — Opening a file for writing fails with EIO while its queued revision lands
**Status:** Fixed, not verified live
**Found:** 2026-10-31, CI: the simulation
`a_file_whose_upload_was_not_read_back_takes_two_writes_at_once` failed once with
`Input/output error` on a write, after `copy queued revision into write scratch failed … No
such file or directory`. It did not fail in 54 local runs.
**Where:** the write open in `crates/pdfs-fuse/src/filesystem.rs`
**Repro:** Write a file, and open it for writing again at the moment its queued revision
finishes uploading. The open fails with EIO, and the program sees its save fail.
**Cause.** The open samples the queued revision, then copies its blob into the new handle's
scratch file. The drain lets go of the pending entry and then deletes the blob when the upload
lands. A blob deleted between the sample and the copy failed the open, although the re-check
just after the copy already retries a revision that changed during it.
**Fix.** A failed copy checks whether the pending entry still names that blob. When it does
not, the revision landed or was replaced, and the open goes around again with the base the
file has now, as the re-check does. Since the drain drops the entry before the blob, a blob
that is gone and still named is a real failure and still returns EIO.
**Test:** none deterministic; the simulation above covers the window.
---
### B273 — A restore proposes another computer's paths when the two share a hostname ### B273 — A restore proposes another computer's paths when the two share a hostname
**Status:** Open **Status:** Open
@@ -416,7 +529,7 @@ folder list has now.
### B271 — One failed listing makes a large first sync start over from nothing ### B271 — One failed listing makes a large first sync start over from nothing
**Status:** Open **Status:** Fixed, not verified live
**Found:** 2026-10-31, code audit after issue #46: a folder of about 19,900 items restored on a **Found:** 2026-10-31, code audit after issue #46: a folder of about 19,900 items restored on a
second computer sat at `scanning…` with nothing downloaded, and the journal showed second computer sat at `scanning…` with nothing downloaded, and the journal showed
`RequestTimeout (http 408)`. `RequestTimeout (http 408)`.
@@ -433,8 +546,13 @@ the walk is done. Files that did download keep their baseline rows, so only the
but on a first sync that is all of it. Progress is also counted only after a folder's full but on a first sync that is all of it. Progress is also counted only after a folder's full
listing, so a large folder shows `scanning…` with 0 checked for a long time. listing, so a large folder shows `scanning…` with 0 checked for a long time.
**Fix.** Not done. Record a failed subtree as a gap, as undecryptable nodes already are: no **Fix.** A listing that fails below the sync root records that folder as a gap in
deletions under it, and the rest of the pass still applies. `RemoteGaps`, next to the folders whose listing came back short. The planner deletes nothing
under a gap, the rest of the pass still applies, and the pass report names the folder as
tried again next pass. A failed listing of the root itself still fails the pass, since there is
nothing left to apply. The progress count inside one large folder is unchanged.
**Test:** `a_folder_that_failed_to_list_hides_everything_under_it` in `sync::planner`.
--- ---
@@ -461,7 +579,7 @@ choice to the user: delete on the other side, or sync from scratch.
### B269 — A restore that skipped every folder reports success ### B269 — A restore that skipped every folder reports success
**Status:** Open **Status:** Partly fixed
**Found:** 2026-10-31, code audit after issue #46: the journal showed `restore complete **Found:** 2026-10-31, code audit after issue #46: the journal showed `restore complete
restored=0 skipped=1` several times, and the user saw nothing. restored=0 skipped=1` several times, and the user saw nothing.
**Where:** the `RestoreSyncFolders` and `RestoreDeviceFolders` handler in **Where:** the `RestoreSyncFolders` and `RestoreDeviceFolders` handler in
@@ -476,15 +594,20 @@ result, `restored 0 folder(s); skipped: …`, is then logged as a successful act
reason is only in the activity text and the journal. When the database insert fails partway, reason is only in the activity text and the journal. When the database insert fails partway,
the `?` also skips the `Rewatch` for the folders already restored. the `?` also skips the `Rewatch` for the folders already restored.
**Fix.** Not done. Log a restore that skipped folders as a failure, name the reason in the app **Fix.** `restore_sync_folders` returns what it restored and what it skipped, with the reason
and the CLI, and offer to replace or reset the folder in the way. Send the `Rewatch` on every for each skip. The daemon logs a restore that skipped any folder as a failed activity, with the
exit path. reasons as its detail, and warns in the journal. A database error partway now sends the
`Rewatch` for the folders already restored before it returns. `pdfs devices join` points to
`pdfs activity` for the result.
Left: the app shows the result only in the activity feed, and nothing offers to replace or reset
the folder in the way.
--- ---
### B268 — `pdfs diagnose` says "No problems found" while sync is stuck ### B268 — `pdfs diagnose` says "No problems found" while sync is stuck
**Status:** Open **Status:** Partly fixed. The sync engine's own state is not checked yet.
**Found:** 2026-10-31, issue #53: a folder that never synced, and `diagnose` found nothing. **Found:** 2026-10-31, issue #53: a folder that never synced, and `diagnose` found nothing.
**Where:** `diagnose` in `crates/pdfs-cli/src/main.rs` **Where:** `diagnose` in `crates/pdfs-cli/src/main.rs`
@@ -497,8 +620,16 @@ folders or for `Diagnostics`, so a folder in error and a stalled worker are not
control socket file with no daemon behind it is also reported `[ok]`; only the next line says control socket file with no daemon behind it is also reported `[ok]`; only the next line says
the daemon does not answer. the daemon does not answer.
**Fix.** Not done. Warn on failing or parked writes, a pause, old staged data and folders in **Fix.** `diagnose` warns about writes that keep failing (with the last error), a write that
error, and report the sync engine's state once B259 exposes it. has waited a day or more to upload, a pause, and each synced folder in `error` or `conflict`
(`sync_health` and `folder_health` in `crates/pdfs-cli/src/main.rs`). A control socket file
that nothing listens on is a warning, not `[ok]`. Parked uploads are left out: a file still
wearing a download's temporary name is normal. Not done: the sync engine's state, which
`Diagnostics` reports once B259's change is in.
**Test:** `a_stuck_queue_or_a_pause_is_reported` and `a_synced_folder_in_error_is_reported` in
`pdfs-cli`. On 2026-10-31 the new `diagnose` against a healthy 3.6.0 daemon still reported
"No problems found".
--- ---
@@ -548,7 +679,7 @@ error, reads as offline. That covers every caller at once, the database ones inc
### B265 — Address keys whose token is for an inactive account key never open ### B265 — Address keys whose token is for an inactive account key never open
**Status:** Open **Status:** Fixed, not verified live
**Found:** 2026-10-31, code audit after issue #40: a Duo account whose mount failed on every **Found:** 2026-10-31, code audit after issue #40: a Duo account whose mount failed on every
start with `fetch My Files root: cryptography error: failed to decrypt: missing key`, after three start with `fetch My Files root: cryptography error: failed to decrypt: missing key`, after three
`failed to decrypt address key token` warnings. The web app worked. B210's fix does not apply, `failed to decrypt address key token` warnings. The web app worked. B210's fix does not apply,
@@ -565,8 +696,17 @@ already reports "token is for inactive account key", so the case is known. Separ
`addresses()` fails as a whole through `?` when one address has no usable primary key, which `addresses()` fails as a whole through `?` when one address has no usable primary key, which
takes the working addresses down with it. takes the working addresses down with it.
**Fix.** Not done, and it belongs in the SDK. Use every account key that opens for decrypting **Fix.** In `proton-sdk` 0.10.1 (narrrl/proton-sdk-rs#3). Tokens are tried with every account
tokens, and only the active ones for signing. Skip and log an address that cannot be opened. key that opens, active ones first, and signing uses active keys only. An address that cannot be
opened is left out with a warning, and the others work; `addresses()` fails only when none
opens. When the left-out address is the default, `default_address` fails with the reason
rather than signing with another address. `key_report` walks the same chain, so `pdfs diagnose`
reports such a key as opening. Not verified against an account with an inactive key, since
none is at hand; issue #40's reporter can confirm.
**Test:** `an_address_key_whose_token_is_for_an_inactive_account_key_opens`,
`an_address_whose_keys_do_not_open_leaves_the_others_working` and
`an_account_whose_addresses_all_fail_to_open_is_an_error` in `proton-sdk`.
--- ---
@@ -797,9 +937,46 @@ are off. A terminal that merely inherited `JOURNAL_STREAM` keeps the plain forma
--- ---
### B256 — `pdfs copy` right after a save copies the old version ### B257 — Deleting a new file while it is open can drop the bytes its handle reads
**Status:** Fixed, not verified live **Status:** Fixed, not verified live
**Found:** 2026-10-27, the simulation
`a_file_deleted_offline_while_open_still_reads_and_takes_writes` failing once on CI with a cache
tag left behind, then on every run with a 50 ms pause added to `ContentCache::store_file`.
**Where:** `retire_create` in `crates/pdfs-fuse/src/drain.rs`; `queue_trash` and
`remove_replaced` in `crates/pdfs-fuse/src/lib.rs`; `trash_child` in
`crates/pdfs-fuse/src/filesystem.rs`; `ContentCache::reconcile` in `crates/pdfs-core/src/cache.rs`
**Repro:** Create a file in the mount, wait until `pdfs status` shows nothing queued, open it,
and delete it while the link is down. If the create was still landing, a partial write or a read
through the open handle fails with "Input/output error", and the cache can keep a tag with no
bytes until the next start.
**Cause.** A create that carries the file's bytes retires its op, then reads the node back,
caches the bytes and only then gives the tree the real uid. Nothing counted that window, so the
queue read empty while it ran, which B119 promises means the file reads without the network. A
delete in that window found the placeholder uid in the tree, and trashed the node by its real
uid. `queue_trash` asked the tree whether that uid was open, found no entry for it, and dropped
the cached bytes the open handle reads, which B186 had kept. When the cache store was running at
the same moment, its tag was written after the eviction, and nothing removed it.
**Fix.** The create's landing is counted like an upload's, from the op's retirement until the
tree has the real uid. The delete and the replace pass whether they found the file open, so
`queue_trash` no longer depends on the tree having the new uid. Opening the cache removes a tag
whose bytes are gone, which also covers a crash between an eviction's two removals.
**Test:** The simulation `a_new_file_deleted_while_open_as_its_create_lands_still_reads` holds a
create's landing and fails without the count. The simulation above passes again with the 50 ms
pause, since it now waits for `Daemon::drained`, which counts the landing. Unit test
`a_tag_without_its_blob_is_swept_at_open` in `crates/pdfs-core/src/cache.rs`. No test hits the
delete inside the cache store itself: the window is too short to hold from the simulation.
---
### B256 — `pdfs copy` right after a save copies the old version
**Status:** Fixed, checked on 2026-10-27 by the acceptance case "regression B256" on a live
account
**Found:** 2026-10-26, the simulation `a_pdfs_copy_copies_on_drive_and_waits_for_unsynced_changes` **Found:** 2026-10-26, the simulation `a_pdfs_copy_copies_on_drive_and_waits_for_unsynced_changes`
failing on most runs. The feature was not released yet. failing on most runs. The feature was not released yet.
**Where:** `release` in `crates/pdfs-fuse/src/filesystem.rs`; `copy_to` in **Where:** `release` in `crates/pdfs-fuse/src/filesystem.rs`; `copy_to` in
@@ -823,7 +1000,8 @@ A move checks the same set.
### B255 — A synced folder whose files are all gone here is stuck for good ### B255 — A synced folder whose files are all gone here is stuck for good
**Status:** Fixed, not verified live **Status:** Fixed, checked on 2026-10-27 by the acceptance case "regression B255" on a live
account
**Found:** 2026-10-26, issue #46: a folder restored from another computer never downloaded. **Found:** 2026-10-26, issue #46: a folder restored from another computer never downloaded.
**Where:** `push_pass` in `crates/pdfs-fuse/src/sync.rs`; `reset_sync_folder` in **Where:** `push_pass` in `crates/pdfs-fuse/src/sync.rs`; `reset_sync_folder` in
`crates/pdfs-fuse/src/devices.rs` `crates/pdfs-fuse/src/devices.rs`
+9
View File
@@ -11,6 +11,15 @@ scratch (user data in `staging/` and `recovery/` is never touched by this).
## [Unreleased] ## [Unreleased]
### Fixed
- **A new file reads without the network as soon as `pdfs status` shows nothing queued**
(B257). A new file's upload counted as done before its bytes were cached, so a script that
waited for an empty queue and then went offline could not read it. A new file deleted while
open in that moment could also lose the bytes its open handle reads, which made a write or
read through that handle fail with "Input/output error". The cache also removes leftover
tags of evicted files when it opens.
## [3.6.0] — 2026-10-27 ## [3.6.0] — 2026-10-27
Schema: **44**, unchanged. Schema: **44**, unchanged.
+1 -1
View File
@@ -246,7 +246,7 @@ remaining cases are skipped. Cases that make many round trips by design get twic
| `--report-json PATH` | Every result and recorded observation. Attach it to a bug report | | `--report-json PATH` | Every result and recorded observation. Attach it to a bug report |
| `--report-junit PATH` | JUnit XML. `known` is reported as skipped | | `--report-junit PATH` | JUnit XML. `known` is reported as skipped |
| `--budget SECONDS` | Flags cases that pass but take longer (`PDFS_ACCEPTANCE_BUDGET`) | | `--budget SECONDS` | Flags cases that pass but take longer (`PDFS_ACCEPTANCE_BUDGET`) |
| `--journal-check` | Fails the run if `proton-drive.service` logged an error during it, or the warnings "queued write conflicts; keeping a conflict copy" and "a fuse worker has held the same job for a long time" | | `--journal-check` | Fails the run if `proton-drive.service` logged an error or a panic during it, or the warnings "queued write conflicts; keeping a conflict copy" and "a fuse worker has held the same job for a long time". A `Failed to send FUSE reply: No such file or directory` during a stop does not count: it is a request answering after the stop aborted its connection |
| `--durability` | Restarts the service mid-suite and checks the bytes written before it. Proves the queue survives a restart. `PDFS_ACCEPTANCE_UNIT` names another unit | | `--durability` | Restarts the service mid-suite and checks the bytes written before it. Proves the queue survives a restart. `PDFS_ACCEPTANCE_UNIT` names another unit |
| `--fail-fast` | Stops at the first failure | | `--fail-fast` | Stops at the first failure |
| `--list` | Prints every case and its targets | | `--list` | Prints every case and its targets |
+29 -5
View File
@@ -3561,8 +3561,36 @@ JOURNAL_WARNINGS = (
"queued write conflicts; keeping a conflict copy", "queued write conflicts; keeping a conflict copy",
"a fuse worker has held the same job for a long time", "a fuse worker has held the same job for a long time",
"re-interned an authority missing from the local tree", "re-interned an authority missing from the local tree",
# A Rust panic goes to stderr without a tracing level, so the ERROR match
# does not see it (`docs/BUGS.md` B276).
"panicked at",
) )
# A stop aborts each mount's FUSE connection so the unmount cannot hang. A
# request still running then replies into a connection the kernel has
# closed, and the kernel refuses that reply with ENOENT. While a stop is in
# progress that error is expected (`docs/BUGS.md` B276); at any other time it
# still counts.
STOP_STARTED = "aborted FUSE connection to unblock unmount"
STOP_ENDED = ("daemon stopping", "control socket listening")
LATE_REPLY = "Failed to send FUSE reply: No such file or directory"
def journal_problems(lines: list[str]) -> list[str]:
"""The lines of `lines` that count as problems."""
problems = []
stopping = False
for line in lines:
if STOP_STARTED in line:
stopping = True
elif any(marker in line for marker in STOP_ENDED):
stopping = False
if stopping and LATE_REPLY in line:
continue
if ERROR_LEVEL.search(line) or any(warning in line for warning in JOURNAL_WARNINGS):
problems.append(line)
return problems
def read_journal(unit: str, since: float) -> list[str] | None: def read_journal(unit: str, since: float) -> list[str] | None:
"""The unit's journal lines since `since`, or None without journalctl. """The unit's journal lines since `since`, or None without journalctl.
@@ -3621,11 +3649,7 @@ class JournalWatch:
# Filtering on syslog priority (`-p err`) finds nothing: the daemon writes # Filtering on syslog priority (`-p err`) finds nothing: the daemon writes
# tracing levels as text on stdout, which journald files at the unit's # tracing levels as text on stdout, which journald files at the unit's
# default priority. Match the level token instead. # default priority. Match the level token instead.
return [ return journal_problems(lines)
line
for line in lines
if ERROR_LEVEL.search(line) or any(warning in line for warning in JOURNAL_WARNINGS)
]
# -------------------------------------------------------------------------- # --------------------------------------------------------------------------