Compare commits

...
26 Commits
Author SHA1 Message Date
nvrl 50483bf3e1 chore(release): 2.8.2 2026-10-01 21:39:05 +02:00
nvrl 4caba0a34c chore(deps): require proton SDK 0.7.2
0.7.2 resolves each shared root with its own key. With 0.7.1, when
several accepted shares came from the same owner's volume, only the first
appeared under Shared with me and the rest were skipped with "missing
key" (B112, proton-sdk-rs#2).
2026-10-01 18:17:25 +02:00
Nils Pukropp e8554d6434 chore(aur): update packages to 2.8.1 2026-10-01 14:58:09 +00:00
Claude ba1fdd65be chore(release): 2.8.1 2026-09-30 17:02:31 +00:00
nvrl 0d4163b9bf fix(service): start with the graphical session, not default.target (#24)
default.target is reached before the desktop exports DISPLAY/WAYLAND_DISPLAY
to the user manager, so After=graphical-session.target never applied. The
daemon's first keyring call D-Bus-activated the Secret Service without a
display; on KDE ksecretd aborted, the first start failed, and every login
left a coredump.

Install into graphical-session.target instead, and use reenable when the
front-ends enable the unit so existing default.target.wants links are
replaced rather than kept alongside the new one.

Fixes #23
2026-09-30 19:01:50 +02:00
Nils Pukropp f3c7a809f4 chore(aur): update packages to 2.8.0 2026-09-30 02:15:19 +00:00
nvrl c1b5378045 fix(aur): download sources from GitHub, not the homepage
73e60f1 pointed url at proton-drive.narl.io, and all three AUR PKGBUILDs
built their source URLs from $url, so the release tarball, the binary
archive and the git clone were fetched from the homepage and 404ed. The
sources now come from a separate _repo on GitHub; url stays the homepage.
2026-09-30 04:13:06 +02:00
nvrl 3d6f53c16d chore(release): 2.8.0 2026-09-30 03:56:47 +02:00
nvrl 8ceee5285b chore(fuse): log how long each queued upload takes
"pending upload landed" now carries upload_ms, the time on the wire, and
total_ms, the whole op including the conflict check and gap-fill. A slow
drain then shows where its time goes without a profiler.
2026-09-30 03:55:48 +02:00
nvrl a181f1a5f0 fix(core,fuse): keep working through network drops
The mount only ever went from offline to online: nothing marked it
offline again, so after the Wi-Fi dropped every create, mkdir, delete and
rename called Drive, waited out the SDK's retries for up to two minutes
and answered EIO. Background loops kept polling the dead link.

Connectivity now lives in pdfs-fuse's new link module and goes both
ways. A call that gets no answer (a transport error or a request
timeout) marks the mount offline; a probe thread checks every 2 s,
backing off to 30 s, and on recovery re-lists folders served from the
database, retries failed operations and wakes the drain. Interactive
calls get a 20 s deadline, so a dead link costs one slow call instead of
many.

While offline:
- create, mkdir, unlink, rmdir and rename fall back to the offline
  queue instead of answering EIO;
- folders list from the database even when not marked listed, and are
  re-listed once the link returns;
- reads of uncached data wait up to 30 s for the link, then retry once
  within the 120 s read deadline;
- statfs serves the last quota instead of fetching, and quota refreshes
  are single-flight;
- the event and photo event loops wait for the link instead of polling.

A create whose request timed out may have landed anyway. When the drain
replays it and finds a same-named twin of the same kind, created within
the last two minutes and still empty, it adopts that node instead of
making a conflict copy. The drain also leaves alone a name still held
by a queued trash.
2026-09-30 03:25:00 +02:00
nvrl 7251f00e91 chore(deps): require proton SDK 0.7.1
0.7.1 sends TCP keepalives on its connections, so a link that dies
silently is noticed instead of leaving a request waiting on a dead
socket until its timeout.
2026-09-30 03:25:00 +02:00
nvrl cb38f79aa3 fix(fuse,scripts): log why a create finds no parent and fit the metadata floor to FUSE
A create that answered ENOENT left no trace in the log. It now warns
when the parent folder is no longer known, and when the parent left the
tree while its children were being listed, naming the parent inode and
the new name.

The acceptance script's metadata throughput floor was 20 operations per
second, which a FUSE mount cannot reach: each create and unlink waits for
Drive before it returns, about 350 ms apiece. On a FUSE mount the floor
is now 3 operations per second; other paths keep 20. The mount type is
read from /proc/self/mountinfo.
2026-09-29 21:22:54 +02:00
nvrl 54e9921efb fix(fuse): stop reads deadlocking on size upgrades and retry stale renames
A read that settles a provisional size invalidated the file's page cache
while the kernel held that read's page lock, so the worker waited on
itself forever (B110). Size upgrades now invalidate attributes only.

Renaming a node right after renaming it could be refused by Drive as out
of date, because the name hash read just before still held the old name.
Every remote rename now goes through Core::rename_remote, which retries
InvalidRequirements for up to ten seconds (B111).
2026-09-29 19:47:20 +02:00
nvrl f252b09757 fix(fuse): keep the size of a file open for writing across re-listings
B109: an online create seals an empty file on Drive, and until the last
close the written bytes live only in the open handle. A re-listing of
the parent, a size upgrade or a create landing re-interned the node with
Drive's empty size, the kernel capped reads at that size, and SQLite
reported "database disk image is malformed". State::keep_open_write_size
now carries the handle's size and local mtime over in all three places.

The acceptance script waits for My files to mount after a daemon restart
instead of failing at once.
2026-09-29 16:49:13 +02:00
nvrl a2992b3b9d fix(core,fuse): keep replaced and vanished files from failing saves, reads and syncs
Fixes from the account acceptance runs:

- B104: a reader open across a rename-over keeps working; the replaced
  node is unlinked like any other open file instead of forgotten.
- B105: a drain worker holding a revision withdrawn by an unlink no
  longer uploads it as a recovered-… conflict copy (Db::op_exists).
- B106: an open victim's queued ops are withdrawn at trash time, so a
  repeated editor save no longer fails with AlreadyExists; the staged
  blob goes with the last close.
- B107: a remote read gives up after 120 s with EIO instead of hanging
  its caller; the timer is built inside the runtime.
- B108: a mirror upload whose local file was deleted after the scan is
  skipped rather than counted as an error.

The acceptance script reports the failing line for non-assert errors,
ignores queue ops that predate the run, purges with delete-forever
first, and diagnoses a malformed SQLite database against a local replay.
2026-09-29 16:20:51 +02:00
nvrl 7b67db5ced fix(core,fuse,cli): harden writes, trash and cleanup found by account acceptance runs
The FUSE acceptance suite now runs on the signed-in account by default and
cleans up after itself; its first live runs found three daemon bugs.

- B101: a pwrite past EOF now authors the hole it leaves, so truncated bytes
  no longer come back from the base. A write attached to a create while that
  create uploads becomes a revision of the real uid instead of being dropped.
- B102: removing a folder drops the queued revisions of files below it, found
  through the node tree, with their blobs and in-flight uploads. `pdfs rm`
  now drops queued ops too.
- B103: each server call of a trash refresh times out after 120 s, so one hung
  call can no longer freeze the trash listing until restart.

Acceptance suite: account mode with a manifest, lock and reaper; cleanup by
uid; queued ops naming a test root count as leftovers; new edge-case,
throughput and B100/B101 cases; the wide-directory case is split from the
deep tree and gets a doubled budget. Adds `pdfs trash --json`.
2026-09-29 10:09:44 +02:00
nvrl 84c9ade8e3 fix(fuse): settle provisional sizes before reads and write opens
A file listed by the cheap enumeration reports its ciphertext size until
the size upgrade lands. Since B84 a read on that size fails with EIO, and
a write open started its handle at it. Reads and write opens now fetch
the real size first. A folder past the size-upgrade cap is queued instead
of dropped, so a large copy no longer leaves most folders provisional
(B100).
2026-09-29 07:37:41 +02:00
nvrl 9e26cd0285 chore(release): 2.7.0 2026-09-29 05:30:38 +02:00
nvrl 73e60f1151 chore: point homepage metadata at proton-drive.narl.io
The About dialog's website link, the README, and the package metadata
(Cargo, PKGBUILDs, AUR .SRCINFO, .deb, .rpm) now use the project
homepage. Source code, issue, and release links stay on GitHub.
2026-09-29 05:30:38 +02:00
nvrl 088e8f3b0d test(acceptance): cover moves between locations in the managed matrix
Every managed mode pair now also moves trees between the two folders in both
directions, moves several sources in one command, and passes a tree through
My files and out again. Each case waits for Drive before and after, then
checks the tree is only at the destination, byte for byte, with no conflict
copies. Refusal cases cover a taken name, a folder into itself, a mirror
source holding a symlink, and an on-demand file still open for writing.
2026-09-29 00:38:40 +02:00
nvrl 261d5dbf84 feat(core,fuse,cli): reuse cached content and take several sources in moves
- A mirror download first copies the file out of the content cache when the
  same revision is cached whole or block by block, so moving a file out of an
  on-demand folder into a mirror costs no traffic for what was already read.
- `pdfs move a b c DEST` moves every source, reports each failure and exits
  non-zero if any failed.
- Mirror-to-mirror moves rename with RENAME_NOREPLACE, so a name that
  appeared at the destination after the check is never overwritten.
- The local copy a move out of a mirror drops is checked again just before
  it is deleted; anything written into it meanwhile keeps the copy and is
  uploaded as new by the next pass.
2026-09-29 00:38:40 +02:00
nvrl 5f9d07aff1 feat(fuse,core,cli,gui): move between locations without transferring content
A move request with two absolute paths is resolved against every
location (My files, on-demand and mirror synced folders) and made as a
server-side move_node. The local side is levelled afterwards: mounts
forget and re-list, a mirror folder renames its copy and baseline along
or drops the copy once Drive holds it.

Anything only this disk holds is refused up front: queued ops, open
writes, unsynced or ignored mirror files and symlinks. The GUI Move
dialog gains a top-level list of locations.
2026-09-29 00:26:56 +02:00
nvrl 3fb7b9be6b chore(release): 2.6.2 2026-09-28 23:58:26 +02:00
nvrl 6c3a50ce7f fix(gui): let the Sync page use the full width
The transfer label did not ellipsize, so a long file name set the
column's minimum width and pushed it past the clamp. The page width
changed with every file.

The label now ellipsizes in the middle, and the Sync page drops the
clamp so Overview and Folders fill the space next to the sidebar.
2026-09-28 23:58:26 +02:00
nvrl f8304a0e86 fix(gui): keep Albums and Places selected when returning to Photos
Every visit to the Photos page loaded the timeline, and a finished
timeline load always shows the timeline. Leaving Albums or Places and
coming back therefore showed the timeline under the Albums or Places
toggle.

A visit now reloads the view the toggle selects. An open album or place
still loads its photos as before.
2026-09-28 23:58:26 +02:00
Nils Pukropp e63f39f8c2 chore(aur): update packages to 2.6.1 2026-09-28 21:44:18 +00:00
56 changed files with 8298 additions and 3104 deletions

No files matched your search

+1 -1
View File
@@ -166,7 +166,7 @@ jobs:
Maintainer: Nils Pukropp <contact@narl.io> Maintainer: Nils Pukropp <contact@narl.io>
Depends: fuse3, libgtk-4-1 (>= 4.12.0), libadwaita-1-0 (>= 1.5.0), libsecret-1-0, libwebkitgtk-6.0-4, libimage-exiftool-perl Depends: fuse3, libgtk-4-1 (>= 4.12.0), libadwaita-1-0 (>= 1.5.0), libsecret-1-0, libwebkitgtk-6.0-4, libimage-exiftool-perl
Installed-Size: ${INSTALLED_SIZE} Installed-Size: ${INSTALLED_SIZE}
Homepage: https://github.com/narrrl/proton-drive-linux Homepage: https://proton-drive.narl.io
Description: Unofficial Proton Drive client for Linux Description: Unofficial Proton Drive client for Linux
Files-on-demand Proton Drive for the Linux desktop, unaffiliated with Proton AG: Files-on-demand Proton Drive for the Linux desktop, unaffiliated with Proton AG:
- FUSE virtual mount with block-level caching - FUSE virtual mount with block-level caching
Generated
+8 -8
View File
@@ -3474,7 +3474,7 @@ checksum = "2ee67f1008b1ba2321834326597b8e186293b049a023cdef258527550b9935b4"
[[package]] [[package]]
name = "pdfs-cli" name = "pdfs-cli"
version = "2.6.1" version = "2.8.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"clap", "clap",
@@ -3489,7 +3489,7 @@ dependencies = [
[[package]] [[package]]
name = "pdfs-core" name = "pdfs-core"
version = "2.6.1" version = "2.8.2"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dbus-secret-service-keyring-store", "dbus-secret-service-keyring-store",
@@ -3513,7 +3513,7 @@ dependencies = [
[[package]] [[package]]
name = "pdfs-fuse" name = "pdfs-fuse"
version = "2.6.1" version = "2.8.2"
dependencies = [ dependencies = [
"base64 0.23.1", "base64 0.23.1",
"fuser", "fuser",
@@ -3533,7 +3533,7 @@ dependencies = [
[[package]] [[package]]
name = "pdfs-gui" name = "pdfs-gui"
version = "2.6.1" version = "2.8.2"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-channel", "async-channel",
@@ -3773,9 +3773,9 @@ dependencies = [
[[package]] [[package]]
name = "proton-drive-rs" name = "proton-drive-rs"
version = "0.7.0" version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0f6996724e51e99f03d2843828de9477afc39baa764403cb69490cabc1c4d93" checksum = "d6ce113d234474821d34787436ee80e1458e3d8a8f97212219615ed315eb70be"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"base64 0.23.1", "base64 0.23.1",
@@ -3800,9 +3800,9 @@ dependencies = [
[[package]] [[package]]
name = "proton-sdk" name = "proton-sdk"
version = "0.7.0" version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91cc25654133bd5714407a1df0996eaf214a6e8e0bb95d8139bc42e7cee31c34" checksum = "2a33c67df9694f90ce60e5b34984275956244138511b4640ca0d100e1dac8b3d"
dependencies = [ dependencies = [
"aes-gcm 0.11.1", "aes-gcm 0.11.1",
"async-trait", "async-trait",
+4 -4
View File
@@ -3,20 +3,20 @@ resolver = "2"
members = ["crates/pdfs-core", "crates/pdfs-fuse", "crates/pdfs-cli", "crates/pdfs-gui"] members = ["crates/pdfs-core", "crates/pdfs-fuse", "crates/pdfs-cli", "crates/pdfs-gui"]
[workspace.package] [workspace.package]
version = "2.6.1" version = "2.8.2"
edition = "2024" edition = "2024"
license = "MIT" license = "MIT"
authors = ["Nils Pukropp <contact@narl.io>"] authors = ["Nils Pukropp <contact@narl.io>"]
rust-version = "1.96" rust-version = "1.96"
repository = "https://github.com/narrrl/proton-drive-linux" repository = "https://github.com/narrrl/proton-drive-linux"
homepage = "https://github.com/narrrl/proton-drive-linux" homepage = "https://proton-drive.narl.io"
readme = "README.md" readme = "README.md"
keywords = ["proton", "drive", "fuse", "sync", "linux"] keywords = ["proton", "drive", "fuse", "sync", "linux"]
categories = ["filesystem", "command-line-utilities"] categories = ["filesystem", "command-line-utilities"]
[workspace.dependencies] [workspace.dependencies]
proton-sdk = { version = "0.7.0" } proton-sdk = { version = "0.7.2" }
proton-drive-rs = { version = "0.7.0" } proton-drive-rs = { version = "0.7.2" }
# Internal crates. # Internal crates.
pdfs-core = { path = "crates/pdfs-core" } pdfs-core = { path = "crates/pdfs-core" }
+2
View File
@@ -8,6 +8,8 @@
local folders, Photos, sharing, and a native GTK4 app with tray and search launcher, plus a local folders, Photos, sharing, and a native GTK4 app with tray and search launcher, plus a
scriptable CLI. scriptable CLI.
**[proton-drive.narl.io](https://proton-drive.narl.io)**
[![CI](https://github.com/narrrl/proton-drive-linux/actions/workflows/ci.yml/badge.svg)](https://github.com/narrrl/proton-drive-linux/actions/workflows/ci.yml) [![CI](https://github.com/narrrl/proton-drive-linux/actions/workflows/ci.yml/badge.svg)](https://github.com/narrrl/proton-drive-linux/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/narrrl/proton-drive-linux?sort=semver)](https://github.com/narrrl/proton-drive-linux/releases/latest) [![Release](https://img.shields.io/github/v/release/narrrl/proton-drive-linux?sort=semver)](https://github.com/narrrl/proton-drive-linux/releases/latest)
[![AUR](https://img.shields.io/aur/version/proton-drive-for-linux)](https://aur.archlinux.org/packages/proton-drive-for-linux) [![AUR](https://img.shields.io/aur/version/proton-drive-for-linux)](https://aur.archlinux.org/packages/proton-drive-for-linux)
+43 -12
View File
@@ -30,7 +30,7 @@ struct Cli {
/// Emit machine-readable JSON instead of formatted text. /// Emit machine-readable JSON instead of formatted text.
/// ///
/// Applies to the query commands (`status`, `ls`, `pins`, `sync list`, /// Applies to the query commands (`status`, `ls`, `pins`, `sync list`,
/// `devices list`, `locations`, `transfers`, `activity`, `cache inspect`). Commands that /// `devices list`, `locations`, `transfers`, `activity`, `trash`, `cache inspect`). Commands that
/// perform an action keep their human output — a script that needs to know /// perform an action keep their human output — a script that needs to know
/// whether one succeeded has the exit code. /// whether one succeeded has the exit code.
#[arg(long, global = true)] #[arg(long, global = true)]
@@ -290,12 +290,17 @@ enum Command {
/// New name (a single path component). /// New name (a single path component).
new_name: String, new_name: String,
}, },
/// Move a file or folder into another folder via the running daemon. /// Move files or folders into another folder via the running daemon.
///
/// The paths may be in different locations: My files and any synced
/// folder, on-demand or mirrored. The move happens on Proton Drive, so
/// nothing is downloaded or uploaded again, unlike `mv` between two
/// locations. The last path is the destination folder.
Move { Move {
/// File/folder path, inside the mountpoint or relative to it. /// Files/folders to move, then the destination folder: each inside a
path: PathBuf, /// location, or relative to the mountpoint.
/// Destination folder path, inside the mountpoint or relative to it. #[arg(required = true, num_args = 2.., value_name = "PATH")]
new_parent: PathBuf, paths: Vec<PathBuf>,
}, },
/// Trash a file or folder via the running daemon. /// Trash a file or folder via the running daemon.
Rm { Rm {
@@ -800,7 +805,10 @@ fn main() -> Result<()> {
scope, scope,
} => cmd_search(query, limit, scope), } => cmd_search(query, limit, scope),
Command::Rename { path, new_name } => cmd_rename(path, new_name), Command::Rename { path, new_name } => cmd_rename(path, new_name),
Command::Move { path, new_parent } => cmd_move(path, new_parent), Command::Move { mut paths } => {
let new_parent = paths.pop().expect("clap requires at least two paths");
cmd_move(paths, new_parent)
}
Command::Rm { path } => cmd_rm(path), Command::Rm { path } => cmd_rm(path),
Command::Mkdir { parent, name } => cmd_mkdir(parent, name), Command::Mkdir { parent, name } => cmd_mkdir(parent, name),
Command::Upload { sources, parent } => cmd_upload(sources, parent), Command::Upload { sources, parent } => cmd_upload(sources, parent),
@@ -2439,15 +2447,34 @@ fn cmd_rename(path: PathBuf, new_name: String) -> Result<()> {
Ok(()) Ok(())
} }
fn cmd_move(path: PathBuf, new_parent: PathBuf) -> Result<()> { /// Move each of `sources` into `new_parent`, one request each. A refusal names
/// its source and the rest still move; the command fails if any did not.
fn cmd_move(sources: Vec<PathBuf>, new_parent: PathBuf) -> Result<()> {
let new_parent = path_arg(&new_parent)?;
let mut failed = 0;
for source in &sources {
let outcome = path_arg(source).and_then(|path| {
match control_request(CtlRequest::Move { match control_request(CtlRequest::Move {
path: path_arg(&path)?, path,
new_parent: path_arg(&new_parent)?, new_parent: new_parent.clone(),
})? { })? {
CtlResponse::Ok { message } => println!("{message}"), CtlResponse::Ok { message } => Ok(message),
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)), CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"), other => bail!("unexpected response: {other:?}"),
} }
});
match outcome {
Ok(message) => println!("{message}"),
Err(e) if sources.len() == 1 => return Err(e),
Err(e) => {
eprintln!("{}: {e:#}", source.display());
failed += 1;
}
}
}
if failed > 0 {
bail!("{failed} of {} could not be moved", sources.len());
}
Ok(()) Ok(())
} }
@@ -2524,7 +2551,11 @@ fn cmd_diagnostics() -> Result<()> {
} }
fn cmd_trash() -> Result<()> { fn cmd_trash() -> Result<()> {
match control_request(CtlRequest::ListTrash)? { let response = control_request(CtlRequest::ListTrash)?;
if emit_json(&response)? {
return Ok(());
}
match response {
CtlResponse::Entries { entries } if entries.is_empty() => println!("(trash is empty)"), CtlResponse::Entries { entries } if entries.is_empty() => println!("(trash is empty)"),
CtlResponse::Entries { entries } => { CtlResponse::Entries { entries } => {
for e in entries { for e in entries {
+94
View File
@@ -730,6 +730,60 @@ impl ContentCache {
Some(buf) Some(buf)
} }
/// Write the whole content of `uid` at `(mtime, size)` to `dst` from the
/// cache alone, and report whether that worked.
///
/// A whole-file blob is cloned. Otherwise every block of the revision has to
/// be cached, under its recorded geometry or, failing that, the uniform one
/// the read path would use. Anything short of the full content leaves
/// nothing at `dst`, so a caller falls back to a download with a clean slate.
///
/// This is what lets a mirror folder take a file an on-demand mount has
/// already read, after a move between them, without fetching it again. The
/// trust is the read path's: the same `(mtime, size)` key, and blocks that
/// were validated when they were fetched.
pub fn copy_cached_to(&self, uid: &NodeUid, mtime: i64, size: u64, dst: &Path) -> bool {
if size == 0 {
return false;
}
if let Some(blob) = self.valid_blob(uid, mtime, size) {
if Self::clone_or_copy(&blob, dst).is_ok_and(|len| len == size) {
self.touch(Self::key(uid));
return true;
}
let _ = std::fs::remove_file(dst);
return false;
}
let geometry = self
.block_geometry(uid, mtime, size)
.unwrap_or_else(|| BlockGeometry::uniform(size));
if geometry.size() != size {
return false;
}
let spans = geometry.spans(0, size);
// Checked up front so a file with one block missing costs stats, not a
// copy of every block before it.
if !spans
.iter()
.all(|&span| self.has_block(uid, mtime, size, span))
{
return false;
}
let written = (|| -> Option<()> {
use std::io::Write;
let mut out = std::fs::File::create(dst).ok()?;
for span in spans {
out.write_all(&self.cached_block(uid, mtime, size, span)?)
.ok()?;
}
(out.metadata().ok()?.len() == size).then_some(())
})();
if written.is_none() {
let _ = std::fs::remove_file(dst);
}
written.is_some()
}
/// Store `bytes` as the cached content for `uid`, tagged with `(mtime, /// Store `bytes` as the cached content for `uid`, tagged with `(mtime,
/// size)`. The blob is written to a temp file then renamed so a concurrent /// size)`. The blob is written to a temp file then renamed so a concurrent
/// reader never sees a partial blob; the meta tag is written last so a /// reader never sees a partial blob; the meta tag is written last so a
@@ -2561,6 +2615,46 @@ mod tests {
assert!(c.cached_block(&u, 100, 5000, blk(5000, 0)).is_none()); assert!(c.cached_block(&u, 100, 5000, blk(5000, 0)).is_none());
} }
#[test]
fn a_cached_blob_is_copied_out_whole() {
let (c, d) = cache();
let u = uid("blob");
c.store(&u, 7, 5, b"hello").unwrap();
let dst = d.path().join("out");
assert!(c.copy_cached_to(&u, 7, 5, &dst));
assert_eq!(std::fs::read(&dst).unwrap(), b"hello");
// Another revision is not this one.
assert!(!c.copy_cached_to(&u, 8, 5, &d.path().join("stale")));
assert!(!d.path().join("stale").exists());
}
#[test]
fn cached_blocks_are_copied_out_in_order() {
let (c, d) = cache();
let u = uid("blocks");
let size = BLOCK_SIZE + 3;
let first = vec![1_u8; BLOCK_SIZE as usize];
c.store_block(&u, 7, size, blk(size, 0), &first).unwrap();
c.store_block(&u, 7, size, blk(size, 1), b"end").unwrap();
let dst = d.path().join("out");
assert!(c.copy_cached_to(&u, 7, size, &dst));
let bytes = std::fs::read(&dst).unwrap();
assert_eq!(bytes.len() as u64, size);
assert_eq!(&bytes[..BLOCK_SIZE as usize], first.as_slice());
assert_eq!(&bytes[BLOCK_SIZE as usize..], b"end");
}
#[test]
fn a_file_with_a_block_missing_is_not_copied_out() {
let (c, d) = cache();
let u = uid("partial");
let size = BLOCK_SIZE + 3;
c.store_block(&u, 7, size, blk(size, 1), b"end").unwrap();
let dst = d.path().join("out");
assert!(!c.copy_cached_to(&u, 7, size, &dst));
assert!(!dst.exists(), "a partial copy must not be left behind");
}
/// The [`BlockSpan`] the uniform geometry gives block `idx` of a file of /// The [`BlockSpan`] the uniform geometry gives block `idx` of a file of
/// `size` bytes — the shape these tests were written against, and the one /// `size` bytes — the shape these tests were written against, and the one
/// this client's own uploads still have. /// this client's own uploads still have.
+4 -1
View File
@@ -339,7 +339,10 @@ pub enum Request {
/// single path component (no separators). Replies with [`Response::Ok`]. /// single path component (no separators). Replies with [`Response::Ok`].
Rename { path: String, new_name: String }, Rename { path: String, new_name: String },
/// Move a file or folder into a new parent folder. Both `path` and /// Move a file or folder into a new parent folder. Both `path` and
/// `new_parent` are mountpoint-relative. Replies with [`Response::Ok`]. /// `new_parent` are mountpoint-relative, or both absolute local paths; two
/// absolute paths may lie in different locations (My files, an on-demand
/// or a mirror synced folder), and the move is still made on Proton Drive
/// without transferring content. Replies with [`Response::Ok`].
Move { path: String, new_parent: String }, Move { path: String, new_parent: String },
/// Trash a file or folder. `path` is mountpoint-relative. Replies with /// Trash a file or folder. `path` is mountpoint-relative. Replies with
/// [`Response::Ok`]. /// [`Response::Ok`].
+10 -1
View File
@@ -559,6 +559,15 @@ impl Db {
if listed != Some(1) { if listed != Some(1) {
return Ok(None); return Ok(None);
} }
drop(conn);
self.known_children(parent).map(Some)
}
/// The children the DB knows under `parent`, whether or not the folder's
/// listing is still marked complete. Only for when nothing better can be
/// had: a listing dropped as stale may miss or keep a child.
pub fn known_children(&self, parent: &NodeUid) -> Result<Vec<Node>> {
let conn = self.read();
let mut stmt = conn.prepare( let mut stmt = conn.prepare(
"SELECT node_json FROM nodes "SELECT node_json FROM nodes
WHERE parent_uid = ?1 AND node_json IS NOT NULL AND trashed = 0", WHERE parent_uid = ?1 AND node_json IS NOT NULL AND trashed = 0",
@@ -568,7 +577,7 @@ impl Db {
for json in rows { for json in rows {
out.push(serde_json::from_str(&json?)?); out.push(serde_json::from_str(&json?)?);
} }
Ok(Some(out)) Ok(out)
} }
// --- Content-cache LRU index (P4) ------------------------------------- // --- Content-cache LRU index (P4) -------------------------------------
+142 -39
View File
@@ -66,6 +66,52 @@ pub fn op_supersedes(kind: &str) -> bool {
matches!(kind, OP_REVISION | OP_RENAME | OP_TRASH) matches!(kind, OP_REVISION | OP_RENAME | OP_TRASH)
} }
/// Delete the ops that removing `uid` takes with it, returning the staged blobs
/// they held.
///
/// That is every op for `uid` or for anything queued beneath it, plus the
/// revisions of files the node tree places beneath it. A revision op carries
/// no `parent_uid`, so the queue alone cannot tell that its file sits inside
/// a removed folder: the upload outlived the folder, failed forever against a
/// trashed node, and kept its blob on disk (`docs/BUGS.md` B102).
///
/// Only revisions are taken from the node tree. A queued rename that moved a
/// node *into* the folder has to land, or the node would stay where it was
/// on the server instead of going to the trash with the folder.
fn drop_doomed_ops(tx: &rusqlite::Transaction<'_>, uid: &str) -> Result<Vec<String>> {
const DOOMED: &str = "
WITH RECURSIVE
queued(uid) AS (
SELECT ?1
UNION
SELECT p.uid FROM pending_op p JOIN queued q ON p.parent_uid = q.uid
),
below(uid) AS (
SELECT ?1
UNION
SELECT n.uid FROM nodes n JOIN below b ON n.parent_uid = b.uid
),
doomed(id) AS (
SELECT id FROM pending_op
WHERE uid IN (SELECT uid FROM queued)
OR (kind = 'revision' AND uid IN (SELECT uid FROM below))
)";
let blobs: Vec<String> = {
let mut stmt = tx.prepare(&format!(
"{DOOMED}
SELECT blob_path FROM pending_op
WHERE id IN (SELECT id FROM doomed) AND blob_path IS NOT NULL"
))?;
let rows = stmt.query_map(params![uid], |r| r.get::<_, String>(0))?;
rows.collect::<std::result::Result<Vec<_>, _>>()?
};
tx.execute(
&format!("{DOOMED} DELETE FROM pending_op WHERE id IN (SELECT id FROM doomed)"),
params![uid],
)?;
Ok(blobs)
}
/// The volume id given to a node that exists only on this machine, so far. A /// The volume id given to a node that exists only on this machine, so far. A
/// real [`NodeUid`] is `{volume}~{link}`, so a placeholder is `local~<uuid>` and /// real [`NodeUid`] is `{volume}~{link}`, so a placeholder is `local~<uuid>` and
/// round-trips through the same `Display`/parse path as any other uid. /// round-trips through the same `Display`/parse path as any other uid.
@@ -174,6 +220,24 @@ pub struct AttachedBlob {
} }
impl Db { impl Db {
/// Whether a queued trash still holds `name` somewhere.
///
/// A replacing rename or a delete-then-recreate queues the trash of the old
/// node ahead of the op that wants its name. With several drain workers the
/// later op can run first and find the name still taken; this tells it to
/// wait for the trash rather than land under a conflict name.
pub fn has_pending_trash_named(&self, name: &str) -> Result<bool> {
let conn = self.read();
conn.query_row(
"SELECT EXISTS(
SELECT 1 FROM pending_op WHERE kind = ?1 AND name = ?2
)",
params![OP_TRASH, name],
|row| row.get(0),
)
.map_err(Into::into)
}
/// Whether a specific desired-state operation is still queued for a node. /// Whether a specific desired-state operation is still queued for a node.
pub fn has_pending_op(&self, uid: &str, kind: &str) -> Result<bool> { pub fn has_pending_op(&self, uid: &str, kind: &str) -> Result<bool> {
let conn = self.read(); let conn = self.read();
@@ -252,27 +316,9 @@ impl Db {
name: &str, name: &str,
created_at: i64, created_at: i64,
) -> Result<(i64, Vec<String>)> { ) -> Result<(i64, Vec<String>)> {
const SUBTREE: &str = "
WITH RECURSIVE doomed(uid) AS (
SELECT ?1
UNION
SELECT p.uid FROM pending_op p JOIN doomed d ON p.parent_uid = d.uid
)";
let mut conn = self.conn.lock(); let mut conn = self.conn.lock();
let tx = conn.transaction()?; let tx = conn.transaction()?;
let blobs: Vec<String> = { let blobs = drop_doomed_ops(&tx, uid)?;
let mut stmt = tx.prepare(&format!(
"{SUBTREE}
SELECT blob_path FROM pending_op
WHERE uid IN (SELECT uid FROM doomed) AND blob_path IS NOT NULL"
))?;
let rows = stmt.query_map(params![uid], |row| row.get::<_, String>(0))?;
rows.collect::<std::result::Result<Vec<_>, _>>()?
};
tx.execute(
&format!("{SUBTREE} DELETE FROM pending_op WHERE uid IN (SELECT uid FROM doomed)"),
params![uid],
)?;
tx.execute( tx.execute(
"INSERT INTO pending_op "INSERT INTO pending_op
(kind, uid, parent_uid, name, blob_path, meta_json, created_at, next_attempt_at) (kind, uid, parent_uid, name, blob_path, meta_json, created_at, next_attempt_at)
@@ -326,6 +372,67 @@ impl Db {
Ok(Some(AttachedBlob { id, superseded })) Ok(Some(AttachedBlob { id, superseded }))
} }
/// Retire a queued create that has just landed as `real`, and repoint
/// everything that named its placeholder, in one transaction.
///
/// The drain uploads the blob the create held when it was picked up. A
/// write that closes while that upload is on the wire attaches a newer blob
/// to the *same row* (`attach_blob_to_create`), so deleting the row by id
/// threw that write away: the server kept the older bytes and nothing was
/// left queued to replace them. When the row no longer holds `uploaded`, it
/// becomes a revision of `real` instead, with its sidecar passed through
/// `rewrite` so it names the real node. Returns that blob and sidecar, or
/// `None` when the create is simply done.
pub fn finish_create(
&self,
id: i64,
uploaded: Option<&str>,
local: &str,
real: &str,
rewrite: impl FnOnce(&str) -> Option<String>,
) -> Result<Option<(String, String)>> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
let row: Option<(Option<String>, Option<String>)> = tx
.query_row(
"SELECT blob_path, meta_json FROM pending_op WHERE id = ?1",
params![id],
|r| Ok((r.get(0)?, r.get(1)?)),
)
.optional()?;
let newer = match row {
Some((Some(blob), Some(meta))) if uploaded != Some(blob.as_str()) => {
rewrite(&meta).map(|meta| (blob, meta))
}
_ => None,
};
match &newer {
Some((_, meta)) => {
tx.execute(
"UPDATE pending_op
SET kind = ?2, uid = ?3, parent_uid = NULL, name = NULL, meta_json = ?4,
attempts = 0, last_error = NULL, next_attempt_at = 0
WHERE id = ?1",
params![id, OP_REVISION, real, meta],
)?;
}
None => {
tx.execute("DELETE FROM pending_op WHERE id = ?1", params![id])?;
}
}
tx.execute(
"UPDATE pending_op SET parent_uid = ?2 WHERE parent_uid = ?1",
params![local, real],
)?;
tx.execute(
"UPDATE nodes SET parent_uid = ?2 WHERE parent_uid = ?1",
params![local, real],
)?;
tx.execute("DELETE FROM nodes WHERE uid = ?1", params![local])?;
tx.commit()?;
Ok(newer)
}
/// Replace the sidecar of a queued op, for a baseline that has moved under /// Replace the sidecar of a queued op, for a baseline that has moved under
/// it. /// it.
/// ///
@@ -428,29 +535,12 @@ impl Db {
/// and in the user's pending count, for the life of the database. /// and in the user's pending count, for the life of the database.
/// ///
/// Only `create`/`mkdir` ops carry a `parent_uid`, so for a file this /// Only `create`/`mkdir` ops carry a `parent_uid`, so for a file this
/// recursion finds nothing and costs one query. /// recursion finds nothing and costs one query. Revisions of files the
/// node tree places beneath `uid` go too; see [`drop_doomed_ops`].
pub fn delete_ops_for_uid(&self, uid: &str) -> Result<Vec<String>> { pub fn delete_ops_for_uid(&self, uid: &str) -> Result<Vec<String>> {
const SUBTREE: &str = "
WITH RECURSIVE doomed(uid) AS (
SELECT ?1
UNION
SELECT p.uid FROM pending_op p JOIN doomed d ON p.parent_uid = d.uid
)";
let mut conn = self.conn.lock(); let mut conn = self.conn.lock();
let tx = conn.transaction()?; let tx = conn.transaction()?;
let blobs: Vec<String> = { let blobs = drop_doomed_ops(&tx, uid)?;
let mut stmt = tx.prepare(&format!(
"{SUBTREE}
SELECT blob_path FROM pending_op
WHERE uid IN (SELECT uid FROM doomed) AND blob_path IS NOT NULL"
))?;
let rows = stmt.query_map(params![uid], |r| r.get::<_, String>(0))?;
rows.collect::<std::result::Result<Vec<_>, _>>()?
};
tx.execute(
&format!("{SUBTREE} DELETE FROM pending_op WHERE uid IN (SELECT uid FROM doomed)"),
params![uid],
)?;
tx.commit()?; tx.commit()?;
Ok(blobs) Ok(blobs)
} }
@@ -765,6 +855,19 @@ impl Db {
/// drain replaces it with the uid the server assigns. /// drain replaces it with the uid the server assigns.
pub const LOCAL_VOLUME: &str = "local"; pub const LOCAL_VOLUME: &str = "local";
/// Whether op `id` is still queued. A drain worker that claimed it can find
/// it gone: the node was deleted, or a newer write superseded it, while the
/// worker was talking to the server.
pub fn op_exists(&self, id: i64) -> Result<bool> {
let conn = self.read();
let count: i64 = conn.query_row(
"SELECT COUNT(*) FROM pending_op WHERE id = ?1",
params![id],
|row| row.get(0),
)?;
Ok(count > 0)
}
/// Drop a queued op, once its upload has actually landed. /// Drop a queued op, once its upload has actually landed.
pub fn delete_op(&self, id: i64) -> Result<()> { pub fn delete_op(&self, id: i64) -> Result<()> {
let conn = self.conn.lock(); let conn = self.conn.lock();
+82
View File
@@ -260,6 +260,88 @@ impl Db {
Ok(()) Ok(())
} }
/// Drop the baseline rows of `rel_path` and everything under it.
///
/// Used when a subtree leaves a mirror folder through a move on Drive: its
/// local copy is about to be removed, and rows left behind would read as
/// "deleted locally" and trash the moved nodes at their new place.
pub fn sync_entries_remove_subtree(&self, folder_id: i64, rel_path: &str) -> Result<()> {
let conn = self.conn.lock();
conn.execute(
"DELETE FROM sync_entry
WHERE folder_id = ?1
AND (rel_path = ?2 OR substr(rel_path, 1, length(?2) + 1) = ?2 || '/')",
params![folder_id, rel_path],
)?;
Ok(())
}
/// Move the baseline rows of `from_rel` and everything under it to `to_rel`
/// in folder `to_folder`, in one transaction.
///
/// Used when a move on Drive took a mirror subtree along with its local
/// copy: the rows still describe both sides correctly, only under a new
/// path, so the next pass on either folder has nothing to do. The two
/// folders may be the same one.
pub fn sync_entries_move(
&self,
from_folder: i64,
from_rel: &str,
to_folder: i64,
to_rel: &str,
) -> Result<()> {
let mut conn = self.conn.lock();
let tx = conn.transaction()?;
{
let mut select = tx.prepare(
"SELECT rel_path, remote_uid, local_mtime, local_size, remote_rev, remote_hash,
local_mtime_ns
FROM sync_entry
WHERE folder_id = ?1
AND (rel_path = ?2 OR substr(rel_path, 1, length(?2) + 1) = ?2 || '/')",
)?;
let rows = select
.query_map(params![from_folder, from_rel], |r| {
Ok(StoredSyncEntry {
rel_path: r.get(0)?,
remote_uid: r.get(1)?,
local_mtime: r.get(2)?,
local_size: r.get(3)?,
remote_rev: r.get(4)?,
remote_hash: r.get(5)?,
local_mtime_ns: r.get(6)?,
})
})?
.collect::<rusqlite::Result<Vec<_>>>()?;
let mut delete =
tx.prepare("DELETE FROM sync_entry WHERE folder_id = ?1 AND rel_path = ?2")?;
for row in &rows {
delete.execute(params![from_folder, row.rel_path])?;
}
let mut insert = tx.prepare(
"INSERT OR REPLACE INTO sync_entry
(folder_id, rel_path, remote_uid, local_mtime, local_size, remote_rev,
remote_hash, local_mtime_ns)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
)?;
for row in rows {
let rel = format!("{to_rel}{}", &row.rel_path[from_rel.len()..]);
insert.execute(params![
to_folder,
rel,
row.remote_uid,
row.local_mtime,
row.local_size,
row.remote_rev,
row.remote_hash,
row.local_mtime_ns,
])?;
}
}
tx.commit()?;
Ok(())
}
/// Drop the entire baseline for a folder. Used when flipping ondemand→mirror: /// Drop the entire baseline for a folder. Used when flipping ondemand→mirror:
/// the local tree was evicted, so the old baseline is stale and would make the /// the local tree was evicted, so the old baseline is stale and would make the
/// next reconcile mistake "locally deleted" for "must re-download". Clearing it /// next reconcile mistake "locally deleted" for "must re-download". Clearing it
+287
View File
@@ -788,6 +788,8 @@ fn children_if_listed_gated_on_flag() {
// Not listed yet → unknown, force a re-fetch. // Not listed yet → unknown, force a re-fetch.
assert!(db.children_if_listed(&uid("root")).unwrap().is_none()); assert!(db.children_if_listed(&uid("root")).unwrap().is_none());
// Still known, for a caller that has nothing better while offline.
assert_eq!(db.known_children(&uid("root")).unwrap().len(), 2);
db.set_listed(&uid("root"), true).unwrap(); db.set_listed(&uid("root"), true).unwrap();
let kids = db.children_if_listed(&uid("root")).unwrap().unwrap(); let kids = db.children_if_listed(&uid("root")).unwrap().unwrap();
@@ -2458,6 +2460,20 @@ fn atomic_trash_replacement_returns_owned_blobs_after_commit() {
assert!(ops[0].blob_path.is_none()); assert!(ops[0].blob_path.is_none());
} }
#[test]
fn a_queued_trash_holds_its_name_until_it_lands() {
let db = Db::open_in_memory().unwrap();
let node = uid("held-name").to_string();
assert!(!db.has_pending_trash_named("report.txt").unwrap());
let (trash_id, _) = db.replace_ops_with_trash(&node, "report.txt", 1).unwrap();
assert!(db.has_pending_trash_named("report.txt").unwrap());
assert!(!db.has_pending_trash_named("other.txt").unwrap());
db.delete_op(trash_id).unwrap();
assert!(!db.has_pending_trash_named("report.txt").unwrap());
}
#[test] #[test]
fn failed_atomic_trash_insert_keeps_prior_revision_and_blob_ownership() { fn failed_atomic_trash_insert_keeps_prior_revision_and_blob_ownership() {
let db = Db::open_in_memory().unwrap(); let db = Db::open_in_memory().unwrap();
@@ -2498,6 +2514,71 @@ fn failed_atomic_trash_insert_keeps_prior_revision_and_blob_ownership() {
); );
} }
/// A revision op names no parent, so the queue alone cannot see that its file
/// sits inside a folder being removed. The node tree can: trashing the folder
/// has to take the uploads of the files below it along, or they fail forever
/// against a trashed node and keep their bytes on disk (B102). A rename that
/// moved a node into the folder still has to land.
#[test]
fn trashing_a_folder_drops_the_queued_revisions_of_files_below_it() {
let db = Db::open_in_memory().unwrap();
db.upsert_nodes(&[
folder("root", None, "My Files"),
folder("doomed", Some("root"), "doomed"),
folder("sub", Some("doomed"), "sub"),
file("deep", "sub", "deep.bin", 1),
file("moved", "doomed", "moved.txt", 1),
file("kept", "root", "kept.bin", 1),
])
.unwrap();
let op = |kind: &str, link: &str, blob: Option<&str>| PendingOp {
id: 0,
kind: kind.to_string(),
uid: uid(link).to_string(),
parent_uid: None,
name: None,
blob_path: blob.map(str::to_string),
meta_json: Some("{}".to_string()),
created_at: 1,
attempts: 0,
last_error: None,
next_attempt_at: 0,
};
db.enqueue_op(&op(OP_REVISION, "deep", Some("/staging/deep")))
.unwrap();
db.enqueue_op(&op(OP_RENAME, "moved", None)).unwrap();
db.enqueue_op(&op(OP_REVISION, "kept", Some("/staging/kept")))
.unwrap();
let doomed = uid("doomed").to_string();
let (_, blobs) = db.replace_ops_with_trash(&doomed, "doomed", 2).unwrap();
assert_eq!(blobs, vec!["/staging/deep"]);
let mut left: Vec<(String, String)> = db
.pending_ops()
.unwrap()
.into_iter()
.map(|op| (op.kind, op.uid))
.collect();
left.sort();
assert_eq!(
left,
vec![
(OP_RENAME.to_string(), uid("moved").to_string()),
(OP_REVISION.to_string(), uid("kept").to_string()),
(OP_TRASH.to_string(), doomed.clone()),
]
);
// Deleting outright follows the same tree.
db.enqueue_op(&op(OP_REVISION, "deep", Some("/staging/deep2")))
.unwrap();
assert_eq!(
db.delete_ops_for_uid(&uid("sub").to_string()).unwrap(),
vec!["/staging/deep2"]
);
}
/// Deleting a folder that was created offline must take the ops queued /// Deleting a folder that was created offline must take the ops queued
/// underneath it with it: they name a placeholder parent that will now never /// underneath it with it: they name a placeholder parent that will now never
/// become real, so nothing could ever drain them and nothing is left to /// become real, so nothing could ever drain them and nothing is left to
@@ -2699,6 +2780,99 @@ fn attaching_to_an_already_drained_create_reports_it_is_gone() {
assert!(out.is_none()); assert!(out.is_none());
} }
#[test]
fn a_write_attached_while_its_create_uploads_becomes_a_revision() {
let db = Db::open_in_memory().unwrap();
let local = "local~abc";
let real = uid("real").to_string();
let create = |blob: Option<&str>| PendingOp {
id: 0,
kind: OP_CREATE.to_string(),
uid: local.to_string(),
parent_uid: Some(uid("parent").to_string()),
name: Some("new.txt".to_string()),
blob_path: blob.map(str::to_string),
meta_json: blob.map(|_| "{\"uid\":\"local~abc\"}".to_string()),
created_at: 1,
attempts: 0,
last_error: None,
next_attempt_at: 0,
};
let (id, _) = db.enqueue_op(&create(None)).unwrap();
// The drain picked the create up with no blob; the file's first write
// closes while the empty upload is on the wire.
db.attach_blob_to_create(local, "/staging/written", "{\"uid\":\"local~abc\"}")
.unwrap()
.expect("create is still queued");
let newer = db
.finish_create(id, None, local, &real, |json| {
Some(json.replace("local~abc", "vol~real"))
})
.unwrap();
assert_eq!(
newer,
Some((
"/staging/written".to_string(),
"{\"uid\":\"vol~real\"}".to_string()
))
);
let ops = db.pending_ops().unwrap();
assert_eq!(ops.len(), 1, "the write is still queued");
assert_eq!(ops[0].kind, OP_REVISION);
assert_eq!(ops[0].uid, real);
assert_eq!(ops[0].blob_path.as_deref(), Some("/staging/written"));
assert_eq!(ops[0].parent_uid, None);
}
#[test]
fn a_create_whose_blob_landed_is_retired() {
let db = Db::open_in_memory().unwrap();
let local = "local~abc";
let real = uid("real").to_string();
let (id, _) = db
.enqueue_op(&PendingOp {
id: 0,
kind: OP_CREATE.to_string(),
uid: local.to_string(),
parent_uid: Some(uid("parent").to_string()),
name: Some("new.txt".to_string()),
blob_path: Some("/staging/sent".to_string()),
meta_json: Some("{}".to_string()),
created_at: 1,
attempts: 0,
last_error: None,
next_attempt_at: 0,
})
.unwrap();
db.enqueue_op(&PendingOp {
id: 0,
kind: OP_CREATE.to_string(),
uid: "local~child".to_string(),
parent_uid: Some(local.to_string()),
name: Some("inside.txt".to_string()),
blob_path: None,
meta_json: None,
created_at: 2,
attempts: 0,
last_error: None,
next_attempt_at: 0,
})
.unwrap();
let newer = db
.finish_create(id, Some("/staging/sent"), local, &real, |_| {
panic!("an unchanged create has no sidecar to rewrite")
})
.unwrap();
assert_eq!(newer, None);
let ops = db.pending_ops().unwrap();
assert_eq!(ops.len(), 1, "only the child is left");
assert_eq!(ops[0].parent_uid.as_deref(), Some(real.as_str()));
}
#[test] #[test]
fn draining_a_folder_repoints_its_queued_children() { fn draining_a_folder_repoints_its_queued_children() {
let db = Db::open_in_memory().unwrap(); let db = Db::open_in_memory().unwrap();
@@ -2920,6 +3094,94 @@ fn clearing_a_stale_pending_mode_preserves_the_newer_intent() {
assert_eq!(db.sync_folder_get(id).unwrap().unwrap().pending_mode, None); assert_eq!(db.sync_folder_get(id).unwrap().unwrap().pending_mode, None);
} }
fn baseline_row(rel: &str) -> StoredSyncEntry {
StoredSyncEntry {
rel_path: rel.into(),
remote_uid: Some(format!("vol~{rel}")),
local_mtime: 1_700_000_000,
local_mtime_ns: Some(1_700_000_000_000_000_001),
local_size: 7,
remote_rev: Some("1700000000".into()),
remote_hash: Some("7".into()),
}
}
#[test]
fn moving_a_baseline_subtree_takes_its_rows_and_leaves_name_prefixed_siblings() {
let db = Db::open_in_memory().unwrap();
let from = db.sync_folder_add("/home/me/A", "vol~a", "share").unwrap();
let to = db.sync_folder_add("/home/me/B", "vol~b", "share").unwrap();
for rel in [
"docs",
"docs/x.txt",
"docs/sub/y.txt",
"docsextra.txt",
"other",
] {
db.sync_entry_upsert(from, &baseline_row(rel)).unwrap();
}
db.sync_entry_upsert(to, &baseline_row("inbox")).unwrap();
db.sync_entries_move(from, "docs", to, "inbox/docs")
.unwrap();
let left = db.sync_entries(from).unwrap();
let mut left: Vec<_> = left.keys().cloned().collect();
left.sort();
assert_eq!(left, ["docsextra.txt", "other"]);
let moved = db.sync_entries(to).unwrap();
let mut keys: Vec<_> = moved.keys().cloned().collect();
keys.sort();
assert_eq!(
keys,
[
"inbox",
"inbox/docs",
"inbox/docs/sub/y.txt",
"inbox/docs/x.txt"
]
);
// Only the path changes: the row still describes the same node on both sides.
assert_eq!(
moved["inbox/docs/x.txt"],
StoredSyncEntry {
rel_path: "inbox/docs/x.txt".into(),
..baseline_row("docs/x.txt")
}
);
}
#[test]
fn moving_a_baseline_subtree_within_one_folder_renames_it() {
let db = Db::open_in_memory().unwrap();
let folder = db.sync_folder_add("/home/me/A", "vol~a", "share").unwrap();
for rel in ["old", "old/f.txt", "new"] {
db.sync_entry_upsert(folder, &baseline_row(rel)).unwrap();
}
db.sync_entries_move(folder, "old", folder, "new/old")
.unwrap();
let mut keys: Vec<_> = db.sync_entries(folder).unwrap().into_keys().collect();
keys.sort();
assert_eq!(keys, ["new", "new/old", "new/old/f.txt"]);
}
#[test]
fn removing_a_baseline_subtree_keeps_name_prefixed_siblings() {
let db = Db::open_in_memory().unwrap();
let folder = db.sync_folder_add("/home/me/A", "vol~a", "share").unwrap();
for rel in ["docs", "docs/x.txt", "docsextra.txt"] {
db.sync_entry_upsert(folder, &baseline_row(rel)).unwrap();
}
db.sync_entries_remove_subtree(folder, "docs").unwrap();
let keys: Vec<_> = db.sync_entries(folder).unwrap().into_keys().collect();
assert_eq!(keys, ["docsextra.txt"]);
}
#[test] #[test]
fn mirror_uid_lookup_covers_the_root_and_synced_descendants_only() { fn mirror_uid_lookup_covers_the_root_and_synced_descendants_only() {
let db = Db::open_in_memory().unwrap(); let db = Db::open_in_memory().unwrap();
@@ -4963,3 +5225,28 @@ fn retry_all_touches_only_ops_that_have_failed() {
assert_eq!(due(debounced), 7_000); assert_eq!(due(debounced), 7_000);
assert_eq!(due(parked), ops::PARK_UNTIL); assert_eq!(due(parked), ops::PARK_UNTIL);
} }
#[test]
fn a_queued_op_stops_existing_once_the_node_is_deleted() {
let db = Db::open_in_memory().unwrap();
let (id, _) = db
.enqueue_op(&PendingOp {
id: 0,
kind: OP_REVISION.to_string(),
uid: uid("journal").to_string(),
parent_uid: None,
name: None,
blob_path: Some("/staging/journal".to_string()),
meta_json: Some("{}".to_string()),
created_at: 1,
attempts: 0,
last_error: None,
next_attempt_at: 0,
})
.unwrap();
assert!(db.op_exists(id).unwrap());
let blobs = db.delete_ops_for_uid(&uid("journal").to_string()).unwrap();
assert_eq!(blobs, vec!["/staging/journal"]);
assert!(!db.op_exists(id).unwrap());
}
+10 -3
View File
@@ -55,9 +55,10 @@ fn systemctl_query(args: &[&str]) -> bool {
/// keyring checks). `--now` is a no-op on an active unit, so the daemon would /// keyring checks). `--now` is a no-op on an active unit, so the daemon would
/// keep sleeping and the mount would not appear until the next poll fired. /// keep sleeping and the mount would not appear until the next poll fired.
/// Restarting makes the new session take effect immediately, and starts the /// Restarting makes the new session take effect immediately, and starts the
/// unit when it is not running at all. /// unit when it is not running at all. `reenable` for the reason given on
/// [`enable`].
pub fn enable_start() -> bool { pub fn enable_start() -> bool {
let enabled = systemctl(&["enable"]); let enabled = systemctl(&["reenable"]);
systemctl(&["restart"]) && enabled systemctl(&["restart"]) && enabled
} }
@@ -97,8 +98,14 @@ pub fn is_enabled() -> bool {
/// Enable the service so it auto-starts on future logins, without necessarily /// Enable the service so it auto-starts on future logins, without necessarily
/// starting it now beyond what `enable` does. Used by the Settings "Start on /// starting it now beyond what `enable` does. Used by the Settings "Start on
/// login" switch; pairs with [`disable`](disable). /// login" switch; pairs with [`disable`](disable).
///
/// `reenable` rather than `enable`: older installs were enabled
/// into `default.target.wants`, and a plain `enable` would add the new
/// `graphical-session.target.wants` link while leaving the old one to start the
/// unit too early. `reenable` drops every existing link and recreates them from
/// the current `[Install]` section.
pub fn enable() -> bool { pub fn enable() -> bool {
systemctl(&["enable"]) systemctl(&["reenable"])
} }
/// Disable auto-start without stopping the running mount. Used by the Settings /// Disable auto-start without stopping the running mount. Used by the Settings
+7 -5
View File
@@ -9,7 +9,7 @@ use crate::shutdown::Shutdown;
/// ///
/// Returns whether this mount actually held the node, so the caller can evict /// Returns whether this mount actually held the node, so the caller can evict
/// its content blob once if *any* mount did. /// its content blob once if *any* mount did.
fn forget_and_notify(st: &mut State, notify: &mut NotifyBatch, uid: &NodeUid) -> bool { pub(crate) fn forget_and_notify(st: &mut State, notify: &mut NotifyBatch, uid: &NodeUid) -> bool {
// Capture the inode before `forget` clears the uid mapping. // Capture the inode before `forget` clears the uid mapping.
let child = st.by_uid.get(uid).copied(); let child = st.by_uid.get(uid).copied();
let Some((parent, name)) = st.forget(uid) else { let Some((parent, name)) = st.forget(uid) else {
@@ -378,9 +378,10 @@ pub(super) async fn run_event_sync(
info!(?cursor, "event sync started"); info!(?cursor, "event sync started");
loop { loop {
tokio::time::sleep(POLL_INTERVAL).await; core.next_poll(POLL_INTERVAL).await;
let events = match client.enumerate_events(&scope, cursor.as_ref()).await { let events = match link::bounded(client.enumerate_events(&scope, cursor.as_ref())).await {
Ok(events) => events, Ok(events) => events,
Err(e) if core.lost_link(&e, "poll events") => continue,
Err(e) => { Err(e) => {
warn!(error = %e, "event poll failed; retrying after interval"); warn!(error = %e, "event poll failed; retrying after interval");
continue; continue;
@@ -471,9 +472,10 @@ pub(super) async fn run_photos_event_sync(core: Core) {
info!(?cursor, "photo event sync started"); info!(?cursor, "photo event sync started");
loop { loop {
tokio::time::sleep(POLL_INTERVAL).await; core.next_poll(POLL_INTERVAL).await;
let events = match photos.enumerate_events(&scope, cursor.as_ref()).await { let events = match link::bounded(photos.enumerate_events(&scope, cursor.as_ref())).await {
Ok(events) => events, Ok(events) => events,
Err(error) if core.lost_link(&error, "poll photo events") => continue,
Err(error) => { Err(error) => {
warn!(%error, "photo event poll failed; retrying after interval"); warn!(%error, "photo event poll failed; retrying after interval");
continue; continue;
+18
View File
@@ -974,6 +974,24 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
}, },
Err(e) => CtlResponse::error(e), Err(e) => CtlResponse::error(e),
}, },
// Two absolute paths may name two different locations, which only
// `move_between` can resolve; a relative pair keeps the old meaning.
Ok(CtlRequest::Move { path, new_parent })
if Path::new(&path).is_absolute() && Path::new(&new_parent).is_absolute() =>
{
match core.move_between(Path::new(&path), Path::new(&new_parent)) {
Ok(name) => {
core.log_activity(ActivityKind::Move, &name, format!("to {new_parent}"), true);
CtlResponse::Ok {
message: format!("moved {name}"),
}
}
Err(e) => {
core.log_activity(ActivityKind::Move, &path, &e, false);
CtlResponse::error(e)
}
}
}
Ok(CtlRequest::Move { path, new_parent }) => { Ok(CtlRequest::Move { path, new_parent }) => {
match ( match (
rel_to_mount(mountpoint, &path), rel_to_mount(mountpoint, &path),
+218 -15
View File
@@ -38,10 +38,10 @@ use pdfs_core::db::{
}; };
use proton_drive_rs::proton_sdk::error::ProtonError; use proton_drive_rs::proton_sdk::error::ProtonError;
use proton_drive_rs::proton_sdk::ids::NodeUid; use proton_drive_rs::proton_sdk::ids::NodeUid;
use proton_drive_rs::{Node, NodeMoveItem}; use proton_drive_rs::{Node, NodeKind, NodeMoveItem};
use tracing::{debug, error, info, warn}; use tracing::{debug, error, info, warn};
use super::state::Intervals; use super::state::{Intervals, PendingRevision};
use super::transfers::CountingReader; use super::transfers::CountingReader;
use super::{ use super::{
Core, DRAIN_BACKOFF_MAX, DRAIN_BACKOFF_MIN, DRAIN_IDLE_POLL, DRAIN_REVISION_DEBOUNCE, Core, DRAIN_BACKOFF_MAX, DRAIN_BACKOFF_MIN, DRAIN_IDLE_POLL, DRAIN_REVISION_DEBOUNCE,
@@ -673,6 +673,9 @@ impl Core {
// *different* name is the non-destructive resolution: it neither // *different* name is the non-destructive resolution: it neither
// clobbers their file nor drops ours, and it is visible. // clobbers their file nor drops ours, and it is visible.
Err(e) if is_already_exists(&e) => { Err(e) if is_already_exists(&e) => {
if self.db.has_pending_trash_named(&name)? {
return Err(held_by_queued_trash(&name));
}
let alt = conflict_name(&name, now_secs()); let alt = conflict_name(&name, now_secs());
warn!(%uid, name, alt, "destination already holds that name; using a conflict name"); warn!(%uid, name, alt, "destination already holds that name; using a conflict name");
self.move_renaming(&uid, &parent, Some(&alt))?; self.move_renaming(&uid, &parent, Some(&alt))?;
@@ -696,8 +699,8 @@ impl Core {
let landed = match self.drain_rename_in_place(&uid, &node.name, &name) { let landed = match self.drain_rename_in_place(&uid, &node.name, &name) {
Ok(landed) => landed, Ok(landed) => landed,
// The node went with it; nothing is left to rename. // The node went with it; nothing is left to rename.
Err(e) if is_gone(&e) => node.name.clone(), Err(e) if is_gone(e.as_ref()) => node.name.clone(),
Err(e) => return Err(e.into()), Err(e) => return Err(e),
}; };
self.log_activity( self.log_activity(
ActivityKind::Rename, ActivityKind::Rename,
@@ -725,16 +728,19 @@ impl Core {
uid: &NodeUid, uid: &NodeUid,
current: &str, current: &str,
name: &str, name: &str,
) -> Result<String, ProtonError> { ) -> Result<String, Box<dyn std::error::Error>> {
if current == name { if current == name {
return Ok(current.to_string()); return Ok(current.to_string());
} }
match self.rt.block_on(self.client.rename_node(uid, name, None)) { match self.rename_remote(uid, name) {
Ok(()) => Ok(name.to_string()), Ok(()) => Ok(name.to_string()),
Err(e) if is_already_exists(&e) => { Err(e) if is_already_exists(&e) => {
if self.db.has_pending_trash_named(name)? {
return Err(held_by_queued_trash(name));
}
let alt = conflict_name(name, now_secs()); let alt = conflict_name(name, now_secs());
warn!(%uid, name, alt, "rename target name is taken; using a conflict name"); warn!(%uid, name, alt, "rename target name is taken; using a conflict name");
self.rt.block_on(self.client.rename_node(uid, &alt, None))?; self.rename_remote(uid, &alt)?;
self.adopt_drained_name(uid, &alt); self.adopt_drained_name(uid, &alt);
self.log_activity( self.log_activity(
ActivityKind::Rename, ActivityKind::Rename,
@@ -744,7 +750,7 @@ impl Core {
); );
Ok(alt) Ok(alt)
} }
Err(e) => Err(e), Err(e) => Err(e.into()),
} }
} }
@@ -842,8 +848,26 @@ impl Core {
// reachable only because the op waited, which is the whole point of the // reachable only because the op waited, which is the whole point of the
// queue. Never overwrite theirs and never drop ours: land under a // queue. Never overwrite theirs and never drop ours: land under a
// conflict name, exactly as the sync engine does. // conflict name, exactly as the sync engine does.
//
// Two things can hold the name that are not someone else's file. The
// node a create made before the network dropped its answer is ours: the
// create was queued because nobody heard back, so adopt it. And a node
// whose trash is still queued is about to let go of it.
let mut name = wanted.clone(); let mut name = wanted.clone();
let mut adopted = false;
let mut real = self.create_drained_node(op, &parent, &name); let mut real = self.create_drained_node(op, &parent, &name);
if real.as_ref().is_err_and(|e| is_already_exists(e.as_ref()))
&& let Some(twin) = self.adoptable_twin(op, &parent, &wanted)?
{
info!(%local, %twin, wanted, "the name is held by our own unanswered create; adopting it");
real = Ok(twin);
adopted = true;
}
if real.as_ref().is_err_and(|e| is_already_exists(e.as_ref()))
&& self.db.has_pending_trash_named(&wanted)?
{
return Err(held_by_queued_trash(&wanted));
}
if real.as_ref().is_err_and(|e| is_already_exists(e.as_ref())) { if real.as_ref().is_err_and(|e| is_already_exists(e.as_ref())) {
name = conflict_name(&wanted, now_secs()); name = conflict_name(&wanted, now_secs());
warn!(%local, wanted, name, "name is taken remotely; creating under a conflict name"); warn!(%local, wanted, name, "name is taken remotely; creating under a conflict name");
@@ -886,20 +910,62 @@ impl Core {
// Retire the op before touching anything else: if we crash here the node // Retire the op before touching anything else: if we crash here the node
// exists remotely and the local placeholder is reconciled by the event // exists remotely and the local placeholder is reconciled by the event
// sync, whereas a surviving op would create the file a second time. // sync, whereas a surviving op would create the file a second time.
self.db.delete_op(op.id)?; // An adopted node was made empty, so a blob this op carries has not been
// uploaded: it goes on as a revision of the node instead.
let uploaded = (!adopted).then_some(op.blob_path.as_deref()).flatten();
let newer = self.retire_create(op, uploaded, &local, &real)?;
self.adopt_real_uid(&local, &real)?; self.adopt_real_uid(&local, &real)?;
// The feed will report this create back to us; the tree already has it // The feed will report this create back to us; the tree already has it
// under its real uid, so that event is ours to ignore (`Core::self_changes`). // under its real uid, so that event is ours to ignore (`Core::self_changes`).
self.note_self_change(&real); self.note_self_change(&real);
if let Some(blob) = op.blob_path.as_deref() { // The uploaded blob is done with either way: a write that replaced it
// while it was on the wire already discarded it when it attached.
if let Some(blob) = uploaded {
self.cache.discard_staged(Path::new(blob)); self.cache.discard_staged(Path::new(blob));
} }
self.pending.lock().remove(&local); if newer {
info!(%local, %real, name, "a write landed during the create; queued it as a revision");
self.wake_drain();
}
self.log_activity(ActivityKind::Upload, &name, "created", true); self.log_activity(ActivityKind::Upload, &name, "created", true);
info!(%local, %real, name, kind = %op.kind, "pending create landed"); info!(%local, %real, name, kind = %op.kind, "pending create landed");
Ok(()) Ok(())
} }
/// The node already holding `name` under `parent`, if it is the one an
/// earlier attempt at this op made before its answer was lost.
///
/// A create that timed out may still have landed; the op was queued because
/// the caller never heard back. Without this, its replay forked a
/// `(sync-conflict)` copy of a file nobody else had touched.
fn adoptable_twin(
&self,
op: &PendingOp,
parent: &NodeUid,
name: &str,
) -> Result<Option<NodeUid>, Box<dyn std::error::Error>> {
let uids = self
.rt
.block_on(self.client.enumerate_folder_children_node_uids(parent))?;
let children = self.rt.block_on(self.client.enumerate_nodes_light(&uids))?;
let Some(twin) = children
.into_iter()
.find(|node| node.name == name && !node.trashed)
else {
return Ok(None);
};
// The light listing carries no file size, and an adoptable file is
// exactly an empty one.
let twin = match twin.kind {
NodeKind::Folder => twin,
NodeKind::File { .. } => match self.fetch_node_remote(&twin.uid)? {
Some(node) => node,
None => return Ok(None),
},
};
Ok(adoptable(op.kind == OP_MKDIR, &twin, op.created_at).then_some(twin.uid))
}
/// Make one queued `create`/`mkdir` real under a given name, and hand back /// Make one queued `create`/`mkdir` real under a given name, and hand back
/// the API's own error so the caller can tell a name clash from a failure. /// the API's own error so the caller can tell a name clash from a failure.
/// ///
@@ -963,6 +1029,62 @@ impl Core {
Ok(uid) Ok(uid)
} }
/// Retire a landed create's op and move its queued state to the real uid.
/// True when a newer write attached to the create while it was uploading,
/// which is now queued as a revision of `real`.
///
/// Runs under the `pending` lock, which `Core::enqueue_staged_write` also
/// holds from attaching a blob to a create until it records the pending
/// entry. So a write either attached before this ran and is carried over,
/// or finds the create gone and queues against the real uid itself. The
/// open write handles are repointed right after, for the same reason:
/// a handle released later must not queue against the placeholder.
fn retire_create(
&self,
op: &PendingOp,
uploaded: Option<&str>,
local: &NodeUid,
real: &NodeUid,
) -> Result<bool, Box<dyn std::error::Error>> {
let real_str = real.to_string();
let newer = {
let mut pending = self.pending.lock();
let newer =
self.db
.finish_create(op.id, uploaded, &local.to_string(), &real_str, |json| {
let mut meta: StagedWrite = serde_json::from_str(json).ok()?;
meta.uid = real_str.clone();
serde_json::to_string(&meta).ok()
})?;
pending.remove(local);
if let Some((blob, json)) = &newer {
let meta: StagedWrite = serde_json::from_str(json)?;
pending.insert(
real.clone(),
PendingRevision {
path: PathBuf::from(blob),
meta,
},
);
}
newer.is_some()
};
self.for_each_state(|st| {
if let Some(ino) = st.by_uid.remove(local) {
st.by_uid.insert(real.clone(), ino);
if let Some(e) = st.entries.get_mut(&ino) {
e.uid = real.clone();
}
}
for aw in st.active_writes.values_mut() {
if aw.uid == *local {
aw.uid = real.clone();
}
}
});
Ok(newer)
}
/// Swap a placeholder uid for the real one across everything that keyed off /// Swap a placeholder uid for the real one across everything that keyed off
/// it: queued children, the DB, the in-memory tree, and the caches. /// it: queued children, the DB, the in-memory tree, and the caches.
/// ///
@@ -985,17 +1107,22 @@ impl Core {
.fetch_node(real) .fetch_node(real)
.map_err(|e| self.errno_error(e, "fetch node"))?; .map_err(|e| self.errno_error(e, "fetch node"))?;
// Repoints queued children and node rows, and drops the placeholder row. // Repoints queued children and node rows, and drops the placeholder row.
// A no-op after `retire_create`, which has already done both.
self.db self.db
.remap_local_uid(&local.to_string(), &real.to_string())?; .remap_local_uid(&local.to_string(), &real.to_string())?;
self.for_each_state(|st| { self.for_each_state(|st| {
let Some(ino) = st.by_uid.remove(local) else { if let Some(ino) = st.by_uid.remove(local) {
st.by_uid.insert(real.clone(), ino);
}
let Some(&ino) = st.by_uid.get(real) else {
return; return;
}; };
st.by_uid.insert(real.clone(), ino); let mut landed = node.clone();
st.keep_open_write_size(ino, &mut landed);
if let Some(e) = st.entries.get_mut(&ino) { if let Some(e) = st.entries.get_mut(&ino) {
e.uid = real.clone(); e.uid = real.clone();
e.node = node.clone(); e.node = landed;
} }
// Where the op said the node goes and where it actually landed can // Where the op said the node goes and where it actually landed can
// differ — a conflict re-homes it — so the tree follows the parent // differ — a conflict re-homes it — so the tree follows the parent
@@ -1024,6 +1151,9 @@ impl Core {
if let Err(e) = self.db.upsert_node(&node) { if let Err(e) = self.db.upsert_node(&node) {
warn!(%real, error = %e, "db upsert_node failed after remap"); warn!(%real, error = %e, "db upsert_node failed after remap");
} }
// A write carried over from the create was made against no revision at
// all; the one just created is what it now replaces.
self.rebaseline_pending(real, &node);
if node.is_folder() { if node.is_folder() {
// It was recorded as listed while local (it was empty and had nothing // It was recorded as listed while local (it was empty and had nothing
// to enumerate). That still holds: its queued children re-intern under // to enumerate). That still holds: its queued children re-intern under
@@ -1407,6 +1537,7 @@ impl Core {
/// Upload a staged revision of a file the server already knows about. /// Upload a staged revision of a file the server already knows about.
pub(crate) fn drain_revision(&self, op: &PendingOp) -> Result<(), Box<dyn std::error::Error>> { pub(crate) fn drain_revision(&self, op: &PendingOp) -> Result<(), Box<dyn std::error::Error>> {
let op_started = Instant::now();
let blob = op let blob = op
.blob_path .blob_path
.clone() .clone()
@@ -1416,6 +1547,16 @@ impl Core {
let uid = parse_node_uid(&meta.uid).ok_or("staged write has an unparseable uid")?; let uid = parse_node_uid(&meta.uid).ok_or("staged write has an unparseable uid")?;
if let Some(reason) = self.revision_conflict(&uid, &meta)? { if let Some(reason) = self.revision_conflict(&uid, &meta)? {
// Trashing the file from the mount drops its queued write, but not
// from under a worker already holding it. That worker then finds the
// node trashed — by us — and would upload the withdrawn bytes as a
// conflict copy (a `recovered-…` file in the root when the tree has
// already forgotten the node). A write nobody queues any more is
// not a conflict.
if !self.db.op_exists(op.id)? {
debug!(%uid, reason, "queued write withdrawn while it drained; dropping it");
return Ok(());
}
return self.keep_as_conflict_copy(op, &blob, &meta, &uid, &reason); return self.keep_as_conflict_copy(op, &blob, &meta, &uid, &reason);
} }
@@ -1452,6 +1593,12 @@ impl Core {
// this is on the wire can stop it rather than paying for a revision the // this is on the wire can stop it rather than paying for a revision the
// op it queued will immediately replace. // op it queued will immediately replace.
let cancel = self.begin_cancellable_upload(&uid); let cancel = self.begin_cancellable_upload(&uid);
// An unlink that withdrew this op before the flag above existed had no
// upload to cancel; the row is the record that it happened.
if !self.db.op_exists(op.id)? {
debug!(%uid, "queued write withdrawn before it went out; dropping it");
return Ok(());
}
let reader = CountingReader::new(File::open(&blob)?, &guard).with_cancel(cancel.flag()); let reader = CountingReader::new(File::open(&blob)?, &guard).with_cancel(cancel.flag());
let started = Instant::now(); let started = Instant::now();
let sent = self.rt.block_on(self.client.upload_new_revision_from( let sent = self.rt.block_on(self.client.upload_new_revision_from(
@@ -1498,7 +1645,15 @@ impl Core {
self.evict_reader(&uid); self.evict_reader(&uid);
self.refresh_after_upload(&uid); self.refresh_after_upload(&uid);
self.log_activity(ActivityKind::Upload, &name, "uploaded", true); self.log_activity(ActivityKind::Upload, &name, "uploaded", true);
info!(%uid, len = meta.len, "pending upload landed"); // Both times, so a slow drain shows whether the upload itself or the
// checks and gap-fill around it are where the time goes.
info!(
%uid,
len = meta.len,
upload_ms = took.as_millis() as u64,
total_ms = op_started.elapsed().as_millis() as u64,
"pending upload landed"
);
Ok(()) Ok(())
} }
@@ -1708,6 +1863,31 @@ fn park_verdict(created_at: i64, now: i64, open: bool) -> ParkVerdict {
ParkVerdict::Release ParkVerdict::Release
} }
/// The retryable error for an op whose name a queued trash has yet to free.
fn held_by_queued_trash(name: &str) -> Box<dyn std::error::Error> {
format!("{name} is still held by a node whose trash is queued").into()
}
/// How much older than its op a remote node may be and still be the one an
/// unanswered call made. The op is queued the moment the call gives up, so the
/// node is at most one call's deadline older; the rest is clock skew between
/// this machine and Proton.
const ADOPT_WINDOW_MS: i64 = 2 * 60 * 1000;
/// Whether `twin`, found holding the name a queued create wants, is the node an
/// earlier, unanswered attempt at that create made: the same kind, created
/// around when the op was queued, and for a file still empty, as the mount's
/// create leaves it.
fn adoptable(is_dir: bool, twin: &Node, op_created_ms: i64) -> bool {
if twin.creation_time.saturating_mul(1000) < op_created_ms - ADOPT_WINDOW_MS {
return false;
}
match &twin.kind {
NodeKind::Folder => is_dir,
NodeKind::File { claimed_size, .. } => !is_dir && *claimed_size == Some(0),
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -2087,6 +2267,29 @@ mod tests {
} }
} }
#[test]
fn only_a_fresh_empty_twin_of_the_same_kind_is_adopted() {
// The op was queued at t=1000 s, after a create that got no answer.
let queued = 1_000_000;
let made = |mut node: Node, secs: i64| {
node.creation_time = secs;
node
};
assert!(adoptable(false, &made(file_node(0, 0, None), 990), queued));
assert!(adoptable(true, &made(folder_node("d", None), 990), queued));
// Made long before the op: someone else's file, not our lost create.
assert!(!adoptable(false, &made(file_node(0, 0, None), 800), queued));
// Our create leaves a file empty; one with content is someone else's.
assert!(!adoptable(false, &made(file_node(0, 5, None), 990), queued));
// The kind has to match what the op makes.
assert!(!adoptable(true, &made(file_node(0, 0, None), 990), queued));
assert!(!adoptable(
false,
&made(folder_node("d", None), 990),
queued
));
}
fn baseline(mtime: i64, size: u64, rev: Option<&str>) -> Baseline { fn baseline(mtime: i64, size: u64, rev: Option<&str>) -> Baseline {
Baseline { Baseline {
mtime, mtime,
+154 -62
View File
@@ -290,7 +290,7 @@ impl Filesystem for ProtonFs {
} }
return; return;
} }
let (uid, mtime, fsize, is_video) = { let (uid, mtime, fsize, provisional, is_video) = {
let st = self.core.state(); let st = self.core.state();
match st.entries.get(&ino.0) { match st.entries.get(&ino.0) {
Some(e) if e.node.is_file() => { Some(e) if e.node.is_file() => {
@@ -300,14 +300,24 @@ impl Filesystem for ProtonFs {
}; };
let is_video = let is_video =
PhotoKind::classify(Some(&e.node.name), media_type) == PhotoKind::Video; PhotoKind::classify(Some(&e.node.name), media_type) == PhotoKind::Video;
let fsize = self let queued = self.core.pending.lock().get(&e.uid).map(|p| p.meta.len);
.core // Still the cheap enumeration's ciphertext size (B12).
.pending let provisional = queued.is_none()
.lock() && matches!(
.get(&e.uid) &e.node.kind,
.map(|p| p.meta.len) NodeKind::File {
.unwrap_or_else(|| node_size(&e.node)); claimed_size: None,
(e.uid.clone(), e.node.modification_time, fsize, is_video) ..
}
);
let fsize = queued.unwrap_or_else(|| node_size(&e.node));
(
e.uid.clone(),
e.node.modification_time,
fsize,
provisional,
is_video,
)
} }
Some(_) => { Some(_) => {
reply.error(Errno::EISDIR); reply.error(Errno::EISDIR);
@@ -330,6 +340,13 @@ impl Filesystem for ProtonFs {
// races with nothing. FUSE does not require replies in request order. // races with nothing. FUSE does not require replies in request order.
let core = self.core.clone(); let core = self.core.clone();
self.core.workers.run(Lane::Transfer, "read", move || { self.core.workers.run(Lane::Transfer, "read", move || {
// The revision disagrees with a provisional size, and B84 fails
// such a read outright, so settle the size first. On the network,
// hence here and not above (bugs.md B100).
let fsize = match provisional {
true => core.resolve_provisional_size(ino.0, &uid).unwrap_or(fsize),
false => fsize,
};
match core.read_range(&uid, mtime, fsize, offset, size as u64, cache_blocks) { match core.read_range(&uid, mtime, fsize, offset, size as u64, cache_blocks) {
Ok(bytes) => reply.data(&bytes), Ok(bytes) => reply.data(&bytes),
Err(e) => reply.error(e), Err(e) => reply.error(e),
@@ -425,10 +442,7 @@ impl Filesystem for ProtonFs {
reply.error(Errno::EBADF); reply.error(Errno::EBADF);
return; return;
}; };
let end = offset + data.len() as u64; aw.record_write(offset, offset + data.len() as u64);
aw.written.add(offset, end);
aw.len = aw.len.max(end);
aw.dirty = true;
let len = aw.len; let len = aw.len;
st.set_size(ino.0, len); st.set_size(ino.0, len);
len len
@@ -1193,7 +1207,7 @@ impl ProtonFs {
/// open. /// open.
fn serve_open(&self, ino: INodeNo, flags: OpenFlags, reply: ReplyOpen) { fn serve_open(&self, ino: INodeNo, flags: OpenFlags, reply: ReplyOpen) {
let write_requested = flags.acc_mode() != OpenAccMode::O_RDONLY; let write_requested = flags.acc_mode() != OpenAccMode::O_RDONLY;
let (uid, base_mtime, base_size, base_revision_id) = { let (uid, base_mtime, base_size, provisional, base_revision_id) = {
let mut st = self.core.state(); let mut st = self.core.state();
match st.entries.get_mut(&ino.0) { match st.entries.get_mut(&ino.0) {
Some(e) if e.node.is_file() => { Some(e) if e.node.is_file() => {
@@ -1206,6 +1220,14 @@ impl ProtonFs {
e.uid.clone(), e.uid.clone(),
e.node.modification_time, e.node.modification_time,
node_size(&e.node), node_size(&e.node),
// Still the cheap enumeration's ciphertext size (B12).
matches!(
&e.node.kind,
NodeKind::File {
claimed_size: None,
..
}
),
node_revision_id(&e.node), node_revision_id(&e.node),
) )
} }
@@ -1251,6 +1273,21 @@ impl ProtonFs {
// `reply` is consumed by whichever arm answers, so the loop yields an // `reply` is consumed by whichever arm answers, so the loop yields an
// outcome and the answer is sent once, after it. // outcome and the answer is sent once, after it.
let outcome = 'install: { let outcome = 'install: {
// The handle starts at the base's size. A provisional one is the
// ciphertext size, so the file would grow by the encryption overhead
// and reading the base would fail (bugs.md B100). A queued revision
// stamps a real size, so this only goes to the network for a file
// the listing has not sized yet.
let base_size = match provisional && self.core.pending_blob(&uid).is_none() {
false => base_size,
true => match self.core.resolve_provisional_size(ino.0, &uid) {
Some(size) => size,
None => {
error!(%uid, "could not settle the size of the base for a write open");
break 'install Err(Errno::EIO);
}
},
};
for attempt in 0..OPEN_BASE_ATTEMPTS { for attempt in 0..OPEN_BASE_ATTEMPTS {
let pending_base = self.core.pending.lock().get(&uid).cloned(); let pending_base = self.core.pending.lock().get(&uid).cloned();
if pending_base.as_ref().is_some_and(|p| !p.meta.complete) { if pending_base.as_ref().is_some_and(|p| !p.meta.complete) {
@@ -1427,6 +1464,12 @@ impl ProtonFs {
return; return;
} }
if let Err(e) = self.core.ensure_children(parent) { if let Err(e) = self.core.ensure_children(parent) {
// The parent was looked up a moment ago, so ENOENT here means it
// left the tree in between. Log it: the caller only sees "no such
// file" on a create.
if e == Errno::ENOENT {
warn!(parent, name, "create: the parent folder is no longer known");
}
reply.error(e); reply.error(e);
return; return;
} }
@@ -1435,6 +1478,10 @@ impl ProtonFs {
match st.entries.get(&parent) { match st.entries.get(&parent) {
Some(e) => e.uid.clone(), Some(e) => e.uid.clone(),
None => { None => {
warn!(
parent,
name, "create: the parent folder left the tree during its listing"
);
reply.error(Errno::ENOENT); reply.error(Errno::ENOENT);
return; return;
} }
@@ -1454,37 +1501,45 @@ impl ProtonFs {
// //
// A parent that is itself still queued forces the same path even when we // A parent that is itself still queued forces the same path even when we
// are online: the API has no folder to put this in yet. // are online: the API has no folder to put this in yet.
let node = //
if !transient && self.core.online.load(Ordering::Relaxed) && !is_local_uid(&parent_uid) // A create that set out online and lost the network on the way lands in
{ // the queue too, so a Wi-Fi drop costs the caller a wait, never an EIO.
let minted = if !transient && self.core.is_online() && !is_local_uid(&parent_uid) {
if let Err(error) = self.core.require_uid_writable(&parent_uid) { if let Err(error) = self.core.require_uid_writable(&parent_uid) {
reply.error(error); reply.error(error);
return; return;
} }
// Create an empty file on the remote so it has a real uid immediately; // Create an empty file on the remote so it has a real uid immediately;
// written bytes are buffered and sealed as a new revision on close. // written bytes are buffered and sealed as a new revision on close.
let new_uid = match self.core.rt.block_on(self.core.client.upload_file( match self.core.block_on_bounded(self.core.client.upload_file(
&parent_uid, &parent_uid,
name, name,
media_type_for(name), media_type_for(name),
b"", b"",
)) { )) {
Ok(u) => u, Ok(new_uid) => match self.core.fetch_node(&new_uid) {
Ok(n) => Some(n),
// The file exists remotely but we could not read it back.
// The queued create finds it by name and adopts it.
Err(e) if e == Errno::EIO && !self.core.is_online() => None,
Err(e) => {
reply.error(e);
return;
}
},
Err(e) if self.core.lost_link(&e, "create") => None,
Err(e) => { Err(e) => {
error!(%parent_uid, name, error = %e, "create file failed"); error!(%parent_uid, name, error = %e, "create file failed");
reply.error(Errno::EIO); reply.error(Errno::EIO);
return; return;
} }
};
match self.core.fetch_node(&new_uid) {
Ok(n) => n,
Err(e) => {
reply.error(e);
return;
}
} }
} else { } else {
match self None
};
let node = match minted {
Some(node) => node,
None => match self
.core .core
.queue_local_node(&parent_uid, name, false, transient) .queue_local_node(&parent_uid, name, false, transient)
{ {
@@ -1493,7 +1548,7 @@ impl ProtonFs {
reply.error(e); reply.error(e);
return; return;
} }
} },
}; };
let new_uid = node.uid.clone(); let new_uid = node.uid.clone();
// The base this handle writes over is the node as it actually exists — // The base this handle writes over is the node as it actually exists —
@@ -1584,39 +1639,43 @@ impl ProtonFs {
// As in `create`: offline — or under a parent that is itself still // As in `create`: offline — or under a parent that is itself still
// queued — the folder becomes a placeholder that the drain turns into a // queued — the folder becomes a placeholder that the drain turns into a
// real one (offline.md Phase 3b). // real one (offline.md Phase 3b).
let node = if self.core.online.load(Ordering::Relaxed) && !is_local_uid(&parent_uid) { // Losing the network on the way queues it the same way.
let minted = if self.core.is_online() && !is_local_uid(&parent_uid) {
if let Err(error) = self.core.require_uid_writable(&parent_uid) { if let Err(error) = self.core.require_uid_writable(&parent_uid) {
reply.error(error); reply.error(error);
return; return;
} }
let new_uid =
match self match self
.core .core
.rt .block_on_bounded(self.core.client.create_folder(&parent_uid, name, now))
.block_on(self.core.client.create_folder(&parent_uid, name, now))
{ {
Ok(u) => u, Ok(new_uid) => match self.core.fetch_node(&new_uid) {
Ok(n) => Some(n),
Err(e) if e == Errno::EIO && !self.core.is_online() => None,
Err(e) => {
reply.error(e);
return;
}
},
Err(e) if self.core.lost_link(&e, "mkdir") => None,
Err(e) => { Err(e) => {
error!(%parent_uid, name, error = %e, "create folder failed"); error!(%parent_uid, name, error = %e, "create folder failed");
reply.error(Errno::EIO); reply.error(Errno::EIO);
return; return;
} }
};
match self.core.fetch_node(&new_uid) {
Ok(n) => n,
Err(e) => {
reply.error(e);
return;
}
} }
} else { } else {
match self.core.queue_local_node(&parent_uid, name, true, false) { None
};
let node = match minted {
Some(node) => node,
None => match self.core.queue_local_node(&parent_uid, name, true, false) {
Ok(n) => n, Ok(n) => n,
Err(e) => { Err(e) => {
reply.error(e); reply.error(e);
return; return;
} }
} },
}; };
let mut st = self.core.state(); let mut st = self.core.state();
let local = is_local_uid(&node.uid); let local = is_local_uid(&node.uid);
@@ -1706,12 +1765,18 @@ impl ProtonFs {
if let Some(uid) = unlinked_uid if let Some(uid) = unlinked_uid
&& release_can_discard_unlinked(&self.core.db, &uid) && release_can_discard_unlinked(&self.core.db, &uid)
{ {
// Ops withdrawn at unlink left the bytes this handle read behind,
// owned by no op any more; they go with the last handle.
let withdrawn = self.core.pending_blob(&uid);
if let Err(error) = self.core.discard_queued_ops(&uid) { if let Err(error) = self.core.discard_queued_ops(&uid) {
warn!(%uid, ?error, "queued-op cleanup failed; retaining unlinked state"); warn!(%uid, ?error, "queued-op cleanup failed; retaining unlinked state");
} else { } else {
if let Err(error) = self.core.db.delete_node(&uid) { if let Err(error) = self.core.db.delete_node(&uid) {
warn!(%uid, %error, "deleting released unlinked node state failed"); warn!(%uid, %error, "deleting released unlinked node state failed");
} }
if let Some(blob) = withdrawn {
self.core.cache.discard_staged(&blob);
}
self.core.cache.evict(&uid); self.core.cache.evict(&uid);
self.core.evict_reader(&uid); self.core.evict_reader(&uid);
} }
@@ -1932,12 +1997,11 @@ impl ProtonFs {
// take the synchronous path, so a genuine API refusal (permissions, a // take the synchronous path, so a genuine API refusal (permissions, a
// name clash) surfaces to the caller instead of becoming a queued op // name clash) surfaces to the caller instead of becoming a queued op
// that can only ever conflict. // that can only ever conflict.
if rename_needs_queue( // The queued rename, as a closure: the network going away under the
self.core.online.load(Ordering::Relaxed), // synchronous path below lands here too, with whatever half already
is_local_uid(&new_parent_uid), // reached Drive. The op records the end state, so the drain finishes
newparent != parent, // the rest.
newname != name, let queue_it = |reply: ReplyEmpty| {
) {
let queued = if replacement_authorized { let queued = if replacement_authorized {
self.core.queue_rename_authorized( self.core.queue_rename_authorized(
ino, ino,
@@ -1978,6 +2042,14 @@ impl ProtonFs {
reply.error(e); reply.error(e);
} }
} }
};
if rename_needs_queue(
self.core.is_online(),
is_local_uid(&new_parent_uid),
newparent != parent,
newname != name,
) {
queue_it(reply);
return; return;
} }
// Rename first if both halves change. Moving first makes the encrypted // Rename first if both halves change. Moving first makes the encrypted
@@ -1996,11 +2068,12 @@ impl ProtonFs {
return; return;
} }
if newname != name if newname != name
&& let Err(e) = self && let Err(e) = self.core.rename_remote(&uid, newname)
.core
.rt
.block_on(self.core.client.rename_node(&uid, newname, None))
{ {
if self.core.lost_link(&e, "rename") {
queue_it(reply);
return;
}
error!(%uid, error = %e, "rename failed"); error!(%uid, error = %e, "rename failed");
self.core.restore_replaced(victim.as_ref(), newname); self.core.restore_replaced(victim.as_ref(), newname);
reply.error(Errno::EIO); reply.error(Errno::EIO);
@@ -2011,8 +2084,7 @@ impl ProtonFs {
let moved = loop { let moved = loop {
match self match self
.core .core
.rt .block_on_bounded(self.core.client.move_node(&uid, &new_parent_uid))
.block_on(self.core.client.move_node(&uid, &new_parent_uid))
{ {
Ok(()) => break Ok(()), Ok(()) => break Ok(()),
Err(e) Err(e)
@@ -2031,6 +2103,10 @@ impl ProtonFs {
} }
}; };
if let Err(e) = moved { if let Err(e) = moved {
if self.core.lost_link(&e, "move") {
queue_it(reply);
return;
}
error!(%uid, attempts, error = %e, "move after rename failed"); error!(%uid, attempts, error = %e, "move after rename failed");
// The rename half landed in the source directory. // The rename half landed in the source directory.
let mut state = self.core.state(); let mut state = self.core.state();
@@ -2092,7 +2168,12 @@ impl ProtonFs {
// it just means its queued creation is no longer wanted. This works // it just means its queued creation is no longer wanted. This works
// offline, which the remote path below cannot (offline.md Phase 3b). // offline, which the remote path below cannot (offline.md Phase 3b).
if is_local_uid(&uid) { if is_local_uid(&uid) {
if !open_now && let Err(error) = self.core.discard_queued_ops(&uid) { let dropped = if open_now {
self.core.withdraw_queued_ops(&uid)
} else {
self.core.discard_queued_ops(&uid)
};
if let Err(error) = dropped {
reply.error(error); reply.error(error);
return; return;
} }
@@ -2103,12 +2184,14 @@ impl ProtonFs {
} }
// Offline: queue it. Trashing is the one mutation a user expects to work // Offline: queue it. Trashing is the one mutation a user expects to work
// regardless — the file is gone from their point of view the moment the // regardless — the file is gone from their point of view the moment the
// command returns (offline.md Phase 3b). // command returns (offline.md Phase 3b). The same when the network goes
if !self.core.online.load(Ordering::Relaxed) { // away under the remote call below.
match self.core.queue_trash(&uid, name) { let queue_it = |reply: ReplyEmpty| match self.core.queue_trash(&uid, name) {
Ok(()) => reply.ok(), Ok(()) => reply.ok(),
Err(e) => reply.error(e), Err(e) => reply.error(e),
} };
if !self.core.is_online() {
queue_it(reply);
return; return;
} }
if let Err(e) = self.core.require_uid_writable(&uid) { if let Err(e) = self.core.require_uid_writable(&uid) {
@@ -2117,10 +2200,13 @@ impl ProtonFs {
} }
if let Err(e) = self if let Err(e) = self
.core .core
.rt .block_on_bounded(self.core.client.trash_nodes(std::slice::from_ref(&uid)))
.block_on(self.core.client.trash_nodes(std::slice::from_ref(&uid)))
.and_then(batch::into_unit) .and_then(batch::into_unit)
{ {
if self.core.lost_link(&e, "trash") {
queue_it(reply);
return;
}
error!(%uid, error = %e, "trash failed"); error!(%uid, error = %e, "trash failed");
self.core self.core
.log_activity(ActivityKind::Trash, name, e.to_string(), false); .log_activity(ActivityKind::Trash, name, e.to_string(), false);
@@ -2131,7 +2217,13 @@ impl ProtonFs {
// Otherwise an EIO response still made the path disappear locally. // Otherwise an EIO response still made the path disappear locally.
self.core.hidden.lock().insert(uid.clone()); self.core.hidden.lock().insert(uid.clone());
self.core.state().forget_or_unlink(&uid); self.core.state().forget_or_unlink(&uid);
if !open_now { if open_now {
// The open handle keeps the staged bytes; the drain must not send
// them to a node that is already in the trash (docs/BUGS.md B106).
if let Err(error) = self.core.withdraw_queued_ops(&uid) {
warn!(%uid, ?error, "remote trash landed but queued-op cleanup failed");
}
} else {
if let Err(error) = self.core.discard_queued_ops(&uid) { if let Err(error) = self.core.discard_queued_ops(&uid) {
warn!(%uid, ?error, "remote trash landed but queued-op cleanup failed"); warn!(%uid, ?error, "remote trash landed but queued-op cleanup failed");
} }
File diff suppressed because it is too large. Load diff
+315
View File
@@ -0,0 +1,315 @@
//! Noticing that the network went away mid-session, and that it came back.
//!
//! The mount used to learn it was offline only at startup: a daemon that had
//! connected once kept `online` true for the rest of its life, so a Wi-Fi drop
//! turned every create, mkdir, delete and rename into a two-minute wait for an
//! `EIO`, while the queue that exists for exactly this sat unused. Now any
//! remote call a handler waits on reports a failure to reach Proton through
//! [`Core::lost_link`], which flips the mount offline; the handler then takes its
//! queued path, and the probe thread here flips it back when Proton answers.
use std::future::Future;
use std::sync::atomic::Ordering;
use std::time::{Duration, Instant};
use parking_lot::{Condvar, Mutex};
use proton_drive_rs::proton_sdk::api::ResponseCode;
use proton_drive_rs::proton_sdk::error::{ProtonApiError, ProtonError};
use tracing::{debug, info, warn};
use crate::{Core, ONLINE_PROBE_MAX, ONLINE_PROBE_MIN, ROOT_INO};
/// How long a remote call a FUSE handler is waiting on may take before the link
/// is treated as down.
///
/// The SDK's own budget for one request is four attempts of up to 30 seconds
/// each, so a dead link held the caller for about two minutes. A metadata call
/// that has had no answer in 20 seconds is not going to be one the user wants to
/// keep waiting for; queueing the change is the better answer.
pub(crate) const INTERACTIVE_CALL_TIMEOUT: Duration = Duration::from_secs(20);
/// How long one probe may wait for an answer. A probe is a single small request,
/// and a probe stuck in the SDK's retry budget would delay noticing the link is
/// back by as long as it waits.
const PROBE_TIMEOUT: Duration = Duration::from_secs(10);
/// First delay between probes after a drop seen mid-session, and the ceiling it
/// doubles to while the outage is young. A blip should end in seconds, not in
/// the minutes [`ONLINE_PROBE_MAX`] allows.
const PROBE_AFTER_DROP: Duration = Duration::from_secs(2);
const PROBE_YOUNG_MAX: Duration = Duration::from_secs(30);
/// How long an outage counts as young. Past it the probe backs off to
/// [`ONLINE_PROBE_MAX`], because a laptop can sit offline for days and each
/// probe is a real API round trip.
const YOUNG_OUTAGE: Duration = Duration::from_secs(10 * 60);
/// How often the idle probe thread checks for teardown while the link is up.
const IDLE_CHECK: Duration = Duration::from_secs(1);
/// The shared half of the online flag: when the link went down, and a condvar
/// that fires on every change so waiters need not poll.
#[derive(Default)]
pub(crate) struct Link {
lost_at: Mutex<Option<Instant>>,
changed: Condvar,
/// Folders listed from the DB alone while offline, to relist once back.
pub(crate) stale_listings: Mutex<std::collections::HashSet<u64>>,
}
/// Whether a failed call failed because Proton could not be reached, rather
/// than because it answered and refused.
///
/// A transport error is the network. So is a request timeout: the SDK reports
/// Proton's own "try again" codes that way, and [`no_answer`] reports our
/// deadline that way. Anything else Proton actually said, and says again on a
/// retry, so it must reach the caller rather than be queued.
pub(crate) fn is_network_error(e: &ProtonError) -> bool {
match e {
ProtonError::Transport(_) => true,
ProtonError::Api(api) => {
matches!(
api.code,
ResponseCode::RequestTimeout | ResponseCode::Timeout
)
}
_ => false,
}
}
/// The error a call reports when it runs past `after` without an answer.
pub(crate) fn no_answer(after: Duration) -> ProtonError {
ProtonError::Api(ProtonApiError {
code: ResponseCode::RequestTimeout,
http_status: 408,
message: format!("no answer within {}s", after.as_secs()),
details: None,
})
}
/// Run a remote call, giving up after [`INTERACTIVE_CALL_TIMEOUT`].
pub(crate) async fn bounded<T>(
call: impl Future<Output = Result<T, ProtonError>>,
) -> Result<T, ProtonError> {
match tokio::time::timeout(INTERACTIVE_CALL_TIMEOUT, call).await {
Ok(result) => result,
Err(_) => Err(no_answer(INTERACTIVE_CALL_TIMEOUT)),
}
}
/// The next delay between probes, given the current one and how long the
/// outage has lasted.
fn next_probe_delay(delay: Duration, outage: Duration) -> Duration {
let cap = if outage < YOUNG_OUTAGE {
PROBE_YOUNG_MAX
} else {
ONLINE_PROBE_MAX
};
(delay * 2).min(cap)
}
impl Core {
/// Whether the mount currently believes Proton is reachable.
pub(crate) fn is_online(&self) -> bool {
self.online.load(Ordering::Relaxed)
}
/// Run a remote call a caller is waiting on, giving up after
/// [`INTERACTIVE_CALL_TIMEOUT`]. The timer is built inside the runtime: a
/// FUSE worker has no reactor of its own.
pub(crate) fn block_on_bounded<T>(
&self,
call: impl Future<Output = Result<T, ProtonError>>,
) -> Result<T, ProtonError> {
self.rt.block_on(bounded(call))
}
/// Report a failed remote call. Returns whether it failed for want of a
/// network, in which case the mount is now offline and the caller should
/// take its queued path instead of failing.
pub(crate) fn lost_link(&self, e: &ProtonError, what: &str) -> bool {
if !is_network_error(e) {
return false;
}
self.mark_offline(what, e);
true
}
/// Flip the mount offline and wake the probe. Only the first caller of an
/// outage logs it; the rest are the same outage.
pub(crate) fn mark_offline(&self, what: &str, error: &dyn std::fmt::Display) {
let mut lost_at = self.link.lost_at.lock();
if self.online.swap(false, Ordering::Relaxed) {
*lost_at = Some(Instant::now());
warn!(during = what, %error,
"lost the connection to Proton; queueing changes until it returns");
self.link.changed.notify_all();
}
}
/// Flip the mount back online, wake everything waiting for it, and let the
/// queue drain.
fn mark_online(&self) {
{
let mut lost_at = self.link.lost_at.lock();
self.online.store(true, Ordering::Relaxed);
match lost_at.take() {
Some(since) => {
info!(
offline_secs = since.elapsed().as_secs(),
"connection to Proton is back"
)
}
None => info!("back online"),
}
self.link.changed.notify_all();
}
self.relist_offline_listings();
// Anything written while offline is queued and waiting on exactly this,
// including ops that failed on the way down and are sitting out a
// backoff sized for a server that keeps refusing.
if let Err(e) = self.db.retry_failed_ops_now(crate::now_millis()) {
warn!(error = %e, "moving backed-off ops up after a reconnect failed");
}
self.wake_drain();
}
/// Wait up to `budget` for the link to come back. Returns whether it is up.
pub(crate) fn wait_online(&self, budget: Duration) -> bool {
let deadline = Instant::now() + budget;
let mut lost_at = self.link.lost_at.lock();
while !self.is_online() {
if self.shutdown.is_stopping()
|| self
.link
.changed
.wait_until(&mut lost_at, deadline)
.timed_out()
{
return self.is_online();
}
}
true
}
/// Wait out `interval` before a background poll of Proton, and while the
/// link is down, until it is back. Returns as soon as it comes back, so the
/// poll catches up on what changed during the outage at once instead of
/// one interval later. A poll into a dead link would only wait out the
/// SDK's retries and fail.
pub(crate) async fn next_poll(&self, interval: Duration) {
let due = Instant::now() + interval;
let mut was_offline = false;
loop {
if !self.is_online() {
was_offline = true;
} else if was_offline || Instant::now() >= due {
return;
}
tokio::time::sleep(IDLE_CHECK).await;
}
}
/// Watch the link for the life of the mount: sleep while it is up, and
/// probe it back while it is down (offline.md Phase 1).
///
/// A mount that started from the cache is down from the first moment; one
/// that started online goes down when [`Core::mark_offline`] says so.
pub(crate) fn run_online_probe(&self) {
// Only an offline start never had a chance to repair its share id.
let mut repair_share_id = !self.is_online();
loop {
{
let mut lost_at = self.link.lost_at.lock();
while self.is_online() {
if self.shutdown.is_stopping() {
return;
}
self.link.changed.wait_for(&mut lost_at, IDLE_CHECK);
}
}
if !self.probe_until_online() {
return;
}
if std::mem::take(&mut repair_share_id) {
self.repair_primary_share_id(&self.primary_root_uid);
}
}
}
/// Probe until Proton answers, then flip the mount online. Returns `false`
/// if the daemon is stopping.
fn probe_until_online(&self) -> bool {
let started = self.link.lost_at.lock().unwrap_or_else(Instant::now);
let mut delay = if started.elapsed() < YOUNG_OUTAGE {
PROBE_AFTER_DROP
} else {
ONLINE_PROBE_MIN
};
loop {
if !self.shutdown.sleep(delay) {
return false;
}
let probe = self.rt.block_on(async {
match tokio::time::timeout(PROBE_TIMEOUT, self.client.get_my_files_folder()).await {
Ok(result) => result,
Err(_) => Err(no_answer(PROBE_TIMEOUT)),
}
});
match probe {
Ok(root) => {
{
let mut st = self.state();
if let Some(e) = st.entries.get_mut(&ROOT_INO) {
e.node = root.clone();
}
}
if let Err(e) = self.db.upsert_node(&root) {
warn!(error = %e, "refresh root after reconnect failed");
}
self.mark_online();
return true;
}
Err(e) => {
debug!(error = %e, ?delay, "online probe failed; still offline");
delay = next_probe_delay(delay, started.elapsed());
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn api(code: ResponseCode) -> ProtonError {
ProtonError::Api(ProtonApiError {
code,
http_status: 422,
message: String::new(),
details: None,
})
}
#[test]
fn only_an_unanswered_call_counts_as_a_lost_link() {
assert!(is_network_error(&no_answer(INTERACTIVE_CALL_TIMEOUT)));
assert!(is_network_error(&api(ResponseCode::Timeout)));
assert!(!is_network_error(&api(ResponseCode::AlreadyExists)));
assert!(!is_network_error(&api(ResponseCode::InvalidRequirements)));
assert!(!is_network_error(&ProtonError::invalid_operation("bug")));
}
#[test]
fn a_young_outage_is_probed_often_and_an_old_one_rarely() {
let young = Duration::from_secs(60);
let old = YOUNG_OUTAGE + Duration::from_secs(1);
assert_eq!(
next_probe_delay(PROBE_AFTER_DROP, young),
2 * PROBE_AFTER_DROP
);
assert_eq!(next_probe_delay(PROBE_YOUNG_MAX, young), PROBE_YOUNG_MAX);
assert_eq!(next_probe_delay(PROBE_YOUNG_MAX, old), 2 * PROBE_YOUNG_MAX);
assert_eq!(next_probe_delay(ONLINE_PROBE_MAX, old), ONLINE_PROBE_MAX);
}
}
+6 -3
View File
@@ -383,6 +383,7 @@ pub fn mount(
shared_generation: Arc::new(AtomicU64::new(0)), shared_generation: Arc::new(AtomicU64::new(0)),
shared_refresh_deadlines: Arc::new(Mutex::new(SharedRefreshDeadlines::default())), shared_refresh_deadlines: Arc::new(Mutex::new(SharedRefreshDeadlines::default())),
online: Arc::new(AtomicBool::new(online)), online: Arc::new(AtomicBool::new(online)),
link: Arc::new(Default::default()),
sync_paused_until: Arc::new(AtomicI64::new(paused_until)), sync_paused_until: Arc::new(AtomicI64::new(paused_until)),
pending: Arc::new(Mutex::new(HashMap::new())), pending: Arc::new(Mutex::new(HashMap::new())),
hidden: Arc::new(Mutex::new(HashSet::new())), hidden: Arc::new(Mutex::new(HashSet::new())),
@@ -411,6 +412,7 @@ pub fn mount(
thumbnail_build_cancel: Arc::new(tokio::sync::Notify::new()), thumbnail_build_cancel: Arc::new(tokio::sync::Notify::new()),
thumbnail_misses: Arc::new(Mutex::new(Default::default())), thumbnail_misses: Arc::new(Mutex::new(Default::default())),
quota: Arc::new(Mutex::new(None)), quota: Arc::new(Mutex::new(None)),
quota_refreshing: Arc::new(AtomicBool::new(false)),
size_upgrades: Arc::new(Mutex::new(HashMap::new())), size_upgrades: Arc::new(Mutex::new(HashMap::new())),
size_waiters: Arc::new(Default::default()), size_waiters: Arc::new(Default::default()),
notifier: Arc::new(OnceLock::new()), notifier: Arc::new(OnceLock::new()),
@@ -484,9 +486,10 @@ pub fn mount(
); );
} }
// Mounted from the cache: watch for the network coming back so the mount can // Watch the link for the life of the mount: a mount from the cache waits for
// stop being read-only-ish without the user restarting the daemon. // the network to come back, and an online one can lose it at any moment and
if !online { // must notice it coming back just the same.
{
let core = core.clone(); let core = core.clone();
workers.push( workers.push(
std::thread::Builder::new() std::thread::Builder::new()
+58 -2
View File
@@ -36,6 +36,21 @@ pub(super) const REPAIR_MAX: u64 = 512 * 1024 * 1024;
/// demand read in the SDK's flat block semaphore. /// demand read in the SDK's flat block semaphore.
pub(super) const PREFETCH_BUDGET: usize = 8; pub(super) const PREFETCH_BUDGET: usize = 8;
/// How long one read may wait on the network before it fails with `EIO`.
///
/// The SDK bounds each request, not a read: a block that times out is retried,
/// and a read can need several blocks, a block table and a repair. On a link
/// that stalled without dropping, one `read` held a FUSE worker for 34 minutes
/// and the application behind it with it (`docs/BUGS.md` B107). An error the
/// caller can retry beats a process stuck in `D` state.
pub(super) const READ_FETCH_TIMEOUT: Duration = Duration::from_secs(120);
/// How long a read of blocks nothing has cached waits for a lost link to come
/// back before failing. Long enough to ride out a Wi-Fi blip, short enough
/// that a program reading while the laptop is really offline gets its `EIO`
/// before the user gives up on it.
const READ_OFFLINE_WAIT: Duration = Duration::from_secs(30);
/// Cap on tracked sequential streams. Only reads in progress matter, so the map /// Cap on tracked sequential streams. Only reads in progress matter, so the map
/// is cleared wholesale rather than aged — losing the ramp costs one read. /// is cleared wholesale rather than aged — losing the ramp costs one read.
const MAX_PREFETCH_STREAMS: usize = 256; const MAX_PREFETCH_STREAMS: usize = 256;
@@ -364,6 +379,7 @@ impl Core {
let result = match self.client.open_revision(uid).await { let result = match self.client.open_revision(uid).await {
Ok(reader) => Ok(Arc::new(reader)), Ok(reader) => Ok(Arc::new(reader)),
Err(e) => { Err(e) => {
self.lost_link(&e, "open a revision");
warn!(%uid, error = %e, "open_revision failed"); warn!(%uid, error = %e, "open_revision failed");
Err(Errno::EIO) Err(Errno::EIO)
} }
@@ -649,6 +665,7 @@ impl Core {
.store_block_geometry(uid, mtime, fsize, reader.block_sizes()); .store_block_geometry(uid, mtime, fsize, reader.block_sizes());
} }
let bytes = reader.read_at(span.start, span.len).await.map_err(|e| { let bytes = reader.read_at(span.start, span.len).await.map_err(|e| {
self.lost_link(&e, "read a block");
warn!(%uid, start = span.start, len = span.len, error = %e, "block read failed"); warn!(%uid, start = span.start, len = span.len, error = %e, "block read failed");
Errno::EIO Errno::EIO
})?; })?;
@@ -939,7 +956,21 @@ impl Core {
} }
if !misses.is_empty() { if !misses.is_empty() {
let fetched = self.rt.block_on(async { // One deadline for the whole read, including any wait for a lost
// link. A fetch that failed because the link went down is tried
// once more when it is back, so a blip is a slow read, not an `EIO`.
let deadline = Instant::now() + READ_FETCH_TIMEOUT;
let mut retried = false;
let fetched = loop {
if !self.is_online() {
let wait =
READ_OFFLINE_WAIT.min(deadline.saturating_duration_since(Instant::now()));
if !self.wait_online(wait) {
debug!(%uid, offset, len, "read of uncached blocks while offline; failing it");
return Err(Errno::EIO);
}
}
let fetch = async {
// Resolve the file's keys and block table once, then read every // Resolve the file's keys and block table once, then read every
// missing block through the shared reader. Previously each block // missing block through the shared reader. Previously each block
// called `download_range`, which redid that resolution per block. // called `download_range`, which redid that resolution per block.
@@ -960,7 +991,32 @@ impl Core {
out.push(joined.map_err(|_| Errno::EIO)??); out.push(joined.map_err(|_| Errno::EIO)??);
} }
Ok::<_, Errno>(out) Ok::<_, Errno>(out)
})?; };
// Dropping the fetch on the deadline aborts the block tasks with
// it; blocks that already landed stay cached for the retry. The
// timer is built inside the runtime: a FUSE worker has no reactor
// of its own, and building it out here panicked every remote read.
let budget = deadline.saturating_duration_since(Instant::now());
match self
.rt
.block_on(async { tokio::time::timeout(budget, fetch).await })
{
Ok(Ok(fetched)) => break fetched,
// The fetch saw the link go down; the top of the loop waits
// for it within what is left of the deadline.
Ok(Err(_)) if !retried && !self.is_online() => retried = true,
Ok(Err(e)) => return Err(e),
Err(_) => {
warn!(
%uid, offset, len, secs = READ_FETCH_TIMEOUT.as_secs(),
"read got no answer from the network in time; failing it"
);
self.evict_reader(uid);
self.mark_offline("read", &crate::link::no_answer(READ_FETCH_TIMEOUT));
return Err(Errno::EIO);
}
}
};
for (bidx, bytes) in fetched { for (bidx, bytes) in fetched {
blocks[(bidx - first) as usize] = Some(bytes); blocks[(bidx - first) as usize] = Some(bytes);
} }
+603
View File
@@ -0,0 +1,603 @@
//! Moving a file or folder between two locations without moving its content.
//!
//! `~/ProtonDrive` and every on-demand synced folder are separate FUSE
//! sessions, and a mirror folder is a plain local directory. The kernel refuses
//! `rename(2)` between two mounts with `EXDEV` before the daemon hears of it, so
//! `mv` falls back to copy and delete: every file is downloaded, uploaded again,
//! and the original is trashed.
//!
//! On Proton Drive the same move is one metadata change, because My files and
//! this computer's device folders all live on the main volume. So a move
//! request that names two absolute paths is resolved here, against every
//! location this daemon knows, and done as a server-side move. The local side
//! is then brought level: a mount forgets the node and re-lists, and a mirror
//! folder either renames its copy along or drops it once Drive has it.
use std::collections::HashMap;
use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use std::time::Duration;
use parking_lot::Mutex;
use pdfs_core::db::{StoredSyncEntry, StoredSyncFolder};
use pdfs_core::{CoreError, CoreResult};
use proton_drive_rs::NodeKind;
use proton_drive_rs::proton_sdk::ids::NodeUid;
use tracing::{info, warn};
use super::background::forget_and_notify;
use super::sync::mirror_subtree_unsynced;
use super::{Core, is_local_uid, parse_uid};
/// How long a move waits for a mirror folder's sync pass to finish before it
/// reports the folder busy. A pass can run for hours on a first sync, so the
/// move does not wait it out.
const MIRROR_LOCK_WAIT: Duration = Duration::from_secs(5);
/// Where one side of a move lives.
enum Place {
/// My files or an on-demand folder: a live inode space. `core` is rooted at
/// that mount and `rel` is relative to its mountpoint.
Mounted { core: Box<Core>, rel: PathBuf },
/// A mirror folder: a local directory kept in step by the sync engine.
/// `rel` is `/`-separated, relative to the folder, as the baseline keys it.
Mirror {
folder: StoredSyncFolder,
rel: String,
},
}
impl Place {
fn mirror_id(&self) -> Option<i64> {
match self {
Place::Mirror { folder, .. } => Some(folder.id),
Place::Mounted { .. } => None,
}
}
fn is_root(&self) -> bool {
match self {
Place::Mounted { rel, .. } => rel.as_os_str().is_empty(),
Place::Mirror { rel, .. } => rel.is_empty(),
}
}
}
/// The mirror folder that holds `abs`, and `abs` relative to it. The deepest
/// folder wins, although folders are not expected to nest.
fn mirror_covering(
folders: Vec<StoredSyncFolder>,
abs: &Path,
) -> Option<(StoredSyncFolder, String)> {
let folder = folders
.into_iter()
.filter(|f| f.mode == "mirror" && abs.starts_with(&f.local_path))
.max_by_key(|f| Path::new(&f.local_path).components().count())?;
let rel = abs.strip_prefix(&folder.local_path).ok()?;
let parts = rel
.components()
.map(|c| match c {
Component::Normal(part) => part.to_str(),
_ => None,
})
.collect::<Option<Vec<_>>>()?;
Some((folder, parts.join("/")))
}
/// `name` inside the folder at `parent`, in baseline form.
fn join_rel(parent: &str, name: &str) -> String {
if parent.is_empty() {
name.to_string()
} else {
format!("{parent}/{name}")
}
}
/// Rename `from` to `to`, failing with `EEXIST` rather than replacing
/// whatever appeared at `to` since it was checked. A filesystem that cannot
/// promise that gets a plain rename after one more look.
fn rename_noreplace(from: &Path, to: &Path) -> std::io::Result<()> {
use std::ffi::CString;
use std::os::unix::ffi::OsStrExt;
let cstr = |p: &Path| {
CString::new(p.as_os_str().as_bytes())
.map_err(|_| std::io::Error::from_raw_os_error(libc::EINVAL))
};
let (c_from, c_to) = (cstr(from)?, cstr(to)?);
// SAFETY: both are valid NUL-terminated paths that outlive the call.
let rc = unsafe {
libc::syscall(
libc::SYS_renameat2,
libc::AT_FDCWD,
c_from.as_ptr(),
libc::AT_FDCWD,
c_to.as_ptr(),
libc::RENAME_NOREPLACE,
)
};
if rc == 0 {
return Ok(());
}
let err = std::io::Error::last_os_error();
match err.raw_os_error() {
Some(libc::EINVAL | libc::ENOSYS) => {
if std::fs::symlink_metadata(to).is_ok() {
return Err(std::io::Error::from_raw_os_error(libc::EEXIST));
}
std::fs::rename(from, to)
}
_ => Err(err),
}
}
/// Whether `uid` is `root` or lies under it, as far as the nodes table knows.
fn under(core: &Core, root: &NodeUid, uid: &str) -> bool {
let root = root.to_string();
uid == root || matches!(core.db.path_relative_to(&root, uid), Ok(Some(_)))
}
impl Core {
/// Move the file or folder at `src` into the folder at `dest_parent`. Both
/// are absolute local paths and may be in different locations: My files,
/// an on-demand folder, or a mirror folder. Returns the moved name.
///
/// Within one inode space this is [`Core::move_to`]. Across locations it
/// is a server-side move followed by local bookkeeping; see the module
/// documentation. Anything the move could lose is refused up front.
pub(crate) fn move_between(&self, src: &Path, dest_parent: &Path) -> CoreResult<String> {
let source = self.locate(src)?;
let dest = self.locate(dest_parent)?;
if let (
Place::Mounted { core, rel },
Place::Mounted {
core: dest_core,
rel: dest_rel,
},
) = (&source, &dest)
&& Arc::ptr_eq(&core.state, &dest_core.state)
{
return core.move_to(rel, dest_rel);
}
if source.is_root() {
return Err(CoreError::invalid(format!(
"{} is the root of a synced location and cannot be moved",
src.display()
)));
}
if dest_parent.starts_with(src) {
return Err(CoreError::invalid("cannot move a folder into itself"));
}
let name = src
.file_name()
.and_then(|n| n.to_str())
.ok_or_else(|| CoreError::invalid(format!("{} has no usable name", src.display())))?
.to_string();
// Hold every mirror folder involved, lowest id first, so no sync pass
// sees the move half done and two moves cannot deadlock.
let mut ids: Vec<i64> = [source.mirror_id(), dest.mirror_id()]
.into_iter()
.flatten()
.collect();
ids.sort_unstable();
ids.dedup();
let locks: Vec<Arc<Mutex<()>>> = ids.iter().map(|&id| self.sync_lock(id)).collect();
let mut guards = Vec::with_capacity(locks.len());
for lock in &locks {
guards.push(lock.try_lock_for(MIRROR_LOCK_WAIT).ok_or_else(|| {
CoreError::conflict("a synced folder is busy syncing; try again when it is done")
})?);
}
let mut baselines: HashMap<i64, HashMap<String, StoredSyncEntry>> = HashMap::new();
for &id in &ids {
let baseline = self
.db
.sync_entries(id)
.map_err(|e| CoreError::internal(format!("db: {e}")))?;
baselines.insert(id, baseline);
}
let src_uid = self.source_uid(&source, &baselines)?;
let dest_uid = self.dest_uid(&dest, &baselines, &name)?;
if src_uid.volume_id != dest_uid.volume_id {
return Err(CoreError::invalid(
"the two locations are on different Proton Drive volumes; copy it instead",
));
}
if under(self, &src_uid, &dest_uid.to_string()) {
return Err(CoreError::invalid("cannot move a folder into itself"));
}
// Mirror to mirror renames the local copy along, so it needs nothing
// proven about it. Another filesystem makes that impossible, and the
// pair then drops the source copy like any other move out of a mirror.
let mut renamed = None;
if let (
Place::Mirror { folder, rel },
Place::Mirror {
folder: dest_folder,
rel: dest_rel,
},
) = (&source, &dest)
{
let from = src.to_path_buf();
let to = dest_parent.join(&name);
match rename_noreplace(&from, &to) {
Ok(()) => renamed = Some((from, to)),
Err(e) if e.raw_os_error() == Some(libc::EXDEV) => {
info!(from = %folder.local_path, to = %dest_folder.local_path,
"move: mirror folders are on different filesystems");
}
Err(e) if e.raw_os_error() == Some(libc::EEXIST) => {
return Err(CoreError::conflict(format!(
"{name} already exists in {}",
dest_parent.display()
)));
}
Err(e) => {
return Err(CoreError::internal(format!(
"could not move {} to {}: {e}",
rel,
join_rel(dest_rel, &name)
)));
}
}
}
if renamed.is_none()
&& let Place::Mirror { folder, rel } = &source
&& let Some(reason) =
mirror_subtree_unsynced(Path::new(&folder.local_path), rel, &baselines[&folder.id])
{
return Err(CoreError::conflict(format!(
"{name} is not fully synced yet ({reason}); wait for the sync to finish"
)));
}
if let Err(e) = self.rt.block_on(self.client.move_node(&src_uid, &dest_uid)) {
if let Some((from, to)) = &renamed
&& let Err(back) = std::fs::rename(to, from)
{
warn!(from = %to.display(), to = %from.display(), error = %back,
"move: could not put the local copy back after the move failed");
}
return Err(CoreError::from_api(&e, "move"));
}
self.settle_source(&source, &src_uid, renamed.is_some(), &dest, &name, src);
self.settle_dest(&dest, &dest_uid, &name);
drop(guards);
if renamed.is_none()
&& let Place::Mirror { folder, .. } = &dest
{
// Nothing local exists yet; the folder's next pass downloads it.
self.sync_now(Some(folder.id));
}
Ok(name)
}
/// Which location `abs` is in. A mount is looked for first: a mirror folder
/// cannot live inside a FUSE mount, and a mount is what the kernel sees.
fn locate(&self, abs: &Path) -> CoreResult<Place> {
if let Some((core, rel)) = self.rooted_at(abs) {
if !rel
.components()
.all(|c| matches!(c, Component::Normal(_) | Component::CurDir))
{
return Err(CoreError::invalid(format!(
"{} escapes its location",
abs.display()
)));
}
return Ok(Place::Mounted {
core: Box::new(core),
rel,
});
}
let folders = self
.db
.sync_folder_list()
.map_err(|e| CoreError::internal(format!("db: {e}")))?;
mirror_covering(folders, abs)
.map(|(folder, rel)| Place::Mirror { folder, rel })
.ok_or_else(|| {
CoreError::invalid(format!(
"{} is not in a Proton Drive location",
abs.display()
))
})
}
/// The node being moved, once everything that must hold before it leaves
/// its location has been checked.
fn source_uid(
&self,
source: &Place,
baselines: &HashMap<i64, HashMap<String, StoredSyncEntry>>,
) -> CoreResult<NodeUid> {
match source {
Place::Mounted { core, rel } => {
let (ino, uid) = core.resolve(rel)?;
if is_local_uid(&uid) {
return Err(CoreError::conflict(
"it is not uploaded yet; wait for the upload to finish",
));
}
let old_parent = core.source_parent_uid(ino, rel)?;
for authority in [&uid, &old_parent] {
core.require_uid_writable(authority)
.map_err(|e| core.errno_error(e, "move access"))?;
}
if self.has_pending_work_under(&uid)? {
return Err(CoreError::conflict(
"it has changes that are not uploaded yet; wait for the upload to finish",
));
}
Ok(uid)
}
Place::Mirror { folder, rel } => baselines[&folder.id]
.get(rel)
.and_then(|entry| entry.remote_uid.as_deref())
.and_then(parse_uid)
.ok_or_else(|| {
CoreError::conflict(
"it is not on Proton Drive yet; wait for the sync to finish",
)
}),
}
}
/// The folder the node moves into. Refuses one that already holds `name`
/// on this disk; Drive refuses a clash on its side by itself.
fn dest_uid(
&self,
dest: &Place,
baselines: &HashMap<i64, HashMap<String, StoredSyncEntry>>,
name: &str,
) -> CoreResult<NodeUid> {
let uid = match dest {
Place::Mounted { core, rel } => {
let (ino, uid) = core.resolve(rel)?;
let is_folder = core
.state()
.entries
.get(&ino)
.is_some_and(|entry| matches!(entry.node.kind, NodeKind::Folder));
if !is_folder {
return Err(CoreError::invalid("the destination is not a folder"));
}
core.require_uid_writable(&uid)
.map_err(|e| core.errno_error(e, "move access"))?;
uid
}
Place::Mirror { folder, rel } => {
let local = Path::new(&folder.local_path).join(rel);
if !local.is_dir() {
return Err(CoreError::invalid("the destination is not a folder"));
}
if std::fs::symlink_metadata(local.join(name)).is_ok() {
return Err(CoreError::conflict(format!(
"{name} already exists in the destination"
)));
}
let uid = if rel.is_empty() {
Some(folder.remote_uid.as_str())
} else {
baselines[&folder.id]
.get(rel)
.and_then(|entry| entry.remote_uid.as_deref())
};
uid.and_then(parse_uid).ok_or_else(|| {
CoreError::conflict(
"the destination is not on Proton Drive yet; wait for the sync to finish",
)
})?
}
};
if is_local_uid(&uid) {
return Err(CoreError::conflict(
"the destination is not uploaded yet; wait for the upload to finish",
));
}
Ok(uid)
}
/// Whether the queue owes Drive anything for `uid` or a node under it, or a
/// file under it is open for writing. Moving such a node away would strand
/// the pending change against a path that no longer leads to it.
fn has_pending_work_under(&self, uid: &NodeUid) -> CoreResult<bool> {
let ops = self
.db
.pending_ops()
.map_err(|e| CoreError::internal(format!("db: {e}")))?;
if ops.iter().any(|op| {
under(self, uid, &op.uid)
|| op
.parent_uid
.as_deref()
.is_some_and(|p| under(self, uid, p))
}) {
return Ok(true);
}
// Collected first so no State lock is held across the nodes-table reads.
let mut writing = Vec::new();
self.for_each_state(|st| {
writing.extend(st.active_writes.values().map(|w| w.uid.to_string()));
});
Ok(writing.iter().any(|w| under(self, uid, w)))
}
/// Bring the source location level with a move Drive has made.
fn settle_source(
&self,
source: &Place,
uid: &NodeUid,
renamed: bool,
dest: &Place,
name: &str,
src: &Path,
) {
match source {
Place::Mounted { core, rel } => {
// Forget it everywhere, so no mount keeps serving it under the
// old path. The content cache is keyed by uid and stays valid.
self.for_each_mount(|st, notify| {
forget_and_notify(st, notify, uid);
});
core.invalidate_parent_listing(rel);
}
Place::Mirror { folder, rel } if renamed => {
let Place::Mirror {
folder: dest_folder,
rel: dest_rel,
} = dest
else {
unreachable!("a local rename needs a mirror destination");
};
if let Err(e) = self.db.sync_entries_move(
folder.id,
rel,
dest_folder.id,
&join_rel(dest_rel, name),
) {
// The next passes see a copy with no baseline on one side
// and a baseline with no copy on the other. That uploads
// the files again and trashes the moved node, but loses
// nothing.
warn!(error = ?e, "move: could not carry the sync baseline along");
}
}
Place::Mirror { folder, rel } => {
// Drive had all of it when the move was checked, so the local
// copy is only a stale duplicate now. Something written into it
// since would go with it, so look again first; if anything is
// new, the copy stays and the next pass uploads it as new.
let changed = match self.db.sync_entries(folder.id) {
Ok(baseline) => {
mirror_subtree_unsynced(Path::new(&folder.local_path), rel, &baseline)
}
Err(e) => Some(format!("db: {e}")),
};
if let Err(e) = self.db.sync_entries_remove_subtree(folder.id, rel) {
warn!(error = ?e, "move: could not drop the sync baseline");
}
if let Some(reason) = changed {
warn!(path = %src.display(), reason,
"move: the local copy changed during the move; keeping it");
return;
}
let removed = if src.is_dir() {
std::fs::remove_dir_all(src)
} else {
std::fs::remove_file(src)
};
if let Err(e) = removed {
warn!(path = %src.display(), error = %e,
"move: could not remove the local copy after the move");
}
}
}
}
/// Bring the destination location level with a move Drive has made.
fn settle_dest(&self, dest: &Place, uid: &NodeUid, name: &str) {
if let Place::Mounted { .. } = dest {
self.invalidate_children_of(uid);
// A lookup that missed before the move may be cached as absent.
self.for_each_mount(|st, notify| {
if let Some(&ino) = st.by_uid.get(uid) {
notify.inval_entry(ino, name.to_string());
}
});
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn folder(id: i64, local_path: &str, mode: &str) -> StoredSyncFolder {
StoredSyncFolder {
id,
local_path: local_path.to_string(),
remote_uid: format!("vol~root{id}"),
remote_share_id: String::new(),
mode: mode.to_string(),
pending_mode: None,
state: "idle".to_string(),
last_sync: 0,
}
}
#[test]
fn mirror_covering_finds_the_mirror_folder_and_the_path_inside_it() {
let folders = vec![folder(1, "/home/u/Docs", "mirror")];
let (found, rel) = mirror_covering(folders, Path::new("/home/u/Docs/a/b.txt")).unwrap();
assert_eq!(found.id, 1);
assert_eq!(rel, "a/b.txt");
}
#[test]
fn mirror_covering_gives_an_empty_path_for_the_folder_itself() {
let folders = vec![folder(1, "/home/u/Docs", "mirror")];
let (_, rel) = mirror_covering(folders, Path::new("/home/u/Docs")).unwrap();
assert_eq!(rel, "");
}
#[test]
fn mirror_covering_prefers_the_deepest_folder() {
let folders = vec![
folder(1, "/home/u/Docs", "mirror"),
folder(2, "/home/u/Docs/Work", "mirror"),
];
let (found, rel) = mirror_covering(folders, Path::new("/home/u/Docs/Work/x")).unwrap();
assert_eq!(found.id, 2);
assert_eq!(rel, "x");
}
#[test]
fn mirror_covering_skips_on_demand_folders_and_name_prefixed_siblings() {
let folders = || {
vec![
folder(1, "/home/u/Docs", "mirror"),
folder(2, "/home/u/Media", "ondemand"),
]
};
assert!(mirror_covering(folders(), Path::new("/home/u/Media/x")).is_none());
assert!(mirror_covering(folders(), Path::new("/home/u/Docs2/x")).is_none());
}
#[test]
fn mirror_covering_refuses_a_path_that_climbs_out() {
let folders = vec![folder(1, "/home/u/Docs", "mirror")];
assert!(mirror_covering(folders, Path::new("/home/u/Docs/../etc")).is_none());
}
#[test]
fn join_rel_names_a_child_of_the_folder_root_without_a_slash() {
assert_eq!(join_rel("", "a"), "a");
assert_eq!(join_rel("x/y", "a"), "x/y/a");
}
#[test]
fn a_local_rename_never_replaces_what_is_already_there() {
let dir = std::env::temp_dir().join(format!("pdfs-relocate-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let (a, b, c) = (dir.join("a"), dir.join("b"), dir.join("c"));
std::fs::write(&a, b"moved").unwrap();
std::fs::write(&b, b"kept").unwrap();
let err = rename_noreplace(&a, &b).unwrap_err();
assert_eq!(err.raw_os_error(), Some(libc::EEXIST));
assert_eq!(std::fs::read(&b).unwrap(), b"kept");
assert!(a.exists());
rename_noreplace(&a, &c).unwrap();
assert!(!a.exists());
assert_eq!(std::fs::read(&c).unwrap(), b"moved");
std::fs::remove_dir_all(&dir).unwrap();
}
}
+17 -6
View File
@@ -610,10 +610,10 @@ impl Core {
/// Proton products (not Drive-only). A remote round-trip; nothing here is /// Proton products (not Drive-only). A remote round-trip; nothing here is
/// cached. /// cached.
pub(crate) fn account_quota(&self) -> CoreResult<(i64, i64)> { pub(crate) fn account_quota(&self) -> CoreResult<(i64, i64)> {
let q = self let q = self.block_on_bounded(self.client.quota()).map_err(|e| {
.rt self.lost_link(&e, "account quota");
.block_on(self.client.quota()) CoreError::from_api(&e, "account quota")
.map_err(|e| CoreError::from_api(&e, "account quota"))?; })?;
*self.quota.lock() = Some((std::time::Instant::now(), q.max_space, q.used_space)); *self.quota.lock() = Some((std::time::Instant::now(), q.max_space, q.used_space));
Ok((q.max_space, q.used_space)) Ok((q.max_space, q.used_space))
} }
@@ -632,11 +632,22 @@ impl Core {
{ {
return Some((max_space, used_space)); return Some((max_space, used_space));
} }
match self.account_quota() { let last = || self.quota.lock().map(|(_, max, used)| (max, used));
if !self.is_online()
|| self
.quota_refreshing
.swap(true, std::sync::atomic::Ordering::AcqRel)
{
return last();
}
let refreshed = self.account_quota();
self.quota_refreshing
.store(false, std::sync::atomic::Ordering::Release);
match refreshed {
Ok(q) => Some(q), Ok(q) => Some(q),
Err(error) => { Err(error) => {
debug!(%error, "account quota refresh failed; serving the last known figures"); debug!(%error, "account quota refresh failed; serving the last known figures");
self.quota.lock().map(|(_, max, used)| (max, used)) last()
} }
} }
} }
+103 -1
View File
@@ -128,6 +128,23 @@ pub(crate) struct WriteHandle {
pub(crate) open_count: usize, pub(crate) open_count: usize,
} }
impl WriteHandle {
/// Account for `[offset, end)` having been written through this handle.
///
/// A write past the end leaves a hole that reads as zeros, exactly like a
/// grow through `setattr`, so the hole is authored too. Left unauthored, a
/// hole below the base size was refilled from the base at commit: a file
/// shrunk and then written past its new end got its old bytes back.
pub(crate) fn record_write(&mut self, offset: u64, end: u64) {
if offset > self.len {
self.written.add(self.len, offset);
}
self.written.add(offset, end);
self.len = self.len.max(end);
self.dirty = true;
}
}
/// A released write whose upload has not happened yet (offline.md Phase 3). /// A released write whose upload has not happened yet (offline.md Phase 3).
/// ///
/// The bytes live in the content cache's staging dir and the intent lives in the /// The bytes live in the content cache's staging dir and the intent lives in the
@@ -783,12 +800,13 @@ impl State {
self.intern_mem_with_access(parent, node, access) self.intern_mem_with_access(parent, node, access)
} }
fn intern_mem_with_access(&mut self, parent: u64, node: Node, access: Access) -> u64 { fn intern_mem_with_access(&mut self, parent: u64, mut node: Node, access: Access) -> u64 {
if let Some(&ino) = self.by_uid.get(&node.uid) { if let Some(&ino) = self.by_uid.get(&node.uid) {
let changed = self let changed = self
.entries .entries
.get(&ino) .get(&ino)
.is_some_and(|entry| entry.access != access); .is_some_and(|entry| entry.access != access);
self.keep_open_write_size(ino, &mut node);
if let Some(e) = self.entries.get_mut(&ino) { if let Some(e) = self.entries.get_mut(&ino) {
e.node = node; e.node = node;
e.parent = parent; e.parent = parent;
@@ -833,6 +851,23 @@ impl State {
/// Allocate (or reuse) a stable inode for a node that came *from* the /// Allocate (or reuse) a stable inode for a node that came *from* the
/// database, which is why nothing is written back. /// database, which is why nothing is written back.
/// Carry the size and mtime of a file open for writing over onto `node`,
/// which is about to replace its entry.
///
/// The open handle is ahead of anything Drive can report until release
/// queues it. Taking the remote's size shrank a SQLite database to its empty
/// create while SQLite held it open, and the kernel then capped every read
/// at that size (docs/BUGS.md B109).
pub(crate) fn keep_open_write_size(&self, ino: u64, node: &mut Node) {
let (Some(aw), Some(entry)) = (self.active_writes.get(&ino), self.entries.get(&ino)) else {
return;
};
if let NodeKind::File { claimed_size, .. } = &mut node.kind {
*claimed_size = Some(aw.len as i64);
node.modification_time = entry.node.modification_time;
}
}
pub(crate) fn intern_from_db(&mut self, parent: u64, node: Node) -> u64 { pub(crate) fn intern_from_db(&mut self, parent: u64, node: Node) -> u64 {
self.intern_mem(parent, node) self.intern_mem(parent, node)
} }
@@ -1863,4 +1898,71 @@ mod tests {
let shared = st.intern(owned_root, shared); let shared = st.intern(owned_root, shared);
assert_eq!(st.entries[&shared].access, Access::Viewer); assert_eq!(st.entries[&shared].access, Access::Viewer);
} }
fn handle(len: u64, base_size: u64) -> WriteHandle {
let path = std::env::temp_dir().join(format!("pdfs-handle-test-{}", std::process::id()));
let file = File::create(&path).unwrap();
let _ = std::fs::remove_file(&path);
WriteHandle {
ino: 2,
uid: uid("file"),
file: Arc::new(file),
path,
written: Intervals::default(),
len,
base_size,
base_mtime: 0,
base_revision_id: None,
dirty: false,
open_count: 1,
}
}
#[test]
fn a_listing_does_not_shrink_a_file_open_for_writing() {
let (mut st, _dir) = state();
let ino = st.intern(0, node("file", "root", "workload.db", false));
let mut h = handle(16384, 0);
h.ino = ino;
st.active_writes.insert(ino, h);
st.set_size(ino, 16384);
st.touch_mtime(ino, 500);
// The remote still holds the empty file the create sealed.
st.intern(0, node("file", "root", "workload.db", false));
assert_eq!(crate::node_size(&st.entries[&ino].node), 16384);
assert_eq!(st.entries[&ino].node.modification_time, 500);
// Once the handle is gone the remote is authoritative again.
st.active_writes.remove(&ino);
st.intern(0, node("file", "root", "workload.db", false));
assert_eq!(crate::node_size(&st.entries[&ino].node), 0);
}
#[test]
fn a_write_past_the_end_authors_the_hole_it_leaves() {
// Opened over a 100-byte base, shrunk to 10, then written at 50: bytes
// 10..50 are zeros now, not the base's bytes 10..50.
let mut h = handle(100, 100);
h.written.clip(10);
h.len = 10;
h.record_write(50, 60);
assert_eq!(h.len, 60);
assert_eq!(
h.written.segments(0, 60),
vec![(0, 10, false), (10, 60, true)],
"only the head still comes from the base"
);
}
#[test]
fn a_write_inside_the_file_leaves_the_rest_to_the_base() {
let mut h = handle(100, 100);
h.record_write(40, 50);
assert_eq!(h.len, 100);
assert_eq!(
h.written.segments(0, 100),
vec![(0, 40, false), (40, 50, true), (50, 100, false)]
);
}
} }
+228 -12
View File
@@ -73,6 +73,9 @@ enum Applied {
Downloaded, Downloaded,
/// A conflict copy was kept. /// A conflict copy was kept.
Conflict, Conflict,
/// The local file to upload was deleted after the scan planned it. There is
/// nothing left to send, and the next pass sees the deletion (B108).
Vanished,
} }
impl Core { impl Core {
@@ -1147,13 +1150,10 @@ impl Core {
Ok(Applied::Dir(rel.clone(), uid)) Ok(Applied::Dir(rel.clone(), uid))
} }
Pending::UploadNew { rel, parent } => { Pending::UploadNew { rel, parent } => {
self.upload_new(folder_id, local_root, rel, parent).await?; self.upload_new(folder_id, local_root, rel, parent).await
Ok(Applied::Uploaded)
} }
Pending::UploadRevision { rel, uid } => { Pending::UploadRevision { rel, uid } => {
self.upload_revision(folder_id, local_root, rel, uid) self.upload_revision(folder_id, local_root, rel, uid).await
.await?;
Ok(Applied::Uploaded)
} }
Pending::Download { Pending::Download {
rel, rel,
@@ -1217,10 +1217,12 @@ impl Core {
local_root: &Path, local_root: &Path,
rel: &str, rel: &str,
parent: &NodeUid, parent: &NodeUid,
) -> Result<(), String> { ) -> Result<Applied, String> {
let path = local_root.join(rel_to_path(rel)); let path = local_root.join(rel_to_path(rel));
let name = base_name(rel); let name = base_name(rel);
let file = std::fs::File::open(&path).map_err(|e| format!("open {rel}: {e}"))?; let Some(file) = open_to_upload(&path, rel)? else {
return Ok(Applied::Vanished);
};
let meta = file.metadata().map_err(|e| format!("stat {rel}: {e}"))?; let meta = file.metadata().map_err(|e| format!("stat {rel}: {e}"))?;
let mtime = system_mtime(&meta); let mtime = system_mtime(&meta);
// Taken from the descriptor being streamed, so it names the content that // Taken from the descriptor being streamed, so it names the content that
@@ -1261,7 +1263,8 @@ impl Core {
self.invalidate_children_of(parent); self.invalidate_children_of(parent);
warn_if_torn(rel, &path, streamed); warn_if_torn(rel, &path, streamed);
self.record_file_baseline(folder_id, rel, Some(streamed), &uid) self.record_file_baseline(folder_id, rel, Some(streamed), &uid)
.await .await?;
Ok(Applied::Uploaded)
} }
/// Upload a changed local file as a new revision of an existing remote node. /// Upload a changed local file as a new revision of an existing remote node.
@@ -1271,9 +1274,11 @@ impl Core {
local_root: &Path, local_root: &Path,
rel: &str, rel: &str,
uid: &NodeUid, uid: &NodeUid,
) -> Result<(), String> { ) -> Result<Applied, String> {
let path = local_root.join(rel_to_path(rel)); let path = local_root.join(rel_to_path(rel));
let file = std::fs::File::open(&path).map_err(|e| format!("open {rel}: {e}"))?; let Some(file) = open_to_upload(&path, rel)? else {
return Ok(Applied::Vanished);
};
let meta = file.metadata().map_err(|e| format!("stat {rel}: {e}"))?; let meta = file.metadata().map_err(|e| format!("stat {rel}: {e}"))?;
let mtime = system_mtime(&meta); let mtime = system_mtime(&meta);
let streamed = LocalSig::from(&meta); let streamed = LocalSig::from(&meta);
@@ -1295,7 +1300,8 @@ impl Core {
.map_err(|e| format!("upload revision {rel}: {e}"))?; .map_err(|e| format!("upload revision {rel}: {e}"))?;
warn_if_torn(rel, &path, streamed); warn_if_torn(rel, &path, streamed);
self.record_file_baseline(folder_id, rel, Some(streamed), uid) self.record_file_baseline(folder_id, rel, Some(streamed), uid)
.await .await?;
Ok(Applied::Uploaded)
} }
/// Download a remote file to its local path (atomically via a temp file), /// Download a remote file to its local path (atomically via a temp file),
@@ -1328,7 +1334,21 @@ impl Core {
.map(|d| d.as_nanos()) .map(|d| d.as_nanos())
.unwrap_or(0) .unwrap_or(0)
)); ));
{ // A file an on-demand mount already read, say one just moved here from
// it, is in the content cache under the same revision key.
let seeded = size > 0 && self.cache.copy_cached_to(uid, mtime, size as u64, &tmp);
if seeded {
std::fs::File::open(&tmp)
.and_then(|f| f.sync_all())
.map_err(|e| {
let _ = std::fs::remove_file(&tmp);
format!("sync tmp {rel}: {e}")
})?;
debug!(
rel,
"sync: placed from the content cache instead of downloading"
);
} else {
let out = std::fs::File::create(&tmp).map_err(|e| format!("create tmp {rel}: {e}"))?; let out = std::fs::File::create(&tmp).map_err(|e| format!("create tmp {rel}: {e}"))?;
// Count the bytes as they land, so the download reports progress like // Count the bytes as they land, so the download reports progress like
// an on-demand hydration does. // an on-demand hydration does.
@@ -1431,6 +1451,23 @@ impl Core {
} }
} }
/// Open a local file for upload, or `None` when it is gone.
///
/// A scan plans uploads from a listing, and the file can be deleted before its
/// upload starts — a build or a test tree cleaned up mid-pass. That is not a
/// failure: the next pass sees the deletion. Counting it as one put the folder
/// in `error` for a file nobody wanted any more (B108).
fn open_to_upload(path: &Path, rel: &str) -> Result<Option<std::fs::File>, String> {
match std::fs::File::open(path) {
Ok(file) => Ok(Some(file)),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
debug!(rel, "sync: file deleted before its upload; skipping");
Ok(None)
}
Err(e) => Err(format!("open {rel}: {e}")),
}
}
/// Say so when a file changed under an upload that was streaming it. /// Say so when a file changed under an upload that was streaming it.
/// ///
/// Purely diagnostic — the correction is already in place, because the baseline /// Purely diagnostic — the correction is already in place, because the baseline
@@ -1602,6 +1639,76 @@ fn walk_local_tree(
Ok(()) Ok(())
} }
/// Why the mirror copy at `rel` holds something Proton Drive does not, or
/// `None` when everything under it is on Drive exactly as it is on disk.
///
/// A move out of a mirror folder removes the local copy once Drive has moved
/// the node, so anything only this disk has would be lost with it: a file not
/// uploaded yet, one edited since, a deletion not carried up yet, or anything
/// the engine never uploads. That is why ignored names count here, and why the
/// walk is its own rather than [`walk_local_tree`], which skips symlinks.
pub(crate) fn mirror_subtree_unsynced(
root: &Path,
rel: &str,
baseline: &HashMap<String, StoredSyncEntry>,
) -> Option<String> {
let mut seen = HashSet::new();
if let Some(reason) = unsynced_local(root, rel, baseline, &mut seen) {
return Some(reason);
}
let prefix = format!("{rel}/");
baseline
.keys()
.filter(|r| *r == rel || r.starts_with(&prefix))
.find(|r| !seen.contains(r.as_str()))
.map(|r| format!("{r} was deleted here but not on Proton Drive yet"))
}
fn unsynced_local(
root: &Path,
rel: &str,
baseline: &HashMap<String, StoredSyncEntry>,
seen: &mut HashSet<String>,
) -> Option<String> {
let path = root.join(rel_to_path(rel));
let meta = match std::fs::symlink_metadata(&path) {
Ok(meta) => meta,
Err(e) => return Some(format!("cannot read {rel}: {e}")),
};
let Some(base) = baseline.get(rel) else {
return Some(format!("{rel} is not on Proton Drive yet"));
};
seen.insert(rel.to_string());
if meta.is_file() {
return (!LocalSig::from(&meta).same_content(&LocalSig::from(base)))
.then(|| format!("{rel} has changes that are not on Proton Drive yet"));
}
if !meta.is_dir() {
return Some(format!("{rel} is not a regular file or folder"));
}
let entries = match std::fs::read_dir(&path) {
Ok(entries) => entries,
Err(e) => return Some(format!("cannot read {rel}: {e}")),
};
for entry in entries {
let name = match entry {
Ok(entry) => entry.file_name(),
Err(e) => return Some(format!("cannot read {rel}: {e}")),
};
let Some(name) = name.to_str() else {
return Some(format!("{rel} holds a name that is not valid UTF-8"));
};
// A download the engine left half done; it is the engine's to clean up.
if name.contains(".pdfs-tmp-") {
continue;
}
if let Some(reason) = unsynced_local(root, &format!("{rel}/{name}"), baseline, seen) {
return Some(reason);
}
}
None
}
/// Publish a conflict copy without overwriting an earlier conflict bearing the /// Publish a conflict copy without overwriting an earlier conflict bearing the
/// same timestamp. The original is removed only after the new copy is durable. /// same timestamp. The original is removed only after the new copy is durable.
fn preserve_conflict_copy(path: &Path, stamp: i64) -> std::io::Result<PathBuf> { fn preserve_conflict_copy(path: &Path, stamp: i64) -> std::io::Result<PathBuf> {
@@ -2155,6 +2262,27 @@ mod tests {
assert_eq!(remote_sig(&e), None); assert_eq!(remote_sig(&e), None);
} }
#[test]
fn a_file_deleted_before_its_upload_is_skipped_not_failed() {
let root = std::env::temp_dir().join(format!("pdfs-b108-{}", std::process::id()));
std::fs::create_dir_all(&root).unwrap();
let gone = root.join("gone.txt");
assert!(open_to_upload(&gone, "gone.txt").unwrap().is_none());
std::fs::write(&gone, b"x").unwrap();
assert!(open_to_upload(&gone, "gone.txt").unwrap().is_some());
// Other open failures still count as errors.
let err = open_to_upload(&gone.join("child"), "gone.txt/child");
assert!(err.unwrap_err().starts_with("open gone.txt/child:"));
let _ = std::fs::remove_dir_all(&root);
let mut o = Outcome::default();
o.record(&Applied::Vanished);
assert!(o.is_empty());
assert_eq!(o.errors, 0);
}
#[test] #[test]
fn outcome_summarises_only_what_it_moved() { fn outcome_summarises_only_what_it_moved() {
let mut o = Outcome::default(); let mut o = Outcome::default();
@@ -2346,6 +2474,94 @@ mod tests {
); );
} }
/// A baseline row recording `path` as it is on disk right now.
fn synced_row(root: &Path, rel: &str) -> (String, StoredSyncEntry) {
let meta = std::fs::symlink_metadata(root.join(rel_to_path(rel))).unwrap();
let sig = LocalSig::from(&meta);
let entry = StoredSyncEntry {
rel_path: rel.to_string(),
remote_uid: Some(format!("vol~{rel}")),
local_mtime: sig.mtime,
local_mtime_ns: sig.mtime_ns,
local_size: sig.size,
remote_rev: None,
remote_hash: None,
};
(rel.to_string(), entry)
}
fn synced_subtree(name: &str) -> (PathBuf, HashMap<String, StoredSyncEntry>) {
let root = sync_test_dir(name);
std::fs::create_dir_all(root.join("dir/sub")).unwrap();
std::fs::write(root.join("dir/a.txt"), b"a").unwrap();
std::fs::write(root.join("dir/sub/b.txt"), b"b").unwrap();
let baseline = ["dir", "dir/sub", "dir/a.txt", "dir/sub/b.txt"]
.into_iter()
.map(|rel| synced_row(&root, rel))
.collect();
(root, baseline)
}
#[test]
fn a_fully_synced_mirror_subtree_may_leave() {
let (root, baseline) = synced_subtree("move-synced");
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert_eq!(reason, None);
}
#[test]
fn a_mirror_subtree_with_a_file_not_uploaded_may_not_leave() {
let (root, baseline) = synced_subtree("move-new-file");
std::fs::write(root.join("dir/sub/new.txt"), b"n").unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/sub/new.txt"));
}
#[test]
fn a_mirror_subtree_with_an_edit_not_uploaded_may_not_leave() {
let (root, baseline) = synced_subtree("move-edit");
std::fs::write(root.join("dir/a.txt"), b"longer now").unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/a.txt"));
}
#[test]
fn a_mirror_subtree_with_a_deletion_not_carried_up_may_not_leave() {
let (root, baseline) = synced_subtree("move-deleted");
std::fs::remove_file(root.join("dir/sub/b.txt")).unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/sub/b.txt"));
}
#[test]
fn a_mirror_subtree_holding_a_symlink_may_not_leave() {
let (root, baseline) = synced_subtree("move-symlink");
std::os::unix::fs::symlink("a.txt", root.join("dir/link")).unwrap();
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert!(reason.unwrap().contains("dir/link"));
}
#[test]
fn a_name_prefixed_sibling_does_not_hold_a_mirror_subtree_back() {
let (root, mut baseline) = synced_subtree("move-sibling");
// Deleted here but not on Drive, yet outside the subtree being moved.
baseline.insert(
"dirt".to_string(),
StoredSyncEntry {
rel_path: "dirt".to_string(),
..baseline["dir/a.txt"].clone()
},
);
let reason = mirror_subtree_unsynced(&root, "dir", &baseline);
std::fs::remove_dir_all(&root).unwrap();
assert_eq!(reason, None);
}
/// The same rule one level up: if the sync root itself cannot be read, the /// The same rule one level up: if the sync root itself cannot be read, the
/// pass sees nothing, and "nothing" must never mean "delete everything". /// pass sees nothing, and "nothing" must never mean "delete everything".
#[test] #[test]
+1
View File
@@ -108,6 +108,7 @@ impl Outcome {
Applied::Uploaded => self.uploaded += 1, Applied::Uploaded => self.uploaded += 1,
Applied::Downloaded => self.downloaded += 1, Applied::Downloaded => self.downloaded += 1,
Applied::Conflict => self.conflicts += 1, Applied::Conflict => self.conflicts += 1,
Applied::Vanished => {}
} }
} }
+10 -1
View File
@@ -668,6 +668,15 @@ fn build_window(app: &adw::Application) {
sync_sidebar(&ui_nav); sync_sidebar(&ui_nav);
match st.visible_child_name().as_deref() { match st.visible_child_name().as_deref() {
Some("browser") => load_browser(&ui_nav), Some("browser") => load_browser(&ui_nav),
// The Albums and Places listings are re-read in place. Loading the
// timeline instead would swap it in under the Albums or Places toggle.
Some("gallery") if !ui_nav.gallery.in_collection() => {
match ui_nav.gallery.view_switch.active() {
VIEW_ALBUMS => load_albums(&ui_nav),
VIEW_PLACES => load_places(&ui_nav),
_ => load_gallery(&ui_nav, false),
}
}
Some("gallery") => load_gallery(&ui_nav, false), Some("gallery") => load_gallery(&ui_nav, false),
// Every visit reads the page again. The rows already on screen stay // Every visit reads the page again. The rows already on screen stay
// up while it does, so arriving costs no "Loading…" flash. // up while it does, so arriving costs no "Loading…" flash.
@@ -1199,7 +1208,7 @@ fn install_window_actions(ui: &Rc<Ui>, window: &adw::ApplicationWindow) {
.application_icon("io.narl.proton-drive-linux") .application_icon("io.narl.proton-drive-linux")
.version(pdfs_core::config::APP_VERSION) .version(pdfs_core::config::APP_VERSION)
.developer_name("Nils Pukropp") .developer_name("Nils Pukropp")
.website("https://github.com/narrrl/proton-drive-linux") .website("https://proton-drive.narl.io")
.issue_url("https://github.com/narrrl/proton-drive-linux/issues") .issue_url("https://github.com/narrrl/proton-drive-linux/issues")
.copyright("© 2026 Nils Pukropp (contact@narl.io)") .copyright("© 2026 Nils Pukropp (contact@narl.io)")
.license_type(gtk4::License::MitX11) .license_type(gtk4::License::MitX11)
+244 -39
View File
@@ -1920,6 +1920,10 @@ pub(crate) fn prompt_rename(ui: &Rc<Ui>, entry: &DirEntry) {
/// Pick a destination folder by browsing Drive's folders, and move `entries` /// Pick a destination folder by browsing Drive's folders, and move `entries`
/// there through the daemon. /// there through the daemon.
///
/// Besides My files the picker offers every synced folder. A move into one is
/// still made on Proton Drive, so nothing is downloaded or uploaded again; it
/// is sent with absolute paths, which is how the daemon tells locations apart.
pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) { pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
if entries.is_empty() { if entries.is_empty() {
return; return;
@@ -1967,6 +1971,7 @@ pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
dialog.set_default_response(Some("move")); dialog.set_default_response(Some("move"));
dialog.set_close_response("cancel"); dialog.set_close_response("cancel");
let mountpoint = ui.dirs.resolved_mountpoint(&ui.dirs.load_config());
let picker = Rc::new(MovePicker { let picker = Rc::new(MovePicker {
ui: ui.clone(), ui: ui.clone(),
dialog: dialog.clone(), dialog: dialog.clone(),
@@ -1974,28 +1979,93 @@ pub(crate) fn prompt_move(ui: &Rc<Ui>, entries: Vec<DirEntry>) {
list, list,
up: up.clone(), up: up.clone(),
sources: sources.clone(), sources: sources.clone(),
folder: RefCell::new(String::new()), folder: RefCell::new(MoveTarget::Locations),
generation: Cell::new(0), generation: Cell::new(0),
}); });
let picker_up = picker.clone(); let picker_up = picker.clone();
up.connect_clicked(move |_| { up.connect_clicked(move |_| {
let folder = picker_up.folder.borrow().clone(); let parent = picker_up.folder.borrow().parent();
let parent = folder.rfind('/').map(|i| &folder[..i]).unwrap_or_default(); picker_up.open(parent);
picker_up.open(parent.to_string());
}); });
// Start where the browser is: moving is most often one level up or down. // Start where the browser is: moving is most often one level up or down.
picker.open(ui.browser.path.borrow().clone()); picker.open(MoveTarget::Drive(ui.browser.path.borrow().clone()));
let ui = ui.clone(); let ui = ui.clone();
let picker_done = picker.clone(); let picker_done = picker.clone();
dialog.connect_response(None, move |_, resp| { dialog.connect_response(None, move |_, resp| {
if resp == "move" { if resp != "move" {
run_bulk_move(&ui, sources.clone(), picker_done.folder.borrow().clone()); return;
}
match picker_done.folder.borrow().clone() {
MoveTarget::Locations => {}
MoveTarget::Drive(folder) => run_bulk_move(&ui, sources.clone(), folder),
MoveTarget::Synced { path, .. } => {
let absolute = sources
.iter()
.map(|rel| mountpoint.join(rel).to_string_lossy().into_owned())
.collect();
run_bulk_move(&ui, absolute, path);
}
} }
}); });
dialog.present(ui_window(&picker.ui).as_ref()); dialog.present(ui_window(&picker.ui).as_ref());
} }
/// Where the Move dialog is looking, and the destination if the user confirms.
#[derive(Clone, Debug, PartialEq)]
pub(crate) enum MoveTarget {
/// The list of places to move to: My files and each synced folder.
Locations,
/// A folder in My files, mountpoint-relative; empty for My files itself.
Drive(String),
/// A folder inside a synced folder, as an absolute local path. `root` is
/// the synced folder. A mirror folder is a plain directory, read from disk.
Synced {
root: String,
path: String,
mirror: bool,
},
}
impl MoveTarget {
/// One level up. The top of a location leads back to the list of them.
pub(crate) fn parent(&self) -> MoveTarget {
match self {
MoveTarget::Locations => MoveTarget::Locations,
MoveTarget::Drive(folder) if folder.is_empty() => MoveTarget::Locations,
MoveTarget::Drive(folder) => MoveTarget::Drive(
folder
.rfind('/')
.map(|i| folder[..i].to_string())
.unwrap_or_default(),
),
MoveTarget::Synced { root, path, .. } if path == root => MoveTarget::Locations,
MoveTarget::Synced { root, path, mirror } => MoveTarget::Synced {
root: root.clone(),
path: Path::new(path)
.parent()
.map(|p| p.to_string_lossy().into_owned())
.unwrap_or_else(|| root.clone()),
mirror: *mirror,
},
}
}
/// `name` inside this folder, in the same form.
fn child(&self, name: &str) -> MoveTarget {
match self {
MoveTarget::Locations => MoveTarget::Locations,
MoveTarget::Drive(folder) if folder.is_empty() => MoveTarget::Drive(name.to_string()),
MoveTarget::Drive(folder) => MoveTarget::Drive(format!("{folder}/{name}")),
MoveTarget::Synced { root, path, mirror } => MoveTarget::Synced {
root: root.clone(),
path: Path::new(path).join(name).to_string_lossy().into_owned(),
mirror: *mirror,
},
}
}
}
/// The Move dialog's folder browser. /// The Move dialog's folder browser.
struct MovePicker { struct MovePicker {
ui: Rc<Ui>, ui: Rc<Ui>,
@@ -2006,71 +2076,81 @@ struct MovePicker {
/// What is being moved, mountpoint-relative. /// What is being moved, mountpoint-relative.
sources: Vec<String>, sources: Vec<String>,
/// The folder on show, and the destination if the user confirms now. /// The folder on show, and the destination if the user confirms now.
folder: RefCell<String>, folder: RefCell<MoveTarget>,
/// Drops a listing that arrives after the user went elsewhere. /// Drops a listing that arrives after the user went elsewhere.
generation: Cell<u64>, generation: Cell<u64>,
} }
impl MovePicker { impl MovePicker {
fn open(self: &Rc<Self>, folder: String) { fn open(self: &Rc<Self>, folder: MoveTarget) {
let generation = self.generation.get().wrapping_add(1); let generation = self.generation.get().wrapping_add(1);
self.generation.set(generation); self.generation.set(generation);
self.location.set_label(if folder.is_empty() { let (label, tooltip) = match &folder {
"Proton Drive" MoveTarget::Locations => (gettext("Locations"), String::new()),
} else { MoveTarget::Drive(rel) if rel.is_empty() => (gettext("My files"), String::new()),
&folder MoveTarget::Drive(rel) => (rel.clone(), rel.clone()),
}); MoveTarget::Synced { path, .. } => (path.clone(), path.clone()),
self.location.set_tooltip_text(Some(&folder)); };
self.up.set_sensitive(!folder.is_empty()); self.location.set_label(&label);
self.dialog self.location.set_tooltip_text(Some(&tooltip));
.set_response_enabled("move", move_target_allowed(&self.sources, &folder)); self.up
.set_sensitive(!matches!(folder, MoveTarget::Locations));
let allowed = match &folder {
MoveTarget::Locations => false,
MoveTarget::Drive(rel) => move_target_allowed(&self.sources, rel),
// What is moved is in My files, so it cannot be in a synced folder.
MoveTarget::Synced { .. } => true,
};
self.dialog.set_response_enabled("move", allowed);
*self.folder.borrow_mut() = folder.clone(); *self.folder.borrow_mut() = folder.clone();
while let Some(row) = self.list.first_child() { while let Some(row) = self.list.first_child() {
self.list.remove(&row); self.list.remove(&row);
} }
self.list.append(&picker_note(&gettext("Loading…"))); self.list.append(&picker_note(&gettext("Loading…")));
let rx = spawn_request(
self.ui.dirs.control_socket(),
Request::ListDir {
path: folder.clone(),
},
);
let picker = self.clone(); let picker = self.clone();
glib::spawn_future_local(async move { glib::spawn_future_local(async move {
let result = rx.recv().await; let listed = match &folder {
MoveTarget::Locations => picker.locations().await,
MoveTarget::Drive(rel) => picker.drive_folders(&folder, rel.clone()).await,
MoveTarget::Synced {
path, mirror: true, ..
} => local_folders(&folder, path.clone()).await,
MoveTarget::Synced { path, .. } => {
picker.drive_folders(&folder, path.clone()).await
}
};
if picker.generation.get() != generation { if picker.generation.get() != generation {
return; return;
} }
while let Some(row) = picker.list.first_child() { while let Some(row) = picker.list.first_child() {
picker.list.remove(&row); picker.list.remove(&row);
} }
let Ok(Ok(Response::Entries { entries })) = result else { let Some(rows) = listed else {
picker picker
.list .list
.append(&picker_note(&gettext("Couldn't read this folder."))); .append(&picker_note(&gettext("Couldn't read this folder.")));
return; return;
}; };
let mut folders: Vec<DirEntry> = entries.into_iter().filter(|e| e.is_dir).collect();
folders.sort_by_key(|e| e.name.to_lowercase());
let mut shown = 0; let mut shown = 0;
for entry in folders { for (title, subtitle, target) in rows {
let path = if folder.is_empty() {
entry.name.clone()
} else {
format!("{folder}/{}", entry.name)
};
// A folder cannot go inside itself, so do not offer to open it. // A folder cannot go inside itself, so do not offer to open it.
if picker.sources.contains(&path) { if let MoveTarget::Drive(path) = &target
&& picker.sources.contains(path)
{
continue; continue;
} }
let row = adw::ActionRow::builder() let row = adw::ActionRow::builder()
.title(glib::markup_escape_text(&entry.name).as_str()) .title(glib::markup_escape_text(&title).as_str())
.activatable(true) .activatable(true)
.build(); .build();
if let Some(subtitle) = subtitle {
row.set_subtitle(glib::markup_escape_text(&subtitle).as_str());
}
row.add_prefix(&gtk4::Image::from_icon_name("folder-symbolic")); row.add_prefix(&gtk4::Image::from_icon_name("folder-symbolic"));
row.add_suffix(&gtk4::Image::from_icon_name("go-next-symbolic")); row.add_suffix(&gtk4::Image::from_icon_name("go-next-symbolic"));
let open = picker.clone(); let open = picker.clone();
row.connect_activated(move |_| open.open(path.clone())); row.connect_activated(move |_| open.open(target.clone()));
picker.list.append(&row); picker.list.append(&row);
shown += 1; shown += 1;
} }
@@ -2081,6 +2161,92 @@ impl MovePicker {
} }
}); });
} }
/// My files and every synced folder, as rows of the top level.
async fn locations(&self) -> Option<Vec<PickerRow>> {
let mut rows = vec![(gettext("My files"), None, MoveTarget::Drive(String::new()))];
let rx = spawn_request(self.ui.dirs.control_socket(), Request::ListSyncFolders);
let Ok(Ok(Response::SyncFolders { mut items })) = rx.recv().await else {
// My files alone is still somewhere to move to.
return Some(rows);
};
items.sort_by_key(|f| f.local_path.to_lowercase());
for folder in items {
let name = Path::new(&folder.local_path)
.file_name()
.map(|n| n.to_string_lossy().into_owned())
.unwrap_or_else(|| folder.local_path.clone());
rows.push((
name,
Some(folder.local_path.clone()),
MoveTarget::Synced {
root: folder.local_path.clone(),
path: folder.local_path,
mirror: folder.mode == "mirror",
},
));
}
Some(rows)
}
/// The folders in a mounted folder: `folder` is mountpoint-relative for My
/// files and absolute for an on-demand synced folder.
async fn drive_folders(&self, here: &MoveTarget, folder: String) -> Option<Vec<PickerRow>> {
let rx = spawn_request(
self.ui.dirs.control_socket(),
Request::ListDir {
path: folder.clone(),
},
);
let Ok(Ok(Response::Entries { entries })) = rx.recv().await else {
return None;
};
let mut names: Vec<String> = entries
.into_iter()
.filter(|e| e.is_dir)
.map(|e| e.name)
.collect();
names.sort_by_key(|name| name.to_lowercase());
Some(
names
.into_iter()
.map(|name| {
let target = here.child(&name);
(name, None, target)
})
.collect(),
)
}
}
/// One row of the picker: its title, an optional subtitle, and where it leads.
type PickerRow = (String, Option<String>, MoveTarget);
/// The folders in a mirror folder's directory `path`, read from disk off the
/// main thread. A mirror folder has no mount for `ListDir` to read.
async fn local_folders(here: &MoveTarget, path: String) -> Option<Vec<PickerRow>> {
let names = gio::spawn_blocking(move || {
let mut names: Vec<String> = std::fs::read_dir(&path)
.ok()?
.flatten()
.filter(|e| e.file_type().is_ok_and(|t| t.is_dir()))
.filter_map(|e| e.file_name().into_string().ok())
.filter(|name| !name.contains(".pdfs-tmp-"))
.collect();
names.sort_by_key(|name| name.to_lowercase());
Some(names)
})
.await
.ok()??;
Some(
names
.into_iter()
.map(|name| {
let target = here.child(&name);
(name, None, target)
})
.collect(),
)
} }
fn picker_note(text: &str) -> gtk4::ListBoxRow { fn picker_note(text: &str) -> gtk4::ListBoxRow {
@@ -2786,8 +2952,8 @@ fn listing_summary(files: usize, folders: usize, file_bytes: u64) -> String {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{ use super::{
hit_location, listing_summary, move_target_allowed, show_thumbnail_build_progress, MoveTarget, hit_location, listing_summary, move_target_allowed,
sort_entries, show_thumbnail_build_progress, sort_entries,
}; };
use pdfs_core::config::{FileSort, FilesView}; use pdfs_core::config::{FileSort, FilesView};
use pdfs_core::control::{DirEntry, ThumbnailBuildStatus}; use pdfs_core::control::{DirEntry, ThumbnailBuildStatus};
@@ -2880,4 +3046,43 @@ mod tests {
status.running = false; status.running = false;
assert!(!show_thumbnail_build_progress(&status)); assert!(!show_thumbnail_build_progress(&status));
} }
#[test]
fn move_picker_climbs_from_a_location_top_to_the_list_of_locations() {
assert_eq!(
MoveTarget::Drive(String::new()).parent(),
MoveTarget::Locations
);
let root = MoveTarget::Synced {
root: "/home/u/Docs".into(),
path: "/home/u/Docs".into(),
mirror: true,
};
assert_eq!(root.parent(), MoveTarget::Locations);
}
#[test]
fn move_picker_climbs_one_folder_at_a_time_inside_a_location() {
assert_eq!(
MoveTarget::Drive("a/b".into()).parent(),
MoveTarget::Drive("a".into())
);
assert_eq!(
MoveTarget::Drive("a".into()).parent(),
MoveTarget::Drive(String::new())
);
let inner = MoveTarget::Synced {
root: "/home/u/Docs".into(),
path: "/home/u/Docs/x/y".into(),
mirror: false,
};
assert_eq!(
inner.parent(),
MoveTarget::Synced {
root: "/home/u/Docs".into(),
path: "/home/u/Docs/x".into(),
mirror: false,
}
);
}
} }
+9 -7
View File
@@ -262,20 +262,22 @@ pub(crate) fn build_locations_page(history: &gtk4::Widget) -> (gtk4::Widget, Loc
(frame.upcast(), widgets) (frame.upcast(), widgets)
} }
/// `child` in a clamp that scrolls vertically only, with page margins. /// `child` across the page's full width, scrolling vertically only, with page
/// margins.
fn scrolled_column(child: &impl IsA<gtk4::Widget>) -> gtk4::ScrolledWindow { fn scrolled_column(child: &impl IsA<gtk4::Widget>) -> gtk4::ScrolledWindow {
let clamp = adw::Clamp::builder().child(child).build(); let column = gtk4::Box::new(gtk4::Orientation::Vertical, 0);
clamp.set_margin_top(18); column.append(child);
clamp.set_margin_bottom(18); column.set_margin_top(18);
clamp.set_margin_start(18); column.set_margin_bottom(18);
clamp.set_margin_end(18); column.set_margin_start(18);
column.set_margin_end(18);
// Never scroll sideways: a location's subtitle holds a full path, and letting // Never scroll sideways: a location's subtitle holds a full path, and letting
// the row grow to fit one pushes the controls at its end off screen. // the row grow to fit one pushes the controls at its end off screen.
// Constrained, the path ellipsizes and the controls stay reachable. // Constrained, the path ellipsizes and the controls stay reachable.
gtk4::ScrolledWindow::builder() gtk4::ScrolledWindow::builder()
.vexpand(true) .vexpand(true)
.hscrollbar_policy(gtk4::PolicyType::Never) .hscrollbar_policy(gtk4::PolicyType::Never)
.child(&clamp) .child(&column)
.build() .build()
} }
+5 -1
View File
@@ -1069,7 +1069,11 @@ pub(crate) fn repaint_transfers(ui: &Rc<Ui>, items: &[TransferItem], jobs: &[Job
let row_box = gtk4::Box::new(gtk4::Orientation::Vertical, 4); let row_box = gtk4::Box::new(gtk4::Orientation::Vertical, 4);
row_box.set_margin_top(8); row_box.set_margin_top(8);
row_box.set_margin_bottom(8); row_box.set_margin_bottom(8);
let label = gtk4::Label::builder().halign(gtk4::Align::Start).build(); // Ellipsized so a long file name cannot widen the page.
let label = gtk4::Label::builder()
.halign(gtk4::Align::Start)
.ellipsize(gtk4::pango::EllipsizeMode::Middle)
.build();
label.add_css_class("dim-label"); label.add_css_class("dim-label");
let bar = gtk4::ProgressBar::new(); let bar = gtk4::ProgressBar::new();
row_box.append(&label); row_box.append(&label);
+13
View File
@@ -539,6 +539,19 @@ A restore is expanded before it is sent (`expand_restore`, `pdfs-fuse/src/lib.rs
The trash view's restore and permanent-delete use the SDK's **streaming** variants (`restore_nodes_streaming` / `delete_nodes_streaming`) and apply local state per node as each batch lands rather than after the last one. For a permanent delete — which is irreversible — that means a daemon interrupted mid-batch has forgotten exactly the nodes the server destroyed, no more and no fewer. Both report per-node failures and count only what actually succeeded; only a batch where nothing succeeded is an error. The trash view's restore and permanent-delete use the SDK's **streaming** variants (`restore_nodes_streaming` / `delete_nodes_streaming`) and apply local state per node as each batch lands rather than after the last one. For a permanent delete — which is irreversible — that means a daemon interrupted mid-batch has forgotten exactly the nodes the server destroyed, no more and no fewer. Both report per-node failures and count only what actually succeeded; only a batch where nothing succeeded is an error.
### Moves Between Locations
**Files**: [`relocate.rs`](../crates/pdfs-fuse/src/relocate.rs), [`control.rs`](../crates/pdfs-fuse/src/control.rs), [`browser.rs`](../crates/pdfs-gui/src/app/pages/browser.rs)
The primary mount and every on-demand folder are separate FUSE sessions, and a mirror folder has no session at all. The kernel answers `rename(2)` between two mounts with `EXDEV` before the daemon hears of it, so `mv` copies and deletes. On Proton Drive the same move is one `move_node`, because My files and this device's folders are on the main volume.
`Request::Move` with two absolute paths is therefore resolved by `Core::move_between` against every location: a live mount through `rooted_at`, otherwise the mirror folder with the longest `local_path` prefix, whose uids come from the `sync_entry` baseline. Two paths in one inode space still go to `move_to`. Relative paths keep their old meaning.
- **Nothing that exists only on this disk may be lost.** A mounted source is refused while it or anything under it has a queued op or a file open for writing. A mirror source is refused unless every file under it matches its baseline and every baseline row still has its file (`mirror_subtree_unsynced`); ignored names and symlinks count as unsynced, because the local copy is removed after the move.
- **Mirror to mirror renames along.** The local copy is renamed first, with `RENAME_NOREPLACE` so nothing that appeared at the destination meanwhile is overwritten, and its baseline rows are moved with `sync_entries_move`, so neither folder's next pass has anything to do. If `move_node` fails, the rename is undone. `EXDEV` between two filesystems falls back to the check above, then drops the source copy.
- **A dropped mirror copy is checked twice.** The check runs again just before the source copy is deleted. A file written into it during the move keeps the whole copy; its baseline is gone, so the next pass uploads it as new.
- **Sync passes are held off.** Every mirror folder involved is locked through `sync_lock`, by ascending id, with a five-second bound. A folder that is still busy is reported rather than waited on.
- **The local side is levelled afterwards.** A mounted source forgets the node in every inode space and notifies the kernel. A mounted destination has its listing cleared. A mirror destination that received nothing locally gets a reconcile, which downloads the node. That download first asks `ContentCache::copy_cached_to` for the file: a whole cached blob, or every block of it, is copied into place instead of fetched, so a move out of an on-demand folder costs no traffic for what was already opened there.
### Open-for-Write Deferral for Mirror Sync ### Open-for-Write Deferral for Mirror Sync
**File**: [`sync.rs`](../crates/pdfs-fuse/src/sync.rs) **File**: [`sync.rs`](../crates/pdfs-fuse/src/sync.rs)
+349
View File
@@ -12,6 +12,355 @@ Conventions:
--- ---
## B112 — Folders shared by the same person disappear from Shared with me
**Status:** Fixed (unverified).
**Found:** 2026-09-30, reported in #25. An account with several accepted shares from one other
account listed only some of them, in both the control listing and the FUSE mount. The journal
showed `skipping undecryptable node ... failed to decrypt: missing key`. Each missing folder
opened on the Proton Drive website and resolved when the SDK was asked for it alone.
**Where:** `proton-drive-rs` 0.7.1, `enumerate_nodes_detail`, which serves both
`enumerate_nodes` and `enumerate_nodes_light`. The app hit it in `list_shared_with_me`
(`crates/pdfs-fuse/src/sharing.rs`) and in the shared-root listing in
`crates/pdfs-fuse/src/lib.rs`.
**Cause.** The SDK cached parent keys per batch by the link's parent id. Every share root has
no parent, so all roots on one volume shared the `None` entry, and the roots after the first were
decrypted with the first root's share key (proton-sdk-rs#2).
**Fix.** Require `proton-drive-rs` 0.7.2, which names each key `Parent(folder)` or `Root(link)`.
No app code changes. The app-side workaround in #25, one SDK call per root, is not needed.
Unverified: no account with several shares from one owner has listed through 0.7.2 yet.
## B111 — Renaming a file back right after renaming it fails with EIO
**Status:** Fixed (unverified).
**Found:** 2026-09-29, by the acceptance case "unusual but legal names", on My files and on the
on-demand folder. The case renames each file to `renamed-N` and back at once. One rename back per
run failed with `EIO`, and the journal showed `rename failed ... proton api error
InvalidRequirements (http 422): This file or folder was out of date, rename failed.`
**Where:** every remote rename: `serve_rename` in `crates/pdfs-fuse/src/filesystem.rs`,
`drain_rename_in_place` in `crates/pdfs-fuse/src/drain.rs`, and the control `rename` in
`crates/pdfs-fuse/src/lib.rs`.
**Cause.** The SDK's `rename_node` reads the node's details from the server and sends its current
name hash as `OriginalHash`. Right after a rename of the same node, that read can still return the
old hash. Drive then refuses the request as out of date. The daemon reported this at once as
`EIO`, although the same request succeeds a moment later.
**Fix.** All remote renames go through `Core::rename_remote`. It retries a rename that fails with
`InvalidRequirements` up to 20 times, half a second apart, the same window the move after a
rename already uses. Any other error still returns at once.
## B110 — A read that settles a provisional size deadlocks on its own page lock
**Status:** Fixed (unverified).
**Found:** 2026-09-29, by the acceptance case "application workloads", which timed out on My
files and on the on-demand folder on a fast network. The journal showed `a fuse worker has held
the same job for a long time worker=pdfs-fuse-8 job=read` growing past 700 s with no network
error. `/proc/<pid>/task/*/wchan` put that worker, and one more, in `__folio_lock`, while the
daemon had no file on any mount open.
**Where:** `crates/pdfs-fuse/src/lib.rs` (`Core::apply_size_upgrade`).
**Cause.** B100 made `read` settle a provisional size before it fetches, through
`resolve_provisional_size` and `apply_size_upgrade`. That function tells the kernel about the new
size with `inval_inode(ino, 0, 0)`, which also drops the file's page cache. The kernel holds the
page lock of the read being served until the daemon replies. The invalidation waits for the same
lock, on the thread that has to send the reply, so neither ever finishes. The process that
issued the read hangs, and so does anything else that touches those pages.
**Fix.** The size upgrade invalidates attributes only (`offset -1`). A size upgrade does not
change content, so the page cache has nothing stale to drop. A daemon already stuck this way
needs a restart.
## B109 — A re-listing shrinks a file that is open for writing, and SQLite reads it as malformed
**Status:** Fixed (unverified).
**Found:** 2026-09-29, by the acceptance case "application workloads" on My files. It failed
several runs in a row with `database disk image is malformed` at the `SELECT COUNT(*)` right
after a 500-row insert, and the daemon logged nothing. A diagnosis added to the case showed
`workload.db` at 0 bytes while SQLite still held it open with 16 KiB written.
**Where:** `crates/pdfs-fuse/src/state.rs` (`State::keep_open_write_size`, called from
`intern_mem_with_access`), the local-uid remap in `crates/pdfs-fuse/src/drain.rs`, and the size
upgrade in `crates/pdfs-fuse/src/lib.rs`.
**Cause.** An online create seals an empty file on Drive. Until the last close, the written
bytes live only in the open handle, and the entry's size is the handle's. Any re-listing of the
parent re-interned the node from Drive, which replaced that size with 0. Such re-listings come
from event echoes and sibling creates, and SQLite's journal is one. The kernel then got size 0
from `getattr` and ended SQLite's reads at EOF without asking the daemon, so the database read
back as empty pages. Queued writes were already protected (B11); open handles were not.
**Fix.** A node that has an open write handle keeps the handle's size and its local mtime when
a listing, a size upgrade or a create landing replaces its entry. After the last close, the
queued write takes over as before.
## B108 — A file deleted during a mirror pass puts the sync folder in `error`
**Status:** Fixed (unverified).
**Found:** 2026-09-29, by the account run of `scripts/fuse-acceptance.sh --quick`. It stopped with
`sync folder … entered error` before the managed-move phase. At the same time the journal
showed `sync: step failed; continuing error=open …/workloads/tree.tar: No such file or directory`
for eleven files. The contract suite had removed its `workloads` tree while the mirror pass
was still uploading it.
**Where:** `crates/pdfs-fuse/src/sync.rs` (`upload_new`, `upload_revision`, `open_to_upload`).
**Cause.** A pass plans its uploads from a scan and runs them later. When a planned file was
deleted in between, opening it failed with `NotFound`. That counted as a failed step, so the pass
ended in `error` and logged "item(s) failed; will retry" for files nobody wanted any more.
**Fix.** An upload whose local file is gone when it starts is skipped (`Applied::Vanished`) and
does not count as an error. The next pass sees the deletion and handles it like any other.
Other open failures are still errors.
## B107 — One read on a stalled connection hangs the reading program for half an hour
**Status:** Fixed (unverified).
**Found:** 2026-09-29, by the acceptance case "application workloads" on My files, which hit its
180 s timeout. The journal showed `a fuse worker has held the same job for a long time
worker=pdfs-fuse-10 job=read` from `age_secs=124` up to `age_secs=2059`. Event polls in the
same window failed with `HTTP transport error` after 91-103 s.
**Where:** `crates/pdfs-fuse/src/reads.rs` (`Core::read_range_remote`, `READ_FETCH_TIMEOUT`).
**Cause.** The SDK bounds each request (30 s for the API, 300 s for a storage block) and retries
failed ones, but nothing bounds a whole read. A read can need a block table, several blocks and a
repair, each with its own retries. On a connection that stalled without dropping, one `read`
held its FUSE worker for about 34 minutes, and the program that issued it could not be
interrupted for that time.
**Fix.** The network part of a read now has a 120 s deadline. When it passes, the read fails
with `EIO`, the file's revision reader is dropped so the next read starts fresh, and blocks that
already arrived stay cached for the retry. A single read is at most a few 4 MiB blocks, so this
only fails a read on a link slower than about 70 KB/s. A block repair that has to download a
large file whole can also hit the deadline on a slow link; pinning the file avoids that.
The first version of this fix built the timer on the FUSE worker thread, outside the runtime.
Every read that went to the network panicked with `there is no reactor running`, and the kernel
got `EIO`. The next account run failed 11 cases on My files this way. The timer is now built
inside `block_on`, and a unit test runs that pattern from a thread without a runtime.
---
## B106 — Saving over a file that was just read fails with "already exists"
**Status:** Fixed (unverified).
**Found:** 2026-09-29, by the acceptance case "application workloads" on an on-demand folder,
after installing the B104 fix: `OSError: [Errno 5]` from `os.replace` in the editor-save loop,
at the third save. The journal had `rename failed … AlreadyExists (http 422): A file or folder
with name already exists` from `rename_node`.
**Where:** `crates/pdfs-fuse/src/lib.rs` (`Core::remove_replaced`,
`Core::withdraw_queued_ops`), `crates/pdfs-fuse/src/filesystem.rs` (`trash_child`,
`finish_release`), `crates/pdfs-fuse/src/drain.rs` (`Core::drain_revision`).
**Cause.** A regression from B104. The save loop writes a temporary file, renames it over the
document, and reads the document back. `close()` returns before the daemon sees the release, so
at the next save the replaced file still counts as open. B104 then left its queued upload in
the queue until the final close, and the drain could send it after the file had been trashed.
The next rename onto the name was refused. `unlink` of an open file had kept its queued upload
the same way.
**Fix.** A file trashed or replaced while open now loses its queued ops at once, and an upload
already under way is cancelled. Only the staged bytes stay, because the open handle reads them,
and they are deleted at the final close. A drain worker that claimed the op before it was
withdrawn checks that the op still exists before it sends any bytes.
---
## B105 — A file deleted while its upload is in flight comes back as a conflict copy
**Status:** Fixed (unverified).
**Found:** 2026-09-29, in the daemon journal of an acceptance run on the account. During the
SQLite workload two `queued write conflicts; keeping a conflict copy` warnings named files called
`recovered-XrPUSW5W` and `recovered-JbZM_pWK`, with the reason `the file was trashed remotely`.
Each was followed by `pending upload failed; will retry` and `No such file or directory`.
**Where:** `crates/pdfs-fuse/src/drain.rs` (`Core::drain_revision`),
`crates/pdfs-core/src/db/ops.rs` (`Db::op_exists`).
**Cause.** SQLite writes, syncs and deletes its journal file within milliseconds. The sync
queues an upload of the journal. The delete trashes the node and drops that queued upload, but a
drain worker that already holds the op carries on. It then asks the server about the node,
finds it trashed, and treats the trash as another device's change: it tries to upload the bytes
as a conflict copy. The tree has already forgotten the node, so the copy is named `recovered-…`
and placed in the Drive root. Here the blob was already gone and the attempt failed, but with
the blob still on disk the copy would land.
**Fix.** Before keeping a conflict copy, the drain checks that its op is still queued. When the
op is gone, the write was withdrawn by the delete, or superseded, while the worker held it, and
it is dropped.
---
## B104 — Reading a file that was replaced by a rename fails with `EIO`
**Status:** Fixed (unverified).
**Found:** 2026-09-29, by the acceptance case "rename patterns" on My files and on an on-demand
folder: `OSError: [Errno 5] Input/output error` at
`os.pread(reader, 64, 0)` after `os.replace(replacement, target)` with `reader` open on `target`.
The mirror folder, a local disk, passed.
**Where:** `crates/pdfs-fuse/src/lib.rs` (`Core::remove_replaced`).
**Cause.** A rename over an existing file first trashes that file, because Drive has no atomic
replace. That step forgot the replaced inode outright and evicted its cache and reader, even
when a handle was still open on it. The next read through that handle found no inode and
failed. `rename(2)`, like `unlink(2)`, keeps a replaced file readable through the handles
already open on it. The unlink path (`trash_child`) already did this, but the rename path
did not.
**Fix.** `remove_replaced` now handles an open victim as `unlink` does. It keeps the inode as
an unlinked entry, and leaves its cache and reader to the final close. That applies both to a
node the server knows and to one whose create is still queued. Its queued ops at first also
waited for the final close; B106 moved them back to the trash.
---
## B103 — One stuck trash refresh freezes the trash listing until the daemon restarts
**Status:** Fixed (unverified). The hang itself has not been reproduced on purpose.
**Found:** 2026-09-29, while timing the acceptance suite's wide-directory case. Its cleanup trashed
the test folder, then waited for it to show up in the trash so it could delete it permanently. It
never did. From 06:55 UTC on, every trash listing logged `trash refresh still running; answering
with what has materialized` and returned the same 362 entries. No `trash refresh: enumerated` line
followed, so the first server call of the refresh never returned.
**Where:** `crates/pdfs-fuse/src/lib.rs` (`Core::refresh_trash`, `within_trash_call`).
**Cause.** Trash refreshes are single-flight: a request joins the refresh already running instead
of starting another. The refresh awaited its server calls with no deadline, so one call that
never returned kept the flag set forever. Every later request joined it, waited out its 20 s
budget, and answered with the stale listing.
**Fix.** Each server call of a refresh (`enumerate_trash_node_uids`, and each `enumerate_nodes`
chunk) fails after `TRASH_CALL_TIMEOUT` (120 s). The refresh then ends with an error, clears the
flag, and the next request starts a fresh one.
---
## B102 — Removing a folder leaves the queued uploads of the files inside it
**Status:** Fixed (unverified). No daemon carrying the change has run the acceptance suite yet.
**Found:** 2026-09-29, after the account run that found B101. Its cleanup removed both test roots
and reported "nothing left behind", but `pdfs sync queue` still listed two `revision` ops for
`pdfs-acceptance-…/boundaries/reshaped`, each holding about 8 MiB of staged bytes. They retried
against the trash forever ("the share has not been writable for 300s").
**Where:** `crates/pdfs-core/src/db/ops.rs` (`drop_doomed_ops`, used by
`Db::replace_ops_with_trash` and `Db::delete_ops_for_uid`), `crates/pdfs-fuse/src/lib.rs`
(`Core::delete`, `Core::release_dropped_ops`).
**Cause.** Two gaps.
- **The queue could not see which files a folder holds.** Removing a node dropped the ops for it
and for anything *queued* beneath it, found through `pending_op.parent_uid`. Only `create` and
`mkdir` ops carry a parent, so a `revision` of a file that already existed remotely was never
found, however deep in the removed folder it sat.
- **`pdfs rm` dropped nothing at all.** The control socket's delete trashed the node remotely but
left every op queued for it, unlike an unlink through the mount.
**Fix.** Removal also follows the node tree (`nodes.parent_uid`) and drops the `revision` ops of
every file below the node, with their blobs, pending entries and in-flight uploads. Queued renames
below it are kept: one that moved a node into the folder still has to land for the node to go to
the trash with it. `pdfs rm` now drops the queued ops once the remote trash succeeds. The
acceptance janitor lists the daemon queue after cleanup and reports any op still naming one of its
roots as a leftover.
Regression test: `trashing_a_folder_drops_the_queued_revisions_of_files_below_it`.
---
## B101 — A shrunk file written past its end gets old bytes back, or fails with `EIO`; a write during a create's upload is lost
**Status:** Fixed (unverified). No daemon carrying the change has run the acceptance suite yet.
**Found:** 2026-09-29, by the new acceptance case "block boundaries and overwrites that change
the block count". It failed with `Input/output error` in My files and in an on-demand folder, every
time. The journal had `the revision's block table disagrees with the node size ... fsize=8388611
reader_size=0`, then `pending upload failed; will retry ... gap-fill from base failed: Errno(5)`
on every retry.
**Where:** `crates/pdfs-fuse/src/state.rs` (`WriteHandle::record_write`),
`crates/pdfs-fuse/src/drain.rs` (`Core::drain_local_node`, `Core::retire_create`),
`crates/pdfs-fuse/src/lib.rs` (`Core::enqueue_staged_write`),
`crates/pdfs-core/src/db/ops.rs` (`Db::finish_create`).
**Cause.** Two defects.
- **The hole of a write past the end was not authored.** A write handle tracks which ranges it
wrote; everything else below the base size is filled from the base when it is committed. A
grow through `setattr` marks its new tail as written, but a `pwrite` past the end did not mark
the hole it leaves. After `ftruncate(fd, 4 MiB)` and `pwrite(fd, data, 4 MiB + 10)`, bytes
4 MiB to 4 MiB + 10 were refilled from the old base instead of reading as zeros. That silently
restored truncated bytes. When the base was itself a queued revision the remote did not have
yet, the fill read a revision of the wrong size, and B84's check failed it: `EIO` on read, and
a drain that retried forever.
- **A write that closed while its file's create uploaded was lost.** The create of a new file
drains on its own, often while the file is still open. A write that closes meanwhile attaches
its blob to the create's queue row. When the create landed, the drain deleted that row by id,
so the server kept the older bytes (often an empty file) and nothing was queued to replace
them. A handle still open after the create landed carried the placeholder uid, found no create
to attach to, and failed its `close` with `EIO`.
**Fix.** A write past the end authors the hole (`WriteHandle::record_write`).
`Db::finish_create` retires a landed create in one transaction: if the row now holds a newer blob
than the one uploaded, it becomes a revision of the real uid instead of being deleted. The drain
does this under the `pending` lock, which `enqueue_staged_write` holds from attaching a blob until
it records the pending entry, and then repoints open write handles to the real uid. A write that
still finds the create gone waits for its inode's real uid (`Core::landed_uid`) and queues a
revision against it.
Regression tests: `a_write_past_the_end_authors_the_hole_it_leaves`,
`a_write_attached_while_its_create_uploads_becomes_a_revision`,
`a_create_whose_blob_landed_is_retired`, and the live acceptance cases "block boundaries and
overwrites that change the block count" (now also read back after the queue drains) and
`regression B101`.
---
## B100 — A freshly copied file reads as `EIO` or 53 bytes too big
**Status:** Fixed (unverified). No daemon carrying the change has read a file with a provisional
size yet.
**Found:** 2026-09-29, checking a 222-file copy into `Documents/pi-backup` on 2.7.0. `find -printf
%s` showed 10 small files 52 to 59 bytes bigger than their originals. `cat` of one of them failed
with `Input/output error`, and the journal had `the revision's block table disagrees with the
node size ... fsize=221 reader_size=169` followed by `repair download did not match the node
size`. The node's DB row and `claimed_size` were correct (169). A few minutes later the same files
reported the right size and matched the originals byte for byte.
**Where:** `crates/pdfs-fuse/src/filesystem.rs` (`read`, `serve_open`),
`crates/pdfs-fuse/src/lib.rs` (`Core::upgrade_sizes`, `Core::run_size_upgrade`).
**Cause.** Three things together.
- A listing built from the cheap enumeration (B12) has no `claimed_size`, so a file reports its
ciphertext size until the size upgrade lands. The upgrade's own documentation said reads were
unaffected because the revision reader carries its own size.
- B84 made that untrue. `read_block` now treats a revision that disagrees with the size it was
given as damage: it repairs from the whole-file download, and when that does not match either,
it fails the read. With a provisional size the two never match, so every read failed.
- `getattr` normally settles the size before answering (B14), which kept this rare. But only
`MAX_SIZE_UPGRADES` (8) folders may upgrade at once, and past that cap `upgrade_sizes` returned
nothing. The `stat` was answered with the ciphertext size at once, and nothing retried the
folder until the next `stat`. A recursive walk over many fresh folders goes past the cap.
A write open had the same fault: its handle started at the provisional size, so the file would
gain the encryption overhead and reading the base would fail.
**Fix.** `read` and a write `open` settle a provisional size before using it
(`Core::resolve_provisional_size`, one node fetch, on the worker that already goes to the
network). A folder past the cap is now queued instead of dropped. When a running upgrade ends,
its thread takes over the oldest queued folder, so at most 8 threads still run.
Regression tests: `a_folder_past_the_cap_is_queued_not_dropped`,
`a_finished_batch_hands_its_thread_the_oldest_queued_one`. The read and open paths need a live
mount and have no unit test; the acceptance case `regression B100` covers them on a real mount.
---
## B99 — Conflict copies in synced folders are logged again on every start and shown as "Resolved" ## B99 — Conflict copies in synced folders are logged again on every start and shown as "Resolved"
**Status:** Fixed (unverified). No daemon carrying the change has been restarted against the **Status:** Fixed (unverified). No daemon carrying the change has been restarted against the
+131
View File
@@ -11,6 +11,131 @@ scratch (user data in `staging/` and `recovery/` is never touched by this).
## [Unreleased] ## [Unreleased]
## [2.8.2] — 2026-10-01
Schema: **35** (unchanged). Requires `proton-drive-rs` **0.7.2**.
### Fixed
- **Folders shared with you no longer go missing when one person shared several.** When
several accepted shares came from the same owner's volume, only the first appeared under
Shared with me, and the rest were skipped with `missing key`. `proton-drive-rs` 0.7.2 resolves
each shared root with its own key (B112, #25).
## [2.8.1] — 2026-09-30
Schema: **35** (unchanged). Requires `proton-drive-rs` **0.7.1**.
### Fixed
- **The service no longer starts before the desktop session.** `proton-drive.service` was
wanted by `default.target`, which is reached before the desktop hands `DISPLAY` /
`WAYLAND_DISPLAY` to the user's systemd. The first keyring call then started the Secret
Service without a display; on KDE `ksecretd` crashed at every login and the first start of the
service failed. The unit is now wanted by `graphical-session.target`. Existing installs switch
over on the next sign-in or "Start on login" toggle, or with
`systemctl --user reenable proton-drive.service` (#23).
## [2.8.0] — 2026-09-30
Schema: **35** (unchanged). Requires `proton-drive-rs` **0.7.1**.
### Added
- **`pdfs trash --json`** prints the trash listing as JSON.
- **The FUSE acceptance suite tests the signed-in account by default.**
`scripts/fuse-acceptance.sh` with no arguments creates test folders in My files and new
on-demand and mirror sync folders, runs every case in each, and removes all of it afterwards,
also on a failure or interrupt. A run that dies is cleaned up by the next one. New cases cover
block boundaries, unusual names, a deep tree, a wide directory, rename patterns, handle coherency,
throughput and B100. `--offline-only` runs the account-free suite as before.
### Changed
- **The mount keeps working through a network drop.** When Drive stops answering, the daemon
now marks itself offline within about 20 seconds instead of failing every call for up to two
minutes. While offline, creating, renaming and deleting files and folders is queued instead of
failing with `Input/output error`, folders list from the local cache, and a read that needs
data from Drive waits up to 30 seconds for the connection. The daemon checks every few seconds
for the connection to return, then uploads the queued changes. A file whose creation timed
out but reached Drive anyway is taken over rather than uploaded again as a conflict copy.
- **A dead connection is noticed sooner.** `proton-drive-rs` 0.7.1 sends TCP keepalives, so a
connection that died silently fails instead of waiting for its request timeout.
### Fixed
- **Truncating and then writing past the end no longer brings old bytes back.** The gap
between the new end and the write now reads as zeros, as it should, instead of the file's
earlier content. When that earlier content had not been uploaded yet, the file failed with
`Input/output error` and its upload retried forever (B101).
- **A write made while a new file is first uploading is no longer lost.** Drive could keep an
empty or older copy of a file written right after it was created, and closing a file held open
across its first upload could fail (B101).
- **Removing a folder also cancels the pending uploads of the files inside it.** Before, an
edit that had not reached Drive yet kept retrying against the trash forever and kept its
staged copy on disk. `pdfs rm` now also drops what was queued for what it removes (B102).
- **A program reading a file keeps its data when another program replaces the file.** Saving
over a file by renaming a new copy onto it (what most editors and `os.replace` do) made
reads through handles already open on the old file fail with `Input/output error` (B104).
- **Short-lived files no longer come back as conflict copies.** A file deleted right after it
was written, such as a SQLite journal, could be uploaded as a `recovered-…` conflict copy in
the Drive root (B105).
- **Editors can save over a file several times in a row.** The fix for B104 could leave the
upload of a replaced file queued, and the next save onto the same name failed with
`Input/output error` (B106).
- **Renaming a file back right after renaming it no longer fails.** Drive can briefly call the
second rename out of date, and the daemon now waits it out instead of returning an I/O error
(B111).
- **Reading a freshly listed file no longer hangs.** When a read had to fetch the file's real
size first, the daemon could wait on the read it was serving, and the program froze until
the daemon restarted (B110).
- **Databases and other files kept open while written no longer read back empty.** When the
folder was listed again while a file was still open, its size dropped to what Drive held,
often 0 bytes, and SQLite reported `database disk image is malformed` (B109).
- **Deleting files during a mirror pass no longer marks the folder as failed.** A file removed
after the pass planned its upload was counted as an error, so the folder showed `error` until
a later pass (B108).
- **A read no longer hangs when the connection stalls.** A read that gets no answer from Drive
within two minutes now fails with `Input/output error` instead of blocking the program for
as long as half an hour (B107).
- **The trash no longer freezes after a server call hangs.** Until the daemon restarted, the
trash kept showing the same entries, and a folder just deleted could not be found there to
delete permanently (B103).
- **Freshly listed files read correctly.** A file whose real size had not been fetched yet
reported its encrypted size, about 53 bytes too big, and reading it failed with
`Input/output error`. This happened most often right after copying a large folder. Reads and
write opens now fetch the real size first, and a big folder walk no longer skips size lookups
(B100).
## [2.7.0] — 2026-09-29
Schema: **35** (unchanged).
### Added
- **Moves between locations without transferring content.** `pdfs move` and the app's
**Move to…** now take a file or folder from My files into a synced folder, or between two
synced folders, on-demand or mirrored. The move happens on Proton Drive, so nothing is
downloaded or uploaded again. `mv` between two locations still copies, because the kernel
refuses a rename between two mounts. The app's Move dialog lists every location at its top
level. A move is refused, with the reason, when the source still has changes that are not on
Proton Drive, when the two locations are on different volumes, or when a synced folder stays
busy with a sync pass.
- **`pdfs move` takes several sources.** `pdfs move a b c DEST` moves all three and
reports each one that could not move.
- **Moving into a mirrored folder reuses cached content.** A file already cached by
an online-only folder is copied into place instead of downloaded again.
### Changed
- **New project homepage: [proton-drive.narl.io](https://proton-drive.narl.io).** The About
dialog's website link, the README, and the package metadata (Cargo, PKGBUILDs, `.deb`,
`.rpm`) point there. The source code link stays on GitHub.
## [2.6.2] — 2026-09-28
Schema: **35** (unchanged).
### Fixed
- **Photos remembers the Albums and Places views.** Leaving Photos from Albums or Places and
coming back showed the timeline while the Albums or Places toggle stayed selected. The page
now reopens on the selected view, and Places keeps its Grid or Map layout.
- **The Sync page keeps a steady width.** A long file name in Transfers no longer widens the
page. Overview and Folders now fill the width next to the sidebar.
## [2.6.1] — 2026-09-28 ## [2.6.1] — 2026-09-28
Schema: **35** (unchanged). Schema: **35** (unchanged).
@@ -1239,6 +1364,12 @@ First stable release: FUSE files-on-demand mount, sync daemon under `proton-driv
- The outstanding FUSE defects tracked in `docs/BUGS.md`, plus a truncate defect, validated - The outstanding FUSE defects tracked in `docs/BUGS.md`, plus a truncate defect, validated
by a new POSIX compliance suite for the filesystem. by a new POSIX compliance suite for the filesystem.
[2.8.2]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.8.2
[2.8.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.8.1
[2.8.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.8.0
[2.7.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.7.0
[2.6.2]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.6.2
[2.6.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.6.1 [2.6.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.6.1
[2.6.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.6.0 [2.6.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.6.0
[2.5.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.5.0 [2.5.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v2.5.0
+2 -2
View File
@@ -53,7 +53,7 @@ talks to the running service (`proton-drive.service`); `login`, `logout`, `mount
| `pdfs search QUERY [--in FOLDER] [--limit N]` | Search names in the local index. | | `pdfs search QUERY [--in FOLDER] [--limit N]` | Search names in the local index. |
| `pdfs mkdir PARENT NAME` | Create a folder. | | `pdfs mkdir PARENT NAME` | Create a folder. |
| `pdfs rename PATH NEW_NAME` | Rename a file or folder. | | `pdfs rename PATH NEW_NAME` | Rename a file or folder. |
| `pdfs move PATH NEW_PARENT` | Move into another folder. | | `pdfs move PATH... NEW_PARENT` | Move one or more files or folders into another folder. The two may be in different locations (My files, an on-demand or a mirrored synced folder); the move happens on Proton Drive, without downloading or uploading anything. |
| `pdfs rm PATH` | Move to the Trash. | | `pdfs rm PATH` | Move to the Trash. |
| `pdfs upload SOURCES... [-t FOLDER]` | Upload local files and folders (recursively) in the background. Follow with `pdfs transfers`. | | `pdfs upload SOURCES... [-t FOLDER]` | Upload local files and folders (recursively) in the background. Follow with `pdfs transfers`. |
| `pdfs refresh [TARGET] [--full]` | Forget a cached listing so the next read fetches it again. `TARGET` is a folder, `trash` or `photos`; `--full` with `photos` re-reads every photo's metadata. | | `pdfs refresh [TARGET] [--full]` | Forget a cached listing so the next read fetches it again. `TARGET` is a folder, `trash` or `photos`; `--full` with `photos` re-reads every photo's metadata. |
@@ -177,7 +177,7 @@ commands are for scripts and for cases where the mount is not available.
## Scripting with `--json` ## Scripting with `--json`
The global `--json` flag makes query commands print JSON: `status`, `ls`, `pins`, `sync list`, The global `--json` flag makes query commands print JSON: `status`, `ls`, `pins`, `sync list`,
`devices list`, `locations`, `transfers`, `activity` and `cache inspect`. Commands that change `devices list`, `locations`, `transfers`, `activity`, `trash` and `cache inspect`. Commands that change
something keep their human-readable output; check their exit code. something keep their human-readable output; check their exit code.
```bash ```bash
+8
View File
@@ -215,6 +215,14 @@ starts; to switch at once:
systemctl --user restart proton-drive.service systemctl --user restart proton-drive.service
``` ```
Coming from 2.8.0 or older, re-enable the service once. Those releases hooked it into
`default.target`, which starts it before the desktop session is ready; on KDE that crashes
`ksecretd` at every login. Signing in again or toggling "Start on login" does this as well:
```bash
systemctl --user reenable proton-drive.service
```
Database migrations run automatically and only move forward. A database written by a newer Database migrations run automatically and only move forward. A database written by a newer
release refuses to open in an older one, so downgrading means clearing the local database (see release refuses to open in an older one, so downgrading means clearing the local database (see
[Troubleshooting](TROUBLESHOOTING.md#start-over-with-a-clean-local-state)). [Troubleshooting](TROUBLESHOOTING.md#start-over-with-a-clean-local-state)).
+104 -10
View File
@@ -25,18 +25,81 @@ Proton:
scripts/fuse-acceptance.sh --offline-only scripts/fuse-acceptance.sh --offline-only
``` ```
Kernel/FUSE behavior and remote convergence need a real mount. Live testing is Kernel/FUSE behavior and remote convergence need a real mount. With no
explicitly opt-in and always runs the account-free contract first. It runs arguments, the script tests the account the running service is logged in to:
destructive POSIX operations only below a fresh `pdfs-acceptance-*` directory.
It refuses paths that are not FUSE mounts and removes its directory on success,
failure, or interruption.
Use a dedicated test account without irreplaceable data: ```console
scripts/fuse-acceptance.sh # every case, every mount kind
scripts/fuse-acceptance.sh --quick # one on-demand/mirror pair instead of four
```
An account run does this, in order:
1. Runs the account-free contract as the reference.
2. Runs the contract in a fresh `pdfs-acceptance-<id>` folder in My files and
diffs it against the reference.
3. Registers two new sync folders under `~/.cache/pdfs-acceptance/run-<id>/`
and runs the contract and the move cases in every on-demand/mirror pairing.
4. Removes everything it created, locally and remotely, and reports a
`cleanup` result. The run fails if anything is left behind.
Cleanup identifies remote folders by uid, never by name alone, so nothing of
yours is touched. It removes each sync folder with `sync rm --delete-remote`,
trashes the My files folder, and then deletes both permanently from the trash.
It only counts a folder as gone once a fresh trash listing no longer shows it.
It then reads `pdfs sync queue` and counts any queued op that still names one
of its folders as left behind too, since such an op keeps its staged bytes and
retries forever (B102).
Cleanup also runs on a failure, Ctrl-C, `SIGTERM` or `SIGHUP`. While it runs,
two more interrupts are ignored; a third abandons it.
Each run records what it is about to create in
`~/.cache/pdfs-acceptance/run-<id>/manifest.json` before it creates it, and
holds a lock on that directory. If a run dies without cleaning up (`kill -9`, a
hard timeout, a power cut), the next run finds the unlocked manifest and
finishes that cleanup first. It then removes stray `pdfs-acceptance-<32 hex>`
folders that no live run claims: in My files (older than
`PDFS_ACCEPTANCE_REAP_AGE`, default one hour), under the acceptance home, and
in the trash. A run does not start while that reap leaves anything behind.
The run creates and destroys real remote folders, so a dedicated test account
without irreplaceable data is still the safest target.
`--live PATH...` tests mounts you set up yourself and leaves the sync folder
configuration alone. It runs destructive POSIX operations only below a fresh
`pdfs-acceptance-*` directory, refuses paths that are not FUSE mounts, and
removes its directory on success, failure, or interruption:
```console ```console
scripts/fuse-acceptance.sh --live /mnt/on-demand-testmount scripts/fuse-acceptance.sh --live /mnt/on-demand-testmount
``` ```
### Edge cases and performance
Besides the syscall contract, the suite covers:
- **Block boundaries.** Files of 0 bytes up to two 4 MiB blocks plus 3 bytes,
including every size next to a boundary, then overwrites that grow, shrink
and reshape them.
- **Unusual names.** A 255-byte name, NFC and NFD forms of the same text,
newline, tab, emoji, characters Windows forbids, leading and trailing
spaces, and a case-only rename.
- **A deep tree and a wide directory.** 24 levels of nesting renamed at the
top; 128 entries created concurrently, concurrent unlinks, then `rm -rf`.
Online, each create is a round trip to the server, and the kernel runs
creates in one directory one at a time. So the wide case gets twice the
per-case timeout and notes the time per create as `wide.create_ms`.
- **Rename patterns.** Swaps, chains, log rotation, replacing a file under an
open reader, the `EISDIR`/`ENOTDIR`/`ENOENT` refusals, and a folder moved out
and back.
- **Handle coherency.** Unsynced writes seen by a second handle, two `O_APPEND`
writers, 30 rewrite generations, and a truncate under a reader.
- **Throughput.** 64 MiB written and read sequentially, 1 MiB in 4 KiB writes,
and metadata operations on 100 files. The rates are printed. The case fails
below `PDFS_ACCEPTANCE_MIN_MIBPS` (default 10) or `PDFS_ACCEPTANCE_MIN_OPS`
(default 20 operations per second, or 3 on a FUSE mount, where each create
and unlink waits for Drive before it returns).
### The mount is diffed against an ordinary filesystem ### The mount is diffed against an ordinary filesystem
The local reference run is not only a self-test of the runner. Every case The local reference run is not only a self-test of the runner. Every case
@@ -78,6 +141,16 @@ in-flight `.crdownload` must not be sealed as a revision) and B74 (a file
renamed after close must keep its content) need a real mount and a reachable renamed after close must keep its content) need a real mount and a reachable
daemon; they are skipped, not failed, without one. daemon; they are skipped, not failed, without one.
**B100** (a copied tree reads back at its exact sizes) copies 48 files of
assorted sizes into the mount at once, as `cp -r` does. On a live mount it
waits for the queue, checks every size and byte, runs `pdfs refresh` on each
folder to drop the cached listing, and checks again. The refresh is what
brought back the provisional sizes the bug reported.
**B101** (a write during its create's upload reaches Drive) holds new files open for
different times before writing, so their empty creates land before, during and after the
bytes arrive, then checks every file after the queue drains.
B74 was found by exactly this mechanism: the B70 case failed, and narrowing it B74 was found by exactly this mechanism: the B70 case failed, and narrowing it
produced a smaller reproduction that got its own case. The defect is fixed and produced a smaller reproduction that got its own case. The defect is fixed and
live-validated (2026-07-26); the case now guards against its return. live-validated (2026-07-26); the case now guards against its return.
@@ -136,13 +209,17 @@ stops at the first failure.
it ran. A suite can pass every assertion while the daemon logs failures behind it ran. A suite can pass every assertion while the daemon logs failures behind
it; that has happened, and it was caught only by reading the journal by hand. it; that has happened, and it was caught only by reading the journal by hand.
Abandoned `pdfs-acceptance-*` roots from a crashed run are removed at startup, With `--live`, abandoned `pdfs-acceptance-*` roots from a crashed run are
but only if they match the exact generated name and are over an hour old removed at startup, but only if they match the exact generated name and are
(`PDFS_ACCEPTANCE_REAP_AGE`), so a concurrent run is never disturbed. over an hour old (`PDFS_ACCEPTANCE_REAP_AGE`), so a concurrent run is never
disturbed. An account run instead uses the manifest and reaper described
above.
### Automated mode matrix ### Automated mode matrix
The managed live runner accepts two existing, empty, unmounted directories. It An account run drives this matrix itself, in folders it creates. To use
directories of your choice instead, the managed live runner accepts two
existing, empty, unmounted directories. It
registers both as new sync folders, waits for initial synchronization, then registers both as new sync folders, waits for initial synchronization, then
tests all four pairs in order: on-demand/on-demand, on-demand/mirror, tests all four pairs in order: on-demand/on-demand, on-demand/mirror,
mirror/on-demand, and mirror/mirror. Every transition must become idle, and an mirror/on-demand, and mirror/mirror. Every transition must become idle, and an
@@ -159,6 +236,23 @@ in `pdfs sync list`. Use `PDFS_ACCEPTANCE_SYNC_TIMEOUT` to extend transition
timeouts and `PDFS_ACCEPTANCE_PDFS` to select a non-installed CLI binary, for timeouts and `PDFS_ACCEPTANCE_PDFS` to select a non-installed CLI binary, for
example `target/debug/pdfs`. example `target/debug/pdfs`.
After the contract, each mode pair also runs the move cases (`--list` shows
them as `managed`). They move trees between the two folders in both
directions, move several sources in one `pdfs move`, and pass a tree through
My files and out again. Every case waits for Drive before and after the move.
It then checks that the tree is only at the destination, that every byte
arrived, and that neither side made conflict copies. The refusal cases prove
that nothing moves and nothing is lost when:
- the destination already has the name;
- a folder would move into itself;
- a mirror source holds something Drive lacks (a symlink);
- a file in an on-demand source is still open for writing.
The My files case writes a test folder into your real My files and removes it
afterwards. It skips when the daemon reports no My files mount, or when the
locations are on different volumes.
### Durability drill ### Durability drill
`--durability` restarts `proton-drive.service` in the middle of the suite and `--durability` restarts `proton-drive.service` in the middle of the suite and
+22
View File
@@ -95,6 +95,28 @@ of the folder, so the files stay visible but take no disk space until opened. Sw
**Synced / Online only** switch on the folder's card, or `pdfs sync mode <id> ondemand` and back **Synced / Online only** switch on the folder's card, or `pdfs sync mode <id> ondemand` and back
with `pdfs sync mode <id> mirror`. `pdfs locations` lists every local place the service occupies. with `pdfs sync mode <id> mirror`. `pdfs locations` lists every local place the service occupies.
### Moving between synced folders and My files
`~/ProtonDrive` and each online-only folder are separate mounts, and a mirrored folder is a plain
folder on disk. `mv`, and dragging in a file manager, cannot rename across them, so they copy
and then delete: every file is downloaded, uploaded again, and the original goes to the Trash.
Use `pdfs move <path> <folder>` or **Move to…** in the app instead. The move happens on Proton
Drive, so it takes seconds whatever the size. For example:
```sh
pdfs move ~/Documents/Scans ~/ProtonDrive/Archive
pdfs move ~/Documents/a.pdf ~/Documents/b.pdf ~/ProtonDrive/Archive
```
The last path is the destination folder; everything before it moves there.
Files already opened in an online-only folder are not downloaded again when
they move into a mirrored one.
A move out of a synced folder waits until its files are on Proton Drive. A file that is still
uploading, has changes that are not uploaded, or is excluded by `.pdfsignore` stops the move,
and the message names it.
Removing a synced folder (**Stop Syncing** or `pdfs sync rm <id>`) keeps the local files and the Removing a synced folder (**Stop Syncing** or `pdfs sync rm <id>`) keeps the local files and the
copy on Proton Drive. Add `--delete-remote` (or tick **Also delete from Proton Drive**) to delete copy on Proton Drive. Add `--delete-remote` (or tick **Also delete from Proton Drive**) to delete
the copy on Proton Drive as well. the copy on Proton Drive as well.
+2 -2
View File
@@ -9,11 +9,11 @@
# Build + install: cd packaging && makepkg -fi # Build + install: cd packaging && makepkg -fi
pkgname=proton-drive-linux pkgname=proton-drive-linux
pkgver=2.6.1 pkgver=2.8.2
pkgrel=1 pkgrel=1
pkgdesc="Unofficial Proton Drive client for Linux: files-on-demand FUSE mount, CLI, GTK4 app + tray" pkgdesc="Unofficial Proton Drive client for Linux: files-on-demand FUSE mount, CLI, GTK4 app + tray"
arch=('x86_64') arch=('x86_64')
url="https://github.com/narrrl/proton-drive-linux" url="https://proton-drive.narl.io"
license=('MIT') license=('MIT')
depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'perl-image-exiftool') depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'perl-image-exiftool')
makedepends=('cargo' 'git' 'gettext') makedepends=('cargo' 'git' 'gettext')
@@ -1,8 +1,8 @@
pkgbase = proton-drive-for-linux-bin pkgbase = proton-drive-for-linux-bin
pkgdesc = Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries) pkgdesc = Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries)
pkgver = 2.6.0 pkgver = 2.8.1
pkgrel = 1 pkgrel = 1
url = https://github.com/narrrl/proton-drive-linux url = https://proton-drive.narl.io
arch = x86_64 arch = x86_64
license = MIT license = MIT
depends = fuse3 depends = fuse3
@@ -21,25 +21,25 @@ pkgbase = proton-drive-for-linux-bin
optdepends = xdg-utils: "Open folder" action in the tray menu optdepends = xdg-utils: "Open folder" action in the tray menu
optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf) optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf)
optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding
provides = proton-drive-for-linux=2.6.0 provides = proton-drive-for-linux=2.8.1
provides = proton-drive-linux=2.6.0 provides = proton-drive-linux=2.8.1
provides = pdfs provides = pdfs
conflicts = proton-drive-linux conflicts = proton-drive-linux
conflicts = proton-drive-for-linux conflicts = proton-drive-for-linux
conflicts = proton-drive-for-linux-git conflicts = proton-drive-for-linux-git
options = !strip options = !strip
options = !debug options = !debug
source = proton-drive-linux-2.6.0-x86_64.tar.gz::https://github.com/narrrl/proton-drive-linux/releases/download/v2.6.0/proton-drive-linux-2.6.0-x86_64.tar.gz source = proton-drive-linux-2.8.1-x86_64.tar.gz::https://github.com/narrrl/proton-drive-linux/releases/download/v2.8.1/proton-drive-linux-2.8.1-x86_64.tar.gz
source = io.narl.proton-drive-linux.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.6.0/packaging/io.narl.proton-drive-linux.desktop source = io.narl.proton-drive-linux.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.8.1/packaging/io.narl.proton-drive-linux.desktop
source = io.narl.proton-drive-linux-tray.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.6.0/packaging/io.narl.proton-drive-linux-tray.desktop source = io.narl.proton-drive-linux-tray.desktop::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.8.1/packaging/io.narl.proton-drive-linux-tray.desktop
source = io.narl.proton-drive-linux.svg::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.6.0/packaging/io.narl.proton-drive-linux.svg source = io.narl.proton-drive-linux.svg::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.8.1/packaging/io.narl.proton-drive-linux.svg
source = proton-drive.service::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.6.0/packaging/proton-drive.service source = proton-drive.service::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.8.1/packaging/proton-drive.service
source = LICENSE-2.6.0::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.6.0/LICENSE source = LICENSE-2.8.1::https://raw.githubusercontent.com/narrrl/proton-drive-linux/v2.8.1/LICENSE
sha256sums = 63893c2f9af8d660151a8687d15472602c76e5dd3cca7547999cd0fc0a833a4d sha256sums = 68cac3de6a4156994d6990a88d6715c05e49da5021a2e18458498364d1ed299a
sha256sums = d190d6771fff0b975271fb62fd3fb0fa8bbf17d7278c10853c18557855b0c123 sha256sums = d190d6771fff0b975271fb62fd3fb0fa8bbf17d7278c10853c18557855b0c123
sha256sums = 1d95e250370220e3bb6aead6ce34d1d6cc759f42c4803b75f1667e78ff47c6c0 sha256sums = 1d95e250370220e3bb6aead6ce34d1d6cc759f42c4803b75f1667e78ff47c6c0
sha256sums = e598a93c2715c3a9fdc7fb937902083d4a1135e8fbdfe6fe7e37f0383ebbd828 sha256sums = e598a93c2715c3a9fdc7fb937902083d4a1135e8fbdfe6fe7e37f0383ebbd828
sha256sums = 25fa3bd9ba52df534be9d8bc66f77d7502fe1423177237d0d14b673dedeb1b60 sha256sums = 77280bff5f693fbc5ffa9f3ccd06a43cfff96d770290aa8d7d0c28e9dde277be
sha256sums = 02c419e7f9af08dda720e4f852f3e57cf5a69e192605440f750e9d244fa2817c sha256sums = 02c419e7f9af08dda720e4f852f3e57cf5a69e192605440f750e9d244fa2817c
pkgname = proton-drive-for-linux-bin pkgname = proton-drive-for-linux-bin
@@ -1,11 +1,12 @@
# Maintainer: Nils Pukropp <contact@narl.io> # Maintainer: Nils Pukropp <contact@narl.io>
pkgname=proton-drive-for-linux-bin pkgname=proton-drive-for-linux-bin
pkgver=2.6.0 pkgver=2.8.1
pkgrel=1 pkgrel=1
pkgdesc="Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries)" pkgdesc="Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (binaries)"
arch=('x86_64') arch=('x86_64')
url="https://github.com/narrrl/proton-drive-linux" url="https://proton-drive.narl.io"
_repo="https://github.com/narrrl/proton-drive-linux"
license=('MIT') license=('MIT')
depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc' depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc'
'hicolor-icon-theme') 'hicolor-icon-theme')
@@ -25,17 +26,17 @@ _tag="v$pkgver"
_raw="https://raw.githubusercontent.com/narrrl/proton-drive-linux/$_tag" _raw="https://raw.githubusercontent.com/narrrl/proton-drive-linux/$_tag"
# The release tarball holds the four binaries only, so the desktop integration # The release tarball holds the four binaries only, so the desktop integration
# files come straight from the tagged tree. # files come straight from the tagged tree.
source=("proton-drive-linux-$pkgver-x86_64.tar.gz::$url/releases/download/$_tag/proton-drive-linux-$pkgver-x86_64.tar.gz" source=("proton-drive-linux-$pkgver-x86_64.tar.gz::$_repo/releases/download/$_tag/proton-drive-linux-$pkgver-x86_64.tar.gz"
"io.narl.proton-drive-linux.desktop::$_raw/packaging/io.narl.proton-drive-linux.desktop" "io.narl.proton-drive-linux.desktop::$_raw/packaging/io.narl.proton-drive-linux.desktop"
"io.narl.proton-drive-linux-tray.desktop::$_raw/packaging/io.narl.proton-drive-linux-tray.desktop" "io.narl.proton-drive-linux-tray.desktop::$_raw/packaging/io.narl.proton-drive-linux-tray.desktop"
"io.narl.proton-drive-linux.svg::$_raw/packaging/io.narl.proton-drive-linux.svg" "io.narl.proton-drive-linux.svg::$_raw/packaging/io.narl.proton-drive-linux.svg"
"proton-drive.service::$_raw/packaging/proton-drive.service" "proton-drive.service::$_raw/packaging/proton-drive.service"
"LICENSE-$pkgver::$_raw/LICENSE") "LICENSE-$pkgver::$_raw/LICENSE")
sha256sums=('63893c2f9af8d660151a8687d15472602c76e5dd3cca7547999cd0fc0a833a4d' sha256sums=('68cac3de6a4156994d6990a88d6715c05e49da5021a2e18458498364d1ed299a'
'd190d6771fff0b975271fb62fd3fb0fa8bbf17d7278c10853c18557855b0c123' 'd190d6771fff0b975271fb62fd3fb0fa8bbf17d7278c10853c18557855b0c123'
'1d95e250370220e3bb6aead6ce34d1d6cc759f42c4803b75f1667e78ff47c6c0' '1d95e250370220e3bb6aead6ce34d1d6cc759f42c4803b75f1667e78ff47c6c0'
'e598a93c2715c3a9fdc7fb937902083d4a1135e8fbdfe6fe7e37f0383ebbd828' 'e598a93c2715c3a9fdc7fb937902083d4a1135e8fbdfe6fe7e37f0383ebbd828'
'25fa3bd9ba52df534be9d8bc66f77d7502fe1423177237d0d14b673dedeb1b60' '77280bff5f693fbc5ffa9f3ccd06a43cfff96d770290aa8d7d0c28e9dde277be'
'02c419e7f9af08dda720e4f852f3e57cf5a69e192605440f750e9d244fa2817c') '02c419e7f9af08dda720e4f852f3e57cf5a69e192605440f750e9d244fa2817c')
package() { package() {
@@ -1,8 +1,8 @@
pkgbase = proton-drive-for-linux-git pkgbase = proton-drive-for-linux-git
pkgdesc = Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (git) pkgdesc = Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (git)
pkgver = 2.6.0.r0.g8bd3867 pkgver = 2.8.1.r0.gba1fdd6
pkgrel = 1 pkgrel = 1
url = https://github.com/narrrl/proton-drive-linux url = https://proton-drive.narl.io
arch = x86_64 arch = x86_64
license = MIT license = MIT
makedepends = cargo makedepends = cargo
@@ -24,8 +24,8 @@ pkgbase = proton-drive-for-linux-git
optdepends = xdg-utils: "Open folder" action in the tray menu optdepends = xdg-utils: "Open folder" action in the tray menu
optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf) optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf)
optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding
provides = proton-drive-for-linux=2.6.0.r0.g8bd3867 provides = proton-drive-for-linux=2.8.1.r0.gba1fdd6
provides = proton-drive-linux=2.6.0.r0.g8bd3867 provides = proton-drive-linux=2.8.1.r0.gba1fdd6
provides = pdfs provides = pdfs
conflicts = proton-drive-linux conflicts = proton-drive-linux
conflicts = proton-drive-for-linux conflicts = proton-drive-for-linux
@@ -1,11 +1,12 @@
# Maintainer: Nils Pukropp <contact@narl.io> # Maintainer: Nils Pukropp <contact@narl.io>
pkgname=proton-drive-for-linux-git pkgname=proton-drive-for-linux-git
pkgver=2.6.0.r0.g8bd3867 pkgver=2.8.1.r0.gba1fdd6
pkgrel=1 pkgrel=1
pkgdesc="Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (git)" pkgdesc="Unofficial Proton Drive client: FUSE files-on-demand mount, CLI, GTK4 app, tray (git)"
arch=('x86_64') arch=('x86_64')
url="https://github.com/narrrl/proton-drive-linux" url="https://proton-drive.narl.io"
_repo="https://github.com/narrrl/proton-drive-linux"
license=('MIT') license=('MIT')
depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc' depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc'
'hicolor-icon-theme') 'hicolor-icon-theme')
@@ -22,7 +23,7 @@ provides=("${pkgname%-git}=$pkgver" "proton-drive-linux=$pkgver" 'pdfs')
conflicts=('proton-drive-linux' 'proton-drive-for-linux' 'proton-drive-for-linux-bin') conflicts=('proton-drive-linux' 'proton-drive-for-linux' 'proton-drive-for-linux-bin')
# LTO has broken GTK/Rust links for this workspace; the release profile keeps it off. # LTO has broken GTK/Rust links for this workspace; the release profile keeps it off.
options=('!lto') options=('!lto')
source=("$pkgname::git+$url.git") source=("$pkgname::git+$_repo.git")
sha256sums=('SKIP') sha256sums=('SKIP')
pkgver() { pkgver() {
@@ -1,8 +1,8 @@
pkgbase = proton-drive-for-linux pkgbase = proton-drive-for-linux
pkgdesc = Unofficial Proton Drive client: files-on-demand FUSE mount, CLI, GTK4 app and tray pkgdesc = Unofficial Proton Drive client: files-on-demand FUSE mount, CLI, GTK4 app and tray
pkgver = 2.6.0 pkgver = 2.8.1
pkgrel = 1 pkgrel = 1
url = https://github.com/narrrl/proton-drive-linux url = https://proton-drive.narl.io
arch = x86_64 arch = x86_64
license = MIT license = MIT
makedepends = cargo makedepends = cargo
@@ -23,13 +23,13 @@ pkgbase = proton-drive-for-linux
optdepends = xdg-utils: "Open folder" action in the tray menu optdepends = xdg-utils: "Open folder" action in the tray menu
optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf) optdepends = fzf: search-as-you-type launcher (pdfs-prompt --fzf)
optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding optdepends = foot: terminal that hosts pdfs-prompt --fzf when started from a keybinding
provides = proton-drive-linux=2.6.0 provides = proton-drive-linux=2.8.1
provides = pdfs provides = pdfs
conflicts = proton-drive-linux conflicts = proton-drive-linux
conflicts = proton-drive-for-linux-bin conflicts = proton-drive-for-linux-bin
conflicts = proton-drive-for-linux-git conflicts = proton-drive-for-linux-git
options = !lto options = !lto
source = proton-drive-for-linux-2.6.0.tar.gz::https://github.com/narrrl/proton-drive-linux/archive/refs/tags/v2.6.0.tar.gz source = proton-drive-for-linux-2.8.1.tar.gz::https://github.com/narrrl/proton-drive-linux/archive/refs/tags/v2.8.1.tar.gz
sha256sums = 6bc29064c23d9cd23158a675a848335a4ec5220c34fdb2b4c8f3d21061f0b516 sha256sums = ee116a75201ae836921865fe9d2179c30f9e32084c983020120a73720923c1d7
pkgname = proton-drive-for-linux pkgname = proton-drive-for-linux
@@ -1,11 +1,12 @@
# Maintainer: Nils Pukropp <contact@narl.io> # Maintainer: Nils Pukropp <contact@narl.io>
pkgname=proton-drive-for-linux pkgname=proton-drive-for-linux
pkgver=2.6.0 pkgver=2.8.1
pkgrel=1 pkgrel=1
pkgdesc="Unofficial Proton Drive client: files-on-demand FUSE mount, CLI, GTK4 app and tray" pkgdesc="Unofficial Proton Drive client: files-on-demand FUSE mount, CLI, GTK4 app and tray"
arch=('x86_64') arch=('x86_64')
url="https://github.com/narrrl/proton-drive-linux" url="https://proton-drive.narl.io"
_repo="https://github.com/narrrl/proton-drive-linux"
license=('MIT') license=('MIT')
depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc' depends=('fuse3' 'gtk4' 'libadwaita' 'webkitgtk-6.0' 'dbus' 'gcc-libs' 'glibc'
'hicolor-icon-theme') 'hicolor-icon-theme')
@@ -22,8 +23,8 @@ provides=("proton-drive-linux=$pkgver" 'pdfs')
conflicts=('proton-drive-linux' 'proton-drive-for-linux-bin' 'proton-drive-for-linux-git') conflicts=('proton-drive-linux' 'proton-drive-for-linux-bin' 'proton-drive-for-linux-git')
# LTO has broken GTK/Rust links for this workspace; the release profile keeps it off. # LTO has broken GTK/Rust links for this workspace; the release profile keeps it off.
options=('!lto') options=('!lto')
source=("$pkgname-$pkgver.tar.gz::$url/archive/refs/tags/v$pkgver.tar.gz") source=("$pkgname-$pkgver.tar.gz::$_repo/archive/refs/tags/v$pkgver.tar.gz")
sha256sums=('6bc29064c23d9cd23158a675a848335a4ec5220c34fdb2b4c8f3d21061f0b516') sha256sums=('ee116a75201ae836921865fe9d2179c30f9e32084c983020120a73720923c1d7')
_srcname="proton-drive-linux-$pkgver" _srcname="proton-drive-linux-$pkgver"
+1 -1
View File
@@ -14,7 +14,7 @@ Version: %{version}
Release: 1%{?dist} Release: 1%{?dist}
Summary: Unofficial Proton Drive client for Linux (FUSE, CLI, GTK4 app + tray) Summary: Unofficial Proton Drive client for Linux (FUSE, CLI, GTK4 app + tray)
License: MIT License: MIT
URL: https://github.com/narrrl/proton-drive-linux URL: https://proton-drive.narl.io
ExclusiveArch: x86_64 ExclusiveArch: x86_64
BuildRequires: cargo BuildRequires: cargo
+7 -1
View File
@@ -32,4 +32,10 @@ MemoryMax=6G
TimeoutStopSec=15 TimeoutStopSec=15
[Install] [Install]
WantedBy=default.target # Not default.target: that is reached before the desktop has exported DISPLAY /
# WAYLAND_DISPLAY to the user manager, so After= above never orders anything
# (the two targets are not in the same start transaction). The daemon's first
# keyring call then D-Bus-activates the Secret Service without a display —
# on KDE ksecretd aborts, the start fails, and every login leaves a coredump.
# Being pulled in by the graphical session starts it only once that is up.
WantedBy=graphical-session.target
+298 -292
View File
File diff suppressed because it is too large. Load diff
+298 -292
View File
File diff suppressed because it is too large. Load diff
+298 -292
View File
File diff suppressed because it is too large. Load diff
+298 -292
View File
File diff suppressed because it is too large. Load diff
+298 -292
View File
File diff suppressed because it is too large. Load diff
+298 -292
View File
File diff suppressed because it is too large. Load diff
+299 -293
View File
File diff suppressed because it is too large. Load diff
+298 -292
View File
File diff suppressed because it is too large. Load diff
+298 -292
View File
File diff suppressed because it is too large. Load diff
+1505 -18
View File
File diff suppressed because it is too large. Load diff
+22 -8
View File
@@ -4,20 +4,28 @@ set -Eeuo pipefail
usage() { usage() {
cat >&2 <<'EOF' cat >&2 <<'EOF'
usage: scripts/fuse-acceptance.sh [--offline-only] [OPTION ...] usage: scripts/fuse-acceptance.sh [OPTION ...]
scripts/fuse-acceptance.sh --offline-only [OPTION ...]
scripts/fuse-acceptance.sh --live MOUNTPOINT [MOUNTPOINT ...] [OPTION ...] scripts/fuse-acceptance.sh --live MOUNTPOINT [MOUNTPOINT ...] [OPTION ...]
scripts/fuse-acceptance.sh --managed-live EMPTY_DIR EMPTY_DIR [OPTION ...] scripts/fuse-acceptance.sh --managed-live EMPTY_DIR EMPTY_DIR [OPTION ...]
scripts/fuse-acceptance.sh MOUNTPOINT [MOUNTPOINT ...] # compatibility scripts/fuse-acceptance.sh MOUNTPOINT [MOUNTPOINT ...] # compatibility
The local reference suite always runs first and requires no account; the mount With no mode, everything runs on the signed-in account: My files, then two
is then diffed against its recorded behaviour. --live runs the same contract on sync folders it registers itself, through every pairing of on-demand and
each mount. Set PDFS_ACCEPTANCE_CONVERGENCE=1 only when all live mountpoints mirror, with moves and copies between them. Everything it creates is removed
afterwards, on failure and on Ctrl-C too, and permanently deleted from the
trash; a run that was killed is cleaned up by the next one.
--offline-only runs only the local reference suite, which needs no account.
It always runs first; every mount is diffed against its recorded behaviour.
--live runs the same contract on each given mount. Set PDFS_ACCEPTANCE_CONVERGENCE=1 only when all live mountpoints
show the same remote folder. show the same remote folder.
Options are forwarded to the Python runner. The useful ones: Options are forwarded to the Python runner. The useful ones:
--list print every case and the targets it runs against --list print every case and the targets it runs against
--timeout SECONDS per-case limit (default 180); a hung case dumps stacks --timeout SECONDS per-case limit (default 180); a hung case dumps stacks
--fail-fast stop at the first failure instead of continuing --fail-fast stop at the first failure instead of continuing
--quick one on-demand/mirror pairing instead of all four
--report-json PATH machine-readable results and recorded observations --report-json PATH machine-readable results and recorded observations
--report-junit PATH JUnit XML for CI --report-junit PATH JUnit XML for CI
--budget SECONDS report any case slower than this --budget SECONDS report any case slower than this
@@ -45,8 +53,11 @@ collect_paths() {
case "${1:-}" in case "${1:-}" in
-h|--help) usage ;; -h|--help) usage ;;
--offline-only|"") "")
shift || true exec "${runner[@]}" --account
;;
--offline-only)
shift
exec "${runner[@]}" "$@" exec "${runner[@]}" "$@"
;; ;;
--live) --live)
@@ -60,10 +71,13 @@ case "${1:-}" in
first="$1"; second="$2"; shift 2 first="$1"; second="$2"; shift 2
exec "${runner[@]}" --managed-live "$first" "$second" "$@" exec "${runner[@]}" --managed-live "$first" "$second" "$@"
;; ;;
--*) --account)
# Bare options with no mode: an offline run with those options.
exec "${runner[@]}" "$@" exec "${runner[@]}" "$@"
;; ;;
--*)
# Bare options with no mode: an account run with those options.
exec "${runner[@]}" --account "$@"
;;
*) *)
collect_paths "$@" collect_paths "$@"
;; ;;