mirror of
https://github.com/narrrl/proton-drive-linux.git
synced 2026-11-03 16:41:26 +01:00
fix(fuse): skip unreadable folders in sync, never evict local-only files
An unreadable subfolder, such as a container's root-owned cache, failed every pass of its synced folder, so nothing else in the tree reached Drive (#33, B196). The scan now leaves such a path out, together with an entry that vanished mid-scan and a name that is not UTF-8, and the pass drops everything under it from classification, the wipe guard and the push pass's deletions. It is never read as deleted and never downloaded into. An unreadable or absent root still fails the pass. The activity feed names unreadable paths once each time the set changes. Two related bugs on the way to online-only: - The push pass read deletions from the raw baseline, so a synced path that later became ignored was trashed on Drive (B197). - The switch evicted everything once the folder was idle, including ignored paths such as .git/, symlinks, unreadable folders and files changed after the pass, none of which are on Drive (B198). It now refuses and names the path. Closes #33
This commit is contained in:
1 parent
c0d211238b
commit
7eb6aa0640
10 files changed
+778
-102
No files matched your search
@@ -842,6 +842,23 @@ impl Core {
|
||||
if folder.state != "idle" {
|
||||
return Err(SwitchBlocked::NotNow);
|
||||
}
|
||||
// `idle` only says the last pass applied everything it tried.
|
||||
// It never tries an ignored path, a symlink or a folder it
|
||||
// cannot read, it defers a file open for writing, and a file can
|
||||
// change after it ends. The eviction below deletes all of those
|
||||
// for good, so it waits until nothing here is missing from Drive
|
||||
// (B198).
|
||||
let baseline = self
|
||||
.db
|
||||
.sync_entries(id)
|
||||
.map_err(|e| SwitchBlocked::Failed(format!("db: {e:?}")))?;
|
||||
if let Some(reason) =
|
||||
sync::mirror_tree_unsynced(&local, &baseline, &self.ignore_rules(&local))
|
||||
{
|
||||
return Err(SwitchBlocked::Failed(format!(
|
||||
"going online-only would delete what is only on this computer: {reason}"
|
||||
)));
|
||||
}
|
||||
let root_uid = parse_uid(&folder.remote_uid).ok_or_else(|| {
|
||||
SwitchBlocked::Failed(format!("bad remote uid: {}", folder.remote_uid))
|
||||
})?;
|
||||
|
||||
@@ -744,6 +744,9 @@ struct Core {
|
||||
/// uploading that same tree — the engine would upload files as they vanish
|
||||
/// and then walk the FUSE mount as if it were local.
|
||||
sync_locks: Arc<Mutex<HashMap<i64, Arc<Mutex<()>>>>>,
|
||||
/// Per-sync-folder paths the last pass could not read, so the activity feed
|
||||
/// says so when the set changes rather than on every poll (B196).
|
||||
sync_unscanned: Arc<Mutex<HashMap<i64, Vec<String>>>>,
|
||||
/// Nodes this daemon changed on the remote itself, and how many echoes of
|
||||
/// those changes the event feed still owes us.
|
||||
///
|
||||
|
||||
@@ -475,6 +475,7 @@ pub(crate) fn mount_with(
|
||||
sync_tx,
|
||||
mounts: Arc::new(Mutex::new(SecondaryMountRegistry::default())),
|
||||
sync_locks: Arc::new(Mutex::new(HashMap::new())),
|
||||
sync_unscanned: Arc::new(Mutex::new(HashMap::new())),
|
||||
states: Arc::new(StateRegistry::default()),
|
||||
};
|
||||
// Before anything can queue work against it: the drain thread below reaches
|
||||
|
||||
+542
-99
@@ -59,8 +59,9 @@ pub(crate) use planner::base_name;
|
||||
#[cfg(test)]
|
||||
use planner::conflict_path;
|
||||
use planner::{
|
||||
FilePlan, classification_order, conflict_path_with_suffix, filter_baseline, guard_local_wipe,
|
||||
join_rel, parent_rel, plan_file, rel_to_path, remote_sig, unchanged_remote_size,
|
||||
FilePlan, Unscanned, classification_order, conflict_path_with_suffix, filter_baseline,
|
||||
guard_local_wipe, join_rel, local_deletions, parent_rel, plan_file, rel_to_path, remote_sig,
|
||||
tracked_baseline, unchanged_remote_size,
|
||||
};
|
||||
/// What a [`Pending`] op did, so the reconcile can update shared state on the
|
||||
/// engine thread (never inside a task).
|
||||
@@ -187,6 +188,7 @@ impl Core {
|
||||
"idle"
|
||||
};
|
||||
let _ = self.db.sync_folder_set_state(folder.id, state, now_secs());
|
||||
self.report_unscanned(folder.id, name, &outcome.unscanned);
|
||||
// Summarise a pass that actually moved something. A poll over an
|
||||
// unchanged folder does nothing and says nothing — otherwise the
|
||||
// feed would fill with "synced, 0 changes" every two minutes.
|
||||
@@ -243,6 +245,37 @@ impl Core {
|
||||
}
|
||||
}
|
||||
|
||||
/// Tell the activity feed what a pass over this folder had to leave out
|
||||
/// because it cannot be read, once each time that changes. Every other path
|
||||
/// in the folder still synced, so this does not make the folder's state an
|
||||
/// error; but a folder the user has no permission for is never going to
|
||||
/// reach Drive, and they should hear that once rather than not at all
|
||||
/// (B196).
|
||||
fn report_unscanned(&self, folder_id: i64, name: &str, unscanned: &[String]) {
|
||||
let previous = {
|
||||
let mut last = self.sync_unscanned.lock();
|
||||
if unscanned.is_empty() {
|
||||
last.remove(&folder_id)
|
||||
} else {
|
||||
last.insert(folder_id, unscanned.to_vec())
|
||||
}
|
||||
};
|
||||
if unscanned.is_empty() || previous.as_deref() == Some(unscanned) {
|
||||
return;
|
||||
}
|
||||
const SHOWN: usize = 3;
|
||||
let mut detail = unscanned[..unscanned.len().min(SHOWN)].join(", ");
|
||||
if unscanned.len() > SHOWN {
|
||||
detail.push_str(&format!(" and {} more", unscanned.len() - SHOWN));
|
||||
}
|
||||
self.log_activity(
|
||||
ActivityKind::Sync,
|
||||
format!("{} item(s) in {name} cannot be synced", unscanned.len()),
|
||||
detail,
|
||||
false,
|
||||
);
|
||||
}
|
||||
|
||||
/// Compile the ignore rules for a synced folder: the global patterns from
|
||||
/// the config, plus any `.pdfsignore` at `local_root`.
|
||||
///
|
||||
@@ -250,7 +283,7 @@ impl Core {
|
||||
/// than to "ignore nothing" — the defaults exclude build and VCS trees, and
|
||||
/// silently uploading those because a config read failed is the outcome this
|
||||
/// feature exists to prevent.
|
||||
fn ignore_rules(&self, local_root: &Path) -> IgnoreRules {
|
||||
pub(crate) fn ignore_rules(&self, local_root: &Path) -> IgnoreRules {
|
||||
let globals = match pdfs_core::config::AppDirs::new() {
|
||||
Ok(dirs) => dirs.load_config().resolved_ignore_patterns(),
|
||||
Err(e) => {
|
||||
@@ -313,15 +346,23 @@ impl Core {
|
||||
self.progress_scan_total(folder_id, baseline.len());
|
||||
|
||||
let mut local: HashMap<String, LocalItem> = HashMap::new();
|
||||
let mut unscanned = Unscanned::default();
|
||||
let writing = open_for_write_set(local_root);
|
||||
self.walk_local(
|
||||
folder_id, local_root, local_root, rules, &writing, &mut local,
|
||||
folder_id,
|
||||
local_root,
|
||||
rules,
|
||||
&writing,
|
||||
&mut local,
|
||||
&mut unscanned,
|
||||
)?;
|
||||
// The guard compares against the paths this pass could still see: an
|
||||
// ignored path is absent from `local` by rule, not by loss, and counting
|
||||
// those as missing would trip the guard on every pass — wedging the
|
||||
// folder's sync permanently — the moment a rule covers the whole tree.
|
||||
guard_local_wipe(&filter_baseline(&baseline, rules), &local)?;
|
||||
// ignored or unreadable path is absent from `local` by rule or by
|
||||
// failure, not by loss, and counting those as missing would trip the
|
||||
// guard on every pass — wedging the folder's sync permanently — the
|
||||
// moment a rule covers the whole tree.
|
||||
let tracked = tracked_baseline(&baseline, rules, &unscanned);
|
||||
guard_local_wipe(&tracked, &local)?;
|
||||
|
||||
// The remote folder uids come from the baseline instead of a walk: every
|
||||
// directory this device has synced recorded its uid there when it was
|
||||
@@ -391,6 +432,7 @@ impl Core {
|
||||
|
||||
let mut outcome = Outcome {
|
||||
deferred: local.values().filter(|item| item.open_for_write).count(),
|
||||
unscanned: unscanned.report(),
|
||||
..Default::default()
|
||||
};
|
||||
let mut order: Vec<&String> = local.keys().collect();
|
||||
@@ -515,13 +557,8 @@ impl Core {
|
||||
// showing the folder's remote contents — otherwise everything they deleted
|
||||
// comes back the moment the switch lands. Shallowest first, skipping anything
|
||||
// under a folder already trashed (its children went with it).
|
||||
let mut missing: Vec<&String> = baseline
|
||||
.keys()
|
||||
.filter(|rel| !local.contains_key(*rel))
|
||||
.collect();
|
||||
missing.sort_by_key(|p| p.matches('/').count());
|
||||
let mut trashed: Vec<String> = Vec::new();
|
||||
for rel in missing {
|
||||
for rel in &local_deletions(&tracked, &local) {
|
||||
if trashed
|
||||
.iter()
|
||||
.any(|dir| rel.starts_with(&format!("{dir}/")))
|
||||
@@ -586,12 +623,18 @@ impl Core {
|
||||
self.progress_scan_total(folder_id, baseline.len() * 2);
|
||||
|
||||
let mut local: HashMap<String, LocalItem> = HashMap::new();
|
||||
let mut unscanned = Unscanned::default();
|
||||
let writing = open_for_write_set(local_root);
|
||||
self.walk_local(
|
||||
folder_id, local_root, local_root, rules, &writing, &mut local,
|
||||
folder_id,
|
||||
local_root,
|
||||
rules,
|
||||
&writing,
|
||||
&mut local,
|
||||
&mut unscanned,
|
||||
)?;
|
||||
// See `push_pass`: the guard must not count rule-excluded paths as lost.
|
||||
guard_local_wipe(&filter_baseline(&baseline, rules), &local)?;
|
||||
// See `push_pass`: the guard must not count excluded paths as lost.
|
||||
guard_local_wipe(&tracked_baseline(&baseline, rules, &unscanned), &local)?;
|
||||
|
||||
let mut remote: HashMap<String, RemoteItem> = HashMap::new();
|
||||
let mut remote_dirs: HashMap<String, NodeUid> = HashMap::new();
|
||||
@@ -605,9 +648,15 @@ impl Core {
|
||||
&baseline,
|
||||
)?;
|
||||
|
||||
let order = classification_order(&local, &remote, &baseline, rules);
|
||||
let mut order = classification_order(&local, &remote, &baseline, rules);
|
||||
// A path the scan could not read is no more a local deletion than an
|
||||
// ignored one, and a remote path under it has nowhere to be written.
|
||||
order.retain(|rel| !unscanned.covers(rel));
|
||||
|
||||
let mut outcome = Outcome::default();
|
||||
let mut outcome = Outcome {
|
||||
unscanned: unscanned.report(),
|
||||
..Default::default()
|
||||
};
|
||||
// Folders to delete, collected here and removed deepest-first at the end
|
||||
// so a parent is never removed before its children.
|
||||
let mut delete_local_dirs: Vec<String> = Vec::new();
|
||||
@@ -873,19 +922,19 @@ impl Core {
|
||||
Ok(outcome)
|
||||
}
|
||||
|
||||
/// Recursively walk a local directory into `out`, keyed by `/`-joined relative
|
||||
/// path. Symlinks and other special files are skipped. Reports each entry to the
|
||||
/// pass's scan progress.
|
||||
/// Recursively walk a local tree into `out`, keyed by `/`-joined relative
|
||||
/// path. Symlinks and other special files are skipped, and paths that cannot
|
||||
/// be read go to `unscanned`. Reports each entry to the pass's scan progress.
|
||||
fn walk_local(
|
||||
&self,
|
||||
folder_id: i64,
|
||||
root: &Path,
|
||||
dir: &Path,
|
||||
rules: &IgnoreRules,
|
||||
writing: &HashSet<PathBuf>,
|
||||
out: &mut HashMap<String, LocalItem>,
|
||||
unscanned: &mut Unscanned,
|
||||
) -> Result<(), String> {
|
||||
walk_local_tree(root, dir, rules, writing, out, &mut |rel| {
|
||||
walk_local_tree(root, rules, writing, out, unscanned, &mut |rel| {
|
||||
self.progress_scanned(folder_id, base_name(rel))
|
||||
})
|
||||
}
|
||||
@@ -1582,51 +1631,120 @@ fn removed_locally(result: std::io::Result<()>) -> std::io::Result<()> {
|
||||
}
|
||||
|
||||
/// The body of [`Core::walk_local`], without a `Core`: recursively collect the
|
||||
/// local tree under `dir` into `out`, reporting each entry to `scanned`.
|
||||
/// local tree under `root` into `out`, reporting each entry to `scanned`.
|
||||
///
|
||||
/// **Every failure here is fatal to the pass, deliberately.** A `read_dir` that
|
||||
/// fails, an entry that cannot be stat-ed, a name that is not UTF-8 — each of
|
||||
/// them would, if skipped, leave `out` describing a *smaller* tree than the one
|
||||
/// on disk. Reconciliation reads a path present in the baseline and absent from
|
||||
/// the scan as "the user deleted this", and propagates that to Drive. So an
|
||||
/// unreadable subdirectory would trash its remote counterpart, and the deeper
|
||||
/// the failure, the more it takes with it (`docs/BUGS.md` B55). Refusing the
|
||||
/// whole pass keeps the previous baseline and leaves the remote alone; the next
|
||||
/// pass retries once the transient condition clears.
|
||||
/// **A path the scan cannot read is left out and recorded in `unscanned`, never
|
||||
/// silently dropped.** Every pass drops what `unscanned` covers from
|
||||
/// classification, so a baseline row under it is neither a local deletion nor a
|
||||
/// download target (`docs/BUGS.md` B55). That covers a subfolder the pass has no
|
||||
/// permission for, an entry that vanished between the listing and its `stat`,
|
||||
/// and a name that is not UTF-8. Failing the whole pass on any of them, as this
|
||||
/// did before, left one root-owned folder stopping everything else in the tree
|
||||
/// from syncing, on every pass, for good (B196).
|
||||
///
|
||||
/// The root is the exception, and so is any other I/O error: an absent or
|
||||
/// unreadable root fails the pass, since leaving the root out would leave
|
||||
/// nothing in, and an error this cannot name a cause for is not one to guess
|
||||
/// about.
|
||||
///
|
||||
/// Split out from the method so exactly that can be tested against a real tree
|
||||
/// with a real unreadable directory in it, which needs no `Core` at all.
|
||||
fn walk_local_tree(
|
||||
root: &Path,
|
||||
rules: &IgnoreRules,
|
||||
writing: &HashSet<PathBuf>,
|
||||
out: &mut HashMap<String, LocalItem>,
|
||||
unscanned: &mut Unscanned,
|
||||
scanned: &mut dyn FnMut(&str),
|
||||
) -> Result<(), String> {
|
||||
walk_local_dir(root, root, rules, writing, out, unscanned, scanned).map_err(|e| e.to_string())
|
||||
}
|
||||
|
||||
/// Why [`walk_local_dir`] could not finish a folder.
|
||||
enum ScanError {
|
||||
/// An I/O error on `what`, which the caller may leave out of the pass.
|
||||
Io { what: String, error: std::io::Error },
|
||||
/// Anything else, which fails the pass.
|
||||
Failed(String),
|
||||
}
|
||||
|
||||
impl std::fmt::Display for ScanError {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
match self {
|
||||
ScanError::Io { what, error } => write!(f, "{what}: {error}"),
|
||||
ScanError::Failed(message) => f.write_str(message),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Leave `rel` out of the pass for `failure`, or hand it back when it is one
|
||||
/// that fails the pass. Permission denied is reported to the user; an entry
|
||||
/// that vanished mid-scan is not, since the next pass sees it settled.
|
||||
fn leave_out(rel: &str, failure: ScanError, unscanned: &mut Unscanned) -> Result<(), ScanError> {
|
||||
let ScanError::Io { what, error } = &failure else {
|
||||
return Err(failure);
|
||||
};
|
||||
match error.kind() {
|
||||
std::io::ErrorKind::PermissionDenied => {
|
||||
warn!(%what, %error, "sync: cannot read; leaving it out of the pass");
|
||||
unscanned.unreadable(rel.to_string(), &error.to_string());
|
||||
Ok(())
|
||||
}
|
||||
std::io::ErrorKind::NotFound => {
|
||||
debug!(%what, "sync: gone mid-scan; leaving it out of the pass");
|
||||
unscanned.vanished(rel.to_string());
|
||||
Ok(())
|
||||
}
|
||||
_ => Err(failure),
|
||||
}
|
||||
}
|
||||
|
||||
fn walk_local_dir(
|
||||
root: &Path,
|
||||
dir: &Path,
|
||||
rules: &IgnoreRules,
|
||||
writing: &HashSet<PathBuf>,
|
||||
out: &mut HashMap<String, LocalItem>,
|
||||
unscanned: &mut Unscanned,
|
||||
scanned: &mut dyn FnMut(&str),
|
||||
) -> Result<(), String> {
|
||||
let entries = std::fs::read_dir(dir).map_err(|e| format!("read {}: {e}", dir.display()))?;
|
||||
) -> Result<(), ScanError> {
|
||||
let io = |what: String| move |error| ScanError::Io { what, error };
|
||||
let entries = std::fs::read_dir(dir).map_err(io(format!("read {}", dir.display())))?;
|
||||
for entry in entries {
|
||||
let entry = entry.map_err(|e| format!("read entry in {}: {e}", dir.display()))?;
|
||||
let entry = entry.map_err(io(format!("read entry in {}", dir.display())))?;
|
||||
let path = entry.path();
|
||||
let meta = std::fs::symlink_metadata(&path)
|
||||
.map_err(|e| format!("stat {}: {e}", path.display()))?;
|
||||
let stripped = path.strip_prefix(root).map_err(|e| {
|
||||
format!(
|
||||
ScanError::Failed(format!(
|
||||
"local path {} escaped sync root {}: {e}",
|
||||
path.display(),
|
||||
root.display()
|
||||
)
|
||||
})?;
|
||||
let rel = stripped.to_str().ok_or_else(|| {
|
||||
format!(
|
||||
"local path {} is not valid UTF-8; refusing a destructive sync pass",
|
||||
path.display()
|
||||
)
|
||||
))
|
||||
})?;
|
||||
let Some(rel) = stripped.to_str() else {
|
||||
// Nothing can be keyed by this name: not the baseline, which only
|
||||
// ever held paths this scan produced, and not Drive, whose names are
|
||||
// UTF-8. Leaving it out cannot read as a deletion of anything.
|
||||
warn!(path = %path.display(), "sync: name is not valid UTF-8; leaving it out of the pass");
|
||||
unscanned.unreadable(
|
||||
stripped.to_string_lossy().into_owned(),
|
||||
"name is not valid UTF-8",
|
||||
);
|
||||
continue;
|
||||
};
|
||||
// Ignore our own in-flight download temp files.
|
||||
if rel.contains(".pdfs-tmp-") {
|
||||
continue;
|
||||
}
|
||||
let meta = match std::fs::symlink_metadata(&path) {
|
||||
Ok(meta) => meta,
|
||||
// Gone since the listing, or in a folder that lists but does not
|
||||
// allow `stat`. Either way this pass cannot say what it is.
|
||||
Err(error) => {
|
||||
let what = format!("stat {}", path.display());
|
||||
leave_out(rel, ScanError::Io { what, error }, unscanned)?;
|
||||
continue;
|
||||
}
|
||||
};
|
||||
// Ignored paths are dropped here as well as from the classification
|
||||
// union — not for correctness (the union filter is what makes this
|
||||
// safe) but so an ignored `node_modules/` is never descended into or
|
||||
@@ -1646,7 +1764,15 @@ fn walk_local_tree(
|
||||
},
|
||||
);
|
||||
scanned(rel);
|
||||
walk_local_tree(root, &path, rules, writing, out, scanned)?;
|
||||
if let Err(failure) =
|
||||
walk_local_dir(root, &path, rules, writing, out, unscanned, scanned)
|
||||
{
|
||||
leave_out(rel, failure, unscanned)?;
|
||||
// What the folder did list before failing goes with it: a
|
||||
// half-read folder is as unknown as an unread one.
|
||||
let prefix = format!("{rel}/");
|
||||
out.retain(|k, _| k != rel && !k.starts_with(&prefix));
|
||||
}
|
||||
} else if meta.is_file() {
|
||||
// Check whether any process holds this file open for writing.
|
||||
// The canonical path is used because /proc/*/fd links resolve
|
||||
@@ -1684,7 +1810,7 @@ pub(crate) fn mirror_subtree_unsynced(
|
||||
baseline: &HashMap<String, StoredSyncEntry>,
|
||||
) -> Option<String> {
|
||||
let mut seen = HashSet::new();
|
||||
if let Some(reason) = unsynced_local(root, rel, baseline, &mut seen) {
|
||||
if let Some(reason) = unsynced_local(root, rel, baseline, None, &mut seen) {
|
||||
return Some(reason);
|
||||
}
|
||||
let prefix = format!("{rel}/");
|
||||
@@ -1695,45 +1821,93 @@ pub(crate) fn mirror_subtree_unsynced(
|
||||
.map(|r| format!("{r} was deleted here but not on Proton Drive yet"))
|
||||
}
|
||||
|
||||
/// [`mirror_subtree_unsynced`] for a whole mirror folder: why its local copy
|
||||
/// holds something Proton Drive does not, or `None` when evicting it loses
|
||||
/// nothing. The switch to on-demand asks this before it empties the folder.
|
||||
///
|
||||
/// A push pass that ends `idle` is not enough on its own. It never uploads an
|
||||
/// ignored path, a symlink or a folder it cannot read, it defers a file open
|
||||
/// for writing, and a file can change after it finishes; the eviction deleted
|
||||
/// all of those for good (B198).
|
||||
///
|
||||
/// A tracked path deleted here but still in the baseline counts, since the
|
||||
/// mount would show it again. An ignored one does not: ignoring never carries
|
||||
/// a deletion to Drive, so its row stays in the baseline for good, and holding
|
||||
/// the switch on it would hold it forever.
|
||||
pub(crate) fn mirror_tree_unsynced(
|
||||
root: &Path,
|
||||
baseline: &HashMap<String, StoredSyncEntry>,
|
||||
rules: &IgnoreRules,
|
||||
) -> Option<String> {
|
||||
let mut seen = HashSet::new();
|
||||
if let Some(reason) = unsynced_local(root, "", baseline, Some(rules), &mut seen) {
|
||||
return Some(reason);
|
||||
}
|
||||
let mut missing: Vec<String> = filter_baseline(baseline, rules)
|
||||
.into_keys()
|
||||
.filter(|r| !seen.contains(r.as_str()))
|
||||
.collect();
|
||||
missing.sort_unstable();
|
||||
missing
|
||||
.first()
|
||||
.map(|r| format!("{r} was deleted here but not on Proton Drive yet"))
|
||||
}
|
||||
|
||||
/// The walk behind both checks. `rules`, when given, only names an ignored
|
||||
/// path as such in the reason; it is still something only this disk has.
|
||||
fn unsynced_local(
|
||||
root: &Path,
|
||||
rel: &str,
|
||||
baseline: &HashMap<String, StoredSyncEntry>,
|
||||
rules: Option<&IgnoreRules>,
|
||||
seen: &mut HashSet<String>,
|
||||
) -> Option<String> {
|
||||
let path = root.join(rel_to_path(rel));
|
||||
let meta = match std::fs::symlink_metadata(&path) {
|
||||
Ok(meta) => meta,
|
||||
Err(e) => return Some(format!("cannot read {rel}: {e}")),
|
||||
let path = if rel.is_empty() {
|
||||
root.to_path_buf()
|
||||
} else {
|
||||
root.join(rel_to_path(rel))
|
||||
};
|
||||
let Some(base) = baseline.get(rel) else {
|
||||
return Some(format!("{rel} is not on Proton Drive yet"));
|
||||
};
|
||||
seen.insert(rel.to_string());
|
||||
if meta.is_file() {
|
||||
return (!LocalSig::from(&meta).same_content(&LocalSig::from(base)))
|
||||
.then(|| format!("{rel} has changes that are not on Proton Drive yet"));
|
||||
}
|
||||
if !meta.is_dir() {
|
||||
return Some(format!("{rel} is not a regular file or folder"));
|
||||
// The folder itself has no baseline row; only what is in it does.
|
||||
if !rel.is_empty() {
|
||||
let meta = match std::fs::symlink_metadata(&path) {
|
||||
Ok(meta) => meta,
|
||||
Err(e) => return Some(format!("cannot read {rel}: {e}")),
|
||||
};
|
||||
let Some(base) = baseline.get(rel) else {
|
||||
if rules.is_some_and(|rules| rules.is_ignored(rel, meta.is_dir())) {
|
||||
return Some(format!(
|
||||
"{rel} is excluded from sync, so it is only on this computer"
|
||||
));
|
||||
}
|
||||
return Some(format!("{rel} is not on Proton Drive yet"));
|
||||
};
|
||||
seen.insert(rel.to_string());
|
||||
if meta.is_file() {
|
||||
return (!LocalSig::from(&meta).same_content(&LocalSig::from(base)))
|
||||
.then(|| format!("{rel} has changes that are not on Proton Drive yet"));
|
||||
}
|
||||
if !meta.is_dir() {
|
||||
return Some(format!("{rel} is not a regular file or folder"));
|
||||
}
|
||||
}
|
||||
let shown = if rel.is_empty() { "the folder" } else { rel };
|
||||
let entries = match std::fs::read_dir(&path) {
|
||||
Ok(entries) => entries,
|
||||
Err(e) => return Some(format!("cannot read {rel}: {e}")),
|
||||
Err(e) => return Some(format!("cannot read {shown}: {e}")),
|
||||
};
|
||||
for entry in entries {
|
||||
let name = match entry {
|
||||
Ok(entry) => entry.file_name(),
|
||||
Err(e) => return Some(format!("cannot read {rel}: {e}")),
|
||||
Err(e) => return Some(format!("cannot read {shown}: {e}")),
|
||||
};
|
||||
let Some(name) = name.to_str() else {
|
||||
return Some(format!("{rel} holds a name that is not valid UTF-8"));
|
||||
return Some(format!("{shown} holds a name that is not valid UTF-8"));
|
||||
};
|
||||
// A download the engine left half done; it is the engine's to clean up.
|
||||
if name.contains(".pdfs-tmp-") {
|
||||
continue;
|
||||
}
|
||||
if let Some(reason) = unsynced_local(root, &format!("{rel}/{name}"), baseline, seen) {
|
||||
if let Some(reason) = unsynced_local(root, &join_rel(rel, name), baseline, rules, seen) {
|
||||
return Some(reason);
|
||||
}
|
||||
}
|
||||
@@ -2154,7 +2328,15 @@ mod tests {
|
||||
let writing = HashSet::new();
|
||||
let mut local: HashMap<String, LocalItem> = HashMap::new();
|
||||
let mut progress = no_progress();
|
||||
walk_local_tree(&root, &root, &rules, &writing, &mut local, &mut progress).unwrap();
|
||||
walk_local_tree(
|
||||
&root,
|
||||
&rules,
|
||||
&writing,
|
||||
&mut local,
|
||||
&mut Unscanned::default(),
|
||||
&mut progress,
|
||||
)
|
||||
.unwrap();
|
||||
assert!(local.is_empty(), "the replacement root is genuinely empty");
|
||||
|
||||
for size in 1..=3 {
|
||||
@@ -2170,7 +2352,15 @@ mod tests {
|
||||
// remounted folder resumes rather than staying wedged.
|
||||
std::fs::write(root.join("file0.txt"), b"back").unwrap();
|
||||
let mut local: HashMap<String, LocalItem> = HashMap::new();
|
||||
walk_local_tree(&root, &root, &rules, &writing, &mut local, &mut progress).unwrap();
|
||||
walk_local_tree(
|
||||
&root,
|
||||
&rules,
|
||||
&writing,
|
||||
&mut local,
|
||||
&mut Unscanned::default(),
|
||||
&mut progress,
|
||||
)
|
||||
.unwrap();
|
||||
let baseline: HashMap<String, ()> = [("file0.txt".to_string(), ())].into_iter().collect();
|
||||
assert!(guard_local_wipe(&filter_baseline(&baseline, &rules), &local).is_ok());
|
||||
|
||||
@@ -2475,33 +2665,176 @@ mod tests {
|
||||
|_: &str| {}
|
||||
}
|
||||
|
||||
/// A directory the pass cannot read must fail the pass. Omitting it would
|
||||
/// hand reconciliation a baseline-minus-subtree, which it reads as the user
|
||||
/// having deleted that subtree and propagates to Drive (B55).
|
||||
/// Scan `root` with no rules, returning what it found and left out.
|
||||
fn scan(root: &Path) -> Result<(HashMap<String, LocalItem>, Unscanned), String> {
|
||||
let mut out = HashMap::new();
|
||||
let mut unscanned = Unscanned::default();
|
||||
let mut progress = no_progress();
|
||||
walk_local_tree(
|
||||
root,
|
||||
&rules_for(&[]),
|
||||
&HashSet::new(),
|
||||
&mut out,
|
||||
&mut unscanned,
|
||||
&mut progress,
|
||||
)?;
|
||||
Ok((out, unscanned))
|
||||
}
|
||||
|
||||
fn sorted_keys<V>(map: &HashMap<String, V>) -> Vec<&str> {
|
||||
let mut keys: Vec<&str> = map.keys().map(String::as_str).collect();
|
||||
keys.sort_unstable();
|
||||
keys
|
||||
}
|
||||
|
||||
/// B196, the report in #33: a folder the user may not read, such as a
|
||||
/// container's root-owned cache, is left out of the pass and named, and
|
||||
/// everything around it still scans. It used to fail the whole pass, so
|
||||
/// nothing else in the tree ever synced again.
|
||||
#[test]
|
||||
fn an_unreadable_subdirectory_fails_the_scan_instead_of_shrinking_it() {
|
||||
fn an_unreadable_subdirectory_is_left_out_and_the_rest_scans() {
|
||||
use std::os::unix::fs::PermissionsExt as _;
|
||||
|
||||
let root = sync_test_dir("scan-eacces");
|
||||
std::fs::write(root.join("keep.txt"), b"a").unwrap();
|
||||
let locked = root.join("locked");
|
||||
std::fs::create_dir_all(root.join("docker/cache")).unwrap();
|
||||
std::fs::write(root.join("docker/compose.yml"), b"c").unwrap();
|
||||
let locked = root.join("docker/cache/locked");
|
||||
std::fs::create_dir(&locked).unwrap();
|
||||
std::fs::write(locked.join("inner.txt"), b"b").unwrap();
|
||||
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o000)).unwrap();
|
||||
|
||||
let rules = rules_for(&[]);
|
||||
let writing = HashSet::new();
|
||||
let mut out = HashMap::new();
|
||||
let mut progress = no_progress();
|
||||
let result = walk_local_tree(&root, &root, &rules, &writing, &mut out, &mut progress);
|
||||
let result = scan(&root);
|
||||
|
||||
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o700)).unwrap();
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
|
||||
let error = result.expect_err("an unreadable directory must abort the pass");
|
||||
assert!(
|
||||
error.contains("locked"),
|
||||
"the error should name the subtree it could not read: {error}"
|
||||
let (out, unscanned) = result.expect("one unreadable folder must not fail the pass");
|
||||
assert_eq!(
|
||||
sorted_keys(&out),
|
||||
vec!["docker", "docker/cache", "docker/compose.yml", "keep.txt"]
|
||||
);
|
||||
assert!(unscanned.covers("docker/cache/locked"));
|
||||
assert!(unscanned.covers("docker/cache/locked/inner.txt"));
|
||||
assert!(!unscanned.covers("docker/cache"));
|
||||
let report = unscanned.report();
|
||||
assert_eq!(report.len(), 1);
|
||||
assert!(report[0].starts_with("docker/cache/locked ("), "{report:?}");
|
||||
}
|
||||
|
||||
/// A folder with read but no search permission lists its names but cannot
|
||||
/// `stat` them. Whatever it did list must go with it: a half-read folder
|
||||
/// is as unknown as an unread one.
|
||||
#[test]
|
||||
fn a_listable_but_unsearchable_folder_is_left_out_whole() {
|
||||
use std::os::unix::fs::PermissionsExt as _;
|
||||
|
||||
let root = sync_test_dir("scan-no-search");
|
||||
std::fs::write(root.join("keep.txt"), b"a").unwrap();
|
||||
let half = root.join("half");
|
||||
std::fs::create_dir(&half).unwrap();
|
||||
std::fs::write(half.join("one.txt"), b"1").unwrap();
|
||||
std::fs::write(half.join("two.txt"), b"2").unwrap();
|
||||
std::fs::set_permissions(&half, std::fs::Permissions::from_mode(0o400)).unwrap();
|
||||
|
||||
let result = scan(&root);
|
||||
|
||||
std::fs::set_permissions(&half, std::fs::Permissions::from_mode(0o700)).unwrap();
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
|
||||
let (out, unscanned) = result.unwrap();
|
||||
// Each entry fails its own `stat` and is left out by name.
|
||||
assert_eq!(sorted_keys(&out), vec!["half", "keep.txt"]);
|
||||
assert!(unscanned.covers("half/one.txt"));
|
||||
assert!(unscanned.covers("half/two.txt"));
|
||||
}
|
||||
|
||||
/// The other half of B196: what was left out is not a deletion, and not a
|
||||
/// download target either. Its baseline rows leave the wipe guard's set
|
||||
/// and a push pass's deletions, and its paths leave classification.
|
||||
#[test]
|
||||
fn a_path_the_scan_left_out_is_never_a_local_deletion() {
|
||||
let mut baseline = baseline_with("keep.txt", 1, 1);
|
||||
baseline.extend(baseline_with("locked", 0, 0));
|
||||
baseline.extend(baseline_with("locked/inner.txt", 1, 1));
|
||||
baseline.extend(baseline_with("gone.txt", 1, 1));
|
||||
let local: HashMap<String, LocalItem> = [("keep.txt".to_string(), local_item(false))]
|
||||
.into_iter()
|
||||
.collect();
|
||||
let mut unscanned = Unscanned::default();
|
||||
unscanned.unreadable("locked".to_string(), "Permission denied");
|
||||
let rules = rules_for(&[]);
|
||||
|
||||
let tracked = tracked_baseline(&baseline, &rules, &unscanned);
|
||||
assert_eq!(sorted_keys(&tracked), vec!["gone.txt", "keep.txt"]);
|
||||
assert_eq!(local_deletions(&tracked, &local), vec!["gone.txt"]);
|
||||
|
||||
// Every tracked path but one is gone, and the guard still lets the
|
||||
// pass run: one survivor proves the tree is there.
|
||||
assert!(guard_local_wipe(&tracked, &local).is_ok());
|
||||
// With everything readable left out, nothing is tracked, so nothing
|
||||
// can be lost and the guard has nothing to object to.
|
||||
let mut all = Unscanned::default();
|
||||
all.unreadable("keep.txt".to_string(), "Permission denied");
|
||||
all.unreadable("locked".to_string(), "Permission denied");
|
||||
all.vanished("gone.txt".to_string());
|
||||
assert!(tracked_baseline(&baseline, &rules, &all).is_empty());
|
||||
|
||||
let remote: HashMap<String, RemoteItem> = HashMap::new();
|
||||
let mut order = classification_order(&local, &remote, &baseline, &rules);
|
||||
order.retain(|rel| !unscanned.covers(rel));
|
||||
order.sort_unstable();
|
||||
assert_eq!(order, vec!["gone.txt", "keep.txt"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unscanned_covers_a_path_and_what_is_under_it_but_not_its_siblings() {
|
||||
let mut unscanned = Unscanned::default();
|
||||
unscanned.unreadable("a/locked".to_string(), "Permission denied");
|
||||
unscanned.vanished("b.txt".to_string());
|
||||
|
||||
assert!(unscanned.covers("a/locked"));
|
||||
assert!(unscanned.covers("a/locked/x/y.txt"));
|
||||
assert!(unscanned.covers("b.txt"));
|
||||
assert!(!unscanned.covers("a"));
|
||||
assert!(!unscanned.covers("a/locked2"));
|
||||
assert!(!unscanned.covers("b.txt.bak"));
|
||||
// A file that only vanished mid-scan is not worth telling anyone about.
|
||||
assert_eq!(unscanned.report(), vec!["a/locked (Permission denied)"]);
|
||||
}
|
||||
|
||||
/// B197: a push pass read its deletions from the raw baseline, so a path
|
||||
/// synced before it became ignored, which the walk skips, was trashed on
|
||||
/// Drive, and the switch the push pass ran for then evicted the local copy.
|
||||
#[test]
|
||||
fn a_push_pass_never_reads_an_ignored_path_as_deleted() {
|
||||
let mut baseline = baseline_with("notes.md", 1, 1);
|
||||
baseline.extend(baseline_with("node_modules", 0, 0));
|
||||
baseline.extend(baseline_with("node_modules/x/index.js", 1, 1));
|
||||
baseline.extend(baseline_with("old.md", 1, 1));
|
||||
let local: HashMap<String, LocalItem> = [("notes.md".to_string(), local_item(false))]
|
||||
.into_iter()
|
||||
.collect();
|
||||
let rules = rules_for(&["node_modules/"]);
|
||||
|
||||
let tracked = tracked_baseline(&baseline, &rules, &Unscanned::default());
|
||||
|
||||
assert_eq!(local_deletions(&tracked, &local), vec!["old.md"]);
|
||||
}
|
||||
|
||||
/// Deletions come out shallowest first, so a trashed folder can take its
|
||||
/// children with it.
|
||||
#[test]
|
||||
fn local_deletions_come_out_shallowest_first() {
|
||||
let mut baseline = baseline_with("d/e/f.txt", 1, 1);
|
||||
baseline.extend(baseline_with("d", 0, 0));
|
||||
baseline.extend(baseline_with("d/e", 0, 0));
|
||||
let local: HashMap<String, LocalItem> = HashMap::new();
|
||||
let tracked = tracked_baseline(&baseline, &rules_for(&[]), &Unscanned::default());
|
||||
|
||||
assert_eq!(
|
||||
local_deletions(&tracked, &local),
|
||||
vec!["d", "d/e", "d/e/f.txt"]
|
||||
);
|
||||
}
|
||||
|
||||
@@ -2593,6 +2926,94 @@ mod tests {
|
||||
assert_eq!(reason, None);
|
||||
}
|
||||
|
||||
// ---- B198: going online-only never evicts what Drive does not have -----
|
||||
|
||||
#[test]
|
||||
fn a_fully_synced_mirror_folder_may_go_online_only() {
|
||||
let (root, baseline) = synced_subtree("evict-synced");
|
||||
let reason = mirror_tree_unsynced(&root, &baseline, &rules_for(&[]));
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
assert_eq!(reason, None);
|
||||
}
|
||||
|
||||
/// The case that lost data: `.git/` is ignored by default, so no pass ever
|
||||
/// uploads it, and the eviction deleted it with the rest.
|
||||
#[test]
|
||||
fn an_ignored_folder_only_this_disk_has_holds_the_switch_back() {
|
||||
let (root, baseline) = synced_subtree("evict-ignored");
|
||||
std::fs::create_dir(root.join(".git")).unwrap();
|
||||
std::fs::write(root.join(".git/HEAD"), b"ref").unwrap();
|
||||
let reason = mirror_tree_unsynced(&root, &baseline, &rules_for(&[".git/"]));
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
let reason = reason.expect("an ignored folder is only on this disk");
|
||||
assert!(
|
||||
reason.contains(".git") && reason.contains("excluded"),
|
||||
"{reason}"
|
||||
);
|
||||
}
|
||||
|
||||
/// An ignored row stays in the baseline for good once its file is gone, so
|
||||
/// counting it as a deletion not carried up would hold the switch forever.
|
||||
#[test]
|
||||
fn a_deleted_ignored_path_does_not_hold_the_switch_back() {
|
||||
let (root, mut baseline) = synced_subtree("evict-ignored-gone");
|
||||
baseline.insert(
|
||||
"build/out.o".to_string(),
|
||||
StoredSyncEntry {
|
||||
rel_path: "build/out.o".to_string(),
|
||||
..baseline["dir/a.txt"].clone()
|
||||
},
|
||||
);
|
||||
let rules = rules_for(&["build/"]);
|
||||
let ignored = mirror_tree_unsynced(&root, &baseline, &rules);
|
||||
let tracked = mirror_tree_unsynced(&root, &baseline, &rules_for(&[]));
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
assert_eq!(ignored, None);
|
||||
assert!(tracked.unwrap().contains("build/out.o"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_edit_after_the_last_pass_holds_the_switch_back() {
|
||||
let (root, baseline) = synced_subtree("evict-edit");
|
||||
std::fs::write(root.join("dir/sub/b.txt"), b"edited since").unwrap();
|
||||
let reason = mirror_tree_unsynced(&root, &baseline, &rules_for(&[]));
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
assert!(reason.unwrap().contains("dir/sub/b.txt"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_new_file_at_the_root_holds_the_switch_back() {
|
||||
let (root, baseline) = synced_subtree("evict-new");
|
||||
std::fs::write(root.join("new.txt"), b"n").unwrap();
|
||||
let reason = mirror_tree_unsynced(&root, &baseline, &rules_for(&[]));
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
assert!(reason.unwrap().contains("new.txt"));
|
||||
}
|
||||
|
||||
/// The folder B196 now leaves out of the pass instead of failing it is,
|
||||
/// by the same token, something the eviction must not delete.
|
||||
#[test]
|
||||
fn an_unreadable_folder_holds_the_switch_back() {
|
||||
use std::os::unix::fs::PermissionsExt as _;
|
||||
|
||||
let (root, baseline) = synced_subtree("evict-eacces");
|
||||
let locked = root.join("dir/sub");
|
||||
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o000)).unwrap();
|
||||
let reason = mirror_tree_unsynced(&root, &baseline, &rules_for(&[]));
|
||||
std::fs::set_permissions(&locked, std::fs::Permissions::from_mode(0o700)).unwrap();
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
assert!(reason.unwrap().contains("dir/sub"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_symlink_holds_the_switch_back() {
|
||||
let (root, baseline) = synced_subtree("evict-symlink");
|
||||
std::os::unix::fs::symlink("dir/a.txt", root.join("link")).unwrap();
|
||||
let reason = mirror_tree_unsynced(&root, &baseline, &rules_for(&[]));
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
assert!(reason.unwrap().contains("link"));
|
||||
}
|
||||
|
||||
/// The same rule one level up: if the sync root itself cannot be read, the
|
||||
/// pass sees nothing, and "nothing" must never mean "delete everything".
|
||||
#[test]
|
||||
@@ -2607,7 +3028,14 @@ mod tests {
|
||||
let writing = HashSet::new();
|
||||
let mut out = HashMap::new();
|
||||
let mut progress = no_progress();
|
||||
let result = walk_local_tree(&root, &root, &rules, &writing, &mut out, &mut progress);
|
||||
let result = walk_local_tree(
|
||||
&root,
|
||||
&rules,
|
||||
&writing,
|
||||
&mut out,
|
||||
&mut Unscanned::default(),
|
||||
&mut progress,
|
||||
);
|
||||
|
||||
std::fs::set_permissions(&root, std::fs::Permissions::from_mode(0o700)).unwrap();
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
@@ -2628,30 +3056,40 @@ mod tests {
|
||||
let writing = HashSet::new();
|
||||
let mut out = HashMap::new();
|
||||
let mut progress = no_progress();
|
||||
assert!(walk_local_tree(&root, &root, &rules, &writing, &mut out, &mut progress).is_err());
|
||||
assert!(
|
||||
walk_local_tree(
|
||||
&root,
|
||||
&rules,
|
||||
&writing,
|
||||
&mut out,
|
||||
&mut Unscanned::default(),
|
||||
&mut progress
|
||||
)
|
||||
.is_err()
|
||||
);
|
||||
assert!(out.is_empty());
|
||||
}
|
||||
|
||||
/// A name that is not valid UTF-8 cannot be keyed into the scan, so it
|
||||
/// cannot be compared against the baseline either. Refusing the pass is the
|
||||
/// only safe reading: skipping it would classify the file as deleted.
|
||||
/// A name that is not valid UTF-8 cannot be keyed into the scan, but it
|
||||
/// cannot be in the baseline or on Drive either, so leaving it out reads as
|
||||
/// the deletion of nothing. It used to fail the pass (B196).
|
||||
#[test]
|
||||
fn a_non_utf8_name_fails_the_scan() {
|
||||
fn a_non_utf8_name_is_left_out_and_reported() {
|
||||
use std::ffi::OsStr;
|
||||
use std::os::unix::ffi::OsStrExt as _;
|
||||
|
||||
let root = sync_test_dir("scan-non-utf8");
|
||||
std::fs::write(root.join(OsStr::from_bytes(b"bad\xffname")), b"x").unwrap();
|
||||
std::fs::write(root.join("good.txt"), b"y").unwrap();
|
||||
|
||||
let rules = rules_for(&[]);
|
||||
let writing = HashSet::new();
|
||||
let mut out = HashMap::new();
|
||||
let mut progress = no_progress();
|
||||
let result = walk_local_tree(&root, &root, &rules, &writing, &mut out, &mut progress);
|
||||
let result = scan(&root);
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
|
||||
let error = result.expect_err("a name the scan cannot key must abort the pass");
|
||||
assert!(error.contains("UTF-8"), "{error}");
|
||||
let (out, unscanned) = result.unwrap();
|
||||
assert_eq!(sorted_keys(&out), vec!["good.txt"]);
|
||||
let report = unscanned.report();
|
||||
assert_eq!(report.len(), 1);
|
||||
assert!(report[0].contains("UTF-8"), "{report:?}");
|
||||
}
|
||||
|
||||
/// The complement: a scan that *can* read everything reports everything, so
|
||||
@@ -2670,9 +3108,14 @@ mod tests {
|
||||
let writing = HashSet::new();
|
||||
let mut out = HashMap::new();
|
||||
let mut seen: Vec<String> = Vec::new();
|
||||
let result = walk_local_tree(&root, &root, &rules, &writing, &mut out, &mut |rel| {
|
||||
seen.push(rel.to_string())
|
||||
});
|
||||
let result = walk_local_tree(
|
||||
&root,
|
||||
&rules,
|
||||
&writing,
|
||||
&mut out,
|
||||
&mut Unscanned::default(),
|
||||
&mut |rel| seen.push(rel.to_string()),
|
||||
);
|
||||
std::fs::remove_dir_all(&root).unwrap();
|
||||
|
||||
result.unwrap();
|
||||
|
||||
@@ -98,6 +98,11 @@ pub(super) struct Outcome {
|
||||
pub(super) errors: usize,
|
||||
/// Files skipped because another process held them open for writing.
|
||||
pub(super) deferred: usize,
|
||||
/// Paths the local scan could not read, left out of the pass. Not part of
|
||||
/// [`is_empty`](Self::is_empty) or the summary: an unreadable folder stays
|
||||
/// unreadable from one poll to the next, so the pass reports it once, when
|
||||
/// the set changes, rather than on every poll.
|
||||
pub(super) unscanned: Vec<String>,
|
||||
}
|
||||
|
||||
impl Outcome {
|
||||
|
||||
@@ -90,6 +90,96 @@ pub(super) fn filter_baseline<'a, B>(
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The paths a local scan left out because it could not read them: a folder
|
||||
/// it has no permission for, an entry that vanished between the listing and
|
||||
/// its `stat`, a name that is not UTF-8.
|
||||
///
|
||||
/// A left-out path is not a deleted one. A pass drops everything an entry
|
||||
/// covers from classification, exactly as it drops ignored paths, so a
|
||||
/// baseline row under an unreadable folder is neither read as a local deletion
|
||||
/// nor answered with a download into a folder the pass cannot write. Failing
|
||||
/// the whole pass instead is what B55 did, and it left one root-owned folder
|
||||
/// stopping every other file in the tree from syncing (B196).
|
||||
#[derive(Default)]
|
||||
pub(super) struct Unscanned {
|
||||
covered: HashSet<String>,
|
||||
/// The subset worth telling the user about, with why: an entry that only
|
||||
/// vanished mid-scan is not, since the next pass sees it settled.
|
||||
unreadable: Vec<String>,
|
||||
}
|
||||
|
||||
impl Unscanned {
|
||||
/// Leave out `rel`, which vanished while the scan looked at it.
|
||||
pub(super) fn vanished(&mut self, rel: String) {
|
||||
self.covered.insert(rel);
|
||||
}
|
||||
|
||||
/// Leave out `rel`, which cannot be read for `reason`.
|
||||
pub(super) fn unreadable(&mut self, rel: String, reason: &str) {
|
||||
self.unreadable.push(format!("{rel} ({reason})"));
|
||||
self.covered.insert(rel);
|
||||
}
|
||||
|
||||
/// Whether `rel` is a left-out path or lies under one.
|
||||
pub(super) fn covers(&self, rel: &str) -> bool {
|
||||
if self.covered.is_empty() {
|
||||
return false;
|
||||
}
|
||||
let mut at = rel;
|
||||
loop {
|
||||
if self.covered.contains(at) {
|
||||
return true;
|
||||
}
|
||||
match at.rfind('/') {
|
||||
Some(i) => at = &at[..i],
|
||||
None => return false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The unreadable paths with their reasons, sorted, for the activity feed.
|
||||
pub(super) fn report(&self) -> Vec<String> {
|
||||
let mut lines = self.unreadable.clone();
|
||||
lines.sort_unstable();
|
||||
lines
|
||||
}
|
||||
}
|
||||
|
||||
/// The baseline paths a pass can see: [`filter_baseline`], minus everything
|
||||
/// under a path the scan could not read. This is the set [`guard_local_wipe`]
|
||||
/// checks and the set a push pass reads deletions from, since a path outside
|
||||
/// it is absent from the local walk by rule or by failure, never by loss.
|
||||
pub(super) fn tracked_baseline<'a, B>(
|
||||
baseline: &'a HashMap<String, B>,
|
||||
rules: &IgnoreRules,
|
||||
unscanned: &Unscanned,
|
||||
) -> HashMap<String, &'a B> {
|
||||
let mut tracked = filter_baseline(baseline, rules);
|
||||
tracked.retain(|rel, _| !unscanned.covers(rel));
|
||||
tracked
|
||||
}
|
||||
|
||||
/// The baseline paths a push pass has to trash on Drive: tracked, and gone
|
||||
/// from the local walk. Shallowest first, so a trashed folder takes its
|
||||
/// children with it.
|
||||
///
|
||||
/// Built from [`tracked_baseline`] rather than from the raw baseline. Reading
|
||||
/// it raw trashed every synced path that later became ignored, because the walk
|
||||
/// skips those, and the switch to on-demand that the push pass runs for then
|
||||
/// deleted the local copy too (B197).
|
||||
pub(super) fn local_deletions<B, L>(
|
||||
tracked: &HashMap<String, &B>,
|
||||
local: &HashMap<String, L>,
|
||||
) -> Vec<String> {
|
||||
let mut missing: Vec<String> = tracked
|
||||
.keys()
|
||||
.filter(|rel| !local.contains_key(*rel))
|
||||
.cloned()
|
||||
.collect();
|
||||
missing.sort_by_key(|p| p.matches('/').count());
|
||||
missing
|
||||
}
|
||||
|
||||
/// Refuse to run a pass whose local side has vanished in its entirety.
|
||||
///
|
||||
/// When every baseline path is absent locally, the likely cause is an unavailable
|
||||
|
||||
@@ -310,8 +310,10 @@ its changes go through the queue like My files.
|
||||
A filesystem watcher (`notify`) and a remote poll every 120 s feed a debounced pass (2 s, at most
|
||||
30 s). Passes are serialized per daemon. A pass:
|
||||
|
||||
1. **Walks the local tree.** Any `readdir`, metadata or permission failure marks the scan
|
||||
incomplete, and an incomplete scan cannot delete anything.
|
||||
1. **Walks the local tree.** A path the user may not read, an entry that vanishes mid-scan and a
|
||||
name that is not UTF-8 are left out, and the pass drops everything under them, as it drops
|
||||
ignored paths, so a left-out path is never read as deleted. An unreadable root and any other
|
||||
I/O error fail the pass.
|
||||
2. **Walks the remote tree.**
|
||||
3. **Loads the baseline**, the `sync_entry` rows: both sides as of the last successful pass.
|
||||
4. **Classifies each path** by comparing local, remote and baseline, with `(mtime, size)` as the
|
||||
|
||||
@@ -100,6 +100,9 @@ a live account since.
|
||||
|
||||
| Entry | Title |
|
||||
|---|---|
|
||||
| [B198](#b198--switching-to-online-only-deletes-local-files-that-never-reached-drive) | Switching to online-only deletes local files that never reached Drive |
|
||||
| [B197](#b197--switching-to-online-only-trashes-synced-files-that-became-ignored) | Switching to online-only trashes synced files that became ignored |
|
||||
| [B196](#b196--one-unreadable-folder-stops-the-whole-synced-folder-from-syncing) | One unreadable folder stops the whole synced folder from syncing |
|
||||
| [B195](#b195--the-app-crashes-when-proton-asks-for-a-captcha-on-secureblue) | The app crashes when Proton asks for a CAPTCHA on secureblue |
|
||||
| [B193](#b193--deleting-a-file-whose-create-failed-can-trash-a-file-of-the-users-on-drive) | Deleting a file whose create failed can trash a file of the user's on Drive |
|
||||
| [B191](#b191--a-file-deleted-after-its-create-failed-and-was-written-again-can-stay-on-drive) | A file deleted after its create failed and was written again can stay on Drive |
|
||||
@@ -298,6 +301,95 @@ a live account since.
|
||||
|
||||
Newest first.
|
||||
|
||||
### B198 — Switching to online-only deletes local files that never reached Drive
|
||||
|
||||
**Status:** Fixed (unverified).
|
||||
**Found:** 2026-10-08, reading the switch while fixing B196.
|
||||
**Where:** `crates/pdfs-fuse/src/devices.rs`, `Core::apply_sync_folder_mode`;
|
||||
`crates/pdfs-fuse/src/sync.rs`, `mirror_tree_unsynced`
|
||||
|
||||
**Repro:** Mirror a project folder with a `.git/` in it, then switch it to **Online only**.
|
||||
`.git/` is gone from the disk and was never on Drive.
|
||||
|
||||
**Cause.** The switch evicts the whole local tree once the folder is `idle`. `idle` only says the
|
||||
last pass applied everything it tried, and a pass never tries an ignored path (`.git/` is ignored
|
||||
by default), a symlink, or a folder it cannot read (B196). It also defers a file that is open for
|
||||
writing without leaving `idle`, and a file can change after the pass ends. The eviction deleted
|
||||
all of those.
|
||||
|
||||
**Fix.** Before the mode is written and anything is evicted, the switch walks the folder against
|
||||
the baseline, as a move out of a mirror folder already does, and refuses when something is only on
|
||||
this disk. The message names the path; an ignored one is called excluded from sync. A tracked file
|
||||
deleted here but not yet on Drive also refuses. An ignored one does not, since its baseline row
|
||||
stays for good and would hold the switch forever.
|
||||
|
||||
**Test:** unit tests `a_fully_synced_mirror_folder_may_go_online_only`,
|
||||
`an_ignored_folder_only_this_disk_has_holds_the_switch_back`,
|
||||
`a_deleted_ignored_path_does_not_hold_the_switch_back`,
|
||||
`an_edit_after_the_last_pass_holds_the_switch_back`, `a_new_file_at_the_root_holds_the_switch_back`,
|
||||
`an_unreadable_folder_holds_the_switch_back` and `a_symlink_holds_the_switch_back`. The switch
|
||||
itself has not been driven against a live account.
|
||||
|
||||
---
|
||||
|
||||
### B197 — Switching to online-only trashes synced files that became ignored
|
||||
|
||||
**Status:** Fixed (unverified).
|
||||
**Found:** 2026-10-08, reading the push pass while fixing B196.
|
||||
**Where:** `crates/pdfs-fuse/src/sync.rs`, `Core::push_pass`; `crates/pdfs-fuse/src/sync/planner.rs`,
|
||||
`local_deletions`
|
||||
|
||||
**Repro:** Mirror a folder until `build/` is on Drive, add `build/` to `.pdfsignore`, then switch
|
||||
the folder to **Online only**. `build/` goes to the Drive trash, and the switch then deletes the
|
||||
local copy.
|
||||
|
||||
**Cause.** The push pass that runs before the switch read its deletions as "in the baseline, not in
|
||||
the local walk". The walk skips ignored paths, so every synced path that later became ignored read
|
||||
as deleted. A full pass does not do this: it drops ignored paths from classification, which is
|
||||
what the user guide promises ("Ignoring never deletes anything").
|
||||
|
||||
**Fix.** The push pass reads its deletions from the same filtered baseline the wipe guard checks:
|
||||
without ignored paths, and without paths the scan could not read (B196).
|
||||
|
||||
**Test:** unit tests `a_push_pass_never_reads_an_ignored_path_as_deleted` and
|
||||
`local_deletions_come_out_shallowest_first`.
|
||||
|
||||
---
|
||||
|
||||
### B196 — One unreadable folder stops the whole synced folder from syncing
|
||||
|
||||
**Status:** Fixed (unverified). The reporter has not run the fix yet.
|
||||
**Found:** 2026-10-08, [#33](https://github.com/narrrl/proton-drive-linux/issues/33). A synced
|
||||
`~/Documents` held a restic cache a Docker container had created, which the user may not read.
|
||||
Every pass failed with `Couldn't sync Documents: read …/restic/cache/…/snapshots: Permission
|
||||
denied`, and nothing else in `Documents` synced.
|
||||
**Where:** `crates/pdfs-fuse/src/sync.rs`, `walk_local_tree`; `crates/pdfs-fuse/src/sync/planner.rs`,
|
||||
`Unscanned`
|
||||
|
||||
**Repro:** In a mirror folder, `sudo mkdir locked && sudo chmod 700 locked`, then sync.
|
||||
|
||||
**Cause.** B55 made every scan failure fatal to the pass, because a path left out of the scan read
|
||||
as a local deletion and was trashed on Drive. That was safe, but a folder the user may not read is
|
||||
not transient: the pass failed on every poll, for good. The same held for a file deleted between
|
||||
the listing and its `stat`, which a busy folder hits now and then, and for a name that is not UTF-8.
|
||||
|
||||
**Fix.** The scan leaves such a path out and records it, and the pass drops everything under it
|
||||
from classification, the wipe guard and the push pass's deletions, as it already did for ignored
|
||||
paths. A left-out path is then neither deleted on Drive nor downloaded into. A folder that lists
|
||||
but does not allow `stat` is left out whole. An unreadable or absent root, and every other I/O
|
||||
error, still fails the pass. The activity feed names the unreadable paths once each time the set
|
||||
changes, and the folder stays `idle`. Adding the folder to `.pdfsignore` was already a workaround
|
||||
and still silences the report.
|
||||
|
||||
**Test:** unit tests `an_unreadable_subdirectory_is_left_out_and_the_rest_scans`,
|
||||
`a_listable_but_unsearchable_folder_is_left_out_whole`,
|
||||
`a_path_the_scan_left_out_is_never_a_local_deletion`,
|
||||
`unscanned_covers_a_path_and_what_is_under_it_but_not_its_siblings` and
|
||||
`a_non_utf8_name_is_left_out_and_reported`. `an_unreadable_root_fails_the_scan` and
|
||||
`an_absent_root_fails_the_scan` still hold.
|
||||
|
||||
---
|
||||
|
||||
### B195 — The app crashes when Proton asks for a CAPTCHA on secureblue
|
||||
|
||||
**Status:** Fixed (unverified). Nobody has run the fix on secureblue yet.
|
||||
@@ -4822,6 +4914,10 @@ subdirectory holding a file, asserting the error names the subtree),
|
||||
strictness must not cost a working pass, so a healthy tree still reports every entry and still
|
||||
skips symlinks by design.
|
||||
|
||||
B196 later relaxed this: a path the user may not read, an entry that vanishes mid-scan and a name
|
||||
that is not UTF-8 are left out of the pass instead of failing it, and the pass drops everything
|
||||
under them from classification, so the B55 guarantee holds without the wedge.
|
||||
|
||||
Not covered: transient I/O errors, and an entry that disappears between `read_dir` and `stat`.
|
||||
Both are races that would need a filesystem shim to force; the code path they take is the same
|
||||
`symlink_metadata` failure the permission test drives.
|
||||
|
||||
@@ -11,6 +11,20 @@ scratch (user data in `staging/` and `recovery/` is never touched by this).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
- **One unreadable folder stopped a synced folder from syncing at all.** A folder you may not
|
||||
read, such as a cache a container created, failed every pass, so nothing else in the synced
|
||||
folder reached Drive. Such a folder is now left out and named once in the activity feed, and
|
||||
everything around it syncs. A file deleted during a scan, and a name that is not UTF-8, no
|
||||
longer fail the pass either.
|
||||
([#33](https://github.com/narrrl/proton-drive-linux/issues/33))
|
||||
- **Switching a folder to online-only could delete files that were never on Drive.** Ignored
|
||||
folders such as `.git/`, symlinks, unreadable folders and files changed after the last sync were
|
||||
deleted with the local copy. The switch now refuses and names what is only on this computer.
|
||||
- **Switching to online-only trashed synced files that had become ignored.** Adding a synced path
|
||||
to `.pdfsignore` and then switching moved it to the Drive trash. Ignoring never deletes now, as
|
||||
the user guide says.
|
||||
|
||||
## [3.1.2] — 2026-10-08
|
||||
|
||||
Schema: **42**, unchanged.
|
||||
|
||||
+6
-1
@@ -96,7 +96,9 @@ app and in the web app.
|
||||
Each synced folder can also be switched to **Online only**. The local copy is replaced by a mount
|
||||
of the folder, so the files stay visible but take no disk space until opened. Switch with the
|
||||
**Synced / Online only** switch on the folder's card, or `pdfs sync mode <id> ondemand` and back
|
||||
with `pdfs sync mode <id> mirror`. `pdfs locations` lists every local place the service occupies.
|
||||
with `pdfs sync mode <id> mirror`. The switch waits until everything in the folder is on Proton
|
||||
Drive, and refuses while the folder holds something that never goes there: an ignored path such
|
||||
as `.git/`, a symlink, or a folder you may not read. The message names it. `pdfs locations` lists every local place the service occupies.
|
||||
|
||||
### Moving between synced folders and My files
|
||||
|
||||
@@ -146,6 +148,9 @@ places, and both apply:
|
||||
Rules are re-read on every pass. Ignoring never deletes anything: a file that becomes ignored is
|
||||
simply no longer tracked, and its copy on Proton Drive stays.
|
||||
|
||||
A folder you may not read, such as a cache a container created as root, is skipped the same way,
|
||||
and the activity feed names it once. Add it to `.pdfsignore` to stop that message.
|
||||
|
||||
## Changes waiting for Proton Drive
|
||||
|
||||
A change you made in the Proton Drive folder exists only on this computer until it reaches
|
||||
|
||||
Reference in new issue
Block a user