fix(fuse): list a file whose create lands while its folder is listed (B166)

A listing from Drive was overlaid with the queue as it was after Drive
answered. A create that landed in between was in neither, and a wide
folder on Wi-Fi dropped files that were never unlinked. The queue is now
read before Drive is asked too; a node that left it meanwhile is listed
as the DB has it.
This commit is contained in:
nvrl committed 2026-10-04 20:10:14 +02:00
1 parent 573a29e596
commit 25c46cfe0a
5 files changed
+133 -3

No files matched your search

+45 -1
View File
@@ -2340,6 +2340,39 @@ impl Core {
Ok(()) Ok(())
} }
/// The nodes of `before`, queued in `folder_uid` when its listing was asked
/// of Drive, that are no longer queued in `after`, as the DB has them now.
///
/// Their create landed while the listing was on the wire, after Drive took
/// it, and so is in neither. One unlinked or moved away meanwhile is not
/// under the folder by that name any more and stays out.
fn landed_meanwhile(
&self,
folder_uid: &NodeUid,
before: Vec<Node>,
after: &[Node],
) -> Vec<Node> {
let still: HashSet<&str> = after.iter().map(|node| node.name.as_str()).collect();
let landed: HashSet<String> = before
.into_iter()
.map(|node| node.name)
.filter(|name| !still.contains(name.as_str()))
.collect();
if landed.is_empty() {
return Vec::new();
}
match self.db.known_children(folder_uid) {
Ok(known) => known
.into_iter()
.filter(|node| landed.contains(&node.name))
.collect(),
Err(e) => {
warn!(%folder_uid, error = %e, "db known_children failed");
Vec::new()
}
}
}
/// Serve a listing without the network: the one already in memory if any, /// Serve a listing without the network: the one already in memory if any,
/// else whatever children the DB still knows. The fallback only answers /// else whatever children the DB still knows. The fallback only answers
/// until the link is back, when [`Core::relist_offline_listings`] drops it: /// until the link is back, when [`Core::relist_offline_listings`] drops it:
@@ -2501,6 +2534,12 @@ impl Core {
let refresh_generation = let refresh_generation =
refresh_foreign.then(|| self.shared_generation.load(Ordering::SeqCst)); refresh_foreign.then(|| self.shared_generation.load(Ordering::SeqCst));
// What is queued before Drive is asked: a create that lands while the
// listing is on the wire is in neither Drive's answer nor the queue read
// after it (`docs/BUGS.md` B166).
let queued_before = (!foreign)
.then(|| self.db.queued_children(&folder_uid).ok())
.flatten();
let uids = match self let uids = match self
.rt .rt
.block_on(self.drive.enumerate_folder_children_node_uids(&folder_uid)) .block_on(self.drive.enumerate_folder_children_node_uids(&folder_uid))
@@ -2537,7 +2576,12 @@ impl Core {
// A shared folder's listing is published as Drive has it. // A shared folder's listing is published as Drive has it.
if !foreign { if !foreign {
match self.db.queued_children(&folder_uid) { match self.db.queued_children(&folder_uid) {
Ok((queued, gone)) => overlay_queued(&mut nodes, queued, &gone), Ok((mut queued, gone)) => {
if let Some((before, _)) = queued_before {
queued.extend(self.landed_meanwhile(&folder_uid, before, &queued));
}
overlay_queued(&mut nodes, queued, &gone)
}
Err(e) => warn!(%folder_uid, error = %e, "db queued_children failed"), Err(e) => warn!(%folder_uid, error = %e, "db queued_children failed"),
} }
} }
+54
View File
@@ -1084,4 +1084,58 @@ mod tests {
eprintln!("lan: syscalls {lan_syscalls:?}, drain {lan_drain:?}"); eprintln!("lan: syscalls {lan_syscalls:?}, drain {lan_drain:?}");
eprintln!("wifi: syscalls {wifi_syscalls:?}, drain {wifi_drain:?}"); eprintln!("wifi: syscalls {wifi_syscalls:?}, drain {wifi_drain:?}");
} }
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn half_of_a_wide_folder_unlinked_at_once_leaves_the_other_half() {
// The acceptance suite's wide directory on Wi-Fi. A create that landed
// while the folder's listing was on the wire was in neither Drive's
// answer nor the queue read after it, and a file that was never
// unlinked dropped out of the listing until the next one (B166).
let drive = FakeDrive::new();
let dir = scratch("wide-unlink");
let daemon = Daemon::start(&dir, drive.client(1, Faults::wifi())).unwrap();
let root = daemon.mountpoint.join("wide");
std::fs::create_dir(&root).unwrap();
let names: Vec<String> = (0..128).map(|i| format!("entry-{i:04}.txt")).collect();
let in_parallel = |op: &(dyn Fn(&str) + Sync), names: &[String]| {
std::thread::scope(|scope| {
for chunk in names.chunks(names.len().div_ceil(8)) {
scope.spawn(move || chunk.iter().for_each(|name| op(name)));
}
});
};
in_parallel(
&|name| {
let mut file = std::fs::File::create(root.join(name)).unwrap();
std::io::Write::write_all(&mut file, name.as_bytes()).unwrap();
file.sync_all().unwrap();
},
&names,
);
let listed = |root: &std::path::Path| {
let mut names: Vec<String> = std::fs::read_dir(root)
.unwrap()
.map(|e| e.unwrap().file_name().into_string().unwrap())
.collect();
names.sort();
names
};
assert_eq!(listed(&root), names);
in_parallel(
&|name| std::fs::remove_file(root.join(name)).unwrap(),
&names[..64],
);
assert_eq!(listed(&root), names[64..]);
let mut settled = None;
while settled.is_none_or(|at: Instant| at.elapsed() < crate::POLL_INTERVAL * 2) {
assert_eq!(listed(&root), names[64..]);
if settled.is_none() && daemon.pending().is_ok_and(|ops| ops.is_empty()) {
settled = Some(Instant::now());
}
std::thread::sleep(Duration::from_millis(50));
}
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
} }
+27
View File
@@ -12,6 +12,33 @@ Conventions:
--- ---
## B166 — A folder drops a file whose create lands while it is listed
**Status:** Fixed.
**Found:** 2026-10-04, by the account run on the working tree after B165 (a wide directory, on
My files and on on-demand pairing 1): "concurrent unlinks left the wrong entries". Every
create and all 64 trashes landed as they should, so the drain was right and the listing was
wrong. The run logs no names. The simulation lost files that were never unlinked.
**Where:** `list_children` in `crates/pdfs-fuse/src/lib.rs`.
**Cause.** A folder whose listing was dropped is listed from Drive, then overlaid with what
the queue still holds for it (B132). The queue was read after Drive answered. A create that
landed in between was in neither: Drive had taken the listing before the node existed, and
the queue had let go of it by the time it was read. The file was missing until the folder
was listed again. On Wi-Fi, with 16 workers landing creates while the event feed drops the
folder's listing, this happened in almost every run of the wide-directory case.
**Fix.** The queued children are read before Drive is asked as well as after. A node queued
before and gone from the queue after has landed meanwhile, and is listed as the DB has it
now, under its name. One unlinked or moved away meanwhile is no longer a child of the folder
in the DB, so it stays out.
**Test:** `half_of_a_wide_folder_unlinked_at_once_leaves_the_other_half` in `sim/daemon.rs`
(failed 4 of 4 runs before the fix, passed 8 of 8 after).
---
## B165 — A stop waits on a stuck thread until systemd kills the daemon ## B165 — A stop waits on a stuck thread until systemd kills the daemon
**Status:** Fixed (unverified). **Status:** Fixed (unverified).
+2
View File
@@ -59,6 +59,8 @@ by its local id, in the queue, the stored tree and pins alike.
upload landed showed it both as Drive had it and as the queue did (B164). upload landed showed it both as Drive had it and as the queue did (B164).
- **Stopping the daemon no longer hangs until systemd kills it.** A stop waits up to 10 s for - **Stopping the daemon no longer hangs until systemd kills it.** A stop waits up to 10 s for
work in progress, then exits and logs what was still running (B165). work in progress, then exits and logs what was still running (B165).
- **A folder no longer drops a new file while others are uploaded.** A file whose upload
landed while the folder was being listed was missing from that listing (B166).
- **A file renamed just as its upload finished keeps its new name.** The mount no longer showed - **A file renamed just as its upload finished keeps its new name.** The mount no longer showed
the old name, and a second rename no longer failed with "No such file or directory" (B152). the old name, and a second rename no longer failed with "No such file or directory" (B152).
- **A change that waits for a name to be freed no longer waits seconds after it is.** A rename or - **A change that waits for a name to be freed no longer waits seconds after it is.** A rename or
+5 -2
View File
@@ -9,7 +9,7 @@ Tracks `docs/MILESTONE-3.0.0.md`. Update it with every commit that moves a phase
- **Phase 2**: local ids, inode = local id, one tree for every mount (2.10.0). - **Phase 2**: local ids, inode = local id, one tree for every mount (2.10.0).
- **Phase 3, first part**: mkdir, create, rename, unlink and rmdir on the mount are recorded - **Phase 3, first part**: mkdir, create, rename, unlink and rmdir on the mount are recorded
locally and sent from the queue; `"local_first": false` brings back the old path (5e0f30c). locally and sent from the queue; `"local_first": false` brings back the old path (5e0f30c).
- Fixes from the simulation and account runs since then: B127 and B141 to B165. - Fixes from the simulation and account runs since then: B127 and B141 to B166.
- **Sync issues, first cut** (§6.2): refusals sorted into issues, shown at once in the app, - **Sync issues, first cut** (§6.2): refusals sorted into issues, shown at once in the app,
tray and CLI; Export button and `pdfs sync export`; `pdfs sync issues`. tray and CLI; Export button and `pdfs sync export`; `pdfs sync issues`.
@@ -101,7 +101,10 @@ Tracks `docs/MILESTONE-3.0.0.md`. Update it with every commit that moves a phase
(2026-10-04: 260 passed, 3 failed) failed a new file that read back empty (B163, (2026-10-04: 260 passed, 3 failed) failed a new file that read back empty (B163,
fixed), a 128-entry folder that listed 129 (B164, fixed), and its cleanup, which ran fixed), a 128-entry folder that listed 129 (B164, fixed), and its cleanup, which ran
while the daemon was restarting: the stop waited on a busy thread until systemd killed while the daemon was restarting: the stop waited on a busy thread until systemd killed
it (B165, bounded). it (B165, bounded). The next run on the working tree (2026-10-04: 259 passed, 4
failed) found a wide folder dropping files whose create landed while it was listed
(B166, fixed). Still open from it: a new file that read back empty again, and a
cleanup that found no control socket.
5. **Release**: 2.10.0 is tagged; the 2.9.0 tag still waits for the user's go-ahead. 5. **Release**: 2.10.0 is tagged; the 2.9.0 tag still waits for the user's go-ahead.
## After 3.0.0 ## After 3.0.0