chore: cite docs/BUGS.md and drop references to notes outside the repo

66 comments cited `bugs.md B<n>`, the ledger's old name; they now say
`docs/BUGS.md B<n>`, and every cited number has an entry. About 90 more
cited offline.md, devices.md, features.md, mount-architecture.md,
plan.md and improvements.md, planning notes that were never in the
repository. Those citations are removed; the comments keep their text.
This commit is contained in:
nvrl committed 2026-10-06 19:29:02 +02:00
1 parent 10c69b0d64
commit aa384ae2eb
31 files changed
+154 -164

No files matched your search

+13 -15
View File
@@ -91,7 +91,7 @@ const TOUCH_FLUSH_AT: usize = 256;
/// ///
/// Public because it is the *contract* a block has to satisfy, not a detail of /// Public because it is the *contract* a block has to satisfy, not a detail of
/// this cache: the network read path validates against it too, so a block that /// this cache: the network read path validates against it too, so a block that
/// comes back short is refused before it can truncate a file (bugs.md B84). /// comes back short is refused before it can truncate a file (`docs/BUGS.md` B84).
pub fn block_len(size: u64, idx: u64) -> u64 { pub fn block_len(size: u64, idx: u64) -> u64 {
let start = idx.saturating_mul(BLOCK_SIZE); let start = idx.saturating_mul(BLOCK_SIZE);
size.saturating_sub(start).min(BLOCK_SIZE) size.saturating_sub(start).min(BLOCK_SIZE)
@@ -311,8 +311,7 @@ struct PinFile {
pins: BTreeMap<String, Pin>, pins: BTreeMap<String, Pin>,
} }
/// What a staged file actually contains, written beside it as `<name>.json` /// What a staged file actually contains, written beside it as `<name>.json`.
/// (offline.md Phase 2).
/// ///
/// A staged file is **not** necessarily valid whole-file content, and that is /// A staged file is **not** necessarily valid whole-file content, and that is
/// the whole reason this exists. A partial overwrite commits by filling the /// the whole reason this exists. A partial overwrite commits by filling the
@@ -439,9 +438,9 @@ pub struct ContentCache {
/// Subdirectory for write-handle scratch files (disk-backed write buffers). /// Subdirectory for write-handle scratch files (disk-backed write buffers).
/// Emptied on open so a crashed run leaves no orphans. /// Emptied on open so a crashed run leaves no orphans.
scratch_dir: PathBuf, scratch_dir: PathBuf,
/// Subdirectory holding the bytes of writes that have not been uploaded yet /// Subdirectory holding the bytes of writes that have not been uploaded yet.
/// (offline.md Phase 2/3) — every released write passes through here, since /// Every released write passes through here, since the upload is a queued
/// the upload is a queued op performed later. Unlike `scratch_dir` this is /// op performed later. Unlike `scratch_dir` this is
/// **never** emptied on open: these are the only copy of content the user /// **never** emptied on open: these are the only copy of content the user
/// authored, and the whole point is that they outlive the daemon. /// authored, and the whole point is that they outlive the daemon.
staging_dir: PathBuf, staging_dir: PathBuf,
@@ -476,7 +475,7 @@ pub struct ContentCache {
thumb_bytes: AtomicU64, thumb_bytes: AtomicU64,
/// Unified metadata DB. Its `cache_entries` table is the LRU index: every /// Unified metadata DB. Its `cache_entries` table is the LRU index: every
/// store/read/evict updates it, and the budget enforcers query it instead of /// store/read/evict updates it, and the budget enforcers query it instead of
/// scanning the cache directories (plan.md P4). /// scanning the cache directories.
db: Arc<Db>, db: Arc<Db>,
/// LRU touches not yet written to `cache_entries`, keyed by cache key. /// LRU touches not yet written to `cache_entries`, keyed by cache key.
/// ///
@@ -989,7 +988,7 @@ impl ContentCache {
/// a miss ([`cached_block`](Self::cached_block) checks), so a bad block /// a miss ([`cached_block`](Self::cached_block) checks), so a bad block
/// cannot escape this cache even though this does not check. Validating a /// cannot escape this cache even though this does not check. Validating a
/// block is the *reader's* job, at the point it is fetched — see /// block is the *reader's* job, at the point it is fetched — see
/// `Core::read_block` (bugs.md B84). /// `Core::read_block` (`docs/BUGS.md` B84).
pub fn store_block( pub fn store_block(
&self, &self,
uid: &NodeUid, uid: &NodeUid,
@@ -1491,9 +1490,8 @@ impl ContentCache {
/// This is what makes a write survive its upload: the caller is releasing a /// This is what makes a write survive its upload: the caller is releasing a
/// write handle and would otherwise delete the file, so until the bytes are /// write handle and would otherwise delete the file, so until the bytes are
/// on the remote, staging holds the only copy. Every dirty handle goes /// on the remote, staging holds the only copy. Every dirty handle goes
/// through here — the upload is a queued op performed later (offline.md /// through here — the upload is a queued op performed later, and a staged
/// Phase 3), and a staged file is also what a human can recover from if the /// file is also what a human can recover from if the queue never drains.
/// queue never drains.
/// ///
/// Falls back to a copy when the rename crosses a filesystem boundary; a /// Falls back to a copy when the rename crosses a filesystem boundary; a
/// failure here means we could not save the bytes at all, so it is reported /// failure here means we could not save the bytes at all, so it is reported
@@ -2707,7 +2705,7 @@ mod tests {
} }
/// The geometry the whole read path validates against, here and over the /// The geometry the whole read path validates against, here and over the
/// network (bugs.md B84): full blocks, then whatever is left. /// network (`docs/BUGS.md` B84): full blocks, then whatever is left.
#[test] #[test]
fn block_len_is_full_blocks_then_the_remainder() { fn block_len_is_full_blocks_then_the_remainder() {
let size = BLOCK_SIZE * 2 + 17; let size = BLOCK_SIZE * 2 + 17;
@@ -2787,7 +2785,7 @@ mod tests {
/// A cached block is addressed by the range it covers, not by its index /// A cached block is addressed by the range it covers, not by its index
/// alone. Re-planning a file onto its real geometry must therefore miss the /// alone. Re-planning a file onto its real geometry must therefore miss the
/// blocks the uniform fallback cached, rather than serve their bytes at the /// blocks the uniform fallback cached, rather than serve their bytes at the
/// wrong offset (bugs.md B85). /// wrong offset (`docs/BUGS.md` B85).
#[test] #[test]
fn a_cached_block_is_not_served_to_a_different_geometry() { fn a_cached_block_is_not_served_to_a_different_geometry() {
let (c, _d) = cache(); let (c, _d) = cache();
@@ -2854,7 +2852,7 @@ mod tests {
/// A block of the wrong length cannot escape the cache even though the /// A block of the wrong length cannot escape the cache even though the
/// store does not check: the read validates it and reports a miss, so the /// store does not check: the read validates it and reports a miss, so the
/// next read refetches rather than serving a hole (bugs.md B84). /// next read refetches rather than serving a hole (`docs/BUGS.md` B84).
#[test] #[test]
fn a_block_of_the_wrong_length_reads_as_a_miss() { fn a_block_of_the_wrong_length_reads_as_a_miss() {
let (c, _d) = cache(); let (c, _d) = cache();
@@ -2919,7 +2917,7 @@ mod tests {
); );
} }
/// A failed upload must never cost the user their bytes (offline.md Phase 2): /// A failed upload must never cost the user their bytes:
/// the scratch file moves to staging intact, and staging survives a reopen /// the scratch file moves to staging intact, and staging survives a reopen
/// the way scratch deliberately does not. /// the way scratch deliberately does not.
#[test] #[test]
+3 -3
View File
@@ -432,7 +432,7 @@ impl AppDirs {
/// typical umask makes them `0755`. Until this was fixed, the only thing /// typical umask makes them `0755`. Until this was fixed, the only thing
/// keeping another local user out was `~/.cache` and `~/.local/state` /// keeping another local user out was `~/.cache` and `~/.local/state`
/// happening to be `0700`, which is a convention of the user's system and /// happening to be `0700`, which is a convention of the user's system and
/// not something this client established (bugs.md B6). /// not something this client established (`docs/BUGS.md` B6).
/// ///
/// Applied on every start, not just at creation: a directory that already /// Applied on every start, not just at creation: a directory that already
/// exists with a permissive mode — restored from a backup that flattened /// exists with a permissive mode — restored from a backup that flattened
@@ -483,7 +483,7 @@ fn restrict_dir(dir: &Path) -> Result<()> {
/// session — enumerate, read, upload, trash, share — without presenting any /// session — enumerate, read, upload, trash, share — without presenting any
/// credential, since the daemon already holds one. It is an authority boundary, /// credential, since the daemon already holds one. It is an authority boundary,
/// not merely private data, and `UnixListener::bind` applies the umask like any /// not merely private data, and `UnixListener::bind` applies the umask like any
/// other file (bugs.md B6). /// other file (`docs/BUGS.md` B6).
/// ///
/// There is a window between `bind` and this call during which the socket /// There is a window between `bind` and this call during which the socket
/// carries the umask's mode. It is closed in practice by the containing /// carries the umask's mode. It is closed in practice by the containing
@@ -504,7 +504,7 @@ mod tests {
std::fs::metadata(p).unwrap().permissions().mode() & 0o777 std::fs::metadata(p).unwrap().permissions().mode() & 0o777
} }
/// bugs.md B6. A permissive mode here is not a cosmetic problem: these /// `docs/BUGS.md` B6. A permissive mode here is not a cosmetic problem: these
/// directories hold decrypted content, a plaintext index of every node name, /// directories hold decrypted content, a plaintext index of every node name,
/// and the control socket that commands the daemon's session. /// and the control socket that commands the daemon's session.
#[test] #[test]
+7 -7
View File
@@ -443,11 +443,11 @@ pub enum Request {
DeleteDevice { uid: String }, DeleteDevice { uid: String },
/// Adopt an existing device as this machine's, pinning its uid in /// Adopt an existing device as this machine's, pinning its uid in
/// `config.json` so a hostname change or reinstall re-attaches to it instead /// `config.json` so a hostname change or reinstall re-attaches to it instead
/// of registering a duplicate (features.md 5.1). `uid: None` clears the pin. /// of registering a duplicate. `uid: None` clears the pin.
/// Replies with [`Response::Ok`]. /// Replies with [`Response::Ok`].
AdoptDevice { uid: Option<String> }, AdoptDevice { uid: Option<String> },
// ---- device folder sync (devices.md) ---------------------------------- // ---- device folder sync ----------------------------------
/// Add a local folder to this machine's device, uploading its tree and /// Add a local folder to this machine's device, uploading its tree and
/// registering the device on first use. Replies with [`Response::Ok`]. /// registering the device on first use. Replies with [`Response::Ok`].
AddSyncFolder { local_path: String }, AddSyncFolder { local_path: String },
@@ -496,7 +496,7 @@ pub enum Request {
/// [`Response::Ok`]. /// [`Response::Ok`].
ResolveConflict { path: String, keep: ConflictKeep }, ResolveConflict { path: String, keep: ConflictKeep },
/// List the folders under this machine's device that can be synced here, /// List the folders under this machine's device that can be synced here,
/// each with a proposed local path (features.md 5.2). Replies with /// each with a proposed local path. Replies with
/// [`Response::RestorableFolders`]. /// [`Response::RestorableFolders`].
ListRestorableFolders, ListRestorableFolders,
/// Attach the given remote device folders to local paths and sync them down. /// Attach the given remote device folders to local paths and sync them down.
@@ -1903,17 +1903,17 @@ pub enum Response {
/// The pin registry. /// The pin registry.
pins: Vec<Pin>, pins: Vec<Pin>,
/// False when the daemon is serving the cached tree because the API is /// False when the daemon is serving the cached tree because the API is
/// unreachable (offline.md Phase 1). Cached and pinned content still /// unreachable. Cached and pinned content still
/// reads; anything else fails until the network is back. /// reads; anything else fails until the network is back.
#[serde(default = "default_online")] #[serde(default = "default_online")]
online: bool, online: bool,
/// Writes accepted locally but not yet uploaded (offline.md Phase 3). /// Writes accepted locally but not yet uploaded.
/// Non-zero means the mount is ahead of the remote — either a copy is /// Non-zero means the mount is ahead of the remote — either a copy is
/// still draining, or it cannot drain because we are offline. /// still draining, or it cannot drain because we are offline.
#[serde(default)] #[serde(default)]
pending_uploads: u64, pending_uploads: u64,
/// Queued mutations that carry no bytes: `mkdir`, `rename`, `trash` /// Queued mutations that carry no bytes: `mkdir`, `rename`, `trash`.
/// (offline.md Phase 3b). Counted apart from `pending_uploads` because /// Counted apart from `pending_uploads` because
/// calling a queued `mkdir` an upload is a lie. /// calling a queued `mkdir` an upload is a lie.
#[serde(default)] #[serde(default)]
pending_changes: u64, pending_changes: u64,
+1 -1
View File
@@ -1,4 +1,4 @@
//! Database health reporting and repair (features.md 4.3). //! Database health reporting and repair.
//! //!
//! Everything here exists for the case where something has already gone wrong: //! Everything here exists for the case where something has already gone wrong:
//! a crash mid-write, a full disk, a cache that has grown out of proportion to //! a crash mid-write, a full disk, a cache that has grown out of proportion to
+5 -5
View File
@@ -577,7 +577,7 @@ CREATE TABLE trash (
); );
"; ";
/// Schema v8: device sync (devices.md). This machine registers as one Proton Drive /// Schema v8: device sync. This machine registers as one Proton Drive
/// **Device** (a share + root folder on the main volume); `device` is a singleton /// **Device** (a share + root folder on the main volume); `device` is a singleton
/// row cached so we reuse the same device across restarts instead of creating a /// row cached so we reuse the same device across restarts instead of creating a
/// new one each run. `sync_folder` is one row per local folder the user added, /// new one each run. `sync_folder` is one row per local folder the user added,
@@ -638,8 +638,8 @@ ALTER TABLE sync_folder ADD COLUMN pending_mode TEXT;
"; ";
/// Schema v11: writes no longer upload inside the FUSE handler. `release` stages /// Schema v11: writes no longer upload inside the FUSE handler. `release` stages
/// the bytes and records the intended upload here, and a drain worker performs it /// the bytes and records the intended upload here, and a drain worker performs it.
/// (offline.md Phase 3). The row outlives the process, so a write survives both a /// The row outlives the process, so a write survives both a
/// dead network and a restart. /// dead network and a restart.
/// ///
/// `blob_path` points into the content cache's `staging/` dir and `meta_json` is /// `blob_path` points into the content cache's `staging/` dir and `meta_json` is
@@ -666,7 +666,7 @@ CREATE INDEX pending_op_uid ON pending_op(uid);
/// Schema v12: `pending_op` also carries mutations that *create* a node, which a /// Schema v12: `pending_op` also carries mutations that *create* a node, which a
/// revision op never had to describe — it always addressed a node the server had /// revision op never had to describe — it always addressed a node the server had
/// already minted a uid for (offline.md Phase 3b). /// already minted a uid for.
/// ///
/// An offline `create`/`mkdir` cannot get a real uid, so the node is invented /// An offline `create`/`mkdir` cannot get a real uid, so the node is invented
/// locally under a `local~<uuid>` placeholder and the op records where it goes /// locally under a `local~<uuid>` placeholder and the op records where it goes
@@ -1056,7 +1056,7 @@ ALTER TABLE pending_op ADD COLUMN access_deferred_since INTEGER NOT NULL DEFAULT
/// as the last sync and left the file the same length — a fixed-size record /// as the last sync and left the file the same length — a fixed-size record
/// rewritten, an image re-exported, a database page updated in place — read as /// rewritten, an image re-exported, a database page updated in place — read as
/// *unchanged* and was never uploaded. The remote copy silently stayed at the /// *unchanged* and was never uploaded. The remote copy silently stayed at the
/// older content until something else about the file moved (bugs.md B25). /// older content until something else about the file moved (`docs/BUGS.md` B25).
/// ///
/// Nullable rather than a converted `local_mtime * 1_000_000_000`: the /// Nullable rather than a converted `local_mtime * 1_000_000_000`: the
/// nanosecond part of an existing baseline is genuinely unknown, and inventing /// nanosecond part of an existing baseline is genuinely unknown, and inventing
+1 -2
View File
@@ -568,8 +568,7 @@ impl Db {
} }
/// Load one persisted node back by uid. Used to recover the My Files root /// Load one persisted node back by uid. Used to recover the My Files root
/// when the API is unreachable, so the mount can still serve the cached tree /// when the API is unreachable, so the mount can still serve the cached tree.
/// (offline.md Phase 1).
pub fn node_by_uid(&self, uid: &str) -> Result<Option<Node>> { pub fn node_by_uid(&self, uid: &str) -> Result<Option<Node>> {
let conn = self.read(); let conn = self.read();
let json: Option<String> = conn let json: Option<String> = conn
+5 -6
View File
@@ -15,7 +15,7 @@ use crate::Result;
pub const OP_REVISION: &str = "revision"; pub const OP_REVISION: &str = "revision";
/// The `kind` of a [`PendingOp`] that creates a file that so far exists only /// The `kind` of a [`PendingOp`] that creates a file that so far exists only
/// locally, under a `local:` placeholder uid (offline.md Phase 3b). /// locally, under a `local:` placeholder uid.
/// ///
/// The written bytes ride along on the same row rather than as a follow-on /// The written bytes ride along on the same row rather than as a follow-on
/// [`OP_REVISION`]: draining the create mints the node's real uid, which would /// [`OP_REVISION`]: draining the create mints the node's real uid, which would
@@ -29,7 +29,7 @@ pub const OP_CREATE: &str = "create";
pub const OP_MKDIR: &str = "mkdir"; pub const OP_MKDIR: &str = "mkdir";
/// The `kind` of a [`PendingOp`] that gives a node a new name, a new parent, or /// The `kind` of a [`PendingOp`] that gives a node a new name, a new parent, or
/// both — the queued form of `mv` (offline.md Phase 3b). /// both — the queued form of `mv`.
/// ///
/// `parent_uid` and `name` hold the node's *desired end state*, not a delta, so /// `parent_uid` and `name` hold the node's *desired end state*, not a delta, so
/// a second rename simply replaces the row (see [`Db::enqueue_op`]) and the /// a second rename simply replaces the row (see [`Db::enqueue_op`]) and the
@@ -55,8 +55,7 @@ pub struct RenameMeta {
pub original_name: Option<String>, pub original_name: Option<String>,
} }
/// The `kind` of a [`PendingOp`] that trashes a node the server knows about /// The `kind` of a [`PendingOp`] that trashes a node the server knows about.
/// (offline.md Phase 3b).
/// ///
/// A node that only ever existed locally never gets one of these: there is /// A node that only ever existed locally never gets one of these: there is
/// nothing to trash remotely, so deleting it just drops its queued ops /// nothing to trash remotely, so deleting it just drops its queued ops
@@ -311,7 +310,7 @@ pub const PARK_UNTIL: i64 = 8_000_000_000_000;
pub const PARK_EXPIRY_MS: i64 = 60 * 60 * 1000; pub const PARK_EXPIRY_MS: i64 = 60 * 60 * 1000;
/// A mutation that has been accepted locally but not yet performed against the /// A mutation that has been accepted locally but not yet performed against the
/// API — the durable half of the write-back queue (offline.md Phase 3). /// API — the durable half of the write-back queue.
/// ///
/// The daemon answers the FUSE call the moment this row and its staged blob are /// The daemon answers the FUSE call the moment this row and its staged blob are
/// on disk, so a `cp` into the mount runs at disk speed and the upload happens /// on disk, so a `cp` into the mount runs at disk speed and the upload happens
@@ -887,7 +886,7 @@ impl Db {
/// make — the node is still only a queued intent, and rewriting that intent /// make — the node is still only a queued intent, and rewriting that intent
/// *is* the rename. Returns false when the create has already drained, in /// *is* the rename. Returns false when the create has already drained, in
/// which case the node has a real uid and the caller must rename it there /// which case the node has a real uid and the caller must rename it there
/// instead (offline.md Phase 3b). /// instead.
/// ///
/// A create backing off is made due: what it failed on may have been the /// A create backing off is made due: what it failed on may have been the
/// name it no longer wants, such as one held by a queued change. A parked /// name it no longer wants, such as one held by a queued change. A parked
+4 -4
View File
@@ -25,8 +25,8 @@ pub struct StoredSyncFolder {
} }
/// One per-file sync baseline row: what a path looked like on both sides at the /// One per-file sync baseline row: what a path looked like on both sides at the
/// last successful sync, so the next reconcile can tell which side changed /// last successful sync, so the next reconcile can tell which side changed.
/// (devices.md Phase 2). `remote_rev`/`remote_hash` hold the remote signature — /// `remote_rev`/`remote_hash` hold the remote signature —
/// its modification time and size as strings — since no cheap content hash is /// its modification time and size as strings — since no cheap content hash is
/// exposed; change detection is `(mtime, size)` on each side. /// exposed; change detection is `(mtime, size)` on each side.
#[derive(Clone, Debug, PartialEq)] #[derive(Clone, Debug, PartialEq)]
@@ -39,7 +39,7 @@ pub struct StoredSyncEntry {
/// ///
/// `local_mtime` alone cannot tell an edit that landed inside the same /// `local_mtime` alone cannot tell an edit that landed inside the same
/// second as the last sync — and left the file the same length — from no /// second as the last sync — and left the file the same length — from no
/// edit at all, so those never uploaded (bugs.md B25). `None` on a row /// edit at all, so those never uploaded (`docs/BUGS.md` B25). `None` on a row
/// written before schema 28, which the comparison reads as "compare whole /// written before schema 28, which the comparison reads as "compare whole
/// seconds", so upgrading does not make an entire mirror look changed. /// seconds", so upgrading does not make an entire mirror look changed.
pub local_mtime_ns: Option<i64>, pub local_mtime_ns: Option<i64>,
@@ -345,7 +345,7 @@ impl Db {
/// Drop the entire baseline for a folder. Used when flipping ondemand→mirror: /// Drop the entire baseline for a folder. Used when flipping ondemand→mirror:
/// the local tree was evicted, so the old baseline is stale and would make the /// the local tree was evicted, so the old baseline is stale and would make the
/// next reconcile mistake "locally deleted" for "must re-download". Clearing it /// next reconcile mistake "locally deleted" for "must re-download". Clearing it
/// leaves an empty baseline + full remote = pure download (devices.md P3). /// leaves an empty baseline + full remote = pure download.
pub fn sync_entries_clear(&self, folder_id: i64) -> Result<()> { pub fn sync_entries_clear(&self, folder_id: i64) -> Result<()> {
let conn = self.conn.lock(); let conn = self.conn.lock();
conn.execute( conn.execute(
+5 -5
View File
@@ -675,8 +675,8 @@ fn foreign_folder(volume: &str, link: &str, parent: NodeUid, name: &str) -> Node
node node
} }
/// Recovering the root by uid is what lets the daemon mount offline /// Recovering the root by uid is what lets the daemon mount offline:
/// (offline.md Phase 1): the uid is remembered in `sync_state`, the node /// the uid is remembered in `sync_state`, the node
/// itself comes back out of `nodes`. /// itself comes back out of `nodes`.
#[test] #[test]
fn node_by_uid_recovers_a_stored_node() { fn node_by_uid_recovers_a_stored_node() {
@@ -796,7 +796,7 @@ fn children_if_listed_gated_on_flag() {
assert_eq!(kids.len(), 2); assert_eq!(kids.len(), 2);
} }
/// The shape of the `mv`-loses-the-file bug (bugs.md B1): a rename deletes the /// The shape of the `mv`-loses-the-file bug (`docs/BUGS.md` B1): a rename deletes the
/// moved node's row, so a destination left marked `listed` is rebuilt from the DB /// moved node's row, so a destination left marked `listed` is rebuilt from the DB
/// without it — the file is gone from the source and absent from the destination, /// without it — the file is gone from the source and absent from the destination,
/// with `rename(2)` having reported success. Clearing the flag is what forces the /// with `rename(2)` having reported success. Clearing the flag is what forces the
@@ -5735,7 +5735,7 @@ fn cache_eviction_candidates_are_lru_ordered_and_limited() {
); );
} }
/// **The B5 checkpoint.** `improvements.md` P2.3 proposes concurrent SQLite /// **The B5 checkpoint.** A performance plan proposed concurrent SQLite
/// reads, on the premise that the single `Mutex<Connection>` serializes the FUSE /// reads, on the premise that the single `Mutex<Connection>` serializes the FUSE
/// workers. This measures what the connection is actually asked to do now that /// workers. This measures what the connection is actually asked to do now that
/// B3 and B4 have landed, so the proposal is decided on evidence. /// B3 and B4 have landed, so the proposal is decided on evidence.
@@ -6291,7 +6291,7 @@ fn migration_v27_adds_the_access_deferral_column_to_a_v26_queue() {
/// back with no nanosecond time. That `None` is the whole point: it makes the /// back with no nanosecond time. That `None` is the whole point: it makes the
/// comparison fall back to whole seconds for rows written before the column /// comparison fall back to whole seconds for rows written before the column
/// existed, instead of asserting a sub-second time of zero and re-uploading /// existed, instead of asserting a sub-second time of zero and re-uploading
/// every already-synced file in the folder (bugs.md B25). /// every already-synced file in the folder (`docs/BUGS.md` B25).
#[test] #[test]
fn migration_v28_adds_sub_second_local_times_without_disturbing_a_v27_baseline() { fn migration_v28_adds_sub_second_local_times_without_disturbing_a_v27_baseline() {
let path = std::env::temp_dir().join(format!( let path = std::env::temp_dir().join(format!(
+1 -1
View File
@@ -102,5 +102,5 @@ impl Db {
Ok(seen) Ok(seen)
} }
// ---- device sync (devices.md) ----------------------------------------- // ---- device sync -----------------------------------------
} }
+1 -1
View File
@@ -1,4 +1,4 @@
//! The machine profile that travels with a device (features.md 5.3). //! The machine profile that travels with a device.
//! //!
//! Sync folder mappings, pins, and the settings that shape them live in //! Sync folder mappings, pins, and the settings that shape them live in
//! `cache.db` and `config.json` — both on the local disk. When that disk dies, //! `cache.db` and `config.json` — both on the local disk. When that disk dies,
+1 -1
View File
@@ -1,4 +1,4 @@
//! Selective-sync ignore rules for mirror folders (features.md 1.1). //! Selective-sync ignore rules for mirror folders.
//! //!
//! A mirror folder otherwise uploads every path beneath its root, which for a //! A mirror folder otherwise uploads every path beneath its root, which for a
//! project directory means `node_modules/`, `.git/`, and `target/` — quota //! project directory means `node_modules/`, `.git/`, and `target/` — quota
+3 -3
View File
@@ -143,7 +143,7 @@ fn apply_event(core: &Core, event: &DriveEvent, dirty: &mut DirtyParents) -> pdf
// create, and re-fetching would replace the size and mtime of the // create, and re-fetching would replace the size and mtime of the
// write we just accepted with the stale revision's — making a file // write we just accepted with the stale revision's — making a file
// that was copied in seconds ago read as empty until its upload // that was copied in seconds ago read as empty until its upload
// lands (offline.md Phase 3). // lands.
// //
// A write that has already drained is the same story one step later: // A write that has already drained is the same story one step later:
// the drain brought the tree level with the revision it sealed, so // the drain brought the tree level with the revision it sealed, so
@@ -375,7 +375,7 @@ pub(super) async fn run_event_sync(
// server head; persist it so the next restart resumes instead of // server head; persist it so the next restart resumes instead of
// reseeding (which would skip everything that changed offline). // reseeding (which would skip everything that changed offline).
// Seeding needs the network, and this task also runs on mounts that // Seeding needs the network, and this task also runs on mounts that
// started offline (offline.md Phase 1) — so retry rather than giving up, // started offline — so retry rather than giving up,
// which used to disable live sync for the life of the daemon. // which used to disable live sync for the life of the daemon.
Ok(None) => { Ok(None) => {
// Nothing says what changed before this cursor, so a persisted SDK // Nothing says what changed before this cursor, so a persisted SDK
@@ -660,7 +660,7 @@ pub(super) fn run_local_index(
scan_local_once(&db, &indexing, &transfers, &mountpoint); scan_local_once(&db, &indexing, &transfers, &mountpoint);
} }
// Interruptible, so teardown joins this thread instead of leaving it to // Interruptible, so teardown joins this thread instead of leaving it to
// keep rewriting the index for a mount that is gone (bugs.md B44). // keep rewriting the index for a mount that is gone (`docs/BUGS.md` B44).
if !shutdown.sleep(LOCAL_INDEX_CHECK) { if !shutdown.sleep(LOCAL_INDEX_CHECK) {
return; return;
} }
+1 -1
View File
@@ -1816,7 +1816,7 @@ pub(crate) fn run_control_socket(
info!("control socket listening"); info!("control socket listening");
for conn in listener.incoming() { for conn in listener.incoming() {
// Checked on every accept, including the deliberate poke teardown sends // Checked on every accept, including the deliberate poke teardown sends
// to wake this thread out of a blocking accept (bugs.md B44). // to wake this thread out of a blocking accept (`docs/BUGS.md` B44).
if core.shutdown.is_stopping() { if core.shutdown.is_stopping() {
info!("control socket closing"); info!("control socket closing");
return; return;
+7 -9
View File
@@ -1,5 +1,5 @@
//! This machine as a Proton Drive Device, and the local folders it keeps in //! This machine as a Proton Drive Device, and the local folders it keeps in
//! step with the remote (devices.md). //! step with the remote.
//! //!
//! A sync folder runs in one of two modes. `mirror` keeps a real local tree that //! A sync folder runs in one of two modes. `mirror` keeps a real local tree that
//! the engine in [`super::sync`] reconciles both ways; `ondemand` evicts that //! the engine in [`super::sync`] reconciles both ways; `ondemand` evicts that
@@ -184,8 +184,8 @@ impl Core {
} }
/// Adopt an existing device as *this* machine's, pinning it in `config.json` /// Adopt an existing device as *this* machine's, pinning it in `config.json`
/// so hostname changes and reinstalls stop registering duplicates /// so hostname changes and reinstalls stop registering duplicates.
/// (features.md 5.1). `None` clears the pin and returns to hostname matching. /// `None` clears the pin and returns to hostname matching.
/// ///
/// Refuses a uid that is not in the account: the whole point of the pin is /// Refuses a uid that is not in the account: the whole point of the pin is
/// that it is authoritative, so accepting a bad one would defer the failure /// that it is authoritative, so accepting a bad one would defer the failure
@@ -260,7 +260,7 @@ impl Core {
}) })
} }
// ---- device folder sync (devices.md, Phase 1) ------------------------- // ---- device folder sync -------------------------
/// Auto-register (or recover) this machine as a Proton Drive Device, caching /// Auto-register (or recover) this machine as a Proton Drive Device, caching
/// it so restarts reuse the same device. Recovery matches an existing remote /// it so restarts reuse the same device. Recovery matches an existing remote
@@ -270,8 +270,7 @@ impl Core {
/// Resolution order: the *adopted* uid pinned in `config.json`, then the /// Resolution order: the *adopted* uid pinned in `config.json`, then the
/// cached DB row (validated remotely), then a hostname match, then create. /// cached DB row (validated remotely), then a hostname match, then create.
/// The pin exists because the hostname heuristic silently registers a second /// The pin exists because the hostname heuristic silently registers a second
/// device after a rename or reinstall, orphaning the first one's folders /// device after a rename or reinstall, orphaning the first one's folders.
/// (features.md 5.1).
pub(crate) fn ensure_device(&self) -> CoreResult<StoredDevice> { pub(crate) fn ensure_device(&self) -> CoreResult<StoredDevice> {
let name = this_hostname(); let name = this_hostname();
// Enumerate the remote devices once: used to validate the pin, validate // Enumerate the remote devices once: used to validate the pin, validate
@@ -617,7 +616,7 @@ impl Core {
} }
/// A sibling Core for a secondary FUSE session rooted at an `ondemand` sync /// A sibling Core for a secondary FUSE session rooted at an `ondemand` sync
/// folder (devices.md Phase 3). It shares everything with this one, /// folder. It shares everything with this one,
/// the tree included, except what belongs to a session: its root and its /// the tree included, except what belongs to a session: its root and its
/// kernel channel, both filled in when the session is built. /// kernel channel, both filled in when the session is built.
pub(crate) fn fork_state(&self) -> Core { pub(crate) fn fork_state(&self) -> Core {
@@ -986,8 +985,7 @@ impl Core {
/// restart (their local dirs are empty on disk — the files live in the cloud). /// restart (their local dirs are empty on disk — the files live in the cloud).
/// Best-effort per folder: a missing local path or a failed remote fetch marks /// Best-effort per folder: a missing local path or a failed remote fetch marks
/// the folder `error` and moves on rather than aborting the rest. Runs on its /// the folder `error` and moves on rather than aborting the rest. Runs on its
/// own thread from `mount` so the network fetches never block startup /// own thread from `mount` so the network fetches never block startup.
/// (devices.md Phase 4).
pub(crate) fn restore_ondemand_mounts(&self) { pub(crate) fn restore_ondemand_mounts(&self) {
let folders = match self.db.sync_folder_list() { let folders = match self.db.sync_folder_list() {
Ok(f) => f, Ok(f) => f,
+6 -7
View File
@@ -1,7 +1,7 @@
//! The write-back drain: turning queued mutations into remote calls. //! The write-back drain: turning queued mutations into remote calls.
//! //!
//! Every write the kernel hands us is answered the moment its bytes and its //! Every write the kernel hands us is answered the moment its bytes and its
//! `pending_op` row are on disk (offline.md Phase 3), which is what lets a `cp` //! `pending_op` row are on disk, which is what lets a `cp`
//! into the mount run at disk speed and lets an offline write succeed at all. //! into the mount run at disk speed and lets an offline write succeed at all.
//! This module is the other half: the worker that walks that queue and performs //! This module is the other half: the worker that walks that queue and performs
//! the uploads, creates, renames and trashes it recorded. //! the uploads, creates, renames and trashes it recorded.
@@ -276,8 +276,7 @@ fn run_authorized_drain(
} }
impl Core { impl Core {
/// Drain the pending-op queue: the background half of every write /// Drain the pending-op queue: the background half of every write.
/// (offline.md Phase 3).
/// ///
/// Runs for the life of the mount. Ops are replayed oldest-first, each /// Runs for the life of the mount. Ops are replayed oldest-first, each
/// retried with doubling backoff and *never* dropped on failure — the staged /// retried with doubling backoff and *never* dropped on failure — the staged
@@ -306,7 +305,7 @@ impl Core {
loop { loop {
// Between ops, never inside one: an op that has started is either // Between ops, never inside one: an op that has started is either
// retired or released, so stopping here cannot leave the queue with // retired or released, so stopping here cannot leave the queue with
// a row claimed by a worker that no longer exists (bugs.md B44). // a row claimed by a worker that no longer exists (`docs/BUGS.md` B44).
if self.shutdown.is_stopping() { if self.shutdown.is_stopping() {
debug!(primary, "drain worker stopping"); debug!(primary, "drain worker stopping");
return; return;
@@ -1150,7 +1149,7 @@ impl Core {
} }
/// Make a node that so far exists only on this machine real, and adopt the /// Make a node that so far exists only on this machine real, and adopt the
/// uid the server gives it (offline.md Phase 3b). /// uid the server gives it.
pub(crate) fn drain_local_node( pub(crate) fn drain_local_node(
&self, &self,
op: &PendingOp, op: &PendingOp,
@@ -1850,7 +1849,7 @@ impl Core {
/// that window the node can be rewritten by another device, trashed, or /// that window the node can be rewritten by another device, trashed, or
/// deleted outright. Sending the blob anyway would silently drop whatever /// deleted outright. Sending the blob anyway would silently drop whatever
/// happened in between, which is exactly the thing the sync engine refuses /// happened in between, which is exactly the thing the sync engine refuses
/// to do (offline.md Phase 3b). /// to do.
/// ///
/// Only checkable against a recorded baseline: a write staged before /// Only checkable against a recorded baseline: a write staged before
/// [`StagedWrite::based_on`] existed, or one against a node that has never /// [`StagedWrite::based_on`] existed, or one against a node that has never
@@ -2009,7 +2008,7 @@ impl Core {
self.db.delete_op(op.id)?; self.db.delete_op(op.id)?;
// Dropping the pending entry hands the node back to the remote's truth: // Dropping the pending entry hands the node back to the remote's truth:
// reads stop coming from the staged blob, and the event sync stops // reads stop coming from the staged blob, and the event sync stops
// skipping it as "ahead of the server" (offline.md Phase 3a). Only if it // skipping it as "ahead of the server". Only if it
// is still *this* write's entry — see [`Core::release_pending`]. // is still *this* write's entry — see [`Core::release_pending`].
self.release_pending(uid, blob); self.release_pending(uid, blob);
self.cache.discard_staged(blob); self.cache.discard_staged(blob);
+17 -18
View File
@@ -132,7 +132,7 @@ impl Filesystem for ProtonFs {
// A provisional size is the *ciphertext* size, which is larger than the // A provisional size is the *ciphertext* size, which is larger than the
// file. Publishing it makes every reader that trusts `st_size` — rsync, // file. Publishing it makes every reader that trusts `st_size` — rsync,
// mmap, sendfile, a sized read loop — run off the end of the file and // mmap, sendfile, a sized read loop — run off the end of the file and
// fail. So resolve it before answering rather than after (bugs.md B14). // fail. So resolve it before answering rather than after (`docs/BUGS.md` B14).
// //
// The cost is one batched round trip for the whole folder, not one per // The cost is one batched round trip for the whole folder, not one per
// file: `ls -l` is one `getattr` per entry and they collapse onto a // file: `ls -l` is one `getattr` per entry and they collapse onto a
@@ -172,7 +172,7 @@ impl Filesystem for ProtonFs {
/// here — once per `opendir`, which the kernel pairs exactly with one /// here — once per `opendir`, which the kernel pairs exactly with one
/// `releasedir` — is what makes the offsets stable, so a create or a trash /// `releasedir` — is what makes the offsets stable, so a create or a trash
/// during an enumeration can no longer make the caller skip or repeat an /// during an enumeration can no longer make the caller skip or repeat an
/// entry (bugs.md B43). /// entry (`docs/BUGS.md` B43).
/// ///
/// Enumerating a cold folder is a remote call, so it goes to a worker for /// Enumerating a cold folder is a remote call, so it goes to a worker for
/// the same reason `lookup` and `readdir` do. /// the same reason `lookup` and `readdir` do.
@@ -346,7 +346,7 @@ impl Filesystem for ProtonFs {
self.core.workers.run(Lane::Transfer, "read", move || { self.core.workers.run(Lane::Transfer, "read", move || {
// The revision disagrees with a provisional size, and B84 fails // The revision disagrees with a provisional size, and B84 fails
// such a read outright, so settle the size first. On the network, // such a read outright, so settle the size first. On the network,
// hence here and not above (bugs.md B100). // hence here and not above (`docs/BUGS.md` B100).
let fsize = match provisional { let fsize = match provisional {
true => core.resolve_provisional_size(ino.0, &uid).unwrap_or(fsize), true => core.resolve_provisional_size(ino.0, &uid).unwrap_or(fsize),
false => fsize, false => fsize,
@@ -517,7 +517,7 @@ impl Filesystem for ProtonFs {
// `> file`. This is the second write path into the API, and // `> file`. This is the second write path into the API, and
// queueing it is what lets a redirect work offline at all: it // queueing it is what lets a redirect work offline at all: it
// never reaches `release`, so without this it failed before any // never reaches `release`, so without this it failed before any
// byte was written (offline.md Phase 2/3b). // byte was written.
// //
// The one branch of `setattr` that can reach the network: // The one branch of `setattr` that can reach the network:
// shrinking keeps the prefix, which `queue_truncate` gap-fills // shrinking keeps the prefix, which `queue_truncate` gap-fills
@@ -607,7 +607,7 @@ impl Filesystem for ProtonFs {
} }
/// `close(2)` calls flush before release. The upload is queued in `release` /// `close(2)` calls flush before release. The upload is queued in `release`
/// and performed in the background (offline.md Phase 3), so there is nothing /// and performed in the background, so there is nothing
/// to push here — the written bytes are already in the scratch file. /// to push here — the written bytes are already in the scratch file.
fn flush( fn flush(
&self, &self,
@@ -985,7 +985,7 @@ pub struct ProtonFs {
} }
/// How many times a write `open` will re-read its base after a `release` /// How many times a write `open` will re-read its base after a `release`
/// published a newer queued revision underneath it (bugs.md B32). /// published a newer queued revision underneath it (`docs/BUGS.md` B32).
/// ///
/// One retry covers the race, which needs a close and an open of the same file /// One retry covers the race, which needs a close and an open of the same file
/// to overlap on different threads. More than that is not contention, it is a /// to overlap on different threads. More than that is not contention, it is a
@@ -1021,7 +1021,7 @@ fn build_listing(st: &State, ino: u64) -> DirListing {
/// ///
/// The cookie handed back is the index of the *next* entry, so a resumed call /// The cookie handed back is the index of the *next* entry, so a resumed call
/// continues exactly where the last one stopped — which is only true because /// continues exactly where the last one stopped — which is only true because
/// the listing behind those indexes is a snapshot (bugs.md B43). /// the listing behind those indexes is a snapshot (`docs/BUGS.md` B43).
fn reply_listing(listing: &DirListing, offset: u64, root: u64, mut reply: ReplyDirectory) { fn reply_listing(listing: &DirListing, offset: u64, root: u64, mut reply: ReplyDirectory) {
for (i, (ino, is_dir, name)) in listing.iter().enumerate().skip(offset as usize) { for (i, (ino, is_dir, name)) in listing.iter().enumerate().skip(offset as usize) {
let ft = if *is_dir { let ft = if *is_dir {
@@ -1094,7 +1094,7 @@ impl ProtonFs {
// `getattr` would, so `ls -l` — which is one `lookup` // `getattr` would, so `ls -l` — which is one `lookup`
// per entry and no `getattr` at all — takes its sizes // per entry and no `getattr` at all — takes its sizes
// from here. Resolving only in `getattr` left the whole // from here. Resolving only in `getattr` left the whole
// listing provisional (bugs.md B14). // listing provisional (`docs/BUGS.md` B14).
let provisional = matches!( let provisional = matches!(
&e.node.kind, &e.node.kind,
NodeKind::File { NodeKind::File {
@@ -1276,7 +1276,7 @@ impl ProtonFs {
// revision in between sends this around again. Without the re-check the // revision in between sends this around again. Without the re-check the
// new handle carried the *older* base, and its own release published // new handle carried the *older* base, and its own release published
// content that silently discarded the revision closed a moment earlier // content that silently discarded the revision closed a moment earlier
// (bugs.md B32). A second attempt is already unlikely and a third would // (`docs/BUGS.md` B32). A second attempt is already unlikely and a third would
// mean something is republishing continuously, so give up rather than // mean something is republishing continuously, so give up rather than
// spin: the caller's `open` fails instead of its data being lost. // spin: the caller's `open` fails instead of its data being lost.
// //
@@ -1286,7 +1286,7 @@ impl ProtonFs {
for attempt in 0..OPEN_BASE_ATTEMPTS { for attempt in 0..OPEN_BASE_ATTEMPTS {
// The handle starts at the base's size. A provisional one is the // The handle starts at the base's size. A provisional one is the
// ciphertext size, so the file would grow by the encryption // ciphertext size, so the file would grow by the encryption
// overhead and reading the base would fail (bugs.md B100). A // overhead and reading the base would fail (`docs/BUGS.md` B100). A
// queued revision stamps a real size, so this only goes to the // queued revision stamps a real size, so this only goes to the
// network for a file the listing has not sized yet. // network for a file the listing has not sized yet.
if provisional && self.core.pending_blob(&uid).is_none() { if provisional && self.core.pending_blob(&uid).is_none() {
@@ -1541,7 +1541,7 @@ impl ProtonFs {
let transient = is_transient_name(name); let transient = is_transient_name(name);
// Offline the server cannot mint a uid, so invent one and queue the // Offline the server cannot mint a uid, so invent one and queue the
// create. The file is real to the caller either way; only its identity is // create. The file is real to the caller either way; only its identity is
// provisional until the drain (offline.md Phase 3b). // provisional until the drain.
// //
// A parent that is itself still queued forces the same path even when we // A parent that is itself still queued forces the same path even when we
// are online: the API has no folder to put this in yet. // are online: the API has no folder to put this in yet.
@@ -1652,7 +1652,7 @@ impl ProtonFs {
.ok(); .ok();
// As in `create`: offline — or under a parent that is itself still // As in `create`: offline — or under a parent that is itself still
// queued — the folder becomes a placeholder that the drain turns into a // queued — the folder becomes a placeholder that the drain turns into a
// real one (offline.md Phase 3b). // real one.
// Losing the network on the way queues it the same way. // Losing the network on the way queues it the same way.
let minted = if self.core.sends_inline() && !is_local_uid(&parent_uid) { let minted = if self.core.sends_inline() && !is_local_uid(&parent_uid) {
if let Err(error) = self.core.require_uid_writable(&parent_uid) { if let Err(error) = self.core.require_uid_writable(&parent_uid) {
@@ -1874,7 +1874,7 @@ impl ProtonFs {
// rename, so the victim has to be removed first and the operation stops // rename, so the victim has to be removed first and the operation stops
// being atomic. Without this the 422 surfaced as a blanket EIO and every // being atomic. Without this the 422 surfaced as a blanket EIO and every
// write-to-temp-then-rename tool — rsync, atomic editor saves — failed // write-to-temp-then-rename tool — rsync, atomic editor saves — failed
// at the very end of its transfer (bugs.md B13). // at the very end of its transfer (`docs/BUGS.md` B13).
// //
// `RENAME_EXCHANGE` has no Proton primitive and cannot be emulated // `RENAME_EXCHANGE` has no Proton primitive and cannot be emulated
// without a window in which one of the two names does not exist. // without a window in which one of the two names does not exist.
@@ -1956,8 +1956,7 @@ impl ProtonFs {
}; };
// A node whose own creation is still queued has no server-side identity // A node whose own creation is still queued has no server-side identity
// to rename: the queued op *is* the node, so rewriting its target is the // to rename: the queued op *is* the node, so rewriting its target is the
// whole rename. Nothing reaches the API, which is why this works offline // whole rename. Nothing reaches the API, which is why this works offline.
// (offline.md Phase 3b).
// A create that lands meanwhile leaves no op to rewrite; the node is // A create that lands meanwhile leaves no op to rewrite; the node is
// then renamed by the uid Drive gave it (docs/BUGS.md B142). // then renamed by the uid Drive gave it (docs/BUGS.md B142).
if is_local_uid(&uid) { if is_local_uid(&uid) {
@@ -2156,7 +2155,7 @@ impl ProtonFs {
.is_some_and(|e| e.open_count > 0); .is_some_and(|e| e.open_count > 0);
// A node the server has never heard of cannot be trashed there; deleting // A node the server has never heard of cannot be trashed there; deleting
// it just means its queued creation is no longer wanted. This works // it just means its queued creation is no longer wanted. This works
// offline, which the remote path below cannot (offline.md Phase 3b). // offline, which the remote path below cannot.
if is_local_uid(&uid) { if is_local_uid(&uid) {
let dropped = if open_now { let dropped = if open_now {
self.core.withdraw_queued_ops(&uid) self.core.withdraw_queued_ops(&uid)
@@ -2185,7 +2184,7 @@ impl ProtonFs {
} }
// Offline: queue it. Trashing is the one mutation a user expects to work // Offline: queue it. Trashing is the one mutation a user expects to work
// regardless — the file is gone from their point of view the moment the // regardless — the file is gone from their point of view the moment the
// command returns (offline.md Phase 3b). The same when the network goes // command returns. The same when the network goes
// away under the remote call below. // away under the remote call below.
let queue_it = |reply: ReplyEmpty| match self.core.queue_trash(&uid, name) { let queue_it = |reply: ReplyEmpty| match self.core.queue_trash(&uid, name) {
Ok(()) => reply.ok(), Ok(()) => reply.ok(),
@@ -2234,7 +2233,7 @@ impl ProtonFs {
self.core.invalidate_trash(); self.core.invalidate_trash();
// Every other trash site records itself; this one did not, which made a // Every other trash site records itself; this one did not, which made a
// file found in the trash impossible to attribute after the fact — the // file found in the trash impossible to attribute after the fact — the
// activity log was the only record and it showed nothing (bugs.md B2). // activity log was the only record and it showed nothing (`docs/BUGS.md` B2).
self.core self.core
.log_activity(ActivityKind::Trash, name, "trashed from the mount", true); .log_activity(ActivityKind::Trash, name, "trashed from the mount", true);
reply.ok(); reply.ok();
+33 -35
View File
@@ -226,7 +226,7 @@ const DRAIN_WORKERS: usize = 16;
const STAGING_ORPHAN_RETAIN: Duration = Duration::from_secs(30 * 24 * 60 * 60); const STAGING_ORPHAN_RETAIN: Duration = Duration::from_secs(30 * 24 * 60 * 60);
/// First and longest delay between probes for the network coming back after an /// First and longest delay between probes for the network coming back after an
/// offline mount (offline.md Phase 1). Doubles from min to max: a laptop shut in /// offline mount. Doubles from min to max: a laptop shut in
/// a bag is the common case, so the steady state must be cheap, while a brief /// a bag is the common case, so the steady state must be cheap, while a brief
/// blip should still recover in seconds. /// blip should still recover in seconds.
const ONLINE_PROBE_MIN: Duration = Duration::from_secs(5); const ONLINE_PROBE_MIN: Duration = Duration::from_secs(5);
@@ -334,7 +334,7 @@ const MAX_THUMBNAIL_MISSES: usize = 8192;
/// ///
/// Matches the SDK's own `MAX_BATCH_COUNT`, so a chunk is exactly one request: /// Matches the SDK's own `MAX_BATCH_COUNT`, so a chunk is exactly one request:
/// chunking smaller would add round trips, larger would be split anyway and /// chunking smaller would add round trips, larger would be split anyway and
/// delay the waiters this chunking exists to release (bugs.md B14). /// delay the waiters this chunking exists to release (`docs/BUGS.md` B14).
const SIZE_UPGRADE_CHUNK: usize = 150; const SIZE_UPGRADE_CHUNK: usize = 150;
/// How many folders may have a size upgrade in flight at once. /// How many folders may have a size upgrade in flight at once.
@@ -344,7 +344,7 @@ const SIZE_UPGRADE_CHUNK: usize = 150;
/// both with the number of folders it walked. Eight keeps an interactive `ls -l` /// both with the number of folders it walked. Eight keeps an interactive `ls -l`
/// (one folder, sometimes a couple) entirely unaffected while putting a ceiling /// (one folder, sometimes a couple) entirely unaffected while putting a ceiling
/// on the recursive case. A folder past the cap is queued, not dropped: one of /// on the recursive case. A folder past the cap is queued, not dropped: one of
/// the running threads takes it over when its own batch ends (bugs.md B100). /// the running threads takes it over when its own batch ends (`docs/BUGS.md` B100).
const MAX_SIZE_UPGRADES: usize = 8; const MAX_SIZE_UPGRADES: usize = 8;
#[derive(Clone, Debug, Eq, PartialEq)] #[derive(Clone, Debug, Eq, PartialEq)]
@@ -535,7 +535,7 @@ struct Core {
workers: Arc<Workers>, workers: Arc<Workers>,
/// Unified SQLite metadata cache: the persistence layer behind the in-memory /// Unified SQLite metadata cache: the persistence layer behind the in-memory
/// `State` maps. Every mutation writes through here, and the maps rehydrate /// `State` maps. Every mutation writes through here, and the maps rehydrate
/// from it on mount (plan.md P1). /// from it on mount.
db: Arc<Db>, db: Arc<Db>,
/// Serializes only shared-list publication and access-loss invalidation. /// Serializes only shared-list publication and access-loss invalidation.
/// Network calls happen before taking it. /// Network calls happen before taking it.
@@ -545,8 +545,8 @@ struct Core {
/// Successful refreshes suppress retries even if persisting their timestamp /// Successful refreshes suppress retries even if persisting their timestamp
/// fails. Shared across primary/fork clones and cleared by access events. /// fails. Shared across primary/fork clones and cleared by access events.
shared_refresh_deadlines: Arc<Mutex<SharedRefreshDeadlines>>, shared_refresh_deadlines: Arc<Mutex<SharedRefreshDeadlines>>,
/// False while the API is unreachable and we are serving the cached tree /// False while the API is unreachable and we are serving the cached tree.
/// (offline.md Phase 1). Set by the probe thread; read by front-ends through /// Set by the probe thread; read by front-ends through
/// `Response::Status` so the UI can say so rather than leaving the user to /// `Response::Status` so the UI can say so rather than leaving the user to
/// infer it from a wall of EIO. /// infer it from a wall of EIO.
online: Arc<AtomicBool>, online: Arc<AtomicBool>,
@@ -557,8 +557,8 @@ struct Core {
/// [`pause::PAUSED_INDEFINITELY`] for "until resumed", `0` when not paused. /// [`pause::PAUSED_INDEFINITELY`] for "until resumed", `0` when not paused.
/// Persisted, so a pause survives a restart (see [`Core::sync_paused`]). /// Persisted, so a pause survives a restart (see [`Core::sync_paused`]).
sync_paused_until: Arc<AtomicI64>, sync_paused_until: Arc<AtomicI64>,
/// Writes accepted from the kernel but not yet uploaded, keyed by node /// Writes accepted from the kernel but not yet uploaded, keyed by node.
/// (offline.md Phase 3). The in-memory face of the `pending_op` table, from /// The in-memory face of the `pending_op` table, from
/// which it is rebuilt on mount. /// which it is rebuilt on mount.
/// ///
/// Two things read it: [`Core::read_range`], because until the op drains the /// Two things read it: [`Core::read_range`], because until the op drains the
@@ -586,7 +586,7 @@ struct Core {
/// the queue instead of waiting out their backoff. /// the queue instead of waiting out their backoff.
drain_wake: Arc<(Mutex<bool>, Condvar)>, drain_wake: Arc<(Mutex<bool>, Condvar)>,
/// The daemon's stop signal, shared with every background loop this `Core` /// The daemon's stop signal, shared with every background loop this `Core`
/// was cloned into so teardown can end and join them (bugs.md B44). /// was cloned into so teardown can end and join them (`docs/BUGS.md` B44).
shutdown: Arc<shutdown::Shutdown>, shutdown: Arc<shutdown::Shutdown>,
/// How long this node's last upload actually took, in milliseconds, with /// How long this node's last upload actually took, in milliseconds, with
/// the time it was measured for eviction. Feeds /// the time it was measured for eviction. Feeds
@@ -695,7 +695,7 @@ struct Core {
/// Size upgrades currently running, per folder inode, across every mount. A `getattr` that /// Size upgrades currently running, per folder inode, across every mount. A `getattr` that
/// needs a real size waits on the entry rather than issuing its own fetch — /// needs a real size waits on the entry rather than issuing its own fetch —
/// `ls -l` of a folder is one `getattr` per file, and they must collapse /// `ls -l` of a folder is one `getattr` per file, and they must collapse
/// onto a single batch (bugs.md B14). /// onto a single batch (`docs/BUGS.md` B14).
size_upgrades: Arc<Mutex<HashMap<u64, Arc<SizeUpgrade>>>>, size_upgrades: Arc<Mutex<HashMap<u64, Arc<SizeUpgrade>>>>,
/// Replies parked on those upgrades. Shared across every mount for the same /// Replies parked on those upgrades. Shared across every mount for the same
/// reason the pool is: the bound that matters is per daemon. /// reason the pool is: the bound that matters is per daemon.
@@ -726,11 +726,11 @@ struct Core {
/// can say what a pass is doing rather than just "syncing". An entry exists /// can say what a pass is doing rather than just "syncing". An entry exists
/// only while that folder's reconcile pass is running. /// only while that folder's reconcile pass is running.
sync_progress: Arc<Mutex<HashMap<i64, SyncProgress>>>, sync_progress: Arc<Mutex<HashMap<i64, SyncProgress>>>,
/// Channel to the folder-sync engine (devices.md Phase 2): nudges it to /// Channel to the folder-sync engine: nudges it to
/// reconcile a folder, reconcile everything, or re-scan its watch set. /// reconcile a folder, reconcile everything, or re-scan its watch set.
sync_tx: std::sync::mpsc::Sender<sync::SyncMsg>, sync_tx: std::sync::mpsc::Sender<sync::SyncMsg>,
/// Secondary FUSE sessions for `ondemand` sync folders, keyed by sync-folder /// Secondary FUSE sessions for `ondemand` sync folders, keyed by sync-folder
/// id (devices.md Phase 3). Each is a `ProtonFs` rooted at the folder's remote /// id. Each is a `ProtonFs` rooted at the folder's remote
/// node, mounted over its local path, sharing this Core's client/cache/db and /// node, mounted over its local path, sharing this Core's client/cache/db and
/// tree but with its own root (`fork_state`). Held so we can unmount on /// tree but with its own root (`fork_state`). Held so we can unmount on
/// toggle back to `mirror` and on daemon shutdown. Each entry retains the /// toggle back to `mirror` and on daemon shutdown. Each entry retains the
@@ -1833,8 +1833,7 @@ impl Core {
info!(nodes = st.entries.len(), "hydrated metadata cache from db"); info!(nodes = st.entries.len(), "hydrated metadata cache from db");
} }
/// Rebuild the in-memory pending map from the `pending_op` table on mount /// Rebuild the in-memory pending map from the `pending_op` table on mount.
/// (offline.md Phase 3).
/// ///
/// A queued write survives a restart — that is the point of persisting it — /// A queued write survives a restart — that is the point of persisting it —
/// so until the drain worker gets to it, reads of that file must still come /// so until the drain worker gets to it, reads of that file must still come
@@ -2774,7 +2773,7 @@ impl Core {
missing.push(uid.clone()); missing.push(uid.clone());
} }
// Returned, not awaited here: the caller is a `getattr` that must not // Returned, not awaited here: the caller is a `getattr` that must not
// answer with a provisional size (bugs.md B14), and it parks its reply // answer with a provisional size (`docs/BUGS.md` B14), and it parks its reply
// on this batch rather than holding a thread until it lands. // on this batch rather than holding a thread until it lands.
self.upgrade_sizes(key, missing) self.upgrade_sizes(key, missing)
} }
@@ -2791,7 +2790,7 @@ impl Core {
/// `total_size_on_storage`, the *ciphertext* size, which is slightly larger /// `total_size_on_storage`, the *ciphertext* size, which is slightly larger
/// than the real one. A read must not plan on it: since B84 a size the /// than the real one. A read must not plan on it: since B84 a size the
/// revision disagrees with fails the read, so `read` settles the size first /// revision disagrees with fails the read, so `read` settles the size first
/// ([`Core::resolve_provisional_size`], bugs.md B100). In `stat` it is a /// ([`Core::resolve_provisional_size`], `docs/BUGS.md` B100). In `stat` it is a
/// discrepancy that closes within a round trip, not a repeat of B11 (which /// discrepancy that closes within a round trip, not a repeat of B11 (which
/// reported **0** and so suppressed reads entirely). /// reported **0** and so suppressed reads entirely).
/// ///
@@ -2865,11 +2864,11 @@ impl Core {
/// Chunked so a waiter is released as soon as *its* file is resolved. A /// Chunked so a waiter is released as soon as *its* file is resolved. A
/// single 793-node batch took ~80 s, which outran [`SizeUpgrade::WAIT`] and /// single 793-node batch took ~80 s, which outran [`SizeUpgrade::WAIT`] and
/// put provisional sizes back in front of callers — the bug this was /// put provisional sizes back in front of callers — the bug this was
/// supposed to fix (bugs.md B14). /// supposed to fix (`docs/BUGS.md` B14).
/// ///
/// When the batch ends, the thread takes over the oldest batch the cap held /// When the batch ends, the thread takes over the oldest batch the cap held
/// back, so at most [`MAX_SIZE_UPGRADES`] threads run however many folders /// back, so at most [`MAX_SIZE_UPGRADES`] threads run however many folders
/// a recursive walk touches (bugs.md B100). /// a recursive walk touches (`docs/BUGS.md` B100).
fn run_size_upgrade(&self, key: u64, uids: Vec<NodeUid>, slot: Arc<SizeUpgrade>) { fn run_size_upgrade(&self, key: u64, uids: Vec<NodeUid>, slot: Arc<SizeUpgrade>) {
let (mut key, mut uids, mut slot) = (key, uids, slot); let (mut key, mut uids, mut slot) = (key, uids, slot);
loop { loop {
@@ -2897,7 +2896,7 @@ impl Core {
/// size it is given against the revision's own block table and fails the /// size it is given against the revision's own block table and fails the
/// read on a mismatch (B84), so reading on the provisional size is an `EIO` /// read on a mismatch (B84), so reading on the provisional size is an `EIO`
/// every time. `getattr` usually settles the size first, but not when its /// every time. `getattr` usually settles the size first, but not when its
/// wait times out, so the read cannot rely on it (bugs.md B100). /// wait times out, so the read cannot rely on it (`docs/BUGS.md` B100).
fn resolve_provisional_size(&self, ino: u64, uid: &NodeUid) -> Option<u64> { fn resolve_provisional_size(&self, ino: u64, uid: &NodeUid) -> Option<u64> {
let parent = self.state().entries.get(&ino)?.parent; let parent = self.state().entries.get(&ino)?.parent;
// Without a network the answer is `None` either way; the bound keeps a // Without a network the answer is `None` either way; the bound keeps a
@@ -3317,8 +3316,7 @@ impl Core {
} }
} }
/// Accept a released write handle's bytes and queue their upload /// Accept a released write handle's bytes and queue their upload.
/// (offline.md Phase 3).
/// ///
/// This is what makes a copy into the mount run at disk speed: the caller's /// This is what makes a copy into the mount run at disk speed: the caller's
/// `close` returns once the bytes are staged on local disk and the intent is /// `close` returns once the bytes are staged on local disk and the intent is
@@ -3785,8 +3783,8 @@ impl Core {
Ok(()) Ok(())
} }
/// Invent a node under `parent_uid` and queue the op that will make it real /// Invent a node under `parent_uid` and queue the op that will make it real.
/// (offline.md Phase 3b). Returns the node to intern, exactly as the online /// Returns the node to intern, exactly as the online
/// path returns the one the server minted. /// path returns the one the server minted.
/// ///
/// The parent may itself be a placeholder — `mkdir -p` offline, or `cp -r` of /// The parent may itself be a placeholder — `mkdir -p` offline, or `cp -r` of
@@ -3842,7 +3840,7 @@ impl Core {
} }
/// Queue giving a node a new parent and/or name, and apply it to the tree /// Queue giving a node a new parent and/or name, and apply it to the tree
/// now (offline.md Phase 3b). /// now.
/// ///
/// The op records the desired end state rather than the step, so it both /// The op records the desired end state rather than the step, so it both
/// supersedes any earlier queued rename and lets the drain skip whichever /// supersedes any earlier queued rename and lets the drain skip whichever
@@ -3950,7 +3948,7 @@ impl Core {
} }
/// Queue trashing a node the server knows about, and drop it from the tree /// Queue trashing a node the server knows about, and drop it from the tree
/// now (offline.md Phase 3b). /// now.
/// ///
/// Anything else queued for this node is discarded first: the user has said /// Anything else queued for this node is discarded first: the user has said
/// the file should not exist, so uploading bytes to it or renaming it are /// the file should not exist, so uploading bytes to it or renaming it are
@@ -4081,7 +4079,7 @@ impl Core {
/// ///
/// The path identifies the revision: publication stages a fresh file and /// The path identifies the revision: publication stages a fresh file and
/// swaps the map entry to point at it, so an unchanged path means the base a /// swaps the map entry to point at it, so an unchanged path means the base a
/// caller sampled earlier is still the base now (bugs.md B32). /// caller sampled earlier is still the base now (`docs/BUGS.md` B32).
fn pending_blob(&self, uid: &NodeUid) -> Option<PathBuf> { fn pending_blob(&self, uid: &NodeUid) -> Option<PathBuf> {
self.pending.lock().get(uid).map(|p| p.path.clone()) self.pending.lock().get(uid).map(|p| p.path.clone())
} }
@@ -5124,7 +5122,7 @@ impl Core {
/// `rename(2)` promises to replace an existing destination atomically. Proton /// `rename(2)` promises to replace an existing destination atomically. Proton
/// offers no such primitive — `rename_node` refuses a name that is already /// offers no such primitive — `rename_node` refuses a name that is already
/// taken — so this is the first half of an emulation that is *not* atomic: /// taken — so this is the first half of an emulation that is *not* atomic:
/// see [`Core::restore_replaced`] for the other half (bugs.md B13). /// see [`Core::restore_replaced`] for the other half (`docs/BUGS.md` B13).
/// ///
/// A node whose own creation is still queued has never reached the server, so /// A node whose own creation is still queued has never reached the server, so
/// dropping its queued ops is the whole removal; nothing goes to the wire and /// dropping its queued ops is the whole removal; nothing goes to the wire and
@@ -5950,8 +5948,8 @@ fn parse_node_uid(s: &str) -> Option<NodeUid> {
} }
/// The uid a node made on this machine goes by until it lands, so it can be /// The uid a node made on this machine goes by until it lands, so it can be
/// interned, listed and written to before the server has ever heard of it /// interned, listed and written to before the server has ever heard of it:
/// (offline.md Phase 3b): its row's local id, as [`pdfs_core::db::local_uid`] /// its row's local id, as [`pdfs_core::db::local_uid`]
/// has it. /// has it.
fn local_node_uid(lid: i64) -> NodeUid { fn local_node_uid(lid: i64) -> NodeUid {
NodeUid::new(VolumeId::from(LOCAL_VOLUME), LinkId::from(lid.to_string())) NodeUid::new(VolumeId::from(LOCAL_VOLUME), LinkId::from(lid.to_string()))
@@ -6114,7 +6112,7 @@ fn dir_is_empty(dir: &Path) -> std::io::Result<bool> {
/// Delete everything inside `dir` but keep `dir` itself (it stays as the FUSE /// Delete everything inside `dir` but keep `dir` itself (it stays as the FUSE
/// mountpoint). Used when a `mirror` folder flips to `ondemand`: the local files /// mountpoint). Used when a `mirror` folder flips to `ondemand`: the local files
/// are the disk we're reclaiming (devices.md Phase 3). /// are the disk we're reclaiming.
fn evict_dir_contents(dir: &Path) -> std::io::Result<()> { fn evict_dir_contents(dir: &Path) -> std::io::Result<()> {
for entry in std::fs::read_dir(dir)? { for entry in std::fs::read_dir(dir)? {
let path = entry?.path(); let path = entry?.path();
@@ -6196,7 +6194,7 @@ struct Progress {
/// ///
/// Waiters are released **per chunk**, not once at the end: a waiter only cares /// Waiters are released **per chunk**, not once at the end: a waiter only cares
/// about its own file, and waiting for the other 792 is what let a large folder /// about its own file, and waiting for the other 792 is what let a large folder
/// outrun the timeout (bugs.md B14). /// outrun the timeout (`docs/BUGS.md` B14).
#[derive(Default)] #[derive(Default)]
struct SizeUpgrade { struct SizeUpgrade {
inner: Mutex<Progress>, inner: Mutex<Progress>,
@@ -6222,7 +6220,7 @@ enum SizeUpgradeClaim {
/// Past the cap this used to return nothing, and the caller answered with the /// Past the cap this used to return nothing, and the caller answered with the
/// provisional size at once. Nothing retried the folder until the next `stat`, /// provisional size at once. Nothing retried the folder until the next `stat`,
/// so a recursive walk over more than [`MAX_SIZE_UPGRADES`] fresh folders showed /// so a recursive walk over more than [`MAX_SIZE_UPGRADES`] fresh folders showed
/// ciphertext sizes for most of them (bugs.md B100). Queued, the batch is still /// ciphertext sizes for most of them (`docs/BUGS.md` B100). Queued, the batch is still
/// fetched, and a waiter gets the real size unless [`SizeUpgrade::WAIT`] runs out. /// fetched, and a waiter gets the real size unless [`SizeUpgrade::WAIT`] runs out.
fn claim_size_upgrade( fn claim_size_upgrade(
in_flight: &mut HashMap<u64, Arc<SizeUpgrade>>, in_flight: &mut HashMap<u64, Arc<SizeUpgrade>>,
@@ -6425,7 +6423,7 @@ impl SizeWaitQueue {
/// Split out from the handler because it is the part that is pure and the part /// Split out from the handler because it is the part that is pure and the part
/// that is dangerous: every `Err` here is a refusal that happens *before* /// that is dangerous: every `Err` here is a refusal that happens *before*
/// anything is trashed, and getting one wrong turns a refusal into the /// anything is trashed, and getting one wrong turns a refusal into the
/// destruction of the destination (bugs.md B13). /// destruction of the destination (`docs/BUGS.md` B13).
/// ///
/// `dst_empty` is only meaningful when `dst_dir`; pass `true` otherwise. /// `dst_empty` is only meaningful when `dst_dir`; pass `true` otherwise.
fn check_replaceable(src_dir: bool, dst_dir: bool, dst_empty: bool) -> Result<(), Errno> { fn check_replaceable(src_dir: bool, dst_dir: bool, dst_empty: bool) -> Result<(), Errno> {
@@ -7155,7 +7153,7 @@ mod size_upgrade_tests {
} }
/// Past the cap a folder used to get no upgrade at all, so a recursive /// Past the cap a folder used to get no upgrade at all, so a recursive
/// walk reported ciphertext sizes for most of what it listed (bugs.md B100). /// walk reported ciphertext sizes for most of what it listed (`docs/BUGS.md` B100).
#[test] #[test]
fn a_folder_past_the_cap_is_queued_not_dropped() { fn a_folder_past_the_cap_is_queued_not_dropped() {
let mut in_flight = HashMap::new(); let mut in_flight = HashMap::new();
@@ -7208,7 +7206,7 @@ mod replace_tests {
/// The case that motivated all of this: rsync renaming its temp file over /// The case that motivated all of this: rsync renaming its temp file over
/// the real one. Two plain files, and it has to be allowed — refusing is /// the real one. Two plain files, and it has to be allowed — refusing is
/// what made every rsync transfer fail at the last step (bugs.md B13). /// what made every rsync transfer fail at the last step (`docs/BUGS.md` B13).
#[test] #[test]
fn a_file_may_replace_a_file() { fn a_file_may_replace_a_file() {
assert!(check_replaceable(false, false, true).is_ok()); assert!(check_replaceable(false, false, true).is_ok());
+1 -1
View File
@@ -248,7 +248,7 @@ impl Core {
} }
/// Watch the link for the life of the mount: sleep while it is up, and /// Watch the link for the life of the mount: sleep while it is up, and
/// probe it back while it is down (offline.md Phase 1). /// probe it back while it is down.
/// ///
/// A mount that started from the cache is down from the first moment; one /// A mount that started from the cache is down from the first moment; one
/// that started online goes down when [`Core::mark_offline`] says so. /// that started online goes down when [`Core::mark_offline`] says so.
+5 -5
View File
@@ -193,7 +193,7 @@ pub(super) fn clear_stale_mount(mountpoint: &Path) {
} }
/// `sync_state` key holding the uid of the My Files root, so a later run can /// `sync_state` key holding the uid of the My Files root, so a later run can
/// recover it from `nodes` without the API (offline.md Phase 1). /// recover it from `nodes` without the API.
const ROOT_UID_KEY: &str = "root_uid"; const ROOT_UID_KEY: &str = "root_uid";
/// The My Files root, and whether we got it from the API (`true`) or from the /// The My Files root, and whether we got it from the API (`true`) or from the
@@ -401,7 +401,7 @@ pub(crate) fn mount_with(
.all_share_access() .all_share_access()
.map_err(|error| std::io::Error::other(format!("load shared access: {error}")))?; .map_err(|error| std::io::Error::other(format!("load shared access: {error}")))?;
// The folder-sync engine (devices.md Phase 2) runs on its own thread and is // The folder-sync engine runs on its own thread and is
// nudged over this channel; the sender lives in Core so control-socket // nudged over this channel; the sender lives in Core so control-socket
// handlers can trigger reconciles. // handlers can trigger reconciles.
let (sync_tx, sync_rx) = std::sync::mpsc::channel::<sync::SyncMsg>(); let (sync_tx, sync_rx) = std::sync::mpsc::channel::<sync::SyncMsg>();
@@ -501,7 +501,7 @@ pub(crate) fn mount_with(
// Every long-lived thread started from here on is kept in `workers` and // Every long-lived thread started from here on is kept in `workers` and
// joined at the end of this function. Leaving them running turned an // joined at the end of this function. Leaving them running turned an
// in-process remount into a second generation of drain workers, sync engines // in-process remount into a second generation of drain workers, sync engines
// and sweeps all operating on a mount that no longer exists (bugs.md B44). // and sweeps all operating on a mount that no longer exists (`docs/BUGS.md` B44).
let mut workers: Vec<std::thread::JoinHandle<()>> = Vec::new(); let mut workers: Vec<std::thread::JoinHandle<()>> = Vec::new();
for worker in 0..DRAIN_WORKERS { for worker in 0..DRAIN_WORKERS {
let core = core.clone(); let core = core.clone();
@@ -738,7 +738,7 @@ pub(crate) fn mount_with(
}; };
// Unmount every on-demand sync folder too, or the kernel mounts linger as // Unmount every on-demand sync folder too, or the kernel mounts linger as
// stale and the next start fails with EBUSY (devices.md Phase 3). // stale and the next start fails with EBUSY.
if let Some(worker) = restore_worker if let Some(worker) = restore_worker
&& worker.join().is_err() && worker.join().is_err()
{ {
@@ -798,7 +798,7 @@ const WORKER_JOIN_DEADLINE: Duration = Duration::from_secs(10);
/// The mount used to return with its drain workers, sync engine, sweep, online /// The mount used to return with its drain workers, sync engine, sweep, online
/// probe, indexer and control listener all still running on `Core` clones. The /// probe, indexer and control listener all still running on `Core` clones. The
/// process usually exited straight afterwards and hid it; an in-process remount /// process usually exited straight afterwards and hid it; an in-process remount
/// did not, and started a second full set on top of the first (bugs.md B44). /// did not, and started a second full set on top of the first (`docs/BUGS.md` B44).
/// ///
/// Two of the waits need waking rather than merely flagging: the drain workers /// Two of the waits need waking rather than merely flagging: the drain workers
/// block on their own condvar (`wake_drain`), and the control listener blocks /// block on their own condvar (`wake_drain`), and the control listener blocks
+2 -2
View File
@@ -1,5 +1,5 @@
//! Reading and writing this machine's `profile.json` in its device root, and //! Reading and writing this machine's `profile.json` in its device root, and
//! the restore that rebuilds a machine from one (features.md 5.2 and 5.3). //! the restore that rebuilds a machine from one.
//! //!
//! The document itself is defined in [`pdfs_core::profile`]; this is the half //! The document itself is defined in [`pdfs_core::profile`]; this is the half
//! that talks to Drive. Saving is best-effort and never blocks a user action: //! that talks to Drive. Saving is best-effort and never blocks a user action:
@@ -246,7 +246,7 @@ impl Core {
Profile::parse(&buf).map(Some).map_err(CoreError::invalid) Profile::parse(&buf).map(Some).map_err(CoreError::invalid)
} }
// ---- restore (features.md 5.2) ---------------------------------------- // ---- restore ----------------------------------------
/// The folders under a device that could be synced here, each carrying a /// The folders under a device that could be synced here, each carrying a
/// proposed local path. `device` names another registered device by uid; /// proposed local path. `device` names another registered device by uid;
+9 -9
View File
@@ -464,7 +464,7 @@ impl Core {
/// it, and the [`BLOCK_SIZE`] assumption when it has not — which is every /// it, and the [`BLOCK_SIZE`] assumption when it has not — which is every
/// first read, because learning it means opening a `RevisionReader`, and /// first read, because learning it means opening a `RevisionReader`, and
/// opening one per read is precisely the per-file key derivation the block /// opening one per read is precisely the per-file key derivation the block
/// cache exists to avoid (bugs.md B12, B85). /// cache exists to avoid (`docs/BUGS.md` B12, B85).
/// ///
/// Being wrong here is not a correctness problem: `read_at` plans over the /// Being wrong here is not a correctness problem: `read_at` plans over the
/// real sizes and clamps to what was asked for either way. It costs a /// real sizes and clamps to what was asked for either way. It costs a
@@ -493,13 +493,13 @@ impl Core {
) -> Result<Vec<u8>, Errno> { ) -> Result<Vec<u8>, Errno> {
// A queued write has not reached the remote yet, so the remote's current // A queued write has not reached the remote yet, so the remote's current
// revision is stale and the staged blob is the truth. Serve from it until // revision is stale and the staged blob is the truth. Serve from it until
// the drain worker lands the upload (offline.md Phase 3). // the drain worker lands the upload.
if let Some(pending) = self.pending.lock().get(uid).cloned() { if let Some(pending) = self.pending.lock().get(uid).cloned() {
return self.read_pending(&pending, offset, len); return self.read_pending(&pending, offset, len);
} }
// A node created offline and never written has no blob and no remote: it // A node created offline and never written has no blob and no remote: it
// is an empty file, and asking the API about a `local~` uid would only // is an empty file, and asking the API about a `local~` uid would only
// earn a 404 (offline.md Phase 3b). Unless its create landed since the // earn a 404. Unless its create landed since the
// read looked: the landing moved the queued write to the uid Drive // read looked: the landing moved the queued write to the uid Drive
// gave it, and the row says which (`docs/BUGS.md` B179). // gave it, and the row says which (`docs/BUGS.md` B179).
if is_local_uid(uid) { if is_local_uid(uid) {
@@ -636,7 +636,7 @@ impl Core {
/// ///
/// A block that comes back shorter than the file's size says it must be is /// A block that comes back shorter than the file's size says it must be is
/// never served, cached or promoted: it is repaired from the whole-file /// never served, cached or promoted: it is repaired from the whole-file
/// download, or the read fails. See [`Core::repair_block`] (bugs.md B84). /// download, or the read fails. See [`Core::repair_block`] (`docs/BUGS.md` B84).
async fn read_block( async fn read_block(
&self, &self,
reader: &Arc<dyn RevisionRead>, reader: &Arc<dyn RevisionRead>,
@@ -651,7 +651,7 @@ impl Core {
// file's boundaries are wrong, and the failure is not confined to short // file's boundaries are wrong, and the failure is not confined to short
// reads: sizes that *overstate* a block shift every block after it and // reads: sizes that *overstate* a block shift every block after it and
// return full-length reads of the wrong bytes. So a disagreement of any // return full-length reads of the wrong bytes. So a disagreement of any
// kind takes the file off the range path entirely (bugs.md B84). // kind takes the file off the range path entirely (`docs/BUGS.md` B84).
let bytes = if reader.size() != fsize { let bytes = if reader.size() != fsize {
warn!( warn!(
%uid, bidx = span.idx, fsize, %uid, bidx = span.idx, fsize,
@@ -664,7 +664,7 @@ impl Core {
// Learned here rather than on a path of its own: this is where a // Learned here rather than on a path of its own: this is where a
// reader is open anyway, and recording the real geometry is what // reader is open anyway, and recording the real geometry is what
// lets the *next* read plan its blocks on the revision's own // lets the *next* read plan its blocks on the revision's own
// boundaries instead of the 4 MiB fallback (bugs.md B85). // boundaries instead of the 4 MiB fallback (`docs/BUGS.md` B85).
if cache_blocks { if cache_blocks {
self.cache self.cache
.store_block_geometry(uid, mtime, fsize, reader.block_sizes()); .store_block_geometry(uid, mtime, fsize, reader.block_sizes());
@@ -706,7 +706,7 @@ impl Core {
/// The range path derives every block boundary from the revision's recorded /// The range path derives every block boundary from the revision's recorded
/// block sizes. When those understate the file, reads come back short with /// block sizes. When those understate the file, reads come back short with
/// no error anywhere, and the caller hands userspace a truncated file that /// no error anywhere, and the caller hands userspace a truncated file that
/// looks like a clean EOF (bugs.md B84). `download_file_to` does not do that /// looks like a clean EOF (`docs/BUGS.md` B84). `download_file_to` does not do that
/// arithmetic — it streams the block list — so it returns the whole file /// arithmetic — it streams the block list — so it returns the whole file
/// where the range path cannot, which is why pinning was the workaround. /// where the range path cannot, which is why pinning was the workaround.
/// This does the same thing on the user's behalf, once, and the resulting /// This does the same thing on the user's behalf, once, and the resulting
@@ -918,7 +918,7 @@ impl Core {
let end = offset.saturating_add(len).min(fsize); let end = offset.saturating_add(len).min(fsize);
let mut out = Vec::with_capacity((end - offset) as usize); let mut out = Vec::with_capacity((end - offset) as usize);
// On the revision's own boundaries where they are known, and on the // On the revision's own boundaries where they are known, and on the
// 4 MiB fallback where they are not (bugs.md B85). // 4 MiB fallback where they are not (`docs/BUGS.md` B85).
let wanted = self.block_geometry(uid, mtime, fsize).spans(offset, end); let wanted = self.block_geometry(uid, mtime, fsize).spans(offset, end);
let (Some(&head), Some(&tail)) = (wanted.first(), wanted.last()) else { let (Some(&head), Some(&tail)) = (wanted.first(), wanted.last()) else {
return Ok(out); return Ok(out);
@@ -1038,7 +1038,7 @@ impl Core {
// — the cache validates it on read, the network path in `read_block` // — the cache validates it on read, the network path in `read_block`
// — so this refuses rather than quietly contributing fewer bytes // — so this refuses rather than quietly contributing fewer bytes
// than the range covers, which is what turned a bad block into a // than the range covers, which is what turned a bad block into a
// truncated file (bugs.md B84). // truncated file (`docs/BUGS.md` B84).
if block.len() as u64 != span.len { if block.len() as u64 != span.len {
error!( error!(
%uid, bidx = span.idx, len = block.len(), expected = span.len, fsize, %uid, bidx = span.idx, len = block.len(), expected = span.len, fsize,
+1 -1
View File
@@ -6,7 +6,7 @@
//! invisible; in an in-process remount — which is how the daemon recovers from a //! invisible; in an in-process remount — which is how the daemon recovers from a
//! mount going away underneath it — it leaks a whole generation of threads that //! mount going away underneath it — it leaks a whole generation of threads that
//! keep writing to the database and calling the API on behalf of a mount that no //! keep writing to the database and calling the API on behalf of a mount that no
//! longer exists (bugs.md B44). //! longer exists (`docs/BUGS.md` B44).
//! //!
//! The primitive is deliberately small: a flag plus a condvar. Every loop that //! The primitive is deliberately small: a flag plus a condvar. Every loop that
//! would otherwise call `thread::sleep` calls [`Shutdown::sleep`] instead, which //! would otherwise call `thread::sleep` calls [`Shutdown::sleep`] instead, which
+4 -4
View File
@@ -183,7 +183,7 @@ impl WriteHandle {
} }
} }
/// A released write whose upload has not happened yet (offline.md Phase 3). /// A released write whose upload has not happened yet.
/// ///
/// The bytes live in the content cache's staging dir and the intent lives in the /// The bytes live in the content cache's staging dir and the intent lives in the
/// `pending_op` table; this pairs them in memory so a read can be served without /// `pending_op` table; this pairs them in memory so a read can be served without
@@ -253,7 +253,7 @@ pub(crate) struct State {
/// two pages shifted every later entry, and the caller silently skipped or /// two pages shifted every later entry, and the caller silently skipped or
/// repeated whatever crossed the page boundary. Each entry is /// repeated whatever crossed the page boundary. Each entry is
/// `(ino, is_dir, name)` — the kernel type is mapped at reply time so this /// `(ino, is_dir, name)` — the kernel type is mapped at reply time so this
/// module stays free of `fuser` (bugs.md B43). /// module stays free of `fuser` (`docs/BUGS.md` B43).
pub(crate) dir_snapshots: HashMap<u64, Arc<DirListing>>, pub(crate) dir_snapshots: HashMap<u64, Arc<DirListing>>,
/// Resident inode attributes whose effective access changed during an /// Resident inode attributes whose effective access changed during an
/// intern/root refresh. Core drains this set and notifies the matching /// intern/root refresh. Core drains this set and notifies the matching
@@ -1198,7 +1198,7 @@ impl State {
/// re-enumerate, which is the cheaper way to stay honest about what the /// re-enumerate, which is the cheaper way to stay honest about what the
/// server did. A queued rename cannot: re-enumerating needs the network, and /// server did. A queued rename cannot: re-enumerating needs the network, and
/// the server has not been told yet in any case — so this *is* the tree's /// the server has not been told yet in any case — so this *is* the tree's
/// new truth until the op drains (offline.md Phase 3b). /// new truth until the op drains.
pub(crate) fn rename_in_place( pub(crate) fn rename_in_place(
&mut self, &mut self,
ino: u64, ino: u64,
@@ -1317,7 +1317,7 @@ impl State {
/// created offline, nothing at all), so this row is the only record that the /// created offline, nothing at all), so this row is the only record that the
/// file is as long as the caller was told it is. Without it a restart serves /// file is as long as the caller was told it is. Without it a restart serves
/// the stale size and the file reads as truncated — or empty — while its /// the stale size and the file reads as truncated — or empty — while its
/// bytes sit safely in staging (offline.md Phase 3). /// bytes sit safely in staging.
/// Returns the node to persist rather than queueing it in the outbox, and is /// Returns the node to persist rather than queueing it in the outbox, and is
/// the one mutation here whose write-through the caller must do itself: /// the one mutation here whose write-through the caller must do itself:
/// failure has to be reported, not logged. Dropping the row silently means /// failure has to be reported, not logged. Dropping the row silently means
+1 -1
View File
@@ -68,7 +68,7 @@ impl Core {
/// docs for the identical/divergent policy. /// docs for the identical/divergent policy.
pub(crate) fn run_conflict_sweep_loop(&self) { pub(crate) fn run_conflict_sweep_loop(&self) {
// Both waits are interruptible, so teardown does not have to sit out a // Both waits are interruptible, so teardown does not have to sit out a
// five-minute interval to join this thread (bugs.md B44). // five-minute interval to join this thread (`docs/BUGS.md` B44).
if !self.shutdown.sleep(CONFLICT_SWEEP_WARMUP) { if !self.shutdown.sleep(CONFLICT_SWEEP_WARMUP) {
return; return;
} }
+10 -10
View File
@@ -1,4 +1,4 @@
//! Two-way folder sync engine (devices.md Phase 2). //! Two-way folder sync engine.
//! //!
//! Each `mirror`-mode synced folder is kept mirrored between a local directory //! Each `mirror`-mode synced folder is kept mirrored between a local directory
//! and a folder under this machine's Proton Drive device. A local filesystem //! and a folder under this machine's Proton Drive device. A local filesystem
@@ -1342,7 +1342,7 @@ impl Core {
/// rename below is the destructive step, and the download in between can run /// rename below is the destructive step, and the download in between can run
/// for minutes, so the destination is re-examined right before it: a local /// for minutes, so the destination is re-examined right before it: a local
/// file that no longer matches the plan is set aside as a conflict copy /// file that no longer matches the plan is set aside as a conflict copy
/// instead of being replaced (bugs.md B39). /// instead of being replaced (`docs/BUGS.md` B39).
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
async fn download_file( async fn download_file(
&self, &self,
@@ -1441,7 +1441,7 @@ impl Core {
/// the pre-upload identity is what makes the next pass see the path as /// the pre-upload identity is what makes the next pass see the path as
/// locally changed and upload a whole, clean revision over it. Recording the /// locally changed and upload a whole, clean revision over it. Recording the
/// post-upload stat instead declared the torn revision synchronized and left /// post-upload stat instead declared the torn revision synchronized and left
/// it as the file's content on Drive (bugs.md B40). /// it as the file's content on Drive (`docs/BUGS.md` B40).
async fn record_file_baseline( async fn record_file_baseline(
&self, &self,
folder_id: i64, folder_id: i64,
@@ -1505,7 +1505,7 @@ fn open_to_upload(path: &Path, rel: &str) -> Result<Option<std::fs::File>, Strin
/// records the identity the upload *read* rather than the one on disk now, so /// records the identity the upload *read* rather than the one on disk now, so
/// the next pass classifies the path as locally changed and replaces the torn /// the next pass classifies the path as locally changed and replaces the torn
/// revision. Without the log there is nothing in the journal to explain the /// revision. Without the log there is nothing in the journal to explain the
/// extra revision (bugs.md B40). /// extra revision (`docs/BUGS.md` B40).
fn warn_if_torn(rel: &str, path: &Path, streamed: LocalSig) { fn warn_if_torn(rel: &str, path: &Path, streamed: LocalSig) {
let now = local_sig(path); let now = local_sig(path);
if !now.is_some_and(|now| now.same_content(&streamed)) { if !now.is_some_and(|now| now.same_content(&streamed)) {
@@ -1537,7 +1537,7 @@ fn local_sig(path: &Path) -> Option<LocalSig> {
/// transfer started, and a download of a large file can run for minutes. Without /// transfer started, and a download of a large file can run for minutes. Without
/// this, a local edit completed inside that window was replaced by the remote /// this, a local edit completed inside that window was replaced by the remote
/// version with no copy of it kept anywhere — the one shape of loss a sync /// version with no copy of it kept anywhere — the one shape of loss a sync
/// engine must not have (bugs.md B39). Keeping the copy makes it re-upload as a /// engine must not have (`docs/BUGS.md` B39). Keeping the copy makes it re-upload as a
/// new file on the next pass, which is the same resolution a both-sides-changed /// new file on the next pass, which is the same resolution a both-sides-changed
/// conflict already gets. /// conflict already gets.
/// ///
@@ -1573,7 +1573,7 @@ fn keep_racing_local_edit(path: &Path, expected: Option<LocalSig>) -> std::io::R
/// exists, so the same removal was re-attempted and re-reported as an error on /// exists, so the same removal was re-attempted and re-reported as an error on
/// every pass, forever. Every other error is still an error, and keeping the /// every pass, forever. Every other error is still an error, and keeping the
/// baseline is exactly right there: the surviving file must not be mistaken for /// baseline is exactly right there: the surviving file must not be mistaken for
/// a new local creation on the next pass and uploaded back (bugs.md B42). /// a new local creation on the next pass and uploaded back (`docs/BUGS.md` B42).
fn removed_locally(result: std::io::Result<()>) -> std::io::Result<()> { fn removed_locally(result: std::io::Result<()>) -> std::io::Result<()> {
match result { match result {
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()), Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
@@ -1830,7 +1830,7 @@ fn system_mtime(meta: &std::fs::Metadata) -> i64 {
/// ///
/// Whole seconds cannot distinguish an edit made in the same second as the last /// Whole seconds cannot distinguish an edit made in the same second as the last
/// sync from no edit at all, and if the size did not change either, that file /// sync from no edit at all, and if the size did not change either, that file
/// was never uploaded (bugs.md B25). `None` for a clock or filesystem that /// was never uploaded (`docs/BUGS.md` B25). `None` for a clock or filesystem that
/// cannot answer, which falls the comparison back to seconds rather than /// cannot answer, which falls the comparison back to seconds rather than
/// asserting an mtime of zero. /// asserting an mtime of zero.
fn system_mtime_ns(meta: &std::fs::Metadata) -> Option<i64> { fn system_mtime_ns(meta: &std::fs::Metadata) -> Option<i64> {
@@ -1845,7 +1845,7 @@ mod tests {
/// A removal of something already gone has produced the state the sync /// A removal of something already gone has produced the state the sync
/// engine wanted, so its baseline is dropped rather than retried forever. /// engine wanted, so its baseline is dropped rather than retried forever.
/// Anything else stays an error and keeps the baseline (bugs.md B42). /// Anything else stays an error and keeps the baseline (`docs/BUGS.md` B42).
#[test] #[test]
fn an_already_absent_path_counts_as_removed() { fn an_already_absent_path_counts_as_removed() {
let root = std::env::temp_dir().join(format!("pdfs-b42-{}", std::process::id())); let root = std::env::temp_dir().join(format!("pdfs-b42-{}", std::process::id()));
@@ -1862,7 +1862,7 @@ mod tests {
} }
/// The download guard: a destination that still matches the plan is /// The download guard: a destination that still matches the plan is
/// replaced silently, one that has moved on since is kept (bugs.md B39). /// replaced silently, one that has moved on since is kept (`docs/BUGS.md` B39).
#[test] #[test]
fn a_download_keeps_a_local_edit_it_did_not_plan_for() { fn a_download_keeps_a_local_edit_it_did_not_plan_for() {
let root = std::env::temp_dir().join(format!("pdfs-b39-{}", std::process::id())); let root = std::env::temp_dir().join(format!("pdfs-b39-{}", std::process::id()));
@@ -2177,7 +2177,7 @@ mod tests {
std::fs::remove_dir_all(&root).unwrap(); std::fs::remove_dir_all(&root).unwrap();
} }
/// A restored folder (features.md 5.2) starts with an empty baseline against /// A restored folder starts with an empty baseline against
/// an empty local directory and a full remote. That must reconcile as /// an empty local directory and a full remote. That must reconcile as
/// "download everything" rather than tripping the wipe guard — otherwise /// "download everything" rather than tripping the wipe guard — otherwise
/// every restore would wedge its folder on the first pass. /// every restore would wedge its folder on the first pass.
+1 -1
View File
@@ -33,7 +33,7 @@ pub(crate) enum SyncMsg {
/// Re-read the folder set and adjust the filesystem watches (after add/remove). /// Re-read the folder set and adjust the filesystem watches (after add/remove).
Rewatch, Rewatch,
/// The daemon is tearing down: finish the current pass and end the engine /// The daemon is tearing down: finish the current pass and end the engine
/// thread, so the mount can join it (bugs.md B44). /// thread, so the mount can join it (`docs/BUGS.md` B44).
Stop, Stop,
} }
+2 -2
View File
@@ -20,7 +20,7 @@ impl LocalSig {
/// Nanoseconds decide it when both sides have them; otherwise the comparison /// Nanoseconds decide it when both sides have them; otherwise the comparison
/// falls back to whole seconds, which is what a baseline row written before /// falls back to whole seconds, which is what a baseline row written before
/// schema 28 — or a filesystem that reports no sub-second time — can support /// schema 28 — or a filesystem that reports no sub-second time — can support
/// (bugs.md B25). /// (`docs/BUGS.md` B25).
pub(super) fn same_content(&self, other: &Self) -> bool { pub(super) fn same_content(&self, other: &Self) -> bool {
if self.size != other.size { if self.size != other.size {
return false; return false;
@@ -161,7 +161,7 @@ pub(super) enum Pending {
/// was there. A download can run for minutes, so the apply step revalidates /// was there. A download can run for minutes, so the apply step revalidates
/// against it immediately before it renames over the destination: a writer /// against it immediately before it renames over the destination: a writer
/// that finished an edit in the meantime must not be overwritten without a /// that finished an edit in the meantime must not be overwritten without a
/// conflict copy (bugs.md B39). /// conflict copy (`docs/BUGS.md` B39).
Download { Download {
rel: String, rel: String,
uid: NodeUid, uid: NodeUid,
+2 -2
View File
@@ -218,7 +218,7 @@ pub(super) fn plan_file(
} }
let local_changed = local.is_some_and(|item| { let local_changed = local.is_some_and(|item| {
// Nanoseconds where both sides have them (bugs.md B25): whole seconds // Nanoseconds where both sides have them (`docs/BUGS.md` B25): whole seconds
// could not see an edit made in the same second as the last sync that // could not see an edit made in the same second as the last sync that
// left the file the same length, so it was never uploaded. // left the file the same length, so it was never uploaded.
baseline.is_none_or(|base| !LocalSig::from(item).same_content(&LocalSig::from(base))) baseline.is_none_or(|base| !LocalSig::from(item).same_content(&LocalSig::from(base)))
@@ -291,7 +291,7 @@ mod file_plan_tests {
/// Whole seconds cannot see an edit made inside the same second that leaves the /// Whole seconds cannot see an edit made inside the same second that leaves the
/// file the same length, so a mirror silently stopped uploading it. With a /// file the same length, so a mirror silently stopped uploading it. With a
/// nanosecond time on both sides it is an ordinary local change (bugs.md B25). /// nanosecond time on both sides it is an ordinary local change (`docs/BUGS.md` B25).
#[test] #[test]
fn a_same_second_same_size_edit_is_still_a_local_change() { fn a_same_second_same_size_edit_is_still_a_local_change() {
let mut base = baseline(10, 20, 30, 40); let mut base = baseline(10, 20, 30, 40);
+1 -1
View File
@@ -7,7 +7,7 @@
//! `~/ProtonDrive` mount, plus each folder this computer backs up, whether it is //! `~/ProtonDrive` mount, plus each folder this computer backs up, whether it is
//! a mirrored local directory or an on-demand FUSE session. The page is called //! a mirrored local directory or an on-demand FUSE session. The page is called
//! *Locations* rather than *Mounts* because a mirror folder is a plain directory //! *Locations* rather than *Mounts* because a mirror folder is a plain directory
//! with no FUSE session behind it (mount-architecture.md §4). //! with no FUSE session behind it.
//! //!
//! The device rows drive the same control requests the Computers page used to: //! The device rows drive the same control requests the Computers page used to:
//! mode switch, sync now, remove. What is new here is that the primary mount is //! mode switch, sync now, remove. What is new here is that the primary mount is
@@ -133,7 +133,7 @@ impl Drop for Busy {
/// from Shared or Shared-with-me may not: it can live under a device folder's /// from Shared or Shared-with-me may not: it can live under a device folder's
/// own inode space, or on someone else's volume, neither of which the primary /// own inode space, or on someone else's volume, neither of which the primary
/// mount's path space can name. Those carry a uid instead, and the daemon /// mount's path space can name. Those carry a uid instead, and the daemon
/// resolves it with `Core::resolve_anywhere` (mount-architecture.md P4). /// resolves it with `Core::resolve_anywhere`.
/// ///
/// The two are separate request variants rather than one request with an /// The two are separate request variants rather than one request with an
/// optional field on purpose: an older daemon that ignored `uid` would resolve /// optional field on purpose: an older daemon that ignored `uid` would resolve