Commit Graph
30 Commits
Author SHA1 Message Date
nvrl 9c93860cd8 refactor(core): rows and ops name a node's parent by local id; a landing rewrites nothing below it 2026-10-04 13:26:19 +02:00
nvrl d5d3b2da3f refactor(core,fuse): find a queued op by its node's local id under either uid 2026-10-04 13:10:19 +02:00
nvrl e5f3f612f8 refactor(core,fuse): a node made here goes by its local id until it lands (schema 41)
A file or folder made on the mount used to get a minted `local~<ms>-<n>`
placeholder. It now gets `local~<lid>`, a function of the row it is given
first (`Db::add_local_row`), so the stand-in never changes and always leads
back to the row, and through it to the real uid once the node lands.
`Db::lid_of` reads it without a lookup.

Migration V41 rewrites the placeholders still stored to their row's
stand-in: the row and its node_json, its children, the ops on it or into
it (uid, parent, sidecar uid, a rename's original parent), and its pins.

Step 6a of the plan for 3.0.0 phase 3.
2026-10-04 12:32:20 +02:00
nvrl 6d75a56b00 fix(fuse,core): adopt a twin holding the create's bytes (B127); trash the node of a withdrawn create that lost its answer (B151)
A retried create now adopts the file its lost answer made when Drive's
SHA-1 matches the staged blob, not only when the file is empty. A delete
after a sent create keeps the op as a trash that finds and trashes the
node that create may have made; a new create of that name waits for it.
The simulation's settle now reads the log, so known damage is explained.
2026-10-04 05:41:57 +02:00
nvrl ae6650f7e0 feat(core,fuse): queued ops name their node and folder by local id
Schema 39 adds pending_op.lid and parent_lid, backfilled from nodes. Every
op queued or retargeted gets both, and a create gets its row before it
is queued. The claim reads a parent's uid through parent_lid, and a
landing folder rewrites the ops made inside it by local id instead of by
its placeholder string. Ops queued against a row Drive listed follow it
when it is merged into the landing one.
2026-10-04 04:45:47 +02:00
nvrl a6f36d56f3 feat(core,fuse,cli,gui): report changes Drive refuses as sync issues
A change Drive refuses because the storage is full, access was taken away,
its folder is gone, a folder limit is reached or the change was rejected
used to look like any other failure until six attempts had passed. The
drain now sorts the refusal into an issue and stores it on the queued op
(schema 38), so it needs attention at once in the app, tray and CLI. The
change stays queued and is retried; nothing is deleted.

The queue listing carries the issue and whether the op has file content.
ExportPendingOp copies that content to a new file: the Sync page offers it
as an Export button next to each issue, and the CLI as
`pdfs sync export <id> <dest>`. `pdfs sync issues` lists what needs
attention, and `pdfs sync queue` shows the issue per change.

docs/MILESTONE-3.0.0-STATUS.md tracks what is left for 3.0.0.
2026-10-04 03:18:57 +02:00
nvrl 5e0f30cc2c feat(fuse,core): record namespace changes locally and send them from the queue
A new folder, a new file, a rename or a delete through the mount no longer
waits for Drive: it is recorded locally, the call returns at once, and the
drain sends it as it does offline. `"local_first": false` in config.json
brings back the old path for this release.

Fixes found by the simulation runs on the way:
- B129: a file deleted while its first upload is on the wire is trashed once
  the upload lands.
- B132, B140: files made in a new folder survive a restart; a landed folder
  hands its queued children over in the stored node too.
- B138: a create waits for the rename or trash that frees its name.
- B139: a file closed while its first upload lands keeps its last write.
- B141: a rename a worker holds stays queued, and the newer one waits behind
  it instead of racing it.
- B142: a rename of a file whose create lands meanwhile goes to the uid Drive
  gave it instead of answering EBUSY.
- B143: a create whose read-back fails as the link drops is adopted before
  the next op for it is sent, so its next write is not a conflict copy.
2026-10-04 01:50:38 +02:00
nvrl c34cc93c45 fix(fuse,core): keep a rename made while its create is on the wire
A rename of a node whose create is still queued rewrites the create's
target, but the drain had already copied the op. When the create
landed, the row was deleted unread and the rename was lost (B136).

finish_create now learns where the create was sent and where it landed,
and queues a rename of the real uid when the row has moved on since.
adopt_real_uid keeps the tree's name and folder while that rename is
queued. A landed folder's children also stop naming its placeholder as
their parent in memory, which put the placeholder back on their rows.
2026-10-03 16:20:00 +02:00
nvrl 1c262eb601 refactor(fuse,core): keep queued-op preconditions in the database
Phase 1 of docs/MILESTONE-3.0.0.md, without a change in behaviour.

- Own sealed revisions live only in the own_sealed_rev table. The
  in-memory map in front of it is gone, so revision_conflict gives the
  same answer before and after a restart.
- A queued rename records the node's original name next to its original
  parent, and keeps both across renames that replace the row. The drain
  logs when Drive has the node neither where the rename found it nor
  where it takes it.

The SDK's NodeUpdated event carries no revision id, so echoes of our own
changes stay recognised by the self-change timer.
2026-10-03 14:49:58 +02:00
nvrl 37c04f1e69 fix(fuse,core): fix B114 to B120 from the account runs
- B114: `pdfs rm`, `pdfs rename` and `pdfs move` tell each kernel
  session that the old name is gone, so it stops resolving at once
  instead of for up to 30 seconds (`Core::forget_everywhere`,
  `Core::relocate_everywhere`).
- B115: `pdfs ls` and the app's file browser wait up to 10 seconds for
  the listing's size fetch, as `stat` does, so they show a file's real
  size instead of its encrypted one.
- B116: trashing a folder also drops the creates queued in its
  subfolders. A parked one stayed queued with its bytes, and a due one
  would have landed in the root.
- B117: `pdfs move` and `pdfs rename` move the node in each mount
  instead of forgetting it, so its queued upload keeps its path in
  `pdfs sync queue` and is not put off once after the resume.
- B118: when a create or mkdir reads back nothing, the mount asks again
  and then queues it as it does offline, instead of failing with ENOENT
  for something Drive has made.
- B119: a landed upload keeps its staged bytes as the file's cached
  content, unless it is larger than a sixteenth of the cache, so
  reading a file just written does not need the network.
- B120: a landed folder's row passes to its real uid in the same
  transaction, so a file queued inside it is not put off for five
  seconds.

docs/BUGS.md records B117 to B120. The account run of 2026-10-03
verified B114 to B117 and B119 in My files.
2026-10-03 02:10:23 +02:00
nvrl a181f1a5f0 fix(core,fuse): keep working through network drops
The mount only ever went from offline to online: nothing marked it
offline again, so after the Wi-Fi dropped every create, mkdir, delete and
rename called Drive, waited out the SDK's retries for up to two minutes
and answered EIO. Background loops kept polling the dead link.

Connectivity now lives in pdfs-fuse's new link module and goes both
ways. A call that gets no answer (a transport error or a request
timeout) marks the mount offline; a probe thread checks every 2 s,
backing off to 30 s, and on recovery re-lists folders served from the
database, retries failed operations and wakes the drain. Interactive
calls get a 20 s deadline, so a dead link costs one slow call instead of
many.

While offline:
- create, mkdir, unlink, rmdir and rename fall back to the offline
  queue instead of answering EIO;
- folders list from the database even when not marked listed, and are
  re-listed once the link returns;
- reads of uncached data wait up to 30 s for the link, then retry once
  within the 120 s read deadline;
- statfs serves the last quota instead of fetching, and quota refreshes
  are single-flight;
- the event and photo event loops wait for the link instead of polling.

A create whose request timed out may have landed anyway. When the drain
replays it and finds a same-named twin of the same kind, created within
the last two minutes and still empty, it adopts that node instead of
making a conflict copy. The drain also leaves alone a name still held
by a queued trash.
2026-09-30 03:25:00 +02:00
nvrl a2992b3b9d fix(core,fuse): keep replaced and vanished files from failing saves, reads and syncs
Fixes from the account acceptance runs:

- B104: a reader open across a rename-over keeps working; the replaced
  node is unlinked like any other open file instead of forgotten.
- B105: a drain worker holding a revision withdrawn by an unlink no
  longer uploads it as a recovered-… conflict copy (Db::op_exists).
- B106: an open victim's queued ops are withdrawn at trash time, so a
  repeated editor save no longer fails with AlreadyExists; the staged
  blob goes with the last close.
- B107: a remote read gives up after 120 s with EIO instead of hanging
  its caller; the timer is built inside the runtime.
- B108: a mirror upload whose local file was deleted after the scan is
  skipped rather than counted as an error.

The acceptance script reports the failing line for non-assert errors,
ignores queue ops that predate the run, purges with delete-forever
first, and diagnoses a malformed SQLite database against a local replay.
2026-09-29 16:20:51 +02:00
nvrl 7b67db5ced fix(core,fuse,cli): harden writes, trash and cleanup found by account acceptance runs
The FUSE acceptance suite now runs on the signed-in account by default and
cleans up after itself; its first live runs found three daemon bugs.

- B101: a pwrite past EOF now authors the hole it leaves, so truncated bytes
  no longer come back from the base. A write attached to a create while that
  create uploads becomes a revision of the real uid instead of being dropped.
- B102: removing a folder drops the queued revisions of files below it, found
  through the node tree, with their blobs and in-flight uploads. `pdfs rm`
  now drops queued ops too.
- B103: each server call of a trash refresh times out after 120 s, so one hung
  call can no longer freeze the trash listing until restart.

Acceptance suite: account mode with a manifest, lock and reaper; cleanup by
uid; queued ops naming a test root count as leftovers; new edge-case,
throughput and B100/B101 cases; the wide-directory case is split from the
deep tree and gets a doubled budget. Adds `pdfs trash --json`.
2026-09-29 10:09:44 +02:00
nvrl 067561564e feat(core,fuse,gui,cli): pause/resume sync and a visible upload queue (phase 2)
Backend (docs/UI_UX_PLAN.md B-1, B-2):
- Request::SetSyncPaused { paused, until } pauses the drain worker and the
  mirror reconcile. FUSE reads, write staging, the watcher and the poll keep
  running. The pause is persisted in the `state` table (`sync_paused_until`),
  so it survives a restart; a timed pause ends by itself through a timer
  thread, re-armed on mount. Resuming wakes the drain and runs a full
  reconcile. Response::Status gains `paused` and `paused_until`.
- Request::ListPendingOps lists the queue with paths, attempts, last error,
  next attempt and parked/failing flags. Request::RetryPendingOp { id } makes
  one backed-off op (or every failed one) due now. Parked ops are never
  un-parked and nothing is dropped from the queue.

Front ends:
- Sync page: a status card with Pause/Resume (split button with 1 h, 8 h and
  24 h pauses) and a "Waiting to Upload" list with Retry now and Retry All.
- The sidebar strip and the tray show the paused state; the tray gains Pause
  Sync / Resume Sync and names failing changes.
- CLI: `pdfs sync pause [--for 30m|1h|2d]`, `pdfs sync resume`,
  `pdfs sync queue`, `pdfs sync retry [id]`; `pdfs status` reports a pause.

Not in this phase: merging Activity into the Sync page, discarding a queued
op (needs a recovery copy first), and per-folder pause.
2026-09-23 02:06:41 +02:00
nvrl 860d29cfe8 fix(core,fuse): un-park abandoned creates, retire orphan staged bytes, serialise refreshes
Four faults found by reading the journal for issues that were never fixed,
tracked as B94-B97 in docs/BUGS.md.

A create parked behind a temp-file rename (B70) had no exit but that rename,
so 43 uploads sat in the queue for weeks holding their staged bytes. The
primary drain worker now sweeps parked creates while the queue is idle: one
whose node is gone is discarded, and one older than PARK_EXPIRY_MS that nobody
holds open is un-parked rather than dropped, since staged bytes are the only
copy of the write. An open node stays parked however old it is.

.tmp and .temp leave TRANSIENT_SUFFIXES. The list served two mechanisms with
different consequences: ignoring a name during mirror reconcile is reversible,
parking a create is not, and a Takeout index.tmp is a final name that nothing
will ever rename.

Staged blobs that match no queued op and name no node were kept forever and
warned about once per blob per mount (95 blobs, ~445 MiB, oldest two months).
Blobs past STAGING_ORPHAN_RETAIN are retired; the rest are one aggregated
warning.

Refresh control requests now run one per scope. Every scope is individually
cheap, but duplicates contend for the database lock the running refresh holds,
which left five in flight for twenty-three minutes. request_kind names the
scope so the report says which view wedged. The photo metadata pass also stops
at shutdown instead of running into the cancelled runtime and logging one
"skipping undecryptable photo" per node.
2026-09-19 20:29:48 +02:00
nvrl 098da2e97a fix(log) log unreachable nodes 2026-08-16 21:34:12 +02:00
nvrl d03d3eac3f fix(drive) locks + 1.7.1 2026-08-16 20:41:03 +02:00
nvrl 35418eea3f fix(perf) read-only connection pool, States doesn't write to db, search optimized 2026-08-16 00:33:27 +02:00
nvrl 7ad9231858 fix(fs) locks, vacuum, recover fsynced writes 2026-08-15 19:17:56 +02:00
nvrl 295bbeadac implemented share_access, POSIX read-only mode 2026-07-29 14:57:56 +02:00
nvrl a091c2f34f implemented sweep mode and better conflict handling 2026-07-26 02:06:58 +02:00
nvrlandClaude Opus 5 bca0fe34bc fix(fuse,core): key conflict detection on revision id, stop transient-name seals (B69, B70)
B69 — spurious `(sync-conflict)` copies came from `(mtime, size)` being used as a
revision identity. The server re-stamps a sealed revision with its own commit
time, so the drain diverted a write into a conflict copy of its own base; and
`plan_file` classified a no-baseline (local, remote) pair as `Conflict`, forking
every pre-existing file on first reconcile.

- `Baseline` carries `revision_id` (from the SDK's new `active_revision_id`),
  captured at write-open and preserved across supersede/rebaseline.
  `revision_changed` trusts the id and only falls back to `(mtime, size)` for
  old sidecars.
- Planner gains `FilePlan::AdoptBaseline`: equal-size no-baseline pairs are
  recorded, not conflicted. Divergent sizes stay a real conflict.
- New `sweep.rs` (`pdfs-conflict-sweep`, 5-min cadence) removes conflict copies
  proven identical by size + `content_sha1` and surfaces divergent ones once as
  an `ActivityKind::Conflict` entry. Never removes what it cannot prove.

B70 — browsers downloading into an on-demand mount had their in-flight temps
sealed as canonical revisions, exiling the finished file to a conflict copy.
`syncignore::is_transient_name` now recognises download/scratch temps, and the
on-demand `release`/`rename` paths hold them (`set_create_hold`) instead of
sealing until the name is finalized.

Workspace deps bumped to proton-sdk/proton-drive-rs 0.2.2 (registry, no patch).

Live validation against a real account still pending for both.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V8P42asymU158CPnhjRwri
2026-07-25 15:24:31 +02:00
nvrl 8be0f04b61 fixed fuse bugs from BUGS.md
- Durable scratch/staging publication and preservation on failure.
      - Transactional pending-operation supersession.
      - Conflict copies cannot overwrite older copies or lose the source.
      - Single-entry wipe protection and incomplete-scan aborts.
      - Failed local deletion retains its baseline.
      - Downloads are fsynced before baseline settlement.
      - Future database schemas are refused.
      - Event cursors advance only after successful invalidation and persistence.
      - IPC frame, timeout, and concurrency limits.
      - Atomic config saves; malformed configs are preserved.
      - Sensitive directories require correct ownership, real directories, and 0700.
      - Debian package now includes service and tray autostart files.
      - Release workflow enforces versions, tests, lint, formatting, and offline acceptance.
      - CI now runs offline filesystem acceptance.
2026-07-22 14:49:06 +02:00
nvrl fe75998cea fix(filesystem): resolve critical data loss, POSIX non-compliance, and deadlock vulnerabilities
- Standardize POSIX unlink (EISDIR) and rmdir (ENOTDIR/ENOTEMPTY) checks
- Defer local directory eviction in mode switch until FUSE mount verification succeeds
- Enforce fsync before atomic renames in ContentCache store operations
- Add directory ancestor loop check in rename (EINVAL)
- Implement punch_hole interval tracking in fallocate
- Purge un-created local node write revisions on unlink
- Fix control socket permission race with octal umask 0o77
- Add unit tests for state, DB, and FUSE POSIX logic (208/208 tests passing)
2026-07-22 00:23:55 +02:00
nvrl 9d55ffb9f6 fixed io lock with alloced files from for example aria2c 2026-07-21 23:23:20 +02:00
nvrl 2275004f79 impplemented fallocate 2026-07-21 22:39:35 +02:00
nvrl 190ea3e9cb fixed some conflict bugs + ui fix 2026-07-19 15:36:04 +02:00
nvrl 1d5fdfb6b9 some performance optimizations 2026-07-19 12:48:48 +02:00
nvrl a90cf83ff5 fixed some bugs 2026-07-18 19:24:11 +02:00
nvrl 232b3b5348 db.rs refactor 2026-07-18 14:20:37 +02:00