mirror of
https://github.com/narrrl/proton-drive-linux.git
synced 2026-11-04 00:47:00 +01:00
fix(drive) locks + 1.7.1
This commit is contained in:
1 parent
433e206203
commit
d03d3eac3f
13 files changed
+701
-51
No files matched your search
@@ -153,8 +153,9 @@ Because Proton Drive does not support partial byte writes, modified files must b
|
||||
2. **Close/Release (`queue_revision`):** When the application closes the file descriptor, the daemon:
|
||||
- Fetches any untouched gaps from the remote base file to compile the full file.
|
||||
- Durably publishes the scratch data and authored-range sidecar into `staging` under a `{uid}-{millis}-{counter}` name. Temporary data and metadata are synced before atomic rename, and their directory is synced before the source is removed.
|
||||
- Transactionally queues or supersedes a pending database operation (`PendingOp`), so insertion failure cannot erase the previously acknowledged upload. For `OP_REVISION` ops, execution is debounced by 2 seconds (`DRAIN_REVISION_DEBOUNCE = 2s`) to give rapid follow-up writes (e.g. `aria2c` preallocation followed by writing) time to supersede the staged blob before network transmission.
|
||||
3. **Async Drain Thread (`run_pending_drain`):** The background drain worker picks up the database operations queue, handles revision uploads, resolves conflicts, and cleans up staging files. Upon landing a revision upload (`refresh_after_upload`), it rebaselines both still-queued ops (`rebaseline_pending`) and open write handles targeting the same node to prevent false self-conflict copies on subsequent writes.
|
||||
- Transactionally queues or supersedes a pending database operation (`PendingOp`), so insertion failure cannot erase the previously acknowledged upload. For `OP_REVISION` ops, execution is debounced to give rapid follow-up writes (e.g. `aria2c` preallocation followed by writing) time to supersede the staged blob before network transmission. The debounce is adaptive (`Core::revision_debounce`): it starts at `DRAIN_REVISION_DEBOUNCE = 2s` and, once a node has been uploaded, widens toward how long that upload actually took, bounded by `DRAIN_REVISION_DEBOUNCE_MAX = 60s`. A file saved faster than it can be sent therefore supersedes in the queue rather than mid-upload.
|
||||
- Signals `Core::cancel_upload` for the node before touching the queue. A drain worker may already be reading the blob this write supersedes; the flag is read by the upload's `CountingReader`, which refuses the SDK's next block. `queue_trash` and `discard_queued_ops` do the same, because they unlink the blob outright.
|
||||
3. **Async Drain Threads (`run_pending_drain`):** `DRAIN_WORKERS = 3` background workers share the operations queue. They pick work up through `Db::claim_next_due_op`, which marks the row `claimed_at` in the same transaction that selects it and excludes any op whose uid another worker already holds — ordering only has to hold *per node*, and that exclusion is what guarantees it. Worker 0 additionally runs the queue's idle chores (LRU touch flush, `recover_fsynced_writes`). A claim is process-local state: the single-writer `flock` means one found at open belongs to a crashed run, and `Db::clear_op_claims` drops the lot so those ops are not invisible forever. Each worker handles revision uploads, resolves conflicts, and cleans up staging files. Upon landing a revision upload (`refresh_after_upload`), it rebaselines both still-queued ops (`rebaseline_pending`) and open write handles targeting the same node to prevent false self-conflict copies on subsequent writes.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
|
||||
@@ -9,6 +9,26 @@ release ships. Migrations are forward-only — a database written by a newer cli
|
||||
refuse-to-open, not a downgrade, so rolling back a release means restoring the cache from
|
||||
scratch (user data in `staging/` and `recovery/` is never touched by this).
|
||||
|
||||
## [1.7.1] — 2026-08-16
|
||||
|
||||
Write-back throughput follow-up to 1.7.0, same audit
|
||||
(`docs/plans/audit-2026-08-15-perf-locks.md`). Schema: 26 — migration V26 adds
|
||||
`pending_op.claimed_at`.
|
||||
|
||||
### Added
|
||||
- Parallel write-back drain: three workers share the queue through a `pending_op.claimed_at`
|
||||
claim column (migration V26), so a large upload no longer holds every queued rename, trash
|
||||
and small write behind it. Ordering is preserved per node — the claim query never offers
|
||||
two workers ops for the same uid — and claims left by a crashed run are cleared at open.
|
||||
- An upload already on the wire is cancelled when a newer write, a trash or a discard
|
||||
supersedes it, instead of spending the uplink on a revision the next operation replaces.
|
||||
|
||||
### Changed
|
||||
- The revision debounce is adaptive: it widens toward how long that node's last upload
|
||||
actually took (bounded to 2–60 s), so a file saved faster than it can be sent supersedes
|
||||
in the queue rather than mid-upload.
|
||||
- Workspace deps bumped to `proton-sdk` / `proton-drive-rs` 0.6.0.
|
||||
|
||||
## [1.7.0] — 2026-08-16
|
||||
|
||||
Performance and robustness pass over the mount, from the architecture audit in
|
||||
@@ -232,6 +252,7 @@ First stable release: FUSE files-on-demand mount, sync daemon under `proton-driv
|
||||
- The outstanding FUSE defects tracked in `docs/BUGS.md`, plus a truncate defect, validated
|
||||
by a new POSIX compliance suite for the filesystem.
|
||||
|
||||
[1.7.1]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.7.1
|
||||
[1.7.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.7.0
|
||||
[1.6.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.6.0
|
||||
[1.5.0]: https://github.com/narrrl/proton-drive-linux/releases/tag/v1.5.0
|
||||
|
||||
Reference in new issue
Block a user