add cache inspect/vacuum and a diagnose self-check

A corrupt database or an oversized cache previously left the user with no
answer but "delete the state directory and re-sync", which for a large
account means re-downloading everything to fix a problem that may have been
a few stale rows.

pdfs diagnose reports on paths, keyring, daemon, and mountpoint, and exits
non-zero if any check fails so it works in a health check. Every check is
written to run with no daemon, because that is the situation it gets run in:
a stopped daemon is reported as a finding rather than aborting the report.
Writability is tested by writing, since permission bits say nothing about a
full disk or a read-only remount.

pdfs cache inspect reports database size, reclaimable space, per-table row
counts, and cache usage; --deep adds SQLite's integrity check, opt-in because
it reads every page. The report distinguishes "checked and clean" from "not
checked" rather than letting an empty problem list imply a clean bill of
health it never gave.

pdfs cache vacuum checkpoints the WAL and compacts the database.

Deliberately no orphaned-cache-row pruning: ContentCache::reconcile already
rebuilds the index from what is on disk at every open, so a second pass here
would duplicate it against a worse view of the truth. The WAL is likewise
left to journal_size_limit, which already caps it -- a periodic checkpoint
would paper over a problem that no longer exists.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFzNQyb4TWrCRWcnegqWxY
This commit is contained in:
nvrlandClaude Opus 4.8 committed 2026-07-20 19:54:42 +02:00
1 parent 3ade754ea2
commit 471f57b8e1
6 files changed
+621

No files matched your search

+38
View File
@@ -54,6 +54,44 @@ already synced, its copy on Drive is left untouched — the file simply stops
being tracked. Removing the rule later picks the existing remote file back up
rather than re-uploading it.
## Diagnostics & Maintenance
When something looks wrong, `pdfs diagnose` checks the installation and prints a
report. It runs without a daemon on purpose — the state worth diagnosing is
usually the state where the daemon will not start:
```console
$ pdfs diagnose
Paths
[ok ] state dir: /home/you/.local/state/proton-drive-linux
[ok ] database: /home/you/.local/state/proton-drive-linux/cache.db (170.3 MiB)
Account
[ok ] keyring session: you@proton.me
Daemon
[ok ] daemon responding
[ok ] mounted at: /home/you/ProtonDrive
[ok ] queued writes: none
No problems found.
```
It exits non-zero if any check fails, so it works in a health-check script.
For the local metadata database and content cache:
| Command | What it does |
|---|---|
| `pdfs cache inspect` | Database size, reclaimable space, per-table row counts, cache usage against budget |
| `pdfs cache inspect --deep` | Also runs SQLite's integrity check — reads every page, so it is slow on a large database |
| `pdfs cache vacuum` | Checkpoints the write-ahead log and compacts the database |
| `pdfs cache clear` | Deletes cached file content, keeping pinned files |
`vacuum` takes a write lock for its duration and needs room for a second copy of
the database while it runs, so it is a deliberate operation rather than
something the daemon does on a timer.
## Performance & Caching
The client includes several optimizations designed for high efficiency, a low memory footprint, and a responsive user experience:
+241
View File
@@ -235,6 +235,34 @@ enum Command {
#[arg(short, long, default_value_t = 50)]
limit: usize,
},
/// Inspect and maintain the local metadata database and content cache.
Cache {
#[command(subcommand)]
action: CacheCmd,
},
/// Check this installation for problems and print a report.
///
/// Runs without a daemon on purpose: the state worth diagnosing is usually
/// the state where the daemon will not start.
Diagnose,
}
#[derive(Subcommand)]
enum CacheCmd {
/// Report database size, row counts, and cache usage.
Inspect {
/// Also run SQLite's integrity check. Reads every page, so it is slow
/// on a large database — worth it when you suspect corruption.
#[arg(long)]
deep: bool,
},
/// Checkpoint the write-ahead log and compact the database.
///
/// Takes a write lock for the duration and needs room for a second copy of
/// the database while it runs.
Vacuum,
/// Delete cached file content, keeping pinned files.
Clear,
}
/// Which collection a share entry lives in, for `share-role` / `unshare`.
@@ -426,6 +454,8 @@ fn main() -> Result<()> {
Command::Invitations { action } => cmd_invitations(action),
Command::Bookmarks { action } => cmd_bookmarks(action),
Command::Activity { limit } => cmd_activity(limit),
Command::Cache { action } => cmd_cache(action),
Command::Diagnose => cmd_diagnose(),
}
}
@@ -1310,6 +1340,217 @@ fn path_arg(path: &Path) -> Result<String> {
.ok_or_else(|| anyhow!("path is not valid UTF-8"))
}
/// Format a byte count for a report line.
fn human_bytes(bytes: u64) -> String {
const UNITS: [(&str, f64); 4] = [
("GiB", 1_073_741_824.0),
("MiB", 1_048_576.0),
("KiB", 1024.0),
("B", 1.0),
];
let b = bytes as f64;
for (unit, scale) in UNITS {
if b >= scale {
return format!("{:.1} {unit}", b / scale);
}
}
"0 B".to_string()
}
fn cmd_cache(action: CacheCmd) -> Result<()> {
match action {
CacheCmd::Inspect { deep } => cmd_cache_inspect(deep),
CacheCmd::Vacuum => match control_request(CtlRequest::CacheVacuum)? {
CtlResponse::Ok { message } => {
println!("{message}");
Ok(())
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
},
CacheCmd::Clear => match control_request(CtlRequest::PurgeCache)? {
CtlResponse::Ok { message } => {
println!("{message}");
Ok(())
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
},
}
}
fn cmd_cache_inspect(deep: bool) -> Result<()> {
match control_request(CtlRequest::CacheInspect { deep })? {
CtlResponse::CacheReport {
schema_version,
db_bytes,
db_reclaimable_bytes,
tables,
cache_used,
cache_budget,
integrity_problems,
integrity_checked,
} => {
println!("Database schema v{schema_version}, {}", human_bytes(db_bytes));
if db_reclaimable_bytes > 0 {
println!(
" {} reclaimable — run `pdfs cache vacuum`",
human_bytes(db_reclaimable_bytes)
);
}
let budget = if cache_budget == 0 {
"unlimited".to_string()
} else {
human_bytes(cache_budget)
};
println!("Content {} cached of {budget}", human_bytes(cache_used));
println!("\nRows");
for (name, count) in tables {
println!(" {name:<16} {count:>10}");
}
if integrity_checked {
if integrity_problems.is_empty() {
println!("\nIntegrity ok");
} else {
println!("\nIntegrity {} problem(s):", integrity_problems.len());
for problem in integrity_problems {
println!(" {problem}");
}
}
} else {
println!("\nIntegrity not checked (pass --deep)");
}
Ok(())
}
CtlResponse::Error { message, kind } => bail!("{}", cli_error(kind, &message)),
other => bail!("unexpected response: {other:?}"),
}
}
/// One line of the `diagnose` report.
fn check(label: &str, ok: bool, detail: impl AsRef<str>) -> bool {
let mark = if ok { "ok " } else { "FAIL" };
let detail = detail.as_ref();
if detail.is_empty() {
println!("[{mark}] {label}");
} else {
println!("[{mark}] {label}: {detail}");
}
ok
}
/// Self-check the installation, reporting each finding and exiting non-zero if
/// any check failed.
///
/// Every check is written to work with no daemon running, because that is the
/// situation a user runs this in. A missing daemon is reported as a finding,
/// not an error that aborts the rest of the report.
fn cmd_diagnose() -> Result<()> {
let mut all_ok = true;
let dirs = AppDirs::new().context("resolve application directories")?;
let config = dirs.load_config();
println!("Paths");
let state = dirs.state_dir();
all_ok &= check(
" state dir",
state.is_dir(),
format!("{}", state.display()),
);
let cache_dir = dirs.cache_dir();
all_ok &= check(
" cache dir",
cache_dir.is_dir(),
format!("{}", cache_dir.display()),
);
// Writability is checked by actually writing: permission bits do not
// account for a full disk, a read-only remount, or an immutable flag.
let probe = state.join(".pdfs-diagnose-probe");
let writable = std::fs::write(&probe, b"probe").is_ok();
let _ = std::fs::remove_file(&probe);
all_ok &= check(" state dir writable", writable, "");
let db_path = dirs.db_path();
let db_size = std::fs::metadata(&db_path).map(|m| m.len()).unwrap_or(0);
all_ok &= check(
" database",
db_path.is_file(),
format!("{} ({})", db_path.display(), human_bytes(db_size)),
);
println!("\nAccount");
match pdfs_core::auth::load() {
Ok(session) => {
all_ok &= check(" keyring session", true, session.username);
}
Err(e) => {
// A locked keyring and an absent login look the same from here, so
// the detail carries the distinction rather than the verdict.
all_ok &= check(" keyring session", false, format!("{e}"));
}
}
println!("\nDaemon");
let socket = dirs.control_socket();
let socket_exists = socket.exists();
check(
" control socket",
socket_exists,
format!("{}", socket.display()),
);
// A stale socket file with no listener is the common failure, so the reply
// is what decides this rather than the file's existence.
match control_request(CtlRequest::Status) {
Ok(CtlResponse::Status {
mountpoint,
online,
pending_uploads,
pending_changes,
..
}) => {
check(" daemon responding", true, "");
check(" mounted at", true, mountpoint);
check(" network", online, if online { "" } else { "offline" });
let pending = pending_uploads + pending_changes;
check(
" queued writes",
true,
if pending == 0 {
"none".to_string()
} else {
format!("{pending} waiting to upload")
},
);
}
Ok(other) => {
all_ok &= check(" daemon responding", false, format!("unexpected: {other:?}"));
}
Err(e) => {
// Not a failure of the installation: a stopped daemon is a normal
// state, and the rest of the report is still worth printing.
check(" daemon responding", false, format!("{e:#}"));
println!(" (start it with `systemctl --user start pdfs.service`)");
}
}
let mountpoint = dirs.resolved_mountpoint(&config);
check(
" mountpoint exists",
mountpoint.is_dir(),
format!("{}", mountpoint.display()),
);
if all_ok {
println!("\nNo problems found.");
Ok(())
} else {
bail!("one or more checks failed; see the report above")
}
}
/// Send one request to the running mount daemon's control socket and read its
/// reply. Errors if no daemon is listening.
fn control_request(req: CtlRequest) -> Result<CtlResponse> {
+38
View File
@@ -136,6 +136,22 @@ pub enum Request {
/// persist it to config so the next mount keeps it. Replies with
/// [`Response::Ok`].
SetCacheBudget { bytes: u64 },
/// Report on the health of the metadata database and content cache: sizes,
/// row counts, and — when `deep` — SQLite's own integrity check. Replies
/// with [`Response::CacheReport`].
///
/// `deep` reads every page of the database, so it is opt-in: worth it when
/// diagnosing a suspected corruption, wasteful as a routine status call.
CacheInspect {
#[serde(default)]
deep: bool,
},
/// Checkpoint the write-ahead log and compact the database, replying with
/// [`Response::Ok`] reporting the bytes reclaimed.
///
/// Takes a write lock for the duration and needs room for a second copy of
/// the database, which is why it is user-invoked rather than periodic.
CacheVacuum,
/// Snapshot what the daemon is working on: in-flight transfers (active
/// uploads/downloads) and the longer jobs around them (scans, folder
/// skeletons, the local index, sync passes). Replies with
@@ -817,6 +833,28 @@ pub enum Response {
},
/// A human-readable success message.
Ok { message: String },
/// Health report for the metadata database and content cache (reply to
/// [`Request::CacheInspect`]).
CacheReport {
/// Schema version the database file is at.
schema_version: i64,
/// Bytes the database file accounts for.
db_bytes: u64,
/// Bytes a vacuum could hand back.
db_reclaimable_bytes: u64,
/// Row counts per table, in display order.
tables: Vec<(String, i64)>,
/// Bytes of cached content blobs, blocks, and thumbnails.
cache_used: u64,
/// Configured soft byte cap (`0` = unlimited).
cache_budget: u64,
/// Problems SQLite reported. Empty means sound; absent when the caller
/// did not ask for a deep check.
integrity_problems: Vec<String>,
/// Whether the integrity check actually ran, so an empty problem list
/// is not mistaken for a clean bill of health it never gave.
integrity_checked: bool,
},
/// The pin registry.
Pins { pins: Vec<Pin> },
/// A directory listing (reply to [`Request::ListDir`]) or a trash listing
+263
View File
@@ -0,0 +1,263 @@
//! Database health reporting and repair (features.md 4.3).
//!
//! Everything here exists for the case where something has already gone wrong:
//! a crash mid-write, a full disk, a cache that has grown out of proportion to
//! what it holds. The daemon never calls these on its own — they are what
//! `pdfs cache inspect`, `pdfs cache vacuum`, and `pdfs diagnose` are built on,
//! so a user with a suspect install can get an answer without being told to
//! delete their state directory and re-sync from scratch.
use rusqlite::OptionalExtension;
use super::Db;
use crate::Result;
/// A point-in-time picture of the database's size and contents.
///
/// Sizes are computed from SQLite's own page accounting rather than by stat'ing
/// the file, so they stay right when the file is sparse or a checkpoint is
/// pending.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DbStats {
/// Schema version the file is currently at.
pub schema_version: i64,
/// Total pages allocated to the database.
pub page_count: i64,
/// Pages allocated but unused — what a `VACUUM` would reclaim.
pub free_pages: i64,
/// Bytes per page.
pub page_size: i64,
/// Row counts for the tables worth reporting, in display order.
pub tables: Vec<(String, i64)>,
}
impl DbStats {
/// Bytes the database file accounts for.
pub fn total_bytes(&self) -> u64 {
(self.page_count * self.page_size).max(0) as u64
}
/// Bytes a `VACUUM` could hand back to the filesystem.
pub fn reclaimable_bytes(&self) -> u64 {
(self.free_pages * self.page_size).max(0) as u64
}
}
/// What a maintenance run reclaimed.
///
/// Deliberately does not carry an orphaned-row count: the content cache already
/// rebuilds its index from what is actually on disk every time it opens (see
/// [`crate::cache::ContentCache`]'s `reconcile`), so a second pruning pass here
/// would be a duplicate of that logic with a worse view of the truth.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct VacuumOutcome {
/// Database bytes before the run.
pub before_bytes: u64,
/// Database bytes after it.
pub after_bytes: u64,
/// WAL frames written back into the database by the checkpoint.
pub wal_frames_checkpointed: i64,
}
impl VacuumOutcome {
/// Bytes handed back to the filesystem. Saturating: a database can legitimately
/// grow across a vacuum (a checkpoint folds WAL frames in), and reporting that
/// as a huge positive number via wraparound would be worse than reporting zero.
pub fn freed_bytes(&self) -> u64 {
self.before_bytes.saturating_sub(self.after_bytes)
}
}
/// The tables `inspect` reports on, in the order a reader wants them.
const REPORTED_TABLES: &[&str] = &[
"nodes",
"cache_entries",
"pins",
"photos",
"trash",
"device",
"sync_folder",
"sync_entry",
"pending_op",
"activity",
"local_files",
];
impl Db {
/// SQLite's own consistency check.
///
/// Returns an empty vector when the database is sound; otherwise each entry
/// is one problem SQLite found. Read-only and safe against a live daemon,
/// though it reads every page, so it is not free on a large file.
pub fn integrity_check(&self) -> Result<Vec<String>> {
let conn = self.conn.lock();
let mut stmt = conn.prepare("PRAGMA integrity_check")?;
let rows = stmt.query_map([], |r| r.get::<_, String>(0))?;
let mut problems = Vec::new();
for row in rows {
let row = row?;
// A sound database reports the single line "ok"; anything else is a
// finding.
if row != "ok" {
problems.push(row);
}
}
Ok(problems)
}
/// Size and row-count snapshot for `pdfs cache inspect`.
pub fn stats(&self) -> Result<DbStats> {
let conn = self.conn.lock();
let page_count: i64 = conn.query_row("PRAGMA page_count", [], |r| r.get(0))?;
let free_pages: i64 = conn.query_row("PRAGMA freelist_count", [], |r| r.get(0))?;
let page_size: i64 = conn.query_row("PRAGMA page_size", [], |r| r.get(0))?;
// The schema version lives in the `sync_state` key/value table, not in
// `PRAGMA user_version` — see `migrations`.
let schema_version: i64 = conn
.query_row(
"SELECT value FROM sync_state WHERE key = 'schema_version'",
[],
|r| r.get::<_, String>(0),
)
.optional()?
.and_then(|v| v.parse().ok())
.unwrap_or(0);
let mut tables = Vec::new();
for name in REPORTED_TABLES {
// A table listed here may not exist yet on an older schema, and a
// report is not worth failing over a missing one.
let exists: Option<String> = conn
.query_row(
"SELECT name FROM sqlite_master WHERE type='table' AND name=?1",
[name],
|r| r.get(0),
)
.optional()?;
if exists.is_none() {
continue;
}
let count: i64 = conn.query_row(&format!("SELECT count(*) FROM \"{name}\""), [], |r| {
r.get(0)
})?;
tables.push(((*name).to_string(), count));
}
Ok(DbStats {
schema_version,
page_count,
free_pages,
page_size,
tables,
})
}
/// Fold the WAL back into the database and compact the file.
///
/// Checkpointing first keeps the `VACUUM` from having to contend with a
/// large WAL.
///
/// `VACUUM` rewrites the whole database into a fresh file, so it needs room
/// for a second copy and takes a write lock for the duration. That makes it
/// a deliberate, user-invoked operation rather than something the daemon
/// does on a timer.
pub fn vacuum(&self) -> Result<VacuumOutcome> {
let before = self.stats()?.total_bytes();
let wal_frames_checkpointed = self.checkpoint_wal()?;
{
let conn = self.conn.lock();
conn.execute_batch("VACUUM")?;
}
let after = self.stats()?.total_bytes();
Ok(VacuumOutcome {
before_bytes: before,
after_bytes: after,
wal_frames_checkpointed,
})
}
/// Checkpoint the WAL and truncate it, returning the number of frames
/// written back into the database.
///
/// `TRUNCATE` blocks on readers rather than giving up part-way, which is
/// what makes the result meaningful: a `PASSIVE` checkpoint against a busy
/// daemon can legitimately move nothing and report success.
pub fn checkpoint_wal(&self) -> Result<i64> {
let conn = self.conn.lock();
// Columns are (busy, log frames, frames checkpointed).
let frames: i64 = conn.query_row("PRAGMA wal_checkpoint(TRUNCATE)", [], |r| r.get(2))?;
Ok(frames.max(0))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_fresh_database_passes_its_integrity_check() {
let db = Db::open_in_memory().unwrap();
assert!(db.integrity_check().unwrap().is_empty());
}
#[test]
fn stats_report_the_schema_and_the_known_tables() {
let db = Db::open_in_memory().unwrap();
let stats = db.stats().unwrap();
assert!(stats.schema_version > 0, "migrations set a user_version");
assert!(stats.page_size > 0);
assert!(
stats.tables.iter().any(|(name, _)| name == "nodes"),
"the nodes table should be reported; got {:?}",
stats.tables
);
// A fresh database has no rows in any of them.
assert!(stats.tables.iter().all(|(_, count)| *count == 0));
}
#[test]
fn total_and_reclaimable_bytes_derive_from_the_page_counts() {
let stats = DbStats {
schema_version: 1,
page_count: 10,
free_pages: 4,
page_size: 4096,
tables: Vec::new(),
};
assert_eq!(stats.total_bytes(), 40960);
assert_eq!(stats.reclaimable_bytes(), 16384);
}
/// A vacuum that leaves the file larger (a checkpoint folded WAL frames in)
/// must report nothing freed, not an underflowed enormous number.
#[test]
fn freed_bytes_does_not_underflow_when_the_file_grew() {
let outcome = VacuumOutcome {
before_bytes: 100,
after_bytes: 400,
..VacuumOutcome::default()
};
assert_eq!(outcome.freed_bytes(), 0);
}
#[test]
fn freed_bytes_reports_the_difference_when_the_file_shrank() {
let outcome = VacuumOutcome {
before_bytes: 4096,
after_bytes: 1024,
..VacuumOutcome::default()
};
assert_eq!(outcome.freed_bytes(), 3072);
}
#[test]
fn vacuuming_a_fresh_database_leaves_it_sound() {
let db = Db::open_in_memory().unwrap();
db.vacuum().unwrap();
assert!(db.integrity_check().unwrap().is_empty());
}
}
+2
View File
@@ -25,6 +25,7 @@ mod activity;
mod cache;
mod devices;
mod local;
mod maintenance;
mod migrations;
mod nodes;
mod ops;
@@ -38,6 +39,7 @@ mod utils;
pub use cache::CacheEntryInput;
pub use devices::StoredDevice;
pub use local::LocalFileHit;
pub use maintenance::{DbStats, VacuumOutcome};
pub use nodes::{SearchHit, StoredNode};
pub use ops::{
AttachedBlob, LOCAL_VOLUME, OP_CREATE, OP_MKDIR, OP_RENAME, OP_REVISION, OP_TRASH,
+39
View File
@@ -404,6 +404,45 @@ fn handle_control_conn(core: &Core, username: &str, mountpoint: &Path, stream: U
),
}
}
Ok(CtlRequest::CacheInspect { deep }) => match core.db.stats() {
Ok(stats) => {
// The deep check reads every page, so it runs only on request;
// `integrity_checked` tells the caller which answer it is
// holding, since "no problems found" and "never looked" are
// both an empty list.
let (integrity_problems, integrity_checked) = if deep {
match core.db.integrity_check() {
Ok(problems) => (problems, true),
Err(e) => (vec![format!("integrity check failed: {e:?}")], true),
}
} else {
(Vec::new(), false)
};
CtlResponse::CacheReport {
schema_version: stats.schema_version,
db_bytes: stats.total_bytes(),
db_reclaimable_bytes: stats.reclaimable_bytes(),
tables: stats.tables,
cache_used: core.cache.usage(),
cache_budget: core.cache.budget(),
integrity_problems,
integrity_checked,
}
}
Err(e) => CtlResponse::error(CoreError::internal(format!("read db stats: {e:?}"))),
},
Ok(CtlRequest::CacheVacuum) => match core.db.vacuum() {
Ok(outcome) => CtlResponse::Ok {
message: format!(
"vacuumed: {:.1} MiB freed ({:.1} MiB → {:.1} MiB), {} WAL frame(s) checkpointed",
outcome.freed_bytes() as f64 / 1_048_576.0,
outcome.before_bytes as f64 / 1_048_576.0,
outcome.after_bytes as f64 / 1_048_576.0,
outcome.wal_frames_checkpointed,
),
},
Err(e) => CtlResponse::error(CoreError::internal(format!("vacuum: {e:?}"))),
},
Ok(CtlRequest::GetQueueStatus) => CtlResponse::Transfers {
items: core.transfers.snapshot(),
jobs: core.jobs_snapshot(),