vault: a lock that lets go, and seals that say what they broke

The idle timeout was checked only when the key was next asked for, so a
vault nobody touched kept its key in memory indefinitely while status
reported it locked. A ticker now drops the key once it is idle, and
status() expires it too. Vault passwords are Zeroizing from the request
body to Argon2.

Re-sealing a file rewrote links with a case-sensitive replace, and only
in documents linked through node_media. Links are now matched by regex,
whatever their case, in every document and card body that names the id.
A folder re-seal carries on past a file that fails and reports how many
failed. Two re-seals of one file no longer both commit.

:share reports how many sealed files the document embeds, since a share
link never serves them, and :encrypt reports how many shared documents
embed what it just sealed. Vault status counts sealed files, so :export!
asks for the password first rather than navigating into a 423 page. A
sealed embed that fails to load while the vault is shut becomes a button
that unlocks it and puts the embed back.

README.md

@@ -209,7 +209,8 @@ and close it, `:passwd` changes the password. A sealed file carries a lock in
 the list, and the bar says `vault open` while it is.
 
 The vault closes itself after `WORKSPACE_VAULT_IDLE_MIN` minutes of nobody
-using it (15 by default), on logout, and whenever the server restarts. While
+using it (15 by default) — the key is dropped from memory then, not merely
+refused on the next look — on logout, and whenever the server restarts. While
 it is closed a sealed file shows as sealed and nothing more; a share link never
 serves one, open or not; and `:export!` refuses until it is open, because an
 archive meant to be read without this application has to hold the files
@@ -222,6 +223,18 @@ id, and its bytes are AES-256-GCM in 64 KiB records, the last one marked as
 last — so a file that is swapped, cut short or tampered with refuses to open.
 Changing the password re-seals the master key and rewrites no file.
 
+A sealed file is served with `Cache-Control: no-store`, so the browser keeps
+no copy of what it showed once the vault shuts. A sealed image in a document
+shows as a button while the vault is shut, which asks for the password and
+puts the image back. `:share` says how many sealed files a document holds,
+since its readers will not see them, and `:encrypt` says how many shared
+documents just lost a picture to the vault. `:encrypt` on a folder seals
+what it can and counts what it could not, rather than stopping at the first.
+
+Only the bytes are sealed. A sealed file's name, and the names and text of
+the documents around it, stay readable in `diary.db` — and in the mirror of
+it — like everything else outside the vault.
+
 **There is no recovery.** Forget the vault password and every sealed file is
 gone — from the server and from every backup — which is exactly what makes it
 worth having. The password is not in `.env`, not in `diary.db`, and not in the

src/main.rs

@@ -114,8 +114,10 @@ async fn main() -> Result<()> {
 
     sweep::spawn(db.clone(), config.uploads_dir());
 
+    let vault = vault::Vault::new(db.clone(), config.vault_idle);
+    vault.spawn_idle_lock();
     let state = AppState {
-        vault: vault::Vault::new(db.clone(), config.vault_idle),
+        vault,
         db,
         config: Arc::clone(&config),
         login_throttle: Arc::default(),

src/routes/seal.rs

@@ -22,7 +22,7 @@ use uuid::Uuid;
 use crate::{
     auth::Session,
     error::{AppError, AppResult},
-    routes::nodes,
+    routes::{media::MEDIA_REF, nodes},
     state::AppState,
     vault,
 };
@@ -33,6 +33,11 @@ pub struct Resealed {
     pub files: usize,
     /// Of what was asked about, how many were already that way.
     pub unchanged: usize,
+    /// How many could not be re-written; the rest still were.
+    pub failed: usize,
+    /// Shared documents that embed a file just sealed: their readers now see
+    /// a gap where it was, because a share link never serves a sealed file.
+    pub shared: usize,
 }
 
 pub async fn encrypt(
@@ -84,21 +89,45 @@ async fn reseal(state: &AppState, id: i64, seal: bool) -> AppResult<Resealed> {
         return Err(AppError::BadRequest("there are no files there".into()));
     }
 
+    // A folder is many files, and one that will not re-write — gone from the
+    // disk, cut short — is no reason to leave the rest as they were.
     let mut files = 0;
     let mut unchanged = 0;
-    for target in targets {
+    let mut failed = Vec::new();
+    for target in &targets {
         if target.encrypted == seal {
             unchanged += 1;
             continue;
         }
-        rewrite(state, &key, &target, seal).await?;
+        match rewrite(state, &key, target, seal).await {
-        files += 1;
+            Ok(()) => files += 1,
+            Err(e) => {
+                tracing::warn!(node = target.node_id, "could not re-write a file: {e:?}");
+                failed.push(e);
+            }
+        }
     }
 
     if files > 0 {
         state.backup.signal();
     }
-    Ok(Resealed { files, unchanged })
+    // Asked about one file, the reason it failed is the answer.
+    if files == 0 && unchanged == 0 && failed.len() == 1 {
+        return Err(failed.remove(0));
+    }
+    let shared: i64 = if seal && files > 0 {
+        sqlx::query_scalar(&format!(
+            "SELECT COUNT(DISTINCT d.id) FROM nodes d
+             JOIN node_media nm ON nm.node_id = d.id
+             JOIN nodes f ON f.media_id = nm.media_id
+             WHERE d.kind = 'document' AND d.share_token IS NOT NULL AND f.id IN ({list})"
+        ))
+        .fetch_one(&state.db)
+        .await?
+    } else {
+        0
+    };
+    Ok(Resealed { files, unchanged, failed: failed.len(), shared: shared as usize })
 }
 
 /// Re-write one file sealed or opened under a new id, then point everything
@@ -136,8 +165,8 @@ async fn rewrite(state: &AppState, key: &vault::Key, target: &Target, seal: bool
     tokio::fs::rename(&partial, uploads.join(&new)).await?;
 
     let committed = async {
-        let mut tx = state.db.begin().await?;
+        let mut tx = crate::db::begin_write(&state.db).await?;
-        sqlx::query(
+        let copied = sqlx::query(
             "INSERT INTO media (id, filename, mime, size, created_at, encrypted)
              SELECT ?1, filename, mime, size, created_at, ?2 FROM media WHERE id = ?3",
         )
@@ -145,21 +174,52 @@ async fn rewrite(state: &AppState, key: &vault::Key, target: &Target, seal: bool
         .bind(seal)
         .bind(&old)
         .execute(&mut *tx)
-        .await?;
+        .await?
+        .rows_affected();
+        // Another re-write of the same file got there first.
+        if copied == 0 {
+            return Err(AppError::NotFound);
+        }
         sqlx::query("UPDATE nodes SET media_id = ?1 WHERE media_id = ?2")
             .bind(&new)
             .bind(&old)
             .execute(&mut *tx)
             .await?;
-        // Every document that embeds the file keeps showing it.
+        // Every document and card that embeds the file keeps showing it —
-        sqlx::query(
+        // however the link was written: an id typed in capitals is the
-            "UPDATE nodes SET body = replace(body, '/api/media/' || ?2, '/api/media/' || ?1)
+        // same file.
-             WHERE kind = 'document' AND id IN (SELECT node_id FROM node_media WHERE media_id = ?2)",
+        let bodies = sqlx::query(
+            "SELECT 'nodes' AS tbl, id, body FROM nodes WHERE instr(lower(body), ?1) > 0
+             UNION ALL
+             SELECT 'cards', id, body FROM cards WHERE instr(lower(body), ?1) > 0",
         )
-        .bind(&new)
         .bind(&old)
+        .fetch_all(&mut *tx)
+        .await?;
+        for row in bodies {
+            let body: String = row.get("body");
+            let moved = MEDIA_REF.replace_all(&body, |caps: &regex::Captures| {
+                if caps[1].eq_ignore_ascii_case(&old) {
+                    format!("/api/media/{new}")
+                } else {
+                    caps[0].to_string()
+                }
+            });
+            if moved == body {
+                continue;
+            }
+            let table: String = row.get("tbl");
+            let update = if table == "cards" {
+                "UPDATE cards SET body = ?1 WHERE id = ?2"
+            } else {
+                "UPDATE nodes SET body = ?1 WHERE id = ?2"
+            };
+            sqlx::query(update)
+                .bind(moved.as_ref())
+                .bind(row.get::<i64, _>("id"))
                 .execute(&mut *tx)
                 .await?;
+        }
         sqlx::query("UPDATE node_media SET media_id = ?1 WHERE media_id = ?2")
             .bind(&new)
             .bind(&old)

src/routes/share.rs

@@ -82,11 +82,22 @@ pub async fn enable(
         .execute(&state.db)
         .await?;
 
+    // A share link never serves a sealed file, so a reader of this one sees
+    // a gap wherever the document embeds one. Worth saying while sharing.
+    let sealed: i64 = sqlx::query_scalar(
+        "SELECT COUNT(*) FROM node_media nm JOIN media m ON m.id = nm.media_id
+         WHERE nm.node_id = ?1 AND m.encrypted",
+    )
+    .bind(id)
+    .fetch_one(&state.db)
+    .await?;
+
     state.backup.signal();
     Ok(Json(json!({
         "token": token,
         "key": key,
         "path": format!("/s/{token}#{key}"),
+        "sealed": sealed,
     })))
 }
 

src/routes/vault.rs

@@ -8,18 +8,18 @@ use crate::{
     error::{AppError, AppResult},
     state::AppState,
     throttle::Attempt,
-    vault::{Status, Unlock, MIN_PASSWORD},
+    vault::{Password, Status, Unlock, MIN_PASSWORD},
 };
 
 #[derive(Deserialize)]
 pub struct PasswordInput {
-    pub password: String,
+    pub password: Password,
 }
 
 #[derive(Deserialize)]
 pub struct ChangeInput {
-    pub current: String,
+    pub current: Password,
-    pub password: String,
+    pub password: Password,
 }
 
 fn strong_enough(password: &str) -> AppResult<()> {

src/vault.rs

@@ -316,6 +316,16 @@ pub fn seal_file(master: &Key, media_id: &str, source: &mut impl Read, target: &
     Ok(size)
 }
 
+/// A vault password on its way to Argon2, wiped from memory once it is done.
+pub type Password = Zeroizing<String>;
+
+fn humantime(d: Duration) -> String {
+    match d.as_secs() {
+        s if s % 60 == 0 => format!("{}m", s / 60),
+        s => format!("{s}s"),
+    }
+}
+
 /// The vault's state: whether it exists, and its key while it is open.
 pub struct Vault {
     db: SqlitePool,
@@ -335,6 +345,8 @@ pub struct Status {
     pub unlocked: bool,
     /// Seconds until it locks itself if nothing uses it.
     pub locks_in: Option<u64>,
+    /// How many files are sealed: what an export needs the vault open for.
+    pub sealed: i64,
 }
 
 /// What a password check came to.
@@ -374,8 +386,38 @@ impl Vault {
         *self.slot() = None;
     }
 
+    /// Drop the key once it has gone unused past the idle time. Without this
+    /// a vault nobody looks at would only notice it should be shut on the
+    /// next look, with the key sitting in memory all the while.
+    fn expire(&self) {
+        let mut slot = self.slot();
+        if slot.as_ref().is_some_and(|open| open.touched.elapsed() >= self.idle) {
+            *slot = None;
+            tracing::info!("vault locked after {} idle", humantime(self.idle));
+        }
+    }
+
+    /// Check for an idle vault every few seconds, for as long as the vault
+    /// itself is around.
+    pub fn spawn_idle_lock(self: &Arc<Self>) {
+        let vault = Arc::downgrade(self);
+        let every = (self.idle / 4).clamp(Duration::from_secs(1), Duration::from_secs(30));
+        tokio::spawn(async move {
+            let mut ticker = tokio::time::interval(every);
+            loop {
+                ticker.tick().await;
+                let Some(vault) = vault.upgrade() else { return };
+                vault.expire();
+            }
+        });
+    }
+
     pub async fn status(&self) -> Result<Status> {
         let configured = self.row().await?.is_some();
+        let sealed: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM media WHERE encrypted")
+            .fetch_one(&self.db)
+            .await?;
+        self.expire();
         let slot = self.slot();
         let remaining = slot
             .as_ref()
@@ -385,6 +427,7 @@ impl Vault {
             configured,
             unlocked: remaining.is_some(),
             locks_in: remaining.map(|left| left.as_secs()),
+            sealed,
         })
     }
 
@@ -396,7 +439,7 @@ impl Vault {
 
     /// Make the vault, and leave it open. Refuses when there already is one:
     /// a second setup would orphan every file sealed under the first.
-    pub async fn setup(&self, password: String) -> Result<bool> {
+    pub async fn setup(&self, password: Password) -> Result<bool> {
         if self.row().await?.is_some() {
             return Ok(false);
         }
@@ -432,7 +475,7 @@ impl Vault {
     }
 
     /// The master key the password stands for, if it is the right one.
-    async fn check(&self, password: String) -> Result<Option<Option<Key>>> {
+    async fn check(&self, password: Password) -> Result<Option<Option<Key>>> {
         let Some(row) = self.row().await? else {
             return Ok(None);
         };
@@ -448,7 +491,7 @@ impl Vault {
         Ok(Some(key))
     }
 
-    pub async fn unlock(&self, password: String) -> Result<Unlock> {
+    pub async fn unlock(&self, password: Password) -> Result<Unlock> {
         Ok(match self.check(password).await? {
             None => Unlock::NotConfigured,
             Some(None) => Unlock::WrongPassword,
@@ -461,7 +504,7 @@ impl Vault {
 
     /// Re-seal the master key under a new password. No file changes: they are
     /// sealed under the master key, which stays what it was.
-    pub async fn change_password(&self, current: String, new: String) -> Result<Unlock> {
+    pub async fn change_password(&self, current: Password, new: Password) -> Result<Unlock> {
         let master = match self.check(current).await? {
             None => return Ok(Unlock::NotConfigured),
             Some(None) => return Ok(Unlock::WrongPassword),
@@ -527,6 +570,23 @@ mod tests {
         Ok(out)
     }
 
+    /// Idle means the key is gone from memory, not only that the next look
+    /// will refuse it.
+    #[tokio::test]
+    async fn an_idle_vault_lets_go_of_its_key() {
+        let db = sqlx::SqlitePool::connect("sqlite::memory:").await.unwrap();
+        let vault = super::Vault::new(db, std::time::Duration::from_millis(20));
+        *vault.slot() = Some(super::Unlocked {
+            key: std::sync::Arc::new(key(1)),
+            touched: std::time::Instant::now(),
+        });
+        vault.expire();
+        assert!(vault.slot().is_some(), "still within the idle time");
+        std::thread::sleep(std::time::Duration::from_millis(30));
+        vault.expire();
+        assert!(vault.slot().is_none());
+    }
+
     #[test]
     fn files_of_every_awkward_size_round_trip() {
         let master = key(7);

web/src/lib/DocumentPane.svelte

@@ -40,6 +40,38 @@
     }
   })
 
+  /*
+   * A sealed file embedded here will not load while the vault is shut: the
+   * server answers 423 and the browser shows a broken image. Errors do not
+   * bubble, so they are caught on the way down, and the broken embed is
+   * swapped for a button that opens the vault and puts it back.
+   */
+  $effect(() => {
+    const root = article
+    if (!root) return
+    const sealed = async (event: Event) => {
+      const element = event.target as HTMLElement
+      const media = element.closest('img, video, audio') as HTMLImageElement | HTMLMediaElement | null
+      if (!media || !(media.getAttribute('src') ?? media.querySelector('source')?.getAttribute('src') ?? '').includes('/api/media/')) return
+      await workspace.loadVault()
+      if (!workspace.vault?.configured || workspace.vault.unlocked || !media.isConnected) return
+      const button = document.createElement('button')
+      button.className = 'sealed-embed'
+      button.textContent = 'sealed — unlock to show'
+      button.onclick = async (click) => {
+        click.preventDefault()
+        click.stopPropagation()
+        if (!(await workspace.ensureUnlocked())) return
+        button.replaceWith(media)
+        if (media instanceof HTMLImageElement) media.src = media.src
+        else media.load()
+      }
+      media.replaceWith(button)
+    }
+    root.addEventListener('error', sealed, true)
+    return () => root.removeEventListener('error', sealed, true)
+  })
+
   /**
    * `/n/<space>/<slug>` links — the ones the importer writes between imported
    * documents — are followed in place rather than reloading the application.
@@ -152,6 +184,19 @@
 </section>
 
 <style>
+  .pane :global(.sealed-embed) {
+    display: block;
+    margin: 0.5rem 0;
+    padding: 1.5rem;
+    width: 100%;
+    border: 1px dashed currentColor;
+    background: none;
+    color: inherit;
+    opacity: 0.7;
+    font: inherit;
+    cursor: pointer;
+  }
+
   .pane {
     display: flex;
     flex-direction: column;

web/src/lib/api.ts

@@ -145,6 +145,8 @@ export interface VaultStatus {
   unlocked: boolean
   /** Seconds until it locks itself if nothing uses it. */
   locks_in: number | null
+  /** How many files are sealed: what an export needs the vault open for. */
+  sealed: number
 }
 
 /** The Proton Drive mirror, as `/api/backup` reports it. */
@@ -261,7 +263,9 @@ export const api = {
 
   /** Publishes the document, or mints a fresh key for one already published. */
   share: (id: number) =>
-    request<{ token: string; key: string; path: string }>(`/nodes/${id}/share`, { method: 'POST' }),
+    request<{ token: string; key: string; path: string; sealed: number }>(`/nodes/${id}/share`, {
+      method: 'POST',
+    }),
 
   unshare: (id: number) => request<{ ok: true }>(`/nodes/${id}/share`, { method: 'DELETE' }),
 
@@ -351,7 +355,7 @@ export const api = {
 
   /** Seal a file, or every file under a folder, into the vault — or take it back out. */
   seal: (id: number, encrypt: boolean) =>
-    request<{ files: number; unchanged: number }>(`/nodes/${id}/${encrypt ? 'encrypt' : 'decrypt'}`, {
+    request<{ files: number; unchanged: number; failed: number; shared: number }>(`/nodes/${id}/${encrypt ? 'encrypt' : 'decrypt'}`, {
       method: 'POST',
     }),
 

web/src/lib/commands.ts

@@ -1,6 +1,6 @@
 import { api, isContainer, type BackupStatus } from './api'
 import { workspace } from './store.svelte'
-import { formatDay, parseDay, parseDue, type ScrollAmount } from './util'
+import { formatDay, parseDay, parseDue, sealedNote, type ScrollAmount } from './util'
 
 export type { ScrollAmount }
 
@@ -596,8 +596,8 @@ export const commands: CommandSpec[] = [
       if (!id) return workspace.say('nothing selected', 'error')
       if (!node?.shared) return workspace.say('not shared — :share first', 'error')
       await workspace.guard(async () => {
-        const url = await workspace.mintShareLink(id)
+        const { url, sealed } = await workspace.mintShareLink(id)
-        workspace.say(`new link → ${url} (copied) — the previous one is dead`)
+        workspace.say(`new link → ${url} (copied) — the previous one is dead${sealedNote(sealed)}`)
       })
     },
   },
@@ -636,6 +636,13 @@ export const commands: CommandSpec[] = [
         // A plain navigation, so the browser streams the archive straight to
         // disk instead of the tab holding all of it in memory first.
         await workspace.flush()
+        // Sealed files go into the archive opened, which needs the vault:
+        // asked here, because a refusal after the navigation would replace
+        // the whole app with an error page.
+        await workspace.loadVault()
+        if ((workspace.vault?.sealed ?? 0) > 0 && !(await workspace.ensureUnlocked())) {
+          return workspace.say('the export holds sealed files — it needs the vault open', 'error')
+        }
         workspace.say('building the archive — the download starts on its own')
         location.href = '/api/export'
       },

web/src/lib/store.svelte.ts

@@ -13,7 +13,7 @@ import {
   type SpaceOverview,
   type VaultStatus,
 } from './api'
-import { formatBytes } from './util'
+import { formatBytes, sealedNote } from './util'
 
 export type Pane = 'list' | 'document'
 /** The tree of documents, or the board of the space being browsed. */
@@ -242,10 +242,16 @@ class Workspace {
 
   /** Seal what is under the cursor, or take it back out, and show the result. */
   async seal(id: number, encrypt: boolean) {
-    const { files, unchanged } = await api.seal(id, encrypt)
+    const { files, unchanged, failed, shared } = await api.seal(id, encrypt)
     const verb = encrypt ? 'encrypted' : 'decrypted'
     const already = unchanged > 0 ? `, ${unchanged} already were` : ''
-    this.say(`${verb} ${files} file${files === 1 ? '' : 's'}${already}`)
+    const broken = failed > 0 ? `, ${failed} could not be` : ''
+    const readers =
+      shared > 0 ? ` — ${shared} shared document${shared === 1 ? '' : 's'} embed it, and their readers no longer see it` : ''
+    this.say(
+      `${verb} ${files} file${files === 1 ? '' : 's'}${already}${broken}${readers}`,
+      failed > 0 || shared > 0 ? 'error' : undefined,
+    )
     await this.refresh()
     if (this.open?.kind === 'file') this.open = await api.getNode(this.open.id)
   }
@@ -1096,8 +1102,8 @@ class Workspace {
       await api.unshare(id)
       this.say(`#${id} is private again`)
     } else {
-      const url = await this.mintShareLink(id)
+      const { url, sealed } = await this.mintShareLink(id)
-      this.say(`shared → ${url} (copied)`)
+      this.say(`shared → ${url} (copied)${sealedNote(sealed)}`, sealed > 0 ? 'error' : undefined)
     }
     if (this.open?.id === id) this.open = await api.getNode(id)
     await this.refresh()
@@ -1110,10 +1116,10 @@ class Workspace {
    * retires the old one.
    */
   async mintShareLink(id: number) {
-    const { path } = await api.share(id)
+    const { path, sealed } = await api.share(id)
     const url = `${location.origin}${path}`
     await this.copy(url)
-    return url
+    return { url, sealed }
   }
 
   async copy(text: string) {

web/src/lib/util.ts

@@ -130,3 +130,7 @@ export function scrollPage(el: HTMLElement, amount: ScrollAmount) {
       el.scrollBy({ top: amount * line * 3 })
   }
 }
+
+/** What a share link's reader will miss: a share never serves a sealed file. */
+export const sealedNote = (sealed: number) =>
+  sealed > 0 ? ` — ${sealed} sealed file${sealed === 1 ? '' : 's'} in it stay hidden from readers` : ''