:passwd used to wrap the same master key under the new password, so an
old diary.db (a pre-* snapshot, an old revision on Proton Drive) and the
old password still opened every sealed file. It now makes a new master
key. The old password opens nothing from that moment; the old master
key is kept, sealed under the new one, in two new vault columns, only
until every sealed file has been re-sealed.
A background worker does that while the vault is open, without holding
it open. Each file gets a new id with its file key wrapped under the new
master key; its records are copied as they are. The move goes through
the same repointing as :encrypt, so the old id is put on backup_retired
and the mirror purges it from Drive. Which files are left is read off
their headers, so a lock or a restart halfway only pauses the work until
the next unlock. Once none is left, the old key is dropped.
Readers take a Keyring that tries the current key, then the retired
one. A second :passwd is refused until the first has finished. :vault
shows the progress. Every sealed file is uploaded to Proton once more.
The share page used to put the key in the API path, and every embedded
picture or video carried it in its src, so a reverse proxy's access log
held whole links. The page now sends the key in the body of
POST /api/share/{token}. The answer sets a diary_share cookie
(__Secure- over HTTPS) scoped to /api/share/{token}/, HttpOnly,
SameSite=Strict, good for twelve hours, and signed over the key's hash.
Media is served from /api/share/{token}/media/{id} only to a browser
holding that cookie, so minting a new key or unsharing voids it.
The old /api/share/{token}/{key} routes are gone; links themselves are
unchanged, since the key was always in their fragment.
A session cookie now carries the id of a row in the new sessions table,
and is refused once that row is gone. :sessions lists the browsers
signed in, newest first, and :sessions rm <n> signs one out without
touching the rest; it locks the vault too, since the vault is open for
the whole workspace. Logging out deletes the row, :logout! and a login
password change delete them all, and the sweep drops expired rows.
last_seen is kept in memory and written at most every five minutes.
Cookies issued before this carry no id and are refused: every browser
logs in once more after the upgrade.
The server no longer starts with WORKSPACE_SECRET or WORKSPACE_PASSWORD
left at the value .env.example ships with. The secret was only checked
for length, and the public placeholder passed: anyone who had read the
repository could sign a session cookie for themselves. A secret under
32 characters is warned about.
With WORKSPACE_SECURE_COOKIE=1 the cookie is named __Host-diary_session,
which the browser keeps to this host and to HTTPS and which no sibling
subdomain can plant; turning it on logs everyone out once. Responses
then carry Strict-Transport-Security for a year. The traefik example
turns it on.
The password is compared as an HMAC under the secret, so the time a
comparison takes no longer depends on the configured password's length.
Every response also carries Cross-Origin-Opener-Policy and
Cross-Origin-Resource-Policy same-origin and a Permissions-Policy that
turns camera, microphone, location, payment and USB off; API answers
are no-store unless they say otherwise.
The file pane decided what to show by MIME type, and uploads keep a
type only when it cannot execute, so a .md, a .json or a .pem arrived
as application/octet-stream and got "no preview". It now goes by name
first (preview.ts), and a file with a name it does not know has its
first 8 KiB looked at: no NUL bytes and valid UTF-8 means text.
- text and code: the first 256 KiB with line numbers, as before
- markdown: rendered through the same sanitiser as documents, with a
source toggle
- CSV/TSV: a table of the first 500 rows, quotes respected
- PDF: the browser's own viewer in a frame. Only a PDF is served
without the sandbox CSP (Chrome's viewer will not start under it),
with frame-ancestors 'self' and X-Frame-Options SAMEORIGIN; the app
CSP allows same-origin frames.
- certificates, keys, keystores: POST /api/nodes/{id}/inspect reads
PEM (chains, keys, requests, public keys), DER and PKCS#12 on the
server, in pure Rust, including the RC2/3DES keystores that
openssl -legacy and older keychains write. It answers with subject,
issuer, validity, key, names, usage and fingerprints, and pairs each
private key with its certificate by public key. A password is asked
for only when needed, used for that one answer and never stored;
no key material is ever sent back. Sealed files are read through the
vault, and only while it is open.
The tests read throwaway fixtures under src/routes/testdata, made with
the openssl commands listed beside them; .gitignore lets their .pem
files past a global *.pem rule.
Media answers Range requests with 206 and Content-Range, so videos seek
and Safari plays them. Sealed files too: the vault's opener now decrypts
by record position (STREAM BE32 through StreamPrimitive, the same
construction and the same files), so a range starts at the record that
holds its first byte. Compression is limited to text, because a gzipped
body no longer matches the byte ranges a player asks for.
GET /api/health answers without a session once the database does, and
the compose healthcheck asks it instead of only opening the port.
Requests are logged at info with share tokens and keys left out of the
path and no query strings. A request that is not answered in 30 seconds
gets a 503, except uploads, imports, the export, re-sealing and backups.
The backup, mail, sweep and SFTP loops run under a supervisor that logs
a panic and starts the loop again. The backup loop starts without a
Proton session and picks up a later proton-login without a restart. A
lock file keeps backup-now and the server from mirroring at once, and an
unreadable session key file turns backups off instead of failing boot.
Numbers in the configuration are checked against bounds at startup, so
a session length or an upload limit can no longer overflow later, and a
switch that is neither on nor off is an error rather than off. Config
and MailConfig print with the password, secret and SMTP URL redacted.
A session epoch is signed into each cookie (migration 0014). :logout!
advances it, which ends every session, and a changed WORKSPACE_PASSWORD
advances it on the next start. Cookies issued before this keep working
in the first epoch.
The idle timeout was checked only when the key was next asked for, so a
vault nobody touched kept its key in memory indefinitely while status
reported it locked. A ticker now drops the key once it is idle, and
status() expires it too. Vault passwords are Zeroizing from the request
body to Argon2.
Re-sealing a file rewrote links with a case-sensitive replace, and only
in documents linked through node_media. Links are now matched by regex,
whatever their case, in every document and card body that names the id.
A folder re-seal carries on past a file that fails and reports how many
failed. Two re-seals of one file no longer both commit.
:share reports how many sealed files the document embeds, since a share
link never serves them, and :encrypt reports how many shared documents
embed what it just sealed. Vault status counts sealed files, so :export!
asks for the password first rather than navigating into a 423 page. A
sealed embed that fails to load while the vault is shut becomes a button
that unlocks it and puts the embed back.
Node update, delete and move each ran as separate statements, with the
slug chosen outside the write that took it. They are now one transaction
each, opened with BEGIN IMMEDIATE so a read-then-write cannot fail with
SQLITE_BUSY halfway. Creating a node, a space or a file does the same.
A delete is one function, delete_subtree, used by the API and the SFTP
mount alike: SFTP deletes used to skip the media cleanup and orphan the
files a document embedded. Rows go in the transaction, bytes after the
commit. Deleting a media row that backs a file in the tree is refused;
the media browser marks those files and stops counting them as
documents that embed them.
A space can no longer be moved into a folder. The diary is found by a
role column (migration 0013) instead of its slug, so renaming it no
longer breaks the reminder or the agenda, and it cannot be deleted. The
same migration adds the unique index root slugs never had.
A card is done exactly when it is in the last list: a done patch now
moves the card, a card made in the done list is stamped, and adding or
deleting lists settles the board. Seeding a fresh board is atomic, and
a card links only to a document.
Uploads are fsynced before the rename, and zip entries are written aside
and renamed like any other upload. A sweep at start and once a day
removes leftover .part files, files no row knows, and uploads nothing
has embedded for a week.
Before a new build migrates the database it writes a VACUUM INTO copy,
diary.db.pre-<version>, since migrations only go forward.
DELETE /api/media/{id} joined the raw, percent-decoded id onto uploads/,
so ..%2Fdiary.db deleted the database. Ids are now parsed as uuids before
they reach the filesystem, and a file is only removed when its row was.
Import panicked on a link like %aé: percent_decode sliced the string
between the bytes of a character. Hex digits are now read as bytes. The
SFTP write_at offset is checked before it is cast, so an offset near
u64::MAX no longer wraps past the size limit and panics the session.
The global body limit was the upload ceiling, about a gigabyte, and
applied to the login too. Only the upload routes take that much now;
other bodies stop at 16 MiB and the login at 16 KiB.
Writes a browser says came from another origin are refused, which
SameSite=Lax alone does not cover for sibling subdomains or for
multipart forms. Logout only locks the vault for a caller who is
logged in, so a cross-site form can no longer shut it.
Login and vault attempts go through one at a time, so a burst of
parallel guesses cannot all pass the throttle before the first failure
is recorded, and parallel Argon2 runs cannot pile up.
Sealed files are served with no-store instead of a year of cache, so
their plaintext does not outlive :lock in the browser. The SFTP host key
is created 0600 rather than narrowed after being written.
SIGTERM is handled: as PID 1 in the container it was ignored, and every
docker stop ended in SIGKILL. Requests get eight seconds to finish, the
pool is closed so the WAL is checkpointed, and compose runs tini.
WORKSPACE_SESSION_KEY_FILE names a file holding a 32-byte key, meant to
be a Docker secret. When it is set, proton-session.json is written as
AES-256-GCM ciphertext, so a copy of the data volume alone no longer
carries a usable session. That is what the OS keyring gives a native
host, for a container that has no Secret Service.
A plain session found while a key is set is sealed the first time it is
read. A sealed file without its key, or a malformed key file, is an
error at startup rather than a quiet fall back to plain JSON.
Argon2id turns the vault password into a key that seals a random master
key; only the sealed copy is kept. Each file gets its own key, sealed
under the master key in its header with its media id, and its bytes are
AES-256-GCM STREAM records, so a swapped, cut or altered file refuses
to open. Uploads with ?encrypt=true are sealed as they stream in.
:vault, :unlock, :lock, :passwd, :encrypt, :decrypt and :upload! drive
it. The vault locks after WORKSPACE_VAULT_IDLE_MIN idle minutes, on
logout and on restart; while locked, sealed files answer 423, share
links never serve them, and :export! refuses rather than leave them out.
Sealing an existing file rewrites it under a new id and retires the old
one: the mirror deletes the plaintext from Proton Drive outright,
trash included, whatever WORKSPACE_BACKUP_PRUNE says, and drops it from
the readable files/ view. The mirror now compares on-disk sizes, so
sealed files are not re-hashed every run.
Add an in-process SFTP server (russh) backed directly by the nodes
table, so a space can be edited from nvim or synced with sshfs without
a separate filesystem mirror. Access is authenticated by public key
against a new ssh_keys table, managed via :sshkey in the app; the
feature stays off until WORKSPACE_SFTP_BIND is set.
The diary grows into a workspace: a flat list of dated entries becomes a
tree of spaces, and the app is renamed to match what it now is.
Spaces and nodes
One recursive `nodes` table replaces `entries`: a space is a root node,
and folders and documents nest under it without limit. Entries keep
their ids through the migration, so share tokens, media links and the
drafts in localStorage all still resolve.
Folder import
A folder or a zip is uploaded as a document tree. Markdown becomes
documents titled from their first heading, everything else becomes
media, and relative links between the imported files are rewritten to
the paths they landed at, so a set of cross-linked notes stays
navigable after the upload.
Boards
A kanban board per space, with the space as the board: lists, cards,
due dates and an optional link to a document. `diary` has none.
Mail
SMTP notifications through an outbox. A mail is enqueued under a key
naming the occurrence it belongs to, so a restart cannot resend and a
relay that refuses is retried with a widening backoff. Four kinds: the
diary reminder, cards due and overdue, a weekly digest, and logins
from a browser that has not been seen before or a burst of failures.
All of it is dormant until WORKSPACE_SMTP_URL is set.
Rename
Everything user-visible says workspace; everything that already exists
outside this repo keeps its name, because renaming it would break a
running deployment for tidiness alone: the database file, the docker
volume, the session cookie, the Proton device, the keyring entry and
the localStorage keys. Configuration reads WORKSPACE_* first and falls
back to DIARY_*, so the deployed .env keeps working untouched.
Proton refuses a file directly in a device's root folder — 422 "Cannot
create file at the root of a device" — so the first real backup died on
RESTORE.txt, before it had written anything. The mirror now lives in a
`data/` folder on the device, which is both what Proton permits and the
honest name for what it holds: the contents of DIARY_DATA_DIR.
Every remote path changed with it, so the bookkeeping describing the old
layout is dropped and rebuilt on the next run. A device root that
already collected an empty `uploads/` folder from the failed attempt
keeps it; it is not ours to guess about, and deleting it is one click.
Also quiet rPGP's packet-header warnings by default. They fire on every
key Proton wrote slightly differently to how rPGP re-serialises it —
cosmetic, once per unlock, and loud enough to bury the backup's own log.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5Gk8FPkRXu1e6Nj2BCRE1
The Proton Drive client verifies TLS against the system CAs, and
debian-slim ships none at all — no /etc/ssl/certs, no bundle — so every
request out of the container died at the handshake with "No CA
certificates were loaded from the system", before it could even be a
login. `webpki-roots` is compiled in, but choosing it happens inside the
SDK when it builds its HTTP client, so the image is the place to fix it.
Also answer `--help` before reading the configuration: a host that has
not been set up yet is exactly where that question gets asked, and it
was being met with "DIARY_USER must be set".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5Gk8FPkRXu1e6Nj2BCRE1
The data directory is the whole diary, and until now keeping a copy of it
was someone's manual job. The server can now register itself in Proton
Drive as a device — its own sync root, next to the desktop clients — and
keep that folder equal to DIARY_DATA_DIR, encrypted client-side by the
same Rust SDK the Linux client uses.
One interactive `narl-diary proton-login`, then nothing prompts again.
The stored session carries the tokens, the mailbox password (the key
chain is rebuilt from it on every start) and the account key salts, which
have to be captured at login: `core/v4/keys/salts` needs the `locked`
scope, and only a password-login token has it. Refresh tokens are
single-use, so every rotation is written back before it can be needed.
It lives as a 0600 file next to the database, on the same volume as the
data it protects; `--features keyring` puts it in the OS keyring instead,
for hosts that have a Secret Service.
The mirror is one-way and change-driven: writes mark the diary dirty and
it is mirrored once the writing stops, with an interval as the backstop —
an hour of writing is one backup, not sixty. `diary.db` goes up as a
VACUUM INTO snapshot, so it is a real point in time rather than a file
copied out from under a writer, and as a new revision each time, so
Proton keeps the older ones. Uploads are immutable, so they are written
once and skipped by size afterwards. Deletions do not propagate unless
asked: a backup that forgets on command is one accident away from being
no backup at all.
Identification follows Proton's third-party rules — an honest
`external-drive-narl_diary@<version>-alpha`, the disclosure printed
before any credential is asked for, and remembered folder uids instead of
walking the tree every run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D5Gk8FPkRXu1e6Nj2BCRE1
The login endpoint answered guesses as fast as they arrived, against a
single static password.
Wrong attempts now back off — 1s, 2s, 4s, up to 30s — and a correct one
clears the count. It is a single counter rather than a table keyed by
address: there is only one account, so an attacker gains nothing by
rotating source IPs, and there is no per-client map to grow. The cap is
there so that someone hammering the login delays the owner by half a
minute rather than locking them out.
Wrong credentials now answer 401 rather than 400. The server also warns
at startup if DIARY_PASSWORD is still the value from .env.example.
The list query selected every entry's full body so that excerpt() could
keep the first 180 characters of it. Listing a few years of diary moved
megabytes over the wire to render a sidebar, on every keystroke of an
incremental search.
It now reads the leading 1024 characters, which is more than an excerpt
can use, and infers truncation from the length of what came back.
Also pairs WAL with synchronous=NORMAL and an explicit busy_timeout — the
usual settings for it, and neither was being set.
Nothing set a CSP, a referrer policy, nosniff or a framing rule, so the
app relied entirely on the same-origin policy.
The referrer policy is the one with a concrete leak behind it: a share
token is part of the URL path, so following any outbound link from a
shared entry handed a third party a working link to a private entry.
`no-referrer` closes that.
The layer is `if_not_present`, so the media routes keep the stricter
sandbox CSP they set for themselves.
Single-user diary served as one Rust binary with the Svelte frontend
embedded.
- axum 0.8 + SQLite (sqlx), migrations run at startup
- stateless HMAC session cookie; credentials from .env, compared in
constant time
- FTS5 full-text search, user input rewritten into quoted prefix terms
- media uploads linked to entries by scanning bodies for /api/media/<id>,
which scopes share-token media access and entry deletion
- unguessable share links (192-bit token) serving a public read-only page
- vim keybindings via CodeMirror plus a modal browse layer and : command
line; disabled on touch devices
nvrl
