files: a file belongs to a node, and a document holds its own

An upload while writing is now a file node of the document, through
POST /nodes/{id}/files, instead of a loose row of media that only the media
browser listed. It moves, seals, goes to the trash and comes back with the
document. Migration 0029 gives every embedded upload that had no node a
place under the document that embeds it (a living one before a trashed
one), keeping its media id, so no body changes; the sealed-folder triggers
now ask a document's folder for the files under it.

- nodes: `holds` and `place_for` say what may go where; a document holds
  files and nothing else, in moves, uploads and restores from the trash.
  A loaded document carries its files and whether its folder seals them.
- seal: a document moving into a sealed folder seals its files first.
- media: POST /media, GET /media and DELETE /media/{id} are gone, and with
  them MediaBrowser and :media.
- import: `plan/` beside `plan.md` comes back as the document's files, the
  shape export writes; an older export's media/ goes to the first document
  that embeds each file.
- sftp: a document's files are `<slug>/` beside its `.md`, listed once it
  has any, holding files only, moving with the `.md`.
- web: paste, drop and :upload while reading go to the document; the info
  sheet has a files section (Enter opens, i embeds, y copies how); the list
  shows +N on a document, za folds its files open, p drops marked files in.
This commit is contained in:
nvrl committed 2026-10-22 11:57:52 +02:00
1 parent 2233fa41a6
commit d8e72c688d
29 files changed
+862 -483

No files matched your search

+21 -18
View File
@@ -141,6 +141,7 @@ it; in the document pane it is that document. In the browse pane:
| `i`, `a` | edit the open entry |
| `R` | rename what is highlighted — a document, a folder or a file — starting from its name (`:name!` renames the space) |
| `m` / `p` | pick nodes up — `m` again puts one back, `M` takes the whole folder — then drop them all into the folder you walk to |
| `za` | fold a document's own files open under it in the list, or away again |
| `Ctrl`/`Shift`-click | pick a row up / every row from the cursor to it; drag picked-up rows onto a folder, a crumb or a space |
| `S` | the board, the agenda and the calendar tiled ahead of what is open, or every view away again; `Alt-h`/`j`/`k`/`l` move between the tiles, `f` gives one the screen |
| `b` | the board of this space, in a tile; a digit there shows another space's |
@@ -165,7 +166,7 @@ it; in the document pane it is that document. In the browse pane:
Inside the editor the full vim keymap is live — `w`/`b`, `dd`, `ciw`, visual
mode, macros, `:w` to write. `:help` lists every ex command; the useful ones are
`:w`, `:wq`, `:q!`, `:new [template] [yyyy-mm-dd] [name]`, `:date`, `:name`, `:share`, `:link`,
`:upload`, `:media`, `:search`, `:backup` and
`:upload`, `:search`, `:backup` and
`:set novim`. `:name` on its own moves up into the title; `Enter`, `Esc` or `↓`
there goes back down into the text. In the text, `]]`, `[[` and `gO` walk the
headings as they do in the reader. `gf` follows the link under the cursor, saving
@@ -443,8 +444,9 @@ readers already hold.
`:export` downloads the open document as markdown. `:export!` downloads the
whole workspace as a zip: the tree as real directories, empty folders too, one
markdown file per document, every file in the tree where it sits and every
other embedded file in `media/`, under their real names, and relative links
markdown file per document, every file in the tree where it sits — a
document's own files in a directory named like it, `plan/` beside `plan.md` —
under their real names, and relative links
between documents, folders and files, so the archive reads in any markdown
viewer without this application. What plain files have no place for — when
each entry was written, names as they were typed, which space is the diary,
@@ -457,15 +459,17 @@ the agenda still know which space it is, and it cannot be deleted — empty it
instead. Before a new build migrates the database it keeps a copy of it as
it was, `diary.db.pre-<version>`, next to it in the data directory.
An upload that nothing embeds any more — pasted, then deleted from the
document — is kept a week and then removed, along with what a killed upload
or a crash leaves in `uploads/`. That runs at start and once a day. A file in
the tree is never one of them: it is deleted from its folder, not from
`:media`, and its bytes stay on the disk until it leaves the trash.
`:media` lists every upload: `j`/`k` walk it, `i` or `Enter` embeds the
highlighted one at the cursor while writing, `y` copies its address and `x`
deletes it, asking first.
Every upload is a file in the tree. One pasted or dropped while writing, or
put in with `:upload` while reading a document, belongs to that document: it
moves, seals, goes to the trash and comes back with it. The info sheet lists
a document's files (`:info files`): `Enter` opens one, `i` embeds it at the
cursor while writing, `y` copies how to embed it elsewhere. In the list a
document with files says `+2`, and `za` folds them open under it, where they
are renamed, deleted and picked up like any other row; `p` with the cursor
among them drops marked files there. A file is deleted where it sits, and its
bytes stay on the disk until it leaves the trash — or for as long as another
document still embeds it. What a killed upload or a crash leaves in
`uploads/` is removed at start and once a day.
## Links
@@ -654,9 +658,9 @@ heading), anything else becomes a file in the tree. `:import!` does the same wit
a zip, which is the shape `:export!` writes. `:import! /` pours an `:export!`
back in at the root: each directory becomes a space again — or goes into the
space already there by that name — and with `.workspace.json` beside it the
dates, names, boards and cards come back as they were, embedded files go back
to being embedded rather than a `media` folder, and a file that happened to end
in `.md` stays a file.
dates, names, boards and cards come back as they were, a document's files go
back to being its own — `plan/` beside `plan.md`, or the `media/` folder of an
older export — and a file that happened to end in `.md` stays a file.
Links between the imported files are rewritten as they land: a link to a
markdown file becomes the document's address, a link to an image becomes its
@@ -903,11 +907,10 @@ sealed or not.
| `GET`/`POST` | `/api/backup` | Proton Drive mirror: status / run now |
| `POST`/`DELETE` | `/api/backup/login` | log the mirror in to Proton, body `{"username","password"}` / forget its session |
| `POST` | `/api/backup/login/totp`, `/api/backup/login/captcha` | the two-factor code `{"code"}`, or the solved human check `{"token"}`, a waiting login asked for |
| `GET`/`POST` | `/api/media` | list / upload (multipart) |
| `GET`/`DELETE` | `/api/media/{id}` | serve (with `Range`) / delete a file nothing in the tree owns |
| `GET` | `/api/media/{id}` | serve a file's bytes (with `Range`) |
| `POST` | `/api/typst` | `{source}` of a ` ```typst ` block → `{svg}`, or `{error}` as typst words it |
| `GET` | `/api/spaces/overview` | every space with what was last written in it |
| `POST` | `/api/nodes/{id}/files` | upload files into this folder as file nodes (`?encrypt=true` seals them) |
| `POST` | `/api/nodes/{id}/files` | upload files (multipart) into a space or folder, or to a document as its own; `?encrypt=true` seals them, and a sealed folder always does |
| `POST` | `/api/nodes/{id}/inspect` | what a certificate, key or keystore file holds (`{password}` opens a protected one) |
| `POST` | `/api/nodes/{id}/encrypt`, `/decrypt` | seal a file, or a folder for good — what is in it and what lands later — or take it out |
| `GET`/`POST`/`PUT` | `/api/vault` | status / set the password once / change it |
+2 -3
View File
@@ -50,11 +50,10 @@ sealed or not.
| `GET`/`POST` | `/api/backup` | Proton Drive mirror: status / run now |
| `POST`/`DELETE` | `/api/backup/login` | log the mirror in to Proton, body `{"username","password"}` / forget its session |
| `POST` | `/api/backup/login/totp`, `/api/backup/login/captcha` | the two-factor code `{"code"}`, or the solved human check `{"token"}`, a waiting login asked for |
| `GET`/`POST` | `/api/media` | list / upload (multipart) |
| `GET`/`DELETE` | `/api/media/{id}` | serve (with `Range`) / delete a file nothing in the tree owns |
| `GET` | `/api/media/{id}` | serve a file's bytes (with `Range`) |
| `POST` | `/api/typst` | `{source}` of a ` ```typst ` block → `{svg}`, or `{error}` as typst words it |
| `GET` | `/api/spaces/overview` | every space with what was last written in it |
| `POST` | `/api/nodes/{id}/files` | upload files into this folder as file nodes (`?encrypt=true` seals them) |
| `POST` | `/api/nodes/{id}/files` | upload files (multipart) into a space or folder, or to a document as its own; `?encrypt=true` seals them, and a sealed folder always does |
| `POST` | `/api/nodes/{id}/inspect` | what a certificate, key or keystore file holds (`{password}` opens a protected one) |
| `POST` | `/api/nodes/{id}/encrypt`, `/decrypt` | seal a file, or a folder for good — what is in it and what lands later — or take it out |
| `GET`/`POST`/`PUT` | `/api/vault` | status / set the password once / change it |
+3 -3
View File
@@ -41,6 +41,7 @@ arguments and `↑` / `↓` recall earlier lines.
| `i · a` | edit the open document |
| `R` | rename what is under the cursor — document, folder or card — starting from its name |
| `m / p` | pick nodes up, then drop them all into the folder you walk to |
| `za` | a document's own files, folded open under it in the list — again folds them away; p drops marked files in there |
| `M` | pick up the whole folder — again puts it all back |
| `Ctrl / Shift-click` | pick a row up · every row up to it — drag them onto a folder |
| `S` | the board, the agenda and the calendar, tiled ahead of what is open with the board focused — S again puts every view away |
@@ -191,7 +192,7 @@ do: `:w!`, `:space! <name>`, `:import!`.
| `:cards` | the cards about the open document — its own, and the ones whose summary links here: j/k picks the one :due, :label, Space and H/L act on, Enter turns it over on its board (gc) |
| `:source` | show an open markdown file as written, or rendered again |
| `:outline, :toc` | the open document's headings to pick from — gO, in the editor too |
| `:info [outline\|history\|backlinks\|cards\|share]` | what is known about the open document: its headings, earlier versions, backlinks, cards and sharing — Tab between them (gi) |
| `:info [outline\|history\|backlinks\|cards\|files\|share]` | what is known about the open document: its headings, earlier versions, backlinks, cards, files and sharing — Tab between them (gi) |
| `:search, :se [text]` | full-text search in this space (empty clears) |
| `:search!` | full-text search across every space — Ctrl-P |
| `:clear, :noh` | clear the search filter |
@@ -223,9 +224,8 @@ do: `:w!`, `:space! <name>`, `:import!`.
| `:name, :title, :rename <text>` | rename what is under the cursor — document, folder, or card on a board; bare, while writing, into the title |
| `:name!` | rename the space you are in — on the board, the list under the cursor |
| `:date <yyyy-mm-dd>` | re-date the open document |
| `:upload, :up` | while writing, attach files at the cursor — otherwise put files into this folder |
| `:upload, :up` | files for the open document — while writing, embedded at the cursor — or, from the list, into this folder |
| `:upload!` | put files into this folder sealed in the vault — encrypted before they touch the disk |
| `:media` | browse everything you have uploaded — j/k, i inserts it while writing, y copies its address, x deletes |
### Sharing & export
+10 -3
View File
@@ -3,13 +3,20 @@
## Files
A file is a node like a document, only with bytes instead of text. It sits in a
folder beside the notes about it, shows its size in the list, and `Enter` opens
it.
folder beside the notes about it, or belongs to a document, shows its size in
the list, and `Enter` opens it.
A document's own files are the ones pasted or dropped into it while writing,
and the ones `:upload` puts in while reading it. They move, seal, go to the
trash and come back with the document. A document with files says `+2` in the
list; `za` folds them open under it, and `:info files` lists them too. With
the cursor among them, `p` drops files picked up elsewhere in.
Put files in:
- drop them onto the list;
- `:upload` picks files for the folder you are in;
- `:upload` picks files for the folder you are in, or for the document you
are reading;
- `:import` uploads a whole folder, and `:import!` a zip
(see [Not losing things](Not-Losing-Things.md#import));
- copy them in over the [SFTP mount](SFTP.md).
+3
View File
@@ -33,6 +33,9 @@ directory.
different first heading back renames it.
- Files sit beside the documents under their own names. A later file with the
same name is told apart by its id: `slides (42).pdf`.
- A document's own files are in `<slug>/`, beside its `.md`. The directory is
listed once the document has a file, and can be copied into either way. It
holds files and nothing else, and moves when the `.md` is renamed.
- Anything copied in that is not a `.md` becomes a file, and its text is
indexed for search.
- `rm` and `rmdir` put things in the trash, as a delete in the browser does.
+1
View File
@@ -79,6 +79,7 @@ reader:
| `gi` | the info sheet |
| `gO` | the headings as a list to jump to |
| `zl` / `zh` | scroll a wide table or code block sideways |
| `za` | fold a document's own files open under it in the list |
| `F` | a letter over every link and button in view; type it to follow |
| `gh` | earlier versions |
| `gb` | the documents that link here |
+5 -3
View File
@@ -28,9 +28,11 @@ note in the scratchpad are written in it too, and behave the same way.
back on. On a touch screen they start off.
Drag, drop or paste an image, a video or a sound file into the text and it is
uploaded and embedded where the cursor is. `:upload` picks files to embed, and
`:media` lists every upload: `Enter` embeds the one under the cursor, `y`
copies its address.
uploaded and embedded where the cursor is. `:upload` picks files to embed. Each
one becomes a file of the document: it moves, seals, goes to the trash and
comes back with it. `:info files` lists them: `Enter` opens one, `i` embeds it
at the cursor, `y` copies how to embed it in another document. In the list a
document with files says `+2`, and `za` folds them open under it.
## Saving
+65
View File
@@ -0,0 +1,65 @@
-- A file belongs to a node. A picture pasted into a document used to be a row
-- of `media` and nothing more: the document embedded it, but only the media
-- browser could list it, and it did not move, seal or go to the trash with
-- the document. Now an upload while writing is a file node of the document,
-- and each of the old ones becomes one too, under the document that embeds
-- it. The media id stays, so no `/api/media/<id>` in any body changes.
--
-- A file embedded in several documents goes to one of them, a document that
-- is not in the trash before one that is, and the oldest of those. The slug
-- cannot be derived from the name in SQL, so, as in 0010, it is the start of
-- the media id: unique, and renaming the file fixes it for good.
WITH RECURSIVE trashed (id) AS (
SELECT id FROM nodes WHERE role = 'trash'
UNION ALL
SELECT n.id FROM nodes n JOIN trashed t ON n.parent_id = t.id
),
owners (media_id, node_id, rank) AS (
SELECT nm.media_id, nm.node_id,
ROW_NUMBER() OVER (
PARTITION BY nm.media_id
ORDER BY nm.node_id IN (SELECT id FROM trashed), nm.node_id
)
FROM node_media nm
JOIN nodes d ON d.id = nm.node_id
WHERE d.kind = 'document'
)
INSERT INTO nodes (parent_id, kind, name, slug, created_at, updated_at, media_id)
SELECT o.node_id, 'file', m.filename, 'file-' || substr(m.id, 1, 8),
m.created_at, m.created_at, m.id
FROM media m
JOIN owners o ON o.media_id = m.id AND o.rank = 1
WHERE NOT EXISTS (SELECT 1 FROM nodes f WHERE f.media_id = m.id);
-- A file node owns its bytes the way a document owns what it embeds, so
-- purging the document still finds every file that goes with it.
INSERT OR IGNORE INTO node_media (node_id, media_id)
SELECT id, media_id FROM nodes WHERE kind = 'file';
-- A document is not sealed itself, but the folder it is in decides for its
-- files, the same as for a file beside it. The triggers of 0015 asked only
-- the parent, which for a document's file is the document.
DROP TRIGGER nodes_sealed_insert;
DROP TRIGGER nodes_sealed_update;
CREATE TRIGGER nodes_sealed_insert BEFORE INSERT ON nodes
WHEN new.kind = 'file'
AND (SELECT CASE WHEN p.kind = 'document'
THEN (SELECT g.sealed FROM nodes g WHERE g.id = p.parent_id)
ELSE p.sealed END
FROM nodes p WHERE p.id = new.parent_id) = 1
AND (SELECT encrypted FROM media WHERE id = new.media_id) = 0
BEGIN
SELECT RAISE(ABORT, 'sealed folder: a file here must be encrypted');
END;
CREATE TRIGGER nodes_sealed_update BEFORE UPDATE OF parent_id, media_id ON nodes
WHEN new.kind = 'file'
AND (SELECT CASE WHEN p.kind = 'document'
THEN (SELECT g.sealed FROM nodes g WHERE g.id = p.parent_id)
ELSE p.sealed END
FROM nodes p WHERE p.id = new.parent_id) = 1
AND (SELECT encrypted FROM media WHERE id = new.media_id) = 0
BEGIN
SELECT RAISE(ABORT, 'sealed folder: a file here must be encrypted');
END;
+3 -2
View File
@@ -166,8 +166,9 @@ This is a complete copy of a ~/workspace.
<space>/ one directory per space, holding the same tree of folders,
documents and files that the workspace shows; one markdown
file per document, and every file under its own name
media/ the files documents embed that are not in the tree themselves
file per document, every file under its own name, and a
document's own files in a directory named like it
media/ files embedded somewhere that belong to no document
.workspace.json what plain files cannot hold: dates, names as they were
typed, the boards. Importing the archive puts it all back.
+166 -34
View File
@@ -4,9 +4,11 @@
//! `docs/`, a folder of lecture notes, an export of this very diary. Storing it
//! as opaque files would make it unsearchable and uneditable, so the importer
//! reproduces the directory as nodes: directories become folders, markdown
//! becomes documents, everything else becomes a file. The links between the
//! files are rewritten as they land, because a set of documents that can no
//! longer reach each other is a worse copy of what was uploaded.
//! becomes documents, everything else becomes a file. A directory beside a
//! document and named like it, `plan.md` and `plan/`, holds that document's
//! files, which is how the export writes them. The links between the files
//! are rewritten as they land, because a set of documents that can no longer
//! reach each other is a worse copy of what was uploaded.
use std::collections::{BTreeSet, HashMap};
@@ -729,13 +731,43 @@ async fn build(
}
let uploads = unique;
// An export keeps the files documents embed in `media/`, beside the spaces;
// they come back as what they were, embedded files, not as a folder.
// An older export kept the files documents embed in `media/`, beside the
// spaces; each comes back as a file of the first document that embeds it.
let embedded = |path: &str| manifest.is_some() && parent_of(path) == manifest::MEDIA_DIR;
let meta = |path: &str| -> Option<&NodeMeta> { manifest.as_ref()?.nodes.get(path) };
// `plan/` is the files of `plan.md` when it holds files and nothing else,
// and the manifest, if there is one, does not know it as a folder.
let mut crowded: BTreeSet<&str> = BTreeSet::new();
for upload in &uploads {
let dir = parent_of(&upload.path);
if matches!(upload.content, Content::Markdown(_)) {
crowded.insert(dir);
}
let mut above = parent_of(dir);
while !above.is_empty() {
crowded.insert(above);
above = parent_of(above);
}
}
let documents_at: BTreeSet<&str> = uploads
.iter()
.filter(|u| matches!(u.content, Content::Markdown(_)))
.filter_map(|u| u.path.strip_suffix(".md"))
.collect();
let attached: BTreeSet<String> = uploads
.iter()
.map(|u| parent_of(&u.path))
.filter(|dir| !dir.is_empty() && documents_at.contains(dir) && !crowded.contains(dir))
.filter(|dir| meta(dir).is_none_or(|m| !nodes::is_container(&m.kind)))
.map(str::to_string)
.collect();
let mut directories: BTreeSet<String> = BTreeSet::new();
for upload in uploads.iter().filter(|u| !embedded(&u.path)) {
for upload in uploads
.iter()
.filter(|u| !embedded(&u.path) && !attached.contains(parent_of(&u.path)))
{
let mut dir = parent_of(&upload.path);
while !dir.is_empty() {
directories.insert(dir.to_string());
@@ -853,8 +885,14 @@ async fn build(
let mut documents: Vec<(i64, String, String)> = Vec::new();
let mut by_path: HashMap<String, i64> = HashMap::new();
let mut files = 0usize;
// The files of `media/`, waiting for the document that embeds them.
let mut unowned: HashMap<String, (String, i64, i64)> = HashMap::new();
for upload in &uploads {
// Documents before files, so a document is there for its files to go in.
let mut ordered: Vec<&Upload> = uploads.iter().collect();
ordered.sort_by_key(|u| !matches!(u.content, Content::Markdown(_)));
for upload in ordered {
let dated = meta(&upload.path);
let (created, updated) = dated.map_or((now, now), |m| (m.created_at, m.updated_at));
@@ -888,37 +926,20 @@ async fn build(
Landed::Media(format!("/api/media/{media_id}")),
);
if embedded(&upload.path) {
unowned.insert(media_id.clone(), (filename.to_string(), created, updated));
continue;
}
// The file sits in the tree under the name it had, and owns its
// bytes the way a document owns what it embeds — so deleting the
// folder it landed in takes it along.
let parent = dir_ids[parent_of(&upload.path)];
// The file sits in the tree under the name it had: in its folder,
// or as a file of the document whose directory it came in.
let dir = parent_of(&upload.path);
let parent = if attached.contains(dir) {
by_path[&format!("{dir}.md")]
} else {
dir_ids[dir]
};
let taken = siblings.entry(parent).or_default();
let slug = next_slug(taken, &slugify(filename))?;
taken.insert(slug.clone());
let node: i64 = sqlx::query_scalar(
"INSERT INTO nodes (parent_id, kind, name, slug, created_at, updated_at, media_id)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
RETURNING id",
)
.bind(parent)
.bind(FILE)
.bind(filename)
.bind(&slug)
.bind(created)
.bind(updated)
.bind(media_id)
.fetch_one(&mut *tx)
.await?;
sqlx::query("INSERT INTO node_media (node_id, media_id) VALUES (?1, ?2)")
.bind(node)
.bind(media_id)
.execute(&mut *tx)
.await?;
file_node(&mut tx, taken, parent, media_id, filename, created, updated).await?;
files += 1;
} else if let Content::Markdown(raw) = &upload.content {
let dir = parent_of(&upload.path);
@@ -975,6 +996,18 @@ async fn build(
}
media::link_to_node(&mut tx, *id, &rewritten).await?;
links::index(&mut tx, *id, &rewritten).await?;
let embeds: BTreeSet<String> = media::MEDIA_REF
.captures_iter(&rewritten)
.map(|c| c[1].to_lowercase())
.collect();
for media_id in embeds {
if let Some((filename, created, updated)) = unowned.remove(&media_id) {
let taken = siblings.entry(*id).or_default();
file_node(&mut tx, taken, *id, &media_id, &filename, created, updated).await?;
files += 1;
}
}
}
// The boards come back only where spaces do, and the scratchpad with
@@ -1014,6 +1047,42 @@ async fn build(
})
}
/// A file node for bytes already in `media`, under `parent`, with a slug not
/// in `taken` — which it then is. It owns its bytes the way a document owns
/// what it embeds, so deleting its folder or its document takes it along.
async fn file_node(
tx: &mut sqlx::SqliteConnection,
taken: &mut BTreeSet<String>,
parent: i64,
media_id: &str,
filename: &str,
created: i64,
updated: i64,
) -> AppResult<i64> {
let slug = next_slug(taken, &slugify(filename))?;
taken.insert(slug.clone());
let node: i64 = sqlx::query_scalar(
"INSERT INTO nodes (parent_id, kind, name, slug, created_at, updated_at, media_id)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)
RETURNING id",
)
.bind(parent)
.bind(FILE)
.bind(filename)
.bind(&slug)
.bind(created)
.bind(updated)
.bind(media_id)
.fetch_one(&mut *tx)
.await?;
sqlx::query("INSERT INTO node_media (node_id, media_id) VALUES (?1, ?2)")
.bind(node)
.bind(media_id)
.execute(&mut *tx)
.await?;
Ok(node)
}
#[cfg(test)]
mod tests {
use std::collections::HashMap;
@@ -1084,6 +1153,69 @@ mod tests {
assert_eq!(back, "private bytes");
}
/// `plan/` beside `plan.md` is the document's files, as the export
/// writes them; a file an older export kept in `media/` becomes a file of
/// the document that embeds it.
#[tokio::test]
async fn a_documents_files_come_back_as_its_files() {
let state = crate::routes::tests::state_in(".".into()).await;
let photo = "1a000000-0000-0000-0000-000000000000";
let old = "2b000000-0000-0000-0000-000000000000";
let file = |path: &str, id: &str| {
text(
path,
Content::File {
id: id.into(),
mime: "image/png".into(),
size: 1,
},
)
};
let uploads = vec![
text(
"work/plan.md",
Content::Markdown(format!(
"# Plan\n\n![p](plan/photo.png)\n\n![o](../media/{old}-old.png)"
)),
),
file("work/plan/photo.png", photo),
file(&format!("media/{old}-old.png"), old),
];
let manifest = crate::routes::manifest::Manifest {
format: crate::routes::manifest::FORMAT,
media: [(format!("media/{old}-old.png"), "old.png".to_string())]
.into_iter()
.collect(),
..Default::default()
};
let report = super::build(
&state,
&super::Into::Root,
uploads,
Vec::new(),
Some(manifest),
)
.await
.unwrap();
assert_eq!((report.folders, report.documents, report.files), (0, 1, 2));
let (plan, body): (i64, String) =
sqlx::query_as("SELECT id, body FROM nodes WHERE kind = 'document' AND name = 'Plan'")
.fetch_one(&state.db)
.await
.unwrap();
assert!(body.contains(&format!("/api/media/{photo}")), "{body}");
let parents: Vec<(String, i64)> =
sqlx::query_as("SELECT name, parent_id FROM nodes WHERE kind = 'file' ORDER BY name")
.fetch_all(&state.db)
.await
.unwrap();
assert_eq!(
parents,
[("old.png".into(), plan), ("photo.png".into(), plan)]
);
}
fn text(path: &str, content: Content) -> Upload {
Upload {
path: path.into(),
+8 -164
View File
@@ -11,7 +11,7 @@ use axum::{
Json,
};
use regex::Regex;
use serde::{Deserialize, Serialize};
use serde::Deserialize;
use sqlx::{Connection, Row, SqliteConnection, SqlitePool};
use tokio::io::AsyncWriteExt;
use tokio_util::io::ReaderStream;
@@ -67,47 +67,6 @@ fn is_inline(mime: &str) -> bool {
mime != "application/octet-stream"
}
#[derive(Serialize)]
pub struct Media {
pub id: String,
pub filename: String,
pub mime: String,
pub size: i64,
pub created_at: i64,
pub url: String,
/// Sealed in the vault: its bytes on disk are ciphertext.
pub encrypted: bool,
/// Every document that embeds this file, which may be none or several.
pub node_ids: Vec<i64>,
/// The file node it is, when it sits in the tree: deleted from there, not
/// from the media list.
pub file_node: Option<i64>,
}
fn row_to_media(row: &sqlx::sqlite::SqliteRow) -> Media {
let id: String = row.get("id");
// `group_concat` gives "3,7" or NULL; queries that do not ask for it at all
// (a fresh upload) get an empty list.
let node_ids = row
.try_get::<Option<String>, _>("node_ids")
.ok()
.flatten()
.map(|joined| joined.split(',').filter_map(|n| n.parse().ok()).collect())
.unwrap_or_default();
Media {
url: format!("/api/media/{id}"),
id,
filename: row.get("filename"),
mime: row.get("mime"),
size: row.get("size"),
created_at: row.get("created_at"),
encrypted: row.try_get("encrypted").unwrap_or(false),
node_ids,
file_node: row.try_get("file_node").ok().flatten(),
}
}
/// The type an upload is stored as: what the browser declared, or what the
/// name suggests when it declared nothing, narrowed by [`sanitize_mime`].
pub(crate) fn mime_of(filename: &str, declared: Option<&str>) -> String {
@@ -200,54 +159,6 @@ pub(crate) fn too_large(state: &AppState) -> AppError {
))
}
pub async fn upload(
_: Session,
State(state): State<AppState>,
mut multipart: Multipart,
) -> AppResult<Json<Vec<Media>>> {
let mut uploaded = Vec::new();
let mut budget = state.config.max_upload_bytes as u64;
while let Some(mut field) = multipart
.next_field()
.await
.map_err(|e| AppError::BadRequest(format!("malformed upload: {e}")))?
{
let filename = field
.file_name()
.map(str::to_string)
.unwrap_or_else(|| "upload".to_string());
let mime = mime_of(&filename, field.content_type());
let id = Uuid::new_v4().to_string();
let size = store_field(&state, &mut field, &id, &mut budget, None).await?;
if size == 0 {
continue;
}
let row = sqlx::query(
"INSERT INTO media (id, filename, mime, size, created_at)
VALUES (?1, ?2, ?3, ?4, ?5)
RETURNING id, filename, mime, size, created_at",
)
.bind(&id)
.bind(&filename)
.bind(&mime)
.bind(size as i64)
.bind(crate::now())
.fetch_one(&state.db)
.await?;
uploaded.push(row_to_media(&row));
}
if uploaded.is_empty() {
return Err(AppError::BadRequest("no files in upload".into()));
}
state.backup.signal();
Ok(Json(uploaded))
}
#[derive(Deserialize)]
pub struct UploadQuery {
/// Seal the files into the vault as they arrive.
@@ -256,7 +167,9 @@ pub struct UploadQuery {
}
/// `POST /api/nodes/{id}/files` — every file in the body becomes a file node
/// in that space or folder, under the name it was uploaded with.
/// in that space or folder, or of that document, under the name it was
/// uploaded with. A picture pasted while writing comes this way too, and the
/// document then embeds it by its `/api/media/<id>`.
/// `?encrypt=true` seals them on the way in, which needs the vault open.
pub async fn upload_into(
_: Session,
@@ -265,8 +178,9 @@ pub async fn upload_into(
Query(query): Query<UploadQuery>,
mut multipart: Multipart,
) -> AppResult<Json<Vec<nodes::Node>>> {
nodes::container(&mut *state.db.acquire().await?, parent).await?;
// A sealed folder seals what lands in it, asked or not.
nodes::place_for(&mut *state.db.acquire().await?, parent, nodes::FILE).await?;
// A sealed folder seals what lands in it, asked or not, and so does a
// document in one.
let key = if query.encrypt || seal::sealed(&state, parent).await? {
Some(state.vault.key().ok_or(AppError::Locked)?)
} else {
@@ -356,7 +270,7 @@ pub(crate) async fn insert_file(
// Checked inside the transaction that takes it, so the slug is still
// free when the node is written; the folder too, which may have gone
// while the upload was streaming in.
nodes::container(&mut tx, parent).await?;
nodes::place_for(&mut tx, parent, nodes::FILE).await?;
let slug = nodes::unique_slug(&mut tx, Some(parent), &nodes::slugify(filename), None).await?;
let now = crate::now();
@@ -397,36 +311,6 @@ pub(crate) async fn insert_file(
Ok(id)
}
#[derive(Deserialize)]
pub struct ListQuery {
#[serde(default)]
pub limit: Option<i64>,
#[serde(default)]
pub offset: Option<i64>,
}
/// A page of the media list, newest first; the client asks for pages until
/// one comes back short.
pub async fn list(
_: Session,
State(state): State<AppState>,
Query(page): Query<ListQuery>,
) -> AppResult<Json<Vec<Media>>> {
let rows = sqlx::query(
"SELECT m.id, m.filename, m.mime, m.size, m.created_at, m.encrypted,
(SELECT f.id FROM nodes f WHERE f.media_id = m.id) AS file_node,
(SELECT group_concat(nm.node_id)
FROM node_media nm JOIN nodes d ON d.id = nm.node_id
WHERE nm.media_id = m.id AND d.kind <> 'file') AS node_ids
FROM media m ORDER BY m.created_at DESC, m.id LIMIT ?1 OFFSET ?2",
)
.bind(page.limit.unwrap_or(500).clamp(1, 2000))
.bind(page.offset.unwrap_or(0).max(0))
.fetch_all(&state.db)
.await?;
Ok(Json(rows.iter().map(row_to_media).collect()))
}
pub async fn serve(
_: Session,
State(state): State<AppState>,
@@ -475,16 +359,6 @@ fn byte_range(
Ok(Some(range))
}
pub async fn remove(
_: Session,
State(state): State<AppState>,
Path(id): Path<String>,
) -> AppResult<Json<serde_json::Value>> {
delete_media(&state, &id).await?;
state.backup.signal();
Ok(Json(serde_json::json!({ "ok": true })))
}
/// Shared by the private and the public-share media handlers. A sealed file
/// is only ever served to the owner, and only while the vault is open — a
/// share link is a door for strangers, and the vault is not behind it.
@@ -665,36 +539,6 @@ pub(crate) fn canonical_id(id: &str) -> Option<String> {
Uuid::try_parse(id).ok().map(|uuid| uuid.to_string())
}
pub async fn delete_media(state: &AppState, id: &str) -> AppResult<()> {
let id = canonical_id(id).ok_or(AppError::NotFound)?;
let mut tx = crate::db::begin_write(&state.db).await?;
// A file in the tree is its node's to delete: going round it here would
// take the node out of its folder without the tree having been asked.
let in_tree = sqlx::query("SELECT 1 FROM nodes WHERE media_id = ?1")
.bind(&id)
.fetch_optional(&mut *tx)
.await?
.is_some();
if in_tree {
return Err(AppError::BadRequest(
"that is a file in the tree — delete it from its folder".into(),
));
}
let deleted = sqlx::query("DELETE FROM media WHERE id = ?1")
.bind(&id)
.execute(&mut *tx)
.await?
.rows_affected();
// No row, no file of ours: nothing on disk is removed on the word of a
// request alone.
if deleted == 0 {
return Err(AppError::NotFound);
}
tx.commit().await?;
remove_upload(&state.config.uploads_dir(), &id).await;
Ok(())
}
/// The bytes of an upload whose row is already gone. Called after the
/// commit, never before: a file a crash leaves behind is swept up later as an
/// orphan, where a row whose file had gone first would be a broken link.
+1 -5
View File
@@ -111,11 +111,7 @@ pub fn api_router(max_upload: usize) -> Router<AppState> {
.route("/vault/lock", post(vault::lock))
.route("/nodes/{id}/share", post(share::enable))
.route("/nodes/{id}/share", delete(share::disable))
.route(
"/media",
post(media::upload).layer(uploads).get(media::list),
)
.route("/media/{id}", get(media::serve).delete(media::remove))
.route("/media/{id}", get(media::serve))
.route("/share/{token}", post(share::read))
.route("/share/{token}/media/{id}", get(share::serve_media))
.route("/share/{token}/typst", post(share::typeset))
+185 -9
View File
@@ -42,6 +42,18 @@ pub fn is_container(kind: &str) -> bool {
kind == SPACE || kind == FOLDER
}
/// Whether a node of kind `parent` may hold one of kind `child`: a space or a
/// folder anything but a space, and a document its own files — the picture
/// pasted into it, the PDF it is about — so they move, seal, go to the trash
/// and come back with it.
pub fn holds(parent: &str, child: &str) -> bool {
match parent {
SPACE | FOLDER => child != SPACE,
DOCUMENT => child == FILE,
_ => false,
}
}
/// The bytes behind a file node, as the client needs them to show or fetch it.
#[derive(Serialize)]
pub struct FileInfo {
@@ -105,8 +117,13 @@ pub struct Node {
pub updated_at: i64,
pub shared: bool,
pub share_token: Option<String>,
/// For a document, whether the files it is given are sealed: the folder
/// it is in decides.
pub sealed: bool,
pub file: Option<FileInfo>,
/// A document's own files, oldest first: what was pasted into it, and
/// what was put beside it.
pub files: Vec<NodeSummary>,
/// This node's ancestors, the space first and the node itself last — the
/// breadcrumb, and everything needed to build its /n/... path.
pub path: Vec<Crumb>,
@@ -1041,7 +1058,6 @@ async fn movable(
ids: &[i64],
parent: i64,
) -> AppResult<Vec<(i64, String, Option<i64>)>> {
container(&mut *db, parent).await?;
let mut nodes = Vec::with_capacity(ids.len());
for &id in ids {
if id == parent {
@@ -1061,6 +1077,7 @@ async fn movable(
"{name} is a space, and a space cannot be moved into anything"
)));
}
place_for(&mut *db, parent, &kind).await?;
// Moving a folder into its own subtree would cut that subtree loose
// from every root, where nothing could reach it again.
if descendants(&mut *db, id).await?.contains(&parent) {
@@ -1076,16 +1093,24 @@ async fn movable(
/// Refuse anything but a space or a folder as the place a node goes into —
/// and one in the trash, where whatever went in would go when it is purged.
pub async fn container(db: &mut SqliteConnection, id: i64) -> AppResult<()> {
let kind: String = sqlx::query("SELECT kind FROM nodes WHERE id = ?1")
place_for(db, id, FOLDER).await
}
/// Refuse `id` as the place a node of `kind` goes into: one that cannot hold
/// it, by [`holds`], or one in the trash.
pub async fn place_for(db: &mut SqliteConnection, id: i64, kind: &str) -> AppResult<()> {
let parent: String = sqlx::query("SELECT kind FROM nodes WHERE id = ?1")
.bind(id)
.fetch_optional(&mut *db)
.await?
.ok_or(AppError::NotFound)?
.get("kind");
if !is_container(&kind) {
return Err(AppError::BadRequest(format!(
"a {kind} cannot contain other nodes"
)));
if !holds(&parent, kind) {
return Err(AppError::BadRequest(if parent == DOCUMENT {
"a document holds only its own files".into()
} else {
format!("a {parent} cannot contain other nodes")
}));
}
if trash::in_trash(&mut *db, id).await? {
return Err(AppError::BadRequest(
@@ -1157,6 +1182,21 @@ pub async fn load(state: &AppState, id: i64) -> AppResult<Node> {
boards::cards_of(&mut conn, id).await?,
)
};
let (files, sealed) = if summary.kind == DOCUMENT {
let files = sqlx::query(&format!(
"SELECT {SUMMARY_COLUMNS} FROM nodes n
WHERE n.parent_id = ?1 AND n.kind = 'file' ORDER BY n.created_at, n.id"
))
.bind(id)
.fetch_all(&state.db)
.await?;
(
files.iter().map(|row| row_to_summary(row, "")).collect(),
crate::routes::seal::sealed(state, id).await?,
)
} else {
(Vec::new(), summary.sealed)
};
Ok(Node {
id: summary.id,
@@ -1174,8 +1214,9 @@ pub async fn load(state: &AppState, id: i64) -> AppResult<Node> {
updated_at: summary.updated_at,
shared: summary.shared,
share_token: summary.share_token,
sealed: summary.sealed,
sealed,
file: summary.file,
files,
path: path_of(&state.db, id).await?,
backlinks,
cards,
@@ -1188,8 +1229,8 @@ mod tests {
use std::collections::BTreeSet;
use super::{
excerpt, fts_query, next_slug, purge_subtree, recent_documents, slugify, EXCERPT_CHARS,
HEAD_CHARS,
excerpt, fts_query, load, move_nodes, next_slug, place_for, purge_subtree,
recent_documents, slugify, DOCUMENT, EXCERPT_CHARS, FILE, FOLDER, HEAD_CHARS,
};
use crate::error::AppError;
@@ -1321,6 +1362,141 @@ mod tests {
assert_eq!(slugs, ["b", "c"]);
}
/// 0029: a file a document embeds, and that was in the tree nowhere,
/// becomes that document's file — a living document's before one in the
/// trash — under the same media id. A file already in the tree stays
/// where it is, and one nothing embeds is left to the sweep.
#[tokio::test]
async fn embedded_files_become_the_files_of_their_document() {
let db = sqlx::sqlite::SqlitePoolOptions::new()
.max_connections(1)
.connect("sqlite::memory:")
.await
.expect("an in-memory database");
let mut before = sqlx::migrate!("./migrations");
before.migrations = before
.migrations
.iter()
.filter(|m| m.version < 29)
.cloned()
.collect();
before.run(&db).await.expect("migrations up to 0028");
let node = |parent: &'static str, kind: &'static str, slug: &'static str| {
let db = db.clone();
async move {
sqlx::query_scalar::<_, i64>(
"INSERT INTO nodes (parent_id, kind, name, slug, body, created_at, updated_at)
VALUES ((SELECT id FROM nodes WHERE slug = ?1), ?2, ?3, ?3, '', 1, 1)
RETURNING id",
)
.bind(parent)
.bind(kind)
.bind(slug)
.fetch_one(&db)
.await
.expect("insert")
}
};
let gone = node(".trash", "document", "gone").await;
let plan = node("uni", "document", "plan").await;
for (id, name) in [("a0", "both.png"), ("b0", "gone.png"), ("c0", "loose.png")] {
sqlx::query(
"INSERT INTO media (id, filename, mime, size, created_at)
VALUES (?1 || '000000-0000-0000-0000-000000000000', ?2, 'image/png', 1, 1)",
)
.bind(id)
.bind(name)
.execute(&db)
.await
.expect("media");
}
for (doc, media) in [(gone, "a0"), (plan, "a0"), (gone, "b0")] {
sqlx::query(
"INSERT INTO node_media (node_id, media_id)
VALUES (?1, ?2 || '000000-0000-0000-0000-000000000000')",
)
.bind(doc)
.bind(media)
.execute(&db)
.await
.expect("embedded");
}
sqlx::migrate!("./migrations")
.run(&db)
.await
.expect("0029 applies");
let files: Vec<(String, i64, String)> = sqlx::query_as(
"SELECT n.name, n.parent_id, n.slug FROM nodes n WHERE n.kind = 'file' ORDER BY n.name",
)
.fetch_all(&db)
.await
.expect("files");
assert_eq!(
files,
[
("both.png".into(), plan, "file-a0000000".into()),
("gone.png".into(), gone, "file-b0000000".into()),
]
);
let owned: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM node_media nm JOIN nodes f ON f.id = nm.node_id
WHERE f.kind = 'file' AND nm.media_id = f.media_id",
)
.fetch_one(&db)
.await
.expect("count");
assert_eq!(owned, 2, "each file owns its bytes");
}
/// A document holds its own files and nothing else; a folder holds both.
#[tokio::test]
async fn a_document_holds_files_only() {
let state = crate::routes::tests::state_in(".".into()).await;
let mut conn = state.db.acquire().await.expect("a connection");
let space: i64 = sqlx::query_scalar("SELECT id FROM nodes WHERE slug = 'uni'")
.fetch_one(&mut *conn)
.await
.expect("the seeded space");
let doc: i64 = sqlx::query_scalar(
"INSERT INTO nodes (parent_id, kind, name, slug, body, created_at, updated_at)
VALUES (?1, 'document', 'Plan', 'plan', '', 1, 1) RETURNING id",
)
.bind(space)
.fetch_one(&mut *conn)
.await
.expect("a document");
place_for(&mut conn, doc, FILE)
.await
.expect("a file in a document");
place_for(&mut conn, space, FILE)
.await
.expect("a file in a space");
for kind in [DOCUMENT, FOLDER] {
assert!(place_for(&mut conn, doc, kind).await.is_err(), "{kind}");
}
drop(conn);
crate::routes::media::insert_file(
&state.db,
doc,
"f0000000-0000-0000-0000-000000000000",
"a.png",
"image/png",
1,
false,
)
.await
.expect("a file of the document");
let loaded = load(&state, doc).await.expect("load");
assert_eq!(loaded.files.len(), 1);
assert_eq!(loaded.files[0].name, "a.png");
assert_eq!(loaded.child_count, 1);
assert!(move_nodes(&state, &[doc], doc, None).await.is_err());
}
/// 0010 widens a CHECK by editing the stored schema. Every connection —
/// not just the one that ran it — has to accept a file node afterwards,
/// the database has to stay intact, and a file an old import hung on a
+14 -4
View File
@@ -117,7 +117,9 @@ pub async fn enter(state: &AppState, id: i64, parent: i64) -> AppResult<()> {
// sealed ones — needs nothing, not even the vault open.
let ready: Option<bool> = sqlx::query_scalar(
"SELECT CASE WHEN n.kind = 'file' THEN m.encrypted
WHEN n.kind = 'document' THEN 1
WHEN n.kind = 'document' THEN NOT EXISTS (
SELECT 1 FROM nodes f JOIN media fm ON fm.id = f.media_id
WHERE f.parent_id = n.id AND fm.encrypted = 0)
ELSE n.sealed END
FROM nodes n LEFT JOIN media m ON m.id = n.media_id WHERE n.id = ?1",
)
@@ -126,7 +128,8 @@ pub async fn enter(state: &AppState, id: i64, parent: i64) -> AppResult<()> {
.await?;
match ready {
None => return Err(AppError::NotFound),
// A document is not sealed; it moves in as it is.
// A document's text is not sealed, so it moves in as it is once its
// files are.
Some(true) => return Ok(()),
Some(false) => {}
}
@@ -145,15 +148,22 @@ pub async fn enter(state: &AppState, id: i64, parent: i64) -> AppResult<()> {
Ok(())
}
/// Whether files that land in this folder or space are sealed.
/// Whether files that land in this folder or space are sealed. A document
/// keeps its files under the rules of the folder it is in.
pub async fn sealed(state: &AppState, id: i64) -> AppResult<bool> {
Ok(sqlx::query_scalar("SELECT sealed FROM nodes WHERE id = ?1")
Ok(sqlx::query_scalar(SEALS)
.bind(id)
.fetch_optional(&state.db)
.await?
.unwrap_or(false))
}
/// [`sealed`] as one statement, `?1` the node, for callers with a pool of
/// their own.
pub(crate) const SEALS: &str =
"SELECT CASE WHEN n.kind = 'document' THEN p.sealed ELSE n.sealed END
FROM nodes n LEFT JOIN nodes p ON p.id = n.parent_id WHERE n.id = ?1";
async fn place(state: &AppState, id: i64) -> AppResult<(String, Option<i64>)> {
let row = sqlx::query("SELECT kind, parent_id FROM nodes WHERE id = ?1")
.bind(id)
+6 -4
View File
@@ -257,7 +257,7 @@ pub async fn restore_node(state: &AppState, id: i64) -> AppResult<bool> {
let from_parent: Option<i64> = row.get("from_parent");
let from_space: Option<i64> = row.get("from_space");
let (target, elsewhere) = match from_parent {
Some(parent) if is_live_container(&mut conn, parent).await? => (parent, false),
Some(parent) if is_live_place(&mut conn, parent, &kind).await? => (parent, false),
_ => (fallback(&mut conn, from_space).await?, true),
};
drop(conn);
@@ -273,12 +273,14 @@ pub async fn restore_node(state: &AppState, id: i64) -> AppResult<bool> {
Ok(elsewhere)
}
async fn is_live_container(db: &mut SqliteConnection, id: i64) -> AppResult<bool> {
let kind: Option<String> = sqlx::query_scalar("SELECT kind FROM nodes WHERE id = ?1")
/// Whether `id` is still there to take back a node of `kind`: a folder, or
/// the document a file belonged to.
async fn is_live_place(db: &mut SqliteConnection, id: i64, kind: &str) -> AppResult<bool> {
let parent: Option<String> = sqlx::query_scalar("SELECT kind FROM nodes WHERE id = ?1")
.bind(id)
.fetch_optional(&mut *db)
.await?;
Ok(kind.is_some_and(|kind| nodes::is_container(&kind)) && !in_trash(db, id).await?)
Ok(parent.is_some_and(|parent| nodes::holds(&parent, kind)) && !in_trash(db, id).await?)
}
/// Where a node goes back to when its folder is gone: the root of its space,
+175 -27
View File
@@ -2,8 +2,10 @@
//! and the files beside them. Directories are spaces and folders; documents
//! are `<slug>.md`, and file nodes appear under their own name — the one a
//! download of them gets — with a later file that shares it told apart by its
//! id, `slides (42).pdf`. There is no on-disk mirror — every call reads or
//! writes `nodes` directly, so a save here is a save in the browser too.
//! id, `slides (42).pdf`. A document's own files are in `<slug>/` beside it,
//! which is listed while it holds any and can be written into either way.
//! There is no on-disk mirror — every call reads or writes `nodes` directly,
//! so a save here is a save in the browser too.
//!
//! A document reads as `# <title>` followed by its body, the same shape the
//! export writes, and a leading heading written back becomes the title again.
@@ -71,6 +73,8 @@ enum Target {
},
/// A document, `<slug>.md`.
Doc(Doc),
/// A document's files, `<slug>/`: a directory that holds files only.
Files { id: i64, updated_at: i64 },
/// A file node.
File(FileNode),
/// A session-only file, keyed by its normalised path.
@@ -388,7 +392,7 @@ impl SpaceFs {
parent: i64,
over_sealed: bool,
) -> SftpResult<Option<Arc<vault::Keyring>>> {
let sealed: bool = sqlx::query_scalar("SELECT sealed FROM nodes WHERE id = ?1")
let sealed: bool = sqlx::query_scalar(seal::SEALS)
.bind(parent)
.fetch_optional(&self.db)
.await
@@ -529,12 +533,12 @@ impl SpaceFs {
.map_err(|_| StatusCode::Failure)?;
let mut taken: std::collections::HashSet<String> = others
.iter()
.map(|row| {
.flat_map(|row| {
let slug: String = row.get("slug");
if row.get::<String, _>("kind") == nodes::DOCUMENT {
format!("{slug}.md")
vec![format!("{slug}.md"), slug]
} else {
slug
vec![slug]
}
})
.collect();
@@ -650,19 +654,61 @@ impl SpaceFs {
Ok(None)
}
/// The files of the document under `parent` whose slug is `leaf`, the
/// directory beside its `.md`. Every document has one, empty or not.
async fn files_of(&self, parent: Option<i64>, leaf: &str) -> SftpResult<Option<Target>> {
let Some(parent) = parent else {
return Ok(None);
};
let slugged = nodes::slugify(leaf);
for slug in [leaf, slugged.as_str()] {
let row = sqlx::query(
"SELECT id, updated_at FROM nodes
WHERE parent_id = ?1 AND slug = ?2 AND kind = 'document'",
)
.bind(parent)
.bind(slug)
.fetch_optional(&self.db)
.await
.map_err(|_| StatusCode::Failure)?;
if let Some(row) = row {
return Ok(Some(Target::Files {
id: row.get("id"),
updated_at: row.get("updated_at"),
}));
}
}
Ok(None)
}
/// Resolve a path of directory segments to a node id, `None` meaning the
/// root.
/// root. The last may be a document's files.
async fn resolve_dir(&self, segs: &[&str]) -> SftpResult<Option<i64>> {
let mut current: Option<i64> = None;
for seg in segs {
match self.dir_in(current, seg).await? {
Some(Target::Dir { id, .. }) => current = Some(id),
let found = match self.dir_in(current, seg).await? {
Some(found) => Some(found),
None => self.files_of(current, seg).await?,
};
match found {
Some(Target::Dir { id, .. } | Target::Files { id, .. }) => current = Some(id),
_ => return Err(StatusCode::NoSuchFile),
}
}
Ok(current)
}
/// Refuse `parent` as the place for a node of `kind`: a document holds
/// files and nothing else.
async fn may_hold(&self, parent: i64, kind: &str) -> SftpResult<()> {
nodes::place_for(&mut *self.conn().await?, parent, kind)
.await
.map_err(|e| match e {
AppError::NotFound => StatusCode::NoSuchFile,
_ => StatusCode::PermissionDenied,
})
}
async fn target(&self, path: &str) -> SftpResult<Target> {
let (dir, leaf) = split_path(path);
let Some(leaf) = leaf else {
@@ -679,6 +725,9 @@ impl SpaceFs {
if let Some(doc) = self.doc_in(parent, leaf).await? {
return Ok(Target::Doc(doc));
}
if let Some(files) = self.files_of(Some(parent), leaf).await? {
return Ok(files);
}
if let Some(file) = self.file_in(parent, leaf).await? {
return Ok(Target::File(file));
}
@@ -702,7 +751,9 @@ impl SpaceFs {
fn attrs_of(&self, target: &Target) -> FileAttributes {
match target {
Target::Root => dir_attrs(crate::now()),
Target::Dir { updated_at, .. } => dir_attrs(*updated_at),
Target::Dir { updated_at, .. } | Target::Files { updated_at, .. } => {
dir_attrs(*updated_at)
}
Target::Doc(doc) => file_attrs(
render_doc(&doc.name, &doc.body).len() as u64,
doc.updated_at,
@@ -725,6 +776,7 @@ impl SpaceFs {
/// An empty document under `parent`, named after the file it was created
/// as until a heading says otherwise.
async fn create_doc(&mut self, parent: i64, stem: &str) -> SftpResult<i64> {
self.may_hold(parent, nodes::DOCUMENT).await?;
let slug = nodes::unique_slug(
&mut *self.conn().await?,
Some(parent),
@@ -1118,7 +1170,9 @@ impl SpaceFs {
let segs = segments(path);
let parent = self.resolve_dir(&segs).await?;
let rows = sqlx::query(&format!(
"SELECT kind, slug, updated_at, {DOC_SIZE} AS size FROM nodes
"SELECT kind, slug, updated_at, {DOC_SIZE} AS size,
EXISTS (SELECT 1 FROM nodes f WHERE f.parent_id = nodes.id) AS has_files
FROM nodes
WHERE parent_id IS ?1 AND kind <> 'file' AND role IS NOT 'trash'
ORDER BY kind, slug"
))
@@ -1127,20 +1181,25 @@ impl SpaceFs {
.await
.map_err(|_| StatusCode::Failure)?;
let mut files: Vec<File> = rows
.iter()
.map(|row| {
let mut files: Vec<File> = Vec::with_capacity(rows.len());
for row in &rows {
let kind: String = row.get("kind");
let slug: String = row.get("slug");
let mtime: i64 = row.get("updated_at");
if kind == nodes::DOCUMENT {
let size: i64 = row.try_get("size").unwrap_or(0);
File::new(format!("{slug}.md"), file_attrs(size as u64, mtime))
files.push(File::new(
format!("{slug}.md"),
file_attrs(size as u64, mtime),
));
// Its files, once it has any; written into, it has.
if row.get::<bool, _>("has_files") {
files.push(File::new(slug, dir_attrs(mtime)));
}
} else {
File::new(slug, dir_attrs(mtime))
files.push(File::new(slug, dir_attrs(mtime)));
}
}
})
.collect();
if let Some(parent) = parent {
for (shown, file) in self.files_in(parent).await? {
@@ -1249,7 +1308,7 @@ impl russh_sftp::server::Handler for SpaceFs {
Target::File(file) if size as i64 != file.size => {
return Err(StatusCode::OpUnsupported)
}
Target::Root | Target::Dir { .. } | Target::File(_) => {}
Target::Root | Target::Dir { .. } | Target::Files { .. } | Target::File(_) => {}
}
}
Ok(ok_status(id))
@@ -1331,7 +1390,9 @@ impl russh_sftp::server::Handler for SpaceFs {
StatusCode::NoSuchFile
});
};
if self.dir_in(Some(parent), leaf).await?.is_some() {
if self.dir_in(Some(parent), leaf).await?.is_some()
|| self.files_of(Some(parent), leaf).await?.is_some()
{
return Err(StatusCode::Failure);
}
@@ -1600,7 +1661,9 @@ impl russh_sftp::server::Handler for SpaceFs {
Target::Scratch(key) => {
self.scratch.remove(&key);
}
Target::Root | Target::Dir { .. } => return Err(StatusCode::PermissionDenied),
Target::Root | Target::Dir { .. } | Target::Files { .. } => {
return Err(StatusCode::PermissionDenied)
}
}
Ok(ok_status(id))
}
@@ -1625,6 +1688,7 @@ impl russh_sftp::server::Handler for SpaceFs {
self.changed();
return Ok(ok_status(id));
};
self.may_hold(parent, nodes::FOLDER).await?;
let slug = nodes::unique_slug(
&mut *self.conn().await?,
Some(parent),
@@ -1650,8 +1714,24 @@ impl russh_sftp::server::Handler for SpaceFs {
async fn rmdir(&mut self, id: u32, path: String) -> SftpResult<russh_sftp::protocol::Status> {
self.authorized().await?;
let Target::Dir { id: node, .. } = self.target(&path).await? else {
return Err(StatusCode::PermissionDenied);
let node = match self.target(&path).await? {
Target::Dir { id, .. } => id,
// A document's files go when the document does; emptied, the
// directory is gone from the listing already.
Target::Files { id: doc, .. } => {
let files: i64 =
sqlx::query_scalar("SELECT count(*) FROM nodes WHERE parent_id = ?1")
.bind(doc)
.fetch_one(&self.db)
.await
.map_err(|_| StatusCode::Failure)?;
return if files > 0 {
Err(StatusCode::Failure)
} else {
Ok(ok_status(id))
};
}
_ => return Err(StatusCode::PermissionDenied),
};
let children: i64 = sqlx::query("SELECT count(*) AS n FROM nodes WHERE parent_id = ?1")
.bind(node)
@@ -1687,7 +1767,8 @@ impl russh_sftp::server::Handler for SpaceFs {
let now = crate::now();
match source {
Target::Root => return Err(StatusCode::PermissionDenied),
// A document's files are renamed and moved with its `.md`.
Target::Root | Target::Files { .. } => return Err(StatusCode::PermissionDenied),
Target::Dir {
id: node,
@@ -1708,6 +1789,7 @@ impl russh_sftp::server::Handler for SpaceFs {
return Err(StatusCode::PermissionDenied)
}
(_, Some(parent)) => {
self.may_hold(parent, &kind).await?;
let below = nodes::descendants(&mut *self.conn().await?, node)
.await
.map_err(|_| StatusCode::Failure)?;
@@ -1754,7 +1836,7 @@ impl russh_sftp::server::Handler for SpaceFs {
return Err(StatusCode::PermissionDenied);
};
match (doc_slug(new_leaf), dest) {
(_, Some(Target::Dir { .. } | Target::File(_))) => {
(_, Some(Target::Dir { .. } | Target::Files { .. } | Target::File(_))) => {
return Err(StatusCode::Failure)
}
// Onto another document: that one takes this one's text
@@ -1765,6 +1847,7 @@ impl russh_sftp::server::Handler for SpaceFs {
self.replace_node(doc.id).await?;
}
(Some(stem), _) => {
self.may_hold(new_parent, nodes::DOCUMENT).await?;
let new_slug = nodes::unique_slug(
&mut *self.conn().await?,
Some(new_parent),
@@ -1813,7 +1896,9 @@ impl russh_sftp::server::Handler for SpaceFs {
return Err(StatusCode::PermissionDenied);
};
match dest {
Some(Target::Dir { .. } | Target::Doc(_)) => return Err(StatusCode::Failure),
Some(Target::Dir { .. } | Target::Files { .. } | Target::Doc(_)) => {
return Err(StatusCode::Failure)
}
Some(Target::File(other)) if other.id == file.id => {}
// Onto another file, as rename(2) does: that one goes.
Some(Target::File(other)) => self.replace_node(other.id).await?,
@@ -1831,7 +1916,9 @@ impl russh_sftp::server::Handler for SpaceFs {
return Err(StatusCode::PermissionDenied);
};
match (doc_slug(new_leaf), dest) {
(_, Some(Target::Dir { .. })) => return Err(StatusCode::Failure),
(_, Some(Target::Dir { .. } | Target::Files { .. })) => {
return Err(StatusCode::Failure)
}
// A temp file renamed over a file, or onto a name worth
// keeping: what a sync tool's atomic write looks like.
(None, Some(Target::File(other))) => {
@@ -1958,6 +2045,67 @@ mod tests {
/// A file put through the mount is a file node, read back from disk, and
/// renamed, replaced and removed the way any file would be.
/// A document's files are a directory beside it, named like it: listed
/// once it holds any, holding files and nothing else, and moving with
/// the document's `.md`.
#[tokio::test]
async fn a_document_keeps_its_files_beside_it() {
let dir = tempfile::tempdir().unwrap();
let mut fs = fs_in(dir.path().to_path_buf()).await;
fs.mkdir(0, "/proj".into(), FileAttributes::empty())
.await
.expect("mkdir");
put(&mut fs, "/proj/plan.md", "# Plan\n\ntext").await;
async fn names(fs: &SpaceFs, path: &str) -> Vec<String> {
let mut names: Vec<String> = fs
.listing(path)
.await
.unwrap()
.into_iter()
.map(|f| f.filename)
.collect();
names.sort();
names
}
assert_eq!(names(&fs, "/proj").await, ["plan.md"]);
put(&mut fs, "/proj/plan/photo.png", "png").await;
assert_eq!(names(&fs, "/proj").await, ["plan", "plan.md"]);
assert_eq!(names(&fs, "/proj/plan").await, ["photo.png"]);
assert_eq!(get(&mut fs, "/proj/plan/photo.png").await, "png");
let (doc, _, _) = node(&fs, "plan").await;
let parent: i64 = sqlx::query_scalar("SELECT parent_id FROM nodes WHERE kind = 'file'")
.fetch_one(&fs.db)
.await
.unwrap();
assert_eq!(parent, doc);
// Files, and nothing else.
assert!(fs
.mkdir(0, "/proj/plan/sub".into(), FileAttributes::empty())
.await
.is_err());
assert!(open_at(&mut fs, "/proj/plan/inner.md", CREATE)
.await
.is_err());
fs.rename(0, "/proj/plan.md".into(), "/proj/roadmap.md".into())
.await
.expect("rename");
assert_eq!(names(&fs, "/proj").await, ["roadmap", "roadmap.md"]);
assert_eq!(get(&mut fs, "/proj/roadmap/photo.png").await, "png");
fs.remove(0, "/proj/roadmap/photo.png".into())
.await
.expect("remove");
assert_eq!(names(&fs, "/proj").await, ["roadmap.md"]);
fs.rmdir(0, "/proj/roadmap".into())
.await
.expect("rmdir of no files");
assert_eq!(names(&fs, "/proj").await, ["roadmap.md"]);
}
#[tokio::test]
async fn files_are_nodes_on_disk() {
let dir = tempfile::tempdir().unwrap();
+5 -4
View File
@@ -18,7 +18,6 @@
import Verify from './lib/Verify.svelte'
import Login from './lib/Login.svelte'
import Overview from './lib/Overview.svelte'
import MediaBrowser from './lib/MediaBrowser.svelte'
import Trash from './lib/Trash.svelte'
import CardTile from './lib/CardTile.svelte'
import Scratch from './lib/Scratch.svelte'
@@ -62,7 +61,6 @@
verify: Verify,
info: Info,
trash: Trash,
media: MediaBrowser,
}
let boardEl = $state<HTMLDivElement>()
/** The first half of `gg`, `dd`, `]]` …, shown until the second comes. */
@@ -195,8 +193,11 @@
input.value = ''
if (files.length === 0) return
await workspace.guard(async () => {
const uploaded = await workspace.upload(files)
hooks.insertText(uploaded.map(embedSnippet).join('\n\n'))
// While writing, they go in at the cursor too; while reading, the
// document only holds them.
const writing = workspace.editing
const created = await workspace.attach(files)
if (writing && created.length > 0) hooks.insertText(created.map(embedSnippet).join('\n\n'))
})
}
+6 -2
View File
@@ -153,7 +153,8 @@
<!--
The readout is the way to everything else about the document: its
headings, its history, what links to it and who can read it — gi.
headings, its history, what links to it, its files and who can read
it — gi.
-->
<button class="meta" title="info (gi)" onclick={() => openInfo()}>
<span class="field"><span class="caption">written</span>{formatStamp(tile.draft.created_at)} {formatWeekday(tile.draft.created_at)}</span>
@@ -162,6 +163,9 @@
{#if doc.backlinks > 0}
<span class="field"><span class="caption">linked</span>from {doc.backlinks}</span>
{/if}
{#if doc.files.length > 0}
<span class="field"><span class="caption">files</span>{doc.files.length}</span>
{/if}
{#if doc.shared}
<span class="field public"><span class="ring"></span>public</span>
{/if}
@@ -207,7 +211,7 @@
</div>
{#if isTouchDevice}
<div class="mobilebar">
<button onclick={() => runCommand('upload')}>+ media</button>
<button onclick={() => runCommand('upload')}>+ file</button>
<button onclick={() => runCommand('w')}>save</button>
<button onclick={() => runCommand('wq')}>done</button>
</div>
+2 -2
View File
@@ -248,8 +248,8 @@
async function uploadFiles(files: File[]) {
if (files.length === 0) return
await workspace.guard(async () => {
const uploaded = await workspace.upload(files)
insertAtCursor(uploaded.map(embedSnippet).join('\n\n'))
const created = await workspace.attach(files, tile)
if (created.length > 0) insertAtCursor(created.map(embedSnippet).join('\n\n'))
})
}
+44 -3
View File
@@ -1,5 +1,7 @@
<script lang="ts" module>
import type { AgendaCard, Backlink, NodeSummary, RevisionSummary } from './api'
import { hooks } from './commands'
import { embedSnippet } from './markdown'
import { workspace, type InfoSection } from './store.svelte'
import { formatStamp } from './util'
@@ -7,9 +9,9 @@
* What the keys in actions.ts do here too, so a key and a click are the
* same thing. The info sheet holds what is known about the open document
* besides its text: its headings, its earlier versions, what links to it,
* the cards about it, and whether it is shared.
* the cards about it, its own files, and whether it is shared.
*/
export const infoSections: InfoSection[] = ['outline', 'history', 'backlinks', 'cards', 'share']
export const infoSections: InfoSection[] = ['outline', 'history', 'backlinks', 'cards', 'files', 'share']
/** Tab and Shift-Tab: the next part along, round the ends. */
export function turnInfo(by: number) {
@@ -38,6 +40,25 @@
if (index !== workspace.historyCursor) void workspace.guard(() => workspace.showRevision(index))
}
/** Enter on one of the document's files: the file is opened. */
export function openFile(file: NodeSummary) {
workspace.closeSheet()
return workspace.guard(() => workspace.openNode(file.id))
}
/** `i`: the file embedded at the cursor, while the document is being written. */
export function embedFile(file: NodeSummary) {
if (!workspace.editing) return workspace.say('embedding works while writing — i on the document first', 'error')
hooks.insertText(embedSnippet(file))
workspace.closeSheet()
}
/** `y`: how to embed it, to paste into another document. */
export async function copyEmbed(file: NodeSummary) {
const copied = await workspace.copy(embedSnippet(file))
workspace.say(copied ? `copied how to embed ${file.name}` : 'the clipboard said no', copied ? 'info' : 'error')
}
/** Enter on a backlink: the linking document is opened. */
export function openBacklink(link: Backlink) {
workspace.closeSheet()
@@ -80,7 +101,7 @@
<script lang="ts">
import Sheet from './Sheet.svelte'
import { diffLines, folded } from './diff'
import { formatDay, relative } from './util'
import { formatBytes, formatDay, relative } from './util'
const doc = $derived(workspace.open)
@@ -116,6 +137,7 @@
['history', `history ${workspace.history.length}`],
['backlinks', `linked from ${workspace.backlinks.length}`],
['cards', `cards ${doc?.cards.filter((card) => card.done_at === null).length ?? 0}`],
['files', `files ${doc?.files.length ?? 0}`],
['share', doc?.shared ? 'shared' : 'private'],
])
@@ -125,6 +147,7 @@
history: workspace.historyCursor,
backlinks: workspace.backlinksCursor,
cards: workspace.cardsCursor,
files: workspace.filesCursor,
share: undefined,
}[section],
)
@@ -232,6 +255,24 @@
<li class="muted">no card about this document — c makes one, and a card's [[title]] counts too.</li>
{/each}
</ul>
{:else if section === 'files'}
<ul class="items">
{#each doc?.files ?? [] as file, index (file.id)}
<!-- svelte-ignore a11y_click_events_have_key_events, a11y_no_noninteractive_element_interactions -->
<li
class="item"
data-index={index}
class:at={index === workspace.filesCursor}
onpointerenter={() => (workspace.filesCursor = index)}
onclick={() => openFile(file)}
>
<span class="name">{file.name}</span>
<span class="muted">{file.file ? `${file.file.mime} · ${formatBytes(file.file.size)}` : ''}{file.file?.encrypted ? ' · sealed' : ''}</span>
</li>
{:else}
<li class="muted">no files of its own — a picture pasted or dropped while writing is one, and so is :upload while reading it.</li>
{/each}
</ul>
{:else}
<div class="share">
{#if sharePath}
-105
View File
@@ -1,105 +0,0 @@
<script lang="ts" module>
import { api, type MediaFile } from './api'
import { hooks } from './commands'
import { embedSnippet } from './markdown'
import { workspace } from './store.svelte'
/*
* What a row's buttons do, out here so the keys in App.svelte do the same:
* i inserts, y copies, x deletes.
*/
export function insertMedia(file: MediaFile) {
if (!workspace.editing) return workspace.say('insert works while writing — i on a document first', 'error')
hooks.insertText(embedSnippet(file))
workspace.closeSheet()
}
export async function copyMedia(file: MediaFile) {
const copied = await workspace.copy(location.origin + file.url)
workspace.say(copied ? `copied the address of ${file.filename}` : 'the clipboard said no', copied ? 'info' : 'error')
}
export async function removeMedia(file: MediaFile) {
if (file.file_node !== null) return workspace.say(`${file.filename} is a file in the tree — delete it there`, 'error')
const used = file.node_ids.length === 1 ? '1 document' : `${file.node_ids.length} documents`
const warning = file.node_ids.length > 0 ? ` it is embedded in ${used}, which will break` : ''
if (!(await workspace.ask(`delete ${file.filename}?${warning}`))) return
await workspace.guard(async () => {
await api.deleteMedia(file.id)
await workspace.loadMedia()
workspace.mediaCursor = Math.min(workspace.mediaCursor, Math.max(workspace.media.length - 1, 0))
})
}
</script>
<script lang="ts">
import Sheet from './Sheet.svelte'
import { formatBytes, formatStamp } from './util'
const usage = (nodeIds: number[], fileNode: number | null) => {
const embeds = nodeIds.map((id) => `#${id}`).join(' ')
if (fileNode !== null) return embeds ? `in the tree · ${embeds}` : 'in the tree'
return embeds || 'unattached'
}
</script>
<Sheet title="media" detail="{workspace.media.length} files" cursor={workspace.mediaCursor} keys={['media', 'sheet']} hide={workspace.editing ? [] : ['insert']}>
<ul class="items">
{#each workspace.media as file, index (file.id)}
<!-- svelte-ignore a11y_no_noninteractive_element_interactions -->
<li class="item" class:at={index === workspace.mediaCursor} data-index={index} onpointerenter={() => (workspace.mediaCursor = index)}>
<div class="thumb">
{#if file.encrypted}
<span class="label">sealed</span>
{:else if file.mime.startsWith('image/')}
<img src={file.url} alt={file.filename} loading="lazy" />
{:else}
<span class="label">{file.mime.split('/')[0]}</span>
{/if}
</div>
<div class="info">
<span class="name">{file.filename}</span>
<span class="muted">
{formatBytes(file.size)} · {formatStamp(file.created_at)} ·
{usage(file.node_ids, file.file_node)}
</span>
</div>
<div class="actions">
{#if workspace.editing}
<button onclick={() => insertMedia(file)}>insert</button>
{/if}
<button onclick={() => copyMedia(file)}>copy url</button>
{#if file.file_node === null}
<button class="danger" onclick={() => removeMedia(file)}>rm</button>
{/if}
</div>
</li>
{:else}
<li class="muted">nothing uploaded yet — attach a file from the editor.</li>
{/each}
</ul>
</Sheet>
<style>
.items > .item { flex-direction: row; align-items: center; gap: 14px; cursor: default; }
.thumb {
width: 52px;
height: 38px;
flex: none;
display: grid;
place-items: center;
background: var(--surface);
border: 1px solid var(--rim);
border-radius: var(--radius-s);
overflow: hidden;
}
.thumb .label { font-size: 9px; letter-spacing: 0.14em; }
.thumb img { width: 100%; height: 100%; object-fit: cover; }
.info { display: flex; flex-direction: column; min-width: 0; flex: 1; font-size: 12px; }
.actions { display: flex; gap: 4px; flex: none; }
</style>
+19 -5
View File
@@ -20,14 +20,20 @@
return node.name.trim() || node.excerpt.trim() || node.slug
}
/* A folder is listed by what is in it, a document by when it was written, a file by its size. */
/*
* A folder is listed by what is in it, a document by when it was written —
* and how many files it has, folded or not — a file by its size.
*/
const aside = (node: NodeSummary) =>
node.file
? formatBytes(node.file.size)
: node.kind === 'document'
? formatTime(node.created_at)
? `${node.child_count > 0 ? `+${node.child_count} · ` : ''}${formatTime(node.created_at)}`
: `${node.child_count}`
/* The row a day heading compares with: the one above, past a document's files. */
const above = (index: number) => workspace.nodes.slice(0, index).filter((n) => !workspace.isNested(n)).at(-1)
const carriesFiles = (event: DragEvent) => event.dataTransfer?.types.includes('Files') ?? false
function drop(event: DragEvent) {
@@ -56,6 +62,9 @@
workspace.openSheet('actions')
}
/* What is in the folder itself, not the files folded open under its documents. */
const items = $derived(workspace.nodes.filter((node) => !workspace.isNested(node)).length)
/* Inside a folder, a phone has no h key and no crumbs on the bar: this is the way up. */
const nested = $derived(isTouchDevice && !workspace.query && workspace.path.length > 1)
</script>
@@ -78,7 +87,7 @@
<span class="marker"></span>
<span class="name">{workspace.space?.slug ?? 'diary'}</span>
<span class="count">
{workspace.nodes.length} {workspace.nodes.length === 1 ? 'item' : 'items'}{#if workspace.foundCards.length > 0}
{items} {items === 1 ? 'item' : 'items'}{#if workspace.foundCards.length > 0}
· {workspace.foundCards.length} {workspace.foundCards.length === 1 ? 'card' : 'cards'}{/if}{#if workspace.foundNotes.length > 0}
· {workspace.foundNotes.length} {workspace.foundNotes.length === 1 ? 'note' : 'notes'}{/if}
</span>
@@ -103,10 +112,11 @@
</li>
{/if}
{#each workspace.nodes as node, index (node.id)}
{@const previous = workspace.nodes[index - 1]}
{@const previous = above(index)}
{@const nested = workspace.isNested(node)}
<!-- Day headings only make sense in a list that is a journal: documents,
in date order, not a search and not a folder listing. -->
{#if !workspace.query && node.kind === 'document' && (!previous || previous.kind !== 'document' || formatDay(previous.created_at) !== formatDay(node.created_at))}
{#if !workspace.query && !nested && node.kind === 'document' && (!previous || previous.kind !== 'document' || formatDay(previous.created_at) !== formatDay(node.created_at))}
<li class="daybreak label rule">
<span><span class="day">{formatDay(node.created_at)}</span> {formatWeekday(node.created_at)}</span>
</li>
@@ -127,6 +137,7 @@
class:active={index === workspace.cursor}
class:open={workspace.tiles.some((tile) => tile.open?.id === node.id)}
class:marked={workspace.isMarked(node.id)}
class:nested
data-index={index}
onclick={(event) => click(event, index, node)}
>
@@ -324,6 +335,9 @@
.item { display: flex; align-items: center; }
.more { flex: none; display: grid; place-items: center; width: 40px; height: 40px; padding: 0; border: 0; background: none; color: var(--muted); }
.up .title { font-family: var(--font-mono); }
/* A document's own files, folded open under it. */
.row.nested { padding-left: 28px; }
.row.nested .title { font-size: 13px; }
/* Roomier tap targets where there is no keyboard. */
@media (pointer: coarse) {
+16 -16
View File
@@ -11,9 +11,8 @@
import { cycleTile, focusTile, followLink, hooks, openInfo, runCommand } from './commands'
import { bindings, type Context } from './keymap'
import { focusHelpFilter, scrollHelp } from './Help.svelte'
import { copyMedia, insertMedia, removeMedia } from './MediaBrowser.svelte'
import { purgeTrashed, restoreTrashed } from './Trash.svelte'
import { goHeading, moveHistory, openBacklink, renewLink, restoreShown, toggleSharing, turnInfo } from './Info.svelte'
import { copyEmbed, embedFile, goHeading, moveHistory, openBacklink, openFile, renewLink, restoreShown, toggleSharing, turnInfo } from './Info.svelte'
import { workspace } from './store.svelte'
import { dwindle, formatDay, neighbour, type Direction } from './util'
@@ -34,7 +33,7 @@ const spaceIndex = (key: string) => {
const reading = () => !!workspace.open && !workspace.editing && workspace.pane === 'document'
/** A cursor down a sheet's list, kept on it. */
function step(cursor: 'outlineCursor' | 'mediaCursor' | 'trashCursor' | 'backlinksCursor' | 'cardsCursor', length: number, by: number) {
function step(cursor: 'outlineCursor' | 'filesCursor' | 'trashCursor' | 'backlinksCursor' | 'cardsCursor', length: number, by: number) {
workspace[cursor] = Math.min(Math.max(workspace[cursor] + by, 0), Math.max(length - 1, 0))
}
@@ -163,22 +162,22 @@ export const actions: { [C in Context]: Record<ActionOf<C>, Run> } = {
go: () => goHeading(),
},
/** The media list: j/k down the files, i inserts, y copies, x deletes. */
media: {
down: () => step('mediaCursor', workspace.media.length, 1),
up: () => step('mediaCursor', workspace.media.length, -1),
bottom: () => step('mediaCursor', workspace.media.length, Infinity),
/** Its files: j/k down them, Enter opens one, i embeds it while writing, y copies how. */
files: {
down: () => step('filesCursor', workspace.open?.files.length ?? 0, 1),
up: () => step('filesCursor', workspace.open?.files.length ?? 0, -1),
bottom: () => step('filesCursor', workspace.open?.files.length ?? 0, Infinity),
open: () => {
const file = workspace.open?.files[workspace.filesCursor]
if (file) void openFile(file)
},
insert: () => {
const file = workspace.media[workspace.mediaCursor]
if (file) insertMedia(file)
const file = workspace.open?.files[workspace.filesCursor]
if (file) embedFile(file)
},
copy: () => {
const file = workspace.media[workspace.mediaCursor]
if (file) void copyMedia(file)
},
delete: () => {
const file = workspace.media[workspace.mediaCursor]
if (file) void removeMedia(file)
const file = workspace.open?.files[workspace.filesCursor]
if (file) void copyEmbed(file)
},
},
@@ -386,6 +385,7 @@ export const actions: { [C in Context]: Record<ActionOf<C>, Run> } = {
// Pick nodes up here, walk to a folder, put them down with `p`.
mark: () => workspace.mark(),
markAll: () => workspace.markAll(),
files: () => guard(() => workspace.toggleFiles()),
drop: () => guard(() => workspace.drop()),
new: () => void runCommand('new'),
// Capture: a thought goes on this space's inbox without leaving here.
+5 -24
View File
@@ -57,6 +57,8 @@ export interface WorkspaceNode extends NodeSummary {
cards: AgendaCard[]
/** Where each `[[title]]` in the body leads, as an /n/ path; a title that leads nowhere is missing. */
wikilinks: Record<string, string>
/** A document's own files, oldest first: what was pasted into it, and what was put beside it. */
files: NodeSummary[]
}
/** A document that links to the one asked about, and the line it does it on. */
@@ -256,20 +258,6 @@ export interface Board {
lists: BoardList[]
}
export interface MediaFile {
id: string
filename: string
mime: string
size: number
created_at: number
url: string
encrypted: boolean
/** Every document that embeds this file — a file may be used by several. */
node_ids: number[]
/** The file node it is, when it sits in the tree; deleted from there. */
file_node: number | null
}
/** One thing a certificate, key or keystore file holds, as `/inspect` describes it. */
export type CertItem =
| {
@@ -627,21 +615,14 @@ export const api = {
moveCard: (id: number, list_id: number, position?: number) =>
request<Moved>(`/cards/${id}/move`, { method: 'POST', body: body({ list_id, position }) }),
listMedia: () => pages((limit, offset) => request<MediaFile[]>(`/media?limit=${limit}&offset=${offset}`)),
/**
* Upload files into a space or folder as file nodes. XHR rather than fetch,
* because a gigabyte going up needs to say how far along it is.
* Upload files into a space or folder, or to a document as its own files.
* XHR rather than fetch, because a gigabyte going up needs to say how far
* along it is.
*/
putFiles: (parentId: number, files: File[], encrypt: boolean, progress?: (fraction: number) => void) =>
send<WorkspaceNode[]>(`/api/nodes/${parentId}/files${encrypt ? '?encrypt=true' : ''}`, files, progress),
/** Files to embed in a document, with the same progress as `putFiles`. */
upload: (files: File[], progress?: (fraction: number) => void) =>
send<MediaFile[]>('/api/media', files, progress),
deleteMedia: (id: string) => request<{ ok: true }>(`/media/${id}`, { method: 'DELETE' }),
vaultStatus: () => request<VaultStatus>('/vault'),
setupVault: (password: string) =>
+10 -15
View File
@@ -93,11 +93,16 @@ export async function openInfo(section?: InfoSection) {
await workspace.loadBacklinks()
// Picked afresh each time, from the first: the open card leads.
workspace.cardsCursor = 0
workspace.filesCursor = 0
if (section) workspace.infoSection = section
workspace.openSheet('info')
})
}
/** A document is open and in focus — being written or read — rather than the list. */
const readingDocument = () =>
workspace.editing || (workspace.focus === 'node' && workspace.open?.kind === 'document')
/** The board's cursor, or the card turned over beside it, has the keys. */
const onBoard = () => workspace.focus === 'board' || workspace.focus === 'card'
@@ -1127,8 +1132,8 @@ export const commands: CommandSpec[] = [
name: 'upload',
aliases: ['up'],
group: 'editing',
help: 'while writing, attach files at the cursor — otherwise put files into this folder',
run: () => (workspace.editing ? hooks.pickFiles() : hooks.pickTreeFiles(false)),
help: "files for the open document — while writing, embedded at the cursor — or, from the list, into this folder",
run: () => (readingDocument() ? hooks.pickFiles() : hooks.pickTreeFiles(false)),
bang: {
help: 'put files into this folder sealed in the vault — encrypted before they touch the disk',
run: () =>
@@ -1137,16 +1142,6 @@ export const commands: CommandSpec[] = [
}),
},
},
{
name: 'media',
group: 'editing',
help: 'browse everything you have uploaded — j/k, i inserts it while writing, y copies its address, x deletes',
run: () => workspace.guard(async () => {
await workspace.loadMedia()
workspace.mediaCursor = 0
workspace.openSheet('media')
}),
},
{
name: 'outline',
aliases: ['toc'],
@@ -1156,13 +1151,13 @@ export const commands: CommandSpec[] = [
},
{
name: 'info',
args: '[outline|history|backlinks|cards|share]',
args: '[outline|history|backlinks|cards|files|share]',
group: 'documents',
help: 'what is known about the open document: its headings, earlier versions, backlinks, cards and sharing — Tab between them (gi)',
help: 'what is known about the open document: its headings, earlier versions, backlinks, cards, files and sharing — Tab between them (gi)',
complete: () => infoSections,
run: (arg) => {
const section = arg.trim()
if (section && !infoSections.includes(section as InfoSection)) return workspace.say('usage: :info [outline|history|backlinks|cards|share]', 'error')
if (section && !infoSections.includes(section as InfoSection)) return workspace.say('usage: :info [outline|history|backlinks|cards|files|share]', 'error')
return openInfo((section || undefined) as InfoSection | undefined)
},
},
+8 -7
View File
@@ -20,7 +20,7 @@ export type Context =
| 'overview'
| 'info'
| 'outline'
| 'media'
| 'files'
| 'trash'
| 'backlinks'
| 'cards'
@@ -103,6 +103,7 @@ export const bindings = [
{ context: 'tree', keys: ['R'], action: 'rename', help: 'rename what is under the cursor — document, folder or card — starting from its name' },
{ context: 'tree', keys: ['m'], action: 'mark', label: 'm / p', help: 'pick nodes up, then drop them all into the folder you walk to' },
{ context: 'tree', keys: ['p'], action: 'drop' },
{ context: 'tree', keys: ['za'], action: 'files', help: "a document's own files, folded open under it in the list — again folds them away; p drops marked files in there" },
{ context: 'tree', keys: ['M'], action: 'markAll', help: 'pick up the whole folder — again puts it all back' },
{ context: 'tree', label: 'Ctrl / Shift-click', help: 'pick a row up · every row up to it — drag them onto a folder' },
{ context: 'global', keys: ['S'], action: 'plan', help: 'the board, the agenda and the calendar, tiled ahead of what is open with the board focused — S again puts every view away' },
@@ -245,12 +246,12 @@ export const bindings = [
{ context: 'outline', keys: ['G'], action: 'bottom' },
{ context: 'outline', keys: ['Enter', 'l'], action: 'go', caps: ['↵'], hint: 'jump there' },
{ context: 'media', keys: ['j', 'ArrowDown'], action: 'down', caps: ['j', 'k'], hint: 'move' },
{ context: 'media', keys: ['k', 'ArrowUp'], action: 'up' },
{ context: 'media', keys: ['G'], action: 'bottom' },
{ context: 'media', keys: ['Enter', 'i'], action: 'insert', caps: ['i'], hint: 'insert' },
{ context: 'media', keys: ['y'], action: 'copy', hint: 'copy url' },
{ context: 'media', keys: ['x'], action: 'delete', hint: 'delete' },
{ context: 'files', keys: ['j', 'ArrowDown'], action: 'down', caps: ['j', 'k'], hint: 'move' },
{ context: 'files', keys: ['k', 'ArrowUp'], action: 'up' },
{ context: 'files', keys: ['G'], action: 'bottom' },
{ context: 'files', keys: ['Enter', 'l'], action: 'open', caps: ['↵'], hint: 'open' },
{ context: 'files', keys: ['i'], action: 'insert', hint: 'embed' },
{ context: 'files', keys: ['y'], action: 'copy', hint: 'copy embed' },
{ context: 'trash', keys: ['j', 'ArrowDown'], action: 'down', caps: ['j', 'k'], hint: 'move' },
{ context: 'trash', keys: ['k', 'ArrowUp'], action: 'up' },
+5 -3
View File
@@ -187,9 +187,11 @@ export function renderMarkdown(
return holder.innerHTML
}
/** The markdown snippet that embeds a freshly uploaded file. */
export function embedSnippet(file: { url: string; filename: string; mime: string }): string {
const name = file.filename.replaceAll(']', '')
/** The markdown snippet that embeds a file node: a picture, a player, or a link to it. */
export function embedSnippet(node: { name: string; file: { url: string; mime: string } | null }): string {
const file = node.file
if (!file) return ''
const name = node.name.replaceAll(']', '')
if (file.mime.startsWith('image/')) return `![${name}](${file.url})`
if (file.mime.startsWith('video/')) return `<video src="${file.url}" controls playsinline></video>`
if (file.mime.startsWith('audio/')) return `<audio src="${file.url}" controls></audio>`
+1 -1
View File
@@ -21,7 +21,7 @@ import { bindings, type Binding, type Context } from './keymap'
import { workspace } from './store.svelte'
/** The sheets with keys of their own, under the same name as the sheet. */
const sheets = new Set<string>(['help', 'overview', 'media', 'trash'])
const sheets = new Set<string>(['help', 'overview', 'trash'])
/** The stack, the topmost first. */
export function stack(): Context[] {
+73 -17
View File
@@ -12,7 +12,6 @@ import {
type Crumb,
type DiaryEntry,
type WorkspaceNode,
type MediaFile,
type Moved,
type NodeSummary,
type Revision,
@@ -35,9 +34,9 @@ export type Pane = 'list' | 'document'
export type TileKind = 'node' | 'board' | 'agenda' | 'calendar' | 'card'
export type View = Exclude<TileKind, 'node'>
/** What floats over the tiles; one at a time. */
export type Sheet = 'help' | 'media' | 'overview' | 'actions' | 'verify' | 'info' | 'trash'
export type Sheet = 'help' | 'overview' | 'actions' | 'verify' | 'info' | 'trash'
/** The info sheet's parts: what the open document holds, and what is around it. */
export type InfoSection = 'outline' | 'history' | 'backlinks' | 'cards' | 'share'
export type InfoSection = 'outline' | 'history' | 'backlinks' | 'cards' | 'files' | 'share'
export type Flash = { text: string; kind: 'info' | 'error' } | null
/**
@@ -404,8 +403,6 @@ class Workspace {
secret = $state.raw<{ text: string; resolve: (value: string | null) => void } | null>(null)
/** The vault, as last heard from the server; null until asked. */
vault = $state<VaultStatus | null>(null)
media = $state<MediaFile[]>([])
mediaCursor = $state(0)
/** What `:trash` lists: everything deleted, the last first. */
trash = $state<TrashItem[]>([])
trashCursor = $state(0)
@@ -420,6 +417,10 @@ class Workspace {
backlinksCursor = $state(0)
/** The info sheet's cards: where its cursor is in the open document's. */
cardsCursor = $state(0)
/** The file picked in the info sheet's files: Enter opens it, `i` embeds it while writing. */
filesCursor = $state(0)
/** Documents whose files `za` has folded open under them in the list. */
unfolded = $state<number[]>([])
overview = $state<SpaceOverview[]>([])
overviewCursor = $state(0)
@@ -991,7 +992,7 @@ class Workspace {
this.globalSearch = false
this.path = folder.path
this.spaceId = folder.path[0]?.id ?? id
this.nodes = await api.listNodes({ parent: id })
this.nodes = await this.#withFiles(await api.listNodes({ parent: id }))
const index = this.nodes.findIndex((n) => n.id === keep)
this.cursor = index >= 0 ? index : 0
this.#syncUrl(folder.path)
@@ -1017,7 +1018,7 @@ class Workspace {
api.scratch(this.query),
])
} else {
this.nodes = await api.listNodes({ parent: this.parentId ?? undefined })
this.nodes = await this.#withFiles(await api.listNodes({ parent: this.parentId ?? undefined }))
this.foundCards = []
this.foundNotes = []
}
@@ -1025,6 +1026,34 @@ class Workspace {
this.cursor = index >= 0 ? index : Math.min(this.cursor, Math.max(this.rows - 1, 0))
}
/** The list, with the files of each document `za` folded open under it. */
async #withFiles(list: NodeSummary[]): Promise<NodeSummary[]> {
const open = list.filter((n) => n.kind === 'document' && n.child_count > 0 && this.unfolded.includes(n.id))
if (open.length === 0) return list
const files = new Map(await Promise.all(open.map(async (doc) => [doc.id, await api.listNodes({ parent: doc.id })] as const)))
return list.flatMap((n) => [n, ...(files.get(n.id) ?? [])])
}
/** `za`: a document's own files, folded open under it in the list, or away again. */
async toggleFiles() {
const node = this.selected
if (!node || this.query) return
const doc = node.kind === 'document' ? node : this.nodes.find((n) => n.id === node.parent_id && n.kind === 'document')
if (!doc) return this.say('za folds open the files of a document', 'error')
if (this.unfolded.includes(doc.id)) {
this.unfolded = this.unfolded.filter((id) => id !== doc.id)
} else {
if (doc.child_count === 0) return this.say('no files — :upload while reading it gives it some', 'error')
this.unfolded = [...this.unfolded, doc.id]
}
await this.refresh(doc.id)
}
/** A row the list shows under its document, not in the folder itself. */
isNested(node: NodeSummary): boolean {
return !this.query && node.parent_id !== this.parentId
}
/** The space a search keeps to; none for one across every space. */
get #searchSpace(): number | undefined {
return this.globalSearch ? undefined : (this.spaceId ?? undefined)
@@ -1234,7 +1263,11 @@ class Workspace {
if (this.overlay !== 'trash') await this.enterSpace(node.id)
this.say(`restored the space ${name}`)
} else {
const parent = node.path.at(-2)
// A document's file comes back to the document, shown folded open in
// the list of the folder the document is in.
const owner = node.path.at(-2)?.kind === 'document' ? node.path.at(-2) : undefined
if (owner && !this.unfolded.includes(owner.id)) this.unfolded = [...this.unfolded, owner.id]
const parent = node.path.at(owner ? -3 : -2)
if (parent && node.path[0]?.id === this.spaceId) {
if (parent.id === this.parentId && !this.query) await this.refresh(node.id)
else if (this.overlay !== 'trash') await this.goTo(parent.id, node.id)
@@ -1608,11 +1641,21 @@ class Workspace {
this.say(`marked ${what} — p drops ${count === 1 ? 'it' : 'them'} in the folder you are in`)
}
/** `p`: drop the marked nodes into the folder the list is showing. */
/**
* `p`: drop the marked nodes into the folder the list is showing — or,
* when they are files and the cursor is in a document's files folded open,
* into that document.
*/
async drop() {
const parent = this.parentId
if (this.marked.length === 0) return this.say('nothing marked — press m on a row first', 'error')
if (parent === null) return this.say('no folder open', 'error')
const at = this.selected
const doc = at && this.isNested(at) ? at.parent_id : at?.id
if (doc && this.unfolded.includes(doc) && this.marked.every((n) => n.kind === 'file')) {
const name = this.nodes.find((n) => n.id === doc)?.name.trim()
return this.moveNodes(this.marked, doc, `to ${name || 'the document'}`)
}
await this.moveNodes(this.marked, parent)
}
@@ -2409,19 +2452,32 @@ class Workspace {
this.say(created.length === 1 ? `uploaded ${this.here}/${created[0].slug}` : `uploaded ${created.length} files`)
}
async upload(files: File[]): Promise<MediaFile[]> {
/**
* Give the document in `tile` files of its own — what a picture pasted
* while writing is too. Answers the file nodes, for the editor to embed.
*/
async attach(files: File[], tile = this.tile): Promise<NodeSummary[]> {
const doc = tile.open
if (doc?.kind !== 'document') {
this.say('files go to a document — open one first', 'error')
return []
}
// In a sealed folder they are sealed on the way, which needs the vault.
if (doc.sealed && !(await this.ensureUnlocked())) return []
const what = files.length === 1 ? files[0].name : `${files.length} files`
const total = files.reduce((sum, file) => sum + file.size, 0)
this.say(`uploading ${what} (${formatBytes(total)}) …`)
const uploaded = await api.upload(files, (fraction) =>
const name = doc.name.trim() || doc.slug
this.say(`uploading ${what} (${formatBytes(total)}) to ${name} …`)
const created = await api.putFiles(doc.id, files, doc.sealed, (fraction) =>
this.say(`uploading ${what} — ${Math.floor(fraction * 100)}%`),
)
this.say(`uploaded ${uploaded.length} file${uploaded.length === 1 ? '' : 's'}`)
return uploaded
// Its list of files, without touching what is being written in it.
if (tile.open?.id === doc.id) {
tile.open = { ...tile.open, files: [...tile.open.files, ...created], child_count: tile.open.child_count + created.length }
}
async loadMedia() {
this.media = await api.listMedia()
await this.refresh()
this.say(`${created.length === 1 ? created[0].name : `${created.length} files`} now belong${created.length === 1 ? 's' : ''} to ${name}`)
return created
}
setVim(enabled: boolean) {