nvrl/proton-stream
1
0 Fork 0
mirror of https://github.com/narrrl/proton-stream.git synced 2026-11-04 16:57:00 +01:00
Code Issues Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
nvrl committed 2026-08-02 19:37:03 +02:00
commit a1b345a029
80 files changed
+28033
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/ARCHITECTURE.md
+123
View File
@@ -0,0 +1,123 @@
# Architecture
## The problem
A Proton Drive public link is normally a download link. This app turns one into a
streamable library.
Nothing about the crypto prevents that: a file's content blocks are each a
self-contained PGP SEIPD packet decrypted under one shared session key, so there
is no sequential streaming session to resume and seeking costs exactly the blocks
the seek lands on. What was missing was plumbing — `RevisionReader`, the SDK's
random-access reader, was welded to the *authenticated* client. Undoing that weld
(`proton-sdk-rs` 0.3.3, `transport.rs`) is what made this app possible.
## Crates
```
pstr-app (bin: proton-stream) pstr-cli (bin: pstr)
│ │
├── pstr-player ── libmpv │
│ │ stream_cb + render API │
│ ▼ │
└── pstr-stream ───────────────────┤
│ │
└── pstr-core ─────────────┘
│
proton-drive-rs ──▶ proton-sdk
```
- **`pstr-core`** — the share list and its secrets, the catalog database, the
filename parser, and the crawl. Knows nothing about playback or pixels.
- **`pstr-stream`** — `RevisionReader` to a seekable byte stream: reader LRU,
in-memory block ring, forward read-ahead, optional on-disk block cache. Only
its `reader` module touches the SDK; everything else runs against a
`BlockSource` trait, which is what makes the caching and read-ahead testable
with no account and no network.
- **`pstr-player`** — libmpv. Fed through a custom `pstr://` protocol registered
with `mpv_stream_cb_add_ro`; rendered through `mpv_render_context_create` with
`MPV_RENDER_API_TYPE_OPENGL` into eframe's glow context. The URL is an opaque
token into a registry, never an identity: the caller opens the stream in async
code where a failure can be reported, then publishes it. mpv's open callback is
a hash lookup that cannot block and cannot leak a share secret into a log line.
Everything the callbacks decide lives in a safe, tested `StreamCursor`; the
`unsafe` is confined to one module of pointer plumbing. What is inside the file
— audio and subtitle tracks — is read from `track-list` one sub-property at a
time (`tracks.rs`) and selected through `aid`/`sid`; the *language* of a choice
is carried to the next file with `alang`/`slang`, which is what makes "Japanese
audio" hold for the rest of a season.
- **`pstr-app`** — the egui UI. Three threads' worth of work kept apart: the UI
thread only draws, an `Engine` runs everything with a network or a disk in it
on the tokio runtime and answers with events, and a player thread owns mpv for
the length of a file. Pages are pure functions of state that push `Action`s
rather than mutating, which is what allows a click to change the page it was
drawn on.
- **`pstr-cli`** — the headless front end, and the thing that tells you which
layer is broken.
## Data flow, playing an episode
```
click egui
│
▼
pstr://<share>/<link_id> mpv opens the URL, calls back on its demuxer thread
│
▼
VideoStream::read_at pstr-stream: ring? disk cache? else fetch
│
▼
RevisionReader::read_at SDK: the blocks overlapping the range, 10 at a time
│
▼
get_storage_blob block storage, pm-storage-token, no session credential
│
▼
ContentKey::decrypt_block one PGP packet per block, on the blocking pool
```
The demuxer thread is allowed to block, so the `stream_cb` callbacks bridge into
a dedicated tokio runtime with `Handle::block_on`.
Two properties of that middle layer are load-bearing enough to state here, both
established by measurement rather than reasoning (`docs/TESTING.md`):
- **A seek cancels outstanding read-ahead before it fetches anything.** Otherwise
the prefetches from the abandoned position hold the bandwidth the seek needs —
worth 5x on worst-case seek latency.
- **Read-ahead never runs deeper than half the ring.** A window larger than the
cache it lands in evicts its own blocks before the player reaches them.
- **How far to read ahead belongs to the consumer, not to the block layer.** mpv
buffers ahead of the picture itself, so under mpv the block layer reads ahead
*not at all*; a second speculative layer only takes bandwidth from the reads
mpv is blocked on. A reader with no buffer of its own still wants twelve.
Every block is fetched exactly once no matter how many readers want it: the
open, and each block fetch, go through a single-flight keyed on the block. The
demand read and the prefetch racing for the same block is the normal case, not
an edge case.
## Trust and secrets
- The share URL fragment **is** the decryption password, and a custom-password
link has a second secret. Both live in the OS credential store; `shares.json`
holds only the id, display name and token.
- Nothing this app holds can write to the share: a viewer link grants
`MemberRole::Viewer`, and the SDK's public-link write side is unported anyway.
- Signature verification is absent on the visitor path by design — an anonymous
visitor cannot read `core/v4/keys/all` to resolve a signer. Decryption still
proves the bytes were encrypted to the node key. Nodes come back with nothing
*claimed* rather than a verification that silently passed.
- Metadata enrichment is off by default because enabling it sends the user's
library titles to a third party.
## Persistence
| Store | Path | Rebuildable? |
|---|---|---|
| Share list | config dir, `shares.json` | **No** — atomic writes, never overwritten when unparseable |
| Share secrets | OS credential store | **No** |
| Catalog | data dir, `catalog.db` | Yes — recrawl |
| Watch state | same DB, own table | **No** — deliberately survives a recrawl |
| Block cache | cache dir, `blocks/` | Yes — must work correctly after deletion |
| Poster thumbnails | cache dir, `thumbs/` | Yes — refetched per file, one small decrypt each |
docs/BUGS.md
+282
View File
@@ -0,0 +1,282 @@
# Bugs
The issue ledger. Open items first, then a log of what was fixed and where.
Format: `Bn — one-line summary`, then symptom, cause, fix, and how it was
verified. Reference entries from PRs.
## Open
_None yet — the app is young enough that everything known is in
`docs/DEVELOPMENT.md` as unbuilt rather than broken._
## Fixed
### B13 — the player's controls ran off the bottom of the window
**Symptom.** On the player page the transport row — previous, ±seconds, play,
volume, the pickers — sat flush against the bottom edge of the window and was
clipped by it. The floating **Skip opening** button sat at the same height as
the seek bar rather than above the controls, four pixels out of line with them.
**Cause.** `CHROME_HEIGHT` was a constant 132 px. The controls are two rows of
themed buttons whose height follows the theme and the platform's text scaling,
so the constant was a guess — and a guess that is low does not clip in egui, it
draws past the rectangle and off the window. The skip button then used its own
22 px inset where the controls used 26.
**Fix.** `ui::player::chrome` returns the height it actually measured
(`ui.min_rect().height()` plus padding), the app carries it in `Overlay`, and the
next frame uses it for the scrim and for floating anything above the controls.
The content is anchored to the bottom of the window rather than laid out from
the top of the strip, so the last row ends a fixed distance above the edge
whatever it turns out to contain. One frame of lag, and only while resizing.
`floating` puts both the skip button and the up-next card at the controls' own
`CHROME_PAD_X`.
### B12 — the skip button believed a chapter's name
**Symptom.** Episode one of Oshi no Ko opens with a chapter called `Intro` that
is eleven minutes of the story. The player offered **Skip opening** over it.
**Cause.** `ChapterRole::of` read the first word of the name and nothing else,
mapping `intro` to `Opening` unconditionally. Nothing looked at how long the
chapter was or at what else the file contained.
**Fix.** Two passes — `Claim` from the name, `roles` from the name *and* the
file. A name that could go either way (`Intro`, `Credits`, `Cast`) is resolved
by length and position: an opening has to be theme-length, near the front, and
the only candidate in the file. Both ambiguities fail towards content. The same
pass is what `credits_start` reads, so the end-of-episode countdown does not
assume the last chapter is the ending either — see the Part C case in
`docs/DEVELOPMENT.md`.
**Verified.** `a_long_intro_is_the_episode_and_not_an_opening`,
`a_theme_length_intro_near_the_front_is_an_opening`,
`content_after_the_ending_keeps_the_ending_out_of_the_tail`.
### B11 — seasons past the first had no episode names, and were one query from having the wrong ones
**Symptom.** Oshi no Ko seasons two and three showed filenames where season one
showed episode names. Season one only "worked" because *its* filenames carry
them (`S01E01-Mother and Children.mkv`); `episode_metadata` was empty for the
whole library.
**Cause.** Two, stacked. A title is looked up once, and on AniList that match is
the *first season's* entry — a sequel is a separate id numbering from one — so
nothing was ever asked about seasons two and three. And `EpisodeGuide::get` fell
back from `(Some(2), 1)` to the provider's absolute `(None, 1)`, so the moment
season one's episodes were fetched, every later season would have been captioned
with season one's names. A wrong caption does not read as a bug; it reads as the
library being wrong.
**Fix.** `Provider::seasons_are_separate_entries` distinguishes the two
providers. For AniList, `MetadataService::season_episodes` searches each season
past the first by the name the provider uses (`<title> 2nd Season`, then
`<title> Season 2`) and tags the result with that season; TMDB already returns
seasons numbered and is left alone. The absolute fallback now stops at season
one.
**Verified.** `an_absolutely_numbered_answer_never_reaches_a_later_season`,
`a_season_is_searched_for_the_way_the_provider_names_it`. Against the live API,
`[Oshi no Ko] 2nd Season` is id 166531 with 13 episodes and `3rd Season` is
182587 with 11 — which is exactly what the share holds.
**Still missing for that show, and not a bug here.** AniList's episode titles
come from `streamingEpisodes`, and all three Oshi no Ko entries have none at
all. Those rows keep their filenames until the viewer switches to TMDB.
### B10 — every episode of season three was called "3rd Season"
**Symptom.** Under `Oshi no Ko`, all eleven rows of season three read
`3rd Season` instead of an episode name.
**Cause.** `naming::parse` knew `S03` and `Season 03` but not `3rd Season`, so
`[DB]Oshi no Ko 3rd Season_-_07_….mkv` parsed as a title of `Oshi no Ko 3rd
Season`. `catalog::build_rows` then took the difference between that and the
folder's `Oshi no Ko` to be the *episode's* own name — which is what it is for,
and here it was a season marker.
**Fix.** `find_worded_season` reads both orders (`3rd Season`, `Season 3`) and
ends the title there, so the season comes out of the filename as a season.
`build_rows` additionally asks `naming::is_season_marker` before keeping any
leftover as an episode name, and folds it into the season instead. The dash is
deliberately not a separator between `Season` and its number: `Show 2nd Season -
04` puts the episode there, and reading across it made episode four season four.
**Verified.** `underscore_separated_numbering_is_read`,
`a_worded_season_is_read_as_a_season`,
`an_ordinal_that_is_part_of_the_title_is_not_a_season`,
`a_season_stated_in_the_filename_does_not_become_the_episodes_name`.
### B9 — a series split into one poster per season on the shelf
**Symptom.** The library wall showed `Game of Thrones - Season 1` … `Season 8`
as eight separate titles, each with its own poster and its own resume point.
**Cause.** `naming::classify_folder` recognised a season folder only when the
whole name was one (`Season 01`, `S03`). A folder named `<Series> - Season 1`
therefore classified as a title, and the season number stayed in the title
string, so the grouping key differed per season.
**Fix.** `naming::trailing_season` splits a `… Season N` / `… SNN` tail off a
folder name, and `catalog::ancestry` takes the season from a folder that names
both. The keyword is required — `Evangelion 3` and `Alien 3` keep their number,
and `Blade Runner 2049` still reads as a year.
**Verified.** `a_folder_naming_series_and_season_yields_both`,
`a_title_ending_in_a_number_keeps_it`; on the real share, 30 titles became 23,
with Game of Thrones one poster of 73 episodes.
### B5 — a multi-byte character in a filename panicked the parser
**Symptom.** The crawl aborted on
`[Anime Time] Neon Genesis Evangelion - Death (True)²-007.mkv` with
`start byte index 34 is not a char boundary; it is inside '²'`.
**Cause.** Year detection scanned bytes and sliced on byte offsets, so a token
boundary could land inside a multi-byte character.
**Fix.** `bare_year` walks `char_indices`; `bracketed_year` verifies the four
bytes are ASCII digits *before* slicing.
**Verified.** `a_multi_byte_character_does_not_break_parsing`.
### B4 — files were catalogued with no title
**Symptom.** Every episode in the catalog had an empty title: a real library is
laid out `Sousou no Frieren/Season 01/S01E01.mkv`, and `S01E01.mkv` states no
series at all.
**Cause.** Rows were built from the filename alone.
**Fix.** `catalog::build_rows` walks the folder path: the folder names the
series, the filename names the numbering, grouping folders (`Movies`, `OVA`,
`Specials`) are stepped over, and a `Season NN` folder supplies a season the
filename omitted. Non-video files (`.nfo`, `.jpg`) are dropped.
**Verified.** Seven `build_rows` tests, and a real 354-node share that went from
0 usable titles to 22 correct ones.
### B3 — several naming conventions lost their episode numbers
**Symptom.** 35 of 310 files in a real share had no episode number.
**Cause.** Four gaps: a bare `E11.mkv`; a bare `11.mkv`; `_-_01_` underscore
separators; and — the subtle one — `Oshi no Ko S2 - 04`, where matching the
season *returned*, so the differently-stated episode was never looked for.
**Fix.** `find_episode_marker`, `find_bare_number`, underscore separators in
`find_dash_numbering`, and `find_numbering` continuing past a season-only match.
**Verified.** Down to 11 unnumbered files, all of which are films or OP/ED
extras that genuinely have no episode number.
### B1 — a public-link download silently truncated past 200 MiB
**Symptom.** `download_file` over a public link returned a file cut off at
exactly 50 blocks, with no error.
**Cause.** The visitor path issued a single un-paged
`GET …/revisions/{rid}` and used whatever `Blocks` came back. The endpoint pages
at 50.
**Fix.** `proton-sdk-rs` 0.3.3: the visitor path now uses the same paged
`list_blocks` as the authenticated one, and fetches blocks concurrently.
**Verified.** `a_visitor_can_seek_within_a_shared_multi_block_file` reads back a
9 MiB file whole and in ranges.
### B2 — a catalog crawl re-derived the same keys per child
**Symptom.** Listing a folder of several hundred episodes over a public link took
minutes.
**Cause.** `resolve_parent_key` walked and re-*unlocked* the whole ancestor chain
for every child. Each unlock is an S2K derivation — tens of milliseconds — so 500
siblings cost 500 redundant unlocks of the same parent key. `enumerate_nodes`
also never chunked its link ids, exceeding the server's 150-id batch limit.
**Fix.** `proton-sdk-rs` 0.3.3: an LRU node-key cache with an iterative
ancestor walk (up to the nearest cached key, then unlock back down caching each)
behind `SingleFlight`; `enumerate_nodes` chunked at 150 and fanned out.
### B6 — read-ahead starved the seeks it was supposed to smooth
**Symptom.** Dragging to a new position took ~2.9 s with the block layer's
read-ahead enabled, versus ~0.57 s with it turned off. Sustained throughput was
also *lower* with read-ahead than without: 5.0 MiB/s against 7.2.
**Cause.** Prefetches queued from the position the viewer had just left kept
running. The block they were fetching was exactly the bandwidth the seek needed,
and a 4 MiB block over this link takes most of a second, so the seek waited
behind up to six of them.
**Fix.** `pstr-stream::stream` tracks where the previous read ended and treats a
read landing more than one block away as a seek. A seek cancels every outstanding
prefetch *before* it fetches anything. Aborting mid-fetch throws those bytes
away, which is the intended trade — they were speculative, and the request slot
they held is what the seek is waiting for. Read-ahead resumes wherever the seek
settles.
**Verified.** Worst-case cold seek back to 715 ms with read-ahead on.
`a_seek_cancels_the_read_ahead_it_invalidates` and
`sequential_playback_never_cancels_its_own_read_ahead` pin both directions, and
`a_block_whose_prefetch_was_cancelled_can_still_be_read` pins that an aborted
prefetch releases its single-flight entry rather than leaving the block
permanently unfetchable.
### B7 — a read-ahead window deeper than the ring evicted its own blocks
**Symptom.** Raising read-ahead from 12 to 32 blocks *reduced* sustained
throughput, 8.7 MiB/s to 7.8, and the ring started reporting evictions.
**Cause.** The default 128 MiB ring holds 32 blocks of 4 MiB. A 32-block
read-ahead window is the entire ring, so blocks at the front of the window were
evicted before the player reached them and had to be fetched a second time.
**Fix.** `clamp_readahead` caps the window at half the ring's capacity in blocks
— half rather than all, because the ring also has to hold what the player has
just passed, so a scrub a few seconds backwards does not refetch. The default
depth is 12, chosen because it never evicted and wasted a third less bandwidth on
cancelled prefetches than 16 did, for indistinguishable throughput.
**Verified.** `read_ahead_is_capped_at_half_the_rings_capacity` and
`a_tiny_ring_still_reads_one_block_ahead`.
### B8 — the block layer's read-ahead made playback worse once mpv was in front of it
**Symptom.** With `pstr-stream`'s tuned default of 12 blocks, mpv took 5.5 s to
show a first frame and 3.6 s to resume after a mid-file seek. Turning the block
layer's read-ahead **off entirely** cut both to 2.5 s and 1.5 s, and fetched a
third as much data for the same playback.
**Cause.** [B6](#b6--read-ahead-starved-the-seeks-it-was-supposed-to-smooth) and
[B7](#b7--a-read-ahead-window-deeper-than-the-ring-evicted-its-own-blocks) were
measured against `pstr bench`, a reader with no read-ahead of its own — for
*that* reader, 12 blocks is right. mpv is not that reader. Its demuxer cache
already runs `demuxer-readahead-secs` ahead of the picture, sequentially and
eagerly, so a second speculative layer underneath adds no buffer at all. It only
competes for bandwidth with the reads mpv is blocked on right now, and it does so
worst exactly when the viewer is waiting: at startup and just after a seek. The
degradation is monotonic in depth.
| blocks ahead | first frame | seek resumed | blocks fetched |
|---:|---:|---:|---:|
| **0** | **2.5 s** | **1479 ms** | **12** |
| 6 | 3.7 s | 2225 ms | 21 |
| 12 | 5.5 s | 3581 ms | 31 |
| 24 | 7.7 s | 6122 ms | 42 |
**Fix.** `pstr_player::READAHEAD_BLOCKS` is 0, and it is what `pstr play` and the
app pass to `StreamConfig`. `pstr_stream::DEFAULT_READAHEAD_BLOCKS` stays 12 for
consumers that genuinely have no read-ahead — the benchmark, and anything that
reads a revision without a demuxer. mpv's buffer is also expressed in *seconds*
rather than bytes (`PlayerConfig::readahead_seconds`, 30 s), because a byte
budget buys two minutes of buffer on a 4 Mbit/s episode and eight seconds of it
on a 4K remux, which is backwards from what either wants; the byte figure is kept
only as a ceiling, sized under the ring so B7 cannot recur one layer up.
**Verified.** Sustained playback holds realtime at depth 0 (178 s of media in
180 s of wall clock, the deficit being the 2.4 s to first frame), with no ring
evictions, against the same 761 MiB episode over a real public link.
docs/DEVELOPMENT.md
+193
View File
@@ -0,0 +1,193 @@
# Development
## Where things stand
| Layer | State |
|---|---|
| SDK streaming surface (`../proton-sdk-rs` 0.3.3) | **done, live-verified** — `open_revision` / `download_range` / thumbnails / `refresh_session` on the visitor path; node-key cache; chunked `enumerate_nodes`; paged `download_file_to` |
| `pstr-core` — shares, config, catalog, naming, library grouping | **done** — 67 offline tests |
| `pstr-cli` — add / crawl / list | **done** — verified against a real 354-node share: 310 playable files, 22 titles, correct seasons and episodes |
| `pstr-stream` — block cache, read-ahead | **done, live-verified** — 68 offline tests; benchmarked against a real 761 MiB episode (see `docs/TESTING.md`) |
| `pstr-player` — libmpv `stream_cb` | **done, live-verified** — 16 offline tests; plays a real 1080p HEVC episode over a public link, seeks, resumes from the disk cache. The render API (embedding) is not started |
| `pstr-app` — egui UI, poster grid | **done, offline-tested** — library wall, title pages, transport, share management; mpv still opens its own window |
| Metadata providers (AniList / TMDB) | not started — and this is what the wall is waiting on: see the note on thumbnails below |
| Packaging | **done, locally verified** — `scripts/build.sh` (tarball, `.deb`, `.rpm`, `.app`, `.dmg`) and `scripts/build.ps1` (portable `.zip`, WiX `.msi`), plus `packaging/PKGBUILD`. See `packaging/README.md`. The Windows and macOS halves are written but only the Linux ones have been run on this machine |
## Build order from here
1. **`pstr-player`, step two.** `mpv_render_context_create` with
`MPV_RENDER_API_TYPE_OPENGL` against eframe's glow context, rendered inside an
`egui::PaintCallback`; overlay controls. If this proves troublesome, the
standalone-window mode already built remains shippable. Nothing in
`pstr-player` except `PlayerConfig`'s window options is about where pixels
land, so this does not reach the streaming path.
2. Metadata providers, docs. Packaging is in place (`packaging/README.md`); what
is left there is running `scripts/build.ps1` and the macOS half on real
hosts, and a release workflow that drives both.
Done: `pstr-player` step one — `mpv_stream_cb_add_ro`, mpv in its own window.
`pstr play --file <text>` plays a catalogued episode over a public link.
Done: `pstr-app` step one — `cargo run -p pstr-app` opens a window with the
library wall, a title page per series, a transport bar and share management. It
plays through the same `Player` the CLI uses, so mpv still owns its own window;
the app's transport drives it from a command channel. Two pieces of that are
worth knowing before touching it:
- **Drawing never mutates.** Pages are handed `&`-state and push `Action`s onto
a list the app applies after the frame. That is what lets a card click change
the page it was drawn on.
- **The UI thread never opens a share, reads the catalog or decodes a JPEG.**
All of it goes through `pstr_app::engine::Engine`, which spawns onto the tokio
runtime and answers with an `Event` — plus a `ctx.request_repaint()`, without
which the event sits in the channel until the mouse moves.
Done: player controls beyond play/pause/seek — volume with mute, an audio-track
picker and a subtitle picker, in the transport bar (so on the player page and on
the bar under the library alike), plus `M` and `↑`/`↓` on the player page. All
four preferences persist to `playback.json` (`pstr_core::prefs`), and a track
chosen by hand sets `alang`/`slang` for every file loaded afterwards — the point
being that picking Japanese audio once picks it for the rest of the season.
A dragged volume slider writes to disk on release, not per frame.
Done: the player page's overlay — a full-width seek bar with the file's chapters
marked on it and the current chapter named beside the clock, a transport cluster
(previous / −10s / play / +30s / next), and volume plus the audio, subtitle and
chapter pickers to the right. Chapters come from mpv's `chapter-list`
(`pstr-player/src/chapters.rs`); a chapter whose name reads as an opening,
ending or preview also puts a **skip** button in the bottom-right corner, which
is drawn whether or not the rest of the controls are up — an opening lasts
ninety seconds and a button you have to wake with the mouse is one nobody uses.
The strip under the library is the same controls at a lower density plus a
**Back to video** button, which is what makes leaving the player page with Esc
reversible.
Done: **next / previous episode and autoplay.** The neighbours come from
`Title::following`/`preceding` in `pstr-core` — display order, so it walks off
the end of a season into the next one — and autoplay fires only on a clean end
of file, never on a failure. `P`/`N` on the player page, and the page stays put
while the next file opens rather than dropping back to the library.
Done: **what a chapter is, and the "up next" countdown.** Chapter names are read
in two passes (`pstr-player/src/chapters.rs`). The first reads the name alone —
`OP`, `ED`, `Next Episode`, and the Japanese forms — and yields a *claim*; the
second (`roles`) settles the claims that a name cannot settle, using the rest of
the file. `Intro` is the opening only when the file has no chapter that says so
outright, the chapter is the length of a theme song and it is near the front:
episode one of Oshi no Ko opens with eleven minutes called `Intro` that are the
story. `Credits` / `Cast` are the ending at about that length or in the last
quarter. Both ambiguities fail *towards content*, because not offering a skip
costs a button and skipping the story costs the story.
`credits_start` then takes the run of chapters that **ends** the file — walking
back from the last one and stopping at the first that is content. That is the
part worth stating: the ending is not the tail if something follows it.
Fullmetal Alchemist Brotherhood episode 46 has a Part C after the ED and the
preview after that, so only the preview is skippable there, while the ordinary
`… ED · Preview` gives up both. Reaching that run starts a ten-second **Up next**
card (`App::tick_up_next`, `ui::player::up_next_card`) with *Play now* and
*Watch till the end*; the latter holds for the rest of the file. It needs
chapters, autoplay on, a next episode, and playback actually running — a paused
episode is one somebody walked away from.
Done: **per-episode metadata.** `Provider::episodes` fetches an episode list for
a title that already matched (AniList `streamingEpisodes`, which is titles and
thumbnails only and no synopsis; TMDB `/tv/{id}` plus every season appended onto
one more request), stored in `episode_metadata` — schema **v4**. The reconciling
between a release's numbering and a provider's lives in one tested place,
`EpisodeGuide::get`: the season and number the filename states, then the
provider's absolute numbering, then season one for a file that states no season
at all. That last fallback is what makes `#057` line up with TMDB filing a
64-episode anime under one season. Episode names show on the title page's rows,
in the player's title strip and in the "next:" line autoplay prints.
**A season is a separate lookup on AniList.** `Provider::seasons_are_separate_entries`
is what splits the two providers: TMDB files every season under one show and
returns them numbered, while AniList makes each sequel its own entry numbering
from one (`[Oshi no Ko] 2nd Season`, id 166531, is not `[Oshi no Ko]`, id
150672). So for AniList each season past the first is searched for by name —
`<title> 2nd Season`, then `<title> Season 2` — and its episodes are stored
tagged with that season. The absolute fallback in `EpisodeGuide::get` stops at
season one for the same reason: without that it answered `S02E01` with season
one's episode one, which is not a missing caption but a wrong one.
Worth knowing before chasing a missing name: AniList's episode titles come from
`streamingEpisodes`, which is what streaming sites published, and for some shows
it is simply **empty** — all three Oshi no Ko entries have none, while Fullmetal
Alchemist Brotherhood has 64. There is nothing to fix on our side for those;
TMDB is the provider that has them.
Note the glyph trap: egui's bundled fonts cover the transport icons
(`⏮ ⏭ ⏸ ▶ 🔊 🔇 ⛶`) and *not* `← 💬 ▣`, which draw as an empty box. Anything new
on a button wants checking on screen before it ships.
Not done: **sidecar subtitle files**. A `.srt`/`.ass` sitting beside the video in
the share is invisible to all of this — the crawl stores video files only, so
there is nothing to offer in the menu. The player side is nearly free once the
catalog has them (`sub-add pstr://<token>` through the same registry, since mpv
reads an external subtitle through the same protocol); the work is in
`pstr-core`: crawl them, and pair them to a video by name.
## Design decisions already taken
- **Fat client, no server.** Each viewer's app talks to Proton directly.
- **Public links only**, but several, merged into one catalog. No account login.
- **Embedded libmpv**, not a bundled decoder stack. HEVC 10-bit, HDR and ASS
subtitles with hardware decode, for free.
- **No transcoding.** mpv demuxes and decodes natively; there is no remux or
re-encode step and no ffmpeg dependency of our own.
- **`stream_cb`, not a loopback HTTP server.** No port to open, no local attack
surface, no plaintext leaving the process.
- **Metadata enrichment is opt-in and off by default**, with the privacy cost
stated in the UI: enabling it sends your library's titles to a third party.
With it off, posters fall back to Proton thumbnails.
- **WASM is out.** Not a small port and not worth reopening: the SDK pulls tokio
with `rt-multi-thread` + `fs`, has four `spawn_blocking` sites, resolves two
`getrandom` versions with no wasm backend, and drags `aws-lc-rs`/`ring` through
rustls. Separately a browser cannot reach the Proton API at all (CORS) without
a proxy server, which the fat-client decision rules out.
## Traps
- **Block sizes are not uniform.** Always `RevisionReader::block_sizes()`; never
a hardcoded 4 MiB. A padded vector shifts every later block's start and serves
full-length reads of the wrong bytes, silently.
- **Cache validity is the revision id**, not `(mtime, size)`. It advances if and
only if a new revision was sealed.
- **Read-ahead must yield to seeks, and must not outrun the ring.** Both were
measured, both cost more than they bought when got wrong: prefetches left
running from the position the viewer left took worst-case seek latency from
572 ms to 2895 ms, and a window deeper than half the ring evicts its own blocks
before the player reaches them. `pstr-stream::stream` cancels on seek and
clamps the depth; the numbers are in its module docs.
- **Read-ahead depth is a property of the *consumer*, not of the block layer.**
mpv brings its own, so under mpv the block layer's read-ahead should be **0**:
at depth 12 the first frame took 5.5 s instead of 2.5 s and a seek 3.6 s
instead of 1.5 s. `pstr_player::READAHEAD_BLOCKS` is what a player-driven
caller passes; `pstr_stream::DEFAULT_READAHEAD_BLOCKS` is for readers that have
none of their own. Do not "unify" them. See `docs/BUGS.md` B8.
- **mpv's buffer is in seconds, not bytes.** A byte budget means two minutes of
buffer on a 4 Mbit/s episode and eight seconds on a 4K remux. The byte figure
survives only as a ceiling, and it must stay under the ring or B7 recurs a
layer up.
- **The anonymous session expires mid-film.** The SDK renews it in place and
readers survive; the app should still be able to reopen and resume at the same
byte offset as a belt-and-braces fallback.
- **A share filled by the FUSE client has no thumbnails at all.** Proton renders
a thumbnail at upload time and the SDK does not generate one, so every
`download_thumbnail` on this library answers `Ok(None)` and every tile falls
back to initials. Nothing is broken — it is why metadata providers matter more
than they looked like they did, and why an empty `thumbs/` cache directory is
not evidence the thumbnail path is wrong.
- **Negative caching matters.** Without it, every grid render pays a round-trip
per unmatched title. `proton-drive-linux`'s photo grid learned this the hard
way. `ThumbnailCache` remembers both the textures it has *and* the files that
turned out to have no thumbnail.
- **A dragged seek bar must seek on release, not per frame.** Every seek cancels
outstanding read-ahead and refetches; a bar that seeks while dragging spends
the whole drag throwing away blocks it just paid for. `ui::transport` acts on
`clicked() || drag_stopped()`.
- **egui 0.35 moved the panels.** `eframe::App` is `fn ui(&mut self, ui, frame)`,
not `update(ctx, frame)`; panels are `egui::Panel::top(id).show(ui, …)` rather
than `TopBottomPanel::top(id).show(ctx, …)`, and `Context::style_mut` is now
`all_styles_mut`. Older egui snippets will not compile as written.
docs/TESTING.md
+200
View File
@@ -0,0 +1,200 @@
# Testing
## Offline
Everything in `pstr-core` is offline and pure. That is deliberate: the filename
parser, the config writer and the catalog migrations are exactly the things whose
edge cases are painful to reproduce against a live share, so they are pinned by
table-driven tests instead.
```bash
cargo test --workspace --locked
cargo test -p pstr-core naming:: # just the parser
```
What is pinned today:
- **`naming`** — the fansub, scene and cross conventions; dashes inside titles;
version suffixes; a bracketed resolution that must not read as a year; a word
starting with `s` that must not start a season; dotted titles vs. extensions;
and that an unrecognisable name still yields a usable title rather than being
dropped.
- **`config`** — round trip, missing-is-`None`, **unparseable-is-an-error** (the
one that protects the share list from a read-modify-write wiping it), and no
temp file left behind.
- **`catalog`** — schema version, refuse-to-open on a newer schema, share replace
semantics (drops removed rows, leaves other shares alone), and that **a recrawl
preserves watch state**.
`pstr-stream` is offline too, and for the same reason: it runs against a
`BlockSource` fake, so the caching, deduplication and read-ahead behaviour that
is nearly impossible to provoke deliberately over a network is pinned exactly.
- **`block`** — byte range → block mapping, including non-uniform sizes, a range
ending exactly on a boundary (which must not pull the next block), and splicing
into the right part of the output buffer.
- **`ring`** — never exceeds its byte budget, evicts least-recently-*used*, a
containment check records no traffic and no recency, a block bigger than the
budget is not stored, reinserting does not double-count.
- **`disk`** — round trip, a block disagreeing with its sidecar is rejected *and
removed*, a block with no sidecar is a miss, eviction deletes the files rather
than merely forgetting them, a reopened cache adopts the previous run's bytes,
and cache paths carry no names.
- **`single_flight`** — N callers run the work once, a waiter sees the leader's
failure, a failed call leaves the key retryable, and a **cancelled** leader does
not wedge the key.
- **`stream`** — reads across block boundaries, a seek fetches only the block it
lands in, a demand read joins an in-flight prefetch instead of refetching, a
failing prefetch never fails the read, **a seek cancels the read-ahead it
invalidates** while sequential playback cancels none, and streaming a file far
larger than the ring stays inside the budget.
`pstr-player` is offline too, everywhere it can be. mpv itself needs a window and
a codec, but nothing mpv's stream callbacks *decide* lives in the callbacks —
they hand straight to `StreamCursor`, a plain safe type over a `BlockSource`
fake, which is what the tests drive.
- **`cursor`** — sequential reads walk the file and stop at EOF, a seek moves
where the next read starts, a read crossing a block boundary is allowed to come
up short but returns the right bytes, seeking exactly to EOF is allowed while
past it is an error, **mpv's post-open seekability probe to 0 succeeds** (fail
it and every seek bar is gone), and a cancelled stream fails its reads and
stays failed.
- **`registry`** — a published stream is reachable through its URL, dropping the
handle revokes it, and **a token is never reused** after the stream behind it
is gone, so a stale URL cannot start playing whatever took the number over.
Only well-formed `pstr://` URLs parse.
- **`player`** — every mpv end-file reason maps to something the UI can act on
(`Eof` means "next episode"; nothing else does), and an observed property
arriving in an unexpected format is *not* read as a position of zero.
## Against a real share
Needs a Proton Drive public link. Use a share you can afford to have indexed;
nothing here writes, but the crawl reads every node's metadata.
```bash
cargo run -p pstr-cli -- add --name anime 'https://drive.proton.me/urls/TOKEN#fragment'
cargo run -p pstr-cli -- add --name anime 'https://…' --custom-password # prompts
cargo run -p pstr-cli -- crawl
cargo run -p pstr-cli -- list
```
Check: node count matches the folder, titles and episode numbers look right, and
the crawl time is seconds rather than minutes (if it is minutes, the SDK's
node-key cache is not doing its job).
The reference run, against a real library: **354 nodes crawled, 310 playable, 22
titles, ~25s**. Eleven files end up with no episode number and that is correct —
they are films and OP/ED extras. Anything else unnumbered is a parser gap; add
the filename to `naming`'s tests before fixing it.
Useful for spotting gaps:
```bash
sqlite3 ~/.local/share/proton-stream/catalog.db \
"SELECT title, name FROM nodes WHERE is_folder=0 AND episode IS NULL;"
```
## Benchmarking playback, without a player
`pstr bench` reads a catalogued file the way a demuxer would — 64 KiB at a time —
and reports the three numbers that decide whether the app is usable: cold seek
latency, sustained throughput, and peak resident memory.
```bash
cargo run -p pstr-cli -- bench --file Frieren --verify
cargo run -p pstr-cli -- bench --file Frieren --readahead 0 # isolate read-ahead
cargo run -p pstr-cli -- bench --file Frieren --disk-cache # run twice
```
`--verify` cross-checks a mid-file read against the SDK's own `download_range`.
Run it whenever anything in the block-mapping or caching path changes; it is the
only check that catches bytes served from the wrong offset, which otherwise
surfaces as video that decodes into garbage rather than as an error.
Reference run, 761 MiB episode, 191 blocks, over a real public link:
| | cold seek (worst) | sustained | peak resident |
|---|---:|---:|---:|
| read-ahead off | 572 ms | 7.2 MiB/s | 60 MiB |
| read-ahead 6, no seek cancellation | **2895 ms** | 5.0 MiB/s | 128 MiB |
| read-ahead 12, seek cancellation | 715 ms | 8.7 MiB/s | 128 MiB |
| read-ahead 32 | 716 ms | 7.8 MiB/s | 128 MiB (evicting) |
| warm disk cache | **14 ms** | 1039 MiB/s | 76 MiB |
Two findings worth not rediscovering. Read-ahead left running from the position
the viewer just left holds exactly the bandwidth the seek needs — 5x on
worst-case latency. And a read-ahead window deeper than half the ring evicts its
own blocks before the player reaches them, so throughput *falls*. Both are fixed
in `pstr-stream::stream`; both are pinned by offline tests.
Expect run-to-run spread of 15–20% on the throughput number. Treat a change under
that as noise.
Note that `bench`'s tuned read-ahead default (12) is **not** the right one for
mpv, which brings its own — see `docs/BUGS.md` B8. A change that improves `bench`
has not thereby improved playback; measure both.
## Playing, with a player
```bash
cargo run --release -p pstr-cli -- play --file Frieren
cargo run --release -p pstr-cli -- play --file Frieren --start 900 # cold mid-file seek
cargo run --release -p pstr-cli -- play --file Frieren --disk-cache # run twice
cargo run --release -p pstr-cli -- play --file Frieren --headless --for-seconds 60
```
Use `--release`. A debug build is watchable but the numbers below are not
reproducible from one.
`--headless` decodes with no window and no sound, so the whole chain still runs
somewhere there is no display; `--for-seconds` ends an unattended run without a
window to close. Together they are the smoke test. Every run prints time to first
frame, seek-resume latency, and what the block layer actually fetched.
Reference run, same 761 MiB / 1560 s episode over a real public link, defaults:
| | cold | warm disk cache |
|---|---:|---:|
| time to first frame | 2.3 s | **0.3 s** |
| mid-file seek resumed in | 1.0–1.6 s | — |
| blocks fetched for 38 s of playback | 16 (61 MiB) | **0** |
| ring evictions | 0 | 0 |
Sustained playback holds exactly realtime: 178 s of media in 180 s of wall clock,
the 2 s deficit being the time to first frame, with no drift over the run. Check
that: a deficit that *grows* is the block layer failing to keep up, and it is the
one failure the smoke test can see but a short run cannot.
`Cannot load libcuda.so.1` on stderr is mpv's `hwdec=auto-safe` probe declining a
backend. Not an error.
## SDK-side
The streaming surface this app depends on is tested in `../proton-sdk-rs`:
```bash
cd ../proton-sdk-rs
PROTON_TOTP_SECRET=... cargo test -p proton-drive-rs --test live_sharing \
-- --ignored --nocapture --test-threads=1
```
`a_visitor_can_seek_within_a_shared_multi_block_file` is the load-bearing one —
it verifies that the `drive/unauth/` revision endpoint returns a usable `XAttr`,
without which block boundaries cannot be placed and seeking is impossible.
Live runs there must stay `--test-threads=1`: Proton throttles repeated logins
and each fresh one burns a TOTP window.
## Acceptance, once playback lands
1. Add the share; the catalog populates with correct titles and seasons.
2. Play a 1080p HEVC episode — first frame in a couple of seconds, not a full
download.
3. Drag the seek bar to ~75%; playback resumes there within a block fetch.
4. Close and reopen; the resume position is honoured.
5. Leave it paused past the anonymous session's lifetime, then seek — the
session renews rather than erroring.
6. Watch the same episode again; it comes from the disk block cache.
7. Repeat on Windows with the bundled `libmpv-2.dll`.