feat(gui,fuse,core): video thumbnails in file views

My files, Shared, Trash and the launcher now ask the daemon for video
thumbnails, not only image ones. Grid tiles with a video thumbnail show
a small play badge.

A `.ts` file is a video from 1 MiB up and a TypeScript source below
that. One rule in pdfs-core (`is_video_file_name`) decides it for the
file icon, the thumbnail gate, the launcher icon and preview, and the
opener's `@media` / `@text` classes. The daemon also checks the MPEG-TS
sync byte before it runs ffmpeg, so a large TypeScript bundle gets no
retries.

A video thumbnail no longer downloads the whole file into memory (B283):
- the server thumbnail is tried first, at no content cost;
- files up to 24 MiB come whole;
- larger ones bring a 16 MiB head, plus the MP4/MOV `moov` box when it
  lies past the head, into a sparse staging file that ffmpeg reads.
The 256 MiB cap is gone. Without ffmpeg, a video answers "no thumbnail"
at once. The folder thumbnail build includes videos.
This commit is contained in:
nvrl committed 2026-11-14 02:17:18 +01:00
1 parent b49631e18f
commit 0183f17435
23 files changed
+4883 -3995

No files matched your search

+63
View File
@@ -1851,6 +1851,34 @@ pub fn is_thumbnail_image_name(name: &str) -> bool {
|| RAW_THUMBNAIL_EXTENSIONS.contains(&extension.as_str())
}
/// The smallest `.ts` file taken for an MPEG transport stream. `.ts` is
/// TypeScript source as often as it is a recording, and the two sit far apart
/// in size: a source file is kilobytes, while a second of broadcast video is
/// already megabytes. Drive often stores either kind as
/// `application/octet-stream`, so the size is the only signal a listing has.
pub const TS_VIDEO_MIN_BYTES: u64 = 1024 * 1024;
/// Whether a file named `name`, of `size` bytes when the caller knows it, is a
/// video. The extension decides, except for `.ts`, which counts as video only
/// from [`TS_VIDEO_MIN_BYTES`] up. A `.ts` of unknown size counts as video, as
/// [`PhotoKind::classify`] has it.
pub fn is_video_file_name(name: &str, size: Option<u64>) -> bool {
if PhotoKind::classify(Some(name), None) != PhotoKind::Video {
return false;
}
let transport_stream = name
.rsplit_once('.')
.is_some_and(|(_, extension)| extension.eq_ignore_ascii_case("ts"));
!transport_stream || size.is_none_or(|size| size >= TS_VIDEO_MIN_BYTES)
}
/// Whether file listings show a thumbnail for a file named `name`, of `size`
/// bytes when known: an image [`is_thumbnail_image_name`] accepts, or a video,
/// whose thumbnail is a frame the daemon takes with ffmpeg.
pub fn is_thumbnail_media_name(name: &str, size: Option<u64>) -> bool {
is_thumbnail_image_name(name) || is_video_file_name(name, size)
}
/// Whether `name` is one of the concrete RAW formats handled by the exiftool
/// embedded-preview path. Deliberately excludes generic/legacy grab-bags whose
/// preview availability is not dependable.
@@ -2675,6 +2703,41 @@ mod tests {
assert_eq!(PhotoKind::classify(Some("x.jpg"), Some("video/mp4")), Photo);
}
#[test]
fn a_ts_file_is_a_video_only_at_the_size_of_one() {
const GB: u64 = 1024 * 1024 * 1024;
assert!(!is_video_file_name("app.ts", Some(3 * 1024)));
assert!(!is_video_file_name(
"bundle.TS",
Some(TS_VIDEO_MIN_BYTES - 1)
));
assert!(is_video_file_name("recording.ts", Some(9 * GB)));
assert!(is_video_file_name("recording.TS", Some(TS_VIDEO_MIN_BYTES)));
assert!(is_video_file_name("recording.ts", None));
// Only `.ts` is in doubt; every other video name is one at any size.
assert!(is_video_file_name("clip.mkv", Some(10)));
assert!(is_video_file_name("clip.m2ts", Some(10)));
assert!(!is_video_file_name("notes.txt", Some(9 * GB)));
assert!(!is_video_file_name("ts", Some(9 * GB)));
}
#[test]
fn listings_thumbnail_images_and_videos() {
assert!(is_thumbnail_media_name("photo.JPG", Some(10)));
assert!(is_thumbnail_media_name("camera.nef", None));
assert!(is_thumbnail_media_name("clip.mp4", Some(10)));
assert!(is_thumbnail_media_name(
"recording.ts",
Some(64 * 1024 * 1024)
));
assert!(!is_thumbnail_media_name("app.ts", Some(2048)));
assert!(!is_thumbnail_media_name(
"song.flac",
Some(64 * 1024 * 1024)
));
assert!(!is_thumbnail_media_name("notes.txt", None));
}
/// Keep every extension accepted by the shared classifier covered. Drive
/// activation uses this classification before it knows whether opening a
/// file should stream through FUSE or materialise a complete cache entry;
+70 -7
View File
@@ -27,6 +27,7 @@ use std::os::unix::process::CommandExt;
use std::path::Path;
use std::process::{Command, Stdio};
use crate::control::is_video_file_name;
use serde::{Deserialize, Serialize};
/// Environment override for the terminal used by `"terminal": true` rules.
@@ -108,11 +109,12 @@ impl OpenWith {
/// a Markdown file). The rules are about *what the user opened*, so the
/// caller passes that name and the argv still gets the real path.
pub fn command_for_named(&self, path: &Path, name: &str, is_dir: bool) -> Vec<String> {
let kind = Kind::of(path, name, is_dir);
for rule in &self.rules {
if rule.command.is_empty() {
continue;
}
if !rule.patterns.iter().any(|p| matches(p, name, is_dir)) {
if !rule.patterns.iter().any(|p| matches(p, name, kind)) {
continue;
}
let argv = expand(&rule.command, path);
@@ -325,16 +327,46 @@ fn expand_env(token: &str) -> Option<String> {
value.filter(|v| !v.is_empty())
}
/// What is being opened, as far as the `@class` patterns care.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Kind {
Dir,
Video,
File,
}
impl Kind {
/// `.ts` names TypeScript and MPEG transport streams alike, so for that
/// one extension the size on disk decides (see
/// [`is_video_file_name`](crate::control::is_video_file_name)). No other
/// name costs a `stat`.
fn of(path: &Path, name: &str, is_dir: bool) -> Kind {
if is_dir {
return Kind::Dir;
}
let size = extension(name)
.eq_ignore_ascii_case("ts")
.then(|| std::fs::metadata(path).ok().map(|meta| meta.len()))
.flatten();
if is_video_file_name(name, size) {
Kind::Video
} else {
Kind::File
}
}
}
/// Whether `pattern` — a glob or an `@class` — matches this name.
fn matches(pattern: &str, name: &str, is_dir: bool) -> bool {
fn matches(pattern: &str, name: &str, kind: Kind) -> bool {
let name_lower = name.to_lowercase();
let file = kind == Kind::File;
match pattern.to_lowercase().as_str() {
"@any" => true,
"@dir" => is_dir,
"@text" => !is_dir && class_text(&name_lower),
"@document" => !is_dir && class_document(&name_lower),
"@image" => !is_dir && class_image(&name_lower),
"@media" => !is_dir && class_media(&name_lower),
"@dir" => kind == Kind::Dir,
"@text" => file && class_text(&name_lower),
"@document" => file && class_document(&name_lower),
"@image" => file && class_image(&name_lower),
"@media" => kind == Kind::Video || (file && class_media(&name_lower)),
other if other.starts_with('@') => false,
other => glob_match(other, &name_lower),
}
@@ -533,6 +565,37 @@ mod tests {
);
}
#[test]
fn a_ts_file_opens_as_media_or_text_by_its_size() {
let policy = OpenWith {
terminal: Some(vec!["alacritty".into(), "-e".into()]),
default: None,
rules: vec![
rule(&["@text"], &["nvim"], true),
rule(&["@media"], &["mpv"], false),
],
};
let dir = std::env::temp_dir().join(format!("pdfs-opener-ts-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let source = dir.join("blob-source");
std::fs::write(&source, "export const answer = 42;\n").unwrap();
let recording = dir.join("blob-recording");
std::fs::File::create(&recording)
.unwrap()
.set_len(crate::control::TS_VIDEO_MIN_BYTES)
.unwrap();
let source_argv = policy.command_for_named(&source, "app.ts", false);
let recording_argv = policy.command_for_named(&recording, "recording.ts", false);
// A file that is not on this machine yet has no size to go by.
let unknown_argv = policy.command_for_named(&dir.join("missing"), "recording.ts", false);
std::fs::remove_dir_all(&dir).unwrap();
assert_eq!(source_argv[..3], ["alacritty", "-e", "nvim"]);
assert_eq!(recording_argv[0], "mpv");
assert_eq!(unknown_argv[0], "mpv");
}
#[test]
fn an_empty_policy_is_xdg_open() {
let policy = OpenWith::default();
+17 -1
View File
@@ -29,7 +29,7 @@ use proton_drive_rs::proton_sdk::error::Result;
use proton_drive_rs::proton_sdk::ids::{DriveEventId, NodeUid, ShareId};
use proton_drive_rs::{
AvailableNameTarget, Device, DeviceType, DriveEvent, DriveEventScopeId, Node, NodeMoveItem,
ProtonDriveClient, RevisionReader, Thumbnail,
ProtonDriveClient, RevisionReader, Thumbnail, ThumbnailType,
};
/// One outcome per node of a batch request; see [`ProtonDriveClient::trash_nodes`].
@@ -118,6 +118,14 @@ pub(crate) trait DriveApi: Send + Sync {
async fn open_revision(&self, uid: &NodeUid) -> Result<Arc<dyn RevisionRead>>;
/// The file's thumbnail of `thumbnail_type`, or `None` when its revision
/// carries none.
async fn download_thumbnail(
&self,
uid: &NodeUid,
thumbnail_type: ThumbnailType,
) -> Result<Option<Vec<u8>>>;
async fn download_file_to_dyn(
&self,
uid: &NodeUid,
@@ -445,6 +453,14 @@ impl DriveApi for ProtonDriveClient {
Ok(Arc::new(reader))
}
async fn download_thumbnail(
&self,
uid: &NodeUid,
thumbnail_type: ThumbnailType,
) -> Result<Option<Vec<u8>>> {
ProtonDriveClient::download_thumbnail(self, uid, thumbnail_type).await
}
async fn download_file_to_dyn(
&self,
uid: &NodeUid,
+428 -68
View File
@@ -9,7 +9,7 @@ use std::collections::{HashMap, HashSet};
use std::ffi::OsStr;
use std::fs::OpenOptions;
use std::io::{Read as _, Write as _};
use std::os::unix::fs::OpenOptionsExt as _;
use std::os::unix::fs::{FileExt as _, OpenOptionsExt as _};
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::sync::atomic::{AtomicU64, Ordering};
@@ -20,14 +20,17 @@ use base64::Engine as _;
use pdfs_core::control::{
FileThumbRequest, MappingProgress, PhotoItem, PhotoKind, PhotoLocation, PhotoThumb, PlaceInfo,
ThumbnailBuildStatus, Topic, is_raw_image_name, is_thumbnail_image_name,
is_thumbnail_media_name,
};
use pdfs_core::db::{self, StoredPhoto};
use pdfs_core::{CoreError, CoreResult};
use proton_drive_rs::proton_sdk::error::Result as DriveResult;
use proton_drive_rs::proton_sdk::ids::NodeUid;
use proton_drive_rs::{NodeKind, PhotoTag, PhotoTagsUpdate, Thumbnail, ThumbnailType};
use tracing::{debug, info, warn};
use super::drive::RevisionRead;
use super::{
Core, PHOTOS_AVAILABLE, PHOTOS_SYNCED_MS, TIMELINE_ENRICH_CHUNK, TIMELINE_TTL, WriteAuthority,
node_size, now_ms, parse_uid,
@@ -58,9 +61,18 @@ const EXIFTOOL_TIMEOUT: Duration = Duration::from_secs(20);
static RAW_TEMP_NONCE: AtomicU64 = AtomicU64::new(0);
static EXIFTOOL_MISSING_WARNING: Once = Once::new();
static FFMPEG_MISSING_WARNING: Once = Once::new();
/// Largest video downloaded whole to grab one frame for its thumbnail. The
/// download lands in memory, so a long 4K recording is left without one.
const VIDEO_THUMB_MAX_BYTES: u64 = 256 * 1024 * 1024;
/// How much of a video's start a thumbnail downloads. The first frame, and
/// the one a second in, sit inside it at any common bitrate. A video is never
/// downloaded whole for one frame: a recording can be many gigabytes.
const VIDEO_HEAD_BYTES: u64 = 16 * 1024 * 1024;
/// A video no larger than this is downloaded whole: its head and tail would
/// cover most of it anyway.
const VIDEO_WHOLE_MAX_BYTES: u64 = 24 * 1024 * 1024;
/// Largest MP4/MOV index (`moov` box) fetched from past the head. An hour of
/// video indexes in a few MiB; anything larger is not worth one frame.
const VIDEO_INDEX_MAX_BYTES: u64 = 32 * 1024 * 1024;
/// Top-level boxes walked looking for the index before giving up.
const VIDEO_BOX_WALK_MAX: usize = 32;
/// A single frame is seconds of work even for 4K; this only stops a wedged
/// ffmpeg from holding a thumbnail permit.
const FFMPEG_TIMEOUT: Duration = Duration::from_secs(30);
@@ -245,8 +257,8 @@ struct RawPreviewUnavailable;
/// the photo will never have a thumbnail, and only that verdict is persisted.
enum ThumbAttempt {
Made(GeneratedThumb),
/// Decoded nothing — a format this build has no decoder for, or a video
/// too large to fetch for one frame. Permanent.
/// Decoded nothing: a format this build has no decoder for, a video
/// ffmpeg takes no frame from, or a `.ts` that is not one. Permanent.
Undecodable,
/// The photo couldn't be downloaded. Transient: try again next time.
Unavailable,
@@ -577,21 +589,6 @@ fn warn_ffmpeg_missing() {
});
}
/// A thumbnail from one frame of a downloaded video, which ffmpeg reads from a
/// staging copy. See [`video_file_thumbnail`].
fn video_thumbnail(
bytes: &[u8],
name: &str,
staging_dir: &Path,
ffmpeg: &OsStr,
timeout: Duration,
) -> ScaleAttempt {
let Some(file) = RawTempFile::create_in(staging_dir, bytes, name) else {
return ScaleAttempt::Unavailable;
};
video_file_thumbnail(&file.0, ffmpeg, timeout)
}
/// A thumbnail from one frame of the video at `path`. A second in, past the
/// black lead-in most clips open on, and the first frame for a clip shorter than
/// that. ffmpeg applies the clip's rotation, so a portrait phone video comes out
@@ -624,6 +621,151 @@ fn video_file_thumbnail(path: &Path, ffmpeg: &OsStr, timeout: Duration) -> Scale
ScaleAttempt::Undecodable
}
/// What a thumbnail is made from.
enum Fetched {
/// The thumbnail itself, from the server.
Ready(Vec<u8>),
/// Nothing: a video with no server thumbnail, and no ffmpeg to take one.
Nothing,
/// A whole image.
Whole(Vec<u8>),
Video(VideoSample),
}
/// The parts of a video its thumbnail needs, downloaded: the start of the
/// file, and for an MP4 or MOV whose index is further in, that index.
struct VideoSample {
/// The whole file's length.
size: u64,
head: Vec<u8>,
/// The `moov` box and its offset, when it is not inside `head`.
index: Option<(u64, Vec<u8>)>,
}
impl VideoSample {
/// Whether this could be an MPEG transport stream: one opens on the 0x47
/// sync byte, and repeats it every 188-byte packet. Only asked of `.ts`
/// files, which are TypeScript source as often as they are video.
fn is_transport_stream(&self) -> bool {
self.head.first() == Some(&0x47) && self.head.get(188).is_none_or(|&byte| byte == 0x47)
}
}
/// Download what a thumbnail of the video `reader` reads needs. A file of up
/// to [`VIDEO_WHOLE_MAX_BYTES`] comes whole. A larger one brings its first
/// [`VIDEO_HEAD_BYTES`], which is all a stream format (Matroska, WebM, MPEG-TS,
/// AVI) needs for a frame, plus the index of an ISO media file (MP4, MOV, 3GP)
/// written with its index after the media data, which is how most cameras and
/// phones write it.
async fn sample_video(reader: &dyn RevisionRead) -> DriveResult<VideoSample> {
let size = reader.size();
if size <= VIDEO_WHOLE_MAX_BYTES {
let head = reader.read_at(0, size).await?;
return Ok(VideoSample {
size,
head,
index: None,
});
}
let head = reader.read_at(0, VIDEO_HEAD_BYTES).await?;
let index = match iso_media_index(reader, &head, size).await? {
Some((offset, len)) if offset.saturating_add(len) > head.len() as u64 => {
Some((offset, reader.read_at(offset, len).await?))
}
_ => None,
};
Ok(VideoSample { size, head, index })
}
/// Where the `moov` box of an ISO media file sits, as `(offset, length)`.
/// `None` for a file that is not one, or whose index is missing, oversized or
/// past [`VIDEO_BOX_WALK_MAX`] boxes. Box headers past `head` are read one at a
/// time, 16 bytes each.
async fn iso_media_index(
reader: &dyn RevisionRead,
head: &[u8],
size: u64,
) -> DriveResult<Option<(u64, u64)>> {
const FIRST_BOXES: [&[u8; 4]; 6] = [b"ftyp", b"moov", b"mdat", b"wide", b"free", b"skip"];
if !head
.get(4..8)
.is_some_and(|kind| FIRST_BOXES.iter().any(|known| kind == *known))
{
return Ok(None);
}
let mut offset = 0u64;
for _ in 0..VIDEO_BOX_WALK_MAX {
if offset.saturating_add(8) > size {
break;
}
let header = match usize::try_from(offset)
.ok()
.and_then(|start| head.get(start..start.checked_add(16)?))
{
Some(header) => header.to_vec(),
None => reader.read_at(offset, 16).await?,
};
let Some((kind, len)) = iso_box_header(&header, size - offset) else {
break;
};
if kind == *b"moov" {
return Ok((len <= VIDEO_INDEX_MAX_BYTES).then_some((offset, len)));
}
offset = offset.saturating_add(len);
}
Ok(None)
}
/// The type and length of the ISO box whose header starts `header`, of a box
/// that may run at most `left` bytes to the end of the file.
fn iso_box_header(header: &[u8], left: u64) -> Option<([u8; 4], u64)> {
let size = u32::from_be_bytes(header.get(0..4)?.try_into().ok()?);
let kind: [u8; 4] = header.get(4..8)?.try_into().ok()?;
let len = match size {
// Runs to the end of the file.
0 => left,
// The real length follows as 64 bits.
1 => u64::from_be_bytes(header.get(8..16)?.try_into().ok()?),
len => u64::from(len),
};
(8..=left).contains(&len).then_some((kind, len))
}
/// A thumbnail from one frame of a downloaded [`VideoSample`]. The sample is
/// laid out in a staging file at the offsets it came from, so ffmpeg finds an
/// index where the file's own boxes say it is. The gap between holds no data
/// and costs no disk.
fn video_sample_thumbnail(
sample: &VideoSample,
name: &str,
staging_dir: &Path,
ffmpeg: &OsStr,
timeout: Duration,
) -> ScaleAttempt {
let transport_stream_name = Path::new(name)
.extension()
.is_some_and(|extension| extension.eq_ignore_ascii_case("ts"));
if transport_stream_name && !sample.is_transport_stream() {
return ScaleAttempt::Undecodable;
}
let Some(file) = RawTempFile::create_in(staging_dir, &sample.head, name) else {
return ScaleAttempt::Unavailable;
};
if let Some((offset, index)) = &sample.index {
let laid_out = OpenOptions::new()
.write(true)
.open(&file.0)
.and_then(|staged| {
staged.write_all_at(index, *offset)?;
staged.set_len(sample.size.max(offset + index.len() as u64))
});
if laid_out.is_err() {
return ScaleAttempt::Unavailable;
}
}
video_file_thumbnail(&file.0, ffmpeg, timeout)
}
/// The thumbnail to attach to an upload of the local file at `path`, which is
/// named `name` in Drive. Without one, Drive's own apps show a generic icon and
/// the gallery has to download the whole file to make its own.
@@ -1406,10 +1548,10 @@ impl Core {
.map(|item| (item.uid.clone(), item.name.clone()))
.collect();
// Proton only stores server-generated thumbnails for nodes in Photos.
// These requests come exclusively from ordinary Drive, Shared and Trash
// views, so downloading a remote thumbnail here would be a guaranteed
// miss. Generate directly from the full image and cache the result.
// Drive makes no thumbnails itself; a file has one only when its
// uploader attached it, as this client and Proton's apps do. The
// generation step asks for that first, and only makes one from the
// file's content without it.
let mut wanted = Vec::new();
let mut seen = HashSet::new();
for (raw, uid, modified) in &parsed {
@@ -1560,7 +1702,10 @@ impl Core {
return (uid, ThumbAttempt::Cancelled);
}
let video = is_video_name(&name);
if video && !ffmpeg_available() {
// Without ffmpeg a video can still have its uploader's
// thumbnail, but Photos has asked the server already.
let can_sample = !video || ffmpeg_available();
if !can_sample && matches!(job, ThumbJob::Photos) {
return (uid, ThumbAttempt::Unavailable);
}
@@ -1609,21 +1754,34 @@ impl Core {
return (uid, ThumbAttempt::Cancelled);
}
// A video is downloaded whole for one frame, so a long
// recording is not worth it — and would not fit in memory.
if video {
match core.photos().get_node(&uid).await {
Ok(Some(node)) if node_size(&node) > VIDEO_THUMB_MAX_BYTES => {
info!(%uid, size = node_size(&node), "video too large for a local thumbnail");
return (uid, ThumbAttempt::Undecodable);
}
Ok(Some(_)) => {}
Ok(None) | Err(_) => return (uid, ThumbAttempt::Unavailable),
}
}
let download_uid = uid.clone();
let download = async move { client.download_file(&download_uid).await };
let drive = core.drive.clone();
let download = async move {
// Photos asked the server in one batch already. A
// file view's or a build's file may carry a
// thumbnail its uploader made, which costs one
// small download instead of its content.
if !matches!(job, ThumbJob::Photos)
&& let Some(bytes) = drive
.download_thumbnail(&download_uid, ThumbnailType::Thumbnail)
.await?
.filter(|bytes| !bytes.is_empty())
{
return Ok(Fetched::Ready(bytes));
}
if !can_sample {
return Ok(Fetched::Nothing);
}
if video {
let reader = drive.open_revision(&download_uid).await?;
sample_video(reader.as_ref()).await.map(Fetched::Video)
} else {
client
.download_file(&download_uid)
.await
.map(Fetched::Whole)
}
};
tokio::pin!(download);
let downloaded = match job {
ThumbJob::Files(_) => loop {
@@ -1661,10 +1819,10 @@ impl Core {
let Some(downloaded) = downloaded else {
return (uid, ThumbAttempt::Cancelled);
};
let bytes = match downloaded {
Ok(bytes) => bytes,
let fetched = match downloaded {
Ok(fetched) => fetched,
Err(e) => {
warn!(%uid, error = %e, "image download for thumbnail failed");
warn!(%uid, error = %e, "download for a thumbnail failed");
return (uid, ThumbAttempt::Unavailable);
}
};
@@ -1674,18 +1832,20 @@ impl Core {
// Decoding + scaling a 20 MP JPEG is CPU-bound and would
// stall the runtime's worker; hand it to the blocking pool.
let staging_dir = core.cache.raw_thumbnail_staging_dir().to_path_buf();
let made = tokio::task::spawn_blocking(move || {
if video {
video_thumbnail(
&bytes,
&name,
&staging_dir,
OsStr::new("ffmpeg"),
FFMPEG_TIMEOUT,
)
} else {
scale_thumbnail(&bytes, &name, &staging_dir)
}
let made = tokio::task::spawn_blocking(move || match fetched {
Fetched::Video(sample) => video_sample_thumbnail(
&sample,
&name,
&staging_dir,
OsStr::new("ffmpeg"),
FFMPEG_TIMEOUT,
),
Fetched::Whole(bytes) => scale_thumbnail(&bytes, &name, &staging_dir),
Fetched::Nothing => ScaleAttempt::Unavailable,
Fetched::Ready(bytes) => match ratio_of(&bytes) {
Some(ratio) => ScaleAttempt::Made(GeneratedThumb { bytes, ratio }),
None => ScaleAttempt::Undecodable,
},
})
.await
.unwrap_or(ScaleAttempt::Unavailable);
@@ -1869,7 +2029,7 @@ impl Core {
if seen_folders.insert(entry.uid) {
folders.push(folder.join(entry.name));
}
} else if is_thumbnail_image_name(&entry.name) {
} else if is_thumbnail_media_name(&entry.name, Some(entry.size)) {
match parse_uid(&entry.uid) {
Some(uid) if seen_images.insert(uid.clone()) => {
found_here += 1;
@@ -2295,15 +2455,19 @@ mod decode_budget_tests {
mod thumb_tests {
use super::{
RawTempFile, ScaleAttempt, THUMB_EDGE, UPLOAD_IMAGE_MAX_BYTES, UPLOAD_THUMB_MAX_BYTES,
can_make_thumbnail, exiftool_binary, ffmpeg_available, fit_upload_thumb, is_video_name,
ratio_of, scale_thumbnail, scale_thumbnail_with_exiftool, similar_hashes,
thumbnail_build_may_start, upload_thumbnails_with, video_thumbnail,
VIDEO_HEAD_BYTES, VIDEO_WHOLE_MAX_BYTES, VideoSample, can_make_thumbnail, exiftool_binary,
ffmpeg_available, fit_upload_thumb, is_video_name, ratio_of, sample_video, scale_thumbnail,
scale_thumbnail_with_exiftool, similar_hashes, thumbnail_build_may_start,
upload_thumbnails_with, video_sample_thumbnail,
};
use crate::drive::RevisionRead;
use pdfs_core::control::{ThumbnailBuildStatus, is_thumbnail_image_name};
use proton_drive_rs::ThumbnailType;
use proton_drive_rs::proton_sdk::error::Result as DriveResult;
use std::collections::HashMap;
use std::ffi::OsStr;
use std::os::unix::fs::PermissionsExt as _;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::Duration;
/// A `width`×`height` JPEG, standing in for a camera photo the server never
@@ -2536,6 +2700,199 @@ mod thumb_tests {
std::env::temp_dir().join(format!("pdfs-{test}-{}", std::process::id()))
}
/// A sample of a video small enough to come whole.
fn whole(bytes: &[u8]) -> VideoSample {
VideoSample {
size: bytes.len() as u64,
head: bytes.to_vec(),
index: None,
}
}
fn block_on<F: std::future::Future>(future: F) -> F::Output {
tokio::runtime::Builder::new_current_thread()
.build()
.unwrap()
.block_on(future)
}
/// A Drive revision of `size` bytes, zero except for the parts written in,
/// that counts the bytes read from it. Stands for a recording far larger
/// than a test should hold in memory.
struct FakeRevision {
size: u64,
parts: Vec<(u64, Vec<u8>)>,
read: AtomicU64,
}
impl FakeRevision {
fn new(size: u64) -> Self {
Self {
size,
parts: Vec::new(),
read: AtomicU64::new(0),
}
}
fn with(mut self, offset: u64, bytes: Vec<u8>) -> Self {
self.parts.push((offset, bytes));
self
}
}
#[async_trait::async_trait]
impl RevisionRead for FakeRevision {
fn size(&self) -> u64 {
self.size
}
fn block_sizes(&self) -> &[u64] {
&[]
}
async fn read_at(&self, offset: u64, length: u64) -> DriveResult<Vec<u8>> {
let end = offset.saturating_add(length).min(self.size);
let mut out = vec![0u8; end.saturating_sub(offset) as usize];
for (start, bytes) in &self.parts {
let part_end = start + bytes.len() as u64;
let (from, to) = (offset.max(*start), end.min(part_end));
if from < to {
out[(from - offset) as usize..(to - offset) as usize]
.copy_from_slice(&bytes[(from - start) as usize..(to - start) as usize]);
}
}
self.read.fetch_add(out.len() as u64, Ordering::Relaxed);
Ok(out)
}
}
/// An ISO media box of `len` bytes, of which only the header is returned.
fn iso_box_header(kind: &[u8; 4], len: u64) -> Vec<u8> {
if let Ok(len) = u32::try_from(len) {
[&len.to_be_bytes()[..], kind].concat()
} else {
[&1u32.to_be_bytes()[..], kind, &len.to_be_bytes()[..]].concat()
}
}
const GIB: u64 = 1024 * 1024 * 1024;
#[test]
fn a_large_stream_video_downloads_its_head_alone() {
let mut head = vec![0u8; 188 * 2];
head[0] = 0x47;
head[188] = 0x47;
let revision = FakeRevision::new(9 * GIB).with(0, head);
let sample = block_on(sample_video(&revision)).unwrap();
assert_eq!(sample.size, 9 * GIB);
assert_eq!(sample.head.len() as u64, VIDEO_HEAD_BYTES);
assert!(sample.index.is_none());
assert!(sample.is_transport_stream());
assert_eq!(revision.read.load(Ordering::Relaxed), VIDEO_HEAD_BYTES);
}
/// A camera MP4 keeps its index after gigabytes of media data. The sample
/// is the head, the box headers on the way, and the index itself.
#[test]
fn a_large_mp4_downloads_its_head_and_its_index() {
let ftyp = [
iso_box_header(b"ftyp", 24),
b"isom\0\0\0\0mp41isom".to_vec(),
]
.concat();
let mdat_len = 6 * GIB;
let moov_at = ftyp.len() as u64 + mdat_len;
let moov = [iso_box_header(b"moov", 4096), vec![7u8; 4088]].concat();
let size = moov_at + moov.len() as u64;
let revision = FakeRevision::new(size)
.with(0, ftyp.clone())
.with(ftyp.len() as u64, iso_box_header(b"mdat", mdat_len))
.with(moov_at, moov.clone());
let sample = block_on(sample_video(&revision)).unwrap();
assert_eq!(sample.index, Some((moov_at, moov)));
assert!(
revision.read.load(Ordering::Relaxed) <= VIDEO_HEAD_BYTES + 4096 + 16,
"read {} bytes",
revision.read.load(Ordering::Relaxed)
);
}
#[test]
fn an_mp4_with_its_index_up_front_downloads_its_head_alone() {
let ftyp = [iso_box_header(b"ftyp", 16), b"isom\0\0\0\0".to_vec()].concat();
let moov = [iso_box_header(b"moov", 64), vec![1u8; 56]].concat();
let revision = FakeRevision::new(GIB)
.with(0, ftyp.clone())
.with(ftyp.len() as u64, moov)
.with(80, iso_box_header(b"mdat", GIB - 80));
let sample = block_on(sample_video(&revision)).unwrap();
assert!(sample.index.is_none());
assert_eq!(revision.read.load(Ordering::Relaxed), VIDEO_HEAD_BYTES);
}
#[test]
fn a_small_video_downloads_whole() {
let revision = FakeRevision::new(VIDEO_WHOLE_MAX_BYTES).with(0, vec![3u8; 64]);
let sample = block_on(sample_video(&revision)).unwrap();
assert_eq!(sample.head.len() as u64, VIDEO_WHOLE_MAX_BYTES);
assert!(sample.index.is_none());
}
/// `.ts` that is TypeScript reaches ffmpeg never: the sync byte is missing,
/// and that is a final answer, not a retry.
#[test]
fn a_ts_without_the_sync_byte_is_no_video() {
let attempt = video_sample_thumbnail(
&whole(b"export const answer = 42;"),
"bundle.ts",
&std::env::temp_dir(),
OsStr::new("pdfs-ffmpeg-deliberately-absent"),
Duration::from_millis(10),
);
assert!(matches!(attempt, ScaleAttempt::Undecodable));
}
/// ffmpeg reads the index where the file's own boxes put it, so the staging
/// file has the full length and the index at its offset.
#[test]
fn the_index_is_staged_at_its_own_offset() {
let tmp = scratch("video-index");
let frame = tmp.join("frame.jpg");
std::fs::create_dir_all(&tmp).unwrap();
std::fs::write(&frame, jpeg(640, 360)).unwrap();
let size: u64 = 3 * GIB;
let at: u64 = size - 4096;
// Arguments: -nostdin -loglevel error -ss <seek> -i <path> …
let ffmpeg = fake_ffmpeg(
&tmp,
&format!(
"[ \"$(stat -c %s \"$7\")\" = {size} ] && \
[ \"$(tail -c +{} \"$7\" | head -c 4)\" = moov ] && cat '{}'",
// The box type, after its 4-byte length; `tail` counts from 1.
at + 5,
frame.display()
),
);
let sample = VideoSample {
size,
head: vec![0u8; 1024],
index: Some((
at,
[iso_box_header(b"moov", 4096), vec![0u8; 4088]].concat(),
)),
};
let attempt = video_sample_thumbnail(
&sample,
"clip.mp4",
&tmp,
ffmpeg.as_os_str(),
Duration::from_secs(10),
);
std::fs::remove_dir_all(&tmp).unwrap();
assert!(matches!(attempt, ScaleAttempt::Made(_)));
}
#[test]
fn a_video_frame_becomes_a_thumbnail() {
let tmp = scratch("video-frame");
@@ -2544,8 +2901,8 @@ mod thumb_tests {
std::fs::write(&frame, jpeg(1080, 1920)).unwrap();
let ffmpeg = fake_ffmpeg(&tmp, &format!("cat '{}'", frame.display()));
let ScaleAttempt::Made(thumb) = video_thumbnail(
b"video bytes",
let ScaleAttempt::Made(thumb) = video_sample_thumbnail(
&whole(b"video bytes"),
"clip.mp4",
&tmp,
ffmpeg.as_os_str(),
@@ -2561,8 +2918,8 @@ mod thumb_tests {
fn a_video_ffmpeg_cannot_read_is_a_permanent_miss() {
let tmp = scratch("video-broken");
let ffmpeg = fake_ffmpeg(&tmp, "exit 1");
let attempt = video_thumbnail(
b"not a video",
let attempt = video_sample_thumbnail(
&whole(b"not a video"),
"clip.mp4",
&tmp,
ffmpeg.as_os_str(),
@@ -2574,8 +2931,8 @@ mod thumb_tests {
#[test]
fn missing_ffmpeg_is_retryable() {
let attempt = video_thumbnail(
b"video bytes",
let attempt = video_sample_thumbnail(
&whole(b"video bytes"),
"clip.mp4",
&std::env::temp_dir(),
OsStr::new("pdfs-ffmpeg-deliberately-absent"),
@@ -2591,9 +2948,12 @@ mod thumb_tests {
let path = std::env::var_os("PDFS_VIDEO_FIXTURE").expect("PDFS_VIDEO_FIXTURE");
let path = std::path::PathBuf::from(path);
let name = path.file_name().unwrap().to_string_lossy();
let bytes = std::fs::read(&path).unwrap();
let ScaleAttempt::Made(thumb) = video_thumbnail(
&bytes,
// Through the same partial download a Drive video gets.
let revision = FakeRevision::new(std::fs::read(&path).unwrap().len() as u64)
.with(0, std::fs::read(&path).unwrap());
let sample = block_on(sample_video(&revision)).unwrap();
let ScaleAttempt::Made(thumb) = video_sample_thumbnail(
&sample,
&name,
&std::env::temp_dir(),
OsStr::new("ffmpeg"),
+106
View File
@@ -3619,4 +3619,110 @@ mod tests {
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
/// A video in My files gets a thumbnail from a frame of its own, but a
/// recording can be gigabytes: only its start, and an MP4's index, are
/// downloaded for it. A video with a thumbnail from its uploader is not
/// downloaded at all, and a `.ts` that is TypeScript gets no thumbnail.
#[test]
#[ignore = "mounts FUSE: run with `cargo test -p pdfs-fuse sim:: -- --ignored`"]
fn a_video_thumbnail_downloads_a_small_part_of_the_video() {
if std::process::Command::new("ffmpeg")
.arg("-version")
.output()
.is_err()
{
eprintln!("skipped: needs ffmpeg");
return;
}
let dir = scratch("video-thumbs");
std::fs::create_dir_all(&dir).unwrap();
// Twenty seconds at 30 Mbit/s: larger than a thumbnail downloads whole.
// An MP4 written this way has its index at the end.
let encode = |name: &str| {
let path = dir.join(name);
let status = std::process::Command::new("ffmpeg")
.args(["-nostdin", "-loglevel", "error", "-y", "-f", "lavfi"])
.args(["-i", "testsrc2=size=1280x720:rate=30", "-t", "20"])
.args(["-c:v", "libx264", "-preset", "ultrafast"])
.args(["-b:v", "30M", "-maxrate", "30M", "-bufsize", "30M"])
.arg(&path)
.status()
.unwrap();
assert!(status.success());
let bytes = std::fs::read(&path).unwrap();
std::fs::remove_file(&path).unwrap();
assert!(bytes.len() > 30 << 20, "{name} is {} bytes", bytes.len());
bytes
};
let (recording, clip) = (encode("recording.ts"), encode("clip.mp4"));
let source = b"export const answer = 42;\n".repeat(64 * 1024);
let drive = FakeDrive::new();
drive.device().write("recording.ts", &recording);
drive.device().write("clip.mp4", &clip);
drive.device().write("app.ts", &source);
let phone = drive.device().write("phone.mp4", &clip);
let mut frame = Vec::new();
image::codecs::jpeg::JpegEncoder::new(&mut frame)
.encode_image(&image::RgbImage::new(320, 180))
.unwrap();
let client = drive.client(1, Faults::lan());
client.set_thumbnail(&phone, frame);
let daemon = Daemon::start(&dir.join("daemon"), client.clone()).unwrap();
let Ok(Response::Entries { entries }) = daemon.request(&Request::ListDir {
path: String::new(),
}) else {
panic!("listing");
};
let items: Vec<control::FileThumbRequest> = entries
.iter()
.filter(|entry| !entry.is_dir)
.map(|entry| control::FileThumbRequest {
uid: entry.uid.clone(),
modified: entry.modified,
name: entry.name.clone(),
})
.collect();
let Ok(Response::FileThumbGeneration { generation }) =
daemon.request(&Request::ReserveFileThumbGeneration)
else {
panic!("generation");
};
let read_before = client.content_read();
let mut paths = std::collections::HashMap::new();
let settled = wait_until(Duration::from_secs(120), || {
let Ok(Response::Thumbs { items: replies }) = daemon.request(&Request::FileThumbs {
items: items.clone(),
generation,
}) else {
return false;
};
paths = replies
.iter()
.filter_map(|reply| {
let name = &items.iter().find(|item| item.uid == reply.uid)?.name;
Some((name.clone(), (reply.path.clone(), reply.pending)))
})
.collect();
paths.values().all(|(_, pending)| !pending) && paths.len() == 4
});
assert!(settled, "{paths:?}");
let read = client.content_read() - read_before;
for name in ["recording.ts", "clip.mp4", "phone.mp4"] {
assert!(paths[name].0.is_some(), "{name} has a thumbnail: {paths:?}");
}
assert_eq!(paths["app.ts"].0, None, "TypeScript is no video");
// Two 16 MiB heads, one MP4 index and the TypeScript file, and nothing
// of the video that came with a thumbnail.
assert!(
read < 34 << 20,
"downloaded {} MiB for three thumbnails",
read >> 20
);
assert!(matches!(daemon.stop(), Ok(MountOutcome::Shutdown)));
let _ = std::fs::remove_dir_all(&dir);
}
}
+43 -3
View File
@@ -62,7 +62,7 @@ use proton_drive_rs::proton_sdk::ids::{
};
use proton_drive_rs::{
AvailableNameTarget, Device as Computer, DeviceType, DriveEvent, DriveEventScopeId, Node,
NodeKind, NodeMoveItem, RevisionState, Thumbnail,
NodeKind, NodeMoveItem, RevisionState, Thumbnail, ThumbnailType,
};
use sha1::{Digest as _, Sha1};
@@ -396,6 +396,8 @@ impl FakeDrive {
hold_answer: Mutex::new(None),
hold_trash: Mutex::new(None),
hold_download: Mutex::new(None),
thumbnails: Mutex::new(HashMap::new()),
content_read: AtomicU64::new(0),
hold_any_landing: AtomicBool::new(false),
hold_landing: Mutex::new(None),
drop_after_create: AtomicBool::new(false),
@@ -677,6 +679,10 @@ struct ClientInner {
hold_trash: Mutex<Option<Arc<HoldState>>>,
/// The download [`FakeClient::hold_next_download`] is waiting for.
hold_download: Mutex<Option<Arc<HoldState>>>,
/// Thumbnails set by [`FakeClient::set_thumbnail`].
thumbnails: Mutex<HashMap<NodeUid, Vec<u8>>>,
/// File content bytes served, by downloads and ranged reads alike.
content_read: AtomicU64,
/// The landing [`FakeClient::hold_next_landing`] is waiting for.
hold_landing: Mutex<Option<Arc<HoldState>>>,
/// Set by [`FakeClient::hold_next_landing_of_any`]: hold whichever node
@@ -821,6 +827,16 @@ impl FakeClient {
/// Hold the next download of `uid`'s content once Drive has answered it,
/// before the bytes reach the daemon, until the answer is dropped.
/// Give `uid` a server thumbnail, as an upload that carried one would.
pub(crate) fn set_thumbnail(&self, uid: &NodeUid, bytes: Vec<u8>) {
self.0.thumbnails.lock().insert(uid.clone(), bytes);
}
/// File content bytes served so far, by whole downloads and ranged reads.
pub(crate) fn content_read(&self) -> u64 {
self.0.content_read.load(Ordering::SeqCst)
}
pub(crate) fn hold_next_download(&self, uid: &NodeUid) -> Held {
let state = Arc::new(HoldState {
uid: uid.clone(),
@@ -1475,6 +1491,21 @@ impl DriveApi for FakeClient {
}))
}
async fn download_thumbnail(
&self,
uid: &NodeUid,
_thumbnail_type: ThumbnailType,
) -> Result<Option<Vec<u8>>> {
self.request(false, |server| {
server
.current(uid)
.map(|_| ())
.ok_or_else(|| api(ResponseCode::DoesNotExist, "file not found"))
})
.await?;
Ok(self.0.thumbnails.lock().get(uid).cloned())
}
async fn download_file_to_dyn(
&self,
uid: &NodeUid,
@@ -1490,6 +1521,9 @@ impl DriveApi for FakeClient {
.await?;
self.wait_if_held(&self.0.hold_download, std::slice::from_ref(uid))
.await;
self.0
.content_read
.fetch_add(content.len() as u64, Ordering::SeqCst);
output.write_all(&content).map_err(io_error)
}
@@ -1733,13 +1767,19 @@ impl RevisionRead for FakeRevision {
async fn read_at(&self, offset: u64, length: u64) -> Result<Vec<u8>> {
let content = self.content.clone();
self.client
let read = self
.client
.request(false, move |_| {
let start = (offset as usize).min(content.len());
let end = (offset.saturating_add(length) as usize).min(content.len());
Ok(content[start..end].to_vec())
})
.await
.await?;
self.client
.0
.content_read
.fetch_add(read.len() as u64, Ordering::SeqCst);
Ok(read)
}
}
+42 -11
View File
@@ -1728,8 +1728,9 @@ fn repaint_thumbnail_build(ui: &Rc<Ui>, status: &ThumbnailBuildStatus) {
let text = if status.scanning {
ui.browser.thumbnail_progress.pulse();
let folders = ngettext_f("{n} folder", "{n} folders", status.folders_scanned, &[]);
let images = ngettext_f("{n} image", "{n} images", status.images_found, &[]);
// Translators: {root} is a folder path or "Proton Drive"; {folders} and {images} are counts such as "3 folders" and "12 images".
// Translators: a count of images and videos that can have a thumbnail.
let images = ngettext_f("{n} file", "{n} files", status.images_found, &[]);
// Translators: {root} is a folder path or "Proton Drive"; {folders} and {images} are counts such as "3 folders" and "12 files".
gettext_f(
"Scanning {root}… {folders}, {images}",
&[("root", root), ("folders", &folders), ("images", &images)],
@@ -1743,7 +1744,7 @@ fn repaint_thumbnail_build(ui: &Rc<Ui>, status: &ThumbnailBuildStatus) {
ui.browser
.thumbnail_progress
.set_fraction(fraction.clamp(0.0, 1.0));
// Translators: {root} is a folder path or "Proton Drive"; {done} and {total} are image counts.
// Translators: {root} is a folder path or "Proton Drive"; {done} and {total} are counts of images and videos.
gettext_f(
"Building thumbnails in {root}… {done} of {total}",
&[
@@ -1756,17 +1757,17 @@ fn repaint_thumbnail_build(ui: &Rc<Ui>, status: &ThumbnailBuildStatus) {
ui.browser.thumbnail_progress.set_fraction(1.0);
let available = status.completed.saturating_sub(status.failed);
match status.failed {
// Translators: {root} is a folder path or "Proton Drive".
// Translators: {root} is a folder path or "Proton Drive"; {n} counts images and videos.
0 => ngettext_f(
"Thumbnails ready for {n} image in {root}",
"Thumbnails ready for {n} images in {root}",
"Thumbnails ready for {n} file in {root}",
"Thumbnails ready for {n} files in {root}",
available,
&[("root", root)],
),
// Translators: {root} is a folder path or "Proton Drive"; {failed} is how many images could not get a thumbnail.
// Translators: {root} is a folder path or "Proton Drive"; {failed} is how many images and videos could not get a thumbnail.
failed => ngettext_f(
"Thumbnails ready for {n} image in {root}; {failed} unavailable",
"Thumbnails ready for {n} images in {root}; {failed} unavailable",
"Thumbnails ready for {n} file in {root}; {failed} unavailable",
"Thumbnails ready for {n} files in {root}; {failed} unavailable",
available,
&[("root", root), ("failed", &failed.to_string())],
),
@@ -2765,6 +2766,11 @@ pub(crate) fn icon_base_for(entry: &DirEntry) -> &'static str {
if entry.is_dir {
return "folder";
}
// Every video type, including a `.ts` recording, which by its name alone
// is TypeScript.
if pdfs_core::control::is_video_file_name(&entry.name, Some(entry.size)) {
return "video-x-generic";
}
let ext = entry
.name
.rsplit('.')
@@ -2775,7 +2781,6 @@ pub(crate) fn icon_base_for(entry: &DirEntry) -> &'static str {
"png" | "jpg" | "jpeg" | "gif" | "webp" | "bmp" | "svg" | "heic" | "tiff" => {
"image-x-generic"
}
"mp4" | "mkv" | "mov" | "avi" | "webm" | "m4v" => "video-x-generic",
"mp3" | "flac" | "wav" | "ogg" | "opus" | "m4a" => "audio-x-generic",
"pdf" | "doc" | "docx" | "odt" => "x-office-document",
"xls" | "xlsx" | "ods" | "csv" => "x-office-spreadsheet",
@@ -3122,7 +3127,7 @@ fn listing_summary(files: usize, folders: usize, file_bytes: u64) -> String {
#[cfg(test)]
mod tests {
use super::{
MoveTarget, badge_for, copy_target_allowed, hit_location, listing_summary,
MoveTarget, badge_for, copy_target_allowed, hit_location, icon_base_for, listing_summary,
move_target_allowed, show_thumbnail_build_progress, sort_entries,
};
use pdfs_core::config::{FileSort, FilesView};
@@ -3149,6 +3154,32 @@ mod tests {
}
}
#[test]
fn every_video_type_gets_the_video_icon() {
const MIB: u64 = 1024 * 1024;
for name in [
"clip.mp4",
"clip.MKV",
"broadcast.m2ts",
"camera.mts",
"old.flv",
] {
assert_eq!(
icon_base_for(&entry(name, false, 10, 0)),
"video-x-generic",
"{name}"
);
}
assert_eq!(
icon_base_for(&entry("recording.ts", false, 900 * MIB, 0)),
"video-x-generic"
);
assert_eq!(
icon_base_for(&entry("app.ts", false, 4096, 0)),
"text-x-generic"
);
}
fn names(entries: &[DirEntry]) -> Vec<&str> {
entries.iter().map(|e| e.name.as_str()).collect()
}
+48 -10
View File
@@ -1,4 +1,4 @@
//! Shared thumbnails for ordinary Drive image files.
//! Shared thumbnails for ordinary Drive image and video files.
//!
//! The Photos gallery has timeline-specific batching and aspect-ratio state.
//! File listings need a smaller, reusable surface: one loader backs My files,
@@ -6,7 +6,7 @@
//! so a replaced image never flashes its previous contents.
use crate::*;
use pdfs_core::control::{FileThumbRequest, is_thumbnail_image_name};
use pdfs_core::control::{FileThumbRequest, is_thumbnail_media_name, is_video_file_name};
const THUMB_BATCH: usize = 32;
const THUMB_CACHE_MAX: usize = 256;
@@ -125,6 +125,18 @@ pub(crate) fn file_thumbnail_widget(size: i32, fallback_size: i32) -> gtk4::Over
.build();
picture.add_css_class("file-thumbnail");
// Over a video's frame, so it does not pass for a photo.
let play = gtk4::Image::builder()
.icon_name("media-playback-start-symbolic")
.pixel_size(16)
.halign(gtk4::Align::Center)
.valign(gtk4::Align::Center)
.can_target(false)
.visible(false)
.build();
play.add_css_class("photo-video-badge");
play.add_css_class("photo-video-badge-small");
let overlay = gtk4::Overlay::new();
overlay.set_width_request(size);
overlay.set_height_request(size);
@@ -133,9 +145,21 @@ pub(crate) fn file_thumbnail_widget(size: i32, fallback_size: i32) -> gtk4::Over
overlay.set_overflow(gtk4::Overflow::Hidden);
overlay.set_child(Some(&fallback));
overlay.add_overlay(&picture);
overlay.add_overlay(&play);
overlay
}
/// The play badge [`file_thumbnail_widget`] puts after `picture`.
fn play_badge(picture: &gtk4::Picture) -> Option<gtk4::Image> {
picture
.next_sibling()
.and_downcast::<gtk4::Image>()
.filter(|image| image.has_css_class("photo-video-badge"))
}
/// Marks a play badge to show once its video's frame is painted.
const VIDEO_BADGE: &str = "video";
/// Resize an existing thumbnail surface. Grid rows are recycled, so changing
/// these requests in place and rebinding the model is enough to implement the
/// status-bar zoom without downloading or decoding an image again.
@@ -179,6 +203,16 @@ pub(crate) fn bind_file_thumbnail(
fallback.set_visible(true);
picture.set_visible(false);
picture.set_paintable(gtk4::gdk::Paintable::NONE);
// List rows are too small for a badge; their icon says video already.
let video = !entry.is_dir && is_video_file_name(&entry.name, Some(entry.size));
if let Some(play) = play_badge(&picture) {
play.set_visible(false);
play.set_widget_name(if video && !symbolic_fallback {
VIDEO_BADGE
} else {
""
});
}
// The widget name is a cheap binding token. List factories recycle their
// children; a reply for the previous entry must not paint into the new one.
@@ -186,7 +220,10 @@ pub(crate) fn bind_file_thumbnail(
picture.set_widget_name(&key);
remove_waiter(ui, &picture);
if entry.is_dir || !is_image_name(&entry.name) || entry.uid.is_empty() {
if entry.is_dir
|| !is_thumbnail_media_name(&entry.name, Some(entry.size))
|| entry.uid.is_empty()
{
return;
}
if let Some(texture) = ui.file_thumbs.textures.borrow().get(&key) {
@@ -214,12 +251,6 @@ pub(crate) fn bind_file_thumbnail(
schedule_file_thumbs(ui);
}
/// Whether the file name denotes an image format for which Proton or the local
/// decoder can reasonably provide a thumbnail.
pub(crate) fn is_image_name(name: &str) -> bool {
is_thumbnail_image_name(name)
}
fn thumbnail_key(uid: &str, modified: i64) -> String {
format!("{uid}:{modified}")
}
@@ -238,6 +269,9 @@ fn paint(picture: &gtk4::Picture, key: &str, texture: &gtk4::gdk::Texture) {
}
picture.set_paintable(Some(texture));
picture.set_visible(true);
if let Some(play) = play_badge(picture) {
play.set_visible(play.widget_name() == VIDEO_BADGE);
}
if let Some(overlay) = picture.parent().and_downcast::<gtk4::Overlay>()
&& let Some(fallback) = overlay.child().and_downcast::<gtk4::Image>()
{
@@ -413,7 +447,11 @@ fn retry_file_thumbs(ui: &Rc<Ui>, items: Vec<FileThumbRequest>) {
#[cfg(test)]
mod tests {
use super::{ReplyDisposition, is_image_name, reply_disposition};
use super::{ReplyDisposition, reply_disposition};
fn is_image_name(name: &str) -> bool {
pdfs_core::control::is_thumbnail_image_name(name)
}
#[test]
fn image_extensions_are_case_insensitive_and_specific() {
+2 -2
View File
@@ -10,7 +10,7 @@ use std::time::Duration;
use adw::prelude::*;
use gtk4::{gio, glib};
use super::row::{fill_slot, thumb_slot};
use super::row::{content_type_for, fill_slot, thumb_slot};
use super::thumbs::Thumbs;
use crate::{Hit, dates, gettext, human_bytes, pgettext};
@@ -235,7 +235,7 @@ impl Preview {
let content_type = if hit.is_dir() {
glib::GString::from("inode/directory")
} else {
gio::functions::content_type_guess(Some(hit.name()), None).0
content_type_for(hit.name(), hit.size())
};
self.kind
.set_label(&gio::functions::content_type_get_description(&content_type));
+18 -5
View File
@@ -6,7 +6,7 @@ use std::path::{Path, PathBuf};
use std::rc::Rc;
use adw::prelude::*;
use gtk4::{gdk, gio};
use gtk4::{gdk, gio, glib};
use super::highlight;
use super::thumbs::Thumbs;
@@ -53,7 +53,7 @@ pub(crate) fn fill_slot(slot: &gtk4::Overlay, hit: &Hit, thumbs: &Rc<Thumbs>, mo
let Some(picture) = fallback.next_sibling().and_downcast::<gtk4::Picture>() else {
return;
};
fallback.set_from_gicon(&icon_for(hit.name(), hit.is_dir()));
fallback.set_from_gicon(&icon_for(hit.name(), hit.is_dir(), hit.size()));
fallback.set_visible(true);
picture.set_visible(false);
picture.set_paintable(gdk::Paintable::NONE);
@@ -63,7 +63,7 @@ pub(crate) fn fill_slot(slot: &gtk4::Overlay, hit: &Hit, thumbs: &Rc<Thumbs>, mo
/// The desktop's own full-colour icon for a file name, so results look like
/// the user's file manager. Folders are Proton's purple ones while the Proton
/// theme is on, as in the app's browser.
pub(crate) fn icon_for(name: &str, is_dir: bool) -> gio::Icon {
pub(crate) fn icon_for(name: &str, is_dir: bool, size: u64) -> gio::Icon {
if is_dir {
let folder = if proton_theme_active() {
"pdfs-proton-folder"
@@ -72,8 +72,21 @@ pub(crate) fn icon_for(name: &str, is_dir: bool) -> gio::Icon {
};
return gio::ThemedIcon::new(folder).upcast();
}
let (content_type, _uncertain) = gio::functions::content_type_guess(Some(name), None);
gio::functions::content_type_get_icon(&content_type)
gio::functions::content_type_get_icon(&content_type_for(name, size))
}
/// The content type of a file named `name` of `size` bytes, guessed from the
/// name. GIO takes every `.ts` for TypeScript (or Qt translations); a
/// recording's size says otherwise (see
/// [`is_video_file_name`](pdfs_core::control::is_video_file_name)).
pub(crate) fn content_type_for(name: &str, size: u64) -> glib::GString {
let transport_stream = std::path::Path::new(name)
.extension()
.is_some_and(|extension| extension.eq_ignore_ascii_case("ts"));
if transport_stream && pdfs_core::control::is_video_file_name(name, Some(size)) {
return glib::GString::from("video/mp2t");
}
gio::functions::content_type_guess(Some(name), None).0
}
/// The sync badge of a Drive file, as in the app's browser: pinned, cached,
+12 -4
View File
@@ -9,8 +9,8 @@
//! matches the file's;
//! 2. for local images, decoding the file itself, and writing the result back
//! to that cache for the next time and for every other application;
//! 3. for Drive images, the daemon's `FileThumbs`, batched after a short quiet
//! period.
//! 3. for Drive images and videos, the daemon's `FileThumbs`, batched after a
//! short quiet period.
//!
//! Workers hand back `gdk::Texture`s, never a `Pixbuf`, which may not cross
//! threads. Each job carries the render epoch it was queued in, and workers
@@ -30,7 +30,9 @@ use gtk4::gdk_pixbuf::Pixbuf;
use gtk4::prelude::*;
use gtk4::{gdk, glib};
use pdfs_core::control::{FileThumbRequest, Request, Response, is_thumbnail_image_name};
use pdfs_core::control::{
FileThumbRequest, Request, Response, is_thumbnail_image_name, is_thumbnail_media_name,
};
use crate::{Hit, spawn_request};
@@ -165,7 +167,13 @@ impl Thumbs {
// The widget name binds the picture to one key: a late reply for
// another hit must not paint into it.
picture.set_widget_name(&key);
if hit.is_dir() || !is_thumbnail_image_name(hit.name()) {
// A video's frame comes from the daemon, which has ffmpeg's help; a
// local video has only what another application cached.
let thumbnailed = match hit {
Hit::Drive(drive) => is_thumbnail_media_name(&drive.name, Some(drive.size)),
Hit::Local(_) => is_thumbnail_image_name(hit.name()),
};
if hit.is_dir() || !thumbnailed {
return;
}
if matches!(hit, Hit::Drive(drive) if drive.uid.is_empty()) {
+1 -1
View File
@@ -94,7 +94,7 @@ pub(crate) fn icon_name(hit: &Hit) -> String {
let name = hit.name();
if is_image(name) {
"image-x-generic"
} else if is_media(name) {
} else if is_media(name) || pdfs_core::control::is_video_file_name(name, Some(hit.size())) {
"video-x-generic"
} else if is_document(name) {
"text-x-generic"
+46
View File
@@ -109,6 +109,7 @@ a live account since.
| Entry | Title |
|---|---|
| [B283](#b283--videos-have-no-thumbnail-outside-photos-and-a-ts-recording-looks-like-source-code) | Videos have no thumbnail outside Photos, and a `.ts` recording looks like source code |
| [B280](#b280--uploading-a-batch-of-large-photos-swaps-the-machine-until-the-daemon-is-killed) | Uploading a batch of large photos swaps the machine until the daemon is killed |
| [B279](#b279--after-a-watchdog-kill-the-mount-never-comes-back) | After a watchdog kill the mount never comes back |
| [B278](#b278--sync-list-and-devices-list-show-the-last-sync-as-epoch-seconds) | `sync list` and `devices list` show the last sync as epoch seconds |
@@ -383,6 +384,51 @@ a live account since.
Newest first.
### B283 — Videos have no thumbnail outside Photos, and a `.ts` recording looks like source code
**Status:** Fixed, not verified live
**Found:** 2026-11-14, from a user report: large `.ts` video recordings in My files showed a text
icon, and no video in My files had a thumbnail although the same videos had one in Photos.
**Where:** `is_video_file_name` in `crates/pdfs-core/src/control.rs`; `generate_thumbs` and
`sample_video` in `crates/pdfs-fuse/src/photos.rs`; `bind_file_thumbnail` in
`crates/pdfs-gui/src/app/widgets/thumbnails.rs`
**Repro:** Put an `.mp4` and a recorded `.ts` of a few GB in My files and open the folder in the
app's grid view. Both show a generic icon, and the `.ts` shows a text one.
**Cause.** Four things:
- File views asked the daemon for thumbnails of images only (`is_thumbnail_image_name`), so a
video was never asked for.
- `.ts` is TypeScript as often as it is an MPEG transport stream. Drive stores both as
`application/octet-stream`, and every place that names a file's type went by the extension:
the browser's icon list had no `.ts`, GIO guesses text, and the opener's `@text` class listed
`.ts`.
- The daemon downloaded a video whole into memory for one frame, so it refused anything over
256 MiB, and the size check behind that refusal asked the Photos client, which knows nothing
of a My files node.
- `file_thumbs` never asked for the thumbnail an uploader attached, on the belief that only
Photos nodes have one. This client attaches one to every upload it can make one for.
**Fix.** `is_video_file_name(name, size)` is the one rule: a video extension, and for `.ts` a
size of at least 1 MiB. The browser icon, the thumbnail gate, the launcher's icon and type, and
the opener's classes use it. For a file view, the daemon first asks Drive for the file's own
thumbnail. Without one, a video of up to 24 MiB is downloaded whole. A larger one brings its first
16 MiB, and for an MP4 or MOV whose `moov` box lies further in, the box headers on the way and
that box. The parts are laid out at their offsets in a sparse staging file, which ffmpeg reads.
A `.ts` whose head lacks the 0x47 sync byte is recorded as having no thumbnail.
**Test:** `a_ts_file_is_a_video_only_at_the_size_of_one` and
`listings_thumbnail_images_and_videos` in `crates/pdfs-core/src/control.rs`;
`a_ts_file_opens_as_media_or_text_by_its_size` in `crates/pdfs-core/src/opener.rs`;
`a_large_mp4_downloads_its_head_and_its_index` and the other `sample_video` tests in
`crates/pdfs-fuse/src/photos.rs`; the simulation
`a_video_thumbnail_downloads_a_small_part_of_the_video`, which reads under 34 MiB for two
videos of over 70 MiB and takes nothing from a video with an uploader's thumbnail.
`real_video_fixture_yields_a_frame` passed on 2026-11-14 with ffmpeg 9.0 for a 46 MB MP4 with its
index at the end, an MPEG-TS and an MKV.
---
### B280 — Uploading a batch of large photos swaps the machine until the daemon is killed
**Status:** Fixed, not verified live
+14
View File
@@ -32,6 +32,20 @@ Existing folders remain device backups; older builds cannot open the upgraded da
My files folder. The first sync preserves one-sided files and differing versions (#67).
Attachments stay in mirror mode, are excluded from device profiles, and cannot delete
the remote root when syncing is stopped.
- **Video thumbnails in My files, Shared, Trash and the launcher**, as in Photos, with a play
badge in the grid. A video's thumbnail is the one its uploader attached when it has one.
Otherwise the daemon takes a frame with ffmpeg from the start of the file, plus an MP4's
index, and never downloads more than about 16 MiB of a video for it, however large the
video is. **Build Thumbnails** covers videos too (B283).
### Fixed
- Large `.ts` recordings (MPEG transport streams) show and open as videos: the video icon, a
thumbnail, and the `@media` open rule rather than `@text`. A `.ts` of 1 MiB or more counts
as a video, a smaller one stays TypeScript source, and the daemon checks the stream's sync
byte before it takes a frame (B283).
- A video over 256 MiB never got a thumbnail, and a smaller one was downloaded whole into
memory for one frame (B283).
## [3.7.0] — 2026-11-02
+440 -430
View File
File diff suppressed because it is too large. Load diff
+453 -441
View File
File diff suppressed because it is too large. Load diff
+439 -429
View File
File diff suppressed because it is too large. Load diff
+440 -430
View File
File diff suppressed because it is too large. Load diff
+440 -430
View File
File diff suppressed because it is too large. Load diff
+440 -430
View File
File diff suppressed because it is too large. Load diff
+432 -426
View File
File diff suppressed because it is too large. Load diff
+450 -438
View File
File diff suppressed because it is too large. Load diff
+439 -429
View File
File diff suppressed because it is too large. Load diff