fix(fuse,core,gui,cli): resumable, robust Google Photos import (#94)

A Takeout import cut short by a restart could never be resumed: the
duplicate check failed on the draft the interrupted upload left behind
("invalid type: null, expected a string"). SDK 0.12.0 skips such drafts,
lets the import replace them across logins, and removes the draft of a
failed upload (B285, B286).

The import keeps its archives and outcome in the state directory, so a
run the service did not finish is reported as interrupted and can be
resumed from the app or with `pdfs import-google-photos --resume`. An
unreadable archive is left out instead of failing the scan, a failure
keeps its counts and still files albums, and a cancel or shutdown is
only reported when work was left undone (B287).

ImportStatus carries the phase, its progress, the running counts and
what can be resumed; the summary says dry_run, to_upload and
error_kind. The app shows these in the user's language, and --wait
prints progress. An empty gallery waits for a timeline refresh already
running instead of starting a second one.

The scan imports only the Photos service of an export, reads geoData
into the photo's Location, pairs live and motion photos (.MOV, .MP) with
their stills, and dates a photo by name and EXIF before Google's upload
time (B288).
This commit is contained in:
nvrl committed 2026-11-15 17:17:47 +01:00
1 parent 2ec3b7ff62
commit 2f2fe96cdf
28 files changed
+5196 -2544

No files matched your search

+81 -2
View File
@@ -279,6 +279,11 @@ pub enum Request {
/// and files what it has uploaded into its albums, then reports
/// `cancelled`. Replies with [`Response::Ok`].
CancelImport,
/// Start the last import again with the same archives, after it was
/// interrupted, cancelled or failed. Photos it already uploaded are found
/// on the account and skipped. Replies like [`Request::ImportTakeout`];
/// [`ErrorKind::NotFound`] when there is nothing to resume.
ResumeImport,
/// Upload the photo at `source_path` under the given name and media type.
///
/// A path, not the bytes: this protocol is line-delimited JSON, and
@@ -1689,10 +1694,75 @@ pub struct ImportSummary {
pub bytes: u64,
/// True when the run stopped early because it was cancelled.
pub cancelled: bool,
/// Why the run as a whole failed, when it did, such as an archive that
/// is not a zip. The counts are then empty.
/// Why the run as a whole stopped early, when it did, such as the
/// account being unreachable. The counts say how far it got.
#[serde(default)]
pub error: Option<String>,
/// What kind of failure [`error`](Self::error) is, so a front-end can word
/// it in the user's language. The text itself is the daemon's English.
#[serde(default)]
pub error_kind: Option<ErrorKind>,
/// True for a scan that uploads nothing ([`Request::ImportTakeout`] with
/// `dry_run`). [`to_upload`](Self::to_upload) then says what an import
/// would upload.
#[serde(default)]
pub dry_run: bool,
/// Photos not yet on the account: what a dry run would upload, and what a
/// real run set out to upload once the duplicates were known.
#[serde(default)]
pub to_upload: usize,
/// Bytes of [`to_upload`](Self::to_upload).
#[serde(default)]
pub to_upload_bytes: u64,
/// Live and motion photos whose video was filed with its still.
#[serde(default)]
pub live_photos: usize,
/// True when the service stopped while the import ran (a shutdown or a
/// restart). [`Request::ResumeImport`] picks it up again.
#[serde(default)]
pub interrupted: bool,
/// Archives that could not be read and were left out, by path.
#[serde(default)]
pub unreadable_archives: Vec<String>,
}
impl ImportSummary {
/// Whether the run did all it set out to do.
pub fn complete(&self) -> bool {
self.error.is_none()
&& !self.cancelled
&& !self.interrupted
&& self.failed == 0
&& self.unreadable_archives.is_empty()
}
}
/// Which step a running Takeout import is on.
#[derive(Serialize, Deserialize, Debug, Clone, Copy, Default, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum ImportPhase {
/// Reading the archives' directories and metadata.
#[default]
Scanning,
/// Asking the account which photos it already has.
Checking,
/// Uploading the photos that are new.
Uploading,
/// Creating albums and filing photos into them.
Albums,
}
/// Where a running import is ([`Response::ImportStatus`]).
#[derive(Serialize, Deserialize, Debug, Clone, Default, PartialEq, Eq)]
pub struct ImportProgress {
pub phase: ImportPhase,
/// Steps done in this phase.
pub done: u64,
/// Steps in this phase; `0` while not yet known.
pub total: u64,
/// The photo being uploaded, during [`ImportPhase::Uploading`].
#[serde(default)]
pub current: Option<String>,
}
/// One album in a [`Response::Albums`] listing.
@@ -2208,8 +2278,17 @@ pub enum Response {
/// report; `summary` is `None` when no import has run this session.
ImportStatus {
running: bool,
/// The counts so far while running, the final report after.
#[serde(default)]
summary: Option<ImportSummary>,
/// The current step, while running.
#[serde(default)]
progress: Option<ImportProgress>,
/// The archives of the last run, when it can be resumed with
/// [`Request::ResumeImport`]: it was a real import that did not
/// finish. Survives a restart of the service.
#[serde(default)]
resumable: Vec<String>,
},
/// How the photo re-date is doing (reply to [`Request::RedateStatus`]).
/// `running` is false once it has finished, when `summary` is the final