mirror of
https://github.com/narrrl/proton-drive-linux.git
synced 2026-11-15 17:41:59 +01:00
fix(fuse,core,gui,cli): resumable, robust Google Photos import (#94)
A Takeout import cut short by a restart could never be resumed: the
duplicate check failed on the draft the interrupted upload left behind
("invalid type: null, expected a string"). SDK 0.12.0 skips such drafts,
lets the import replace them across logins, and removes the draft of a
failed upload (B285, B286).
The import keeps its archives and outcome in the state directory, so a
run the service did not finish is reported as interrupted and can be
resumed from the app or with `pdfs import-google-photos --resume`. An
unreadable archive is left out instead of failing the scan, a failure
keeps its counts and still files albums, and a cancel or shutdown is
only reported when work was left undone (B287).
ImportStatus carries the phase, its progress, the running counts and
what can be resumed; the summary says dry_run, to_upload and
error_kind. The app shows these in the user's language, and --wait
prints progress. An empty gallery waits for a timeline refresh already
running instead of starting a second one.
The scan imports only the Photos service of an export, reads geoData
into the photo's Location, pairs live and motion photos (.MOV, .MP) with
their stills, and dates a photo by name and EXIF before Google's upload
time (B288).
This commit is contained in:
1 parent
2ec3b7ff62
commit
2f2fe96cdf
28 files changed
+5196
-2544
No files matched your search
+125
@@ -109,6 +109,10 @@ a live account since.
|
||||
|
||||
| Entry | Title |
|
||||
|---|---|
|
||||
| [B289](#b289--the-takeout-import-mixes-in-other-services-and-drops-location-and-live-photos) | The Takeout import mixes in other services and drops location and live photos |
|
||||
| [B288](#b288--an-interrupted-takeout-import-reads-as-finished-and-one-error-ends-it) | An interrupted Takeout import reads as finished, and one error ends it |
|
||||
| [B287](#b287--a-photo-draft-left-by-an-interrupted-upload-blocks-that-name) | A photo draft left by an interrupted upload blocks that name |
|
||||
| [B286](#b286--resuming-a-takeout-import-fails-on-a-draft-with-no-content-hash) | Resuming a Takeout import fails on a draft with no content hash |
|
||||
| [B285](#b285--a-restarted-secret-service-leaves-the-process-on-a-dead-session) | A restarted Secret Service leaves the process on a dead session |
|
||||
| [B284](#b284--a-keyring-entry-reported-missing-discards-refreshed-session-tokens) | A keyring entry reported missing discards refreshed session tokens |
|
||||
| [B283](#b283--videos-have-no-thumbnail-outside-photos-and-a-ts-recording-looks-like-source-code) | Videos have no thumbnail outside Photos, and a `.ts` recording looks like source code |
|
||||
@@ -388,6 +392,127 @@ a live account since.
|
||||
|
||||
Newest first.
|
||||
|
||||
### B289 — The Takeout import mixes in other services and drops location and live photos
|
||||
|
||||
**Status:** Fixed, not verified live
|
||||
**Found:** 2026-11-15, auditing the Google Photos import alongside
|
||||
[issue #94](https://github.com/narrrl/proton-drive-linux/issues/94).
|
||||
**Where:** `crates/pdfs-core/src/takeout.rs`; `plan` and `upload_takeout_photo` in
|
||||
`crates/pdfs-fuse/src/takeout.rs`
|
||||
|
||||
**Repro:** Import a Takeout export that also holds `Drive/` or `YouTube/`, with Pixel motion
|
||||
photos, iPhone live photos and photos that have a place in Google Photos.
|
||||
|
||||
**Cause.** Five gaps in what the import carried over:
|
||||
- The service folder was dropped without being checked, so media from other Google services in
|
||||
the same export landed in the photo timeline.
|
||||
- `geoData` was never read, although `docs/PHOTOS.md` said the import takes locations from the
|
||||
export, and the SDK had no way to upload a location.
|
||||
- The two halves of a live or motion photo uploaded as unrelated photos, the `.MOV` half usually
|
||||
without a date, and the `.MP` half of a Pixel motion photo was skipped as unknown.
|
||||
- `creationTime`, the day the photo reached Google, outranked the date in the file name and the
|
||||
photo's own EXIF.
|
||||
- A sidecar with `null` in `favorited`, a numeric `timestamp` or a `0` timestamp lost its other
|
||||
fields or filed the photo under 1970.
|
||||
|
||||
**Fix:** The Photos service is the folder whose sidecars carry `photoTakenTime`, and only it is
|
||||
imported (everything, when no folder does). `geoData`, else `geoDataExif`, becomes the photo's
|
||||
`Location` attribute through SDK 0.12.0's `PhotoUploadMetadata::location`; `0, 0` is no position.
|
||||
Stills and videos with the same stem in one folder (`IMG_1.HEIC` + `IMG_1.MOV`,
|
||||
`PXL_1.MP.jpg` + `PXL_1.MP`) are paired unless their dates are minutes apart: the still is
|
||||
tagged as a live or motion photo, the video is uploaded after it with `main_photo_uid` and takes
|
||||
its date and place. The capture time is `photoTakenTime`, then the name, then EXIF, then
|
||||
`creationTime`. Favorites are also set on photos that were already on the account.
|
||||
|
||||
**Verified:** `takeout::tests` in `pdfs-core` (services, locations, timestamps, live keys) and
|
||||
`pdfs-fuse` (pairing, skew, folded copies). Not run against a real export and account.
|
||||
|
||||
---
|
||||
|
||||
### B288 — An interrupted Takeout import reads as finished, and one error ends it
|
||||
|
||||
**Status:** Fixed, not verified live
|
||||
**Found:** 2026-11-15, auditing the Google Photos import alongside
|
||||
[issue #94](https://github.com/narrrl/proton-drive-linux/issues/94).
|
||||
**Where:** `crates/pdfs-fuse/src/takeout.rs`; `refresh_takeout` in
|
||||
`crates/pdfs-gui/src/app/pages/takeout.rs`; `cmd_import_google_photos` in
|
||||
`crates/pdfs-cli/src/main.rs`
|
||||
|
||||
**Repro:** Start an import of several archives, one of them damaged, and restart the service
|
||||
while it uploads.
|
||||
|
||||
**Cause.**
|
||||
- The import state lived only in memory. After a restart `ImportStatus` had no summary, and the
|
||||
app announced "Import finished". The archives had to be chosen again to continue.
|
||||
- A daemon shutdown broke the upload loop but still filed albums and logged a success.
|
||||
- One unreadable archive failed the scan, and a failed request in the middle of the run
|
||||
discarded every count and skipped the albums for what had uploaded.
|
||||
- The progress total counted photos already on the account, so the bar stopped short of the
|
||||
end; a dry run reported "Would upload: 0"; the job's phases reached the app in English.
|
||||
|
||||
**Fix:** The last run's archives and summary are kept in `takeout-import.json` in the state
|
||||
directory. A run still marked running at start-up is `interrupted`, and `ImportStatus` lists
|
||||
its archives as `resumable`; `ResumeImport` (`pdfs import-google-photos --resume`, **Resume** in
|
||||
the app) starts it again, and the duplicate check skips what is there. An unreadable archive is
|
||||
listed in `unreadable_archives` and the rest imports. A failure keeps the counts and still files
|
||||
albums. `ImportStatus` carries the phase and its own progress, the summary says `dry_run`,
|
||||
`to_upload` and `error_kind`, and the app words them in the user's language. The inline
|
||||
timeline refresh of an empty gallery waits for a refresh already running instead of starting a
|
||||
second one (`refresh_timeline_joined`).
|
||||
|
||||
**Verified:** `takeout::tests` in `pdfs-fuse` (saved state, resumable runs) and `pdfs-core`
|
||||
(unreadable archive), and the CLI tests for `--resume` and the exit status. Not run live.
|
||||
|
||||
---
|
||||
|
||||
### B287 — A photo draft left by an interrupted upload blocks that name
|
||||
|
||||
**Status:** Fixed, not verified live
|
||||
**Found:** 2026-11-15, reading the SDK upload path for
|
||||
[issue #94](https://github.com/narrrl/proton-drive-linux/issues/94).
|
||||
**Where:** `upload_photo_from`, `upload_file_from_inner` and `abandon_file_draft` in
|
||||
`proton-drive-rs` (SDK 0.12.0)
|
||||
|
||||
**Repro:** Interrupt a photo upload after its draft is created, sign in again, and upload a
|
||||
photo with the same name.
|
||||
|
||||
**Cause.** Photo uploads replaced a leftover draft only when its `ClientUID` was the current
|
||||
session's. A new login is a new session, so the next upload of that name failed with 2500. A
|
||||
failed upload also never deleted its own draft, which the C# and TypeScript SDKs do.
|
||||
|
||||
**Fix:** SDK 0.12.0 adds `PhotoUploadMetadata::replace_existing_draft`, which the Takeout import
|
||||
sets, and deletes the draft of a new file or photo whose upload fails. After a failed seal it
|
||||
only does so on a definite rejection, because a seal lost in transit may have been applied.
|
||||
|
||||
**Verified:** SDK unit test `draft_cleanup_only_after_a_definite_seal_rejection`. Not run live.
|
||||
|
||||
---
|
||||
|
||||
### B286 — Resuming a Takeout import fails on a draft with no content hash
|
||||
|
||||
**Status:** Fixed, not verified live
|
||||
**Found:** 2026-11-15, [issue #94](https://github.com/narrrl/proton-drive-linux/issues/94)
|
||||
(3.7.0, Fedora). An import stopped by a restart failed every re-run at once with
|
||||
`check for existing photos: serialization error: invalid type: null, expected a string at line
|
||||
1 column 2063`.
|
||||
**Where:** `FoundPhotoDuplicateDto` in `proton-drive-rs` (SDK 0.11.0)
|
||||
|
||||
**Repro:** Interrupt a photo upload after its draft is created, then run the import again.
|
||||
|
||||
**Cause.** The duplicates endpoint sends `ContentHash` (and can send `Hash`) as `null` for a
|
||||
draft, as its API schema says. The SDK declared both as `String` with `#[serde(default)]`,
|
||||
which covers a missing field but not `null`, so one draft failed the whole batch of 150 names
|
||||
and the import ended before uploading anything.
|
||||
|
||||
**Fix:** SDK 0.12.0 makes both fields optional and skips drafts, trashed links and candidates
|
||||
without hashes, like the TypeScript SDK. Every other defaulted DTO field reads `null` as its
|
||||
default too, and an undecodable response is logged with its path.
|
||||
|
||||
**Verified:** SDK unit tests `duplicate_response_tolerates_null_fields_and_skips_drafts` and
|
||||
`duplicate_response_tolerates_a_null_list`. Not run against the reporter's account.
|
||||
|
||||
---
|
||||
|
||||
### B285 — A restarted Secret Service leaves the process on a dead session
|
||||
|
||||
**Status:** Fixed (unverified)
|
||||
|
||||
Reference in new issue
Block a user