feat(inventory): CSV/JSON export and JSON import with LWW reconciliation

Interchange export/import for Phase 1 (data-model §7):

- JSON: canonical, versioned envelope (formatVersion 1) with all sync
  metadata verbatim, photos embedded as base64, tombstones excluded.
  Species are re-resolved on import by scientific name (catalog ids are
  per-install). Reader tolerates unknown fields/enum values (§5.2) and
  rejects newer format versions with a clear error.
- CSV: export-only spreadsheet flatten, one row per lot, RFC 4180
  escaping, never any photo bytes.
- Import merges by UUIDv7 id in one transaction: insert-if-unknown
  preserving original stamps, last-writer-wins by packed HLC for known
  mutable rows, append-only movements; afterwards the local clock
  receiveEvent()s the newest imported stamp so it never runs behind.
- Settings gains a Backup section (export CSV/JSON, import JSON with
  confirmation); file dialogs behind a FileService interface backed by
  file_picker (MIT).
- Tests: codec round-trip and tolerance, reconciler LWW, repository
  export→import round-trip (fresh DB, idempotent re-import, newer-local
  wins, clock monotonicity, species re-resolution), backup widget flows.
This commit is contained in:
vjrj 2026-07-09 12:46:53 +02:00
parent 136ed701a7
commit 2812c99280
25 changed files with 2207 additions and 7 deletions

View file

@ -226,3 +226,20 @@ Together: Drift handles the *local database* migration; the compatibility rules
- Is `offer_status` on `Lot` enough, or does a `Variety`-level default help? (Lean: Lot only.)
- Do we need `Variety`-level provenance, or is per-Lot/Movement provenance sufficient? (Lean: Movement DAG is enough.)
- `category` as free enum vs linked to `Species.family` when a species is set. (Lean: free, prefilled from family when available.)
## 7. Interchange export/import (Phase 1) — decided & implemented
The user-triggered **interchange export** (Settings → Backup). Distinct from the future encrypted full backup (`.tanemaki`, [backup-and-recovery.md](backup-and-recovery.md) mechanism 1): the interchange files are the *only* plaintext the app ever writes, and only because the user explicitly asks for them. Import reads the picked file straight into memory — no temp copies.
Two formats, one canonical:
- **JSON** — canonical, versioned, importable; a precursor of the future sync payload. Envelope: `formatVersion` (currently 1), `schemaVersion` (the writer's DB generation), `appId: "tane"`, then one array per table: `varieties`, `lots`, `vernacularNames`, `externalLinks`, `germinationTests`, `movements`, `parties`, `attachments`. Every row carries its sync metadata verbatim (`id`, `createdAt`, packed-HLC `updatedAt`, `lastAuthor`, `schemaRowVersion`); enums serialize by name. **Photos/docs are embedded as base64** (`bytesBase64`) — the JSON export is the complete copy of the data (~33% size overhead accepted). **Tombstones are excluded**: this is a user-facing export of the live inventory, not the sync wire format (real sync, Block 2, will carry tombstones on its own channel).
- **CSV** — export-only, for spreadsheets. One row per Lot (a variety with no lots gets one row); vernacular names and links joined into single columns; latest germination rate derived; ISO-8601 dates; never any photo bytes. Not importable — the JSON is.
**Species are not exported.** The bundled catalog rows get per-install ids, so `species_id` is not portable. Each exported variety carries a denormalized `speciesScientificName`; the importer re-resolves it against its own catalog by scientific name (no match → `species_id` stays null).
**Import = merge by id (UUIDv7), never duplicating.** Mutable rows: unknown id → insert preserving the original metadata (no re-stamping); known id → last-writer-wins on the packed HLC `updatedAt` (§4), incoming wins only if strictly newer — so re-importing the same file is a no-op, and a locally-deleted row only resurrects if the incoming version is newer than the tombstone. `movements` are append-only: insert-if-unknown, never updated. Everything runs in one transaction; afterwards the local HLC clock `receiveEvent`s the newest imported stamp so the next local write out-stamps everything imported.
**Reader compatibility (per §5.2):** unknown JSON fields are ignored; unknown enum values map to a safe default (`lot.type → seed`, `offer_status → private`) or drop the row when none exists (an unknown `movement.type`); a `formatVersion` newer than the reader supports fails with a clear "update the app" error. `formatVersion` changes must be additive and documented here.
Implementation: `apps/app_seeds/lib/data/export_import/` (codecs + reconciler), `VarietyRepository.exportInventory()/importInventory()`, `lib/services/export_import_service.dart`.