vjrj
1e2c66a982
Product-name mentions → Tane; backup-file extension .tanemaki → .tane; website tanemaki.app → tane.comunes.org. Etymology-bearing docs (README, VISION, PLAN, CLAUDE, intros) handled separately.
88 lines
4.4 KiB
Markdown
88 lines
4.4 KiB
Markdown
# Chat storage — spec
|
|
|
|
*Where 1:1 chat history lives on each device, and why it is a separate encrypted database from the inventory.*
|
|
|
|
## Problem
|
|
|
|
Chat history was persisted as a **JSON blob per conversation in the OS keystore**
|
|
(`flutter_secure_storage`), via `MessageStore`. That is the wrong tool once a
|
|
conversation is used:
|
|
|
|
- **Read-modify-write on every message.** Each incoming/sent message read the
|
|
whole conversation JSON, decoded it, checked duplicates in O(n), re-encoded
|
|
the whole list, and wrote it back — O(n) per message, O(n²) over a
|
|
conversation.
|
|
- **Silent data loss.** A hard cap of 200 messages per peer dropped the oldest
|
|
without warning. It was a recent-cache, not storage.
|
|
- **Keystore misuse.** The OS keystore (Android `EncryptedSharedPreferences`,
|
|
iOS Keychain) is for small secrets (keys, tokens), not growing blobs.
|
|
- **Manual index.** Peers were tracked in a `\n`-joined string because the
|
|
keystore can't enumerate keys; the inbox (`conversations()`) then read every
|
|
peer's full blob just to get its last message.
|
|
|
|
## Decision
|
|
|
|
Move chat history to an **encrypted Drift/SQLCipher database**, in its own file
|
|
(`tane_chat.sqlite`) — `ChatDatabase`, separate from the inventory
|
|
`AppDatabase` (`tane_inventory.sqlite`). Same SQLCipher key (from the OS
|
|
keystore), so "no plaintext at rest" holds.
|
|
|
|
### Why a *separate* database, not a new table in `AppDatabase`
|
|
|
|
Chat is **not inventory**. It is an ephemeral, per-device cache of what the
|
|
relays delivered; it never enters an inventory snapshot, a `.tane` backup,
|
|
or the device-to-device sync. Keeping it out of `AppDatabase` means:
|
|
|
|
- It carries **no CRDT metadata** (no HLC `updatedAt`, tombstones, `lastAuthor`,
|
|
`schemaRowVersion`) — those are meaningless for a network cache.
|
|
- It stays out of the inventory's **versioned migration series** (no `v9` bump,
|
|
no `drift_schema_v9.json`, no new `migration_test` case). `ChatDatabase` is
|
|
`schemaVersion = 1`, created fresh; there is nothing to migrate.
|
|
- It never trips the inventory **sync stream** (`AppDatabase.tableUpdates()`
|
|
feeds `SyncService`); a message arriving can't spuriously republish the
|
|
inventory.
|
|
|
|
### Schema — `Messages`
|
|
|
|
One flat, indexed table (see `apps/app_seeds/lib/db/chat_database.dart`):
|
|
|
|
| column | notes |
|
|
|-----------------|----------------------------------------------------|
|
|
| `id` | autoincrement — local rowid, also the stable tie-break for equal timestamps |
|
|
| `accountScope` | namespaces rows per social identity (was the keystore key prefix) |
|
|
| `peerPubkey` | the conversation |
|
|
| `fromPubkey` | sender (self vs peer → bubble side) |
|
|
| `body` | message text |
|
|
| `sentAt` | ms since epoch (NIP-17 timestamp) — ordering + read mark |
|
|
|
|
- **Index** `(accountScope, peerPubkey, sentAt)` serves history and the inbox.
|
|
- **Unique key** `(accountScope, peerPubkey, fromPubkey, sentAt, body)` makes
|
|
de-duplication a DB invariant: a relay re-delivers the same gift wrap on every
|
|
(re)subscribe, and an identical row simply isn't inserted twice.
|
|
|
|
### Behaviour vs. the old store
|
|
|
|
`MessageStore` keeps its public API (`history` / `append` / `conversations`), so
|
|
`MessagesCubit`, `InboxService`, `UnreadService` and the inbox screen are
|
|
unchanged. What changes:
|
|
|
|
- **`append`** is a single indexed insert (dedup-checked in a transaction),
|
|
not a whole-conversation rewrite. Returns `true` only when newly stored.
|
|
- **`history`** is uncapped, ordered by `sentAt` then `id` (stable insertion
|
|
tie-break for equal timestamps).
|
|
- **`conversations`** is one indexed scan; the newest message per peer is picked
|
|
in Dart (peer counts are small).
|
|
|
|
## Not migrating old data
|
|
|
|
Pre-release: there are no users, so the old keystore blobs are **not** migrated.
|
|
The keystore keys (`tane.social.chat.*`, `tane.social.chats`) are simply
|
|
abandoned. If that changes before release, a one-shot import can be added.
|
|
|
|
## Related, deliberately not changed
|
|
|
|
The same keystore-blob anti-pattern exists, at trivial scale, in `OfferOutbox`
|
|
(pending lot ids), `TrustReferents` (WoT referent set) and `SocialSettings`
|
|
(relay URLs). These are small, bounded, rarely-written lists — the keystore is
|
|
acceptable there. Only chat grows with everyday use, so only chat moved. If any
|
|
of those grows unexpectedly, the same Drift treatment applies.
|