# User Wallet Data Synchronization Ty Everett () ## Abstract This specification defines an interoperable synchronization protocol for user wallet data between wallet storage providers. It standardizes the chunked, resumable sync model implemented by Wallet Toolbox so independent storage backends can exchange wallet state without proprietary adapters. The protocol is incremental. A consumer requests successive sync chunks for a specific wallet identity using a `since` watermark and per-entity offsets. The producer returns updated records in dependency order. The consumer merges those records, maintains a remote-to-local ID mapping, and advances the sync state until all entity arrays are present and empty. ## Motivation Wallets increasingly need to replicate user state across multiple storage backends: * local and remote storage * active and backup providers * device migration and recovery flows * managed wallet infrastructure built from interchangeable components Without a standard sync protocol, each wallet implementation must define its own export/import format and state-reconciliation logic. That prevents interoperability even when both systems already implement the same wallet behaviors. This specification addresses that problem by defining the synchronization behavior already used in Wallet Toolbox, including: * resumable incremental replication * per-entity pagination * identity-safe replication scoped to one wallet user * convergent merging through persistent ID mapping ## Specification ### Scope This specification applies to synchronization between wallet storage providers for a single wallet user. It does not define: * wallet-to-application APIs such as [BRC-100](/wallet/0100.md) * backup file formats * encryption of exported wallet data at rest or in transit This specification is transport-agnostic. Any transport may be used provided it can faithfully convey the request and response structures defined below. ### Terminology * **Producer**: The storage provider supplying wallet records. * **Consumer**: The storage provider receiving and merging wallet records. * **Sync cycle**: A sequence of one or more chunk requests sharing the same `since` value until the producer reports completion. * **Sync state**: Consumer-maintained state for a specific remote storage provider and wallet identity. * **Entity offset**: The number of records for a given entity type already received within the current sync cycle. * **ID map**: A mapping from producer-local numeric IDs to consumer-local numeric IDs. ### Record Model Wallet data is synchronized as typed entity records. This specification defines the following entity names: 1. `provenTx` 2. `outputBasket` 3. `outputTag` 4. `txLabel` 5. `transaction` 6. `output` 7. `txLabelMap` 8. `outputTagMap` 9. `certificate` 10. `certificateField` 11. `commission` 12. `provenTxReq` In addition, a producer may return a single `user` record describing the synchronized user. Each synchronized entity record: * MUST belong to the wallet user identified by the request `identityKey` * MUST include `created_at` and `updated_at` * MUST NOT contain `null` values; omitted fields MUST be omitted rather than sent as `null` When serialized through JSON, timestamps SHOULD use RFC 3339 / ISO 8601 date-time strings. ### Sync State The consumer MUST maintain a sync state for each pair: * wallet user identity * remote producer storage identity The sync state MUST include: * producer `storageIdentityKey` * producer `storageName` * current `since` watermark, if any * per-entity `count` offsets for the current sync cycle * per-entity `maxUpdated_at` observed during the current sync cycle * per-entity `idMap` The consumer MUST treat the sync state as durable. If synchronization is interrupted, the next attempt MUST resume from the stored `since` and offsets. ### Request Structure The consumer requests a chunk using the following structure: ```json { "fromStorageIdentityKey": "", "toStorageIdentityKey": "", "identityKey": "", "since": "2026-04-23T12:34:56.789Z", "maxRoughSize": 10000000, "maxItems": 1000, "offsets": [ { "name": "provenTx", "offset": 0 }, { "name": "outputBasket", "offset": 0 }, { "name": "outputTag", "offset": 0 }, { "name": "txLabel", "offset": 0 }, { "name": "transaction", "offset": 0 }, { "name": "output", "offset": 0 }, { "name": "txLabelMap", "offset": 0 }, { "name": "outputTagMap", "offset": 0 }, { "name": "certificate", "offset": 0 }, { "name": "certificateField", "offset": 0 }, { "name": "commission", "offset": 0 }, { "name": "provenTxReq", "offset": 0 } ] } ``` #### Request Rules * `fromStorageIdentityKey` MUST identify the producer. * `toStorageIdentityKey` MUST identify the consumer. * `identityKey` MUST identify the wallet user whose records are being synchronized. * `since` MUST be omitted for an initial full sync. * `maxItems` MUST bound the total number of records returned across all entity arrays. * `maxRoughSize` MUST bound the approximate serialized size of the returned chunk. * `offsets` MUST be supplied in the exact entity order defined in this specification. If an `offsets` entry is missing, duplicated, or out of order, the producer MUST reject the request. ### Producer Behavior For a given request, the producer MUST build the response as follows: 1. Initialize the response with `fromStorageIdentityKey`, `toStorageIdentityKey`, and `userIdentityKey`. 2. If `since` is absent, or if the user record `updated_at` is later than `since`, include the `user` record. 3. Process entity types in the exact order defined in this specification. 4. For each entity type, return records for the requested user whose `updated_at` is greater than or equal to `since`. 5. Skip the first `offset` matching records for that entity type. 6. Continue adding records until either: * no more records remain for that entity type, or * `maxItems` is exhausted, or * `maxRoughSize` is exceeded The producer MUST include the record that causes `maxRoughSize` to be exceeded, then stop adding further records. If the producer begins processing an entity type in the response, it MUST include that entity property in the chunk, even if the resulting array is empty. If the producer stops before reaching a later entity type because of size or count limits, the unattempted later entity properties MUST be omitted from the chunk. The producer MUST use a deterministic record order that remains stable throughout a sync cycle so the consumer can safely resume by offset. ### Response Structure The producer returns a `SyncChunk`: ```json { "fromStorageIdentityKey": "", "toStorageIdentityKey": "", "userIdentityKey": "", "user": { "...": "..." }, "provenTxs": [], "outputBaskets": [], "outputTags": [], "txLabels": [], "transactions": [], "outputs": [], "txLabelMaps": [], "outputTagMaps": [], "certificates": [], "certificateFields": [], "commissions": [], "provenTxReqs": [] } ``` Each entity array is interpreted as follows: * `undefined` / omitted: the producer did not attempt that entity type in this chunk * `[]`: the producer attempted that entity type and found no further matching records at the current `since` and offset * non-empty array: records to be merged ### Completion Condition A sync cycle is complete only when all entity-array properties are present and each of them is empty. The presence or absence of the optional `user` record does not by itself determine completion. ### Inclusive `since` Semantics The producer MUST treat `since` as an inclusive lower bound: * a record matches when `updated_at >= since` This means a resumed sync cycle will typically repeat at least one previously seen record. Consumers MUST therefore merge records idempotently and MUST NOT treat repeated records as an error. ### Consumer Merge Behavior For each returned entity record, the consumer MUST: 1. determine whether the producer record matches an existing local record under that entity's convergent equality rules 2. update the existing local record if necessary, or insert a new local record if no match exists 3. update the entity's `idMap` so the producer-local primary ID maps to the consumer-local primary ID 4. update the entity's `maxUpdated_at` with the greatest `updated_at` value observed for that entity during the current sync cycle 5. increase the entity `count` by the number of records received for that entity in the chunk The consumer MUST preserve `idMap` consistency. If an existing producer ID maps to a different local ID than previously recorded, synchronization MUST fail. For entity types whose identity is relationship-based rather than primary-ID-based, an implementation MAY omit `idMap` usage. Wallet Toolbox does this for: * `certificateField` * `txLabelMap` * `outputTagMap` ### Advancing Sync State If the current chunk does not complete the sync cycle: * the consumer MUST retain the current `since` * the consumer MUST retain the updated per-entity `count` offsets * the consumer MUST request the next chunk using those updated offsets If the current chunk completes the sync cycle: * the consumer MUST set `since` to the maximum `updated_at` observed during the cycle, if any * the consumer MUST reset every entity `count` offset to `0` * the consumer MUST begin the next cycle from the new `since` If a completed cycle produced no merged records and no new maximum timestamp, the consumer MAY leave `since` unchanged. ### Authentication and Authorization A producer MUST only return data for the authenticated wallet identity associated with `identityKey`. A producer MUST reject requests that attempt to synchronize another user's records. This specification does not mandate a specific authentication protocol, but an implementation MUST ensure: * the caller is authorized to read the requested user's wallet data * `fromStorageIdentityKey` and `toStorageIdentityKey` are not forgeable within the synchronization context ### Error Handling Synchronization MUST fail if: * the requested `identityKey` is unknown or unauthorized * the `offsets` list is malformed or out of dependency order * required timestamps are missing or invalid * any returned entity contains `null` values * the consumer detects a conflicting `idMap` assignment On failure, the consumer MUST preserve enough sync state to either retry safely or report the last durable state to the operator. ### Interoperability Notes This specification intentionally follows the chunked sync model implemented by Wallet Toolbox: * `getSyncChunk` on the producer side * `processSyncChunk` on the consumer side * durable per-remote `syncState` * persistent `syncMap` for remote-to-local ID reconciliation Implementations that follow this specification can plug into Toolbox-style sync flows interoperably without reverse-engineering private behavior. ## Implementation Wallet Toolbox implements this synchronization model across its storage providers and storage manager. An implementation is considered conformant if it: * produces request and response objects compatible with this specification * honors inclusive `since` semantics and resumable offsets * merges records convergently and durably maintains sync state ## References * [BRC-36: Format for Bitcoin Outpoints](/outpoints/0036.md) * [BRC-37: Basket and Custom Instructions Extension for Bitcoin Outpoints](/outpoints/0037.md) * [BRC-100: Unified, Vendor-Neutral, Unchanging, and Open BSV Blockchain Standard Wallet-to-Application Interface](/wallet/0100.md) --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://bsv.brc.dev/outpoints/0040.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.