User Wallet Data Synchronization

Ty Everett ([email protected])

Abstract

This specification defines an interoperable synchronization protocol for user wallet data between wallet storage providers. It standardizes the chunked, resumable sync model implemented by Wallet Toolbox so independent storage backends can exchange wallet state without proprietary adapters.

The protocol is incremental. A consumer requests successive sync chunks for a specific wallet identity using a since watermark and per-entity offsets. The producer returns updated records in dependency order. The consumer merges those records, maintains a remote-to-local ID mapping, and advances the sync state until all entity arrays are present and empty.

Motivation

Wallets increasingly need to replicate user state across multiple storage backends:

• local and remote storage

• active and backup providers

• device migration and recovery flows

• managed wallet infrastructure built from interchangeable components

Without a standard sync protocol, each wallet implementation must define its own export/import format and state-reconciliation logic. That prevents interoperability even when both systems already implement the same wallet behaviors.

This specification addresses that problem by defining the synchronization behavior already used in Wallet Toolbox, including:

• resumable incremental replication

• per-entity pagination

• identity-safe replication scoped to one wallet user

• convergent merging through persistent ID mapping

Specification

Scope

This specification applies to synchronization between wallet storage providers for a single wallet user.

It does not define:

• wallet-to-application APIs such as BRC-100

• backup file formats

• encryption of exported wallet data at rest or in transit

This specification is transport-agnostic. Any transport may be used provided it can faithfully convey the request and response structures defined below.

Terminology

• Producer: The storage provider supplying wallet records.

• Consumer: The storage provider receiving and merging wallet records.

• Sync cycle: A sequence of one or more chunk requests sharing the same since value until the producer reports completion.

• Sync state: Consumer-maintained state for a specific remote storage provider and wallet identity.

• Entity offset: The number of records for a given entity type already received within the current sync cycle.

• ID map: A mapping from producer-local numeric IDs to consumer-local numeric IDs.

Record Model

Wallet data is synchronized as typed entity records. This specification defines the following entity names:

1. provenTx

2. outputBasket

3. outputTag

4. txLabel

5. transaction

6. output

7. txLabelMap

8. outputTagMap

9. certificate

10. certificateField

11. commission

12. provenTxReq

In addition, a producer may return a single user record describing the synchronized user.

Each synchronized entity record:

• MUST belong to the wallet user identified by the request identityKey

• MUST include created_at and updated_at

• MUST NOT contain null values; omitted fields MUST be omitted rather than sent as null

When serialized through JSON, timestamps SHOULD use RFC 3339 / ISO 8601 date-time strings.

Sync State

The consumer MUST maintain a sync state for each pair:

• wallet user identity

• remote producer storage identity

The sync state MUST include:

• producer storageIdentityKey

• producer storageName

• current since watermark, if any

• per-entity count offsets for the current sync cycle

• per-entity maxUpdated_at observed during the current sync cycle

• per-entity idMap

The consumer MUST treat the sync state as durable. If synchronization is interrupted, the next attempt MUST resume from the stored since and offsets.

Request Structure

The consumer requests a chunk using the following structure:

{
  "fromStorageIdentityKey": "<producerStorageIdentityKey>",
  "toStorageIdentityKey": "<consumerStorageIdentityKey>",
  "identityKey": "<userIdentityKey>",
  "since": "2026-04-23T12:34:56.789Z",
  "maxRoughSize": 10000000,
  "maxItems": 1000,
  "offsets": [
    { "name": "provenTx", "offset": 0 },
    { "name": "outputBasket", "offset": 0 },
    { "name": "outputTag", "offset": 0 },
    { "name": "txLabel", "offset": 0 },
    { "name": "transaction", "offset": 0 },
    { "name": "output", "offset": 0 },
    { "name": "txLabelMap", "offset": 0 },
    { "name": "outputTagMap", "offset": 0 },
    { "name": "certificate", "offset": 0 },
    { "name": "certificateField", "offset": 0 },
    { "name": "commission", "offset": 0 },
    { "name": "provenTxReq", "offset": 0 }
  ]
}

Request Rules

• fromStorageIdentityKey MUST identify the producer.

• toStorageIdentityKey MUST identify the consumer.

• identityKey MUST identify the wallet user whose records are being synchronized.

• since MUST be omitted for an initial full sync.

• maxItems MUST bound the total number of records returned across all entity arrays.

• maxRoughSize MUST bound the approximate serialized size of the returned chunk.

• offsets MUST be supplied in the exact entity order defined in this specification.

If an offsets entry is missing, duplicated, or out of order, the producer MUST reject the request.

Producer Behavior

For a given request, the producer MUST build the response as follows:

1. Initialize the response with fromStorageIdentityKey, toStorageIdentityKey, and userIdentityKey.

2. If since is absent, or if the user record updated_at is later than since, include the user record.

3. Process entity types in the exact order defined in this specification.

4. For each entity type, return records for the requested user whose updated_at is greater than or equal to since.

5. Skip the first offset matching records for that entity type.

6. Continue adding records until either:

   • no more records remain for that entity type, or

   • maxItems is exhausted, or

   • maxRoughSize is exceeded

The producer MUST include the record that causes maxRoughSize to be exceeded, then stop adding further records.

If the producer begins processing an entity type in the response, it MUST include that entity property in the chunk, even if the resulting array is empty.

If the producer stops before reaching a later entity type because of size or count limits, the unattempted later entity properties MUST be omitted from the chunk.

The producer MUST use a deterministic record order that remains stable throughout a sync cycle so the consumer can safely resume by offset.

Response Structure

The producer returns a SyncChunk:

{
  "fromStorageIdentityKey": "<producerStorageIdentityKey>",
  "toStorageIdentityKey": "<consumerStorageIdentityKey>",
  "userIdentityKey": "<userIdentityKey>",
  "user": { "...": "..." },
  "provenTxs": [],
  "outputBaskets": [],
  "outputTags": [],
  "txLabels": [],
  "transactions": [],
  "outputs": [],
  "txLabelMaps": [],
  "outputTagMaps": [],
  "certificates": [],
  "certificateFields": [],
  "commissions": [],
  "provenTxReqs": []
}

Each entity array is interpreted as follows:

• undefined / omitted: the producer did not attempt that entity type in this chunk

• []: the producer attempted that entity type and found no further matching records at the current since and offset

• non-empty array: records to be merged

Completion Condition

A sync cycle is complete only when all entity-array properties are present and each of them is empty.

The presence or absence of the optional user record does not by itself determine completion.

Inclusive since Semantics

The producer MUST treat since as an inclusive lower bound:

• a record matches when updated_at >= since

This means a resumed sync cycle will typically repeat at least one previously seen record. Consumers MUST therefore merge records idempotently and MUST NOT treat repeated records as an error.

Consumer Merge Behavior

For each returned entity record, the consumer MUST:

1. determine whether the producer record matches an existing local record under that entity's convergent equality rules

2. update the existing local record if necessary, or insert a new local record if no match exists

3. update the entity's idMap so the producer-local primary ID maps to the consumer-local primary ID

4. update the entity's maxUpdated_at with the greatest updated_at value observed for that entity during the current sync cycle

5. increase the entity count by the number of records received for that entity in the chunk

The consumer MUST preserve idMap consistency. If an existing producer ID maps to a different local ID than previously recorded, synchronization MUST fail.

For entity types whose identity is relationship-based rather than primary-ID-based, an implementation MAY omit idMap usage. Wallet Toolbox does this for:

• certificateField

• txLabelMap

• outputTagMap

Advancing Sync State

If the current chunk does not complete the sync cycle:

• the consumer MUST retain the current since

• the consumer MUST retain the updated per-entity count offsets

• the consumer MUST request the next chunk using those updated offsets

If the current chunk completes the sync cycle:

• the consumer MUST set since to the maximum updated_at observed during the cycle, if any

• the consumer MUST reset every entity count offset to 0

• the consumer MUST begin the next cycle from the new since

If a completed cycle produced no merged records and no new maximum timestamp, the consumer MAY leave since unchanged.

Authentication and Authorization

A producer MUST only return data for the authenticated wallet identity associated with identityKey.

A producer MUST reject requests that attempt to synchronize another user's records.

This specification does not mandate a specific authentication protocol, but an implementation MUST ensure:

• the caller is authorized to read the requested user's wallet data

• fromStorageIdentityKey and toStorageIdentityKey are not forgeable within the synchronization context

Error Handling

Synchronization MUST fail if:

• the requested identityKey is unknown or unauthorized

• the offsets list is malformed or out of dependency order

• required timestamps are missing or invalid

• any returned entity contains null values

• the consumer detects a conflicting idMap assignment

On failure, the consumer MUST preserve enough sync state to either retry safely or report the last durable state to the operator.

Interoperability Notes

This specification intentionally follows the chunked sync model implemented by Wallet Toolbox:

• getSyncChunk on the producer side

• processSyncChunk on the consumer side

• durable per-remote syncState

• persistent syncMap for remote-to-local ID reconciliation

Implementations that follow this specification can plug into Toolbox-style sync flows interoperably without reverse-engineering private behavior.

Implementation

Wallet Toolbox implements this synchronization model across its storage providers and storage manager.

An implementation is considered conformant if it:

• produces request and response objects compatible with this specification

• honors inclusive since semantics and resumable offsets

• merges records convergently and durably maintains sync state

References

• BRC-36: Format for Bitcoin Outpoints

• BRC-37: Basket and Custom Instructions Extension for Bitcoin Outpoints

• BRC-100: Unified, Vendor-Neutral, Unchanging, and Open BSV Blockchain Standard Wallet-to-Application Interface