# Overlay Network Data Synchronization Ty Everett () ## Abstract This document proposes a solution for synchronizing data across UTXO-based overlay networks by specifying a mechanism for submitting transactions and receiving topic-specific admittance instructions. In current interoperable implementations, overlay services accept BEEF-formatted transaction submissions together with a set of topic names and return a per-topic result describing newly admitted outputs and retained historical inputs. This standard defines the parameters and processing steps needed to track topical UTXOs. ## Motivation As discussed in [BRC-59](/opinions/0059.md), UTXO-based overlay networks provide a secure and scalable solution to managing states derived from the Bitcoin network. In order to maintain a consistent and up-to-date state across nodes, a method of data synchronization is required. Additionally, it is essential to provide a standard way for network participants to submit new transactions and notify network operators of their relevance to hosted topics. This document aims to address these needs by outlining a clear and well-defined process for submitting transactions and processing them based on topic-specific rules. ## Specification We start with a server that is set up to track the state of one or more topics. Every individual server implementing this specification decides what specific topic labels mean, and which rules they apply when admitting new UTXOs. ### Status Note The earlier JSON-and-[BRC-8](/transactions/0008.md)-envelope framing described by this document is now historical. Current interoperable implementations in `bsv-blockchain/ts-sdk` and services built around it use: * a POST `/submit` endpoint, * an `application/octet-stream` request body containing a serialized BEEF transaction, * an `X-Topics` header carrying a JSON array of topic names, * and a structured JSON acknowledgment describing topical admittance instructions. Implementers SHOULD treat the older JSON envelope framing as deprecated unless they are explicitly maintaining backwards compatibility with a legacy deployment. ### API Endpoint and Parameters The overlay service hosts a POST `/submit` API endpoint. Current interoperable request framing is: * **Body:** BEEF for the submitted transaction. * **Content-Type:** `application/octet-stream` * **Header:** `X-Topics`, containing a JSON array of topic names. * **Optional Header:** `x-includes-off-chain-values: true` when the request body appends off-chain values after the BEEF payload. When `x-includes-off-chain-values: true` is present, current SDK facilitators frame the body as: ``` VarInt(beefByteLength) || beefBytes || offChainValueBytes ``` Without that header, the body is just the raw BEEF bytes. Topic names used with current SDK tooling are expected to begin with `tm_`. ### Overlay Network Node Processing Steps Upon receiving a transaction, the overlay network node performs the following steps: 1. Parse and validate the submitted BEEF transaction. 2. Check whether the node hosts at least one of the requested topics. If none are hosted, the node returns an early response indicating that no outputs were admitted. 3. Verify the transaction according to the node's policy and validation stack. This commonly includes SPV and/or policy checks, but the exact validation pipeline is implementation-defined. 4. Apply topic-specific logic to the transaction by iterating through all specified topics and performing the following for each topic: * Check the transaction's inputs for UTXOs that are already part of the current topical overlay. * Apply the topic-specific management logic and protocol rules to determine: * which outputs are newly admitted into the topic, * which consumed topical coins should be retained for historical traversal, * and which topical coins are removed from the live set. * Update the node's tracking system, associating admitted outputs with the topic and marking removed coins as spent. If implementing [BRC-24](/overlays/0024.md), notify lookup services about the new incoming and outgoing entries. 5. Return a JSON response object containing, for each topic, an **Admittance Instructions** object: * `outputsToAdmit`: output indices newly admitted into the topic. * `coinsToRetain`: input indices whose spent topical predecessors should remain queryable for history. * `coinsRemoved` (optional): input indices whose spent topical predecessors are removed from the live topical set. 6. Pursuant to any peering or synchronization arrangements the operator maintains, notify peer nodes about the transaction. Authentication, payment, or additional transport requirements MAY be layered on top of this baseline by deployment-specific facilitators, but they are not part of the current minimum interoperable SHIP submission behavior. ### Example We provide an example of an HTTP request and response. #### Request ``` POST /submit HTTP/1.1 Host: example-overlay-node.com Content-Type: application/octet-stream X-Topics: ["tm_example_1","tm_example_2"] ``` ``` ``` #### Response ``` HTTP/1.1 200 OK Content-Type: application/json ``` ```json { "tm_example_1": { "outputsToAdmit": [0, 2], "coinsToRetain": [1] }, "tm_example_2": { "outputsToAdmit": [1, 2], "coinsToRetain": [], "coinsRemoved": [0] } } ``` In this example, a client submits a BEEF transaction to the `/submit` endpoint together with two topics. The node determines which outputs are newly admitted and which consumed topical coins should be retained or removed, then returns topic-keyed admittance instructions. ## Implementation To implement this data synchronization solution, developers should expose a POST `/submit` endpoint that accepts BEEF transaction submissions, applies topic logic, and returns per-topic admittance instructions. Services that also support history queries should preserve enough ancestry information to satisfy [BRC-64](/overlays/0064.md). --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://bsv.brc.dev/overlays/0022.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.