Overlay Network Data Synchronization

Ty Everett (ty@projectbabbage.com)

Abstract

This document proposes a solution for synchronizing data across UTXO-based overlay networks by specifying a secure and efficient mechanism for submitting and processing transactions. By utilizing a BRC-31 protected server hosting an API endpoint, overlay network nodes can process and admit transactions based on their relevance to the hosted topics. This standard defines the parameters and processing steps needed to track topical UTXOs.

Motivation

As discussed in BRC-59, UTXO-based overlay networks provide a secure and scalable solution to managing states derived from the Bitcoin network. In order to maintain a consistent and up-to-date state across nodes, a method of data synchronization is required. Additionally, it is essential to provide a standard way for network participants to submit new transactions and notify network operators of their relevance to hosted topics. This document aims to address these needs by outlining a clear and well-defined process for submitting transactions and processing them based on topic-specific rules.

Specification

We start with a server that is set up to track the state of one or more topics. Every individual server implementing this specification decides what specific topic labels mean, and which rules they want to apply when admitting new UTXOs.

API Endpoint and JSON Parameters

The BRC-31 protected server will host a POST /submit API endpoint, which accepts JSON request payloads. The JSON request body will include a BRC-8 transaction envelope (rawTx, inputs, mapiResponses, proof) and an additional field called topics. The transaction envelope facilitates SPV checking, while the topics field denotes which overlay network topics should be notified about the transaction.

Overlay Network Node Processing Steps

Upon receiving a transaction, the overlay network node will perform the following steps:

  1. Verify the BRC-31 identity of the sender and establish the required level of identity for submitting transactions, negotiating under the BRC-31 protocol to the satisfaction of the network operator and the transaction sender.

  2. Check whether the node hosts at least one of the tagged topics from the payload. If none are hosted, the node returns an early response indicating no outputs were admitted. Otherwise, it can drop any unhosted topics from the list before proceeding.

  3. Use the BRC-9 process to verify the submitted BRC-8 envelope and confirm the transaction's legitimacy.

  4. Apply topic-specific logic to the transaction by iterating through all specified topics and performing the following for each topic:

    • Check the transaction's list of inputs for UTXOs that are part of the current topical overlay. Flag these inputs and communicate the information to the topic's management logic.

    • Apply the topic-specific management logic and protocol rules to determine output admittance for each transaction output.

    • Update the node's tracking system, associating each relevant output with the topic labels of topics that have admitted those outpoints. If implementing BRC-24, notify lookup services about the new incoming entries.

    • Remove any now-spent inputs that were previously associated with a topic from the tracking system, as they have now become spent by this transaction. If implementing BRC-24, notify lookup services about the old outgoing entries.

  5. Generate and return a JSON response object containing the transaction processing results, including a status key with a value of success, and a topics key with an object containing topic labels and arrays of output numbers denoting which outputs were admitted into the specific topics.

  6. Pursuant to any peering or data synchronization arrangements or contracts that the node operator may have negotiated with other node operators with whom they would like to remain in sync, use the agreed-upon exchange mechanisms to notify the other node operators about the received transaction, potentially in conjunction with an incoming or outgoing BRC-41 payment.

Example

We provide an example of an HTTP request and response.

Request

POST /submit HTTP/1.1
Host: example-overlay-node.com
Content-Type: application/json
X-Authrite: 0.1
X-Authrite-Identity-Key: ...
X-Authrite-Signature: ...
X-Authrite-Nonce: ...
X-Authrite-YourNonce: ...
X-Authrite-Certificates: ...
{
  "rawTx": "0100000001abcdef...",
  "inputs": {
    "...": "..."
  },
  "mapiResponses": [
    {
      "payload": "0100000001abcdef...",
      "signature": "MEQCIGXFM...",
      "publicKey": "036b8c86..."
    }
  ],
  "topics": [
    "example_topic_1",
    "example_topic_2"
  ]
}

Response

HTTP/1.1 200 OK
Content-Type: application/json
X-Authrite: 0.1
X-Authrite-Identity-Key: ...
X-Authrite-Signature: ...
X-Authrite-Nonce: ...
X-Authrite-YourNonce: ...
X-Authrite-Certificates: ...
{
  "status": "success",
  "topics": {
    "example_topic_1": [0, 2],
    "example_topic_2": [1, 2]
  }
}

In this example, a client submits a transaction to the /submit API endpoint. The JSON payload contains the BRC-8 transaction envelope (rawTx, inputs, mapiResponses, proof) and an additional field called topics, which includes two example topics: "example_topic_1" and "example_topic_2".

The overlay network node processes the transaction and determines that outputs 0 and 2 are admissible for "example_topic_1" and outputs 1 and 2 are admissible for "example_topic_2". The node then returns an HTTP response with a status of "success" and a topics object containing the admitted outputs for each topic.

Implementation

To implement this data synchronization solution, developers should first set up a BRC-31 protected server to host the POST /submit API endpoint. The server should be capable of processing JSON request payloads and performing the necessary transaction validation using BRC-9.

Overlay network nodes must be programmed to process incoming transaction submissions following the steps outlined in the specification. This includes verifying the sender's BRC-31 identity, checking for hosted topics, validating the transaction using BRC-9, and applying topic-specific logic to determine output admittance.

Developers should ensure that their implementation adheres to the JSON request and response structures specified in this document, enabling seamless interaction between overlay network participants.

PreviousMandala Token ProtocolNextConfederacy Host Interconnect Protocol (CHIP)

Last updated