Overlay Network Lookup Services
PreviousConfederacy Host Interconnect Protocol (CHIP)NextConfederacy Lookup Availability Protocol (CLAP)
Last updated
Was this helpful?
Last updated
Was this helpful?
Ty Everett (ty@projectbabbage.com)
This document proposes a solution for querying the state of UTXO-based overlay networks by specifying a mechanism for accessing data using lookup services. By utilizing a protected server hosting an API endpoint, overlay network nodes can return relevant UTXOs based on the topics they host and the user's query. This standard defines the parameters and processing steps needed to query for and access topical UTXOs.
As introduced in , UTXO-based overlay networks provide a secure and scalable solution to managing states derived from the Bitcoin network. In order to enable users to query the state of these overlay networks and retrieve relevant UTXOs, a standardized method for lookup services is required. This document aims to address this need by outlining a clear and well-defined process for submitting queries to overlay network nodes and retrieving the resultant UTXOs.
We build on the node first described in and hook into the output admittance and spend events that naturally occur as transactions are submitted. When transactions are received, topical logic determines which outputs will be part of which overlay networks, and we specify that the lookup services respond to these events.
The protected server will host a POST /lookup API endpoint, which accepts JSON request payloads. The JSON request body will include a provider, which is a string denoting the chosen provider, and a query field, whose meaning is determined by the specific provider being chosen.
Upon receiving a query, the overlay network node will perform the following steps:
Engage in authentication and learn the identity of the person making the request, to the extent required in order to fulfill the request.
If consistent with the policy set by the node operator, charge a payment based on the query being performed, providing a mechanism by which the node can monetize its operations.
Check that the provider stipulated by the request is supported on this node. If not, return a JSON error response with a status key of error, a code of ERR_LOOKUP_SERVICE_NOT_SUPPORTED, and a description comprising a human-readable error message.
Send the query to the provider for processing. The provider will use the current known state of the overlay in conjunction with its data storage and retrieval system to fulfill the query.
The provider returns a responsive list of current topical UTXO identifiers (topic labels + TXIDs + output numbers) to the overlay network node, where each identifier is for a UTXO that is currently admitted into a topical overlay.
After retrieving the list of UTXOs responsive to the query from the lookup service provider, the overlay network node will hydrate each UTXO with its output script, amount, encompassing transaction, and other information. The format for the UTXOs constructed by this process is defined by .
The node will then return a JSON response to the consumer, comprising an array of -style UTXOs.
Each lookup service is assigned a provider identifier by the overlay network node so that multiple can be installed simultaneously. For each provider, the overlay network node will send events when new UTXOs are added and when they later become spent. The events will contain the output script, the number of satoshis, the TXID, the output number from the transaction, and the topic identifier for the topic where the output was added or removed.
Each lookup service maintains its own data storage and retrieval mechanism independently. This allows each service to use the most appropriate solution for the specific data being managed. Each service chooses how it will respond to these events, which could involve adding, removing, incrementing, decrementing, replacing, updating, or otherwise mutating data storage and retrieval systems applicable to its specific use-case or indexing methodology. Providers are under no obligation to process events for all topics and can freely drop events they deem irrelevant to their operations. Each service will do this in a way so that it can later provide effective responses to queries.
Below are examples of an HTTP request and response for the /lookup route.
In this example, a client submits a query to the /lookup API endpoint. The JSON payload contains a provider field specifying the desired lookup service provider and a query field with the specific query parameters.
The /lookup route may be behind a paywall to provide a monetization mechanism for overlay network server operators. The rules for this paywall are determined by the specific server being queried and can be based on any aspect of the query being made. All state consumers must be prepared to furnish a payment for lookup requests if enabled by the server, unless the client knows for certain that a payment will not be charged.
This means that consumers should be prepared to handle potential payment requirements when making lookup requests, as the server operator may have implemented a paywall to monetize access to the overlay network's state.
In either case, all lookup provider queries are always protected by authentication, as with 's /submit route, ensuring there are digitally signed records of all requests and responses between the parties.
The overlay network node processes the query and returns an HTTP response containing an array of -style UTXOs that match the query's criteria. The response includes the topic, TXID, output index, output script, number of satoshis, and other fields for each UTXO.
Developers should extend their protected server to host the POST /lookup API endpoint. For monetization, a paywall may be configured in front of the /lookup endpoint, and may charge for lookups based on the queries being performed or the volume of information requested. Developers should ensure that their implementation adheres to the JSON request and response structures specified in this document.