# Overlay Services Synchronization Architecture Ty Everett () ## Abstract This document outlines the Overlay Services Synchronization Architecture, detailing the Services Host Interconnect Protocol (SHIP) and Services Lookup Availability Protocol (SLAP). These protocols enable efficient peer discovery and data synchronization across UTXO-based overlay networks. This BRC defines the components, interactions, and responsibilities of nodes running these protocols to maintain reliable and up-to-date overlay services. ## Motivation The proliferation of UTXO-based overlay networks necessitates robust mechanisms for peer discovery and data synchronization. While [BRC-22](/overlays/0022.md) defines transaction submission and processing, SHIP and SLAP extend this by providing standardized methods for discovering the overlay hosts that run particular services. This architecture ensures seamless interaction between network participants, improving the efficiency and reliability of data propagation. It can also enable the users of services to find hosts that are interested in their transactions. ## Background Readers of this document should be familiar with BRCs 45, 59, 22, 24, 23, and 25. ## Specification ### Overview The Overlay Services Synchronization Architecture comprises two fundamental protocols: 1. **Services Host Interconnect Protocol (SHIP)** 2. **Services Lookup Availability Protocol (SLAP)** Each node running overlay services will implement topic managers and lookup services for both SHIP and SLAP, ensuring alignment between advertised services and their actual configuration. ### Status Note The core SHIP/SLAP concepts in this document remain current, but several transport details have evolved in current interoperable implementations: * SHIP submissions are commonly BEEF over `POST /submit` with topics provided separately. * SLAP/lookup resolution is performed against **service** names such as `ls_kvstore`. * Topic names used with current SDK tooling are expected to start with `tm_`. * Returned lookup data is now standardized around `output-list` responses backed by BEEF, rather than hydrated JSON UTXO objects. * Host resolution includes caching, stale-while-revalidate refresh, and host backoff/reputation behavior. Readers should therefore interpret this document together with the updated [BRC-22](/overlays/0022.md), [BRC-24](/overlays/0024.md), and [BRC-64](/overlays/0064.md) documents. ### Components * **SHIP Topic Managers**: Manage topic-specific transaction admittance for topics such as `tm_example`. * **SLAP Lookup Services**: Provide mechanisms for querying indexed overlay state for services such as `ls_example`. * **Advertiser**: Handles the creation and revocation of SHIP and SLAP advertisements. * **Overlay Services Engine**: Coordinates the processing of transactions, alignment of advertisements, and synchronization of data across various services. ### Key Responsibilities #### 1. Ensuring Alignment Each node must ensure alignment between the SHIP and SLAP advertisements it has sent out and the current set of topic managers and lookup services it is configured with. #### 2. Hosting SHIP/SLAP Overlay Networks Nodes host copies of the SHIP and SLAP overlay networks, ensuring they are updated with new advertisements or revocations. * **Submission**: Any node can submit SHIP/SLAP advertisements to another node if it knows its host base URL, updating the overlay network. * **Internal Updates**: Nodes submit their own advertisement transactions to themselves to update the overlay networks about new advertisements or revocations, thereby ensuring their hosted copies are always current and propagating them to other nodes. #### 3. Transaction Propagation As part of the transaction submission process, nodes notify each other about transactions on topics they host and care about. When new transactions are submitted to a node, the Engine: * Ensures transactions are propagated to relevant nodes based on SHIP advertisements. * Updates lookup services with new or spent outputs. * Delegates transaction submission, verification, and output admittance to the relevant topic managers. * Returns per-topic admittance instructions indicating admitted outputs and retained historical coins. Current resolver implementations also: * query multiple SLAP trackers in parallel, * cache resolved competent hosts per service, * reuse stale host sets while refreshing them in the background, * and temporarily back off failing hosts rather than hammering them continuously. Current broadcaster implementations also: * reject topic names that do not start with `tm_`, * use `ls_ship` lookups with a query shaped as `{ "topics": ["tm_example"] }` to find interested hosts, * use `http://localhost:8080` for local-development presets, * require HTTPS host URLs in production unless an implementation explicitly enables local HTTP, * and, by default, consider a broadcast successful only when at least one responsive host acknowledges every requested topic. Applications may tighten this by requiring acknowledgment from all interested hosts for selected topics, or may configure more specific host requirements. ### SHIP and SLAP Token Formats #### SHIP Token Format | Field | Meaning | | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Field 1 | The string `SHIP`, denoting a SHIP advertisement. | | Field 2 | The [BRC-31](/peer-to-peer/0031.md) identity key of the advertiser. | | Field 3 | The host base URL advertised for submissions. Current interoperable implementations expect an HTTPS base URL in production and allow plain HTTP only for local development. | | Field 4 | The topic name hosted by the advertiser. | #### SLAP Token Format | Field | Meaning | | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Field 1 | The string `SLAP`, denoting a SLAP advertisement. | | Field 2 | The [BRC-31](/peer-to-peer/0031.md) identity key of the advertiser. | | Field 3 | The host base URL advertised for lookups. Current interoperable implementations expect an HTTPS base URL in production and allow plain HTTP only for local development. | | Field 4 | The service name, represented by a [BRC-24](/overlays/0024.md) service ID. | ### Token Creation and Submission The advertiser creates a transaction output containing the token and submits it to known nodes, including their own. The process involves: 1. Encoding the advertisement fields as a PushDrop token. 2. Locking the token using the advertiser's wallet identity under the BRC-100 protocol `[2, "Service Host Interconnect"]` for SHIP or `[2, "Service Lookup Availability"]` for SLAP, key ID `1`, counterparty `self`. 3. Creating and submitting the transaction output containing the token. ### Token Validation and Admission Nodes validate the identity key's link to the signature-producing key before admitting the output. The steps include: 1. Extracting token fields. 2. Verifying the SHIP/SLAP identifier. 3. Verifying the locking key and signature. 4. Admitting the token if valid. ### Implementation Developers should implement a server that hosts SHIP topic managers and SLAP lookup services. Production interoperability currently assumes HTTPS host URLs. Local-development interoperability commonly uses `http://localhost:8080`. The custom multi-scheme facilitator ideas discussed in [BRC-101](/overlays/0101.md) are not part of the current minimum interoperable behavior and should be treated as future extensions rather than baseline requirements. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://bsv.brc.dev/overlays/0088.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.