Multicast Shard Manifest Announcement Protocol

Jeff Harris ([e-mail])

Abstract

This BRC specifies the shard manifest announcement protocol for the BSV multicast transaction distribution pipeline. It defines a single UDP datagram format — ShardManifest (MsgType 0x40) — by which any participant periodically declares its shard_bits configuration and the set of shard groups it has joined. Manifests are emitted directly to the control-plane beacon group (0xFFFD, see BRC-129), the same group used by BRC-126 ADVERT beacons, and are distinguished by their MsgType byte. The protocol provides operator visibility into network-wide sharding configuration, enables cross-peer divergence detection, and defines a normative consumer profile for automated, rate-limited shard-bit coordination. It is observation-only by default: there is no retransmission and no acknowledgment.

Copyright

This BRC is licensed under the Open BSV License.

Motivation

Every component of the multicast pipeline — ingress proxy, listener, retry endpoint, and transaction producer — must agree on a single shard_bits value to interoperate, because the shard group index for a transaction is derived as groupIndex = binary.BigEndian.Uint32(txid[0:4]) >> (32 − shardBits) (BRC-129). Today this value is configured manually on each component and there is no on-network signal to verify agreement. A silent mismatch causes senders and receivers to use disjoint group spaces, dropping traffic without an obvious cause.

BRC-139 introduces a small, dedicated announcement that lets every participant:

1. Periodically declare its current shard_bits and the set of shard group indices it claims to have joined.

2. Let observers detect inconsistent configuration across peers.

3. Carry identity, timestamp, TTL, and a GenerationID so a normative consumer profile can implement automated, rate-limited shard_bits shifts safely.

The protocol reuses the existing beacon group rather than allocating a new control-plane index, and requires no proxy involvement: manifests do not transit the BRC-124 ingress path.

Specification

Common Message Preamble

ShardManifest datagrams share the BRC-126 control-message preamble: the MsgType byte at offset 6 occupies the same position as FrameVersion in BRC-124 data frames. Value 0x40 identifies a ShardManifest and is distinct from both the data-frame version codes (0x01–0x07) and the BRC-126 control messages (0x10–0x2F). All multi-byte integers are big-endian.

ShardManifest Wire Format (MsgType 0x40) — 64-byte header + variable payload

Offset	Size	Field	Description
0	4	Magic	0xE3E1F3E8 (BSV mainnet P2P magic)
4	2	ProtoVer	0x02BF
6	1	MsgType	0x40 (ShardManifest)
7	1	Flags	See Flags
8	16	SrcIPv6	Announcer's primary IPv6 (informational; datagram source is authoritative)
24	4	InstanceID	CRC32c of hostname; stable across restarts
28	4	Epoch	Unix seconds when the announcement was generated
32	2	TTL	Validity in seconds; 0 = use consumer default
34	2	AnnounceInterval	Seconds between sends; consumers compute TTL hint = 3 × this
36	1	ShardBits	0–12 (MUST be ≤ 12 per BRC-129)
37	1	RoleHint	Informational role (see RoleHint)
38	2	GroupCount	N: number of groupIndex entries in list form (0 if bitmap or no claim)
40	2	BitmapBytes	M: length of trailing bitmap; 0 ⇒ list form
42	2	SourceCount	K: number of trailing 16-byte source-IPv6 entries; 0 unless SourcesValid
44	4	ManifestCRC	CRC32c over the whole datagram with these 4 bytes treated as zero
48	16	GenerationID	Operator-supplied 128-bit value; bumped whenever ShardBits changes
64	*	Payload	Groups, then Sources, then Successor block (see below)

The payload sections appear in this order after the 64-byte header:

1. Groups — when GroupsValid=1: either GroupCount × 2 bytes of big-endian groupIndex (sorted ascending, no duplicates), or exactly BitmapBytes bytes of bitmap (LSB-first, bit i = group index i). Empty when GroupsValid=0.

2. Sources — when SourcesValid=1: SourceCount × 16 bytes of source IPv6 addresses in network byte order.

3. Successor — when SuccessorValid=1: a 24-byte Successor block (see Successor block).

Total datagram size = 64 + max(N×2, M) + K×16 + (24 if SuccessorValid else 0). Implementations SHOULD keep the total size ≤ 1232 B to avoid IPv6 fragmentation on typical paths. With ShardBits=12 (4,096 groups) the bitmap form is exactly 512 B; the list form is 2 B per joined group. Operators with large source lists SHOULD spread them across multiple announcers (each advertising its own source as the lone entry) so individual datagrams stay within the recommended MTU.

Encoding-form rules

GroupsValid	BitmapBytes	GroupCount	Groups payload form
0	0	0	identity-only (no group claim)
1	> 0	0	bitmap, exactly BitmapBytes bytes; bit positions 0..(BitmapBytes×8)-1
1	0	> 0	list, exactly GroupCount × 2 bytes, sorted ascending, no duplicates

A datagram with GroupsValid=1 and both BitmapBytes=0 and GroupCount=0 is malformed and MUST be rejected. A datagram with GroupsValid=1 and both BitmapBytes>0 and GroupCount>0 is also malformed. For bitmap form, bits at positions ≥ 2^ShardBits MUST be zero and MUST be ignored by consumers.

Sources payload (when SourcesValid=1)

Each 16-byte entry is a publisher source IPv6 in network byte order, contributed by this announcer. Consumers MUST reject the datagram when SourcesValid=1 && SourceCount=0 or when SourcesValid=0 && SourceCount>0, treat entries as set-valued (order not significant), and deduplicate across the union of all currently-valid manifests. When SourceModeSSM=1, consumers feed the union into the Source-Specific Multicast (S,G) join calls for data-plane groups derived from the announced parameters. Each sender SHOULD announce only its own data-plane source (typically a single entry) rather than the operator-curated full fleet list, so per-datagram size stays small and the source set tracks publisher lifecycle.

Successor block (when SuccessorValid=1)

The Successor block signals an in-flight generation transition: the announcer commits to a future ShardBits (and optionally SourceModeSSM) value that becomes the sole active generation at TransitionEpoch. It enables live re-sharding consumers to enter a bridging window before cutover. When present, a 24-byte block is appended immediately after the Sources payload:

Offset (in block)	Size	Field	Description
0	16	SuccessorGenerationID	The incoming generation's 128-bit ID
16	1	SuccessorShardBits	1–12; MUST satisfy |Successor − ShardBits| ≤ 1
17	1	SuccessorFlags	Bit 0 = SuccessorSourceModeSSM; bits 1–7 reserved (0)
18	2	Reserved	MUST be 0
20	4	TransitionEpoch	Unix seconds at which the successor becomes the sole generation

Consumer rules (normative when auto-configuration is enabled):

• Reject the datagram when |SuccessorShardBits − ShardBits| > 1.

• Reject the datagram when SuccessorValid=1 && Authoritative=0 (live re-sharding signals require operator authority).

• Apply the existing adoption gates (quorum, hysteresis) to the Successor block as a unit; the tuple (SuccessorGenerationID, SuccessorShardBits, SuccessorFlags, TransitionEpoch) is the candidate value.

• Consumers that implement live re-sharding MAY enter a bridging window between the moment the Successor block first satisfies quorum and local_clock ≥ TransitionEpoch. Consumers that do not implement live re-sharding MUST treat Successor-block adoption as a divergence event and wait for the pilot to roll GenerationID (promoting the successor to the active generation) before reacting on the new value.

Pilots MUST choose TransitionEpoch ≥ now + 2 × AnnounceInterval (RECOMMENDED ≥ now + 4 × AnnounceInterval) and MUST reject configurations below that floor. Clock skew between pilot and consumers MUST stay below AnnounceInterval / 2; operators SHOULD run NTP.

Flags

Bit	Name	Meaning
0	GroupsValid	The trailing payload carries a valid joined-groups encoding.
1	Authoritative	Operator-curated authoritative announcer; counts toward adoption quorum.
2	Shutdown	Final announcement before graceful shutdown; consumers MAY evict immediately.
3	SourceModeSSM	Data plane uses Source-Specific Multicast (FF3x::/32, RFC 4607); consumers MUST use the SSM prefix.
4	SourcesValid	The trailing payload includes SourceCount × 16 bytes of publisher source IPv6 addresses. 0 ⇒ none.
5	PilotOnly	Groups describe desired fleet state, not the announcer's own joins; implies Authoritative=1.
6	SuccessorValid	The trailing payload includes a 24-byte Successor block; requires Authoritative=1.

Bit 7 is reserved and MUST be 0. Consumers MUST reject PilotOnly=1 && Authoritative=0 and SuccessorValid=1 && Authoritative=0 as malformed.

RoleHint

RoleHint is informational; consumers SHOULD NOT make filtering decisions on it.

Value	Role
0	generic
1	proxy
2	listener
3	retry-endpoint
4	producer
5	manifest-only

Values ≥ 6 are reserved.

ManifestCRC

ManifestCRC is a CRC32c (Castagnoli polynomial) computed over the entire datagram with the four CRC bytes themselves treated as zero. Consumers MUST verify the CRC before acting on the manifest.

Multicast Group and Scope

Manifests are sent directly to the beacon group defined in BRC-129 (index 0xFFFD), the same group used by BRC-126 ADVERT:

Index	Scope	Compressed Address
0xFFFD	FF05	FF05::B:FFFD
0xFFFD	FF08	FF08::B:FFFD
0xFFFD	FF0E	FF0E::B:FFFD

The announcer chooses one or more scopes; when multiple are configured the same datagram is sent to each. The proxy is not involved. Because BRC-126 ADVERT (MsgType 0x20) shares this group, listeners MUST dispatch on the MsgType byte (offset 6) before parsing: 0x40 ⇒ ShardManifest; 0x20 ⇒ BRC-126 ADVERT.

Cadence and Freshness

Parameter	Default	Notes
AnnounceInterval	300 s	Every 5 minutes
Default TTL	900 s	3 × AnnounceInterval (used when TTL = 0)
Send jitter	±10 %	RECOMMENDED, to avoid global synchronisation

Consumers MUST treat entries older than Epoch + TTL (or Epoch + 3 × AnnounceInterval when TTL=0) as expired. There is no retransmission and no NACK semantics; loss is tolerated by re-announcement, and an empty registry is valid.

Identity and State

Consumers SHOULD key registry entries on (SrcIPv6, InstanceID). The authoritative SrcIPv6 is taken from the IPv6 datagram header; the in-frame SrcIPv6 field is informational only. InstanceID is the CRC32c of the announcer's hostname, matching BRC-126 ADVERT semantics, and is stable across restarts. GenerationID is an operator-supplied 128-bit value (typically a UUID) that operators MUST bump whenever ShardBits changes; it is opaque to the protocol and consumers compare it for change detection.

Consumer Behaviour — Observation (informative)

A consumer MAY join the beacon group(s) of interest and:

1. Dispatch incoming datagrams on the MsgType byte. 0x40 ⇒ ShardManifest.

2. Verify ManifestCRC; reject on mismatch.

3. Upsert into a registry keyed on (SrcIPv6, InstanceID).

4. Evict on Epoch + TTL (or Epoch + 3 × AnnounceInterval when TTL=0).

5. Surface metrics: per-peer ShardBits, joined-group count, last-seen, and the count of distinct ShardBits values currently observed (divergence).

Consumer Behaviour — Auto-configuration (normative when opted in)

A consumer that opts in to automatic configuration MUST implement the observation requirements above and additionally MUST satisfy the following. Components that do not opt in are unaffected.

1. Authoritative-only adoption. Manifests with Authoritative=0 MUST NOT contribute to any adopted value (they MAY still be indexed for visibility and contribute to the source-set union below).

2. Quorum. A candidate value is eligible for adoption only when reported by at least pilot-quorum distinct authoritative announcers (keyed on (SrcIPv6, InstanceID)) within their TTL window. pilot-quorum MUST be configurable; default 2.

3. Hysteresis. A candidate that satisfies quorum MUST hold quorum continuously for ≥ 2 × AnnounceInterval before adoption. A change in adopted value resets the timer.

4. ShardBits shift bound. A consumer MUST NOT adopt a ShardBits value differing from the currently adopted value by more than ±1 within any rolling AnnounceInterval window.

5. Manual pin precedence. An operator-pinned value is the local authority and MUST NOT be overridden by adoption; the consumer MUST still evaluate quorum and emit divergence telemetry when the adopted candidate differs from the pin.

Fields subject to adoption: ShardBits, SourceModeSSM, and the deduplicated union of SourcesValid payloads (the source set is not gated by quorum — it is the union of every currently-valid manifest's sources, irrespective of Authoritative). MCGroupID is not carried in the payload; consumers MUST derive it from the destination address of the beacon socket on which the manifest was received. Consumers SHOULD rate-limit source-set additions and removals before feeding them into kernel join calls to avoid thrashing the multicast forwarding information base, and MUST NOT label any metric with raw source IPv6 addresses.

Safety Guidance

1. Bump GenerationID whenever ShardBits changes.

2. Keep authoritative announcers to a small operator-curated set (RECOMMENDED: three instances across failure domains).

3. Deploy authoritative announcers with PilotOnly=1 so consumers can distinguish operator intent from an announcer's own joins.

4. Treat non-authoritative manifests as observational (and, when SourcesValid=1, as per-publisher source contributions).

5. Warn — do not auto-shift — on observed ShardBits divergence; the adoption gates already prevent unsafe shifts, but operator visibility is where misconfiguration is caught.

Interactions With Other BRCs

• BRC-126 (Retransmission / ADVERT) — shares the beacon group 0xFFFD and listen port. Distinguished by the MsgType byte (0x20 ADVERT vs 0x40 ShardManifest). BRC-139 is neither retransmitted nor a retransmitter.

• BRC-127 (Subtree group announcements) — orthogonal: BRC-127 announces SubtreeID→GroupID bindings on 0xFFFC via the proxy; BRC-139 announces participant configuration directly on 0xFFFD.

• BRC-129 (Multicast addressing) — no new index allocated; manifests reuse the existing beacon group, and ShardBits ≤ 12 per BRC-129.

References

• BRC-82: Defining a Scalable IPv6 Multicast Protocol for Blockchain Transaction Broadcast and Update Delivery — Overall multicast protocol architecture

• BRC-124: Multicast Transaction Frame Format — Data-plane frame format and HashKey/SeqNum derivation

• BRC-126: Multicast Transaction NACK Retransmission Protocol — ADVERT beacon sharing the same beacon group

• BRC-129: IPv6 Multicast Group Address Assignments — Beacon group and shard_bits bound

Constants Reference

Name	Value	Hex	Description
MagicBSV	3823236072	0xE3E1F3E8	BSV mainnet P2P magic
ProtoVer	703	0x02BF	Protocol version
MsgTypeShardManifest	64	0x40	ShardManifest datagram type
ShardManifestHeaderSize	64	0x40	Fixed header size in bytes
GroupBeacon	65533	0xFFFD	Control-plane beacon group index
FlagGroupsValid	1	0x01	Flags bit 0: groups payload present
FlagAuthoritative	2	0x02	Flags bit 1: operator-curated announcer
FlagShutdown	4	0x04	Flags bit 2: final pre-shutdown announcement
FlagSourceModeSSM	8	0x08	Flags bit 3: data plane uses SSM addressing
FlagSourcesValid	16	0x10	Flags bit 4: sources payload present
FlagPilotOnly	32	0x20	Flags bit 5: groups describe desired fleet state
FlagSuccessorValid	64	0x40	Flags bit 6: Successor block present
DefaultAnnounceInterval	300	—	Default re-announce period (s)
DefaultTTLMultiplier	3	—	TTL = 3 × AnnounceInterval when TTL=0