Transaction Extended Format (EF)
Abstract
Regular Bitcoin transactions do not contain all the data that is needed to verify that the signatures in the transactions are valid. To sign an input of a Bitcoin transaction, the signer needs to know the transaction ID, output index, output satoshis and the locking script of the input transaction. When sending a Bitcoin transaction to a node, only the previous transaction ID and the output index are part of the serialized transaction, the node will look up the locking script and output amount of the input transaction.
We propose an Extended Format (EF) for a Bitcoin transaction, that includes the locking script and the amount in satoshis of all inputs of the transaction. This allows a broadcast service to validate all aspects of a transaction without having to contact a node or an indexer for the utxos of the inputs of a transaction, speeding up the validation.
Copyright
This BRC is licensed under the Open BSV license.
Motivation
Verifying that a transaction is valid, including all signatures, is not possible at the moment without getting the unspent transaction outputs (utxos) from the transactions that are used as inputs from a Bitcoin node (or a Bitcoin indexer). This lookup of the utxos always happens inside a Bitcoin node when validating a transaction, but for a broadcast service to be able to fully validate a transaction (including the fee being paid) it also needs to look up the utxos being spent, which complicates scalability, since this lookup needs to happen on a node (via RPC), that might be too busy to react within an acceptable time frame.
A broadcast service would be able to validate a transaction almost in full if the sender would also send the missing data (previous locking scripts and satoshi outputs) from the utxos being used in the transaction. When creating a new transaction, the previous locking scripts and satoshi outputs are needed to be able to properly sign the transaction, so the missing data is available at the time of the transaction creation. Serializing the transaction to Extended Format, instead of the standard format, is at the point of creating the transaction no extra work, but does make it much easier for a broadcast service to validate the transaction when being received, before sending the transaction to a node.
The main motivation for this proposal is therefore scalability. When incoming transactions contain all the data that is needed to validate them, without having to contact an external service for missing data, the broadcast service becomes much more scalable.
Specification
Current Transaction format:
Version no
currently 2
4 bytes
In-counter
positive integer VI = [[VarInt]]
1 - 9 bytes
list of inputs
Transaction Input Structure
qty with variable length per input
Out-counter
positive integer VI = [[VarInt]]
1 - 9 bytes
list of outputs
Transaction Output Structure
qty with variable length per output
nLocktime
if non-zero and sequence numbers are < 0xFFFFFFFF: block height or timestamp when transaction is final
4 bytes
The Extended Format adds a marker to the transaction format:
Version no
currently 2
4 bytes
EF marker
marker for extended format
0000000000EF
In-counter
positive integer VI = [[VarInt]]
1 - 9 bytes
list of inputs
Extended Format transaction Input Structure
qty with variable length per input
Out-counter
positive integer VI = [[VarInt]]
1 - 9 bytes
list of outputs
Transaction Output Structure
qty with variable length per output
nLocktime
if non-zero and sequence numbers are < 0xFFFFFFFF: block height or timestamp when transaction is final
4 bytes
The Extended Format marker allows a library that supports the format to recognize that it is dealing with a transaction in extended format, while a library that does not support extended format will read the transaction as having 0 inputs, 0 outputs and a future nLock time. This has been done to minimize the possible problems a legacy library will have when reading the extended format. It can in no way be recognized as a valid transaction.
The input structure is the only additional thing that is changed in the Extended Format. The current input structure looks like this:
Previous Transaction hash
TXID of the transaction the output was created in
32 bytes
Previous Txout-index
Index of the output (Non negative integer)
4 bytes
Txin-script length
Non negative integer VI = VarInt
1 - 9 bytes
Txin-script / scriptSig
Script
-many bytes
Sequence_no
Used to iterate inputs inside a payment channel. Input is final when nSequence = 0xFFFFFFFF
4 bytes
In the Extended Format, we extend the input structure to include the previous locking script and satoshi outputs:
Previous Transaction hash
TXID of the transaction the output was created in
32 bytes
Previous Txout-index
Index of the output (Non negative integer)
4 bytes
Txin-script length
Non negative integer VI = VarInt
1 - 9 bytes
Txin-script / scriptSig
Script
-many bytes
Sequence_no
Used to iterate inputs inside a payment channel. Input is final when nSequence = 0xFFFFFFFF
4 bytes
Previous TX satoshi output
Output value in satoshis of previous input
4 bytes
Previous TX script length
Non negative integer VI = VarInt
1 - 9 bytes
Previous TX locking script
Script
<script length>-many bytes
Backward compatibility
The Extended Format is not backwards compatible, but has been designed in such a way that existing software should not read a transaction in Extend Format as a valid (partial) transaction. The Extended Format header (0000000000EF) will be read as an empty transaction with a future nLock time in a library that does not support the Extended Format.
Implementation
The Extended Format has been implemented in go-bt and a standalone JavaScript library bitcoin-ef.
Last updated