Introduction to Bitcoin concepts
In this overview, you’ll learn how the Bitcoin system works in general and what terminology we use to describe each part. This is recommended reading for anyone who is new to Bitcoin development. If you’ve been around for awhile, you can probably just skim it or skip it altogether.
In addition to this overview, we highly recommend reading Satoshi Nakamoto’s original high-level description of the Bitcoin system, published in 2008.
We’ll refer often to 21, which is designed to provide everything you need to follow the tutorials on 21.co, including a technology that lets you receive a steady stream of satoshis (fractional bitcoins) for use in creating actual transactions that will appear on the blockchain.
One of the most frequently used data types in Bitcoin is a cryptographic hash. (We’ll simply call them “hashes” from this point on, but you should be aware that there are non-cryptographic hashes that don’t have the same essential security properties.)
A hash function takes an arbitrary amount of input data and turns it into a short fixed-length string (byte array). Every time you hash the same input data with the same hash function, you get the same output hash. But every time you hash different data—even if you use the same function—you get a completely different hash. The data is so different that there’s no way to predict what any part of it will be.
## Hash three different strings $ echo foo | sha256sum b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c - $ echo foo | sha256sum b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c - $ echo bar | sha256sum 7d865e959b2466918c9863afca942d0fb89d7c9ac0c99bafc3749504ded97730 -
We’ll look at some specific examples of hashes in Cryptographic Hash Functions.
Proof of work
Cryptographer Adam Back realized in 1997 that it was possible to use hash functions to prove that you did a certain amount of computational work. His discovery took advantage of the special property of strong cryptographic hashes where each hash looks like random data regardless of what input was used to create it.
For example, the main hash in Bitcoin produces a 256-bit output, which means that it’s output is similar in randomness to flipping 256 coins in sequence and recording each heads as a 0 and each tails as a 1.
So if you create two hashes from two different inputs, you would expect (on average) that the first bit of one of those hashes would be a 0 (“heads”). That means a hash starting with a zero bit is proof that you did the computational work (“number of coin flips”) necessary to create (on average) two hashes.
Extending this, a hash starting with a zero byte (8 zero bits, or “8 heads”, in a row) is proof you did the computational work (on average) to create 256 hashes.
The nice thing about hashes rather than coin flips is that it’s possible for anyone else in the world to verify you did the work by simply hashing the same input data you used with the same hash function. It doesn’t matter whether it took 256 hashes or 256 trillion hashes to get your result, anyone else in the world can verify it by doing a single hash.
The Bitcoin block header
Bitcoin block headers are 80 bytes long and they’re hashed to produce proof of work (we’ll go into detail about how this is done in Mine the Genesis Block). It’s useful to understand how this proof of work protects Bitcoin.
Each block header contains the hash of the previous block. Remember that changing any input totally changes the hash output. This means that if block header 2 contains the hash of block 1’s header, block 1’s header can’t be changed without making block 2 no longer link to it.
Bitcoin requires that every block header link to a previous block header back to the first block (called the Genesis block). If a Bitcoin program can’t verify the link between a block header and its parent block header, it considers that block invalid (called an orphan block).
(We'll discuss stale blocks below in the Header Chain Forks section.)
This means that proof of work in Bitcoin is cumulative. The proof of work (average number of hashes) demonstrated by block header 2 adds to the proof of work demonstrated by block header 1. A block header 3 that linked to block header 2 would further increase that accumulated proof of work.
When a header has a single child, it’s considered to be confirmed once. If that child header has its own child header, the original header has two confirmations, and so on. As of this writing, the first block header, created in January 2009, has more than 380,000 confirmations, demonstrating more proof of work than the current fastest supercomputer in the world could generate in a million years.
This link between block header 2 and block header 1 is called a header chain. A link from block header 3 to block header 2 would extend that chain.
Every Bitcoin client looks for the best header chain: the chain of valid block headers that has the most cumulative proof of work. We’ll describe why this chain is treated specially in a moment.
Some resources, even Satoshi Nakamoto’s original Bitcoin paper, incorrectly describe this special chain as “the longest chain”; but since different headers can demonstrate different amounts of proof of work, what matters is the cumulative proof of work, not the number of headers chained together.
The process of generating proof of work for block headers is called mining, so named because the people who create the first 6.93 million bitcoin blocks receive newly-minted bitcoins as a reward for their work, similar to how gold miners receive gold for their work.
Based on the bitcoin inflation graph by TheRealSteve, which was released under the CC0 public domain grant
Recall from above that the output hash for a particular piece of input data seems random the first time you generate that hash. This means it's not possible to predict when a miner will produce a header with the amount of proof of work necessary to add that header to the best header chain. As a consequence, mining is competitive—a large miner who can create hashes twice as fast as a smaller miner will add twice as many headers to the chain, but the smaller miner will still add half that number of hashes and receive roughly half the compensation—so if the smaller miner has a better profit margin than the larger miner, the smaller miner could still be better off.
Bitcoin miners today all use hardware specially optimized for generating Bitcoin-style proof of work hashes. This hardware is generically called ASICs after the circuits they use, Application-Specific Integrated Circuits. The 21 Bitcoin Computer includes a latest-generation Bitcoin mining ASIC.
Header chain forks
One complication of miners competing with each other is that two miners may each create a new valid header at the same time and both headers will refer to the same parent header. This is called a chain fork. Since each header can only refer to one previous header, every Bitcoin miner needs to choose which of those competing previous headers to use in the next header they try to create.
When the next block after the chain fork is created, it makes that part of the chain into the best header chain—the chain demonstrating the most proof of work. At this point, the miner who created the header that is not part of the best header chain loses his chance at the Bitcoin reward and may be considered to have wasted his proof of work.
In the illustration above two miners create a header referring to the previous header with a hash of #90. When the block with hash #48 is received, it makes header hash #19 part of the best header chain and turns header hash #32 into a stale header—a header that is not part of the best header chain.
Short chain forks are a natural part of Bitcoin, although a lot of engineering goes into trying to minimize their occurrence so that the maximum amount of proof of work is added to the headers chain.
Blocks of transactions
Bitcoin is a transactional system, but there are no transactions in the block headers. Instead, miners collect a group of transactions called a block of transactions and then hash them using a special formula. The hash of the transactions, called a merkle root, is included as 32 of the 80 bytes in the block header. Together the block of transactions and the corresponding header are called a block.
Since hashing the same block of transactions will always produce the same hash as output, this proves the miner chose those particular transactions; since the block header is protected by proof of work, it’s not possible to change that hash and not possible to change which transactions appear in the block.
For reasons that will be explained elsewhere, a simple hash is not used; instead a structure called a merkle tree is built out of multiple hashes (currently a few thousand hashes is common). The merkle tree structure eventually resolves down to a single hash, the merkle root. It’s this merkle root that is placed within the block header.
Because blocks of transactions are attached to the block header, and because those transactions are a critical part of Bitcoin, we commonly talk about the blockchain. It’s not possible to use the blockchain without the header chain—but it is possible to use the header chain without the blockchain (which is a distinct advantage in some situations because the header chain is currently less than 50MB while the blockchain is currently over 55GB). We’ll discuss more about the block chain in a moment.
Transactions, keys, and signatures
Bitcoin transactions spend bitcoins. There are a large variety of transaction types, but by far the most common uses a very simple system involving public key cryptography (PK).
To use PK, you first generate a private key. As the name implies, you must keep this key private or other people will be able to spend your bitcoins. Using the private key, you generate a public key. As that name implies, you can share the public key with other people.
Bitcoin adds an extra wrinkle by letting you compress and encode the public key into something called a Bitcoin address, which you can also share with other people.
When you receive bitcoins, they're sent to your public key or your address. When you spend bitcoins, you use your private key to create a signature which you add to the spending transaction to prove that your private key authorized spending those bitcoins.
Script and opcodes
The description above about using PK to receive and spend bitcoins is accurate, but not precise. Instead of using PK directly, Bitcoin includes a scripting language (whose only name is "Script") that allows the bitcoins to be encumbered by arbitrary conditions. The encumberance is placed in a public key script (pubkey script).
When you want to spend that transaction, you create a signature script that satisfies those conditions. In this sense, public key scripts function like programmable public keys and signature scripts function like dynamic signatures. There are a few different common scripts used, such as pay-to-public-key-hash (P2PKH) and pay-to-script-hash (P2SH).
This extra flexibility lets you do things you couldn't do with regular public keys and signatures. For example, you could specify that your bitcoin transaction had to be signed by two signatures instead of one. Much more complex conditions are possible and you'll see some of them in our other documentation.
When you add your signature to a transaction to prove that you're authorized to spend those bitcoins, somebody has to verify that proof. This is similar to how you have to login to your bank website to pay a bill online. Of course Bitcoin is decentralized, so there's no single person or company who is responsible for verifying transactions are correct—instead, it's users of bitcoin who each must choose whether they think it's important for them to perform the verification.
Full nodes and consensus
Full verification of transactions requires special software, called a full (verification) node. Each full node verifies every transaction on the blockchain, and they often also verify transactions before miners add them to the blockchain.
Each full node does its verification independently of every other full node, but they all follow exactly the same rules, and they process exactly the same data, so they come to exactly the same conclusions about which transactions are valid. This is called consensus, and the rules the full nodes follow are called consensus rules.
The consensus rules are applied against the blocks miners add to the blockchain. If a full node considers any block to contain invalid transactions (or an invalid block header), it rejects that block and any descendant blocks. This doesn’t stop miners from adding invalid transactions to blocks, but it does prevent those miners from receiving the bitcoin reward for creating a block.
The blockchain that all full nodes agree has the most proof of work and contains only valid data is called the best blockchain. Most other people simply call it the blockchain.
Wallets and Simplified Payment Verification (SPV)
Wallet software generates and stores your private and public keys, lets you know when you’ve received a transaction, and lets you spend those bitcoins when you're ready. Wallets don’t have to be full nodes and most today aren’t, which means they operate at reduced security. However, that security can still be quite strong.
Using the merkle tree structure described earlier, Simplified Payment Verification (SPV) wallets can prove a particular transaction was part of a block protected by proof of work. Because proof of work is hard to generate and because other people run full nodes that only reward miners who create valid blocks, it is widely believed that SPV wallets provide reasonable security for moderate amounts of bitcoins even though they can’t verify that they’re receiving real bitcoins.
The final piece of the Bitcoin puzzle when it comes to development is the network protocol that allows wallets, full nodes, and miners to communicate. The main Bitcoin protocol is a Peer-to-Peer (P2P) network designed as a gossip network—that is, software sends its data to all of its peers, and those peers send that data to all of their peers, and so on until everyone has all the data they need.
This works reasonably well, although miners who need low latency have partly split off and formed their own network, the Block Relay Network.
One advantage of the main P2P network is that it operates well over the Tor anonymity network, which helps people send their transactions with increased privacy. Another advantage is that there are no privileged nodes whose downtime would significantly hurt the network. Only the loss of thousands of nodes or a break in the actual Internet (called a partition) would adversely affect the network.