Bitcoin Q&A: Iterating nonces and the block reward

Let me explain what the nonce is, in mining.
We have talked about cryptographic hash functions, which produce a specific fingerprint [of] data,
such that if you make a tiny change to the data, it will change the fingerprint, allowing you
to determine that the data has changed. The hash [could be] a completely different number
even if you only change one bit in the input. Essentially, what hash functions do is take all the bits
of your data and stirs them around algorithmically. If you look at how the algorithm works,
it adds up binary numbers with a modulo. If you add up to more than 32 bits, it wraps around
and starts again, which mixes those bits. There are also binary shift operations where the
algorithm will shift a chunk of bits to the left or right, and then add them to other bits. It is [similar to] shuffling cards. The data that
goes in to the cryptographic hash algorithm… is cut into chunks and then mixed together, until you
eventually [produce] a 256-bit number as the output. Your cryptographic hash is the fingerprint.
If one of those bits of data is different, it causes the mixing process to go down a different path,
which produces a completely different fingerprint. Just changing one bit changes the cryptographic hash
function’s output, which is effectively a random number. It is a random number in that
you can’t predict what it will be. But it is also a deterministic number,
because if you input the same data, the algorithm will give you the
exact same fingerprint every time. So it would be very easy for anyone to verify the data.
If you put the same data through the algorithm [again], you will get the same fingerprint
as everybody else who hashes it. This is a very interesting trick. Changing the
data even by a tiny bit will seemingly produce… a completely different random number;
if you present the same data to someone else, and they put it through the hash function,
it will produce the same fingerprint. What if you set a target? What if I want my fingerprint
to start with a zero? What are the chances it will? If you are considering it in binary [terms],
the first digit can either be zero or one. If you keep [generating] fingerprints, half of them
will start with zero and half will start with one. You have a 50 / 50 chance. What if you wanted both of the
first two binary digits to be zeros? In that case, you have a 25% or 1 / 4 chance
that the first two binary digits will be zeros. If you keep generating fingerprints, hashing and hashing,
one out of four will start with two binary zero digits. If you keep going and require [another] zero, you
will have half as many hashes which fit that target. [Let’s say that you require] thirty zeros. Only one in a
quintillion hashes will [begin] with thirty zero digits. In order to find that [hash], you must modify
your block data slightly, calculate the hash, and see if it has thirty leading zeros. It [probably] won’t. Try again a quintillion times until
you produce a hash that starts with thirty zeros. What is the nonce? The nonce is the
part of the block that you are changing. The block header contains a timestamp,
block number / height, and the Merkle root, which is like a fingerprint of all
the transactions in the block. It contains the hash of the previous block.
For a block to be valid, this information cannot change. You can change the timestamp by a few seconds,
but the rest of this data can’t be changed. The previous block and fingerprint of transactions
will stay the same in the candidate block you built. There is a special 32-bit part of block that doesn’t do
anything; you can put any number you want in there. Why does that space exist it? It exists to allow you to
change the block header and produce a different hash. We call that the “nonce.” Let’s say you [try to generate
a block hash] with a certain number of leading zeros. Does it have the necessary number of zeros in
the beginning to meet your target? No, try again. As you keep incrementing that 32-bit number, every time
you [generate a block] hash, it is a different fingerprint. You need [to generate about] five quadrillion [hashes]. Eventually, you will produce a hash
which starts with [enough] zeros by chance. Then you can tell everyone [in the network], “Here is a
block! If you put this nonce in, the number 6,475,322, and [generate] the block header,
it has thirty leading zeros in it.” How do you know that it took me a lot of time to find?
You know that the only way I could have [found it]… was by plugging [random] numbers into the nonce,
[generating a block header], and checking the results. That is what a nonce is, the space in the block
where you can just plug in a random number. [You must] keep trying in order to produce
a fingerprint [that matches the target]. Then you [share] that random number with everybody
[in the network], and they can verify it by producing… the fingerprint for the block [with the nonce],
with a special pattern of zeros in the beginning. [They know] it must have [required] a lot
of work to find that random number. It is proof that you did the work. It is proof-of-work.
That is how the nonce [functions within mining]. There are some complications, because 32 bits
is not enough space to do all the hashing you need. Miners also change some bits in the
coinbase, and part of the timestamp. They need to move a lot of bits in the input. The nonce isn’t only in one place,
but that is an implementation detail. There are a couple of follow-up questions on that, which
I find interesting. Good questions in the chatroom today. In talking about nonces… “Do the miners know the amount of zeros required
for the next block?” Yes. That is the difficulty target. The difficulty target is how many zeros the hash
needs to have, in binary. Just think of it as a number. If I would say that a number starts with [some amount
of] zeros, effectively the target number is less than that. If ‘x’ is the target, the hash must be a smaller number
than the difficulty target, which is known by every node.. on the network; every node calculates the difficulty
target using the same formula every 2016 blocks. They input the same numbers into that
formula and produce the same answer. [They look at] what the difficulty target
was for the previous two-week period, how many blocks have been mined since,
and how long did those blocks take to mine? In two weeks, we want to mine 2016 blocks,
and each block should be mined in ten minutes. Theoretically, that would take 20,160
minutes in total to mine these blocks. Well, how long did it take this time?
Oh, it took 10% longer than usual. This means that miners are not able to find
blocks every ten minutes, they are a bit slower. It is 10% too difficult.
The difficulty needs to be reduced by 10%. The number of blocks is fixed, so everybody knows how
long it took to mine them. They count it the same way. They can do division and find exactly
how much to adjust the difficulty target. Everybody will know the difficulty target for the previous
2016 round, and the difficulty target in the new period. ‘Based on my calculations, I expect the
difficulty target for the next period will be ‘x.’ ‘I will only accept blocks where the hash is less than ‘x.’
Everybody will calculate the same answer for ‘x.’ Everybody is [expecting] miners to achieve
the same amount of proof-of-work for the next block. The difficulty retargeting happens every two
weeks, or more specifically every 2016 blocks. Everyone on the network will know exactly what
the difficulty should be, at any moment in time, and arrive at the same conclusion. Miners know what they need to do to succeed. If they have the target, [they evaluate], is the hash
less than the target? If yes, it is a valid block. If no, keep hashing. Try more nonces. Another question is, “I wonder, is it possible to guess
the nonce on the first try?” Yeah, it is possible. You might find it on the first try, you might find it
on the quintillionth try. It doesn’t matter. The probabilities of finding it on your
first, second, or third try are the same. These are independent events; the probability of each
event does not affect the previous or the next event. There is [no dependency] link between them. Just as if I were to flip a coin. Whether it comes up
heads or tails, the chances the next time I flip it… are still 50%, even if I flip a hundred times
and it comes up tails one hundred times. On the hundredth and first flip, the probability is 50%
that it will come up heads, 50% that it will come up tails. The probability doesn’t change.
It is the same thing for picking a nonce. Could the first nonce I plug in produce
a hash that is below the difficulty level? Yes. And it is exactly the same [for] the second nonce
I try. I could try them randomly or in decreasing order. I could start from any value of nonce, then
add one and keep going. It doesn’t matter. It makes no difference where you choose
to start, or how you move from nonce to nonce. However, choosing randomly involves some more
computation. You need to generate randomness; there is no reason to do that, because there is
plenty of randomness in cryptographic hashes. Instead, just increment them by one.
That is the fastest way to do it. Incrementing by one in binary is something
that computers can do very quickly. Just pick a starting point for your nonce,
increment by one, and you have the same chance… of finding a valid nonce as
everybody else on the network. Here is an interesting question. This is difficult
to understand, but it is a very important concept, on how pools work with lower difficulty. “How does a pool work with a lower
difficulty level for its nodes to solve?” “If my node finds a nonce that produces a hash that
is lower than the expected difficulty set by the pool, how does that relate to the difficulty
set on the Bitcoin network?” This is a really great question and needs
more discussion to answer, so you can… grasp what is happening, but it is a neat trick. Let’s say you have a room full of
people who each hold two dice. You have a team working together, with the
goal to throw dice below a certain number. If they [throw] below that certain number, they win. [Think of] that low number for the game
the difficulty target for the blockchain. The players can throw dice between two and twelve;
you can get the lowest (two 1s) or the highest (two 6s). We win if we roll less than five total between
the two dice, so rolling a six won’t work. Six and five doesn’t work, five and five doesn’t work, five
and four doesn’t work, four and three doesn’t work, etc. Two and two works, one and two works,
one and one works. Any of those throws. Below five is the target for the whole game. Your team starts throwing dice, and you want to
reward the people who are doing the most throwing, because they are doing more work for the team. How do you know whether they are throwing dice often
enough? You could just wait until one of them wins. But if you wait until one of them wins and give
them [the whole reward], then it is not really a pool. That is the same as if they
were competing by themselves. [Instead, you create] a pool strategy that gives
a portion of the reward [to each team member], no matter who in the team throws
[the winning dice]; the whole team wins. But you should split those winnings based on how
much everybody contributed to the dice throwing. How would you do that? You set a difficulty target.
Every time someone throws dice less than ten, they will [receive] one share of the team’s contribution. So the team members throw dice.
Twelve, eleven, and ten don’t count. Nine is below the team’s difficulty,
though it is not below the game’s difficulty. Because I threw a nine, I count one share and keep
throwing, because the team difficulty is below ten. That will give us a pretty good metric
as to who has done the most work. They will tend to throw more dice that is nine
and below, so they will earn more shares. We will allocate shares based on throwing low numbers,
but the probability [of hitting the target] is still the same. A winning dice throw below five is also below nine. The chances of throwing below nine
are [greater] than throwing below five. They are not losing anything by announcing [scores].
“We didn’t win the game, but I did win the pool [target].” That [strategy] will still help you get closer to winning
the game, and it gives you a way to track performance. By setting these two difficulty levels,
[the network difficulty versus the pool difficulty], you have the [target] at which we win the game,
and the target at which you are recognized… as a team member for your work,
though that is a less difficult throw. Everybody will start throwing their dice, trying to win the
game as a pool, and then allocating winnings correctly… by counting the shares [for each team member].
That is how the pool difficulty works. I hope that was understood. “If I have an ASIC miner and I [win the block] reward,
how will it be technically linked to a wallet of mine?” “Can you explain the process I should follow?
That is a great question, Scott. How does that work know who or
where to pay when you win a block? It is quite simple, when you think about it.
When you you construct a candidate block, the block you try to find proof-of-work for,
that candidate block is made by your own computer. Your computer, connected to the ASIC miner, is
collecting transactions and blocks from the network. It will construct a candidate block by taking
a whole bunch of unconfirmed transactions, stuff them into a block, and then propose it
as a new block to the rest of the network. You won’t show this [candidate block]
to anyone until you have proof-of-work, because no one is interested in
a block that doesn’t have proof-of-work. Within that block, there is a special transaction called
the coinbase, which pays out the mining reward. Who creates that transaction?
Well, your computer creates it. When you are a miner, your computer creates
the candidate block and the coinbase transaction, which is currently a reward of 12.5 bitcoin
plus fees from all the transactions in there. Therefore, you decide where that transaction pays to.
Your computer [will insert] your destination address… into that candidate block while you are mining. If you are part of a pool, the pool operator does that. If you are mining by yourself, which you would
only do if you have a lot of mining equipment, your computers will construct candidate
blocks with a coinbase transaction that pays you. You write that transaction. Effectively, miners are
writing cheques to themselves to collect the reward. A key part to understand: that cheque isn’t valid
unless the block is valid and there is proof-of-work. When a miner [produces] a proof-of-work solution
for a block, then they write a cheque to themselves… for 12.5 bitcoin plus fees in the coinbase,
which is a valid transaction. If that block is accepted by the rest of the network,
it will show clearly in the blockchain that this miner… is being paid their reward. Of course, the reward needs to be the
correct amount, the block subsidy plus fees. Everything in the block must be valid.
Otherwise, it will rejected. If it is valid, there you go. The miner paid themselves [in the coinbase], [included] it
in the candidate block, found the proof-of-work, and then telling everybody else so that
block [will be included] in the blockchain. One hundred blocks later, the miner can spend
[the reward from] that coinbase transaction. Therre is a one hundred block [waiting period] for
coinbase transactions, where they can’t be spent… until a hundred blocks have passed; in other words,
coinbase transactions need one hundred confirmations. This rule is to prevent miners from using chain
reorganizations to double-spend their own reward. “When a miner creates a block, am I right that this
block is transmitted to all other nodes on the network, available for any miner to validate?” Well, not exactly. It is transmitted, but
only after the proof-of-work is found. A miner only transmits a block once that miner has
already proven it by finding a nonce for proof-of-work… within the difficulty target. The miner must [generate] quadrillions
of hashes in order to find a suitable nonce. Only once that block contains a correct proof-of-work
can the miner send it to the rest of the network. When the miner [broadcasts] that block to the first node,
the node needs to validate the block’s proof-of-work. If it doesn’t have proof-of-work, the node will
reject it and even disconnect from that miner. Every node on the network validates
every transaction and block; miners cannot propagate blocks
until those blocks are valid. They must have valid proof-of-work.

19 thoughts on “Bitcoin Q&A: Iterating nonces and the block reward”

  1. Never heard you speak about BCH vs That SEGWIT COIN called BTC… Are you too, controlled opposition? Me thinks so 🖕

  2. Okay, my comment is wired but: i used to listen to hours of your MOOCs and other lectrues when they just started and (re)play Morrowind while at it.
    I've listened to you talking about nonces, hashes and other topics a million times, but in these vids i see you sitting in a room with a very Morrowind-esque art and interior design behind you, which MY GOSH feels so spot on!

  3. Mr Andreas i have been trying to reach you trough twitter but you have so many followers there is no hope in it. Please could you overview technology behind Bitfi DMA-2 wallet ? is it from technical perspective possible that private key never leave device after its generated for fraction of second ? they claim it only sign up the transaction and device never store any data itself. it has to be tested but please give me your opinion if this technology can actually work if its proven by hackers. thank you !

  4. Both adding numbers and shifting/shuffling bits are linear operations, without enough non-linear substitution ciphers it would be trivial to do cryptanalysis on the function (just look at Iota 🙂 ) and be able to reverse it.

  5. Excellent explanations as always — Its great to hear such a straightforward discussion on these details. Thanks Andreas

  6. 18:06 The chances, for a single throw of the dice, of throwing below 9 are NOT the same as the chances of throwing below 5. That is the whole point of having a pool difficulty lower than the network difficulty. Andreas clearly knows this and just misspoke, but it might confuse people. Part of the genius, possibly unintended, of proof of work is this ability to create a mining pool that does not trust any of its workers to tell the truth. If a miner is mining in a pool, when the miner finds a nonce that produces a hash below the pool target, the miner submits that nonce to the pool. The pool verifies that in fact that nonce produces a hash below the pool target when used in the candidate block header, and credits the miner accordingly. Essentially, the miners are producing "proof of pool work" that they submit to the pool, and eventually one of those "proof of pool work" submissions is also good enough to be a "proof of work" for the larger network. I think this was incredibly important in terms of helping bitcoin become established, and it is a critical reason why mining is not as centralized as many think.

  7. Some watching this might wonder:
    When mining as part of a pool, if a participant finds a nonce that meets the difficulty target, why not just keep the whole reward themselves?

    This is is not possible because the block was constructed by the pool itself and has the pool's address in the (payout) coinbase transaction. By the time a miner finds a nonce, it is too late to change the coinbase transaction as that would change the hashed data and make the nonce invalid. Nice, eh?

Leave a Reply

Your email address will not be published. Required fields are marked *