10. PoW Recap, Other Fork Types


The following content is
provided under a Creative Commons license. Your support will help
MIT OpenCourseWare continue to offer high-quality
educational resources for free. To make a donation, or to
view additional materials from hundreds of MIT courses,
visit MIT OpenCourseWare at ocw.mit.edu. PROFESSOR: OK. So yeah, problem set 2. There’s a lot of work done. Congratulations,
all the workers. 16 trillion hashes performed. How can we prove that? So this is a personal
gripe that I hear a lot. That people say that proof
of work doesn’t scale. And that really bugs
me because sometimes I think I sort of know what
they’re talking about, and they mean like, bitcoin
doesn’t scale, or like, these block chains have
poor scalability properties, which sure, sure. They definitely do. But proof of work,
it scales perfectly, in a theoretical sense. There’s nothing that
can scale better. You can prove an arbitrary
amount of work in 0 of 1. Right? So in this case, well, how
big were these headers? They were less than
100 bytes, right? 100 characters. And with that space,
you can prove actually the entire work that happened
over the entire problem set. So yeah. block chains, whole bunch
of scalability problems. There’s complex systems,
all sorts of scaling issues. But proven work itself, in this
pure form, scales really great. OK. So question. Not super intuitive,
but how do you prove all the work ever done
throughout the entire problem set in one line? Does anyone have any
intuition about this how do you prove all the work
from all 1,800 blocks with just one piece of data? AUDIENCE: Well you know that
for each block, 2 to 33, work had to go into it. So you just need
to know the number of blocks produced times the– PROFESSOR: OK. Yeah, so the thing is how do
I prove the number of blocks without showing
all of them, right? So OK, it’s a weird
trick question. Andrew– I think– I remember Andrew Miller, who’s
now a professor at somewhere, Cornell? Who was not, during the time,
wrote about this initially in the bitcoin forums. What you do is you just
show the luckiest block, and I have not yet
defined luckiest block. But in this case, it
was mined by turtle. This is the block, the previous
block reference was of 0065a2 turtle 1654244 and the hash of
that block is 000c49a414 blah, blah, blah. So anything interesting or novel
about this particular block that you can see? It’s not– AUDIENCE: [INAUDIBLE] PROFESSOR: What? AUDIENCE: It’s better than… PROFESSOR: It’s better. There’s more work. So we didn’t count the things
as having more or less work just by a sum number of zeros. We just looked at the threshold,
did it have enough 0 bits and accepted or rejected. But in this case,
these 4 bytes, right? You needed 4 bytes
and then 1 extra bit. You needed 33 bits. So these 4 are always worth 0. But then in c and red,
there’s another extra byte that’s all 0s. And another extra half
a byte that’s all 0s. And then a c means
for that nibble, that the highest
bit was 1, right? So you’ve got this red stuff. There’s a byte and a half
extra– or almost a byte and a half extra. So, yeah. So what you can do for
a compact proof work. So if you look at this again,
there’s 4 green bytes, byte and a half is red, right? So that’s 5 and 1/2
bytes, so 44 bits. And 2 to the 44 is
17 trillion, right, if you do out 17 something,
which is what we expect, that’s our proof, right? We did 16 trillion hashes
for our calculation, and it shows up here. And another way to look
at it is we needed 33 bits for a valid block. We have 44 bits here. That’s 11 bits extra. That’s 11 bits of being lucky. And so that’s 11 bits
to the 11, that’s 2,048, which is pretty
close to the 1,862 that we actually
performed, right? So in this case,
we’re a little lucky. We might have only
had 2 of the 10 and then we could
only prove that we’d done like 8 trillion work
instead of 16 trillion work. So there’s probabilities
here, but this is an interesting property
that actually does work. You can prove all the work,
to some approximation usually within a factor of 2,
that the system has ever done just with 1 block header. So, yeah. This is fun because
another way to look at it is that you’ve
got this metagame, where for every block found,
take the, I’m doing a hash and I need to find a
hash with a lot of 0s to prove that I’ve done work. And you go a level deeper and
say, OK, I’m finding a block. And I want to prove that I
found an even better block than everyone else, right? The entry for admission
here is find a valid block. And then from those
blocks, since it’s a uniform distribution
with 1s and 0s, you’re going to have this
tail-end of like, happened to have lots of 0s that you
didn’t need in the beginning. And that can prove all
the work ever done. There’s a really interesting
paper called, HyperLogLog, that uses this for
non-bitcoin applications that uses it for set counting. Where, like on a website,
you want to see how many unique visitors you’ve
gotten or something. And you can store
that in 0 of 1 space because you could
keep track of OK, let me keep track of every IP
address that’s ever visited or something like
that, or cookie. But instead, you hash them. See if the hash starts
with a bunch of 0s or any arbitrary character,
and then just store the lowest. And then, every time someone
visits, hash it, compare. If it’s lower, replace. If not, ignore. And then you have a
very compact indication of how many visitors
have visited. Anyway, that’s like a super
high-level view of it. But if you’re interested in this
stuff, HyperLogLog is a paper. It builds off of
some other things. It has nothing to
do with bitcoin, other than this property, where
you’ve got this random function and you see how
many 0s are in it. But I think these are cool,
and so to me, this is a fun– this is not used
in bitcoin, right? In bitcoin, you actually
download all the headers, but people have written papers
about how you could use it. If long term the headers
are big, you could prove. Have some checkpoint,
where look, I proved all the previous work
and now I build from there. Any questions about this idea? Yes? AUDIENCE: It’s not
really a proof. It’s just a
probability weighted. PROFESSOR: Yes, but the proof
of work itself is the same. Not of real proof because
you might have gotten lucky. So finding a block, it’s
like, well that’s not a proof. There could be luck involved,
there could be probability, but it’s the exact same
luck and probability that the underlying
proof of work uses. So there’s no further reduction
in security or certainty because of this. Not really. And so I remember talking
about this with someone a few years ago and saying, yeah
proof of work is a misnomer. It’s not really a proof, right? Maybe it’s an argument of work
or some probabilistic argument of work. How could you make it
more certain as a proof? There’s a bunch of ways. One way would be to have
multiple nonces, where instead of just finding one
nonce that satisfies it, you have to find several
replaceable nonces and then iterate through them. That would be a much
more certain proof. It would remove the idea of
probability, to some extent. It would also completely
break the system in a way that’s like
fairly unintuitive, and so I always was
sort of joking like, I should make an altcoin, where
you’ve got multiple nonces, and you could be like, yes, for
more security and then people would buy it, but it
completely breaks. The completely broken incentives
and system, maybe I’ll get to– I’ll let you guys think
about it til Wednesday, and then draw it out and be
like, why wouldn’t this work? It’s fun. It breaks in subtle
but bad ways. This is talking about
proof of work optimization. So if you look at
this slide anyway, you’ve got these
headers or blocks or whatever we’re mining. So you’ve got kezike17,
tomriddle, Thalita, all these people mining. It’s interesting to
see what people use. Some people use
looks like base64, some people just use
a decimal number, all sorts of different things. Who knows. There was also a lot
of invalid blocks with valid work
submitted, where there was four or five different
things in my spaces. So there’s been count,
but actually a lot more work was done,
and that’s not even counting the human work of
doing all these assignments. So sending this over the
wire, or storing it on disk, some inefficiencies
may jump out at you. What do you think you could
do to make this more efficient or compress it or? Yeah? AUDIENCE: [INAUDIBLE] PROFESSOR: OK. That’s an easy one. Oh, this doesn’t
work when I gave you the slides because you can
just look at the next slide. Whoops, OK. Never mind. Yeah, so the first 8
characters are always going to be 0s by definition. If it’s not, it’s
an invalid block so don’t even bother sending it. So the first characters are 0,
don’t send them over the wire. Just have that implied and just
start at the ninth character. That saves, well, really you’d
have the serializing binary, so it only saves 4 bytes. In our case, it
would say 8 bytes. That’s cool. And then, also the
entire previous hash. When you’re sending a list of
5 here, like here, in order for it to be valid,
this has to be the hash of the line above it. So just don’t send the
whole thing, right? Just send name nonce. The hash is also computable
from anyone receiving it. That takes almost
all of this space and we could compress
this entire blockchain to just the first
line, and that’s the only full line we need. After that, it’s just
named nonce named nonce, and we get rid of 70-something
percent of this base. That’s pretty cool, right? Yeah this kind of
header optimization is also very much
possible in bitcoin, and it’s not implemented. It’s not done. If you want to, you could
program it, change bitcoin, make a pull request. I think– I mean,
we’ve discussed it, and people are like, yeah that’d
be cool but, no one’s done it. So if you want to leave
your mark and be like, I’m a bitcoin core
contributor, and I optimized the proof of work propagation
system or something sounds cool, you can do this. It’s not too hard. You got to learn
how bitcoin works and all the different
messages and stuff, so it’s a little annoying. But I think the main reason
people haven’t done is because it’s– this is not the
slow part, right. This is not the critical
path, not a bottleneck in the actual system. Generally, the proof of work
verification is pretty quick. The headers are a total
of 40 something megabytes now, 50 megabytes maybe. So you could you could
definitely reduce that by a significant
extent, but no one’s bothered because there’s
so many other scaling issues that are more pressing. But it’s kind of cool, I think. So, yeah. That’d be a fun thing to do. If you did that would that
be a soft or hard fork? If you said, OK, I’m going to
now send header messages that are truncated. I’m going to leave off the
4 bytes that are always 0s. Bitcoin is also the first– the difficulty requirement
here is basically the same as it is in bitcoin. I’m going to have the
implied previous block hash, things like that. Would that be a fork? Actually, it wouldn’t, right? It’s a non-fork. There are lots of
changes you can make. So I know Neha talked
about soft forks and hard forks changes you can
make in the system that affect consensus, but there’s
a lot of changes you can make that optimize
it that don’t really affect other people. So in this case, it would
just be a wire protocol change and you could easily maintain
backwards compatibility right? So in this case, you say, the
header optimization is not a fork, right. What you do is you’d
have a new message type like, truncated
header or something, and then, when you
connect to nodes you say, hey, you know about this
new message type I’m using? And if they don’t know
what you’re talking about or they usually they
say what version they are when they connect. You’re like, oh
you’re an old version, you don’t know about this. I’ll just keep saying
the old header type. And even if I store the new
truncated headers on disk, I can recreate the
old one pretty quickly by performing the hashtag and
then on sending it to you. So I can be backwards compatible
and forwards compatible. No soft forks needed. The old nodes, they don’t
even see that this happens. They might see that
there’s oh, there’s a new version or a new
message I’m not aware of. They ignore it. Everything seems fine. So these are the easiest– they’re not forced– the
easiest changes in the system get through because there’s
no real coordination needed and it’s backwards and forwards
compatible, so that’s cool. So some example non-forks. A lot of them are internal only. You can’t even see from outside. So for example,
compressing blocks or compressing your database. That’s fairly
straightforward, right? Intuitively it seems
like, well, these are all random numbers and hashes. You can’t really compress
those because they’re random. In practice, you actually can. People reuse public keys
a lot, and so you just see the same pub
key over and over. So you do some pretty
simple encoding and you can make those smaller. Also, the amounts is 8 bytes. So if you’re sending someone
one bitcoin, that’s 100 million. And people like to
use round numbers, and so those get
compressed pretty well. And generally,
they’re much smaller. So the top bytes are usually 0s. So you can compress
it a decent amount. But no one has to know that
you’re compressing, right? That’s all transparent. When someone
connects to you, they have no idea if you’re
compressing or not on disk. Something like faster
signature verification, where there’s been
enormous amounts of work in optimizing
the code for that. Making assembly,
stuff like that. Nobody knows you’re doing
it, they’re just like, oh, he’s asking for blocks
quicker than this other person. Maybe his network’s faster,
maybe his CPU faster. So these are changes
that are purely internal. Nobody needs to know. That’s cool. Other non-forks are
peer-to-peer non-forks. So the truncated headers,
maybe, where you can say, hey, I’m going to send you
less data over the network. You identify at
connect time and you default to the old behavior. People don’t know what
you’re talking about. So there’s one called
compact blocks. I didn’t describe it,
but you can probably guess what the idea
of compact blocks is. Anyone want to venture
a guess what those do? Block. So it’s not a header that’s
come back, but the whole block. How would you compact a block? AUDIENCE: Get rid of all the
fields that aren’t necessary. Like version… PROFESSOR: Yeah,
actually, that would work. But that’s not what they do. There’s a really
big 2x redundancy. And so the basic idea is
transactions are propagated, and then a block’s propagated. Where’s the redundancy there? AUDIENCE: [INAUDIBLE] PROFESSOR: Yes, the
transactions the block. You’ve Probably already
seen them, right? You see the transactions,
and the block comes out. Most of it, in general,
90-something percent, it’s like, yeah we’re
going to see all this. So compact blocks is a way to
say, hey, here is the block. Here are all the
transactions in it, but I don’t show the
whole transaction. I just show the TXID the hashes. And then you can say, OK. 90% of those I’ve already
seen so we’re good. Here’s these 50
transactions I have not seen, please give them to me. So it’s interactive
here’s the blocks with just the
transaction identifiers. OK, what do you need? OK, I need these 10. OK, here’s the 10, and now I
can reconstruct the whole block. So the block goes from being
a megabyte over the wire to something like 10 kilobytes? But it is a little slower in
that it’s like a multi round thing, right. It’s like, here’s
the compact block, OK, I need these extra things,
OK, here’s the extra things. So a little bit more complexity. If you’re really
optimizing for latency, then you don’t want to use this. But in general, it’s
a pretty big gain in terms of bandwidth, which
can be taxing on full nodes. I run a– there’s a full node
on the first floor in one little rack and it uploads
three terabytes a month or so. Depends on how much
people are using bitcoin. In December, everyone starts
downloading it and installing it, and there’s a
lot of bandwidth needed to sync people up. Another non-fork was
the Bloom filters, which note full nodes
can then say, hey, I will perform Bloom filter
calculations for you. And light nodes can
connect in, like I said two weeks ago with SPV. Light nodes can submit
a Bloom filter say, hey, when I download a block from
you, first, filter the block. Match all the transactions
against this Bloom filter and only send me
things that match. That’s not a fork, but it’s
a fairly involved change in the peer-to-peer code. OK, any questions about
these peer-to-peer non-forks? Cool. There’s another aspect called
standardness, where you haven’t soft forked something
out, you haven’t declared something
invalid, but you can declare it non-standard. And what that means,
is when you’re node sees a transaction
coming over unconfirmed, the transaction being
propagated through the network. And it’s got this
property, and you say, oh, that’s non-standard. I’m going to drop it,
I’m going to ignore it. I won’t propagate
it onto my peers. I won’t ban, I don’t– it depends. I don’t know. Do I ban the person
submitting it to me? I think you don’t,
but I ignore it. I don’t propagate it,
so it doesn’t really get around the network. When most of the
peers on the network have these rules
of non-standardness it’s going to be very difficult
to get your transaction out there. However, if you see this
non-standard transaction in a block, you
accept the block. You say, OK, well that was this
weird thing that I didn’t like, but since it’s in a block
and someone did a lot of work on it, I will accept it. It’s a little weird, right? Why have this? It’s something that’s not
quite a soft fork, right? It’s showing that we’re
discouraging this, we think it’s non-standard. The miners software,
by default, will also consider this non-standard
and not mine it. But if someone else is
mining it, we’re OK with it. And so what you
can do, is you can stage future soft
forks this way, right. So for example, in SegWit,
oh, I didn’t talk about SegWit at all. I’m going to have to do
that next class, next week. OK, so SegWit was the biggest
soft fork ever in bitcoin, and it occurred last year. It changed the output
scripts to say– so before, you said, OP_DUP,
OP_HASH160, the hash, OP_CHECKS, OP_EQUAL, whatever. Here, it just says 0. Just pushes a 0 byte, and then
pubkey hash, and that’s it. And if you actually
interpret that in the stack, no signature is needed, right? You push a 0 to the bottom the
stack, you push a pubkey hash on top of that, and then
your execution halts, and you’re like, well,
there’s a non-zero piece of data on the top. I interpret non-zero data
as true, same way he does, so it’s true. You don’t need a
signature at all. So that’s the weird
SegWit soft fork where they said, no, what
used to be considered true without a signature, we
now template and we say, this means check
pubkey hash, right? This means the same as OP_DUP,
OP_HASH160, hash, OP_EQUALS, OP_CHECKS, OP_CHECKSIG, right. So what you actually
do, is you need to provide the pubkey
that this hashes into, and then check a signature. It also defined
1, 2, 3, up to 16, and left this undefined,
and said, look, these are now non-standard. Before, if you– I think they
were already non-standard, but the idea is that if you
just push a 1 on the stack, and then push some data, well,
I guess no signatures needed. But now they’re non-standard
because it means we’re going to use these next. The next soft work will define
what one, some piece of data means. Maybe it’s a new
signature scheme, maybe it’s a new program
where you put some data here, but it’s non-standard. So if you try to make
a transaction that’s using 2 and then a data
push, all the nodes will be like, yeah,
I’m not ready for that. I haven’t I haven’t seen that. And if you see, I
think in your air logs, if you see a block with a
bunch of these kinds of things, it’ll give you a warning. It’s like, warning. People are using stuff that your
software doesn’t know about. You might need to upgrade. There’s a bunch of warnings
like that where like, warning some percentage
of the last few blocks had these things, so people
are doing stuff that you’re not considering invalid, right? You’re not going to refuse the
block, but you’re also like, this is something
I don’t understand and I’ve specifically
coded it as nonstandard. OK so any questions
about non-standardness? Neha talked about soft
forks and hard forks, and I will go through
a bit more detail about how these end up
working and how these interact with miners and notes. Did people have questions
about soft forks and hard forks before we start? Sort of got the
general idea, right? Soft forks add new rules, hard
forks remove rules, in general. And this is minors. So the miners have
a unique role here. It’s not just the
same as a full node. A miner decides what to put into
a block that they’re mining. And so they do have
a bit more influence in this fork decisions. OK, so a soft forks would
be, for example, saying, OK, all output amounts
must be odd, right. You can’t send an even number
of coins to anyone anymore. That would be a
weird, silly fork. Wouldn’t really impact
the usability system, but it’d be dumb,
but you could do it. And you could say,
OK, well, if I see a block, if I see a
transaction, which outputs, if any of the outputs have
an even number of Satoshis, invalid. You’ve got to do odd. Potentially leading to the loss
of 1 Satoshi per transaction– with the fees, 1
Satoshi per block may end up being
lost due to this. So here I’m saying, an A for
adopter and I for ignorer. Now people may ignore the fork
because they disagree with it. They don’t want to
do this fork, or they may do it because they
may just not even know that this software exists. It’s a giant
decentralized system, and it’s hard to know
how to communicate with everyone, right? There is bitcoin.org. There’s also bitcoin.com,
where the guy doesn’t like the
bitcoin developers, and says they’re bad. Anyone can just
register these things. There’s a bitcoin
Twitter account that was purchased recently
by someone who wanted to argue about these things. So there’s no one really
in charge of this. And then there’s also
different implementations. There is the real
bitcoin, which is run by this bunch
of crazy people who say they control
bitcoin, and that everyone has to pay them taxes
in bitcoins, yeah. But they’re all
running bitcoin– they all are in consensus
and doing these transactions. So ignoring could be
any number of things. If you have a soft
fork, where you say, OK, we’re now adding this
rule, but none of the miners are enforcing it. None of the miners even
know about it, potentially. Here, it just stops, right? You say, no, I require that
all output amounts are odd. And then every block
has these even amounts. And you’re just
like, OK, no that’s not a valid block,
that’s not a valid block, you will never see a
valid block again, right? None of the miners are enforcing
this rule, but you are. Everyone’s ignoring it, and they
say, everyone’s ignoring it. Everything seems fine. You just self-imposed
this new rule, making you incompatible with
the rest of the network, and from your perspective,
everything stops and no more blocks occur and
the system is over. Or potentially, if you’re
soft fork is some weird rule that nobody knows about
and nobody breaks anyway, we say, OK, the sum of
the outputs of all– the sum of the outputs
in a transaction must not be a Carmichael number. OK, you could have that rule. Probably no one’s break– wait. There’s a lot of small– something like that, right? Where no one’s
breaking it anyway. Then, from your
perspective, everything’s cool because everyone’s
already obeying your rule even though they
don’t know about it, it’s silly. Another possibility is let’s say
a minority, somewhere 1 to 50% of the miners adopt
this rule and say, yeah, we’re going to enforce
this new rule, right? All output amounts need to be
odd, so the idea is you say, yes, only odd numbers. And a bunch of the majority,
actually, of people don’t care about odd or even. So the majority, they still
go off on their own chain, but you split off
into your own faction, you say, no we’re the odd bunch. And both of those
chains are viable. Blocks come out here
maybe quite slowly, if it’s only a few percent. Blocks still come out here. The fact, so you’ve
got these odd thing, and then you’ve got regular. And the regular is going
to be longer, right? The regular is going
to be potentially a lot longer, but
from the people here, they’re like, we don’t
care if it’s longer. It’s wrong. They use even numbers,
that’s just plain, old wrong. And these people are like,
yeah we can sometimes see it, but actually we lose
track of it very quickly. After here, we start seeing
block advertisements like this, and we’re like,
we’re over here now. We’re way past that. Why are you talking about this
stuff from like weeks ago? So they just disconnect. It’s pretty ugly,
but that can happen. Now, if you have the
majority of the hash power, this ends up being longer, the
odd blocks end up being longer, and everyone gets
dragged along, right? No split, and now
we have a new rule even though they didn’t know
about the rule potentially. So they’re like, what the
heck, half my transactions don’t work. Some of them do,
Some of them don’t, I don’t know what’s going on. I just randomly adjust my
fees until it seems to work, and then my
transactions go through. They should probably find
out from someone, oh, yeah, there’s a new rule. Only odd numbers,
and then they– that rule is imposed on
them from the miners, essentially, in the
rest of the network. And essentially,
the same thing here. When you get to 100%, there’s
none of these orphans, but it worked– oh, sorry–
these orphans would actually be like this because everyone
agrees that this is valid, and then some of the people
aren’t aware of the rule and keep mining
off of these what they consider valid blocks. So it’s a little
different topology. OK, so that makes sense, right? Any questions about soft
fork, mining power rules? One other aspect, is if you
split here, and then later on you get a majority and you
pull ahead, you will reorg out the ignoring side, so we split
off with 10% of the hash power. We’ve got our much
shorter chain, where we only have odd numbers. At some point, we convinced
the rest of the miners, that, no, this is the way to go. The even numbers are really
screwing up the system. And we get the majority
of the hash power, and then we overtake the
even and odd mix chain. The people who have not
yet updated their software, and are ignoring
the fork, they will reorg out because from
their perspective, OK, I was on a longer chain
now there’s this other longer chain. They both look valid, and
when I see two valid chains, my way to decide is
who has the most work? And so this one pulled ahead,
in terms of work, so I switched. So it’s a weird– this
has never really happened, that I’m aware of,
they were threatening to do it last summer
[INAUDIBLE] I don’t know. So yeah there is
all sorts of stuff on the internet, and Reddit,
and Twitter about doing this with a minority of hash power. They didn’t though, or they did. They say they did, but everyone
else says they didn’t though. A lot of arguing. OK. So that’s another
weird aspect of it. OK, hard forks. No minor support. What happens to
those adopting it? Nothing, right? Everything just keeps working. If you say, OK, we’re now going
to allow every transaction output. Every transaction can have 1
extra Satoshi gets created. It’s just 1, it’s no big deal. It’s quite limited, but we
want to compensate people for using bitcoin, so
when you have your inputs, you have your outputs,
you can add 1 Satoshi. You get a free Satoshi
per transaction. The previous
software, absolutely does not allow that, right? If you’re just
generating money out of nowhere in these
transactions, not OK. But these guys, they’ll see that
the transactions they do that with are not confirming,
but otherwise, they’re OK if you don’t add a Satoshi. And so the system works. Nothing happens,
nothing happens. They just see everything. With a minority of the hash
power, something like 10%, 20%, you get all these orphans,
get all these dead ends, where you see a
block, OK, and it’s got this 1 Satoshi
per transaction bonus. Great, but it keeps
getting orphaned out because you still consider– all right, so you see OK,
here’s this longest chain without the bonus. And then you say, oh, here’s
a block with the bonus, cool. Maybe someone builds 2, great. But this keeps getting
longer and you keep trying, but you keep getting overpowered
because you see both of them as valid. You’re not requiring
that there’s this 1 Satoshi bonus
per transaction, you’re just allowing it. And so you say,
oh, this is cool. Cool, oh, no. Got reorged out,
got reorged out. So you see all these little
starts, they get reorged out. And you basically stay
with the same chain. You don’t split. These people also see a bunch
of invalid blocks, right? You’ll see it on
the network, hey, someone keeps sending these
invalid blocks much more frequently than usually. I don’t know why
they’re doing that, but they’ve got invalid
transactions, I ignore them. Here, majority of
hash power is split, so once the majority and these
are the bonus, the plus ones, they pull out ahead. These guys don’t actually care
that they have a majority. After the first block,
they don’t see the rest because they ban. So if someone submits to you
an invalid block, you ban them. You ban their IP address
for 24 hours or something. You’re like, I don’t know what
you’re doing, you’re crazy. Disconnect. So you won’t really see
this pretty quickly. These guys don’t have the bonus. They’re still on their
same old blockchain. It may be much slower because
a lot of the harsh power now moved to this other
chain, and these guys say, oh, it worked, cool. We’ve got our new
bonus coin chain. Now we’re stimulating the
economy, everything like that. Job creation. OK, so and then
these guys slowers. Slows down. And then if you have
100%, well stops. For the non-adopters,
no more blocks come out. This is the end. Everyone’s gone to the
job creation train. And there’s not really
a split anymore, it’s just the new rule. OK. So any questions there? This grid so far? Then another way you
can do it is combine this to say OK we’re going to
do a soft fork and a hard fork at the same time, and
actually, many times that people say hard fork,
they actually mean this. So the nomenclature
is pretty ambiguous. I like keeping these terms
very distinct and pure, so like a soft fork is purely
increasing the number of rules, where it must be odd, and a hard
fork is just reducing rules. And yes, we will allow but
not require a transaction to have this property. And then to combine them would
be something like saying, we allow this new thing
that was not allowed before and we require it to be true. So for example,
every transaction must introduce 1
new Satoshi bonus. That would be both
hard and soft work because now if
you’re doing this, you no longer consider
the old rules appropriate. And there’s a complete mutual
disagreement on the rules. Some people have called this
full fork, who called it that? I forget. Greg called it a
bilateral hard fork. We don’t have good
terms for these things. A lot of people refer to forks
that have both as hard forks, so it’s somewhat ambiguous. I think it helps to keep
these different terms, but it’s a different setup. It’s the union of these two
things in some way in that, we allow this new thing
that was prohibited before, and not only that,
but we require it. So if 0% enforce the new
hard fork, well the adopters, it just stops, right? There requiring this new thing,
it’s not showing up, it ends. These guys nothing
happens, right? When there’s a minority,
it will split off. It’ll split off with the new
rule set and the new bonus coins or whatever. And the ignorers, they
see that it’s slower because some people have
left some mining powers left. When you have a majority,
you also split off. And ignorers, again, it’s slow. They’re not going to
adopt because they see the new fork is invalid. In this case, as
well, these guys won’t adopt here because
they see it as invalid. And then for the full thing– for the 100%, the
adopters, new rule, and these guys system halts. OK so any questions about
full fork or bilateral fork, or whatever you want to call it. Hey, it works. OK, cool. Yes. Bilateral, hard, full, we don’t
know the good names for these. But yeah. It’s essentially a soft fork and
a hard fork coupled together. And this is much simpler
or easier to produce, and if you just start
changing the code, you’re probably going to
create one of these, right? You have to be very careful. If I want just a
hard fork, I’m very careful that everything that
used to be valid is still valid and I’m just rescinding one
rule or one set of rules. It’s very easy to
inadvertently create this when you’re trying to
make consensus changes. Yeah, there have been– did Neha talk about
the 2013 fork? I don’t think so. OK, so a little anecdote. I remember I was in the airport. I got to Nagoya
airport and opened my laptop and bitcoin went
down to like $20 from $30, and it was all over the
internet like, oh, no. And there was a inadvertent
hard fork due to the Berkeley DB to level DB transition
in the software. AUDIENCE: [INAUDIBLE] PROFESSOR: No, no, no. It was 0.7 was
using Berkeley– it used to be everything was using
Berkeley DB for the UTXO set in the blocks. And then they
switched to level DB, and then, it was OK
for a month or two, and then someone wanted
like a block that had a bunch of inputs or something. I don’t remember
exactly the reason. And the new software was
like, yeah, that’s cool, and the old software
said it wasn’t OK. The thing is it wasn’t
clear why, right? There was no defined like,
this should be invalid, like, this looks valid. But the Berkeley DB
layer gave an error. And so it’s like,
well, the database says it’s bad, so
it’s not a good block. So that was a weird
unintentional consensus change. What happened was– so it
seemed like it was a hard fork. The thing is it was like
a compiler time option dependent hard fork,
if you compiled it with a different Berkeley
DB cache setting, then it would work. It was sort of ambiguous and
people rolled back though. They were on IRC and they were
talking to the different miners and are like, what’s going on? There’s two different
forks being built. And they told the people
who were, to some extent, in the right, the new version,
which seemed more correct, to stop mining. And they did, and then the old
version with the Berkeley DB caught up, and then they
restricted their block size. It was something to do with like
having too many file locks open or something like that. So that was essentially
a hard fork. They stopped it, and
then went back so there wasn’t a hard fork, but then
months later they’re like, OK, we’re going
to all transition to level DB because we’re not
even sure what the rules are. Yeah? AUDIENCE: [INAUDIBLE]
the number of blocks update they produce block was
way too many, so level DB was like, OK, [INAUDIBLE] PROFESSOR: Yeah. And the thing is,
they definitely did– at the time, people were running
around screaming like, why– after the fact, they’re
like, that’s why. But at the time, people
were talking really quick and like, what’s going on? Bitcoin has failed,
sell all your bitcoins, the system doesn’t work
because no one really knew what was going on or why
some versions weren’t working. So yeah, that was
an unintended fork. It was a little scary,
and then the price went back up after
that, It was like, hey, we can work through this guys. So that was a hardcore, right? The old software would
not allow these blocks that opened a bunch
of file locks, and the new software did. And so that was probably
the one real hard fork that bitcoin has been through. There were a few maybe
in 2009 that like are fairly ambiguous because
there were no there was no actual split. I think there’s also a
hard fork that’s happened, but it didn’t, it’s
a little weird. It has to do with the
timestamp in the block header and how it like expires
in 2106, once you run out of bits from Unix time. 1970 plus 2 to the 32 seconds
is like 2,106 in January, or something. And so they actually
did a hard fork, but the thing is the hard
fork wouldn’t diverge until 100 years from now. So it’s like whatever, everyone
will have updated by then. If someone’s still running
software from 2015 in 2106, they will diverge, but
that seems unlikely. So there’s a lot of
weird stuff like that. OK. Firm variance. Some people call it firm fork,
people call it evil fork, I’m [INAUDIBLE] call it
evil and firm, I don’t know. This is a fun one that
has not been attempted, and I remember talking to
people and they’re like, don’t talk about this. The miners don’t know they
can do this, so just shh. I’m like, come on, they don’t
know they could do this? That’s kind of like– OK. So how would you do this? Make a soft fork. It’s a hard fork, right? It completely changes
the rules, however, it looks like a soft fork
to non-adopting notes. It’s kind of crazy. Well, the proof of
work for the new chain is a empty but valid
block on the old chain. So instead of your
proof of work being, hey, have a header
that hashes to this, say, have a header
that hashes this. Also a block that is valid
but completely empty. We’ll take that as
our header, right? Our header now gets bigger. Instead of 80 bytes, it’s going
to be 200 something bytes, but that’s doable. Now our header chain is
a chain of empty blocks. And our actual blocks
point to that, right. We can put our Merkle
root in the output address or something. We can put our Merkle root
somewhere in this empty block transaction– the one
transaction in the block. The old nodes will see it
and say, yep, that’s a block. Someone’s mining, here’s
where the money went. But there’s no
transactions in it. My transactions never confirm. You don’t actually have to
connect to the old network to do this, it’s
totally deterministic. You just say, OK,
my new proof of work is a valid but empty
block on the old one, and I commit somewhere
in this to my new block. That’s evil because what happens
is, here’s the chart for this. It’s basically a firm fork
plus this little evil thing. The adopting, if you have no
hash firewall system holds, then nothing changes here. If you have 1 to
50%, the adopting split off with the new
rule, so in this case, it’s like a hard fork. However, the main
difference from a hard fork, if you have majority
hash power, the ignoring the system essentially halts. It doesn’t halt in that the
blocks keep coming out, right. You’ll still see your software
won’t give you any warnings. It’ll just be like,
yep, block height keeps progressing as
normal, as expected. We keep selling all
these blocks, however, no transactions occur. And you can never
receive or send money, but according to your software,
everything’s working fine. So this is the firm, evil,
sneaky, whatever part, where if you’re able to
get 51% of the mining and you implement this
new fork, you’re basically forcing everyone to update
because if you don’t update your software, if you’re
ignoring this fork, you can’t do anything. You’ve got to adopt a new rule. So this is scary. And I can see why
some people are like, don’t tell miners
they can do this. Also I remember last year
with SegWit2x stuff, I think, they were arguing about this. And it really seemed
that they were not aware of this possibility,
which was like, huh, they don’t know about this. Cool, I guess that helps keep
things safer because they were arguing about how they were
going to rent like hundreds of millions of dollars of hash
power to mine empty blocks on the old chain. It’s like, you know you can
do that for free if you just change your software to
make your new proof of work and empty block on
the old proof work, but I don’t think they
were aware of that, so we’re like, OK let them go. Anyway, and so the thing is it
you can see how it would really quickly turn into that, right? If You’ve got 75%,
well why would anyone try to mine on this? You could keep making blocks
that did contain transactions, but they would get orphaned out. And everyone would, you
might occasionally see, hey, a block came out with
transactions in it, I just got reorged out, and
nothing would ever come of it. So you can see how the
miners can never really get paid on the minority
fork and minority chain, so they’re all going to start
switching to the new thing, if they want to get paid. So that’s a little ugly. This has not happened yet. Hopefully, this doesn’t
happen, but on the other hand, despite it being called
evil, I know Luke Jr, he’s kind of crazy,
but he’s cool. He was saying this is how we
should do a hard fork, right? We should also give
people the option to say, look, there’s
going to be a fork. We give you the option to adopt
a different system or doing this so that no one’s
inadvertently left behind. So we say, hey, a year from
now, this is going to happen. You can either say we
actively refuse this new rule and we’ve got our own chain now. We make a soft fork
before that point, or we adopt this new
rule, update our software, and now I’ve got this
new proof of work. The thing is, the
new proof of work, you don’t need new chips, right? You just need to do a little
bit different software. OK any questions
about evil forks? Kind of fun. Yeah? AUDIENCE: [INAUDIBLE]
an empty block? PROFESSOR: Well, OK every block
has to have one transaction, so you’d have the
coinbase transaction, but any user created
transactions would not be. And, yeah, in the case where
there’s only one transaction the Merkle root just
becomes the TXID of that single transaction,
and you can put arbitrary data in that single transaction. The input field– yeah
we said– the input field for that coinbase transaction
that generates new coins can be any arbitrary data you want. So you could put a Merkle root
from your real block in there, and then write your new software
and say, OK, the new proof of work is the header
with the nonce, also it’s got to have a
coinbase transaction, and then a real Merkle root in
the coinbase transaction, and then build out
your treat from there. So it’s a little ugly. It’s a little more complex,
it would totally work though. And all the old
software would just be like, huh, no transactions. And all the new software knows
that that’s not a real coinbase transaction. It’s just a part of
the extended header. OK. Cool. Don’t try– well, I mean, try
this at home, if you want. I don’t know. Yeah, seems coercive,
thus evil, people call it. OK, yeah, fork coordination. How do you go a
level up from this. How do we know to
do all these things? Reddit, IRC, Twitter, there’s– these systems exist on the
real world and people talk. Like the meeting I
was at last week. We actually didn’t
talk about forks much, we sort of did, but the
developers get together, companies using bitcoin,
all sorts of stuff. Yeah, no, on
Wednesday, people were arguing about MAST
versus Schnorr which is more important,
which we should try to soft fork in first. Not much gets done. So it used to be called BIP9. That’s still there. Bitcoin Improvement Protocol 9. It was the idea in the header
field in the version field, you can set these flag
bits for which soft forks you are adopting,
right so you indicate before adopting a fork which
one you’re ready to adopt. And then you don’t actually
implement the adoption until you see some threshold
to say, OK, once 95% of people are signaling that we
have this new operation, we’ll all enforce it. Because quite likely
the software people don’t want this. A lot of times you say,
look, I want this new rule. I want this new
signature system or I want this new even, odd thing. It would be really better if
everyone only used on numbers. However, I’m not
willing to split off because of this, right. I want everyone on board, or
at least the majority on board. So we have a new split. We get the new one,
this is what I want. I like the rule but
I’m not willing to put a stake in the ground say,
look, I’m making my own network if you don’t like it. So in order to do
that, we say we want to get a majority
of mining power to adopt it before
we start doing it. And so that way we can
signal in the header that, hey, I’m aware
of this new rule, and I will enforce it
if everyone else says they’re going to enforce
it so a staging process. So that was called
BIT9, and the idea is once 95% are signaling
it, then you activate it. This didn’t actually
work in practice. I think it worked once with
the OP_CHECKS sequence verify, and then last year,
people were just arguing in the miners
are like, no we’re not going to activate
any new soft forks, or then, they started making
all these deals– it was a mess. Governance, yeah. OK, so, yeah, the future of soft
forks is definitely unclear. This is very much in flux. How is this going to
work in the future? How are people going to
agree on these things, right? It seems, a lot of the times,
from the developer perspective, it seems like, why not? Like, hey, we made this
cool new signature system. It’s faster, it’s more secure,
it saves space, let’s use it. And then people say, no
and you’re like, well, why? Things like that. But on the other hand,
if it’s like, no. We only have odd outputs. It’s like, why? Who cares. Let’s use, even and odd numbers. OK so another aspect
with the forks. Transaction replay. So this is tricky. So a split happens. Let’s say in the case
of a minority soft fork, or a majority, but not
unanimous hard fork, or a full fork,
something like that. There’s a split, right? There’s two chains that
are now being extended. You make a transaction
on the old chain what happens on the new chain? Yes? AUDIENCE: [INAUDIBLE] PROFESSOR: Yeah. [INAUDIBLE] it and it
happens on both, right? In many cases,
these transactions are valid on either. And so they can be
rebroadcast or relayed between the two networks. And if it can be relayed between
the two networks, it will. Someone’s going to set
up a little script that grabs all the transactions
on one chain, broadcast them on the other, even if you
don’t want them to someone will do that, right? And if it’s valid on both,
it gets confirmed on both. And so you say,
OK, it now splits. There’s now two
different histories. At the time of the split, now
I’ve got coins on both, right? I don’t usually think of
it when it’s a short term split as I now have two
types of coins, but I do. If these extend indefinitely
and they’re never going to reconverge, well, I’ve
still got the UTXOs on both. I can make a transaction here,
and maybe it gets relayed here and now they move on both sides. Or maybe it doesn’t. So you can you can potentially–
and eventually the UTXO sets will diverge, right. How do you diverge these things? Well, if you mix it with
coins that have been mined, so you know that the
mine, the new coinbase, the new coins here, are
definitely different, right? Those are going to
have different TXIDs. Coins that got mined here will
not exist on this one, and vise versa. So eventually,
more and more coins start getting mixed in with each
other and they will diverge. That takes a while though. Another thing you
can try to doing is a spamming double spends. I’m going to send this– I’ll make a transaction,
Alice pays Bob, I also make a transaction,
Alice pays Carol. I just send one to one
place, one to the other, hope they get in. Eventually, they’ll start
diverging just by chance. You can also try
exploiting locktime deltas. So in many cases, the
heights will be different, and you can say, OK, I’m here. I’m going to make a
transaction that’s only valid after block 5. And if someone replays
it here, they’re going to have to wait 2
blocks before it’s valid. And then this gets
confirmed here, and then I make a different
transaction spending the coins here without a time lock. And so I can try to exploit
the fact that there’s timing differences
between the two chains to make a transaction
A occur on the top one, and transaction B occur
on the bottom one, and split my coins off that way. So those are potential
ways to split your coins, despite these
transaction replays. And then you can now
say, OK, I have two separate wallets, two separate
keys on the different chains. However, yes, this is expensive. This is ugly. It’s possible, but it’s ugly
because you’re basically going to spam the network
and in many cases, you’re not actually
trying to send money, you’re just moving your own
money around internally. So if everyone does
that in the system, it can overload the system,
have tons of transactions, also if you’re doing
this, it might not work. And you’re like,
OK, I just confirmed a transaction for no point
whatsoever, got to keep trying. It’s pretty ugly. Also, people don’t know,
so why is this a problem? Well, in many cases, you want
to sell one and not the other. In addition to these
software rules, such as bonus coins
for each transaction or only odd numbers
allowed, there are often philosophical and cultural rules
that get associated with it, and people hate each other
and yell at each other and insult each other on
the internet all the time. This is– I don’t think
this is unique to bitcoin or cryptocurrencies,
I think it’s just, you got money involved, you
got trolls on the internet, it’s a rich mixture of the
best parts of humanity. So a lot of times people want
to sell one or the other. So they say, I think
the odd coin is stupid. I’m going to sell it, and
someone wants to buy it and I’ll get these new coins. That’s difficult if transaction
replays are occurring. Another problem
is that many users could be unaware of the forks. I know a lot of people– there have been a bunch of
full forks in bitcoin recently, where different rules
have to been adopted. In many cases, entirely
different proofs of works, things like that. I’m not aware of all of them. I know of some of them. Most people I know don’t
know of all of them or even any of them. So users might unknowingly
send both or not be aware of these things. That’s an issue. There’s also all sorts
of crazy legal issues. Talking to exchanges, where
like, OK, this fork happens. Do we owe our customers both? Do we only owe
them the one that– and which does the
exchange have to let people decide to adopt
or ignore new rules set? There’s all sorts of
weird legal issues that are still being settled. And for one example, you can
do a replay attack on exchange and this is not a theoretical
example, this has happened. So let’s say the
network splits, right? You get bonus coin
and regular old coin. And the bonus coin has
a majority hash rate, and there’s no kind
of replay protection. All the transactions
that are valid in one are valid in the other. OK, so the network splits
into coinA and coinB. And the exchange is
only running coinB. They say, look, this
has the most hash power and that’s what
defines the system. They adopt a new rule, fine. There’s this new rule that
you can generate a coin out of nothing. So the user says
to the exchange, OK, I’m going to deposit coinB. The exchange says,
yes, I acknowledge your deposit of coinB. That’s the network I’m running
on, I see your transaction, it’s in a block. Cool, you’ve got a balance. And the user says,
changed my mind. I’m withdrawing coinB
and the exchange– so what happens next? The exchange says, sure. Here’s coinB. Oh, and coinA, right? The exchange doesn’t implement
any replay protection. They don’t know. They don’t acknowledge the
existence of this other chain. They don’t know they
have coinA, and they should because they
actively split but whatever. And then the users like,
cool I got both, right. I relayed this transaction
between the two networks. I was now able to
deposit only coinB and withdraw both
coinA and coinB. And now I redeposit– I split again, I redeposit
coinB, and I keep doing that. And so I can drain the
exchange of all of their coinA with the same
amount of coinB just looping through depositing
and withdrawing. So, yeah, this happens. Does anyone– I
mean, I’m not going to say which exchange was
susceptible to this attack. Does anyone know? It shares a name with the
first transaction in the block. Anyway, so yeah, that was
almost two years ago, a year and a half ago
with the ethereum, ethereum classic hard fork. That happened. So that was– it happened. I’m not saying, yeah,
it’s not obvious, right? These are some attacks
that are like huh. In retrospect, it wasn’t
that hard to find out. I’m sure the people at the
exchange were like huh, shoot. Yeah, we probably
should’ve seen that coming, and we lost a couple
of million bucks. Shoot. It’s also weird because all
of their users like generally are identified, and
they have their password or where they live, and
so you could probably tell like, hey, come
on dude, give it back. But maybe they didn’t
because they like, well, tech– because they might
have a program in a way where they could deny it. And say, look, I just deposited
and withdrew a couple times. That’s what I always
do, I don’t know. But yeah, there were
definitely warnings. There were there were a
lot of people saying, hey, this is dangerous. You need to really
implement replay protection. If there is a fork without
replay protection implemented, the exchanges
really need to honor and try to split both before
offering both for a deposit and withdrawal,
things like that. And so there’s been a
lot, last year as well, there’s a lot of argument
because in bitcoin, one group of people
wanted to SegWit2x, was what it was called. And they wanted to
implement a hard fork and not implement
replay protection. And so that was a big argument
where people were saying, look, if you’re going to make a
hard fork, make it a full fork. Make it implement so that
you’re going to go off, but also implement replay protection. Make it so that transactions
that you guys signed are slightly different
than the old way. And it’s not too hard to do. What you can do is when
you’re making a signature, flip some bits. Well, OK. You can’t flip bits
in the signature itself because those
can be flipped back, but what you can do is
you can like flip a bit or two in the hash
that you’re signing, and then the old software won’t
be aware of that flip and say, look, this doesn’t look
like a valid signature because it’s trying to compare
it against a different message. Or you can like a pen,
something at the end of the message you’re
signing, things like that. So that on the new network,
the signatures look different. And that helps in
terms of safety because then the old
software that’s ignoring will not inadvertently
send transactions. And also for the
new network, they will not inadvertently
send transactions on the old network. So that’s– and there’s a lot of
ideas of opt in versus opt out replay protection, where you can
like allow the option to sign differently, but not require it. All sorts of weird
ways you can do it. But yeah, this is a fairly
recent mostly last summer, last fall, people were trying
to do different things. And so in practice– I think this is the end of it. Let me go two more minutes. But yeah, consensus
change is hard. In practice, there’s been
some full forks recently. The last soft fork was
Segregated Witness SegWit happened sometime in
September last year. It was it was a mess, and
there was also some full forks. Bitcoin Cash, and then
later Bitcoin Gold, which is a lot smaller,
and they completely changed the proof of work. And now they’re a bunch
that are like pushing the definition of full
fork, where they’re basically like also called
airdrops, where it’s sort of a completely different
coin that just happens to have the UTXO side of the old coin. And so it’s like, why even
bother with the history. We’re just like
look, it’s a new coin that you inherit all
these other coins. Yeah, so there’s
a bunch of those. It’s a mess, it’s fun being– I could not have given
this lecture a year ago. A lot of these things
had not happened. A lot of these terms
were not well defined. A few years ago, the
idea of soft, hard forks were not even defined. It’s pretty clear that Satoshi
later, after releasing bitcoin started to understand
this system. But in the beginning, there
was not a clear understanding. Probably the biggest,
contentious software in 2009 was that Satoshi added a 1
megabyte block size limit. And to reverse a soft fork,
is a hard fork, and so this blocks out his hard fork. And then there is a very
clever way with SegWit to make it a software, but
also increase the block size in a weird way that the
old software wouldn’t recognize. I might have to explain a
little SegWit to you next week. OK, so yeah. It’s a feature and a bug, right? Consensus changes in these
systems can be very difficult. On the one hand, you want
your coins to stay put. You don’t want your
money to change. You want to be able to
just have a bunch of money, and a year later, you still
have a bunch of money, and that’s what you want to do. On the other hand,
new features are cool. And these are not– you don’t want these to be
ossified legacy systems, you want this to be like
new, cool technology and you go make all
these new cool things. And make it faster,
and better, stronger. And the role of miners is also a
big point of contention, right? They seem to have outsize
influence in some ways, right? Up here is the mining power
and how that affects things. And why should these miners
have outsize influence? Shouldn’t the users
themselves be able to vote? But they can’t, right? If the users could
vote, maybe you wouldn’t need mining at
all to verify block chain. So there will continue
to be a lot of debate on this stuff going
into the future. I hope this helped explain
the general thinking as of early 2018, but
it’ll probably change. Cool. Any other questions
about this whole thing? If you have a light–
like I don’t know, James helps develop
Vertcoin, right? Are hard forks and soft
forks difficult in Vertcoin? No, you’re like, hey,
we’re doing a hard fork. AUDIENCE: [INAUDIBLE]
the [? exchanges ?] PROFESSOR: Yeah. So in smaller communities,
smaller coins, where there’s not as many people
involved and people are all on the same page, these changes
can be made fairly regularly, not a huge deal. Bitcoin is very messy. Bitcoin everyone
hates each other, they’re always
trolling each other on the internet and
hacking each other, death threats, all
sorts of stuff. So yeah, future forking
methods– there’s probably new, cool ways you can
add to the bottom of that chart some new idea that
maybe works better. So stay tuned.

Leave a Reply

Your email address will not be published. Required fields are marked *