ETHDay - Tim Beiko - AllCoreDevs - Ethereum Foundation

good morning everyone okay this works um cool i'm timbago i work at the ethereum foundation and i'm here to give you a quick update of the ethereum world map um yeah so first thing on the roadmap you've probably heard about it is the merge uh so i'm gonna walk you through what the merge actually is where we are in the process and and what's left to do um and to do this it's worth looking at like the ethereum two dollar world map that we uh publicized extensively several years ago and and how what we have today is kind of changed from that um so if you remember like around 2018 2019 it was this idea that uh ethereum 2.0 would come in three phases so there was phase zero phase one phase two um and it would be kind of sequential upgrades that get us to a fully scalable ethereum running on proof of stake and the first part of that phase zero was the beacon chain um so the beacon chain is the part of ethereum that runs the proof of stake algorithm and that launched in late 2020 after that the plan was to create what we call data shards which would store a bunch of data for ethereum and then the last part of the plan was something called execution environments which would convert these shards which store data to basically evm chains which run computation and that was always the most complicated part of the plan it was always something that kind of sounded good that we would do but it was never like a great spec for how to get there and so in parallel to launching the beacon chain we started to see rollups develop and gain adoption so um roll-ups are basically a different way to scale ethereum where instead of having everything kind of live in the protocol these roll-ups use layer one as a sort of anchor and source of security but then create their own evm or zero knowledge computation environment where they run the transactions and because this basically happened in parallel to us trying to deliver on this roadmap it just made sense at some point to instead of trying to do this really complicated phase two part of the plan rely on roll-ups as the main way to scale ethereum and so we had kind of this roll-up-centric uh robot-centric road map in the middle uh you can see and and the idea was like well we're going to have the beacon chain we're going to have a single evm chain and then we're going to have the roll up scale execution and then there was like a final insight that led to the current architecture we had which is that today on on ethereum the clients that run the ethereum software they can already run more than one consensus algorithm so if you're on a node on mainnet using nethermine basugeth you run under proof of work obviously but if you run on any of the test nets you already run on a different algorithm so none of the ethereum test nets use proof of work and so the idea that we had was well if there are if these clients already built in a way that they can use a different consensus mechanism can we just have them use proof of stake instead and use the beacon chain as a source of consensus and so that's what we came up with and this is kind of what the current post-merge architecture for ethereum will look like so if you look on this slide um an ethereum client after the merge basically becomes the combination of both a beacon node and an execution engine and we we call the beacon node and the general proof of stake chain the consensus layer of ethereum because it is what uh kind of provides consensus on the latest state of the chain and the execution engine is basically what is known today as an eth one client so uh you can think of something like geth or or basu uh or something like that so after a merge um basically a full node will run the combination of both these things so you'll have a consensus layer client so something like prism uh prism lighthouse uh loadstar which talks to an execution engine uh which is geth or basu or aragon and to make this work we introduce a new communication api between both layer called the engine api um but aside from that things are pretty much unchanged from a network perspective so that means that like if you're an application and you use json rpc to interact with geth all of that is still going to work as it currently does or say you're like a block explorer and you're using the beacon apis all of these will keep working as is and so what is great about this is like we're reusing all of the software that's currently battle tested on mainnet and the beacon chain um keeping the user experience kind of the same and from a smart contract's point of view the merge is going to happen with basically no downtime and if we zoom in a little bit this is kind of what a post merge block will look like so um at the outer kind of the outer part here the consensus layer um this is what's on the beacon chain today so the beacon chain comes to consensus on blocks but they only have these outer parts they don't have this execution layer bit so for example every block in the beacon chain you know tells you what's the slot number what's the parent route there's a signature from the validator there's all the attestations all the new deposits all the validator exits um but aside from that it's empty and today we have kind of this execution layer which contains all of like the transactions on ethereum so if you send a transaction if a smart contract makes a transaction all of that is included there and there's also some other important information for execution like the base fee and things like that but today this kind of execution blob has a proof of work layer around it so after the merge you basically combine these two and get this so not a lot changes actually so the the few things that are worth noting is the block times go from being about 13 second but having a lot of variance because of proof of work to always being 12 seconds or a multiple if a validator is offline when they need to produce a block um there's a couple fields that we set the zero in the blocks just because they don't matter anymore anything related to proof of work um the one thing we don't set the zero though is the difficulty op code uh so if you're a smart contract developer you'll know there's a way to get the difficulty up code in in ethereum and it's like a source of bad randomness but anybody who uses it on chain basically uses it for pseudo-randomness and if we set this to zero it would go from being bad randomness to pretty terrible randomness so instead we return the randall value from the beacon chain and so how do we get from like the current proof of work system to the post merge architecture um there's a couple steps the first thing to note is that unlike previous upgrades on ethereum we're not using a block number for this um and there's basically security considerations around that for uh how miners can fake chains with a large block number uh so instead what we use is the total difficulty uh of the proof of work and just to give a quick overview of how that works uh proof of work basically there's a difficulty to mine every block if you add the difficulty of every block that's ever been mined in the chain you get a total difficulty value and so what what we do to trigger the merge is we set it a total difficulty value which we call the terminal total difficulty and once a block hits or exceeds that terminal total difficulty we consider this to be the last proof of work block on the network and so once we see a block that that meets this condition um we then rely on the validator on the beacon chain who's set to produce the next block to simply include transactions as part of their block and from that point you basically have you know the last proof of work block which exceeds the ttd then the next block is produced by a validator on the beacon chain and we're not using proof of work anymore and then the last thing we want to see after the merge is once we've had this first block produced by proof of stake we want to see this finalize on the proof of stake side and finalization is basically the proof of work equivalent of having a high number of confirmations where you have a high degree of certainty that the chain is not going to be reorg and so you can consider like the first post merge block to be final so i know this is a lot but i have a diagram to walk you through it um bear with me please okay so this diagram basically shows what happens before and after the merge so if you start way to the left on the top side uh you have a proof of work block uh like we like we saw earlier so you know there's some proof of work stuff we didn't include it uh and then within that proof of work block you have all of the execution layer content so the the parent hash of the block the state route the base fee and the list of all the transactions included in that block in parallel to that we have the beacon chain running so if if you go right under that block you can see there's a beacon chain block which has all of this proof of stake information but no transactions um if you forward uh kind of one step in the process this would be kind of our final proof of work block so if if we had set the ttd the ttd would have been like crossed between the first two proof of work blocks um and and that kind of triggers the merge so at that time we see there's like the last block on the beacon chain without any transactions and then the merge basically happens so the next block on the beacon chain simply has transactions included as part of it the validator kind of run those make sure that they're secure sorry that they're valid um and then references kind of the the transactional content of the the last proof of work block um but then after that one you can see that the next block on the proof of stake side literally has no reference to proof of work so from that point we're running a hundred percent under proof of stake and so again you know from like the perspective of a smart contract or anyone using the chain you literally have kind of just two blocks coming the last one's on proof of work the next one's on proof of stake and there's about 12 seconds between them but there's not like a major delay or anything so um how do we actually ship this thing uh so i'll walk you through like what we've been doing and and and kind of what we what we hope to do next uh so earlier in the talk i kind of showed you know how we got to this this world map of using kind of both the execution layer consensus layer having the engine api so we got this idea kind of about a year ago and in may of last year we decided to actually prototype this with all the client teams to check if it would work um so we spent the month uh getting your first prototype and and we managed to get it working um so at that point all we had was like a post-merge network it didn't run through the merge or anything but we were able to say that like this idea of having the validators run like a guest node and produce and execute transactions in that and a test that they're correct we were able to confirm that that would work then we obviously had a ton of bugs in that spent the summer debugging everything in the fall we were ready for kind of another round of prototyping so we got all of the client teams together for a week and we prototyped a network which started on proof of work and ran through the actual transition and ended up on proof of stake um so after that week we kind of had high confidence that um we could actually run through the process on an existing proof-of-stake network and it it would work again tons of bug we found i spent the next two months fixing them right before the holidays we launched a first new uh test net called kinsugi and the idea there is we were at a spot where we thought the spec was pretty final um and we wanted to get some early feedback from applications and users to see you know this does this make sense is there anything that's breaking um obviously some things broke and and we fixed them um and just a month ago we launched kiln which we hope to be our last new test net for the merge um so the idea with kiln was that we found all these issues with with kinsugi a bunch of edge cases in the spec fixed them um but now we don't expect to make any major changes to how the how the merge is specified um since we've launched kiln it's been about a month we've started to find bugs basically in the client implementations and the difference here is that um the bugs we found on kinsugi basically were telling us like the approach we're using for doing the merge has issues whereas now the bugs we're finding tells us that one of the teams who implemented the specification kind of had an issue with how they did it um so over time you know we're getting into more and more stable things and finding issues farther and farther up the stack which is a promising sign the next thing that we started doing and that's what we've been uh doing extensively in the past month is called shadow forks um so this is a new thing we've added as part of our testing cycle which is which is extremely helpful so what we do is we take an existing network something like gourley or even mainnet and we launch a small number of nodes on it which actually run through the merge um so you can think of it like a hard fork but there's only like a tiny number of people who are running the nodes going through the hard fork so the rest of the network kind of has no idea this is happening but in parallel we've run through this hard fork and and kind of see if it works um and the thing that's really useful there is that once uh we actually upgrade to the merge we can still replay the transactions from the main network on our shadow fork so it gets you to test this in an environment that's basically the same as either gourley or mainnet which helps us uncover issues that we would otherwise find on those networks so um basically how we get the mainnet um so on the first side here is how we'll get through the testnet upgrade so once we've run more of these shadow forks and that they're consistently stable we're not finding major issues in any of the clients and and things are generally looking good then we would we would move to forking the existing test nets and the reason we want to be careful with test nets is that today a lot of applications actually use them for production things even though they're test nets so for example reddit has a big application that's on one of the test nets and even though there's not like real financial value on them we'd rather not break those applications if we can avoid it so once we have these these these stable shadow forks we'll move to forking the test nets um and there we we have four test nets live today uh we have gordy sapolia robson and rinkeby gordy's probably the most well-known test net it has like a super active community so this is going to be maintained post-merge and it's a great testing environment because there's a ton of activity on it um sipoli is a new test net that we launched one thing with testnets is like over time they do have a ton of data kind of like mainnet and it becomes harder to sync a node so if they're not super well maintained it kind of makes sense to periodically just shut them down and create a new one so sepolia is our new one which we hope to maintain kind of longer term post merge um robstem the ruffs intestine that right now works under proof of work so it's the only test net that actually uses proof of work uh and because it's a test net you get no money when you mine it so it's a very chaotic proof of work environment so we'll run it through the merge to just make sure everything everything looks right but we'll sunset it sometime after it and then ringkibby is an old testament that's maintained mostly by the get team um and and uh we're not even gonna transition that one to proof of stake we'll do sepolia instead so if you are using rankinb you should start looking at other test nets we're not going to shut it down right after the merge but at that point it won't be like a perfect replica of what's on my net and then once we have all these understable and we know it works uh then we would move to fork in mainnet um so the way the way it would work there is that um we would have to find the terminal total difficulty for mainnet um and um and it's kind of hard to estimate this because you have to well you can set a number but then you need to figure out how long it's going to take you to hit that number and because the hash rate varies on main net um the farther away your estimation is the the harder it is to to get it right uh so we're gonna we're gonna try and have this on the order of like a couple weeks um and the one thing that we can do so i think some people are probably well aware of the difficulty bomb uh which is uh a thing that happens on mainnet um if this is an issue we can kind of couple the merge upgrade with the difficulty bomb delay um and then obviously uh you know the number one priority is making sure mainnet stays up that there's no issues um so you know if we find an issue with the merge the day before we would probably delay it because we want to be sure that you know ethereum does not go down it never has and and that's something we're hoping to keep through this so after the merge we do have more stuff coming um so the next upgrade we have planned is shanghai um and there's basically nothing is set in stone for shanghai yet but there's like three main themes that we're going to try to to get and then a bunch of small improvements that we're looking at as well um the first big thing is beacon chain withdrawals so one thing that's worth noting with the merge because it's like literally the most complex upgrade we've done to ethereum uh we wanted to simplify it as much as possible and cut any excess feature and one kind of big thing we removed from the merge was the ability to withdraw your stake from the from the beacon chain um obviously a lot of validators have been waiting for this for a long time uh so right after the merge basically the next upgrade uh what will introduce the ability to withdraw from the beacon chain um the second big thing that that we're working on is this idea called evm object format um so everybody wants to improve the evm and the challenge we always get when we do it is finding a way to improve the evm without breaking every contract that's live on mainnet if we add new functionality it kind of interacts with contracts that are currently there if we change functionality might change how current contracts behave and we want to avoid that so we finally found like a way to do it where we can have a new kind of type of contract that people can use which the evm is able to recognize and execute under these different rules so that's another big initiative that we're working on the other big one is layer two fee reductions so the kind of really big pressing issue with ethereum is that the fees are really high um we have layer twos today which offer lower fees um but if we get another you know 10 or 100 x uh surge in demand on ethereum then the layer two fees will be incredibly high as well so ideally we get we get a bit ahead of that and we managed to reduce the layer two fees um there's two different solutions that we're looking at one is a bit more complicated one is just a straight up cost reduction um and we're thinking through you know the the trade-offs of of both of them and then finally there's a bunch of other small eips that provide kind of nice features for for developers that were were looking to include um all of these are quite small changes and there's a link to the slides at the end if you want to dive into all of those and then the last thing i have here is um some changes to the process itself um so currently how we make changes on the execution layer so like the application layer that people use today to send transactions on ethereum is different than how we make changes on the beacon chain um and the reason for that was like when we started developing the beacon chain we wanted to move a bit quicker uh especially because it was a new thing there wasn't as much a stake on it as there is on on the main ethereum network uh so so we kind of came up with a different process but now that we're actually merging both networks together it would be nice to have kind of a single unified process to propose changes um and neither of the processes we have today are perfect so on the execution layer side we use eips um which are are great to like uh get the community to to know about change and and to like specify them in english but they're not actually like a specification for ethereum so the way ethereum is specified is really weird we have this yellow paper which is like a math document that specifies all of the protocol rules and then we have this eips which are mostly like code or pseudocode which tell you like look at the last version of the yellow paper and then make these changes and there are two documents in like different formats um that are pretty unrelated to each other um so it's obviously not great um and then beyond that we then need to write tests for all these eips and there's no like linking between the yellow paper or the eips or the tests so you basically have humans looking at all of that and making sure it lines up which is obviously error-prone on the other side on the consensus layer we learned from this so we we started with an executable spec and the idea there is you basically have a python version of the beacon chain that's implemented um it's not efficient so like you can't run it like a node but there is all of the functionality there um so that's you know from like a specification point of view it's really well encapsulated and it's also really easy to make changes to because generally the people who propose changes to this are client developers and so they're usually more engineers than mathematicians and they can just make a pr against the spec with their change the thing that is a bit annoying with them is there is no like english or like accessible document to to to track these changes so they're usually you know scattered throughout the repo of the of the specs um in pr's and comments and so you can't just say like this is like eip1234 and it's a bit harder to like reason about them because of that uh but then the next thing is you can generate tests from that spec and you get some better security coverage so um ideally we'd like to combine both of those two things uh so we there's been work on a project called eels for ethereum execution layer specification um which is basically creating a code specification for the execution layer on ethereum so that we wouldn't have to rely on the yellow paper and eips and there's really a neat way where we could marry these two processes where we could keep using eips to specify changes at a high level give them like a tractable number have like an english description of like what is this change and why do we want it but then the actual code specification would be in either the ei either the execution layer spec or the consensus layer spec and another thing we're seeing more and more is features that span both so for example beacon chain withdrawals something has to be withdrawn from the beacon chain and then credited on the execution chain so having a model like this might be able to to allow us to just have a single eip which says we're doing withdrawals here's the execution part here's the consensus part um that said it's really easy to speak about this at a high level like what i do but uh actually getting it all to work uh still has a lot of things to figure out if you're interested in this there's an active discussion on ethereum magicians uh to contribute to oh okay um that's all i had i don't know if i have time for questions i do nice um and if you're on the slides the qr code is there all right i'm gonna be taking some questions that we got okay one of them is a bit what if proof-of-work miners start shutting down machines just before the merge in order to prevent ttd from being triggered what do we do then yeah that's good um so so yeah so right so i'll go back to my my uh thing here um right so what do you do like you know we set a terminal total difficulty miners are like well screw this i don't want to mine anymore um i'll just stop and then instead of hitting it in three weeks we're like targeting it to hit it in like six months and the network is not secure anymore that's really bad um so we we have a feature as part of the merge called terminal total difficulty override which basically allows us to lower the value so if we were in that spot where like miners stop mining we simply lower the terminal total difficulty um so we hit it sooner and the thing that's neat with that too is like you're always gonna run through the merge before the initial value that you set so like if you know instead of hitting it in two weeks we're gonna hit it in four because miners are dropping off we can set to another flag and then anybody who's like not upgraded there would have been at least a couple weeks where the merge that has actually happened and we can like reach out to those people um so it's not ideal but i think it works and we've also we've also tested this on one of the shadow forks and and it worked all right um another question is i mean i think you covered it but i want how are you going to verify the new proof of stake protocol will be reliable and not buggy so the proof of stake protocol doesn't change and i mean we have a ton of tests for that and i guess there's two things one is like it's been live for over a year with tens of billions in it and we we wanted to wait until we were confident that it was secure before even starting to work on the merge so a year ago we were already in a spot where we thought it was uh it was safe enough to use okay actually one of that one of those questions was asking that what has the beacon chain been validating exactly all this time since it's launched until the merge yeah that's a great point so um basically this diagram shows you uh and you can't see my kicker but if you look at like the first beacon chain block all the stuff there is what it validates so it's kind of weird it's just like all the validators are making sure that the other validators are validating but they're not actually running transactions through the evm and making sure that those are correct um so you validate you know how many new deposits come in do people get slashed um did was this validator the one who should produce the block you know and all those things but you're not actually executing end user transactions okay okay one last question and i think everybody has this question in mind i think i'm gonna walk thank you for asking it you must get this like 300 or a thousand times per day on twitter what is the current eta on the merge there is none um no i'm not going to say a date you're being live streamed soon um soon yeah trent if you stay around for trent's talk after trent will tell you at the end of his talk [Laughter] that's a nice hot potato well thank you so much tim thank you thanks everyone [Applause] [Music]