ETHWarsaw 2023: Daniel Lumi - How to Scale ETH ‘Infinitely’ Without UX & Liquidity Fragmentation

um welcome to my speech on scaling eth infinitely without ux and liquidity fragmentation so first as any good programmer I actually really hate the word infinite it's marketing nothing really scales infinitely but what we can try to do is scale eth linearly according to the instances of rollups or execution sharding without ux and liquidity fragmentation so first a little bit about me um I'm a consultant researcher specializing in ZK l2s privacy and Dows most of what I work on right now is decentralizing provs and sequencers as well as how to maintain Atomic interoperability in factal z3s so today we'll be talking about how does blockchain currently scale how do we get it to scale infinitely and then what do blockchains look like in the future as well as when is that going to happen so why this talk um uh I'm a bit of a grumpy be programmer so I always looked at blockchain like this doesn't scale and the reality is we have very few users right now if we ever want to have even like a 100 million uh active users monthly it's not going to work blockchains will burn and you'll be paying thousands of dollars so how do we actually fix these issues and it wasn't until the last couple years that we actually started having like really good options and methods to actually make blockchain scale in a sort of similar fashion to web 2 so how do blockchains currently scale and this is comparing web 2 compared to web 3 right or blockchain um so how does web 2 scale so let's say apple launches a new phone and you want to pre-order it maybe the first thousand users go to one server and they pre-order the phone and everyone's happy right but now if there's like 2,000 people trying to access the same server it just kind of burns down it runs out of resources crashes and then no one can order anything right so what Apple kind of does is they kind of go through like osmosis they copy the state of the server onto another server and then the next thousand people get it from server number two and they don't actually need to know what server they get it from they're just pre-ordering the phone right everyone's happy now um so Web Two scales linear to according to the amount of resources that you allocate to it so if you have ser with X resources you get One X scale and you add another one of those you get 2 x and it goes on and on you have thousand servers you get thousand X scale so how do blockchain scale compared to this and the whole kind of point in blockchain is we don't really want to trust anyone um so we essentially go through this game of every single server on the network re-executes every single transaction to see the current state and to come to consensus on Daniel does not have a million e right and just to give an idea of how this is bad for scalability is let's say that you have a network with 3,000 nodes and a block takes 5 Seconds to execute essentially you're wasting around the world about 4 hours of re-executing the same transactions over and over again and this is honestly just wasteful and not how web 2 works so blockchain's kind of scale linear to the uh or the time that it takes essentially to execute a block is linear to the amount of transactions so if you have one transaction maybe it's that first dot there and then you have two transactions that's how much longer it takes to execute that block and that simplified a bit cuz every transaction is different a Unis swap transactions more expensive than uh just send transfer but the problem is we want because it's a decentralized network we want a normal computer like a MacBook to be able to run the network so we essentially put an artificial limit on this this is how many transactions we allow in a single block and in ethereum it's it's what is it 30 million gaps so it's about 15 to 30 uh transactions per second depending on what kind of transactions those are so essentially in blockchain if you have one server you get that much scalability but now if you add two servers you still only have one X scalability if you add 3,000 servers it doesn't help with scalability at all it stays static right uh it helps with decentralization and it helps with censorship resistance but it doesn't scale the network at all adding more resources to it so we came up with this concept of ethereum 2.0 right and the idea is okay let's split up ethereum into 64 shards and then we'll split up the validator committee uh into 64 chunks and then 164th of each validators like or total validators validate each separate Shard but there is issues with this like each of those 64 shards is now has the security of 164th of the whole network and then the other problem was interoperability like how do these sharts talk to each other like do we lose Atomic composability like Unis swap being able to call Price feed and we didn't really have a good plan for how to actually maintain interoperability and so we scrapped a theum 2.0 and that made metallic very sad so we came up with this new concept um of scaling e infinitely and that's called the rollup Centric ethereum right and the idea is okay what if we do execution sharding vertically so you still have a Shard a rollup but instead of it running directly on L1 it does it's a separate blockchain that does computation off chain and then it submits um the both the data as well as a proof onto L1 to prove that everything ran correctly ly and again this is execution charting um it's just a better version and um hopefully since you're here you know a little bit about how l2s work again it's so I won't go much into details on this but again you have a bunch of transactions on a separate blockchain the data for that gets posted to L1 as well as a proof showing that essentially those were executed correctly so when someone tries to withdraw money ethereum knows that there it's like the correct transactions and so in a r Centric ethereum essentially we get back to kind of web to like scaling these are made up numbers by the way it just for sake of thought like let's say Stark net scales etherum by 10x and then op is another 20x then you get 20x scaling and then you add some more of those you get 40x scaling right and the benefit of this Ro Centric theorum is now each of those shards actually inherit it's the full security of ethereum it's not fractionalizing the the security of the network but then there's lots of problems with this design right so we'll just go through these uh one by one and essentially like let's go through the first one if you all of a sudden move all of the ethereum users onto I don't know scroll or one of these protocols at some point scroll is going to become more expensive of ethereum right it's going to have higher transaction fees in ethereum and then what's the point of that right so how do we fix this so the crackpot team at starkware had this idea they just make more layers l3s l2s l3s L1 15s and now everyone gets a layer you get a layer you get a layer cool and actually it sounds silly but it actually helps scale it's sharding you just create more and more shards essentially and this design is called recursive ZK or fractal hyper Bridges or super chain we like marketing terms so the idea is that if you bridge from let's say L2 to l14 every one of those Bridge steps is actually just as secure as using the eth uh L2 right there's actually no additional security assumptions cuz each of the bridges use exactly the same circuit for bridging so you can safely Bridge from L2 to L3 to L4 without working about it and actually it's not even really bridging in this sort of fractal design it's more of accounting you just kind of Mark where the funds are now right and this kind of shows it so let's say you're going from that L5 to that other L5 right instead of like Cosmos needing to jump through every single one of those points to get to that bridge uh to that new spot essentially in the recursive model all of the funds are sitting in the same exact Bridge contract so every any ZK sync um L3 L4 doesn't actually move the money anywhere it's sitting on eth L1 in the same Bridge contract so you can just kind of Mark it over like it's not on this L5 anymore it's over here and you don't have to worry about bridging and also optimism and polygon are doing the same version but instead of going vertically they're going horizontally so essentially you deploy a bunch of l2s onto the same common bridge and again they're the funds are in the same place so you can just kind of Mark them over and so now with this design we fixed fees right now each of those shards has less users so it doesn't become as expensive as one chain having every user using it at once right but now the problem is interoperability right both from a uux standpoint of view as well as like it might be expensive to move funds from arbitrium to um ZK sync safely and it's not composable anymore it's not a synchronous transaction it's asynchronous so you can have sort of uh interoperability of contracts within them that happens automically so here's an example of how current L2 uh interoperability works so let's say you have fun some funds on Stark net and you want to move it to ZK sync you can just Bridge it over with like multi-chain right right now please don't let's not create cross-chain Bridges that's a horrible design like there's lots of security risks uh with cross chain Bridges so how do we do this trustless right so what you can actually do is you can batch transactions from stocket L2 to L1 and then that sends it to the starnet uh Bridge or sorry the ZK sync bridge and the funds appear back on ZK sync and this way you actually have a trustless bridge design you don't have to do this individually we can make bridges that essentially batch 10 users into the same uh bridging transaction action and then essentially the eth gas cost that you pay is 1/10th of a actual Bridge transaction right and this works you get the bridge funds but it's also asynchronous it's expensive even if it's one/ tenth of a e e transaction cost and it takes a while especially with optimistic protocols it might take 7 days to do this so it's better we we can kind of solve this in the designs the fractal hyper Bridge designs where you essentially again you have all the funds on the same uh contract on L1 and then any of the l3s or l2s building on the same Bridge contract essentially don't have to go that way through L1 right and this is so let's say the money is here in the bridge contract which is actually a contract on L1 so when you want to move it from coinbase L2 to optimistic L2 again you just kind of Mark that it's over there and you don't have to have this well you'd still have to have the same day delay but you don't have to pay the same amount of gas fees right and this works for bridging but it's still asynchronous and now there's me someone might see 7 days before you move that transaction over that oh this guy's going to trade on Unis Swap and they can front run your transaction and steal money from you so the way that we could like one example the original example of how we could actually fix this asynchronous uh uh comp component is okay let's have coinbase L2 and optimism L2 use the same sequencer essentially and this way um the sequencer essentially uh orders transactions from both of the chains and now you have atom acidity the same way that you have on eel1 and this way you actually have composability and no me but now it's a beefy node if you have 100 different l3s all using the same server like we're not going to get that much scalability from it it might be cheaper but we still kind of want to keep l2s decentralized it's not the same kind of security assumptions that you have on normal blockchains because even if it's not decentralized they can't steal your money but we want some sort of censorship resistance and decentralization for the sequencer role and you can call this one BP sequencer to rule them all not a great design uh even if the ring had a good thought behind it and so how do we do this how do we keep um like decentralization and keep the nodes relatively small so what we can do is we can have one sequencer um that is shared between both of those chains that sequencer doesn't execute the transactions it literally just orders them so it puts the coinbase transaction then the optimism transaction and it doesn't actually execute them which is the expensive part ordering super cheap and just adding values to if you've used X sell it's like adding another value to it and so we can keep that server very small and then every other L3 or L2 can essentially execute its own transactions later and this way each of their sequencers are relatively small they only have to worry about their own State and their own transactions and uh again they agree when they're going to insert the transaction and then you can Bridge funds over and this actually keeps um atomic composability and it's small nodes right but there's still the issue of like now coinbase chain doesn't know the state of optimism so maybe you submit the transaction to swap E from uh coinbase to optimism chain and then back but by the time it gets executed it's like the price has moved and all of a sudden your transactions fails now your money is somewhere else and it becomes a pain in the ass again again because it gets executed later anyone that runs nodes for both of those networks can front run your unisoft transaction and they have lots of time to do that so that's not ideal so we can essentially add if you've heard of PBS and ethereum um this is a bit more in depth just this topic alone but we can essentially have Suave style proposer Builder separation where essentially the Suave Builders run nodes for both of the chains and then they see that okay you have a swap coming through and they're running the state so they know the state of each chain and they can actually promise you that this Unis swap transaction will not fail so when it executes a little bit later it's like it's guaranteed to happen right and now you have composability and no fail transactions but me is still an issue again now the Suave Builder see your transactions coming in and they can be like oh we can make some money off of this guy um and even without telling the network they can again front run your trans transaction so now we can add threshold yeah this T this is not a simple thing the the interoperability of like doing this but the hope is that again the user wouldn't have to know anything about this so we can have a new transaction type called a threshold encryption transaction and essentially you send that transaction with the transaction header public but now you encrypt the body so they don't see that you're swapping uh on uni Swap and at this point they includeed in the block like they promised and then block n+ one they essentially so the next block they decrypted and then they say oh I could have made some money off of this swap but now it's too late it's already included in the chain right and now you have composability small nodes and no me in this sort of like rollup Centric ethereum world and a cooler sort of thing that you could do with this just to explain why this is exciting is like like let's say you have Unis swap as L3 or Unis swap on ethereum and you want to swap some tokens through it cool it works you get your money but the problem is if 100 people are swapping on Unis Swap all of a sudden your gas fees are going to get really really expensive right so what you can do with this sort of sharded fractal design is you can dynamically keep creating new instances of Unis swap the same way that you do in web 2 with the Apple server keeps creating new instances as the demand needs it and again your coins are not in those shards they're on the lower level so now you can dynamically swap against any of those instances and you don't actually even need to know I'm connecting to this L3 or this L3 or this L3 it's just like through RPC level we just show a swap happened and now you have you know some usdc instead of e and now you actually get like a Unis swap contract that can scale d dynamically essentially infinitely as long as you keep adding uh new instances of it and in the future like when that demand goes down you can actually delete those instances because it doesn't matter what the state of that um specific chart is it matters where the money is so now we actually fix like interoperability so all of a sudden you don't need to think of which L3 or l-15 you're on it just kind of works you can trigger a contract and and it happens that interaction but now you have all these rollups all these L1 15s l12s that essentially need to submit their data onto ethereum and that's a lot of data and ethereum can't really handle that kind of data so what can we do about that so this guy named danrad thought about that and he's like hm wait we do that already um every data center uses something called eraser coding is and um it's we could apply the same to blockchain chain right so here's kind of how dank sharting works and I'm sure you guys have heard about this and this is more of a mental model than a accurate picture of exactly what percentages of rasor encoded data is and it doesn't go into leptic curves the idea is currently in E theum every single node on the network needs to hold all of the data right and that's why it becomes a problem because if you have 15 rollups putting the data on chain your your node requirements are going to go up a lot right so with Dank sharting instead we can essentially hold just one chunk of the data right and then the problem with if you only hold one chunk of the data is like what if this guy disappears now that Blue Block is gone right so what we can do is we can hold one chunk of data plus another 25 to 50% of eraser coded data of the data from other people's blocks right so now if the same guy disappears we the three that are left can essentially use their raser coded data to reconstruct the missing data right and but now the problem is you're only holding a small part of the data you also have the Eraser coded data but how do you know that all the data in ethereum's history is available right and that would be a very big problem if any of it disappears because it would break the blockchain it could never forward anymore so essentially you can um send some queries to the network like random queries so let's say there's 100,000 blob data points um you just pick a random number between 1 and 200,000 right and that's the data that you're asking for and they from the first sample essentially if someone Returns the data there's a 50% chance just statistically that all the data in ethereum's history is available then you do this again random number you guy give me some data now there there's a 25% chance that anyone could have lied right and by the time you do this 30 times there's essentially a one in a billion chance that that data might not be available and you can keep doing this as long as you want to the point that it's like even if there is a chance that someone could have hidden data or deleted data it's like statistically next to Impossible right and cool now you know all the data is available right but what happens if someone actually does delete the data right like what if part of that data is gone now you don't have the full version so ethereum's broken right so what you can essentially what you need with eras your coded data and full dank sharting is you need at least 75% of all the data to be available to reconstruct the missing 25% but the benefit of this is like that sounds a bit scary cuz maybe it does disappear but the reality is like any person in the world can and hold a full copy and even if most people don't have that data they can't lie about the data they can't just like create fake transactions into it essentially we can check again the data against the hashes so as long as coinbase or you at home or literally anyone else holds all the data we can essentially continue ethereum and there's like no problem every node uh node storage requirements are smaller and we'll never end up in the situation where the data is not available and a cool part of this dank shorting thing is like sometimes we have these shitcoins I mean sorry if there's anyone that likes frogs um that happen on Main net and all of a sudden gas fees Spike up a crazy amount and that's bad for l2s right maybe l2s can continue but with Dank sharting you actually have this sort of two-dimensional gas market the gas market that rollups use is completely different than the gas market that normal contract execution does so even if there's another shitty nft that spikes up the cost of ethereum rollups can continue functioning without any negative externality so all of a sudden arbitrum doesn't stop cuz there's a nft Min and so cool we fix all these issues eth is infinitely scalable right right eh not really we still need to fix other on L1 and unlike some Bitcoin and Salon people have been claiming we haven't forgotten about L1 it's just like this is execution sharting is the thing we were working on S since e 2.0 so that's kind of prient um so how do we fix issues in L1 what can we do so the first thing that we can do is we can do parallel execution and localized fee markets and essentially we already have the ability when we make a transaction we can declare access list so we can declare we're about to touch this part of Unis swap what part of State we're touching and this wallet right and if we declare those if we're forced to declare those essentially we can do two very cool things the first is all of a sudden any transaction of me sending Miko one e you can parallelize all of those transactions so you can execute them at the same time without worrying about it but that only has limited applications like maybe if Visa wants to run transactions on eth Parallel execution would be great if we're going to have 100,000 transactions a day um or second um but for the shared State you can't really do anything about that but we can do another cool thing uh with that called localized fee market so again if there's a random frog coin or nft that spikes with localized fee markets essentially that contract's interactions would get much more expensive but it wouldn't affect the price of uni swap at all so people could continue swapping on Unis swap without being hurt and again we can do this on L1 um if if we can't if we don't do it on L1 because it'll break some contracts which it will we can at least do this on l2s the other thing that we want in ethereum L1 is statelessness and essentially we can move the storage structure to something called veral trees and um again right now the kind of issue for nodes is every single node needs to hold the entire state of um ethereum and that's about 1 tbte currently it'll keep growing but you can't really load one tbyte into RAM into memory so you have to store it on a hard drive but a normal hard drive is not fast enough so you need a SSD hard drive and a 2 TB SSD hard drive is kind of expensive right and and if we want this protocol to be decentralized like we can't have that kind of barriers to entry of like 200 might be cheap for us but if someone in a developing country wants to run the node that's prohibitively expensive so with statelessness essentially we create full Witnesses of each block and essentially this way each node does not need the entire history of ethereum to run the next transactions they can essentially just download the current state and with that you could essentially have like mobile phones like this would take about one gigabyte per device so your iPhone would have enough space to be a full node on ethereum um ZK for L1 this is not a if this is a when this will happen we will make ethereum ZK like the actual execution layer and the consensus layer and the benefit of this is like going back to how blockchains currently scale again we have to kind of limit the amount of transactions that can happen because we don't want it to be a salon supercomputer that runs the network right so with ZK essentially whoever proves the transaction still has to do a lot of work but every other node on the network doesn't have to run that transaction at all they can just verify the proof and if there's a proof for those transactions you know the other guy executed it correctly right and actually we don't even need one computer to do this it's really easy to parallelize proving um let's say you have a block of 100 transactions you can have a 100 different computers proving one transaction completely separate of each other not worrying about it in this way the verifiers are small computers as well as um uh any execution noes are small computers and that's essentially the scalability we can get in um L1 from ZK and with this essentially you could again you could have a iPhone essentially being a full node you could have your metamask be a full node on your computer and you wouldn't have to connect to any RPC and we get much more decentralization as well as then we can increase actually that block limit of how many transactions are allowed um in Trident rollups I'm running low on time but essentially with inin rollups um you essentially the consensus of the L1 has uh the the rules of the rollup instead of being a separate blockchain they would share the same contract for verifying proofs this is under active research it's going to come take us 10 15 years to come to consensus on how we should do this but we will have that in the future as well and so how does this affect blockchains in the future the hope is that by using in production and researching all these topics in l2s whether it's ZK or shared sequencing for interop between shards we can actually use this to apply it back to ethereum and we can come up with an execution sharding 2.0 that's actually interoperable and inherits the full um security of ethereum right and so when token just kidding uh when scalable soon give us like 10 20 years it's it's going to take a while this shit's not easy but if we do get here we can actually get blockchain adopted by like the world like there's no way Visa or normal companies are going to use this if it doesn't scale and if we do figure this out you'll see metallic clap it's really fun like it's worth it just all these worth years of work and Engineering is enough just to get vitalic happy uh and the point of this presentation is unlike some blockchains Bitcoin uh some blockchains nothing no code is perfect no protocol is perfect we need to keep working on this and figuring out some of these scalability issues if we actually want adoptation in the real world and we need to do it in a way that your grandma or your mom can use it and they don't keep having to switch chains or worry about about the security in in moving funds between different uh chains and we talked today a lot about ZK um it was a small field of us cryptographers math nerds that liked ZK for a long time but don't ask a cryptographer how to explain ZK they'll go into polinomial commitments and Friday based for corrosion this is a mostly mathless version of understanding how ZK Works um bridging is a long conversation whether it's cross cross train bridging or an L2 um I do have a speech on that just to break down each security compromise um and again l2s have their own security issues we had a panel at eth Gathering where we talk with some teams about what their issues are and how to fix those and that's it uh you can follow me on Twitter at ZK Lumi I don't do a lot of uh posting in the small community of ZK l2s we have tog Ru for that so uh but I do put up some research sometimes or any new speeches and you can follow me there thank [Music] you