Everything you need to know about state expiry by Han | Devcon SEA

[Music] all right cool so it's the first session so I want to kick things off a little bit of fun I'm going to give away 0.01 if to whoever that remembers the first ever Smart contract that they have deployed you can raise up your hand if you still remember the address okay I actually expected a few but I didn't expect like none but it's but it's okay like I didn't remember the first ever Smart contract that I deployed and that's normal when we enter the space we probably think that smart contracts are cool we play around with a template or two um deploy some um like fake token or like nft contracts and just send some test transactions and that's totally fine but I just forget about it but the thing is that the data is still there right but you me and the rest of of the ecosystem will likely never use it again so why story onchain so today what I'm going to talk about is really I'm going to go through the problems that we have with State growth the proposed solutions that we have with State aspir and where we are headed next so here's everything you need to know about State aspir let's get started first let's do a quick revisit on state what does state mean on ethereum is that state is represented by a mer P Tri MPT so um we call first we call it the state try in the leaf notes of the State Trial we have all the account object objects so each account will contain balance nons root has and code has and if it's a contract account then it would has its own storage Tri as well it's also an MPT and in the leaf notes of the MPT of the storage Tri will contain the 32 bytes slot values right so this is the overview of how state looks like on ethereum let's zoom in a little bit by focusing on individual account object and Slot keys so given an account address we can actually Traverse from the root of the state try down to the leaf node and we will get the corresponding account object right likewise for slot key um we can Traverse from the root node of the storage try down to the leaf note then we can get the corresponding slot value So This Is How We Do the mapping so really if we really zoom in and um just be abstract right ethereum state is really consists of like a mappings of um um series of mappings between addresses and account objects and also slot keys to slot values so these key value pairs you can really think of it as the basic unit to measure State on ethereum when ethereum started from Genesis um it started with a small set of key value pairs as the chain progresses as more blocks are included you get more more and more key value Pairs and the number just keep increasing increasing and increasing it will never stop because state is unbounded and this phenomena is called State growth right so this is the problem that we are trying to solve today but why is this the problem to quote Peter from GFF ethereum is slow because the state grows like crazy what happens when the state side grows is that the number of notes in the MPT Grows Right so in the client software implementation that means that for every read write event um what happens is you it will require more database lookups it will require more um hash calculations so all these are expensive operations so when when you have those scenarios then your Noe will become slower and it's just not just that right you just think about Hardware storage every few years you probably have to change your Hardware to upgrade your SSD so that uh it can contain more um state so if you really think about it as a flow when the state grows what happens is that running a node costs more because you have to upgrade your Hardware every few years whether it's storage or like CPU or whatever um when you have that kind of scenario that means that um less people will be incentivized to run nodes like normal people like maybe just you and me and when less people nodes then um e become less decentralized right so that's like a effect of State growth but I know what some of you may be thinking right some of you may be thinking that it's fine right um The Innovation in Hardware will probably outpa State growth and that's true but is it really as simple as changing hardware and your um note will magically run again it's not as simple as this the reality is that you have to first go to a physical store or online store store purchase the hardware you have to once your Hardware arrived you have to reassemble your note take out the screw take out the cover swap the hardware put a hardware back I don't know like put a cover back put a screw back then um maybe you have to migrate your storage from the old disc to the new disc during this process if you messed up if you made a mistake what you need to do is that you need to wait hours to resing again from zero and yeah it's just a very terrible user experience so no it's not as simple as people may think yes if your technical is simple to you but not for normal people no writing notes should be easy or at least we shouldn't make it harder than what it is right now so looking back at the state if you look at all the key value pairs ask yourself this question how many key value pairs were actually accessed in recently let's say in the past one year based on the analysis that I did is it was only about 20% so really what we are storing right now for the execution clients are 80% of the state they are unused in a long time and what happens when we eliminate this 80% of the state wouldn't we get a slower State growth wouldn't we be able to reduce the hardware um storage requirements wouldn't we be able to build better execution client a more efficient one so that is what we are trying to do with State aspiring the concept of state aspir is simple it has one key objective which is to expire now and revive later we want to provide a way to expire the still data on chain while providing a way for users to revive those data there are two key question that we we need to answer for this objective first how do we know if a state is expired second how do we revive expired State and let's think this true together first we can look at the current MPT right so this is the structure of the MPT we are only going to focus on the values because those are those values are the ones that you want to expire to answer the first question how do we know if a state is expired well you can take a few seconds to think about it the most straightforward way is to add some sort of a metadata to measure the state or the values right so here what I'm going to do here is to add something called an access period to it so an access period can think of it as like measure in blocks or basically like one period is equals to 6 months maybe so when you have um a period that corresponds to value then we can already answer our first question in regards to the expiry rule we can simply set like for example a value is considered expired if it hasn't been accessed for maybe more than a year right you can change this rule so we answer our first question what happens when a Noe or a value expired right so for leaf node we can simply replace the leaf node expired Lea Noe with the hash of the leaf node or um if the for branch note if all the all of the children are um are expired then we can simply replace the branch note with the hash as well so that's a relatively straightforward um expir rule so for the second one which is to revive how do we need how can users revive the state is by perhaps submitting a transaction uh here we will introduce a new transaction type to revive um certain state that user wants so in the revive transaction um what user needs to do is to provide the key value and the period if it's a expired Leaf note or if it's a expired Branch then they will have to provide a Merle proof or some sort so yeah um we seem to be able to answer these two questions but is this really the best solution that we have if you really look at the before and after it's not really that effective because if you only look at the leaf note you are simply replacing the value with the hash and not just that for all the values that we have um in the try we are adding an additional metadata so in the worst case we can actually have the opposite effect instead of reducing State size we might have a scenario in the worst case where we are increasing the state St in sa because of all this metadata so this is not the best solution we that we can have let's look at another solution instead of putting the metadata the period to each of the values here we actually just put it on the branch node so the expiry rule is similar right but here for each child index we're going to have the excess period and the concept of period is still the same the concept of expir rule is kind of same as well it expires when a child it was considered last access for example a year ago so and what happens when a child is expired right same thing we replace the entire sub tree with the hash so just by looking at before and after you can already see from the diagram right you are replacing a bunch of notes with the hash itself so we are saving up quite a lot of space in in regards to the revive part is relatively straightforward you simply provide the path and the mle proof to the expired sub tree so yeah seems like a viable solution G easy seems like we got the best solution and this is actually one of the um solution that BNB chain uh raised last year I was part of that team we had a slight variance of this solution but the core concept Remains the Same but if this solution was truly viable why hasn't they implemented yet of course there are hidden issues um one big issue is with proof size EMP proof size is too large so when we have large proof size that means we inevitably increase block size as well increasing block size will have effect on network trut and network latency so in the worst case where um if a block is too large then the validator couldn't process it in time then we would make the network to become unstable and might potentially even halt it so in regards to State expir solution MP from all of the um solution that I have right now is probably not viable because of the the proof size so we have to look for the next one and thankfully we have one which is veral so I'm not going to go through everything about veral if you have no knowledge about it you can check out viral. info web page to know more about it or you can check out gom's talk later in the evening um to go through the veral structure right um You have internal notes which are similar to mpt's Branch noes and you have extension Noe which basically contains a group of values why veral is important just to go through it briefly it has small proof sizes so it solves the problem that we have it allows for stateless clients it reduces um Hardware requirements improves not syncing experience ZK friendly and more one major change is that we are for in workco we are using a single tree approach instead of double layer what this means is that we no longer have the concept of State try and um storage tries but we put everything in veral tree so without even understanding how veral tree works you can come up with a state aspir solution as well and we do the same thing by looking at the structure what would you think we need to add in order to answer the two key questions that we have it's simple we do the same thing by adding the period to the extension node because in veral tree one very special property is that values are grouped together so because you have this very nice property then it makes this um solution very effective and let's go through the flow um like before for the expiry rule the concept per is still the same it expires when the note the extension note was last accessed for example a year ago so we got our first question done the SEC um so what happens when um an extion Noe is expired is the same as well we replace it with the commitment of the expired node the commitment you can think of it as the same as hash of the MPT yeah and in regards to the revive rule what users need to provide in the revive transaction is to provide the path to the extension node the period And also the all the values in the extension node it's pretty straightforward and this is exactly what um we have proposed in EIP 7736 Leaf level state aspir in voal trees um um yeah so it's a relatively straightforward solution the advantage of this solution is first it's simple relatively simple second is it has clear gas cost um the revive PS are relatively smaller as well because you just provide all the things that you need in the extend and then it's also Backward Compatible with veral in the sense where if you choose to enable it together with veral that's good it's possible but if you choose to enable this EIP after veral it's also possible right so that's what it means by back compatible one major disadvantage is it only expires States partially because you can think of a scenario where majority of the leaf values are hasn't been existed for for a long time but there could be one single leaf value that has been access for right quite actively so in that case the extension node is not considered as expired so that's why this solution is called the partial State expir solution what if we want full State aspir solution well we have to go back two years ago to vitalic multiperiod Tre approach published in 2021 how it works is very different from the approaches that I've shown here so you have the concept of like multi period tree the concept of period is still the same um here what's different is that for every single new period a new state tree is created the expir rule here is that um the state older than two periods are expired so for a client software what they need to store at a time is really the state three of the current period And also the previous period okay for the revive part is more complicated than this if you look at the original article but essentially what we need to do is that we given an um expired account for example you will require the last known State Tre of the account then you also need to provide the proof of all the subsequent periods up until current periods minus one that's a very simplified explanation my problem with this uh proposal is that first it's super complex and that's why I believe that um the research in this direction has been stagnant in the past few years secondly it requires something called address based extension so address based extension by itself is already a very complicated um and controversial proposal because it brings many Breaking changes to our etherum protocol so we are literally stacking problems onto problems and secondly what I can um what I can foresee is data duplication because you are storing two state tree at once so in some cases or in even in the worst case you might have um duplicated data which again in the worst case you can actually double the state size instead of like reducing it so that that could happen and compared to the partial State expir solution you will have larger proof size because you have to prove different things now so yes we get full state aspir but is it worth it I don't know the future where are we hided next so we know that for State asir we get slower state goow or even eliminated we get reduce Hardware requirements and we get a more efficient execution client but to get those benefits we exchange it for worst user experience because now users have to care about Reviving their state they have to submit another transaction to in order to revive it and those transactions will incur additional State revive costs right and the another thing is that how do these users like normal users get those proofs from right the first part is definitely centralized providers um yeah but we don't want to relies um rely on centralized Solutions we have to come up with a decentralized solution and that's why in order to really enable State expiry whilst still providing a good enough user experience we need a decentralized p proving service such as the portal Network so yeah and we might not even go for vcal tree in state we might go for binary tree so if it go for binary tree then EIP 7736 wouldn't work um depending on how the binary tree structure that we would want to go for we have to rethink um another csir solution if it's similar to the veral tree one you probably work if it's totally different then we have to rethink it but if you follow through the thinking process that have shown here we can probably come up with a few variants so yeah by the way ethereum State just grew by 1.9 megabytes TR this talk so let's find a viable solution to State growth and let's build an ethereum that scales in time thank you [Applause] thank you so much so we're going to go through the Q&A now um and we're going to go for upvoted questions first um so why do you need State expiry if you can have stateless clients in the future and have a smaller subset of nodes store the full tree yeah so in a stateless world what would happen is that um even with epbs is validators will be stateless then only block Builders will have the full state but even that we are not solving the root cause which is the state the fact that a 80% for example um of the state is still considered as still or hasn't been used for a long time so State stess um stess Nur doesn't solve this root problem but State expiry does right we we want to eliminate it so in the world where we have state less and state asiry we can give a better experience to um even block Builders as well and make it more efficient yeah great um is State partitioning an option I think State part partitioning is a like um general term like I need to know like in details what what do you mean by state partitioning but because I don't know the details I will say maybe yeah um and would it be possible to charge a yearly fee to the account if it has no eth delete the whole state this would be a financial incentive for efficient State use so this is something like a renting model um it definitely is possible um but the thing about etherum is that if you are adding such a renting model you are changing the entire economics so it might break things as well and um in my perspective I don't think the community would be ready or would support such a feature um because it might break a lot of things and economic wise is so might not align to their incentive so it is possible but it's a social problem yeah and how about using snarks to compress proofs for reev revivals and do batch revivals um that's possible but um again in regards to how we want to use snars it really depends on how we want to um change up the the again the two key questions that we have like how how first how do you want to maure State um how do you want to I mean how do you know if a state is expired and secondly how um can we um revive expired state so here you seem to answer the second question but how about it first we have to explore that yeah um and the next one is kind of a question only right will alter the X node no not read um in our proposal yes because if for example if a read operation um changes the the period in the extension node you are essentially doing a right operation so if you are doing something like that you um you have to change up the gas cost for that particular read operation because it's read but it's actually right um so there are a few proposed solutions for example if if it's the first read on in the new period then we consider as for the gas cost we consider this as the same as right but yeah um there could be other potential other Solutions as well yeah what would be the incentive of running an expired State provider node expired State provider node so uh I'm guessing is that you um you have to provide you are running a service that provides the proof um honestly there are no incentive unless you're running like a you if you're a centralized provider um that's why I say um we probably have to rely on the decentralized service p network perhaps to be able to provide those proofs to everyone yeah good intro to the next talk what was uh was the 20% State turn measured on BC or mainnet and if mainnet have you tried measuring Beyond 14 million um is it's your main net measuring beond 40 million no because my note crashed are there any gas refunds at State expiry and gas costs involved while recovering State at a later time um in regards to funding or we haven't explored it yet but yeah that that would be a good direction to look at yeah how about archiving Old State compressed in cheap slow storage instead of expiring it if you're talking about um like a client implementation level is definitely possible but if you're talking about um protocol level um I foresee there are plenty of attacks that can that can occur so I'm not too sure about this direction yeah who is responsible for Reviving an old's expired value from the hindsight it's going to be a users but if you have like a sponsorship kind of model then the sponsors can revive it for you Eon attraction thing so yeah that would be ideal for the users but by default I guess the users have to revive it themselves and where does a node fetch a sub tree from uh when Reviving it from some centralized or for some provider not centralized provider sorry yeah from some provider yeah perfect thank you so much H thank you [Music]