Presenting Parity: A Light Client for a Heavy Chain

[Music] hello okay great hello everyone my name is Robert Hobart Mayer I work at Paradis technologies and one of the projects that I worked on was a light client for aetherium if you're not familiar with Paradis basically will be billed an aetherium node so i'm gonna talk about one mode of operation of that node so first I'd like to split up the classifications of aetherium clients so actually we can split these pretty easily into three categories the first would be a full node so when you run a node today you're mostly just running a full node and that means that you download all the block headers and you check the consensus process and then you download the block bodies as well and you check that all the transactions have been executed correctly and that they're actually valid and this gives you basically full security you've checked absolutely everything you can but what we've seen is that as the blockchain grows heavier and longer it takes a very large amount of time resources both for storage and computation and memory to actually synchronize a full client and use it so it's not exactly user friendly in this sense so if you wanted to run a client on your laptop you couldn't necessarily run a full node so the other kind of client that we talked about is a light client so in a light client what you do is you verify the block headers but you're not checking the transactions and that means that you're checking that mining or staking has been done correctly but you're not actually executing those transactions to see if the output state is correct from the previous state and the new transactions that have been added and the last kind of client that we have which I'm not going to talk about too much today but is relevant to the protocol that I'm going to discuss now is a thin client so a thin client isn't checking the census process itself but rather has somebody else do that checking for them and checks it and and trusts that and then can fetch data about the chain in the same way that a light climber would so a light client has these security guarantees so it's checking the validity of headers it doesn't check the validity of state transitions necessarily and this might leave us open to some kind of attacks so perhaps a minor would introduce an invalid state transition and the thing is these kinds of attacks are not particularly likely because they have to be first of all targeted so you have to target a specific person and that's because of the assumption that we have that minors will only build on top of valid blocks or at least a majority of minors when we talk about the group of minors as a whole we say they're acting honestly so they couldn't necessarily perform this kind of attack on the whole network but on a specific targeted person they could if they took up all their peer slots and started sending them fake headers but we also have some kinds of routing mechanisms for example something called K buckets which are Cybil attack resistant which means that it's actually if you have an established peer slot it's very difficult for an attacker to fill up those peer slots and push your valid peers out so I would say light clients are very useful for low value or medium value use cases but of course if you're transacting millions of dollars you would probably want to have the security of a full node because that would that kind of activity would leave you maybe open to attack and those kinds of attacks just to clarify are really just about tricking a user into taking different kinds of actions based on showing them a wrong state so they can't actually steal money from them without the user taking action themselves so we have some network protocol goals so the first would be to minimize round trips in bandwidth because in a light client you are requesting data from your peers and if you want to run this on a mobile device or on low bandwidth you need to minimize those round trips you want to reduce that latency we also want provable data which means that you don't rely on any more trust assumption then the fact that miners are updating state transitions correctly and of course we have full clients who are serving the light nodes over the network and we don't want these guys to get dust we don't want them to get overloaded we need a sensible way to meter the requests that are being made to them so in the next section I'm going to basically discuss a few of the tools that we use to achieve this so the first is back referencing when you make a network request what you can do is put multiple requests in the same packet but further requests in the packet can rely on unresolved outputs of the prior requests so for example you can say I have a block number and I want to get the header for that block but once I have the header for that block I want the state of this account at that block and once I have that account I'd like to inspect some of its storage and this is actually very powerful tool because we can express very fluid kinds of requests in a single round-trip to the network which means that the response time of the light client is going to be much lower another tool that we have is mostly useful for doing things like checking the result of a transaction execution on the light client which is that what you could do is start to execute the code so you fetch the code from the network and then you see that the code execution needs to get some account so we go one round-trip to get that account and then back executable further and we get some more data from the state and you see that over time we're jumping back and forth into the network just to execute that transaction and see what the result would be but rather what we can do is get a full subset the subset of the state tree so the state is stored in a tree which is necessary to execute that transaction fully so if you are a client writer on a on a if you're a adapt writer for the light clan and you're calling this eath underscore call RPC to see what the output of some transaction would be this can be resolved in a single round trip on the light plant as opposed to many and it's unpredictable actually what kinds of data are going to be requested even if the contract state itself has not changed so another thing we're looking into is ways of expressing that a contract in fact doesn't do things like branch on the amount of gas provided to the transaction and that we can prove that if this contract state itself has not changed that the output will always be the same and that will let us re execute from this cached state proof over and over and over again until finally it does change that's very useful for reducing bandwidth as well so the metering system we use is something that we would call request credits so serving nodes are giving peers credits so these credits are pretty arbitrary each node comes up with its own pricing scheme which is based on data it gathers over time on how long these requests are taking to serve and then the light client peers which want to request data are sort of spending these credits to get things like block bodies or to see what the outputs of transaction execution would be and we could make this micro payment based in the future that's another active area of research because in fact in the future I think it's very unlikely that your average person will be running a full node yet we need full nodes on the network to be able to serve the requests to light clients so you need an incentivization layer and request credits although right now they're simply just time-based just provide a sort of foundation that we can build micro payments on to in the future and it's a very generic mechanism that we could plug a whole lot of different schemes into and of course these different requests have different costs so in the case of getting a transaction execution you're actually paying per unit of gas per unit of computation whereas if you're fetching a bunch of headers you're paying per header so another cool is publish/subscribe so that's kind of fun to bake into the protocol because when you're polling for things like log events changing and you that means that on the light client you may be checking bloom filters and things like that and fetching a bunch of headers from the network which may be false positive so in fact the full node peers know better than we do when something has changed so allowing subscriptions at the protocol level means that we can have a much more efficient light client the caveat is that it's semi difficult to catch out peers who don't publish when they're supposed to so for example if we're tracking log events in some contract and this peer is supposed to send us a message when log events change or log event is issued but doesn't it's fairly difficult to catch them out without doing some kind of periodic checking of the validity of data but if the network is mostly honest and the damage that they can do by omitting log events or such isn't that high it's very unlikely that they would do it so I would like to now that I have you all here make a little bit of a block pruning proposal so for example right now we have all nodes storing all the blockchain and that's a pretty high storage requirement and in fact we can we can do much better so one thing we can do on a light client is actually store almost none of the history and we can do this through a cool tool called canonical hash trees which will abbreviate to CHT and the basic idea is that for every and these so so death actually has a canonical hash trees but they work slightly differently and I can explain the difference but in parody what they do is every batch of say 2048 or 4096 blocks you basically make a tree like a Merkel tree mapping block number to the canonical block hash as the light client has seen so the my client sinks the network and it sees that in ancient history block one had and hash so we have a little diagram here and they can make this table and essentially condense this tree made of this table down into a single 32 byte hash so instead of a light client storing all those 2048 block headers it's just storing a single 32 byte hash and if we have a full client peer who has the same history they can give us a merkel proof of that 32 byte hash of the tree starting at that 32 byte hash to give us that block header even though the light client itself has discarded it which is a pretty powerful tool it means that instead of storing megabytes and megabytes or possibly gigabytes of headers we're just storing a few kilobytes no matter how long the chain gets even to the hundred hundreds of millions of blocks and the reason we choose to have separate trees is so that the light client can calculate them as it goes and furthermore that full nodes can recalculate them on the fly so the full nodes don't necessarily need to store these roots but when it receives a request for the canonical block hash of a certain block number then it can rebuild reconstruct that tree on the fly when it's going to serve that request so we can do something similar on full nodes so we already use this distributed hash table style routing in the network protocol although it's not really used for anything right now we expect that all nodes are storing all data which is an assumption that we'd like to relax so we have this idea of splitting up nodes by node ID so every node has a 160-bit ID or it may be a slightly different number but it's not particularly important exactly how long the ideas but we split up in the routing mechanism of each of each node we split up all the peers that it has into buckets prefixed by certain IDs so that means that each peer is trying to have a certain number of other peers whose IDs are distributed fairly you formerly across that ID space so in the DHT or distributed hash table world what we're doing is saying nodes with an ID close to the hash of the file we're trying to store are responsible for storing this hot file we can do something fairly similar for block pruning and say nodes whose ID starts with zero zero zero one are going to serve the first out of every batch of blocks so we can split up the whole block chain and to say batches of 2048 or 4096 and we can say okay these batches are gonna proceed in cycles of 16 or 32 or 256 and a node whose ID starts with 0 0 0 1 is going to serve every one out of 256 batches or the person whose ID starts with 0 0 0 4 is going to serve every 4th batch out of every group of 256 batches so the amount of space we can cut down is really determined by how many peers the average node is likely to have so if we have right now it's not really reasonable to assume that every node is going to be able to connect to 256 nodes but 16 or 32 is fairly reasonable and that means that you can always find a peer who's going to be able to serve you the next 2048 or 4096 blocks but those peers themselves have cut down their storage requirement by quite a large amount so I'm actually really interested to hear what you guys think about this proposal please find me after I think it's a pretty good idea so another thing that we can do on light nodes to get up to speed really quickly is warp sync so the way it works right now is that you have to synchronize the whole blockchain essentially you have to go from header 0 from the Genesis and walk the whole chain all the way to the head of the chain but in fact most of the time users don't care about that very ancient data so we can introduce something like warp sync to try and get users caught up to the head of the chain quickly and in the current formulation of aetherium you have this sort of thin-client security level as soon as you've warp synched but then when you download the ancient block headers later you can jump up to like a regular light client level of security so you have exactly the same security proposition as if you had synced directly from the Genesis but in future versions of aetherium most specifically IP 96 or in proof of steak that finalizes blocks economically you can jump directly to the head of the chain basically without sacrificing security and ancient block download might still be useful in that case if you want to see what the state of a contract was months ago but that can be done say on Wi-Fi on a mobile device where you warp sync when you first start the app regardless of your connection but you don't want to waste the bandwidth over data to download all those blocks all those ancient blocks so you wait until you're on Wi-Fi to finish that process and that's pretty a pretty useful way to do things so I'd like to talk a bit about some RPC pitfalls so if you're developing on a light client what works and what doesn't because we have all these RPC requests that seem fairly simple ways to gather data about the blockchain from the node but some of them under the hood might be doing a lot of work even on full nodes and on light nodes it's important to know actually which requests are going to be the most expensive so first of all getting logs can be very expensive especially if you're trying to get all the logs in the history of everything that means that if you're on a light client which isn't storing all the headers you need to then download all these headers then check bloom filters which may have false positives and then finally get receipts and and scan for logs so that's a lot of bandwidth whereas if you're just watching the head of the chain for log events it's actually fairly fairly performant but if you're trying to get historical logs it can be very difficult the reason is they aren't embedded into that Merkle tree of the state it's why they're cheaper but it's also why they're more expensive to for more ancient data so there are a few approaches that you can have so you could have you could have your peers search for you but again you don't necessarily know that they've given you all the data they were supposed to you can search locally which is what I just described which is to actually go through this long and involved process of downloading everything or you can keep some metadata so that's also another approach where the light clients keep some metadata that helps them search but again it raises that storage requirement significantly and there's not really a good way to bring that down and if we're trying to target say the mobile space where users don't have that much storage it may not really be feasible whereas on the laptop case you do have that storage and may be searching logs can be effective in that case another problem is getting stuff by hash so for recent data transactions and receipts you don't know anything but the hash if you just broadcast a transaction because that transaction may have not even been mined yet but the problem is hashes by design leak no information about the block number so if it's a block hash you don't know which pier to ask for the the block whose hash is that transaction hashes are even a little bit worse because you have to find out where in the blockchain that transaction hash occurred and there are multiple different Forks where that could occur and actually with one of the new IPs we can have the same transaction exact transaction included in the chain multiple times which just means even more work for the light client to check and for checking these transactions and receipts it means that the full nodes have to maintain this fairly expensive index mapping transaction hashes to their location in the blockchain and lastly estimating gas so these state proofs that I talked about before can be used for estimate gas and eath underscore call but the thing is with estimate gas the typical approach is to binary search to find out exactly how much gas a transaction will take so you start at some very high upper bound and then you try at a lower bound and then you sort of narrow and you zoom in on exactly where that transaction begins to succeed the problem is that in the general case contracts can branch on the amount of gas provided so that means that the same state proof that you fetched for the upper bound case may not be enough to re-execute on the lower bound case and that means you might have to take more round trips in general so this is something to be used fairly sparingly only when really necessary and also think about the contracts you write so that they don't play poorly with light clients and lastly I'd like to take a little bit of time to talk about the light client plus whisper ecosystem so we've seen that light clients can be very useful in mobile devices but then there's also things like whisper which people are going to talk about a bit more which is for decentralized encrypted messaging essentially with darkness as well so it's like a global publish/subscribe system where people can't actually see what topics you're listening for so that actually makes for some pretty interesting platforms like status to emerge where you can have gaps running on your phone that are messaging with people doing medium latency state channels it could even be used for example as the micro payment for the state channels micro payments layer of the light client request credits and of course we have a roadmap so this is actually something you can mostly use today so it's in our beta releases you can just run parity with the dash dash light flag the UI is still being rebuilt around light client best practices because it was written mostly with a full client in mind but actually we're undergoing this rewrite basically now but if you access the our pcs directly or if you're using the Chrome extension and you just have some DAP that's running independently if the UI completely it basically will work I even registered a few names on the ENS using it it does work in that case if you have peers and another really interesting project that we're thinking of that we're starting to do is compiling the light client down to webassembly so we're writing everything in this language called rust which is very low level it's it's that roughly the same level as C or C++ which means it has no garbage collection or runtime and that makes it very amenable to being compiled down to web assembly and that means that we could have this same light client protocol implementation storage embedded directly into a browser window and I think that'll be very interesting for to see how the app developers use that and if you'd like to follow the progress I would recommend you do so at the parody tech slash parody repository it's all open source we accept contributions I know you guys are developers and you love open source better we get writing well thank you very much [Music]