Practical Applications of Off Chain Computation in the Light Client

hello everyone I'm George featherly and today I would like to talk about talk about some potential applications of offering computation in the light client ecosystem maybe you have already heard about octane computation and interactive validation these are really promising and exciting new concepts but the basic idea is really simple if there's a complex but deterministic calculation that would be too expensive to diversity process on the blockchain it is still possible to have multiple parties evaluate the same function check the results against each other and only start an interactive validation process that could prove one of the parties wrong if there's a disagreement and this validation process can be realized inside a contract so eventually the results of such a calculation can be canonized on the blockchain this technology can be used to process large amounts of external data but which could be located anywhere like a swarm but in this case I would like to present some use cases where the input data is a blockchain itself either the same chain where the validation happens or can be another one to these use cases are mostly related to even filtering in a current single blockchain scenario in even filtering is quite simple we have the contract log events and bloom filters but in order to do a full history search with ghost world performance we already needed some clever performance optimizations which I will shortly talk about later I would also like to talk about the future challenges of even filtering with light clients we are going to have a sharding and stationer technologies like plasma or polka dot and eventually we'll end up with a massive hierarchy of chains and massive amounts of chain data and we will definitely need some sophisticated filtering methods to make sense of all this data I suppose let me just quickly talk about the current filtering system in the go ATM client you probably know bloom filters they are really simple data structures just a 2048-bit long bit vector and for every log address on topic three quasi random bits are set and if someone else is looking for the same events later they can just check for these three bits if they are set and only check the work receipts if bits are set and this is already a good performance improvement compared to checking all the block receipts in the entire history but still checking checking them means that you have to read the entire header chain which is already more than two gigabytes and it was kind of slow even on a foo node and with a light client it is even worse because downloading the and keeping the entire hydrogen locally is something that some devices just don't want to do cannot do and that's also why we implemented the checkpoint syncing so that we can avoid downloading all the headers but then we don't have all the movie theaters either so we needed some flavor travel data structure and what we did is we take a fixed length sections of consecutive blocks and put the bloom filters of them under each other imagine that there's a bitmap and you can see on the left blue box then when doing a simple search we are interested in three vertical columns in this bitmap which is still if you read this you had to read all the bloom filters from this because it's a that ties it back to the interesting bits are not tightly packed together but if we do a 90 degree rotation of this bitmap we will get horizontal lines horizontal lines will be interesting for us just three lines out of the mm so we only have to read those and this optimization already used a two or three orders of magnitude performance improvement it looks searching and it also verse nicely with life science by the way this requires the second version of the earliest protocol which has just recently been merged into the GUI terraeum code base and and it also works very nicely it can filter the entire history in a second few seconds but there was another problem we had to solve in order for this to work namely that these data structures are not a part of the consensus so we needed some so a nice thing is going to directly validate it even though lightsabers can generate and serve it and in order to solve this we created a special try the bloom filter try and organize all the bits between this try so that the light client only needs to know the root hash of the bloom filter try and use math to validate everything else of course the question still remains how a light client can trust the bloom filter root hash computer train route - and currently we are doing checkpoint syncing with hard-coded trusted checkpoints which is only a temporary solution and right now the bloom filter tries also hard-coded into this checkpoint but soon we would like to get rid of hat for the checkpoint and use trustless validation of checkpoints and in order to do this we need to somehow validate the bloom filter try on chain and this is where of chain validation comes into the picture because all the light servers know the input data which is the header chain and this is a deterministic calculation that they do anyway so servers can send the route - new route hashes to a judge contract and only do well - and if necessary if a route has remains unchallenged on the chain for quite good Afghan time then the clients can trust it of course we still also need some security deposits and other incentives to make the system work but that's also part of the plan now let me also talk about how I imagine the future challenges the massive chain higher his and scaling will mean for life lions and what we can do about it here's an example situation where there's a user who wants to observe a subset of the change in hierarchy which may could still be a quite a subset right now our imaginary user uses a decentralized the marketplace which has a multiple state channels for listing different market offers like one for listing crypto versus the administrator first and other for listing secondhand cars and the new service which has a state channel for weather alerts and the user wants to observe this change and get notified about interesting result and if filtering criteria for interesting events might be even more complex than what could be realized with our current simple log address and topic systems so it would be great if clients could somehow get some help for filtering also it's possible with some station or technologies that you will all redone that use law redundancy that their security security model is based on an assumption that some interested parties actually do validate the chain and that's also something that like clients cannot do so they would also require some insurance to be able to trust the validity of these chains I think it is possible that they can have the higher right servers to do all of this for them and also it would be even better and still possible to build a filtering and observing hierarchy for each client who is running complex applications this higher he could run on multiple light servers and the deliver just the interesting results from the entire computer to a clients who can them run with very low resource requirements I would like to show too I would like to show how I think this is passable so let me just define two very simple primitives that could achieve this one of them is called the chain theater a chain theater is a deterministic set of operations performed on an input block chain and it is specified preferably in a virtual machine that is suitable both for just-in-time compilation and interactive validation and hm theater can have its own state but not its own consensus mechanism because a chain theater blocks are deterministic functions of previous chain theater blocks and new import blocks so whatever consensus marries in the input chain uses the chain theater will just follow and these chain filters can be used for realizing user specific filtering criteria and so these are also costs use cases of efficient computation but alas the bloom filter try they are user specific and the other primitive I would like to show is called the observer chain an observer chain belongs to a single node a single ice light server and it is also validated by a single signature and what an observer does is that it processes multiple observed chance and create observer blocks that contain the new latest heads of these chains and it observer chain is also backed by a security deposit at the a judge contract and the observer has to defend the validity and availability of these chains or at least the latest action some of them on the request the observer chain can follow public chains private chains state channels chain theaters and either even other observers chains and now let's see how we can realize a filtering and observing hierarchy with these primitives here's an example scenario where there's a public chain and two state channels and the user has its own change filters defined for all three of them which I called my event and these chains and chain filters are processed by hired servers servers one two and three in this case who gives certificates about the validity of the input chains and also the results of the users chain filters may be in some case it would be enough to just hire a few servers and collect the results but it is possible that we have such a huge hire he had been driven to observe so many chains that we might need additional layers of servers in this case they are servers four and five who are observing the observer chains of service one two and three and run their own chain theater could collect events that could filter and collect all the interesting events for our client so they find can very conveniently just contact the last layer of helping servers and receive just the interesting results of course we need some redundancy to make sure this system works correctly so if we have multiple paths leading to every interesting chain or chain filter then the client can always detect if it receives different results from different directions and then it can try to investigate the the observer chains and the three filters of the entire service or maybe raise an alarm and notify about other clients about a suspected fraud and eventually if there seems to be a fraud and start an interactive validation process to try to prove it all of this is of course quite far from the original idea of the right client but I believe it still fits pretty well with the philosophy of what the original light duty call is based on because the idea is that in this massive computer ecosystem there are nodes with very different capabilities and a small and it is we always try or have from bigger and it is and what we can do to avoid the concentration of power is that we can create a standardized protocols that make the bigger happier and it is interchangeable and therefore create a liquid market of services so that whenever someone wants to stop providing services or raise the price is very high than other servers can just take over and no one can stop them from doing so so we can ensure a continuous service with reasonable prices and and also we can still provide security not with the Mercure proof mirco proofs are in the classical I try and but we can still increase the cost of an attempted fraud by using security deposits and also reduce the risk of a successful fraud by using multi-purpose and detecting any attempted fraud right away of course the there are still some details that are not worked out and right now our main development priority is still the making the classical light client reliable and usable but when setting the directions of a new development it is always important to keep keep the future challenges in mind and the long-term goals and I think the long-term goal is a massively scalable high performance per computer that even small embedded and mobile devices can safely access and I believe that in this massive in addition to the massive chain hierarchy that provides the global consensus the the client specific observing and filtering higher he is belonging to these unique perspectives of some applications will be equally important in this ecosystem and I hope you found my own unique perspective on the future deuterium interesting and thank you for your attention you [Music]