Networking in ETH2.0 by Antoine Toulme & Rene Lubov (Devcon5)

so but before we start I'll talk a little bit about what we'll go over today so talk about a little bit about who we are at white black and what our role is in eath two challenges to networking and eath 2 we'll go over a recap of the Interop lock-in that we had in September and what our role was there and a roadmap as well as a Q&A at the end so white block is a team of engineers building tools to test blockchains servitude systems protocols what have you we have a core product called Genesis what Genesis you can fully automate the creation and deployment of a blockchain you can configure network topologies deterministically tests how they'll react in real world and worst case conditions we can do things like implement a network partition to force forking events and see how your blockchain will react and reconciliate that information and the reason why we're talking about it is because we will use its functionality to test interoperability and eath - so here we have everybody's favorite diagram the networking old paradigm right so sorry it's a little messed up we just downloaded to PowerPoint and didn't have time to fix this but we so you have a client currying server for information that it desires server acts as the centralized point of truth for that information right everybody knows this model that's my world here cool so then you have blockchain networking which is very very difficult you have a bunch of nodes in the network that are tasked with challenge to communicate over a large network and communicate with unknown peers at that so it gets pretty difficult in eath to specifically this means that you have nodes receiving messages of blocks and add to stations and propagating them to other nodes and they do this over an asynchronous communication pattern called gossip sub and we'll talk a little bit more about that later so nodes will receive a message and pass it on to their interested peers and the idea what gossips is that a message will efficiently propagate throughout the network once the originating node broadcasts it out so nodes will request information they don't have from other nodes so if a node has adequate resources it learns of a new block it will RPC request that the block or the node that has that information to receive it cool so there are a lot of challenges in distributed networking I don't know how where you are of them but when we listed very few of them but we're actively at web block looking at these challenges along with eath to client teams if two research teams and other teams in the space looking at them so there's the issue of peer discovery how do you as a node find two other nodes that you want to talk to in a large network the issue of distributed hash tables so dhts how do you store information about those peers in the network there's a issue of security how do you increase tolerance to certain kinds of attacks that are prevalent in distributed networks like eucalyptus attacks what have you the issue of block and attestation propagation for which we will use gossip so again we'll talk a little bit more about that in a second the issue of loot nodes in the network which are points of centralization perceived to be points of weakness issue of cryptographic verification checking to make sure that signatures are valid hashes match block information and then the issue of syncing so how do you how do you efficiently sync and securely sync information against other nodes in the network so I'm sure that you guys have already had like your ears talked off about it would be 2p at this conference it's been like a super hot topic at Def Con and if you want to go we're only gonna cover it briefly but if you want to go like more in depth about it roll Kripalani from the p2p did it talk about gossip sub like two days ago I'm sure the slides are already up on Twitter somewhere and the video will be uploaded but he did a really good deep dive into gossip 7 but we will we will cover just like very basically what gossip sub does so that we have a to start off at so gossip sub is an epidemic broadcast protocol it's an implementation of a pub/sub communication pattern and it's designed in such a way that information will efficiently propagate across the network so nodes will listen for information that they want and pass it on to their peers that want that information pattern will continue until all nodes that are interested in that information have received it and in the network so gossip sub is an efficient protocol for propagating blocks and at the stations in eath too which is why it's used as the eath to networking stack so back in april we actually did a few preliminary tests on the PDP and they were not so great but again like role mentioned that flip p2p is in alpha mode right now so there are a lot of parts of it that still need to be like massaged and and improved to make sure that it's going to be ready for Phase II our launch so we at white block will actually continue testing and auditing the p2p to ensure that it will be ready for a successful phaser and I'm gonna talk about that thanks for me so why are we testing us observe right so the point of this is to really assess the real-world performance of gossip sub under very adverse conditions dropping gears bad bandwidth packet loss huge latency between the nodes knowing that in the case of if - we have 6 seconds per block so if you want to really gossip everything you would want to have 3 seconds to go sip the whole information between all the peers so they can decide and start voting correctly on the new block one thing that we can reach week during that testing is a different network topologies so if you went to Randall's talk he said that you have a minimum of 4 peers a maximum of 12 is that really what we want to go for should we have more peers we have a few better peers should we start working a little bit on some of the working wants to do for episode where we're going to have optimizations where we're going to prefer nodes which are closer to us which have lower latency which are more responsive we're going to look into that so to get to work first thing we need is when we did our past name Pro we used to go to p2p demon this time we're starting from scratch with a company called the agency Enterprise and they're really great they work very closely with protocol labs so Adam Hannah has been building a leap PGP client which is based on the Goldie p2p core library and it's a very simple client that just is going to output a bunch of metadata information about the message right when was it sent who sent it who is it for how many hops DC for that message you know how long has it been around what type of like how long was the hop between those different peers we also hired an academic on our team who is now doing a literature review to look at the different topologies everything should be done there namely right now I'd like to try also to do some discovery you know associations so we can do academia approach to just let the network self organize but it might actually not be the best way to organize appears so we're testing nozzles at different aspects so talking about network topologies we're going to strain it or gain credit sequence of peers we're going to create a mesh of tears just like redundant hubs between each other so we're going to really see that you know from node 1 to 100 we're going to you have to traverse all the other nodes to get to that one so this is in progress you can check out those two repos the first one here is our project management repo this is all the issues kind of reflect the work from the team you can take a look contributions are very welcome your feedback in issues you see the work itself the benchmark tool is actually under agency enterprise it can take a look at it you can run it right now the idea is that we're going to talk right at work and then deploy it on Genesis we're getting around you know 3100 of those in parallel and we had some unexpected good news and of familiar but we got access to an academic facility to run all those tests something called amil abhinaya so we're starting testing actually this week you got a guy on this we're going to publish all the raw data sets for every test suite on Google Cloud Storage you can download those there you know one point five gigabytes each so just quite a bit of data and then we'll also share the Python parsing script that we use to read all the logs to make sense of them to compare all the statistics the median the all these data in will make that open source as well probably and there's a the PM repo so the idea is that we want to be completely transparent make sure that our work is being reviewed by the community and if you have an idea or a suggestion for a different type of testing you can come to us you can ask us questions we can have a conversation about this and we can come together and collaborate on all those things so to switch gears this is coming a little bit also from a workforce if you're m2 so just going to do a little recap on the interrupt workshop that we did in September here you can see these are stand up in the morning have Danny Ryan F telling us how things are going we were going by team here you have like different teams working together on working on the same laptops so the purpose of interrupt for us was to establish communications between the teams to make sure that they had a good process to work together coming in to interrupt we just wanted people to just take one laptop to clients make them work and see he was able to even just gossip a blog say hello have some information we had some great tooling from pato lambda from the Fenian foundation so he gave us a way to regenerate genesis states like 30 seconds or 1 minute in the future that everybody could ingest as well as the list of validators that would be associated with each client and then would give us the ability to very quickly set up this type of scenarios you know initially what the clients did was only one client in the test net with a validators everybody else would be passive and just ingesting old data and as we got closer to the end of the week everybody was able to very same time so everybody was processing blocks we had to get everybody going a good caveat of this is that solo jury because we had to kind of go with the flow of those different clients some cons did not support sync for example so if they studied after to Genesis time they would not have a good time working with the others right so you also to give you an idea it was very manual they would have a Tmax kind of window at first start first client look at the inno dove that client paste that into the start script of the second client start the second client start a circle i n't cetera et cetera very manual but you know for first try that was that was a good try right so we had a few panics we had a few interesting crashes we saw all the things that could go wrong and well eventually they managed to get some clients on laptop which is a pretty good result much better than we anticipated going in so yeah eventually what we want to do now is to actually productize this so recap of the success we had some clients talking to each other on a local machine a little harder for us we're using docker and now we want to publicize what they've done into weather is completely automated so for us it was a little harder because each client is ever so slightly different I'm gonna go into that the way we run things with Genesis is that we pretty much take whatever docker image are blocked in client has and we just automate its creation in its lifecycle as part of a platform so it's very easy for us once everything is in place to go very quickly to a destination to just generate many of them so things that kind of took a bit of time to figure out we saw that you know if even so everybody is using the p2p when you have an address for a client for a node in Limpy GP can do a couple different things you can have the public key of your client being published to all your peers or you can choose not to do that and when they connect to each other that's when the public key get exchanged right so in that case for example sometimes just barf because didn't want to have the public key of the other node in there so we had to start tweaking for that like night-hunting like that or should we have to change the start scrape just for them etc we saw also that they have different ways of storing the identity key the key that actually represents the identity of the note lead PGP by default is going to use a portable format which adds two bytes to the beginning of the the string that would store some clients like that as a exactly similar string being passed some like that as a basic ste for string to be passed and something like that into a file so we had to account for all those different variances of the way they would consume the information all this of course works very well when you're doing static peering at this point I think we have two clients that do this v5 it might be wrong so that's something that we need to look at next so that we're going to talk a little bit about that quickly for us it was kind of a great learning experiment working to generalize or framework to make it much easier to recognize those those clients without changes as much as possible so discovery I love discovery v5 I need to make stickers for this it's a big change from each one you know an if one you used to have just the URL being broadcasted to all your peers and you would have those kimia buckets where it would be organized in Discovery v5 you now have the notion of if am not records which are still signs or still making sense it's a little bit of security can do and handshake so you can kind of make sure not being spammed as much but what comes with in ours is now the ability for you to have in those key value pairs you can arbitrary stowing them a notion of topics so you can actually when you come in you know podcast you stuff to others tell that you're in if - node or you're an East 1 node or you'll poke it out node right so this allows all the block chains to also participate in a global DHT is nucleate immunity HT which is also simpler than it used to be so it's going to allow better security right so like six months ago French came to New York and one thing that I mentioned is that dc5 will only work if you have over thousand nodes running it otherwise the security is going to be compromised by eclipse attacks very quickly so this is a huge effort going to allow if one in if to to also share some of the work and to share some of the infrastructure so really excited about this is a network geek so next steps for us so we're recipient on EF grant we think we think if M foundation for helping us in sponsoring this effort we're going to continue working on some of the tests for SPECT conformance in terms of networking so we're going to be able to replay traffic send can request make sure that we're getting the response we expect across all the clients it's kind of at the unit testing level if you want and we've been talking along long last about the lipids because if sub testing so we'll continue eating that we want to kind of have a hand in this make sure that all the configuration parameters are kind of optimized for the workload of if to and we're pretty close to being able to do a lot a client test met with all the production ready clients does the idea there is that every time there's a new docker image for Trinity or lighthouse or Artemis or anything we can pick that up insert it into the test net try it out see if it's behaving correctly it's still able to find out the blog sources good stuff so not to put words into the if to clients implementers mouth but their heads down finishing up all the work all the details of each cases all the things that are kind of sticking out really stabilizing the implementations and we want to help them get the safety that they need to declare facial release and that they call your questions you're asking how how things went and how who started first is it a spec effort or is it uh yeah so it's it's interesting so there's been first like some min Devine talking about if two four years right so there's a lot of like different initiatives and efforts by vitalik eventually the research team around if to kind of solidified with Danny ryan kind of helping also get to a point where he could name releases of the spec right the spec was written by Justin Drake to be more in a table format so it could actually become part of the testing I think Pato Liam that really helped as well to really create testing and tooling around it and in parallel all the I mean the implementation I'm most familiar with is Artemis for example right and what happened is I had a lot of churn because lots of moving pieces came back and forth right so SSD for example at three or four incompatible designs to each other until the point that they agreed on all of this so every little piece of the spec kept changing as part of decisions down the road yeah networking is not done anything so they're a couple fees networking one is the wire protocol we're getting to a point where we have a good idea gossiping is actually pretty easy because you just use gossip sub just send all the blocks all the stations that you have Susana is not too much of a problem in terms of sinking the the team right now are talking about different sinking strategies if you familiar with if one and all the ways you can sink there are so many ways you can do this right it's still something that is an active research and our time is up so yeah there's more and there's no RPC interface for if two yet as Greg mentioned yesterday on this very stage when we did the first round of research we found that the median time was pretty low and pretty pretty good but we had uh pliers which were really really bad so it looks like the routing strategy is not always optimal I think that's the only hunch I really have at this point okay so how we scale to thousands of nodes in our testing scenarios yes so right now what we do is we do small-scale testing about 30 to 50 nodes our firstly p2p testing was 100 in required quite a bit of compute right now we are in touch with that research labs came to give us pretty much infinite resources across the u.s. continent so we'll see how to go about that we run docker containers so we're pretty agnostic about the underlying hardware we you know we run on virtualized cloud software like GCP Amazon Azure so the idea is we just add more and more machines about 100 nodes per machines to scale to that we try not to do the other islands of nodes from experience from practical experience we study notes you get already a lot of insights because we can we can really tear apart all the network connections like we can simulate a lot of latency we can actually have nodes being submitted between New York and Tokyo for example to really show what happens if the transaction takes a long time to even come back to you so second part of your question I'm gonna get killed by this guy yes