# P2P Networking in Ethereum 2.0 by Jannik Luhn & Kevin Mai-Hsuan Chia (Devcon4)

- Speakers: Jannik Luhn
- Channel: [Devcon](https://streameth.org/devcon)
- Date: 2019-10-09
- Duration: 21:39
- Topics: blockchain, smart contracts, distributed ledger, Science & Technology
- Watch: https://streameth.org/watch/yt-rMo67_5gA_s
- YouTube: https://www.youtube.com/watch?v=rMo67_5gA_s

## Description

Visit the https://archive.devcon.org/ to gain access to the entire library of Devcon talks with the ease of filtering, playlists, personalized suggestions, decentralized access on IPFS and more.
https://archive.devcon.org/archive/watch/4/p2p-networking-in-ethereum-20

Sharding and Casper promise to greatly improve performance, sustainability, and security of the Ethereum blockchain. Alongside a novel and much-discussed consensus protocol, they also entail fundamental changes to the requirements and constraints imposed on the peer-to-peer (p2p) networking layer. In this talk we report on ongoing research as well as the current implementation state of the latter. We begin by defining key qualitative and quantative properties the network should have. Then we discuss several options for both node discovery and gossip protocols, comparing their performance on the basis of simulationresults. Finally, we give an update on the current state and future developments of protocol implementations.

Speaker(s): Jannik Luhn, Kevin Mai-Hsuan Chia
Skill level: Intermediate
Track: Scalability
Keywords: sharding, casper, security, node, gossip

Follow us: https://twitter.com/efdevcon, https://twitter.com/ethereum
Learn more about devcon: https://www.devcon.org/
Learn more about ethereum: https://ethereum.org/ 

Devcon is the Ethereum conference for developers, researchers, thinkers, and makers. 
Devcon 4 was held in Prague, Czech Republic on Oct 30 - Nov 2, 2018.
Devcon is organized and presented by the Ethereum Foundation, with the support of our sponsors. To find out more, please visit https://ethereum.foundation/

## Transcript

let's begin my name is yannick I'm from Rainbow and I'm here with Kevin from the theorem foundation and we're going to talk a little bit about Italy and networking in issue in Prato or how we call it now in serenity I will start and give a little bit of background and try to introduce you to the problem that we are trying to solve and then and also show some simulations to see if our approach makes somewhat sense and then I will hand it over to Kevin who will talk about the implementation of all that stuff but let's start with the very basic things our gods of networking what we actually want to do we have some notes in that produced data and others want to obtain this data so we need a way to distribute this and that way the way how we want to do this should it should be efficient it should be fast and it should also be safe what kind of data in in the original theorem it's mostly transactions and blocks and the important thing is that all notes basically download the same data so there's no distinction at all basically all notes do the same thing in serenity things gets different mostly because of shouting and shouting means we basically split the data up into different charts and one particular node only downloads the data for that particular shot so and that way we can save a lot of bandwidth I plotted in this picture it's 8:00 and reality it's a thousand but it's not too many to to draw on this picture but just to yeah in addition to this shot data we also still need some kind of global data that are notes download this is the beacon chain data and from that we can basically infer how nodes should connect to other nodes in original aetherium all nodes basically are the same so one node simply basically picks random Lee from the set of nodes and connects to them in serenity things will be different if you pick peers you first look at what shot you're assigned to or what shot you want to download and in this in this example shot three and then you pick peers from that shot and so that you they can give you useful information so here we have three additional notes from a sharp three and then in addition to that we pick notes from the global set that can provide you with beacon data in this example one and seven and then we can basically abstract from that essentially what you do is we now have two networks and then one is the sharp network which consists of all the nodes that are connected to a certain shot and are connected to each other and then with the beacon network that consists of all the different nodes and in this network only the beacon data is transmitted now another thing that's different to the existing ischium network is that we have a new class of nodes called the validators and they are quite similar to normal nodes in that sense that they are interested in the data that concern belongs to a certain chart but in addition to that in what's different to them is that they only care about the recent history so they don't don't don't download and like the whole chain up until Genesis but only the recent history and another thing that's different is that they switch regularly between charts because they basically get assigned to a new chart which they are supposed to validate randomly and a third thing and that's different is that they want not they want not and that the rest of the network knows that they are billy data or or which really did have they are because they stakes and some ether and so they want to have extra protection and if people would know that they are validated they might get attacked now that's basically all in all the changes we have and now we need to think about what protocols we have basically there are three functionalities and the first one is we need a way to find suitable peers and that's done by a discovery protocol the second a second thing we need to do is to distribute data and we do this using a ghost protocol and finally four new nodes that join the network or validators that I signed to a new network a new shot then need a way to is synchronized in the chain on the history of the chain using RPC calls and I will not talk about the last thing because that's basically the there's basically no difference to how if you wondered or does it but I will focus on gossiping and the discovery protocol I said what's gossiping gossiping is a very simple peer-to-peer protocol that's being used in a lot of projects including Bitcoin and including if you own one at all the idea is we have one node in the network that lets produced some data and it wants to inform the rest of the network of that data so what they they do is they pick a random peer and send this data to them and now we have two kids that know the data and in the second round they basically do the same thing again they pick randomly nucleus and sent the data to them they and do this a couple of times and after a few rounds the whole network knows the data and that's a very simple and yeah a static protocol that's also very efficient and this process is also very fast and safe in case of network failures we want to apply this in in two different settings gossiping once in the shot networks and once in the beacon network and the difference between these two settings is that in the shot networks be only a few nodes because they distribute themselves over the whole set of nodes distribute themselves over the different shots but we have a comparatively high throughput so a lot of data is transmitted there and in the beacon network we have a lot of nodes because all of the nodes in the network participate here but it's much much less data so we did some simulations to basically answer two questions the first one is can the network handle is handle our and the numbers we have or the the settings we have and how fast does it is the propagation how much time does it take to propagate data and to do this we implemented we needed in implementation and reuse gossips up and that's been designed by a lipid B and we which will be probably or most likely will use in the end and we implemented that and then another thing we need if we do a simulations we need to make sure that they represent reality in some sense and in particular we need to know what the the connections of the net the nodes have which each other what what bandwidth they have and unfortunately there's a scientific paper that has measured this and we just picked those numbers from the current if your network and here's what we got we basically looked at and how much time it takes to the propagation of a message through the network and over time so at the beginning time zero no one knows networks no one knows the message the network the message is created at time zero and then it starts to be transmitted or propagates through the network and at some point it reaches one meaning that the the whole network has seen the message and we did this for a couple of block sizes one thing I forgot this is for four shots so for a single shot network where we have maybe a thousand nodes and the numbers we get here is so first the first thing we see is that it works so it's not like it takes forever or it does never reach this one messages are always transmitted and the larger the block size of course the longer it takes but for a small block size of one or two kilobytes maybe it takes two seconds and for larger ones it gets longer and longer but even one megabyte is still eight seconds now another thing we should look at is what bandwidth is actually used and we see here and this is a histogram of all the nodes and how much data how much of what fraction of their beta bandwidth they use we see that most most nodes use about 20% of the bandwidth and some a little bit more but no nodes use more than 60% of the bandwidth this is important because it tells us basically two things the first one is that we're not operating at capacity even at large box sizes so if there were notes that use a 100 percent of the bandwidth we would be we could be pretty sure that they will not keep up in the long run and this is not what's happening the second thing is we see that there's a lot of bandwidth available if we get new nodes that join the network and download a new data and charts the download the data because this only simulates the propagation of data that's created newly and not synchronization of chain chains and so on so that's good and now we looked at the beacon Network where we have a lot more nodes I simulated here 100,000 nodes and we see the same thing basically at time 0 a block is created and it propagates through the network and at some point it reaches all the nodes block sizes are smaller here 64 kilobytes to 512 kilobytes we will see how large they will be in the end and here even though the network is much larger we still get propagation times of a couple of seconds so this seems to work very well and we're happy with that peer discovery that's more complicated and the job of peer discovery is to find peers in the beacon network and also to find peers in a specific shot and this is the challenging or one of the challenging points here because there are a lot of shots and if you just pick a random node in the network you need to pick a lot of them to find one that's suitable to you so we would like to have a way that's more efficient than that we also need this also should be very fast so in that validators who is assigned to a new shot can start operating as quickly as possible so that at that time is as short as possible and fourth requirement is that the valid data as I said earlier they would like to be private and if it's easy to discover them using this discovery protocol then yeah this is not good we are considering a bunch of options here and three seem to be viable the first one is discovery version 5 it has been designed by Felix from the film foundation to be used in the existing if you network but it actually has some nice properties meaning that we can could maybe use it for us as well so and it looks very promising I think second one is a simple variation of Kedah million and the third one is to simply use a global gossiping channel to propagate the shots we the yet the sharp preferences in some sense which seems kind of boot forth but it might actually be viable but we're still not finished with that we're still evaluating yeah but that's it from my side and now Kevin will talk about implementation and I'm going to introduce p2p meditation status on our site so this project them Charlene pee-peed of a POC and it is for for a concept of the current design for if you're in turmoil it'll be there and we implement it using the p2p so what is the day p2p it is library that has many useful peer-to-peer networking components so you can choose the components you want or you need to view your own peer-to-peer applications and currently we're using the TCP cumberlands and kadam layout DHT and pops up and the goal of our project is we want to see if our current design meets the needs of ether and 2.0 and we also want to see if the able to be faced our needs and it can also serve as temporary layer for if you're into foil network a Python implementation until the Python let beta P implementation is rate so the requirement for our networking layer for if you're in tutorial the clients should be able to subscribe while to one or more shards so in this graph that clients subscribe to to shards wines flagon one is red and the client should only receive the data from the shards it has subscribed so in this case that client should only received the plug-in back and read instead of the blue one and the time to subscribe to a char should be sure which means a node should be able to find the sharp ears in a short time so the design currently we mapped shards to topics in pops up so the concept of the pops out is subscribers subscribe to some topics and they should only receive that data published to those topics in this way each have a forms a separate channel so we can segregate the chars with topics and about discovery we use puh notes for you notes to join the network and to find the initial peers and we use Kadhim the Abbott DHT and we for to discover the sharp ears our approach is we have a global topic for notes to broadcast the chars they're currently subscribing so if a know wants to to find appear in a specific shark it has e it can do it through subscribe in the topic and we're still exploring other options and each node provides the obviously for other nodes to request for data so currently we support the request collation collision is the block in the shark and we're going to change to use the epitome demon so what is the pdb demon it is capable of supporting liberty across languages and you if you want to use it you need to implement the bindings so as you can see in this graph the left hand side is the B demon and it is a standalone process and in handles they to be components and you can control the demon through the UNIX domain socket and in currently it supports multiple methods so I didn't identify you can get the peer ID from the demon and you can connect your other peers and you can open strings to other peers and you can set up you can register a function to handle the incoming strings and the DI DHT operations and the hops are still underway and this graph shows how we will change our structure so in this graph the blue ones means the part we need to implement and the lemons is still the goal and right hand sighs - so currently we implement our large Expo both in gold and Python and we we handle the communication through G RPC and we're changing to use the p2p daemon and in this way we can move off our logics to Python side and we need to implement the Python bindings and after party PHP is ready we can use it directly so all the logics will be off of them will be in a zone sign and about the implementation status so we have finished the essential functionalities so including the joining subscribing to the chars and broadcast data to the chars and request messages and we have a gobo topic for discovery and condom validations and we have a tracing for the testing and we have the findings of our call for python and was in progress so um the white bakhtin they're supporting us to do the testing with networking they were network emulation and we're still doing our own deployment and testing for a testing and also we are also implementing the python the p2p diamond bindings and we still have to finish the care management and reputation mackensen and do the further optimization for the operating and we currently will have corporations with original interim p2p designers and got a lot of advice from them and protocol labs they support us on the a p2p and pops up white block how busted - to do the testing things and that's it so I want to give credits to FedEx in from Paris and Anton from EF they give us a lot of instructions and my a great word from my colleagues thank you um if anybody in the audience has any questions well there is one right now was the propagation data you showed and ventilation data based on testing on a local area network or a global network and either so it was it was a simulation it was running all on one machine but we simulated the the latency and the bandwidth between these nodes based on global yes my question is I know there's the pr2 the dev p2p implementation and now there's the 2.0 my question is if the previous implementation is kind of going to everything is going to be switching to nip it to be or there's going to be different areas there so in the dev p2p right now there's no lip b2p implementation there's CAD Emilia for discovery etc etc but in this version you're showing that lipid 2 piece kind of everywhere yes so so our current plan is to use just you sleepy dopey and not use the FPTP for the Serenity stuff you going for the go implementation currently we have a grant for the Python API implementation so after it is ready because we implement in Python so after it is ready we can change it to that so other and this is why you also have the the UNIX the daemon right yeah that email is to stop this problem so you can commit for different languages they can use the daemon and without implement without the actual they paid to be is implemented thank you thank you are there any more questions from the audience are there any comments from the audience who they like thank you like you guys alright great that's it round of applause thank you [Applause]