Evolving devp2p

[Music] hey guys so many people here yeah I'm Felix what is it should I use this okay hi guys so I'm Felix I work for a theorem foundation on the Goya theorem client and my role there is mainly bug-fixing in feature development but I guess there's a lot less time for new features lately and my passion and this project has always been you know like taking care of the way that nodes talk to each other and this is also what I'm talking about today hmm so death p2p wasn't came into existence I guess three about three years ago and at the time the vision was to as you can read there provide the lightweight abstraction layer that provides low level algorithms protocols and services in a transparent framework so you know like it was a pretty grand vision in 2017 though deaf p2p is just this thing that you need to implement to talk to the ethereum blockchain and it's part of all noni theorem implementations so all of the 6-7 implementations that are alive on the network have an implementation of def PDP and all of the stuff that's in it and there have been very few actual protocol changes since 2014 so if you wanted to you could count them on one hand basically deaf p2p has a bunch of elements to it so the first one is the no discovery protocol which is a way of finding other nodes to talk to then there is the RL PX transport protocol which is what it's spoken on the on the TCP connections between nodes and finally there's an application layer protocol that sits on top of the ROP X transport protocol and this one is somewhat confusingly also called FPTP so both the overall system and this particular protocol are called FPTP let me just walk you through the protocol that's in use on the network today and then maybe like we'll come to the part that can actually be improved about this so this dark circle there that's that's us so that we're node that wants to connect to the etherium Network so how do we do that so we'll join the DHT first and the DHT is basically the part that is this thing where you can find the other nodes that are on the network yeah so there are some other nodes they are also registered there and then basically we walk to the HT at random to find someone to connect to and then we try to establish a TCP connection to them and the TCP connection might actually fail so this happens quite a lot because the node you know might no longer be live or you know it might be too busy handling all the connections but let's just assume that it works this time so once the connection is established we actually exchange capabilities with the other side so basically this is now the part where like the DEF PDP application layer kicks in and in this particular case we see that the shared capabilities or is it is just one shared capability and it's it's the eath capability in version 63 so now once each version 63 is running we can exchange information about a blockchain that we're both on and once that matches we have a new Pia yeah so that's the current system and it's not super efficient and well there's a whole bunch of details that I haven't really talked about but you know I I guess you kind of get the idea so what can we improve about this though so first of all you know it's kind of annoying that there are so many round trips just to figure out whether someone is on you know the right blockchain I mean I guess that's kind of obvious and then it would be really nice to just sort of know that like before even connecting maybe and then another issue is that the whole system is basically frozen so making any change to any of the protocols requires like really tight coordinations requires implementation consensus and any change we make needs to be backwards compatible and to achieve upgrades at all what we've done in the past is we we've made all the upgrades backwards compatible and we've tied them to a theorem main that hard Forks because well everyone has to upgrade their node anyways and then once the Heart Walk is successfully launched we can start phasing out the old stuff and just you know only speak the new protocol but you know hard folks don't happen all that often and we'd rather you know make changes on an accelerated schedule but it's like really not that possible and then finally because no discovery only relays information about the oral TX protocol there's really not there isn't like any sort of room for experimentation because well we essentially stuck with our lpx and the crypto system that it uses so improving these things is the no discovery version five effort and with no discovery version five we want to make two changes in particular or rather we want to achieve two things in particular so the first one is we'd rather you know be able to find notes more efficiently and then the second one is we'd rather know more about those nodes before we even connect so the first part of our solution to these issues is called E&R which stands for aetherium node records so earlier in the v4 overview you saw that the DHT holds all those a node addresses and then a node address is really just a public key and IP address in two ports if there are m node records can hold arbitrary information about a node so that's the main difference so this arbitrary information that can be information about the capabilities of the node it can be information about other transport protocols spoken by the node it can be initial key material follows for those transports anything really like as long as it fits into 300 bytes we're basically like almost anything can be relayed there and the limitation of 300 bytes is important because E&R is a separate format a seperate spec even and it's not at all connected to the DHT so you can relay those records through any other means if you want to including say a DNS record or something like that and then finally node records are signed and also versioned so if you have two versions of the record that described the same that describes the same node you can determine which one is newer for example and we think that you know is a good solution for the transportability problem because again information about arbitrary transports can be related to this protocol and well yeah because there's just a lot more room to put you know information about anything so we still need implementation consensus though because you know in order to be able to talk to everyone everyone kind of has to agree on you know what the language is that they're speaking to they're using when speaking to each other and it is very likely that for a considerable amount of time the lowest common denominator will be our lpx but eventually one CNR is launched we can actually try out different transports find a viable alternative and then you know maybe like at the end of 2020 like delete or LP access so in order to get in our launch though we need to upgrade the discovery protocol this is something that needs to happen once and it will be a backwards incompatible upgrade so likely what the way this will work is that the discovery version 5 DHT will be a totally separate DHT that will run in parallel to the current system and because this is kind of a unique opportunity for us in addition to you know just including support for en our we want to make a bunch of other changes to the protocol in particular one problem that the v4 discovery one problem that that v4 discovery protocol has is its reliance on absolute time so maybe you know some of you have actually experienced this so if your clock is off let's say bye-bye to minutes you won't really be able to connect to the network and we've worked around this by like alerting users when the clock seems off but that's a really ugly workaround and you know like we might just fix it in the protocol this time and then with many of the notes in the DHT you'll find that the information that's listed there isn't very accurate so you might you know not be able to connect to this node at all even though it's listed there and in v5 we want to introduce this concept of endpoint proofs where the DHT ensures that if if nodes record is listed in the DHT you there's a pretty fair chance that you'll actually be able to talk to it and then finally we have another we have another improvement which is also which can be considered an extension of the of the DHT protocol and this is the this one is about finding notes more efficiently so in a classical THC so to say the THC is an index of nodes by by their public key and it maps public keys to know the endpoints but well so the nodes in the DHT though they're not in any kind of useful order so you'll find an if there are may not note next to an a theorem classic node next to a node that doesn't really participate in the ethereum blockchain at all and there isn't really like an efficient way of like knowing about you know just in those that you care about and contrasting that the the topic index that we have in mind is sort of like in index of all the nodes by by that by the topic or the service that they are providing I don't have a lot of time today to really go into the detail or how this works but I can give you an overview of the design constraints that we set when when when making this protocol so the first big constraint that we had is that we don't want to split up the DHT because the DHT is bigger is better right I mean fundamentally the security of a DHT depends on the number of participants in it so you'd want to like have always have a very large number of participants and another design constraint we had is that these topics kind of have to scale to an arbitrary number of participants so there might be topics that everyone is advertising like everyone who's in the whole network but then on the other hand there might also be topics that are very small and only advertised by let's say five or six different nodes and these topics shouldn't really compete with each other so you should be able to resolve both equally quickly and then finally with all of these systems there is always like the danger of you know people spamming it with like arbitrary registrations that nobody ever cares about and those particular registrations should not drown out you know the actual useful ones that you really want to see and to combat many of these attacks the topic advertisement protocol includes this thing called advertisement inertia so this is really just you know an artificial delay enforced by the protocol before a certain registration for topic and go live and we feel that you know in addition to maybe reducing misuse of this in addition to you know combating these attacks this also reduces the misuse of the topic index because fundamentally topics are meant to be used for announcing big decisions like way ahead of time and those big decisions can be something like which blockchain you're on were which short of the blockchain you're on or well you know things like that and not so much like let's say the URL of the video chat that you know you're just starting so there's a we really want those things to be used you know like four four four four facts that so to recap the no discovery version five is about finding notes more efficiently and knowing more about those notes before we connect a prototype of the system although without the ENR has been in use by The Gaslight client since early 2017 we're still working on the e IPS so nothing is published yet but once it is we will have like a separate spec for ian's or a document that goes into detail about you know like the semantics of the topic advertisement protocol and finally a description of the actual wire protocol that spoken via UDP but there is really nothing set in stone yet so if you feel like you know there is a certain change that absolutely has to be made or a certain feature there to really be included were not included just come talk to us [Applause] [Music]