8. Discussion: Sharding P2P networking roundtable

hello everyone welcome to the sharding p2p roundtable session and this session is going to discuss about the Charlene PV requirements and the implementation and hopefully we can sell down to a single implementation and start working on it so agenda first I'm Kevin from a theorem I will introduce the shutting people requirements and their current idea first and then Lars from protocol ABS you will talk about the p2p and Casas up and then after that we will have a roundtable discussion so about Charlene PV clients requirements the first one we need a client should be able to subscribe to multiple shards and you should be able to jump jump between shards really fast and so it also means that the nodes the clients should be able to find those in other charts in a short time so after the option we're considering to use is pops up so the pops up about how sup it is basically a way for subscriber to subscribe to certain topic and and publisher can publish some data to that topic and subscriber will only receive the top the data of the topic they subscribe to so in this pop star system Anisa overlay network underlying and for each topic there will be a channel on top of the pops up so the reason why we are considering to use pub/sub is you can imagine that the shards can be mapped to topics so a client can choose to subscribe to a certain chars and it will only receive the data from it's interesting or it is listen listening charts and the current implementation for pops up is there is one available in the p2p and it is called first up and it is the first version of the pops up in the p2p and its implementation is go and Jas and rust and his idea is the note when they receive or publishing data just for the messages to office peers and then the gossip sub comes out and it is still in the poor request in Libby to the repository and the spec is also available and its ideas for each topic an omen hence a certain number of peers to gossip and release the messages only to those peers so and its strategy is there are two strategies the first one is eager push eager push means that it just sended whole data to a specific peer and lady push means that it just sent a summary of the data and usually it is a hash of the data so in this graph you can see the black arrow and this is the eager push so the note just push the data entirely and for other notes for this to knows the link on the and they the node just push the summary of the data so it is the lazy push sorry so after a note is pushing after a note pushed at a summary to other nodes if other nose like this note if is after after a time period if it doesn't receive the corresponding body after to the summary it will find that it elects off some data so it will ask for the original note which said that it ought to it so it will carry the data so it is called lazy push and gossips up use both of them both of those strategies and and this two strategies make casas up more efficient I mean because in close stop you just broadcast all of the messages and they are there will be a lot of redundancy and casas all can read reduce this redundancy and gossiped up has further optimizations the first one is broadcast tree and its ideas in gaza we still need to broadcast multiple broadcast a message to multiple peers but it still has redundancy so if we constructed if if we embed a protists tree into the gossip protocol we can the message will only be broadcast once across the peers so it largely reduced redundancy and the second one is it introduced membership management protocol so the idea of it is it allows peers to swap to exchange their harsher review part of their power I mean they can exchange their peers information and that makes the whole network more reliable and the proximity awareness so this optimization allows appears request appears to keep track of its peers latency so if note latency is too high it will be replaced by the note by the peers by nose with lower latency and the process accounts it is a summary from me so in my opinion the pros of using Casas tab as the p2p layer will be introduced the development effort for us because we will no longer need to to consider - - - to solve the complexity of implementing so many maintaining so many shards simultaneously and it is also supported by the p2p team and it possibly be used by Falcone and the counts might be because some of the features are not yet implemented like the authentication and the encryption so for now they are not implemented and other features so yeah and another why is it only supports calling in JavaScript so for us like charting implementation in research team we use Python so we need to write Fighting's to support to use that and the current idea for us for shouting p2p so we our idea is we're using pops up to do gossiping and we will have global and local gossip channels so the go-go means that it is subscribed by every node and the local means that it might be a Shar so some of the nose can substract subscribe that but while others not so we use the global gossip channels to go cast the Shar subscription preference so a node every know where broadcast is currently interested chars so everyone will know which peer is listening to which char and also we use the global gossip channels to broadcast a mention beacon Chen messages and a local gossip channels both is shard specific informations and about the peer discovery and we use footnotes and besides we use the global channel I just mentioned and so every node will procaster ensures there currently subscripting charts in this case every node can construct a sharp reference table and when they are going to join a specific char they can just use this table to find the corresponding peers and we also provide the API for each node and for this like this API a request sharp reference table so if a node it just joined the network it will not have the information about the table so it can read ask this information from its peers and also the request collation will be can be used to synchronize the whole charge hands and ongoing works and we are currently researching on Kosovo and keep testing Casas up and for now we're testing a smaller scale and mainly focusing on the performance but heaven have a test like the reliability in others so we'll keep doing that and I will keep finishing the POC of the sharding PD idea we just mentioned and I have a pretty imaginary version but it is not it is incomplete for now and after we build this GOC we can we can write the Python bindings to make - to make the Python side to call this the the p2p layer so that we can do some do more tests yeah and that's it any question [Applause] thank you yeah thanks for having me many misses and I work on the Lapierre P team I live here in Berlin so I feel right at home we just giving a bit of context of the p2p real quick so when we build when we started building ipfs we noticed that everybody is reinventing the same stuff over and over again just because well distributed networking or decentralized networking never really dominated anything so nothing got into any Colonels or anything like that so there's the p2p wheel the cross-platform wheel the upgradability wheel and so we started to the p2p project which is a modular p2p networking stack there's currently three usable implementations the co implementation which was extracted from go ipfs the JavaScript implementation which was largely a greenfield project and runs both on servers and in the browser and the youngest one is the Rustler p2p implementation built for polka-dot better find parity people and more implementations are welcome of course so users are ipfs and definitely file coin life peer who are running the stream and my life right now thank you polka dot obviously Optimas are meta mask media chain in the past and and a lot more that i that i forgot to put on there the p2p is basically just a bunch of components each of these components is just an basically an interface just a couple of methods just a couple of functions the implementations will implement there's transport scripted channels pops up as we already heard discovery for just finding any notes peer routing for finding addresses for specific nodes content routing for finding nodes that have specific content there's three maxing to put multiple streams into well one stream and this relays and by the way the ones that are in italics here are currently works in progress so the project roadmap is well the PTB just this year became a first-class project of its own so we're still doing a lot bootstrapping that project with dedicated planning's and RFPs and RFC's and meetings we are gonna increase the work that's put into testing into simulations and benchmarks interoperability testing between all those different implementations we're gonna work a lot more in documentation which is you didn't put it on the list of cons but it clearly is one right now which are aware of and one question that keeps coming up is the one about security audits and we already are doing a couple of them and we're we're taking this approach of having individual components of lipid Apiata and we're starting from the bottom up so to to to increase the strength of the foundation going up so to speak and the short term feature map is definitely well gossips up because we need it for five coin other projects needed as well and we are pretty close to having a first version usable and suspect there's ago poll requests as kevin already mentioned and there's gonna be a lot more work into simulation into improving it to its security hardening it against the tags and about more implementations that you mentioned like using python and all that we want to have a demon ala p2p demon similar to the ipfs demon which you can run and it exposes an api and then you can have bindings in various languages too to control that if you type in a demon to bind your own protocols and different languages to it that's I think it's gonna help a lot it's gonna help me personally a lot so just yeah we want to improve connectivity a lot the TCP transport currently is pretty heavy on resources and on little home routers because it turns out 600 700 connections just simply killed some routers and then there's smaller changes for DHT performance and the migrations away from RSA key pairs for example which are really CPU intensive to January well edie keys are simply random data and yeah you already went over some of pops up so I yeah flats up was the very first naive implementation of the of the pops up interface just simply flooded everybody and gossips up is meant to be a bit smarter a building block for more advanced because pops up protocols as well and kind of put a lot more work into that going forward there's been a pretty interesting discussion of of those easier ami requirements you were mentioning the shredding requirements and that could have issue up there and yeah there's respect documents there's a simulator that you can try out yeah I one point that's pretty dear to me and that's the last slide is that if if you pick PDP as as a foundation for for your application and you make it run on the general public the p2p network you gain you gain with every node of another project that joins the network you gain connectivity advantages you the DHT is strengthened not just by theorem nodes for example but by all nodes out there that use the p2p you get connectivity in web browsers for free net busting all that you get privacy enhancing transports for free so open Bazaar I have had a lot of success with an onion transport in the p2p which we're having audited and and included into the p2p and in the not-too-distant future and end of this year beginning of next year we want to do like properly routed packets which overlay networks on the p2p so that when you want to communicate with a different node you don't actually have to directly connect to it but you send stuff over the overlay and once we have that feature then the first point that every node of other applications strengthens the network really starts to pay out yeah so much for the context thank you so may I invite colleagues in Florence and I prepare some questions and I'm wondering if you guys have some questions already yeah thank you so I had two questions first of all when you talk about pops up model where I guess their subscriptions who is actually keeping those subscriptions and what about can they disappear and the second question is that I in none of the I'm sure there is something but none of the presentations there was any mention of adversarial environment and how we deal with those okay from the subscription when you join first you you have to join the overlay network and after you join you what when you subscribe to what you lose sorry when you do subscribe you will broadcast sorry you will send the message as a subscription message to your peers and your peers will know the subscription status of you yeah so every no will keep is his peers subscription about the second question yeah but that was area model reason it's not up there is I didn't work on Gus except myself yeah I don't have too much time for preparation anyway that is documentation and discussion about it I couldn't find it in in the short time and I'm gonna get it for you and put it in the documentation of this of this day I guess of the agenda yeah I just don't have a good reply for it right now I mean to be more specific is that I don't think that we can build this peer-to-peer protocols are completely isolated from the network they're going to be operating for example you want appears to actually filter out the messages without relaying them blindly so you can't relay messages blindly because they could be the spamming of the network they have to apply some kind of rules and these rules will inevitably be the network specific yes so that's what adversarial thing I meant I had in mind so in general these like epidemic protocols like the lots of approach are sort of more resilient than say like a routed solution so that they're suffering something to keep in mind so it's kind of hard to break like like an on and unrouted network like the one that he just presented but the with the spam issue there is always this thing that like in the end I think the way to see this now this effort is to say that you know like the sharding prototype the the actual prototype it doesn't really matter what it runs on so it can run on like any sort of insecure protocol as long as it's efficient enough to support a couple thousand nodes a couple hundred nodes and there will definitely not be a lot of value in the system initially but then over time as the proof-of-concept gets more mature and you know more value is being put in the system then there will definitely be like my guess is that it will just evolve into some kind of custom protocol that has these these these these checks built-in so I don't think that the a theorem sharding will run on like a generic pops-up protocol forever the PDP sounds really awesome I'm just wondering if you guys consider to any other p2p systems or libraries for sharding and sort of why did you settle on lid p2p what makes it especially well-suited for this application [Music] okay we did survey the debt free to be or use the via Twitter le before but in that case you have to maintain like you are like the pure discovery if you separate the network for each shard then how do you do never care discovery you might rely on the who knows right for each shark you will need some certain number of the who knows so you in that case you will entirely rely on the blue notes so unless you use other kinds of the top topology like like Yannick he previously proposed a torus network topology so in that case you can ask for you as your peers about I do you know any peers in specific char so you so you will know that for when in the torus topology you will know you you you will ask the next char spears and you would so so III mean you need some certain way to find the peers in other sharks in that case so it is for me this kind it is more difficult I mean for now we if we just you'd pass up it can solve this problem I have a very basic question can nodes running lid p2p talk to nose running dev p2p no not yet so there's definitely an effort underway to make it possible to share transport implementations at least with the PTV project so the idea would be that in the long future even if they're in 1.0 will have the p2p transport option in that case it might be possible to for example download information about maniacs on anything 1.0 1.0 using the p2p but that's not possible right now so right now all the implementations of a theorem basically run the custom protocol they always round and it's just yeah and and there's no like no connectivity option but I have a comment for your for your earlier question as well so there is like another system it's like within reach and I could totally be tried and that's the same as PSS so you you all heard about it on the first day the swamp teams just released it as part of their our POC 3 and it's available in the in the Goyang repository it also has it comes with like a standalone daemon and everything so it's kind of comparable to the two to two to the the PDP team into the the p2p demon in the end so basically there's a demon you can connect to it you can talk to its API and send messages through a rotted overlay and I guess it's kind of funny that like these things tonight sorry you know we kind of have like the same stuff so but it's it's like yeah so this is like even in the like in another corner of the etherium ecosystem you have like another system that could be tried and I think these that could be like an interesting experiment someday to just check you know like let's just run like the the PSS daemon and see how that performs it does come with it has a bit of a different like privacy trade-off so that that can be considered but it does have on the other hand it does have encryption which isn't really built into floods up or coasters up at the moment but I guess you can just always there that on later so it is there's definitely like alternatives but there aren't that many alternatives that's the thing so basically when you when you're building a system like that like the like the like the sharding prototype you always have to choose do you have to do either your shoes like some some library you make your own protocol we use one of the well things that already exist and somehow adapt them and I think the PDP right now is really the best choice because it gives you it's like it it's like a very modular toolkit you can pick all the different trade-offs you want it gives you a lot of stuff for free so I think is kind of a the ideal ideal candidate for the state of the project is in right now and the question I probably don't have a full picture of lip-lip p2p environment but it seems to me quite limiting to encourage have much more implementation like for every programming language have another implementation for lis p2p have you thought about having some kind of inter process communication for network stack to have some kind of demon running in the system and then just communicate locally with with this process from it would be much easier to implement for like Python for example except like implement all of these protocols you currently have four new languages yeah absolutely definitely there's a bit of initial design for the daemon and I'd like for the first version of the lipid PD minute just like to largely take the model the FS demon uses and but there's a lot more work or a lot more down the road for that demon after the first version which is hopefully gonna be worked on soon so we have this developer meeting in two weeks where we're gonna decide on the roadmap for the next half a year and I'm gonna push for a demon very hard but anyway after that first version of a daemon which will be very simple and where the mango will simply be for other applications and other languages to just be able to speak to an API and use the network through the API after that there's gonna be probably a UNIX socket API to the daemon just for better performance because HTTP is well can be limiting and then there's even more options there's the option of pushing the lower levels of the p2p into the Linux kernel for example the Kripa channels and some of the transports and the last future point up there was a routed a packet switch routed overlay and some part of that could also be pushed into the room the Linux kernel so there's for example regarding crypto channels there's this very interesting new VPN project called wire guard which is a kernel module and and thanks for being a kernel module it's performance is just stellar you can get one gigabit per second of throughput on the tiniest VPS you can get for a dollar at any Oster and so we'd like to use that as a crypto channel in kernel and for example but that's way down the road some some time next year okay so final comment how can we actually like participate in this discussions about that it's like where should be the first point I should like visit and check actually this particular subject yeah definitely the all discussions and documents and and all that is on github so there's the lipid EP /li peer-to-peer repo for more general discussion that's not specific to one specific implementation and then there's the lipid IP / specs repo and the three repositories for the three implementations so the p2p / call the p2p /a asleep p2p / rest the PDP and there's also in two weeks these developer meetings what day is it gonna be yeah it's gonna be Thursday and Friday and one and a half weeks so I'm I'm not a hundred percent sure who from the etherium community applied to come there but if you haven't yet if you haven't yet and you're really interested in like lip it appeared development and planning then send an email - oh gosh well I guess Jimmy and I'm gonna relay it has any work been done on mobile like does this work on mobile if not what would need to be done to make it work on mobile yeah but it does certainly there's an experiment fish Android app called activist droid and it runs well go ipfs on mobile but it's not optimized for mobile use cases yeah so it so it can't pause or resume connections it just keeps running and of course keeps eating a battery because as are constantly on the CPU is constantly on there's a couple of more optimal optimizations which have already been done for example the DHT client mode where you don't act as a as a server so to speak in the DHT but only make requests but that's just a tiny step and being able to to suspend and resume connections or the whole demon is pretty critical for for mobile so actually just so the reason why we have Guillaume here is because you're already a question was also that if there's like any way to for the p2p to talk to deaf PDP systems so one thing that we're doing right now is the reporting like yet another part if your maker system which is the whisper protocol which is also surprised a post-op protocol to let p2p at the moment so that's also something that's going on there's actually something there as far as you know parodies also participating in that effort so I guess we'll have a lot of options yeah yeah I mean we okay so whispers is supposed to be a messaging system well the definition can vary depending on who you ask but yes the goal is to ensure darkness so you when you have when you send a message to someone else an external observer should not know if you're sending a message who you're sending it to and of course what you're writing so yeah we have this this PR that's currently open about implementing no there was not a ring mark you know need to know need to sneer otherwise you okay yes so we have this PR and indeed like we we just we add a new like we use we usually p2p like the multi the protocol streaming sorry the multiplexing to send deaf p2p deaf p2p packets over over IP FS yeah so like the reason why we did that in the first place was because we wanted to see if we could build okay because whisper uses gossip so it's really heavy on the network I think there was a gentleman saying earlier how do you filter how do you filter messages well in a whisper you don't you just send everybody your message to you messages to everybody so a lot of nodes are not activating whisper because that's a lot of processing time that they spend just just decrypting messages that are completely there are not internet for them so yeah we we wanted to see if we could build our own network with only parties interested in in whisper and yes so that's also been used for for prototyping because if we can I mean maybe that's a bit controversial to say that but if we can chuck deaf p2p altogether that will save us some maintenance time as well I actually have a question hello if we if we settle down to use Casas up as the p2p layer do you have idea any idea of building the test net because I have some thoughts in mind that I think if we stay down we can make a spec and it is cross languages so we we just regulate the messages and the protocols and we can in ten different implementation of the causative clients we can build it can build the test net together I mean from the Python side we can also build use the Python bindings to call right we just write some coding Golding and write the biting is to call from the Python and we also can can do the testing so is there any is there anyone has any thoughts on this and about I have I also have some questions especially to ours I'm just wondering the how open however the costs are be used in Falcone possibly well it will be used certainly that's one one of the huge priorities that that the fact one team have brought to two hours to the p2p team get us get as good pops up and gossip stuff is the first step to that how exactly they're using it I actually have no idea there's just so much stuff to follow in ipfs and a p2p that I I don't find the time to also follow five coin so sorry about that but i but I can get an answer to that and add it to the documentation of the day any questions is that broadcast three a part of gossip suspect or expect somewhere else yeah yeah it's in the gossip sub stack also I'm wondering if there is a plan for the test net of Gaza and how will it be I mean that same idea yeah I mean we'd like to get good automated continuous test beds like yesterday because not just for pops up I mean for all the rest of the p2p for a PFS so if I can we just need good go test nerds and simulations and and benchmarks and yeah it's gonna be priority like really high priority and and in the rest of this year just just because we can you can't be huge large scale distributed networks without without any of that right [Music] and about the features like okay and I just come up with one question about the attacks on the Gaza because I previously I already have in my like you can just send the data which is not surprised subscribe it by the peers so you just send it and like you can just occupy is and with and about this do is there possible to have improved building prevention or protection mechanism and on any prevention of the attacks mmm not yet really I mean um on that particular attack you can lead us or you can you can exhaust the network resources off of pretty much any net network service right it's not something that gossips up alone can prevent all it can do is flex sources of traffic for for an external tool to pick them up a gossip service in an application layer protocol so it so it can drop drop packets on layer 3 for example and even if they're already even if it could they already would be at the host and the network if the traffic is already at that point then then it's already too exactly too late but if you want to prevent any network exhaustion the text you need to do it somewhere at the at the boundary of your own network look at a firewall at a router at a switch but yeah attacks Andy he doesn't text specifically or other exhausting attacks and and also texts in general on gossip sir I don't have a good reply for you I know the question what we on that note so you said that like maybe you could flag the sources of traffic so that would be definitely an interesting discussion how to build a system like that allows you to sort of like also stay a bit anonymous because one of the worries that we have for the for the shorting Network specifically it is like certain classes of actors in the shorting network that are very prone to dogs for example like if you're if you're a POS validator for example you don't really want to be dosed because you're gonna do is money over that so this is kind of important to stay to stay anonymous and then my question would be if there is they specifically like in in floods up well floods up maybe not in cost of subs there's like any or in the underlying protocols it's like any prevention of like you know your actual IP leaking or something so it's not so much about you know tossing a note within the protocol a bit more like figuring out what is the notes actual IP on and on the internet and then just like tossing that because that would be that could be catastrophic for our system like Provost ache where a specific machine is need to be online in a certain time of day to actually do something in otherwise say those money yep privacy enhancing transport most most of all so the onion transport comes comes - is the very first thing obviously tor not not high-throughput not lower latency the onion transport is store based yeah exactly exactly so on the one hand you can dial through toralei's and on the other hand you can actually listen on as a make your lip you know listen as a as a tor hidden service and possibly more to come but that's the one that's that's tangible right now yeah so if anyone's interested in doing like cool research that will be like really cool research to have like sort of like because like especially with something like tor so I'm guessing like a lot of people in the room have used tor or if they tried it or read something about it it's kind of a famous project or anything but it's just is like tor is always this thing that's not really like you can't just really plug it into your application it's like a it's like a service that you can you can use you can't on your traffic through it but they actually techniques behind tor is like something that it would be really useful in a lot of protocols so if someone had like something where you could just like make that pluggable that would be great yeah just so just a quick addition to that there's actually no a pretty young but really useful go library that embeds tor right into into your binary and so you don't have to run the tour service separately it's pretty young not auditor super useful okay so now then you can like import tour but it doesn't really yeah so yeah that's cool and all but like this like the this like there is no like this is I think something that could be really cool to have an in the p2p as well would be to have like a native sort of like a lipid Fe native onion routing thing because there's something that is useful in in a lot of cases yeah so probably well before that there's this layer in canoe net but I think you guys still don't have it's called cadet which it handles routing between hops at the you guys only route like through one let you you do have an indirection capability if it's only one hop or something anyway but you probably something like that even before and there can be some weird layering things to think about there another time so anything else or is that it oh there's a stack ratchet so you're mentioned in your presentation of leaked p2p there there's a benefit of having more participants and network and I don't quite understand why like why is it do they all participate in some kind of global network or is it and how does it work because if you do like you know your you're not really paired with the with the network you want to be paired but you appeared with this so many like what is the number of peers and in the pizza P so it's I can answer it or so it's like basically when you if you participate in the in the in the DHT for example to use its services and basically everyone else who is also using the DHT in their project it's also in that very same DHT so in that case you're not really like peering as in like directly talking to these to these nodes about your application but it's more like like certain basic like certain base layer services I just used bye bye everyone and and one more feature that doesn't play out yet because it's not there it is is is well this router Picard switched over there and the advantage the advantage there is basically the same idea just a layer lower on the connectivity layer that there's certain base services / building blocks that are shared by everybody and that gets well to get more throughput more more bandwidth or more resilience if there's more nodes for example if there's if if packet pays the p2p overlay in one and a half years for example has a million nodes on the world or ten million or in the best case then it's much more likely that you have that that you're gonna find find peers that are on the one hand physically pretty close to you or otherwise otherwise well located in the network for your purposes then then if the network is really small and only only focused on your own application and and the idea with with the routed over there is definitely to to cover as much of the networked world in the long term and also try and transport IP traffic over it have it have it peer with with the existing Internet infrastructure of a over B TP and yeah one more addition because you were saying a native a native anonymity layer Olympia P would be nice that's also something you can you can layer on top of such a of such a packet switching overlay building block so to speak and then and then the bigger the bigger your network gets the more advantageous it is to your anonymity purposes just the last quick questions yeah you said you wanted to install embedding the Lea p2p what demon whatever into the Linux kernel so you know I used to be a colonel dealt Lewis Lena Stovall has said like his main job was to say no to stuff and like the way I see is you know this is a library that can be used on other operating system like FreeBSD like if you start admitting that like what would be the purpose what would be the advantage of doing this this thing yeah maybe maybe to clarify we wouldn't probably we probably wouldn't but the whole demon and to the kernel but just the lowest layers of it thank the UDP transport the TCP transport the very simple basic transports and probably reuse wire guards crypto channels and then later once once the packet switched overlay is there probably also the packet switch just for performance reason you don't need to copy packets between kernel and user space it's much easier to to have dedicated hardware for it fpga is for for example and yeah and one girl apart from performance is obviously like not just performance but performance on commodity hardware what about freebsd i haven't thought about it yet but it why not you know so that they do have like a kernel module system so you think just like their neck you can just think you can load your own modules and stuff you have that on Windows - thank you okay I guess that's it