PSS - Node to node Communication Over Swarm

hello everyone my name is Lewis Holbrook I'll talk to you about postal services silver swarm I'm part of the jack team jackets company that developing decentralized micro place for metadata in the media industry and Jack is building on swarm and we are heavy contributors to the development of us work before I worked in Jack though I did a lot of different things I was a classical pianist I was started and edited the cultural magazine I was a music manager for a while funding bureaucrat and also taxi driver in Berlin for two years and while I was driving taxi one day I picked up these guys saying quites bag they were totally starstruck cuz they just met Snowden and they were blockchain people and I draw those somewhere they told me about this bar that they've been in the bitcoin bar I didn't even know they were existed one but half a year later I tried it down there I met this guy Lichter and you know apart from all these things all my life I've been involved in computers like most of you I presume and we got into talking about sea pointers and really really nerdy and then he said look you have to see something so he sent me a bunch of these videos of feels warm and and and saving the world from the clutches of these evil data analysts and stuff and I was sold so then who started working and that's why I'm here in Cancun talking about PSS so I'm very honored that Viktor is trusted and trusted me the development of PSS I've been working on this for the last seven months and this talk will be try to explain what makes PSS tickets internals but it's also an attempt to explain it in a easy-to-understand way so that you don't have to be a total nerd to to follow the narrative be sure to also stick around for after my presentation because up then are the guys from mainframe they're actually built a chat app that's running on PSS and that all of you guys can install and use today now first of all what is PSS well then we have to ask ourselves for this swarm first of all by the way a lot of the examples I'll be using here are going to be pretty contrived so you know real world real world situation might not look exactly as easy as this but it's you know to try to make it clear so sworn as a network of nodes and swarm is a metric of nodes that makes it possible to store data without trusting the nodes that are storing it and since the swarm network is rooted it always knows to short the distance from one node to another now everything in swarm that stored on swarm is hacked into chunks and every piece every of these chunks has an address assigned to it the address being the hash of the piece itself now the chunk is stored at the node that has the most similar address to it and similar here means that they have as many bits as possible in the start to be the address now the same logic is used when retrieving the chunks of course since you always through this logic know where to look for a particular chunk then there's no need for evil centralized tracking systems and stuff to guide your way in crude terms this is what Kadima is routing all about is finding the way to the data just by the properties of the data itself now PSS uses the same kind of logic but for sending messages so you know messages is pretty much just content to write but instead of getting the address by hashing the message the sender actually just adds the address to the message manually now this lets us send any content to any particular location on the swarm network and that's exactly what messaging is of course to reach its destination messages are forwarded from node to node and every one node that forwards compares the address on the message with the address of its respective peers and then forward the message to the peer that is closest to the message all the forwarding Nords compare the first part of the message to figure out where to send them but if we have less Sandra's information then the address information we need that we have probably will match more than one node that means that by specifying less adverts information of the message we can define more than one notice the destination of course we can also just choose to simply omit the address information and in this case the message isn't rooted at all instead the message is sent by all peers to all peers this is of course not particularly efficient but for certain scenarios it can be useful whisper the existing dark messaging implementation of ethereum uses this way to transport messages because it provides full darkness there's no way of knowing you know where the messages are going just from looking at the traffic but of course it comes that's a great cost when a message is sent and swarm all of the nodes that are forwarding the message are also storing the message on swarm as a cache the hash of those messages is used to determine for example whether this same message has been seen before now as you can imagine from what previously has been explained if some nodes wanted to just bomb the network with a bunch of messages you know that wouldn't be so hard so therefore hard-coded into PSS is this cache mechanism that is a crude a crude version of flood guarding if you will so basically what that means is that when a message has been sent through and the message tries to send let's say a hundred thousand of exactly the same message within one second or something they will already have the digest of the message they hash it again when they get it they see oh this is the same thing so I will drop this since the messages pass potentially pass through unknown nodes to get to where they're supposed to be they should be encrypted of course so the nodes on the way can't snoop on the contents this means that the actual definition of a recipient in PSS is not the message no sorry is not the address of the node itself but it's actually who can decrypt the message and this is necessarily only one node either if Worden more than one node could decrypt the message have the required keys then those that group of nodes will be defined as the actual recipients when we talk about routing in swarm we talked about the notion of distances between nodes this can easily cause some confusion this already mentioned swarm roots by how similar the addresses are to each other and notes that are closer and swarm could you know basically be totally different sites on the globe so in measuring distance in swarm we should just forgets about sorry forget about the notion of geography altogether when they measure distance in swarm we put in an America Mount on how many bits two addresses shares so if a node shares the first six bits of an eight that address let's say then the distance would be two and if five then the three and if just one bit then the distance would be seven this address that we're talking about there's a swarm overlay address now this address here is not one by long it's actually 32 bytes long it's also randomly generated that is to say that it's impossible to infer from what information in the node this address is derived there are other notions of addresses in the theorem as well one of them is the p2p node address which is combination of the TCP address of the node along with a private call node called a node key and PS s this is the address to use for step-by-step forwarding furthermore every PSS node has its own unique private and public key pair that it uses for encryption this is currently the same public he has the account pyramid parameter is used when you invoke swarm also PS s note might have any number of symmetric keys for encryption as well and these can be any arbitrary 32 byte value so how exactly does the process of forbidding work every sworn node has a hive and that's the definition of the collection of peers that it's directly connected when you have a message that's to be forwarded you ask the hive about a list of peers that are closer to the message stand the peer itself the note itself the high will respond and appear where the best matching address is contacted and passed a message and the same process is continued now how can you make sure that a the hive has enough peers so that the message actually can be sent to a closer address for this there is the notion of health and to be healthy every node has to have a certain amount of peers in various distances so first of all we define a number of peers that it has to has is in closest neighborhood in this case this is 3 this is what this we call the proc speed now here we see and the next step when we've done some connections that the peer has at least three peers and its closest environment but that's not enough to be healthy in addition in every increment of distances above this we have to have at least one peer so this is closer to the node this is further away we have three in the closest neighborhood and we have at least one and the other bins that's a healthy node and now since its help that we can be sure that it can forward a message to anywhere in the network now one of the attack vectors for spying powers is traffic analysis that's to say that if somebody can determine where the message stops then they could deduce that that particular message was for that node and therefore the closer we get to the node the message was addressed two messages are forwarded to more than one peer and in fact that means that messages are forwarded to all appears in their procs bin their closest neighborhood and this is even done by the peer that actually is the Intendant recipient so it can decrypt the message but it also passes it on so nobody can see where it stops so what does the PSS message look like every message has an address or not as we have seen before an expiry time a topic and an arbitrary bright blob that's the message itself actually PSS uses whispers envelope structure for anything but the address field and it uses whispers methods for encryption of the messages encryption can be done using symmetric free encryption or also public key encryption with the message is also signed by the sender one of the nice features of whispers D able to the ability to pad messages that means that all messages are guaranteed to be a multiple of certain size before encrypting the message is padded with random bytes up to this limit this threshold and it inserts also the first byte which is the number of bytes that constitute the signing actually there's nothing wrong with embedding some secret information that padding itself as well and then you can have a harmless message where the message is expected to be and hide the spice see stuff where a spy maybe and that boy won't look for them now what's the advantages of padding well without padding all the messages will have different sizes and a very very skilled attacker could analyze this pattern and order a messages and determine what the communication is all about when all the messages have the same size it makes that kind of attack less likely as we saw before a recipient at PSS is defined as you can decrypt the message that means as a sender we have to store the keys to keep track of our different recipients and furthermore these keys may have one or more topic associated with them which in turn is linked to a swarm or relay address and keep in mind that it's fully possible also to have for one recipient to have multiple symmetric keys also now what is this about topics topics are what defines what actions to take when you receive a message PSS has a registration method that links handler code to topics and in fact even though this diagram doesn't show it you can register more than one handler function to each topic and they will all be executed when you receive a message with that topic when receiving a message using public key photography we can use the signature to determine which sender that is it's public key the message came from so thus we know who to send back to when we want to reply and we might even have some address information on file for this spear we also inspect the topic and pull out the handling code or codes for the topic and that performs the actual action on the message what symmetric encryption it's a little bit different because here there is no signature to inspect so we basically just go through all the keys until one fits this is of course not so efficient but PSS has a caching mechanism also here so the last used key will be the first one that's tried and then the other ones are tried in chronological order with the logic being that whichever key you last communicated with maybe you know is more likely to have sent you the next message PSS also offers a handshake module which six changes some a pheromone ephemeral symmetric keys using public key encryption for the exchange so when a handshake is initiated the initiating node sends a number of symmetric keys with an expiry count to the peer and requests a number of keys in return the peer stores the keys it gets then it generates some keys and sends it back to the peer Y to collection of keys and PSS it's implemented so that one set of the keys is used for outgoing messages and one is used for incoming so then when one message is sent using one of these keys the expiry count is decremented and when it reaches zero the key cannot be used anymore so the notes to switch to different key and ideally also replenish the one that expired so PSS has an API that can be used through reach through RPC that's through sockets or through WebSockets I'll go quickly through some of the methods here not all of them so to get the public key of the nodes the host node it's this function get public key base address is to get the swarm overlay address of the host node to make it PSS topic there is a convenience function for this you send it a string you get a topic back a topic is actually just a four byte value and it can be any four byte value but it's kind of defined to be the hash of the first four bytes of a specific hash so this is what you get when you call this method to send the message you set using public key cryptography you have to set the public key for a specific pair so you supply the key of course you supply the topic and the address you know how much whatever you want to give or have that and then it's just a matter of calling the Sen awesome function with the key with the topic and the message itself the message just being bytes right the symmetric key one looks a little different do you have to in the same manner set the key the topic the address this is a boolean that tells should this key be among the keys you attempt to use to attempt decryption so if you don't put it in there it won't be put in the pile of the keys that it goes through to match the incoming symmetric message if you do that now we use whisper for key handling and for symmetric keys whisper gives you unique string ID that's a reference to the actual symmetric key and it's back-end so the parameter here is a little bit different this is then the ID that you get back from this function so you have to use this when you send a symmetric message apart from that is the same as the a symmetric one now it's no fun to just talk right you want to somehow sometimes have feedback as well so to get the incoming messages you subscribe to to as you've subscribed through them through the socket and this is a pretty simple as a keyword hardcore keyword receive and also the topic for which you want to receive messages for if you're implementing this in golang by the way there is a slightly different syntax that you can use I will defer explanation of this but it looks like this as far as handshaking goes you have to activate handshaking on each topic you remove handshakes from each topic two you can do that this is for initiating a handshake key topic and a couple of running out of time so I'll cut it shorter and get tan Shakey's which returns to all the valid handshake keys for a specific topic and a specific public key now it's possible also to use PSSs transport mechanism for dev p2p protocols you can use existing dev p2p protocol code and put it on top of PSS now this quickly turns a bit messy code wise so I didn't want to put it here but at least this is to prove that protocols are possible now a couple of caveats for pss-pss is a little bit like UDP you can't really guarantee that a message will get there because the notion of connectivity is just its opinion it's opinion that it's actually connected to this it also depends on you know the whole interlocking here so this needs to be handled in the application layer also since we are the closer you get to know the more of the message is passed around the end recipient might end up with more than one copy of the message now there is a small deduplication guard in mp SS but we don't guarantee that deduplication will happen in all cases so mind that as well and of course as with anything a cryptography there is absolutely no guard against stupidity right so I mean it's it's not a magic solution don't send your key to someone or etc etc for future development of so this is the feature set that's currently in NTS s now this will be part of PLC 3 and for further development on the roadmap is a male boxing function which will say how to store and tell notes that are offline that a message came to them while they were offline we will look at multi casting capabilities that is to say simplified routing when several notes when the notes aren't necessarily in the same address neighborhood and also using PSS to to modify states and nodes that can be used for for off chain database updates and stuff so the until this is merged into master this will be the branch that you use for development in here is also a full documentation of the API with a few more functions that what I showed now there is also a peer to peer to peer tutorial that shows a lot of basic step by step code examples when programming p2p and aetherium and that also includes a bunch of PSS code as well and lastly of course always happy to answer emails or join in on the chat on getter we're pretty much there all the time since we love computers and don't really have other lives that's it for me and there are 33 seconds for questions sorry about that [Applause] [Music]