# "History of Node Infrastructure" - Michael O'Rourke

- Channel: [ETHGlobal](https://streameth.org/ethglobal)
- Date: 2021-10-09
- Duration: 27:39
- Topics: Ethereum, Michael O'Rourke, Pocket Network, ETHOnline
- Watch: https://streameth.org/watch/yt-lXYiHz0wuiw
- YouTube: https://www.youtube.com/watch?v=lXYiHz0wuiw

## Description

Michael O'Rourke of Pocket Network gives an overview of the history of Node Infrastructure.

## Transcript

josh stark from the global team and uh josh will be here uh taking over for me and uh introducing our remaining talks for the day so josh is already here and i want to welcome josh here to uh say hi to everybody on this chat and uh i'll hand it over to josh thanks hey everybody uh thank you for joining us today uh we just had a couple more talks left uh this afternoon and i'm really excited to do something um let me just quickly share my screen as i take over here for a moment all right um yeah so i'm josh i'm also part of the global team and i'll be your mc for just the next hour so next up we've got michael from pocket network and he's going to give us a history of node infrastructure so michael please join the call and take over share your screen and uh yeah it is your stage awesome thank you josh i'm going to go ahead and share my screen here hi everyone hope everyone is doing well okay great hopefully um everyone is seeing my screen um so with that uh uh hi everyone my name is michael o'rourke i am one of the co-founders of pocket network and pocket network is a protocol that incentivizes people to run full nodes for blockchain developers so when a developer is building an application and needs access to the rpc pocket network provides full access to that rpc in a in a fully decentralized manner for a little bit more context when you when your application is using pocket um the protocol uh picks suit five pseudorandom nodes to to to do uh work for your application for an hour and then swaps out another five set of random nodes uh for your application so um and it and it pulls from a global set of nodes so um that's a little bit of background on pocket and uh what i'm going to do here is give a brief history on the note of of node infrastructure um so with that um before i do though i'm going to give you a little bit about myself i was born in dominican republic i live in tampa florida for those of you who know me i shall tampa pretty hard a really awesome place to live i studied international studies in college i learned to code as an ios developer before i graduated and i work out of a blockchain co-work space called block spaces here in tampa um so uh i i want to uh preface this talk with um this is meant to be a pretty high level uh talk i studied basically history in college it's something i'm fascinated by and what i really want to do is give you all a context of um where we are today and what the architecture looks like by starting from from the very beginning so with that um i'm going to go into part one which is a kind of pre-internet and basically we're gonna go over the first nodes what are these regional data owners and effectively uh pre-internet was very low bandwidth uh in this respect so um the first node uh was uh the interface message processor this was the kind of first server by funded by the us government this was to actually protect the us in the case of nuclear catastrophe and the idea was okay what happens if we build a redundant network that you know if our major cities are bombed for example um it won't go down so that's really where this research started um this builds on the on the ideas of the amazing jcr lick lighter who is frankly a legend i would recommend reading anything about him if you're curious on seeing how kind of these ideas that ethereum and every one of these decentralized networks is built on this thing basically just did packet switching dynamic routing uh and some hypertext uh it was very basic um and the first computers were connected in in 1969. uh if you want to learn more about this i really really recommend where the wizards stay up late this is a really great book the narrative is really well done and it really tells you the history of of how this barely happened actually and it's fascinating um so that's the first node in terms of when it comes to history of new infrastructure so the next one uh the next kind of phase i want to talk about is is this idea of or the first or one of the first bulletin board systems or or bbs this is one of the larger ones that occurred this one's called phytonet this was created by someone named tom jennings and the nodes here are effectively the people running the servers that you would connect to over your modem or your phone um uh these are you know this is you know you see the the cds and the floppy disks um that's the database that's where the data was stored and uh these these nodes uh would have uh massive really frankly piles or or um organizations of these cds uh to upload on these servers so that people could dial in and and see this information on these bbs's and um it's really interesting because you can only connect one person per modem at a time so which is which is kind of you know crazy thinking about it uh today uh and really tom jennings designed this in a way uh uh actually he he defined it as a cooperative anarchy he wanted it to be cooperative anarchy so the idea would would be that he would provide minimal costs uh public access to electronic mail and and some other things it was really important for tom jennings that every node to be self-sufficient and needing no support from other nodes to operate and the way this worked was that there was a node list and it would contain the modem telephone number of all the nodes and uh this will allow nodes to communicate with any other node without the consent or or or need to ask any political groups at any level and there's a lot of you know kind of interesting history here in terms of governance and and how this whole thing kind of came along but um a little bit about the network architecture uh of of the fido network um and for context this grew to have 20 000 nodes all around the world um so so really the the the people who had the most power were the regional coordinators um uh and and the list of basically uh the list of all the nodes would be automatically updated and distributed weekly um this list would basically just have all the information of the hosts the telephone numbers the geographic location and and and everything like that right again this was you know cooperative anarchy um so certain random things happened right they would they would uh cooperate and write some policy uh in terms of how to manage this network um the regional coordinators actually kind of de facto made themselves the bosses and they would be the ones who would nominate the network coordinators the zone coordinators and the international coordinators and for context here um the regional coordinators were like the big city ones right um so the ones who had the most data of their specific but what i really want to want to highlight here is you've got a bunch of self-sufficient siloed networks nonetheless dependent on on these regional coordinators and these other uh network zone and international coordinators um so the way i like to think about it uh uh is is it was like being like a turtle um they were pretty siloed the data moved very slowly through shareware and physical mail um and they could work together um although it was kind of limited in capacity and and how they would how they would work together so um i kind of thought kind of remind me of a turtle so um so that's the first part um basically kind of pre pre-internet things are very slow not much data then the internet happens in the 90s bbs's die they become isps there's exponentially more data being passed around as the internet of course grows and in this case large corporations end up owning our data for a really important in my opinion user experience trade-off which is a big reason why the internet is exists the way it is today and frankly this is we're kind of at the edge of part two and part three right now but a really important piece here is that this transition left a void in this kind of node hosting uh environment right uh because those bbs end up turning into isps um so as the 90s went on the internet started exploding um a lot of startups had to buy their own servers uh uh this was originally known as a slashdot problem um you'd get posted on slashdot and your website would break um this was really expensive uh people you know startups would spend millions of dollars buying servers to just keep up their website right this was before we had any idea of of you know sophisticated devops and all the amazing tooling we we have today but this really kind of helped push um a really big movement in the late 90s early 2000s into the cloud this is really the the state of where we are today the cloud is huge um we have really four big um entities that run most of the cloud um amazon microsoft google cloud and and ibm there's a bunch of independent infrastructure providers as well that run their own data centers but those have been progressively going down or getting getting smaller and the way i like to think about this is we are all basically mr fantastic i was trying to draw this mr mrs fantastic i was starting to draw this out and i kind of started to see that we had really long arms because we could really reach anywhere at any time and it's it's really frankly incredibly powerful i don't want to discount how amazing um you know things are right now in this context because we are able to communicate with anyone around the world but there comes an important trade-off here where effectively all our data is siloed amongst uh just a few companies right um you know some uh companies like facebook might you know that data is is their um kind of edge right that's what that's kind of their their competitive edge if you will whereas you've got kind of got these broad infrastructure providers uh which is really kind of our big focus more than anything as pocket but um you know they provide you know servers for people to run their startups so you know before uh when one started to run you know spend millions of dollars to to build in to run their servers and build this infrastructure now they can you know auto scale uh uh you know in seconds with with amazon web services right um so so you know that's kind of the the metaphor i'm i'm going uh there so um uh so then part three um this is web3 uh uh this is you know uh owning your own data and and most importantly for for for me and for pocket is um shared publicly accessible databases this is a really really important concept particularly in the context of the way blockchain infrastructure is working today because in the web 2 world just like facebook and any company while they might be running on amazon web services or gcp or or whatever it might be um uh their databases where they were where that are hosted on these servers um are private right for most companies um most companies this is kind of their their their edge right um and their competitive advantage and kind of how they how they make their secret sauce right um an important and really important distinction in web3 is that these are shared and publicly accessible databases which has changed the dynamics of how this infrastructure gets hosted and served to every single blockchain developer in the world doesn't matter who you are uh uh we've seen you know we're starting to see these trends so um before i do though i wanna talk about um what a node is uh in the blockchain context right uh uh in web two you have an api your server and your database um yes it's all three in one and um uh and frankly is not optimized for um uh you know rapid retrieval right for grabbing data quickly um it's focused on being a redundant uh uh peter pure uh in this peer-to-peer network that exists right um and as a result we've had lots of solutions above and below gap like you know when you say below i mean like the networking layer uh uh and above uh like indexing that has helped these sorts of things uh uh the the limitations of kind of just the raw gap server by itself right um or a raw death node by itself um but i really wanna uh kind of highlight that um a geth node or or an open ethereum node or or whatever it might be um is really all three of these at once whereas in web 2 you know the api and the database and the server all really really optimized for their specific tasks um and as a result it's a lot you know uh and many times more difficult more expensive more slowly to to do what we take for granted in web 2 world so we've seen a lot of awesome solutions to come in and really help uh improve those things um so now we kind of have a note of the way or what kind of a node is out of the way um i i want to talk about the first node um uh i got in crypto in 2013 i managed to to download bitcoin qt for the first time it took forever to download uh bitcoin then eve on my on my local computer bitcoin qt um and it's pretty magical of course the first time that you used one of these full nodes right this is a nice ui on top of a bitcoin node that you'd be running on your on your local laptop so so yeah uh this is a quick thing on on and there's been a lot of information on on bitcoin core and what was bitcoin qt and now is bitcoin core but um it's just a really important moment in history in my opinion um uh and because it was so hard to run these nodes um no one really wanted to uh it kind of limited the um the growth of of of our industry right early on um this is a screenshot of coinbase i think in 2012 or or 2011 or 2013 and um they had an important insight that wasn't you know they weren't the only one to have it but um they're kind of uh of course a very massive reason of why we are here today they were like hey brian armstrong was like uh it's hard to run a bitcoin node um let us run it for you so you can just access your wallet right um this is a complete departure from from kind of the ideals that a lot of us really care about in terms of sovereign ownership owning your money and being able to verify a lot of these things right so this is the first steps are the first steps to kind of the uh traditional web 2 kind of web 3 mash-up that uh we're in today right and and frankly um almost every single application is is working on this context um in in terms of at least the the infrastructure um bitcoin decided to go the castle route right um we have a ton of amazing applications that are not custodial as well but um that's an important uh piece in my mind um so the first appearing clients i don't know if you all remember if you if you've used mist or or the original parity client um but uh these were uh uh also magical uh for me just as magical as the um uh uh bitcoin nodes um uh parity uh uh came out on february 8th 2016. so um you know these were the first ones and um since then we've seen which i'll show in a little bit um a nice growth and diversity in some of these nodes as well um and in 2016 uh defcon 2 uh inferior was born um so uh in fear is in my opinion one of the most important aspects of the ethereum ecosystem um and i want to make sure i highlight that uh this is not in fear did not lead this uh this would have happened anyway in my opinion uh because ethereum just kept getting bigger and harder to run um and uh i i show these graphs to show that developers application developers on ethereum most of the time don't want to run their own nodes they want to focus on building their amazing gap right whatever it might be at the same time we saw the the number of nodes going down right um i i want to highlight that i think it's a little bit overblown in terms of the the uh you know the the amount of nodes being risked to or the amount of nodes being so risky that you know things can be censored and so on and so forth so um uh but frankly um they've been a really important piece and and this is part of the old paradigm where that developers access an api with a bunch of servers controlled by um ran by you know centralized entity and this growth has helped us see a massive explosion of these right from geth to open ethereum of course nethermind and besu um are some of the major major clients um and we've also seen an explosion of of providers as well um vera alchemy rivet i'm just naming a few here pigment networks quick node um i want to highlight rivet they've done a really interesting thing where they've forked geth and allowed um have allowed a much more allowed for much simpler and easier scaling of your death nodes particularly when you're when you're hitting um high throughput or when there's an application that just is absolutely hammering your servers um so so as a result and i've just again i've just named a few here there's a ton of them um it's actually led to a massive diversity in my opinion so which is nothing but good in my opinion um also uh i consider these the kind of next generation of protocols that are really actively trying to um uh decentralize this infrastructure stack because um we see a lot you know i don't know from crypto twitter but we see all the time um you know these are is your dap really decentralized because they are you know using a centralized provider right and um i also do think it's a little bit overblown that the government's gonna come down and shut down or aws um but that said um i think it's important to point out that we have um these kind of web two infrastructures and architectures um kind of placed on top of this web 3 new paradigm right and it doesn't always exactly fit right so we see things like it's expensive to run ethereum infrastructure because again these are you know they take up a lot of data as peer-to-peer networks or by persisting in particular networks but you also need them to [Music] provide data to to applications at scale um so i just want to highlight for me you know of course pocket but i want to show how each part of the stack these two or these three projects are in and if you took that node picture that i showed a few minutes ago um the graph works on the indexing layer um and they are incentivized as people incentivizing people to run full nodes in their own way right because um the more uh traction their their their queries have right the more nodes those those indexers need to run right so but they're really focused on kind of this indexing layer and that index layer is super important because uh again death you know these full nodes are not optimized to to grab this information right pocket network purely focuses on the full node um we just open up the rpc uh for anyone access and when a node serves some data through the rpc api um they earn our native cryptocurrency and also i think marlin is really interesting um on the networking layer um so they're doing their own incentive for full nodes um but specifically through the uh the networking layer which i think is is really fascinating in my opinion these are kind of three networks that are um that are in my opinion well designed and and um are really doing um uh pushing kind of this decentralization of the infrastructure stack uh which in my opinion is is really exciting um and this in my opinion will lead to a new wave of uh web3 friendly infrastructure in architecture um because instead of kind of going through many layers to access some data i believe in the future we're going to be accessing many different kinds of nodes through many different ways of paying with all of these things abstracted through all the tooling and frankly i do think through all this traction that all these applications on ethereum are seeing we're also going to see that the nodes go up i think as well allowing for for really interesting things to happen uh uh down the road in my opinion so so for me i think right now we're at a really interesting clinical time because kind of got these new protocols coming out that are really making um honest attempts at really decentralizing the stack and i think that's incredibly important um that said um we don't have to solve really really hard problems um we don't have billions of dollars like amazon does and there's a lot of investment and money that needs to go into networking bandwidth and hardware in my opinion um uh this is you know for me this takes time a lot of research and uh frankly you know a lot of investment and money for for open source protocols like like the ones i just mentioned um to reach kind of the similar economies of scale as previous ones um that said we we are able to tie in these kind of crypto economic incentives really tightly in in our protocols which in my opinion gives us an advantage um at least in the short to medium term um uh and i i think this is changing the narrative uh uh because i think this is uh uh a really interesting quote that i've always believed but in my opinion software has always implied trust in those hosting the hardware um i think networks like like ours are are a reversal in this trend uh providing new reasons for everyday people to run more hardware for themselves in my opinion because we're starting to see all these networks that are you know frankly incentivizing you to run nodes in infrastructure i'm just amazed at all the people in pocket who are now you know becoming devops pros who before had no idea uh uh about any of the stuff right i just think these incentives are just starting to get baked in i think it's going to be an incredibly powerful trend um so yeah that's that's it uh uh thank you um thank you for for having me uh uh really happy to be sponsoring earth global uh if you guys have any questions about how pocket works uh please reach out to us in the channel we'll be more than happy to to answer any questions and point in the right way great thanks michael uh that was a great presentation if you have a just a couple minutes just a couple questions to run through um yeah absolutely one uh so and i'll just uh i'll stop your screen sharing if that's okay too or maybe andrew can do that behind the scenes um so one question i mean you know you mentioned it seems like a main tension here is well you know in the long term uh will all of this end up you know back in centralized infrastructure right there there are often efficiencies in scale you know whether that's just the expertise to run the nodes or you know whatever it happens to be whether it's proof of workforce or something else um like do you like how can we avoid that future you know i guess you mentioned at the end here that the opportunity to bake in crypto economic incentives into these networks gives us a powerful toolkit but i mean i guess tell us more about what are the different levers that we can push to try and create the incentives to avoid you know just ended up with another centralized infrastructure inventory yeah um i i think the number one thing is tooling uh uh meaning you know i can do a one-click deployment at a local data center right um uh and and manage my my validator you know for example right um i think for me that that's probably the biggest inhibitor uh uh when it's at at least pure proof of stake network uh i think one of a really important actually line for me is as having non-delegation right because delegation is is easy when you have that uh hard line right there uh people are actually much more incentivized to build this tooling uh to make it easier for people to run these notes and i think that's an important aspect but for me it's there you know what's interesting about these open source networks like the graph pocket and and marlin is that uh through collective uh uh efforts and and kind of building on on these infra legos if you will um it will only get better right and and and i think that's how we kind of avoid that that future got it um and uh one more question you mentioned at the end uh there are you know hard problems open in this space um you know for anyone watching today who who's looking for a hard problem to work on um big or small um what would you suggest is the next hard problem that we have to tackle as an ecosystem yeah um uh uh specifically in the in the in the uh infrastructure space i'd say coordination amongst these turtles if you will right um uh so so how do these turtles um speak to each other quickly efficiently and easily um uh you know if i'm aws i've got infrastructure all around the world and i know exactly where everything is going right um i think understanding how these uh various infrastructure networks play together uh and interact i think is going to be increasingly a more difficult problem so especially as they get bigger and it gets more unwieldy people have different setups and with different you know bandwidths in different parts of the world when it's not one person who owns all of it coordinating all that is going to get increasingly more difficult okay well thanks