Geth Team AMA | Devcon Bogotá

foreign [Music] I've been with the guest team for two years now what were you doing before you were in the guest team oh I was working on permanent State channels and actually mining software my name is Guillaume I've been working on the guest team since 2017 initially on a project called whisper uh didn't work out so I moved on to uh more core development and I'm currently working on the topic of stateless ethereum and vocal trees uh before that I was an engineer at a various non-crypto related companies and I'm very glad not to work there anymore I'm Peter I've been on the gas team for about eight years now I started working on whisper and figured out that it won't work out in about two months the essential ethereum is my first real job so hello I am Matt I've been on the guest team for four months uh I've been working in the ethereum space for about four years it's also my first real job I worked on the cool team previously doing various r d efforts on future ideas for ethereum and now working on the Json RPC for the Geth team so this panel was supposed to be an ask me anything so it would be super nice if you have any questions starters we woke up with the news of arbitrum acoustic acquisition for prism how do you how your team looks for an acquisition and do you think prism will be leader consensus clients for ethereum I actually just heard it two minutes ago before we went on stage so I didn't really read anything about it or made up my mind about it I think consolidation is um it's always this two side of the side of this of a coin right we want to be as decentralized we want to be as um distributed as possible and that also means um that the team should be independent um on the other hand if you have uh like a bigger company uh working on different uh things um they can achieve stuff faster and so let's see what the future holds for for arbitrary and prism uh plus just to add to that I think the ethereum vision long term the idea is that the clients will be more the most important clients will be maintained by fairly important companies whether those companies grow out of the ecosystem or our external companies that's I mean out for debate but I think in general it's good for ethereum if you have solid well-funded teams behind them so whether this particular case is good or bad I mean that's Up For Debate but the idea is okay all right I have a question for you guys right here so thank you for being here you guys are rock stars so this is a question for the you know the ones I have been here many years about the what is your most memorable experience working for Geth I remember Defcon 2 the Shanghai attacks so I don't know what's your experience about that so I would like to know what's the most memorable experience for all you for what is the fundest experience working with the get team what about the merch like how do you celebrate it is that was that a happy celebration was that full of anxiety where the the merge celebration was basically okay it's done now let's let's move on um so for me the the I think the most memorable I've haven't been in this team for for that long uh the most memorable thing I think was the uh Greece interrupt um where we worked on the merge together with all of the different client teams uh consensus layer client teams execution layer client teams uh in a really nice hotel in Greece and we spent like the whole day sitting in the basement hunched off over our computers uh working while there was it was like 30 degrees outside and after the third day I I made the decision that after dinner after lunch I'm going to take one hour or two hours to go to the beach to enjoy life a bit between working uh in in the basement and uh as a I'm in a space around five years so as I'm a co-developer I had opportunity to read your code and uh participate something in your Discord um but after the years passing I I was feeling that less and less maybe all Europe so occupied or with less support I could saw you less and less in Discord I also the documentation will become more and more outdated now after I saw Peter's I think of last year one of her tweets that say oh we need more support the other stuff did you get some support for the feeding Foundation to support your team maybe to hire new people to help the documentation the stuff because you did an amazing work go is still being a good platform for for beauty stuff but I see a lot of people say okay you are going to use rust other stuff but you you have a we have a very good base and knowledge in goal that you guys built so how is the future for gef so basically this is my question yeah so that's actually a very good question and I'm kind of happy that I have a positive answer for that is that um well in the past we've uh most of the documentation that we wrote or stuff we we published were all written by us and I mean that's kind of nice it's kind of we have to be the origin for those but uh we didn't really have the capacity to do it and uh for the past couple months we've actually had somebody on the team who helps write documentation I kind of feel that we still don't help him enough but we're trying our best plus it was seen as initiative to actually revamp our entire website which will come in a few weeks months I'm not entirely sure but there again we kind of a huge shout out to the EF essential ethereum.org website team because they are the ones uh doing their our new website and so we know that there's a lot of things to improve but there we have received an enormous help from the EF towards improving it I'm sure there will be a lot of things that still depend on us but it things are getting better Also regarding support on our Discord and and everywhere I think with the increasing complexity of ethereum and the the increasing demand of shipping stuff um just being there like our time being spent on supporting individuals in the community is very limited and we need more people from the community to take over to to help educate uh educate others about how to use gas and this kind of stuff uh yeah I just wanted to answer the question about the future of the guest team uh so far people were were quite positive and upbeat so I wanted to be the down here Downer here uh I think I mean it makes sense that so much of ethereum has been relying on the guest team before uh there's been an effort like the help also comes from other teams like for example uh the the client sorry the consensus layer clients are taking part of the the trouble the burden away from us there will probably be this kind of uh of other efforts in the future and hopefully we can just become a bit more uh redundant in a way and uh this this is this excuse me that would be uh true decentralization to answer as well also talking about support if you guys have experience with get and want to help us with the Discord yeah we're really looking for for people who can do it although I know if you guys can work with guests like you have experience then you probably get better jobs elsewhere but you know just yeah for technical support yeah this is Cena by the way he's also part of the guest team [Applause] yeah so Cena didn't want to be here but uh thank you for being here but Cena gave a good lead into my question which is to ask everyone who who's in the Geth team either on stage or off stage where do you most need help individually like in parts of projects you're working on whether it's like strategic stuff for the future or like something right here right now is there anything you can share with us as Community to help us better support what you guys are doing like can that work the first thing that comes to mind is please be polite on PRS yeah so I guess with the guest team our bottleneck our single actually single bottleneck is pull request reviews we have a lot of very nice members on the team who can really churn out code super fast maybe a few people way too fast like Gary and we can't keep up with him and the the and we also have a lot of external contributions and there's the problem is that the code is super sensitive so it kind of always boils down to a couple people having to review everything and having to do a lot of context jumps to review them and unfortunately this is our Achilles heel because we have absolutely no idea how to solve it definitely more people on the team more reviews help but uh it's not an easy to solve situation because we can't just hire somebody to do code reviews since I mean the entire network kind of depends on it so that's uh we're really open for suggestions on how to solve that I think two things also come to mind for me the first is helping create better onboarding mechanisms for people to contribute to Geth right now it's kind of you know look at the code base find something to fix look at the issues find something to fix um I'm taking you don't like your onboarding experience no I think my onboarding experience has been excellent it's been such a fun time working on the kids no but I think seriously a lot of people have talked to me saying I have no idea how to start working in core development and you know some people like work through that filter themselves and they find issues and they start contributing and they end up in that position but I think that we could build some mechanisms for more people to contribute and I had this idea that I've started working on a little bit is like having some sort of like capture the flag that has like basic flags for people to capture where you do like a core development task like add a new OP code or fix some sort of bug or go through an invalid trace of the evm and you're able to resolve it and run a program and verify that you did it correctly and so that's like a very you know binary system of like I did this right and so now I can move on to the next thing and then once you go through that then it's a little bit easier to come into the whole guest process and figure out like okay why did they not like my PR how do I get this PR to be accepted and everything so I think that would be a cool thing for people to contribute if that's something you want to contribute to I'm happy to talk offline about that and the other thing that I think that we could use help with I think like all of the execution layers could use help with is working on the Json RPC so that this is an interface that's a little bit more standard across all clients I think the ideal world is that a client is just simply a black box and no matter what clients you're running you can always interact with them you know almost exactly the same through the Json RPC and we're trying to get to that point but we're still not quite there and it's just generally one of these things that's not as high interest for a lot of people and so it gets put on the back burner for the most part and I guess one more thing I would add is that uh no shade for the other execution clients but very often I do feel that we the guest team is driving a lot of the changes a lot of protocol changes networking changes so it's kind of a bit asymmetric in that we are the ones bringing the for example the networking eips to the table and everybody else just well okay if it's backed out and it gets implemented it then we'll just roll with it and I mean that's perfectly fine just it kind of means that our our tax tasks are always not only to maintain gas but also somehow to try to advance the execution layer features whereas the other clients are kind of I feel that they are sometimes just playing catch-up and this is fine for them which is less fine for us um I think that that also goes that on not only goes for features but also for uh the testing so a lot of the testing efforts uh right now are driven by uh bright by Geth and we're currently trying to build out a new team within the ethereum foundation uh for specifically for testing for cross-client testing um so um yeah and the deposit has been Geth is the majority client and we have to take care of the network so we have to make sure that everything is really well tested and we can create we can create tests for that can also be used by the other clients but in the future we would like to get to a state where researchers and the testing team create tests for everyone and so we don't like and they don't have to rely on on us doing doing the work for testing uh yeah and something kind of uh in this area is also um there's there's gas gas is its own monolithic piece of software um but there's a lot of I feel there's a huge gap still between the solidity development world and Geth and I I would really love to see a lot more tooling developing in between like to bridge this Gap so I don't really have any suggestion about such tooling but I I feel there's a there's a gap here that would be great to to fill um I have to think of a question because what I really want to do is just use this as an opportunity that I may never have again to say just how sincerely appreciative I am of everything you guys have been doing over the last couple years [Applause] um from one developer to another I know that sometimes that doesn't come through as much as it needs to and I it's just what you guys have done has really helped me um help me personally and get me involved in the ecosystem so uh if I came up with a question right now I guess it would be uh does clef have any future so whether it has a future or not um I would say so cleft was one of those projects which we really think would be very very useful however I kind of feel that we as developers kind of took it to a point where it's uh secure and it's highly unusable because it's very very console based and and everything and the only way to make clef more usable or friendly at even the least bit is to actually turn it into a product and unfortunately I will admit that that is way outside our capabilities because there you would probably need a UI team you would need a completely different team to to do that work and whilst I do think it would be super awesome it's uh again the question is I we could hire somebody or multiple people to work on it we could maybe get an EF to work on it but there are a lot of other wallet software out there and the question is is it worthwhile to try to compete with them I'm not sure so I would say clef isn't going anywhere but I don't really see it going into a product quality either so it probably it will be a bit in this limbo space for now but something that we're currently thinking about retiring is the the personal namespace and the the the the wallet within uh within guess and so if you're depending on that we're sorry no actually there's a discussion to be had but we would really like to get rid of this yes so definitely that's direction we so Geth is kind of like this huge monolithics monster and that was kind of borne out of the necessity of when ethereum launched there was no other software so everything the clients had to do everything but obviously having your accounts managed by a node is a bit wonky that was partially the reason why we built clef and we will definitely try to take clef up until the point where we can remove account management from Geth and hopefully it should be as easy to manage your accounts via clef as if you were managing it via Geth but I think that's the that's the threshold where we will probably stop and if somebody picks it up awesome if not then it will be used as kind of as a developer tool in the future so a long time ago there used to be a very nice proof of concept running that on Android and I feel that now after the merge maybe it's more relevant and it can be actually useful that people are running their own clients on on Android is there any work being done on that so originally we when we shipped guest for Android we shipped it as a full node which obviously doesn't work anymore then later jealous shipped the light client it's still I wouldn't call it really production ready but it kind of worked however it turned out that the pre-merge light client is still too heavy for a phone so now that we are post merge that's actually an interesting discussion to be had whether we could just somehow pick this thread up again I think it definitely would be interesting however usually with mobile phones and Android you have very very strict limits on how much I mean okay on iOS you have very strict limits on how much uh your background process can run Android is a bit more relaxed but you're eating still eating battery very very fast so I could imagine some Alias clients or some like client on demand where you just whenever you just want to interact with your thing then very temporarily just pull some data from the network and and that's it then I think the the post merge world is kind of compatible with that so I I think that would be interesting however at least using the gath code base I think it will always be a bit heavier than ideal so if you if you were to really so the get light client even if we ship it as production ready we're probably more cater towards running on a laptop and if you want to run it on a mobile phone in a production at the environment my guess would be that it would take a different team maybe a slightly different approach to to get there no so it's it's like an additional thing that we have to maintain and we would whatever Android users cross compilation is always broken and there's always something some issues there and so it never really works and sometimes is a bit clunky the code that that gets produced and so I think another team just building a a light client from scratch for for mobile phones especially after working uh would we will definitely some uh see some stuff there and I hope that at that point we can just get rid of this so um there's also a bit of a road map issue here it seems that you know uh now the direction for ethereum is to have all of the day-to-day operation happening in rollups and having the core layer be something more I'm not going to say inaccessible but maybe not the thing you do on a day-to-day basis if that is the case if if that Trend continues I don't think it makes sense to to worry about mobile because no one is going to try to access access the layer one from a mobile one more thing I would add is that it's an interesting twist post merge world that before the merge even a light client had to have a lot of connections I mean you either connected to a trusted server which kind of Beats the purpose or you had to connect to multiple light servers so that you kind of got the ground truth kind of follow the correct chain based on proof of work now post merge you don't really need that because you have the signature so in theory post merge you could connect to an untrusted webto provider and download the necessary data you could still verify it so this kind of opens up a bit of a different design direction for making ethereum trustlessly work on mobile phones uh can you repeat the question yes this level is good so what's your take on the open ethereum parity Saga and what can the community what can we do to make sure that alternative good clients don't die again I think that um I think that's inevitable so in my opinion good clients will die it's uh I mean look at the gas team gas team currently has is about 10 people now I would say that out of these 10 people you have probably four to five people that are kind of more familiar with the very very internal details now should these four or five people leave it is very very hard to find replacements for them and this is essentially what happened with parity okay they're leaving it was a bit different but the idea is that if you actually manage to simultaneously lose enough of your main contributors then it's very very hard to onboard enough people fast enough so that the the project survives and I think that it's as bad as it is to lose a client I don't really see what you can do I mean the same happens in open source software too that eventually you have a couple maintainers that just I mean life happens they go do something else either because they get bored or whatever reason and I'm not sure that um yeah so I I I think it is a very real chance and it will definitely happen that good clients will die there's always a I mean one way to protect it is if there's a very very good funding behind it maybe a large corporation but again if there's a large corporation behind it they can always decide that it's not worth it to keep going so it's a I guess that it boils down to the fact that you need this is why we would need client diversity because that kind of affords us good clients occasionally disappearing and and I think within the guest team we're trying to prevent this by onboarding new people and making sure the the uh the new people that come in get familiar with um a lot of the different parts of the code and so but it's very hard for us being relatively new um to meaningfully contribute because there are so many just invariants within the code that are not explicitly stated and so sometimes it happens that we break an invariant and it's usually Peter who has all of the invariants in his head and and says okay we're breaking in and variant here don't do this this is going to to to fail at some point and it just takes a long time to actually learn all of these invariants that are implicitly in the code plugging back to one idea I was talking about before just the fact that for example we kind of spin off consensus if another one of those steps happens in the future like maintaining a client becomes easier and easier so that would be a good I forgot the word but it would be a good way to make clients not be at risk of dying but until until we do until we simplify the protocol it's uh yeah it's going to happen and one thing that I also really like on the roadmap is these uh the perch where we try to get try to eliminate some of the um all outdated stuff and so for example history expiry would allow us to delete all of the rules that we have for executing all transactions and at some point if there's a way to execute them in a different client and and all of these caveats but it would make maintaining a client a lot easier another issue that we always run into is that someone wants a feature and we don't really think about it and care and implement the feature and and merge it in and two years later or sometime sometime later we realized that this features not really used by anyone not and and we cannot change something because we would break this one feature and at that point we have a decision to make either we like either we just don't do anything or we delete the feature and someone is going to be upset and um we don't want to make people upset but it's inevitable I think with the with the way Geth is right now yeah and I guess just one final thought there um Mario said that uh if we Implement The Purge and get rid of a lot of features then that would really help other clients get up to speed I think if we So currently it's very very hard to write a client and starting a new client is essentially impossible because ethereum is moving so fast that you you never catch up but I guess once we reach the point when uh when ethereum starts to ossify that would be a nice place when actually new client developers can join in because then you can actually say that well I want to make a client just specifically for iOS that has this and this and these properties and they can work on it for three years without the protocol constantly changing the invariance and I think my expectation will be that when we reach that point of stability we will have the original clients be quite marginalized by new clients that are very very focused to some specific sub task or sub some use case thanks again for caring so much of the ethereum ecosystem on your shoulders what is the current yeah sustainable business model for get and probably other clients that want to copy that and the follow-up question is what is the plan to do something similar to what's been done for the core protocol right we want a certain percentage of of it to be staked as a security um a percentage of the total value right so is there some sort of research going on on how do we get certain percentage of the development effort in the ethereum ecosystem to be going towards get security well to answer your first question uh it absolutely makes no sense to write a client because I think it's a there's no business model behind it it's currently most of the clients are funded by I mean we are fully funded by the EF we have absolutely no income I'm assuming other clients usually how they try to fund themselves is that many clients get grants from other projects so that they support different blockchains different layer tools different scaling Solutions that's one way to somehow try to build a business model around it but at the end of the day since ethereum is kind of like this public platform a client as a client you cannot really make money out of it it's a yeah so that there's no business model behind uh creating the client itself the only thing that I would say is that uh there's the the Proto what what's the protocol protocol Guild yeah exactly um that allows projects for example D5 projects to allocate a percentage of their token distribution uh to a pot and out of this pot uh client teams get right now they get bonuses um uh and this is the the general idea behind it was not to fully fund client teams but to provide an upside that they wouldn't have if at that they currently don't have and that they would have if they were to switch to be uh to to defy uh because we've seen a lot of uh good client developers uh just say okay I can make 10 times the the money if I if I create the next token um and so the protocol support a protocol Guild thingy is like a way to give some some of that upside from the default projects yeah appreciate like I was wondering if you can start experimenting with things like maybe Kodak funding towards PRS right there's you said there's a big backlog of PRS and only a couple of people that are capable of reviewing but you could imagine that with some sort of codewriting funding you can't hire or at least bounties for other people to layers of developers to review and write tests and integration your tests before it gets to core developer so I guess this depends on the granularity generally people do do that at the EIP level so famously uni swap is pushing certain eip's very very hard and I'm assuming they are actually funding the people who are doing the necessary research doing necessary presentation etc etc to get an EIP through now that I don't think that really works at the client level because we've tried for example at some point we gave out a couple bounties for some work the the place where it kind of backfires is twofold one of them is that you usually get contributions of not the greatest quality because a lot of people see that oh there's a I don't know one ether bounty on this so let's just jump on it so you will have 10 different implementations all just kind of trying to hack it as fast as you they can to get the bounty so the code is not the best it requires a lot of effort from our site to somehow try to fix it up or try to try to guide that person and the other side is that after the bounty has been paid out since this was a bounty work they just disappear and maintenance is our problem so we kind of have more problems than gains with uh funding PRS at that level the what we have done previously is we have funded the research teams for example to who have helped us on the Discovery protocol the Discovery V5 we I think Felix was managing a small research team at I'm not entirely sure which university and the EF was funding them for a year or so to just investigate find possible solutions to different challenges write some papers that one works so that definitely works but at the client level I feel it's um it's the wrong granularity I just wanted to say to the original question of like how what is the business model for clients I agree like no there's no reason to build the client there's not a lot of money in doing this it's always going to be like considered a public good but I think one way of making it sustainable as like Maria said this protocol Guild project in a way that people can like help make this sustainable for the long term is whenever you're developing new projects to consider adding a small allocation to the protocol Guild in your initial like token launch if some of these are if this would have been something that was around in 2018 or 2019 when a lot of these Blue Chip D5 protocols we're starting we would have over 100 million dollars dedicated just to core development and so I think like going forward it's like a good thing to consider because you want ethereum to be around for a long period of time and the best way to do that is to make sure that the people who are making it happen continue having the funding that they need to do it yeah and uh adding to this we not only do is there no business model for a client but I don't think there should be because otherwise you would kind of adapt your strategy to increasing their revenue stream and you lose your Independence basically my question is related but with what Marius just said and well I'm still a student and I'm still trying to figure out what I want to work on after I graduate I see their stuff that you guys do and I think it's amazing and I still see something bar on the application side of things so my question is really why did you decide to go to core development and stay in court development instead of going to something higher in this in the stack because it's way more fun and way more interesting and so for me the the big thing is also I want to work on something um that makes sense to me where I have the the the the the feeling that I'm doing something good for the creator of humanity and uh I don't think diva is a lot of the defense stuff is is uh is this and so that's why that's why I I was so interested in in working on the merch and and helping push push the merge um because I felt like I'm this is like the biggest thing that I'll ever be part of uh regarding um like the CO2 consumption and everything um and so just having this big level of like my work can my small work can have a really big impact that was something that was pretty magical from the beginning and uh yeah that's that's that's why I am in core development I don't know about you guys it's it's not the money I mean I would so my answer is kind of boring because there was nothing when I started working at ethereum so I think Matt's answer would be much much more interesting since he did have the choice of picking one or the other picking one or the other isn't I mean working on protocol versus working on that player stuff right I mean you know I originally started working on the top layer like when I came out of University I was very interested in how you know the types of applications that you could build on ethereum and I was very interested in like dispute resolution because I had done a lot of like e-commerce and buying and selling things on the internet growing up and I was like frustrated with the way these systems were built so I was very excited about this as a application on on the protocol so I joined consensus and I was working on something pretty similar to this and it just immediately realized that you know a the user experience for ethereum at that time and like still today was too bad to really on board like hundreds of thousands of users to you know have this kind of dispute resolution system and even if 100 000 users decided to show up tomorrow like we didn't have the scalability to support that many users on the protocol and so that was kind of like where I started getting really interested in protocol development it's like I was like this is broken we need to fix it so that we can build an application on it and I've just like slowly become more and more indoctrinated in the idea and now I'm kind of at the point where I feel like I would rather be a small piece of a very large puzzle that I feel is going to become extremely important and impactful for Humanity rather than trying to build an application that may or may not have any kind of impact for anyone uh so I guess your reasoning for joining platform development plus you guys suck I'm gonna fix it myself naively maybe so yeah I don't do that for Humanity sorry you guys suck no yes of course uh I think also it's uh it's where the the biggest problem I mean like the impact indeed is going to have uh like reverberating consequences in the future so that's that's where it's the most interesting but uh it's also simply because uh yeah you want to have good tools to build better societies to build better companies to be better software and I think simply the core I mean if you want to work on your tools you always end up working on the core no matter I've worked on uh on Linux kernel before it was the same thing you get dragged into the you can get dragged into the protocol as long as you want to to improve your tools hi there um I have a question about databases the underlying database and get as being viewed as a database um you guys have talked about Geth as being a monolithic and wanting to remove or modularize components and I was wondering um since database uh throughput had been traditionally uh kind of constricting aspect if uh there were any changes to that coming up and this question comes from seeing a remote DB option in the command line options and not being able to find any documentation with respect to that so that's a very nice technical question uh so one um one aspect I'll touch I'll ask a previous question a lot of people sometimes ask us on on social media why did we pick level DB and that was actually also my question when I joined the team and Jeff's answer was because Bitcoin was using it so that's uh that's essentially how guests started out using level DB we've actually we've tried switching databases many times generally what's not so visible from the outside is that uh databases are kind of built up have two components they have a storage layer which is kind of like this very dumb layer that just has some very primitive ways to store data and retrieve data and then they build on top various transaction mechanisms journaling all kinds of stuff and most people kind of convolute the two and most people don't realize that level DB is essentially a storage engine it's not a full-fledged database and there have been databases built on top of level DB which have all the bells and whistles now the issue is that the moment you are adding uh if we were to use something higher level then essentially we're not only paying the costs of the storage layer rather we also have to pay the cost associated with running running the transactions running indexing the tables etc etc so at this point since Geth was kind of architected from day one to just use a storage layer as his database if we were to plug in any full-fledged database instead of it everything becomes just insanely slow because Geth was not architected to use these high-level Primitives get always assumes that it has more or less direct access to the data and because of that it's we've tried we've have a lot of PR's trying different databases and they always crashed and burned one thing that we currently are working on uh actually Jared is working on it is to switch out level DB to Pebble pebble is kind of like a Next Generation version of uh level DB but it's still just the storage engine as for remote databases the problem is that one of the bottlenecks of ethereum of running an ethereum node is disk access i o operations per second now the moment you move the data away from the node the actually the bottleneck is getting way way worse so it's uh instead of making making things better you are making them a lot worse since accessing I mean usually you have if you want to access SSD a modern SSD can do maybe half a million i o Ops per second that's not the high end SSD that's a payable affordable SSD you cannot do 100 million round trip times on the network per second so it gets a lot slower if we go down that path of course you could always say that why we could create an ethereum node architecture which has a remote database and then you could have multiple clients using that same database that's a very interesting architectural decision but that is essentially writing a completely new client from scratch so you cannot really retrospectively retrofit gas to use it it would be its own new client hey guys seconding all the previous comments thank you for the incredible work you're doing in the ecosystem you guys are the unsung heroes of ethereum um quick question if you were to hypothetically re-architect Geth from scratch today knowing everything we know factoring in the merge what would you do differently uh I think we would at least I would be more careful about the features that that we accept into the code there's some stuff that barely anyone uses but it's it's still used and uh something that that kind of bugs me is that we write a lot of like small tools and instead of having them separate from the client they end up somehow in Geth and uh yeah I would I would have like a stronger policy of not including them for example for example the ABI gen stuff should in my opinion be a different thing um but I I I already know what Peter is going to say but then you need someone to actually maintain it and if it's not in the client it's not going to be maintained if it's not in our code base no I actually agree with you no so I uh I just wanted to add what Mary said previously it was brought up that the problem is that one get was started we had to be this monolithic thing that does everything and this kind of makes it leaves its mark on the code base and it's very very hard to get rid of stuff and I mean we could get rid of stuff but some people somewhere always depend on it so I think the if we were to start over get from way back eight years ago we would end up in this exact same situation because uh we would need to make the same tools and we would and so I don't think the fact that we have a lot of Legacy junk in the code base that's not a bug it's just the way the ecosystem evolved so I don't think that would have been preventable if we were to start over now then all of a sudden you can rely on all the awesome tools that people created and that allows the client to be a lot Slimmer hey um I think it's a lot of stuff changed over the last like one year and a half but especially what's changed significantly I think it's that a lot of other networks appeared that actually locked a lot of billions of dollars like somehow that actually are like Gaff Forks if some changes and was it somehow like useful for you to take a look on it are you somehow monitoring what is happening over there and if so were there any cases when you somehow wasn't like inspired by some changes in these clients and you want like to bring them into the Gap like the original Gap and like some cases when some bugs were opened in like get Forks that actually are pretty close to the original Gap as well one one thing I would wanted to say is about bugs so whenever we find a buck in gas then there's usually a lot of different uh clients that are um that are also vulnerable to this and so we are looking into them and making sure that they are not uh before we actually um uh before we actually uh fix the bug and and publish it um the problem there is in which in which layer ones are we going to look and are we only going to look at uh ethereum Alliance uh layer ones or or not and so I think the way we handled it up until now is that we um kinda look in the biggest ones and um just sent them an email that hey you should expect an announcement at a certain point and then we're going to make a public announcement because we don't want to be king makers we don't want to say which client which which Fork gets the bucks first over over another Fork um and yeah there's also been bugs that have been exploited on other networks before they were exploited on ethereum and uh we're kind of like not really monitoring the the other networks but we're trying to be in with in on good terms with uh with the other client uh client devs I guess one one kind of satellite question here would be that uh way back there were a lot of um drama around when it turned out that we fixed a bug and we haven't necessarily announced it other networks had a problem with it people very often asked us why don't we have a standardized way of reporting bugs reporting vulnerabilities have um usually what big web 2 companies do is that they have their own little private consortia where they share bugs and then when they publish it everybody in that consortia already updated and we people were always very very vocal that we should do the same thing we kind of felt that that's a bit problematic because in in the blockchain world if you if we were to create one of these consortia it's not really clear whether everybody in that group would be friendly or how friendly they would be so usually what we try to do every time we found a hairy vulnerability or bug is that depending on the nature we always try to announce external people or give enough details to external people external teams to minimize any potential damage now for example in the case of ethereum what we did is that if we knew that we are going to fix something that um that only affects miners or if the miners are good then the words that have cap happens for average user is that their note crashes usually what we did is that we just gently pinged a couple of the bigger mining pools that hey we will release a release you really want to be on this release nothing more just a gentle reminder that it contains something that you want to run and by having the majority of the hash power updated the network is kind of safe and there's not much damage that can be done but this is kind of a completely arbitrary bug by bug decision on how best to proceed how best to minimize any damage to anybody in the either in the ethereum network or or the external blockchains that use ethereum so we really try to be as friendly as possible within the limit of keeping the ethereum network live hey it's me again uh I just want to add that um apart from the people sitting here and some of us off stage we also have people watching the stream from the team I want to give a shout out to Martin who is from far away who couldn't make it unfortunately and uh yeah uh who's from far away trying to contribute to the conversation so uh some somebody asked about the remote DB flag uh I I have an answer to that um so it's basically um it does the low level uh database get over RPC so basically you can have like what you have a node on a server and you can have your like local node um connect to that to the remote one for some like only read upper operations so for example if you do DB metadata command locally then it would basically get give you the metadata of the remote node yeah that's it uh do you see snapshots as the get as get solution for the state database layout for the first UB feature yes what one question we are in this beautiful Columbia South America Bogota I just want to know what each of you have like most loved about Bogota so far hiking in the rain at the waterfalls I actually I actually really liked working from here it was it was really fun with like we had some workshops a day with like all of the client devs and today we had a really nice working uh session in the morning [Laughter] yes if you haven't noticed Marius is the new workaholic on the team it was it was really nice just getting the the people in the room that need to be in the room and work on some of the uh interesting things that are coming up and I'm really really really excited about the new stuff that is coming for ethereum I also enjoyed the hike quite a bit that's the only time I've got a hotel besides coming to here so I heard different math I heard different I also really like the energy at Devcon it's been a while since we've all been together and I think it's easy to forget that the community is so large now and there's so many people excited about this protocol so I found that very energizing for myself uh Jared tells me the food the food uh no yeah it's uh the food has been nice uh I like the hotel as well uh no but uh I don't know uh I I saw the the center the other day when we went when uh Luke's looks quite interesting uh I guess we will Explore More this weekend after the work is done one last question who from the team is not on stage today besides Cena who from the team is not on the stage we have Jared beside you so shout out huge shout out to actually in the back uh Felix who's uh in Berlin Martins in Stockholm and and a giant uh actually a giant shout out to Gary who wasn't able to make it to any of the guest meetings for the past three plus years due to him being stuck in China and the Chinese rules on leaving the country and coming back due to kovid are super super strict so and insane shout out to to him for still tolerating us and working for us without all the upsides of uh having the fun like we are having now big round of applause guys thank you so much