# Merge Implementers' Call 7

- Channel: [Devcon](https://streameth.org/devcon)
- Date: 2021-10-09
- Duration: 46:10
- Topics: blockchain, smart contracts, distributed ledger, ethereum, Science & Technology
- Watch: https://streameth.org/watch/yt-6d944TCNpqc
- YouTube: https://www.youtube.com/watch?v=6d944TCNpqc

## Description

https://github.com/ethereum/pm/issues/345

## Transcript

alex okay so let's get started um to welcome everyone to the merchant bloomers call number seven um first congrats on the girly fork uh this is one uh this is yet one more step towards the verge which is great uh okay so we have not that tight agenda today so it's gonna be relatively fast uh we'll go through um a few updates and i'll discuss plans for q3 then finish with some random discussions or probably spec discussions so we'll start with implementation updates um everyone i'll start for myself um as usual so i've been recently working on the prototype of the transition process um the the spec of it have been merged the part of the spec of it have been merged has been merged like a couple of weeks ago so it's been implemented in tekku i played uh with it um in a local network um a few a few things i would like to share about this testing i've tried it like in a positive test scenario in the negative test scenario where the block proposer tried to produce blocks before the computed transition total difficulty has been reached so it went well [Music] but obviously we need more thorough testing with more mass on the network side like withholding um the proof-of-work blocks by part of by some part of the nodes of the network and then releasing them and so forth um also one thing to bear in mind here is that i use local miner so it's a result it's like in a high fluctuations of block time intervals and yeah one of the goals of the total difficulty computation is the predictable emerge time but yeah that was that wasn't like uh checked uh well in locally because of uh fluctuations so it should be like checked like with some real minor um which can produce more hash power so that's the update from my side but anyway the prototype showed that the algorithm general works which is great any questions here um very nice how do you think is best to test um some of these more complex scenarios for example like a partition in the network for you know two epochs after uh hitting the transition difficulty and and things like that um do we have anything in our tool kit to test that kind of stuff or do we need to build out some custom stuff that's a good question if we i was thinking about simulation like simulating the network stack if we want something some some predictable scenario and so we can like have some parameters of the mass that we want to have an on the network layer um no i don't think we have any tools for that so we could do on the consensus side we we could write for choice tests that essentially like a chain's being built and then another chain is revealed with different difficulty and stuff so there's a little bit we can do there in kind of an isolated fashion that we should but yeah simulation probably makes sense for yeah yeah right uh um yeah probably um i've been correct uh seeing this simulation of the network part yeah what what you have just said is what i was thinking about just to stop the network layer with some predictable network layer with which can be managed by by some process require messages according to some time intervals or something like that yeah so it needs to be done also about yeah these fluctuations um i don't know um i've been watching like uh do we have like the stable time intervals on the um i know under robson for example the block time intervals all right what about the block time interval um either like stable on the robstone i mean is it like really um i i mean more like stable in terms of uh the difference between the mean time and my understanding is it kind of depends on the day and who's mining on it but i would suppose more stable than what you were doing locally yeah we'll see okay so i guess we can move on yeah given that clique still uses total difficulty can can this um what you've written be anchored on gourley or clique network relatively easily interesting thought because if so that would give you good good block time right sometimes uh it probably is worth considering if that can be ported pretty easily i think because it uses total difficulty it should be able to be um just because we do have some test nuts we will want to fork off of poorly the one catch with predictability on goalie is that an intern block gets a difficulty of two and out of turn one with difficulty of one there's a lot of out of turn blocks so you are kind of halving or doubling your difficulty a lot okay gotcha yeah thanks thomas on the information on their obsting let's see so let's bear it in mind and get back to this question a little bit later on how to check the predictability uh and yeah the this historical data on difficulties could be pretty valuable cool okay um any other implementation updates that anybody anyone wants to share anything yeah i think he primarily landed an altair yeah yeah makes sense london and altair okay let's go to research updates um a couple of pr's that have been announced uh on the previous call which are the cleanups in um in the beacon chains pack by justin and eden randall failed to the execution payload they have been urged um so um cool uh also i've been like a bit looking into the current implementation of consensus json rpc let me share the talk this is rather a problem statement then the particular uh proposal on how these json on housing this consensus api should look like i got a got a 403 forbidden on that doc if you can open it up oh sorry really um let me open them up right now oh yeah yeah see now it should work yeah woman so um let me give a bit of the context on that we have the consensus json rpc implementation which we used for ionizing and it worked well for the purpose of friendism um and it could probably work well in production but some of us were suspicious about that that it's like production ready and here is like a few arguments uh contributing to this uh um um like uh okay so just yep and the main question here is um if we um go with the json rfc based consensus api um which has some restrictions and which lays some restrictions on the use cases we might probably want to replace it at some point in time in the future so the main question i would ask and the main question that this document states is is that whether we are ready to develop some something uh some new communication protocol before the merge or should we take an easy path right now and like think about it and do it later so that's the main question um i can go through the problems just briefly that i have found in the current implementation and design and before that i would say whatever does actually get implemented is probably likely to be very sticky in terms of difficult to replace once it's in production so i would i haven't read through the stock yet but if there are actually problems i suggest we fix them soon micah has his hands up yep that's mike i just just a quick question did we answer the question from a few meetings ago as to whether this needs bidirectional communication or unidirectional like is it always request from one end to the other end with a response going another direction or sometimes does the other need to initiate um i see and this document denotes some cases where the bi-directional communication is needed or highly desirable so yeah this this is one of the also design considerations right so we use the sorry sorry go on yes please go ahead yeah i mean the the only other or not the only another big question to answer is whether we want to go to rest instead since that's what the new clients use um right and okay there isn't like some topic in this dog also okay let me just go to the uh problems um and yep um i'm not i don't think we will come to any uh conclusion or any solution um at this call and this is just a food for thought for uh the next calls and next minutes okay so the first um the first problem with the existing protocol uh will lack we seem to lack one of the messages that will uh tell the execution client that the beacon block the consensus of this beacon block is valid is validated which is obviously required because if the execution payload of invalid con beacon block will be stored and served uh through the user's json urban seat would lead to some bugs in the services and software um so we need this message we need this explicit message because uh we also have a set head message that tells what is the head of the chain but um not every block is becomes the head of the chain after it gets imported it gets imported so we need a separate message to signify that consensus is valid um it's essentially a commit after initial processing right so it will look like the new payload is sent to the execution client it's being processed and while it's being processed it will receive this new type of message that display that the beacon block the consensus block of this payload is valid or isn't reality then the payload will be either discarded or persisted after the processing yeah the next thing is we will we have like several messages that are causally dependent like a new block um like new payload and the set head and this um new type of consensus processed i didn't um consensus process message so and the current protocol relies on the assumption that all causally dependent messages will be uh the order of all causally dependent messages will be preserved on the consensus side and on the execution client side so they will they need to be pipelined uh which is um like which is just bug prone so we might want to reduce we might want to just release this assumption to get rid of it and in order to do this we will have to uh this um like execution client will have to store some state of of the messages received from the consensus client like if it's received the new payload it can then receive this um set head or consensus process messages and the order of these messages will will not be preserved then it can um gather it and perm uh gather this whole information in this kind of state in this kind of cache and then decide what to do with the payload so this is one of the things also uh the next one is http overhead which requires the new connection each time the request is being sent also um we can't do asynchronous communication with http only we can do this with some kind of techniques that allows for this so we might want to use something like web sockets which opens a way to bi-directional communication and like the last use case uh it's a failure recovery uh let's assume that the execution client just crashed and it and the consensus client stored persisted some block uh while the execution client doesn't have the payload for it for it so the payload hasn't been persisted so it starts up and uh with the and the consensus client will send like the next block uh the block uh which execution client doesn't have a parent for so and it will and according to the current state of the arts it will the execution client will have to go to the network to pull the state um and to continue the execution which is suboptimal we might want to um like for more we might want to look at more optimized scenarios where like the execution client starts and sends the status message with the head of the chain to the um consensus client then consensus client decides what to do if this gap is only one or two blocks it can replace those blocks without making execution client go into the network and so forth uh also and yeah and the last like that was the last use case um and the overall thought is um would be great to have it extensible so in the future so uh yeah it would be great to design a protocol that can be extended with new messages with new use cases uh without need in designing the new protocol because we have some restrictions that's it that's it you you said that for the failure recovery the execution client can go to the extensive client and ask for the last two blocks does the consensus client keep um it can send the message and say uh here is my head and consensus client can decide what to do it can ignore this message and execution plan will have to go to the network to pool the state or to prove those two blocks um a consensus client might store like the last few blocks in memory or something and so you can opportunist optimistically ask hey by the way do you have this because i know you're local if so given to me if not i'll go find it myself right right this this kind of behavior it is what could be like more optimal than just going straight to network does um okay yeah that's fine um yeah it's even if it's these blocks are not stored in the memory they are stored in the um database so they can be replayed but not the bodies right you still have to go to network for the bodies um now the bodies are also stored am i not mistaken no i mean the bodies are certainly in the beacon block uh but you could imagine using the execution engine locally to store the body so you don't have uh redundancy there but and i mean we already have a state blow problem this feels like we're doubling that is that mine correct like if if the kinetic line and the extrusion plant are both storing full blocks full bodies then the only difference is just the state which is only like you know a quarter a third or whatever of our total state flow problem um i i guess that at some point um like beacon block clients will not store blocks beyond the weak subjectivity checkpoint ah okay yeah that makes sense yeah the failure recovery case is one of the um cases that depends on the bi-directional communications and also it could be a syn process we can if we need some rich scenarios for statesing we could it could also be relied it will also rely on the bi-directional communications right i think the one thing missing from this document is potentially messages required to communicate during state sync yeah it's been mentioned like in the last section but just briefly okay gotcha explicitly so just one small comment from me that um each http request requires its own connection that is avoidable there are ways to have persistent http connections not sure if every library we are using supports it but i see you see okay yes so um we can use web sockets which is which is already supported by yeah yeah but even even there's a keeper live uh in http one uh header that allows you to keep the connections first until it time's up oh it's equal yeah yeah great um good point i was uh thinking about http 2 um but it's yeah but why not to use web sockets as they are already supported by clients i feel like websockets is like my gut tells me given the like the problems you laid out um websockets is probably seems like the way to go just because it's easy like yes you can do http keep live and not too hard but people aren't familiar with it libraries often don't support it out of the box so you have to you know fiddle some bits um websockets are just kind of just out of the box they'll do exactly what you need to keep the connection alive they'll let you know when the connection dies and the connection won't die randomly like with uh you could have a timeout in theory with an hp uh just keep alive um and they give you you know that bi-directional communication so you don't have to just open you don't have a don't have to run an http server on both um clients you just have you know one of them is a server one of them is a client that's how you establish the connection and then it just runs from there so maybe one last comment from me uh http two connections are also persistent but i agree the word web cycles are probably better for our use case um maybe you mentioned this is one of the reasons another reason that you might want bidirectional here is async processing of insert block so that the execution layer can tell you once it's done rather than you waiting right yeah that could be can be done with http uh like servers and events techniques or anyway yep how long does that do uh expect to take kind of in the worst case scenario like how async is that how long does it take to press this in each one block okay it's like 250 milliseconds order magnitude yeah and maybe like the worst os block is sometimes uh yeah sure okay i was wondering if it's like we have to worry about like htv timeouts kicking in at two minutes or something yeah no i don't think so so maybe that's not actually a design use case um so but is is gary's comment correct and that that's the primary reason that we want bi-directional is the uh failure recovery case um yeah but potentially other cases that we might not identify this far and uh yep asynchronous communication would yep so what would be implemented with this bi-directional application channel as well um also i don't want to speak about this today but if we will um looking into redesigning the protocol we might also want to look into encoding so json um it's also gives in our head so it's better to use some binary encoding and then we can ask whether to use ssc or rlp or whatever else yeah i mean i think that's an important consideration that we already have like two protocols that solve the same thing which is basically talking between components and the more we have the more we increase that security surface and audits become tricky and it's just annoying to write a client like it if we have web sockets json rpc http rest uh and maybe grpc somebody will soon mention um that's a burden for developers right isn't the idea you're correct if i'm wrong there were this conversation like websocket json rsp or whatever would replace http rest in this client or no so there's the user facing apis which are defined in that restful http which there's this is independent of that and should be discussed independent of that but the fact that that exists in the stack already yeah one question from me about the payload size wouldn't it's a question so i don't know wouldn't it be even like better to keep it as json but just enable some compression methods then rather than doing binary uh not sure which one would produce smaller payloads um yep also good point because we deal with a lot of numbers binary will almost certainly uh be smaller just because json numbers are gigantic because they're strings that means that json does compress a lot and you get do gain a lot by compressing it yeah we do web sockets uh support like something like any ow yeah okay so it could be on top of the sockets version yeah website because it's just bytes on a wire you you can compress websockets uh text messages with js with gzip the support in servers was bit hit and miss that was a few years ago um [Music] and yeah it's surprising what you can get away with when you gzip json like very surprising i used to do market data over json yeah be careful about discussing modifications to [Music] the payload format to something that's kind of opaque um if we don't necessarily need it so i'd want to see some numbers before we swap that yeah i mean the other thing is in the standard api we've started using the accept header to negotiate content types um so you can get an ssd formatted block or state for example um but the default is json and that's really useful to be able to upgrade and say hey i support ssd i want to save some bandwidth or whatever does anyone have an argument against json or sorry against websockets here i see a lot of people saying jazz websocks sound great does anybody disagree with that i think it's probably a good fit here the only concern i've had with websockets in the past is that it doesn't always go through i've always done websockets with a for the plain hdp fallback because inevitably you find firewalls things that just don't do the websockets upgrade or kill off the connection regularly um but i don't think that's really a design consideration for us that's a more public website type stuff i'd i just echo what justice said in adding another thing i i think that we should work through the design considerations that mikhail has placed forth here and at least fully validate that we really need the bi-directional before committing to taking on another protocol so websockets i believe are implemented in all clients right now all major clients for the json rpc endpoint you can do websocket or http so i don't think for just the websocket part i don't think we'd be adding any new technology if we did like rest over websocket or something new or i guess rest over stock but maybe not quite anyways we did something new on top of web sockets um that might be new but json over websockets is already on the execution engine uh yeah that's true that's true yeah that's usually engine only yeah but libraries that are to implement clients of json rfc are supporting web sockets i guess i mean like what whether j for example curiosity do you all have websockets implementations in in them and on the nimbus side uh yeah we do i mean it's not a problem that way really it's more that it becomes an incredible zoo like all right so so you want to use this beast right and then you have to have a websocket server running and an http rest server running and an http json rpc server running and a dev p2p port and um a p2p port and the discovery v4 port and the discovery v5 port and that's well shouldn't have before well you know so i mean that's six already uh and i just ripped them off the top of my head and and that's where the complexity lies it it's just difficult to uh or not difficult it's it's just a lot for even the user to manage and set up and imagine the firewall rules for everybody and blah blah blah so i mean it's not really a question of which libraries are available because there's a ton of them but each library also brings in its own dependencies uh configuration complexity um the overhead to learn those frameworks really like the ins and outs and the details of webs pockets versus playing http versus rest over http which has a different framing and so on like that that's more of what i'm talking about like the complexity overhead in general not not whether a library is available or not i agree with all those the latter point you made for the first point though i feel like even if we did http here you should still be exposing this on a different port than the public-facing json rpc stuff so at least for the firewall stuff you will need a new port i believe or you should have i agree with the letter half of what you said though that it does increase complexities at another frame yep thanks everyone for your valuable inputs um i guess we should think more about it before making any decision and look forward to the use cases to potential use cases that we might see in the future before we are binding with one or another solution um what can like what can a little bit reduce the complexity of design and implementing this protocol is that these two parties that are communicating via this protocol are going to trust each other so but i don't think it reduces it significantly my two cents on danny's question as to whether we need bi-directional or not um again my gut feeling from the conversations i've been overhearing is that there's enough situations where we think it would be valuable that it feels like eventually we're going to need it like it's one of these things where sure you could argue that any of the individual examples you know we could get we can get away with not having bi-directional communication um but they'd be a little bit better with bi-directional but i feel like there's enough of those that like my my my background tells me that eventually that's just going to continue to pile up and you're just going to end up making sacrifice after sacrifice after sacrifice if you don't have that bi-directional communication whether that's long-lived http or websocket or whatever i think matters less but i do feel like bi-directional is again just from what i've been overhearing feels like the right way to go i guess the question here is like um isn't kind of the future that the the execution client um becomes more and more minimal in its feature sets like um will more just be there for verifying blocks and maybe producing blocks so i guess like i wonder if that is really true like if if we if the long term isn't different like it like should users really long term rely on the eth1 um rest api for example to get that data no like i mean they should use one api and that's probably the eth2 api because that's the only one that can get you consensus yeah but there is a lot of there is a bunch of um api that's exposed by execution clients so um and i guess a lot of services are using it it will not be easy to replace one with another or to move it to another endpoint and uh another protocol i mean i think like we'll just have to do it long term because like if we don't do it then like it will always be like i mean it's always going to be pretty weird like why why do i do you really think in the long term users will want to install like two separate piece of software configure them and so on like one of them should be really really minimal in my opinion and like be more like a library so yeah what i've envisioned which i know not everybody agrees with is that um over time many years we will probably move into more pieces not less but they will become packaged better and so from an end user's perspective you double click an installer or whatever and it installs you know three pieces of software like three services on your host you don't know that because you're a user you just double-click the thing but there's three pieces of software and one of those pieces is like a basically like a reverse proxy it's just a thing that you connect to as a user and then that connects to the two back end pieces um this would be the more traditional architecture and i think packaging matters a lot there like we do want it to package into a single double click for users but for more enterprise focus customers you know they will benefit from having those individual services that they can talk to separately yeah i agree i guess i disagree in that i don't see why except for compatibility reasons you'd want to talk to the eth1 client directly in the future because you care about not about some random state you always care about relevant state as an in consensus state when you ask questions about ethereum and so it doesn't make much sense to me in the future i think to ask each one client except that's it if that's the only thing you can do because you have been written before this existed so i think the the primary reason i would want to do that is because each client has different feature sets that are added on beyond the base feature set and the you know your execution client may have a particular features that you want like tracing or something that other clients don't and because it's not a standard feature you can't access it through the um client like you need to go directly to your client because it's added a special feature just for you or another mine for example has plug-ins so i can write a plug-in for another mind the consent client knows nothing about that and so if i want to talk to other mind i have to go directly to it yeah i mean and the execution engine would have it does know what the head is so for many of the things that you do on a query it does have an idea of consensus in that sense but again once you package this up nicely and have just kind of a standard proxy to get get to it all um i think the the common end user doesn't really have to think about it yeah right so every execution client will be accompanied with the consensus client which is which is which is this through and by yeah also for this use case it's probably where we need the consensus um data as well so it could be like the unified advocate that can request data from the consensus and combine them with the data from the execution clients potentially this is one of the financial design solutions here and then get back with this data to the user so it will be one interface well does all the things okay so um let's stop here um thanks everyone again for this discussion um okay do we have any other research updates okay okay like let's move to the plans for q3 this is the first day of q3 i think it worries um like shares like speak about plans for a little bit because we're expecting london in a month we're expecting altair in a couple of months right or something like that and with all this in mind we're expecting um more focus on the merge during this quarter and uh regarding the plans for this quarter we have uh um okay so we so far we have like a beacon chain or consensus specs are in the feature complete state so definitely there will be some refinements bug fixes and some additions like on the networks back and uh if the api changed as well uh but in general uh we have the design we have the transition process so far and it makes sense for q3 to focus on the execution client specs on the eips on the consensus api um that's what we are going to do in this quarter also would be great i think we will have the more test nets coming in the second part of this quarter so that's the high level of view on the plans yeah i think um yeah the consistent specs will also rebase on altair relatively soon and also integrate um london changes the execution payload which i think would include something related 1559 um and also figuring out standard how we're going to be testing i think we already have uh consensus side test factors being generated we'll be extending that um and then figuring out how the execution layer leverages the existing tests and extends them in this new context i think that's something important to figure out in q3 yeah those agree we need that like but by default all the evm stuff should just continue and they should operate independently but i think we just need to kind of touch it and make sure that we're happy with the way things are structured also at some point we are going to stop making the separate merge calls probably we have one or two um after this one and then go to um and then we'll keep discussing marriage uh during the all-core deaths and the proof-of-stake consensus um call dependent on the part which has been discarded which is going to be discussed um yeah so that's how that plans let's mika and i've been working on a check like very high level checklist of all the things that we'll share soon and probably put in the pm repo okay um any questions or suggestions to the plans great um any spec discussions actually we already have one okay any other discussions does anybody want to say anything else before wrap up i'm good great okay thanks everyone um see you on the different places soon um i will not make the next call uh the stick call so i'll see you thanks everyone for coming thank you bye bye thanks everyone thank you thank you cheers hey hey danny is it awesome did you already leave i just wonder if it's possible to have the same um zoom server for both calls since they're back to back just a little more convenient uh let's see uh okay actually uh this is like uh um a link provided by us like cara does so it goes in there and the call that danny manages it is from a gm foundation so they may have different links it's not a huge deal if it's complicated or hard just be convenient right this goal will be deprecated soon anyway so one or two yeah that's true okay i i do have the new link i believe danny sent in the morning email today so i will thank you michael okay thanks bye-bye thank you later