Evaluating the PBS Experiment: Early insights from MEV-Boost and the Builder Market by Jolene Dunne

foreign [Music] hi everyone uh my name is Jolene and I work at flashbots research and today I'm going to be talking to you about evaluating the architecture called proposer Builder separation given the things that we've learned and the insights that we've gained since the deployment of math boost and I'm also happy to take some questions at the end if we have time so as a brief introduction uh what's the point of this how are PBS and mavoos linked or how are they different so probably most of you are familiar with these ideas already but just to make sure that we're all on the same page proposer Builder separation refers to an architecture that separates the functions of block building and block proposing within the core ethereum protocol so instead of the block proposer in this case the validator going to the mempool and building their own block which they would then proposed to the rest of the network all block construction would be outsourced to actors that we called block Builders so the idea is that the only thing that the validator would have to do Under full PBS would be to choose the highest value block that's presented to it when it's time for its thought block building is a you can think of it as like a revenue generating activity there are economies of scale involved in it and opportunities for things like private order flow that might give you an advantage over other builders so basically there are incentives for Block Builders to centralize and to become more powerful as time goes on so that's a trade-off that was kind of known about and made and accepted at the time the PBS would take this centralizing Force out of the ethereum consensus layer and put it into the Builder Market and then logically the only kind of interface point would be where the validator is sending these calls to the Builder Market on picking the best block that comes back like I said so the idea is that kind of stuff can get uh very ugly in the Builder Market but we don't care because the core of ethereum the consensus layer is safe from the effects of Mev so where does Mev boots come into this and how is this related so meth boost allows validators to engage in an experimental version of PBS on this Builder market so it's completely optional but if you have a validator and you're now running mob boost you can add Source your block building to a kind of a burgeoning market of block Builders relays Searchers that are all competing to give you the best block and the reason that this happened was because um if you're familiar with the old flashbots auction this wasn't going to work after the merge so we knew that we needed to kind of upgrade it and do something different and so flashbots worked with the ethereum foundation um to make this piece of Open Source software that could act as a prototype for this longer term Mev solution for ethereum which was PBS so in this way we are able to test this idea of PBS through mad boost rather than kind of putting it in the protocol and then being in trouble if it doesn't work or something bad happens here is a quick diagram of what I'm talking about when I say the Builder Market um and I really just want to go through this quickly just to make sure that we're all on the same page because the later things will be a little confusing um if you don't understand this so what we have right now kind of on the left hand side is a validator running mode Boost Mobile Boost allows it to connect to this Builder Market if the validator is not running move boost officially it should not be participating in this Builder Market so in order for this to work there are certain rules that need to be carried out in the Builder market so over on the right hand side we start with the users so they originate transactions and those transactions either go to the public mempool on the top or maybe some private mempool that Searchers or Builders are allowed to access um and then we have Searchers who would be monitoring the public mempool as well as any private sources of transactions that they have and they would be forwarding their bundles along to these specialized actors that we call block builders block Builders will be receiving bundles from Searchers maybe they're also monitoring the public mempool maybe they also have their own private sources of transactions too and they will be trying to build the most profitable block for a slot so they'll forward blocks to relays and the relays here it's different to if you're familiar with the flashbots architecture before the relay here kind of has a different role so it allows validators to register with it and then when a validator requests a block for a particular slot the relay will forward the block that has the highest payment attached to it and then that validator will receive that payment for proposing this block to the rest of the network so when I say the Builder Market experiment webmav boost this is what I mean uh and when we talk about full and protocol PBS we are trying to figure out a version of this that has the important bits that enable this Market kind of baked into the ethereum protocol so that means nomev boost and probably no relays as well um so the question that I wanted to ask and kind of answer try to answer in this presentation is how is this experiment going so map boost in this early version of the Builder Market have been it's been alive for about a month now so can we tell if things are going the way that they thought we would would it be safe to put PBS in the core protocol today you know as this experiment is without any further features or enhancements and are there any warning signs or maybe early indicators of things that we should be worried about when I was thinking about this question like how could we tell if the experiment is working I started thinking about PBS as a kind of an organizational principle so it seems to be very widely accepted that this kind of clean separation um between the Mev activity on the rest of ethereum with only this Market in between as an interface point is desirable and also possible so it's probably not very helpful that the idea of PBS is I think relatively simple and easy to understand and sometimes we can be biased towards things that are that are like that when the real problem that we're trying to address is much more nuanced and complicated so the previous question kind of turned into these questions is abstracting the Mev away from the core protocol a good obstruction or does it leak are there more interface points between where the Mev is happening on the core protocol then people would lead you to believe and if things get very very bad in the Builder Market maybe imagine a very small number of very powerful very centralized actors that dominate block building are there really no knock-on effects on core ethereum assuming at some point in the future we have a full and protocol PBS in this uh in protocol PBS world I started thinking about what parts of ethereum are still influenced by Mev and I came up with kind of two families of effects so the first order effects are kind of the Direct effects on the core protocol so these are places where even though this Mev is supposedly over on the other side of the room and it doesn't affect ethereum it's still it still influences part of it and the other type are more kind of second order effects so how would how are actors incentivized to try and influence the base layer if it meant that they could gain more favorable conditions for themselves so my idea for this was to try and identify a few concrete examples of these effects and then kind of check data to see if there's any indication that these things are happening already and then if these effects were to get more pronounced as the Builder Market develops are these really an issue and if so what kind of medications might we need so let's have a look at the first kind what parts of ethereum are still affected by Mev under fall in protocol PBS so it might be more than this but these are the main ones that I came up with for now so the first one is the contents of the block so we are now getting these blocks from the Builder market so now all transactions in the block can be directly influenced by Mev and this wasn't the case before because it was possible for minors or validators to kind of opt out of using my booster MAV geeth if they really wanted to so in the future if most or all of the blocks come from the Builder Market the purpose of the builders is to reorder and rearrange transactions their block can be as profitable as possible so hopefully that one's kind of straightforward this kind of idea of like losing control of plot content the second one is maybe not so obvious so the contents of the public mempool are influenced by Mev Under full PBS so how can that be well in a world where we have full-in protocol PBS Builders are incentivized to have private sources of transactions that we talked about earlier and we call this exclusive order flow so if Builders are competing just on the contents of the public mempool there is and there's no difference between the kinds of transactions that they have available to them then their success comes down to um maybe their algorithm or their trading strategies however the more order flow that they have access to that other builders do not the more likely it is that they can build a better block with that on our flashbots we generally think that order flow is going to be a big differentiator in how successful block Builders are going to be and if this order flow is exclusive in that it's going to a private mempool instead of the public mempool um it's it's not going into the public Med pool so you you could have this situation where um as as there is more Mev activity because the the Builder um what is happening with the builders is associated with this Mev than the contents of the public mempool might get smaller um and then the last one which kind of relates to the first but maybe a little bit of a different Focus um blog posters as block proposers used to have control over um what the contents of their blog would be so Under full PBS there used to be sorry the the big difference would be the validators will now accept full Blocks From the Block Builders and that didn't happen before so in the old kind of flashbots model you we would now never have um full blocks coming from um external sources um and then the second thing is that uh I'm not sure if this is for certain yet but we think that you know Under full in protocol PBS there may not be an option for Builders to or for validators to opt out of that so you might have no choice but to get your blocks from um from an external market and then kind of going on to the second family effects so how are actors in the Builder Market incentivize is to mess with the base layer the consensus layer in order to gain an advantage on the big issue that I see with this is that the Builder Market is actually out of protocol so there's nothing enforcing participants in the Builder Market to stick to the structure that we saw in the diagram earlier the map boost architecture that exists is kind of a template for what is safest for like independent ecosystem aligned parties like flashbots to run under a certain number of assumptions baked into it in reality we can't stop anyone from using for example their profits to buy validators their profits from building creating their own private version of mevust or offering services around Builders or other parts of the system and as we're about to see measuring what comes through a relay might not necessarily be the full picture of what's happening in the Builder market so the only part of the Builder Market that's in the core protocol is this Builder specification which divines like the API calls and sequence between the validator and some software that looks like MAV boost on that side of that kind of anything goes by now we've identified a few kind of concrete areas where we think that Mev might still affect the core protocol Under full PBS so what can we do with this well there have been a lot of really awesome data collection efforts by various parties in the ecosystem in the last few weeks since the merge what if we took a look at some of these and see if there are any clues based on these areas that we've identified it that this prototype Builder market via Mev boost is starting to affect the core protocol in an undesirable way um and so the first question that we want to ask is going back to what we identified previously how much block content right now is currently decided by the Builder market so here is a graph that measures how many blocks have come from relays since the merge so as you can see um as of today we're up to 60 percent of blocks coming through relays and I looked at these this graph last week and it was about 45 so it's increasing uh quite quickly and again it's important to note here that this might not be the full picture so many of these dashboards measure what comes from relays and this might not be an accurate reflection of what blocks are coming from the Builder Market as a whole so it's probably best to think of this as a lower bound on how much block content is coming from the Builder Market that seems to be increasing over time and then on to the second question so is the public mempool changing since math boost launched so here is a graph of the count of unconfirmed transactions in the mempool over the last 30 days and if we look back at when the the merge happens September 15th it it looks like something may be different so the Baseline seems to be a little bit lower and what I would expect to see over time if my speculation is correct is this Baseline kind of trending downwards however I don't think I could say at this point that this is definitely because of Mev um my plan is to to keep monitoring this and look into other things that might be affecting the mempool since the merge and the next question is linked to the idea of things kind of getting nasty in the Builder market and if there are actors with big advantages that they might you know use they might be in a position to use those to start to influence the base layer so are there early indicators that Builders are centralizing so this chart measures block Builder's success since the merge so percentage of blocks that landed that came from a particular Builder so if we think of that 60 that we saw in the diagram a few slides ago how many of that 60 came from a particular Builder and the orange one is flashbots so that doesn't really look great right now however if you look closely it does seem like flashbots is losing its lead a little bit as time goes on or as some other people have pointed out to me that the non-flashbot slice looks as though it might be kind of diversifying a little bit but the point that I'm trying to make with this is that it doesn't look like flashbots you know is as the dominant Builder has a lead that's growing and growing over time um but this is probably another good thing to keep an eye on um and so when I talked about actors trying to influence core ethereum in ways that we haven't thought of yet I think the big one for me is whether Builders and validators will eventually decide not to bother with the out of protocol Builder market and just start making deals with each other directly and this is the kind of activity that's much more difficult to see we really need to keep in mind that the Builder market so this thing that validators are able to send their requests to at the moment on what goes through Mev boost on the relays is not the same thing so what we would need to find out here is if there is a way to tell if there are blocks produced via some external source that are not going through math boost or a relay and I think this is possible but and managed to do it in time for this talk but by examining this number over time we could deduce that perhaps Builders are being connected directly to validators bypassing the relay components entirely because if you're a big party with a lot of order flow and either access to a lot of validators or perhaps the means to acquire validators you don't need to participate in all this relay and Builder competition business so I think we want to keep as much of this activity in the light as possible this is something that I think is really important and will continue to work on so looking at the indicators that we found are these things actually problems if the trends that we identified continue and possibly become more pronounced what negative effects on ethereum could we expect so we saw previously that we are currently around 60 of ethereum blocks now coming from the Builder market so extrapolating from this should we be worried if 90 of blocks are coming from the Builder market and I think the main problem with this is censorship and hopefully you can see by now why censorship is the big kind of topical issue with PBS that a lot of people are talking about if the centralizing Dynamics that we're expecting start to play out it's possible that you know 90 plus of blocks start to come from a few actors and as we know it's it's easier to censor at the Block Builder level than it is to censor at the consensus level we also saw that there seems to be a decrease in transaction volume in the public man pool post merge and if that speculation is correct and it turns out that this is a symptom of transactions being diverted from the public mempool and into private pools what implication might this have and one of the solutions that people propose um or maybe one of the kind of additional features that people want to see for math boost or full PBS is this idea of a fallback solution so that would be allowing you know if something goes wrong you allow your validator to build blocks locally and then compare those to Builder Market blocks that have been produced and then you would only accept the Builder Market blocks under certain circumstances so the issue here would be if the public mempool you know maybe got so small that effective fallback Solutions like this either weren't possible or relied on the validator being altruistic in that building from the public mempool alone is completely not competitive compared to um the Builder Market um and for this so we still don't know if this is happening yet but if it did happen that we end up with builders that have ad size profits it might make sense for Builders to start staking their profits and running validators and there might be entities that own a lot of validators already that want to start block building and there's likely to be a business model where Builders can offer services based on how many validators that they own so I don't know if anyone saw Alex Stokes talk yesterday on block building after the merge but he had a slide with like a giant list of different ways that you know block Builders could monetize this kind of relationship and decentralizing the Builder role itself might not be feasible here as we're kind of out of protocol at this point and therefore Limited in what we can force Builders to do we've identified a few different things that if left unchecked might cause US problems down the road and I think there's good awareness around most of these already especially the censorship resistance schemes which is very topical but there are a few other things that given these early indicators might also be um a priority so censorship resistance I think we know about that already something called transaction inclusion lists I believe is being actively worked on by flashbots and other people in the ethereum ecosystem um many it's over the other things many of the effects that I talked about can be lessened if we take steps to ensure that things can't get very centralized and ugly in the Builder market so trying to figure out how to effectively decentralize the Builder role might be important while also keeping in mind that a decentralized builder has to be competitive compared to a centralized builder in order to make any difference another thing that I think is important are mechanisms to ensure that Under full in protocol PBS validators can still build their own block or perhaps augment blocks that they receive and the reason that this is important is that a lot of these kind of extreme effects that we extrapolated assume that Under full PBS 100 of proposers will be getting blocks from the Builder market so this would be a way to kind of get back control of that block content and then if at all possible reducing incentives from validators and block Builders to work together um is something else that I think is important and following on from that working on the ability to monitor the Builder Market as a whole by putting something in the protocol that can tell whether blocks have been built locally or not so that's pretty much it thank you very much for coming I'm wondering if there's a method where some of these solutions could be made by the community and be somewhat attached to something like math boost kind of almost like a module that you can add if you opt in yeah um I think I think that's a really great idea uh I think it depends on the solution math boost at the moment is a fairly lightweight piece of software and we have thought of these things a lot of the things that we want to do with like transaction lists and other things like that require knowledge of State of the ethereum State on map boost just doesn't have that but I think that there are definitely maybe some other things and other ways that mebus can be improved and um you may have already seen we're also trying to make math boost more of a kind of community effort as well and take Solutions and from from other other parties yeah hi Jolene thank you that was a great talk just wondering the the um solution where you allow Builders um sorry validators to augment the block doesn't that still require or still rely on them being altruistic like isn't there is there is there a situation where it just doesn't make sense because it's so much more profitable and not much goes into public Med pool perhaps yeah um I I think I think you're completely correct about that in terms of incentives and I generally think that it's it's really important for us to kind of design these things in a way to make sure that people aren't missing out by doing things that are good for the protocol um so yeah but I I think one maybe one example of where this has actually worked well is around the time of the merge uh when there were some was there some problem with my Boost or something and a lot of people said that you know they weren't going to run math boost so it was more kind of like a social like altruistic thing so I think ethereum is kind of special enough in that um it does happen that people have an interest in in the protocol and want to do the right thing by it but you're completely correct yeah if it got to the case where it's just not profitable to augment your block we'll have to think of something else