Building for a World with a Thousand Blockchains - TOKEN2049 Singapore 2024

as introduced I'm matz I'm the co-founder in CTO of Dune I'm here to talk about what happens practically when we go from building on one blockchain to building on 10 100 a thousand so just quickly run through the agenda I'll talk a bit about myself and about Dune how blockchains scale and essentially what kind of trouble you get into when you're starting to program against 10 or 100 or maybe a thousand in the future so I'm MZ as I mentioned uh co-founder and CTO of Dune started Dune in 2018 with my friend Frederick and a fun fact is that you could buy eth for under $100 that year if you could stomach it after losing a lot of money in the 2018 crash one ask from the audience that if you have a question or want to follow up on something you can reach out to me on foraster uh just at mods or mods d.com so if you don't know Dune we're V crypto data platform been around since 2018 and we essentially organize uh decode and make human readable blockchain data uh pretty sure the numbers here are a little bit outdated but we support 60 plus chains across 1.5 million tables and data sets uh 25 evm chains and Counting we support salana apto sui Bitcoin Stark net pulka do and more than 50 parach chance the unique thing about D is that when you log into the platform you're dropped in with half a million crypto data experts it's not like a traditional uh platform res restricted to your team or restricted to your company sort of open and Limitless and we believe that sort of knowledge compounds when you work in the open so that's a sort of unique thing about June on Dune we store a petabyte of data a th000 terabytes now with the caveat that this isn't like optimally stored there's a few different copies of the data but this is what's needed in order to serve uh j.com half of this is salana by the way uh but by definition and this is what we'll talk about today blockchains only grow and so this number is only going to get larger in the future and if this number represents maybe 70 chains and what will it represent what will it be if there's a thousand l2s or l3s or l4s who knows just want to quickly touch on how to scale a blockchain um this picture on the right hand side is from playground AI it's complete gibberish uh but I thought I'd include it anyways because it's a cool looking diagram but essentially when we talk about scaling blockchains uh the goal is to improve the um increase the available block space or increase increase the throughput or bandwidth of the system so the good news is that a blockchain is just a database and we've been scaling databases for 20 years so there generally two ways to do it you can vertically scale a database or you can horizontally scale a database and we all know salana sort of having taken the vertical scaling of a blockchain to the extreme like you essentially build a larger faster blockchain um utilize all the available Hardware and you know try to make it go as fast as possible then what we'll focus on in this talk is actually the horizontal scaling this is what we see in ethereum you have op stack arbitrum polygon Etc all building l2s and l3s that connect to etherium uh with cryptography so I pulled some numbers from my favorite data platform Dune um and this is just the Opie stack on the right hand side here so we're looking at the number of transactions that are being settled across these l2s uh on a daily basis so with the addition of Base uh coin bases breakthrough L2 the op stack is clearing around 5 million transactions a day and that's all settling to ethereum it's doing 1.6 million transactions today so if we include sort of the arbitrum stack the polygon stack the CK syn stack Etc let's roughly say that we're clearing around you know 6 to 10 million transactions today using the horizontal scaling methodologies in ethereum having a look at salana uh who's doing the vertical scaling they're clearing 40 million transactions a day um 40 million successful transactions a day and that's somewhere between five and 10 times the amount of the horizontal scaling um anyways today we're going to focus on the horizontal part because um that's really poses some challenges for application Developers and for users so just taking a step back and reasoning about the problem so fundamentally the blockchain is a globally replicated and distributed database um and program programming against one blockchain it's you know it's not trivial but it's not super hard the problems come once we add l2s and l3s and we start deploying apps on them and now we have like these collections of replicated interconnected databases and unlike in the traditional uh sense you don't have a single entry point to all the data across the the horizontally scaled databases you have to go to all of them so the fundamental problem that we're going to explore in this talk is that now user data and app state is distributed across all the different chains that you're deployed on this could be like token balances it could be nfts it could be app interactions so what we're going to talk about is how do you get this data and how do you present it to users either analytically or through your app so as the co-founder of Dune there's probably no surprise to you that one of the first thing I think about is analytics so one of the problems that you'll face when you deploy an app on all these chains is that you have data fragmentation and so measuring adoption and traction becomes really tricky because if you omit certain chains um you might be lucky and that just paints you know like it just seems like you have a little bit less adoption uh but at worst it could paint the entirely wrong picture so Dune uh you know Dune nonen kaito all data platforms out there were all uh working with chance to get more coverage to paint a complete picture and that's really important the second dimension that I think a lot about as the CTO of Dune is how difficult it is to program applications against multi blockchains so I believe in in order to have the right ux for the users when your app is deployed on 10 15 100 Chains It's critical to retrieve the full user state from all the chains to have the right ux think about like uh on Unis swap today you have to select your chain or yesterday you have to select your chain before you could swap uh today or tomorrow you see your tokens uh across all chains and I think all apps are going to go this way the big problem though is that um when you Deploy on many chains you end up with a phenomenon called tail latency essentially your rpcs and the apis that you program against are going to have varying latency and you're bound by the slowest one so when your users really care about how Snappy your app feels and you're working with slow rpcs or slow apis this really you know really kills users vibes so just a side note that dun is sort of working on both dimensions you know on the first hand side I I mentioned like dun is onboarding tens of chains every quarter to June to paint an right picture for blockchains to understand the activity on their chain for developers to understand their activity on their protocol and for users to understand what the hell's going on out there in crypto secondly um we've extended our mission to make crypto data accessible not only for you know chains and users and developers but for app developers like we think we play a role in abstracting away the complexity in retrieving this state from all the different blockchains we found that this isn't necessarily that easy though we're taking on a lot of complexity that developers otherwise need to take on I think developers take this on uh anyways so we'll talk about a few things that um will get you in trouble once you start deploying your app across many chance so one of the things that's tricky that we see is finality like ideally you don't think about finality but uh in the edge cases you definitely need to and something as simple as a token swap or a token transfer it might only be confirmed after two five 10 30 minutes uh depending on whatever chain you're deployed on so your code need really needs to be reor aware uh we thinking about like an app that you really want that Snappy update you want the optimistic update in the UI when a user does an action well what happens if you notify your user uh that the swap went through and then 5 minutes later the chain has reorder what do you do then um so it's really important for apps to think about you know does my backend support this and how do I actually uh give my users this information if you program against multiple evm chains you'll also find sort of uh devil in the details type of thing so all the rollup Stacks out there uh are evm compatible on the Virtual Machine level um we found that that doesn't necessarily mean that the Json RPC uh apis are according to spec so we found anything from like missing Fields added Fields altered Fields changing meaning changing types Fields only appearing after 10 15 minutes uh wrong ordering inside arrays like all kinds of things that you can get into once you start supporting 100 blockchains if you're doing anything analytical you probably use the Tracer debug endpoints which are the worst culprit of them all and if you rely on some third party libraries that may not have support for the network that you're working on it might outright be impossible without forking it so we run on a fork of ethers instance because we uh need to support CK sync and Trum both of which have some data validation issues so I I started in crypto in 2018 and we talked a lot about decentralization back in the days so but essentially today I think it's virtually impossible to build a self-hosted application once you go beyond three chance you have to self-host noes backends and data platforms for this stuff and it's really expensive really tricky so essentially all the app teams they need to rely on services such as dune or quick node or Alchemy um and I think that's good but it it has implications for decentralization because it centralizes infrastructure I mentioned we would talk about the horizontal scaling a little bit but one pattern that I see a lot right now is that apps that are mainly deployed across evm chains uh are also adding salana and um salana is actually trickier than you think so the detail is that messages on salana or transactions on salana are binary strings which is fine same on ethereum salana has Bas 58 um ethereum they're hex encoded the problem is that how you serialize and deserialize data to and from different programs is entirely up to the deployed program so you could end up in a worst case scenario where like every program defines their own Cy um you have anchor IDL standard which helps a lot but if you're deploying a program that's or deploying a program or building an offchain application that interprets uh instructions you end up with a matrix of different serialization and deserialization techniques and it's really tricky to do that at uh scale to quickly sum up what we've talked about today is that you know building apps across chains is actually harder than it sounds um sure you can deploy your smart contract on 100 evm chains but the supporting infrastructure needed to run a Snappy web application it's not uh easy you're going to need solutions for cross chain analytics and cross chain State and data retrieval and yeah it's um you're basically bound to have to partner with infrastructure providers and some of the tricky parts we touched on were finality data inconsistency decoding decentralization and just in general the infrastructure burden of building across chains that's my talk thank you very much again if you have comments or questions I'm available on foraster or email my name is mats and uh final little Shameless plug is that if you want to build a multi-chain application you should try out or developer apis there the classes in the industry with the widest coverage supporting 27 evm chains in a single call and it's free while in beta thank you very much