Exploring the Web3 Data Gold Rush | ETHDam 2023

foreign [Music] so moving on we are going to be exploring the web 3 data gold rush I like dids and decentralized identity but I absolutely love data so I'm very much looking forward to this and I've got an absolutely amazing lineup so I'd like to welcome to the stage Ben from API 3 as well as Lewis from gelato aka the ice cream man as well as uh yeah Anna from anchor and Simon from the graph it's uh great that you guys all managed to be here on time and not too hungover so congratulations we made it cmgm it's a good start of the day yeah we're good we're good um so to jump right into it um please could you introduce yourself very very briefly and what type of data you care about so let's start sure hi um my name is Simon I'm a developer relations engineer for eterno that's one of the core devs working on the graph protocol and um I care about open data I think data should become more of a public good and everybody should able be able to access it and uh and not having this this silos that capitalize on it cool hey I'm Anna I'm from anchor we kind of uh I would say we Amazon web service for web frame we proceed at least 25 percent of the whole market and we care about the whole data and we believe that all developers should have an access to good quality of API requests to note infrastructure and so on hey I'm Luis I'm from gelato Network and yeah specifically I would say we care about validated aggregated and verifiable data um that a set of validators commits to and has accountability for this data so that it can be consumed safely by any application out there without worrying about this data being corrupted or unsafe for their business logic hey I'm Ben I'm from apr3 I kind of lead the ecosystem function um I think you know touching on the points of you know what kind of carrying on from the fellow participants I mean including the API providers in the web for ecosystem is crucial these are the people that are processing the data sourcing the data and ensuring that they can you know look to monetize that API into what is the web free economy as such I think that's an important kind of shift in the way that people consume in web 3. cool thanks a lot um please feel free to disagree with each other on these topics we've had some of that today and it was more than yesterday and I like it um so the the first big question that I have is kind of from your perspective what is like the state of data within web3 today it's very open so you know many different directions with this um yeah the three is a big it's a big domain anyways um but like on blockchain data but a lot about blockchain data is per se the design is open so like everybody can run a ethereum archive node and start to uh see that data but also like it grew a lot and so um this this data is very big so it's not easy to just do it at home and I would say like now we see that a lot of different companies also go in there and then and provide that data and create this data sets for the for the developers which I think is good like the everybody sees it's important and it's here but it's also starting to be more fragmented the apis are incompatible with each other every company tries to build some mode in having developers build on top of the of their kind of uh proprietary apis and I think we should kind of work towards together like a public and compatible apis and tools that everybody can use and that are interpre interoperability interoperable thank you good for Sunday and I would also say that um one very interesting data Trend um that is maybe not so obvious is that um l1s are somewhat pivoting to become data availability Solutions so if you look at ethereum it's actually like you have the single chain but if you look into the L2 contracts on ethereum if you take like a um a periscope or whatever these things are called um you can actually see that there's so many infinite l2s on ethereum like technically embedded in in call data I guess with Deng sharding these data blobs or whatever so um the ethereum network itself is becoming a major data layer data provider um for new blockchains that are built on top of it and that yeah it's like a huge data market essentially for for blockchains suddenly yeah I think if you take a step back you you I mean there's a great talk by Andreas and Top-Up office uh just to make sure I got that correctly about infrastructure inversion right we're in terms of oracles we're four years into five years into Oracle development in what is what a 15-year wider development of the space and you've had the first iteration of Oracle infrastructure I think you're seeing at the entrance to the market it's taken a couple of years for people to realize they're kind of first go-to-market product and uh yeah I think you're going to see a lot more not just amongst oracles but within you know people like gelato the graph rpcs for example this infrastructure is still being built um and yeah I think the next couple of years you're going to see the the kind of the fruits of that labor as such in regards to what people can do when they're developing smart contract applications [Music] um yeah yeah I agree and I would say that now kind of from another point of view we have 90 percent listeners for all chains because we proceed um 8 billion requests every 24 hours and we can say that 90 of requested just to read not to write so actually it's technically it's 95 but okay if you're a builder you still need to read and then write so only five percent is request to write something into any blockchain for some blockchains is more for some it's less but like 90 percent is just listeners we have more listeners we have more you know um stalking for the builders talking for the real data and everybody try to find you know there's some Secrets try to find the trends and we have less people who really developed something yeah so it's kind of interesting it's typical for for web 2 I would say but for the free initially it was you know we had a lot of developers it calls like oh okay it's only Geeks here only developers but now we have a lot of data analysts and I agree that it's very interesting how the L1 and L2 chains are compete to each other sometimes in terms of data and how a layer 1 ethereum become the you know the start for all everyone who wants to understand what happened for the industry and Duna created an amazing product then everybody can follow the insights based on the real data but I agree that still a lot of problems because you know the blockchain is not the consistent database yeah and we have a lot of issues with the data so it's kind of interesting what happened inside Market and the topic of uh obviously we just heard Justin talk about l2s and kind of that narrative within the ecosystem we've also just mentioned l2s now like there's liquidity fragmentation with l2's attention fragmentation as well as the information fragmentation so I'm interested to hear kind of what your thoughts are on that and and yeah um yeah that's an interesting question uh so with with the graph uh I mean you can have sub graphs on every chain or like every L2 that's evm compatible so that way that again kind of gives the possibility to have this abstracted layer on top of the blockchain so like when you query the graph it's just graphql and then you don't like it's not so important which chain actually was was behind um but but I agree like the then the problem is that all these steps um deploy you know on all the chains and then they'll have like and then they need to combine the data again in the front end and then it's a little bit questionable so like where is the liquidity actually or um what's that price on ethereum made it or do I see a price on polygon and then that also confuses the user then but that's just the the nature of the thing huh yeah I agree and now we have a lot of them compatible chains and they all kind of compete to each other yeah so and at the same time surprisingly they compete with ethereum so uh we have like the very strange situation because I believe that layer one ethereum is not for humans so we see even before it was more than 100 per transaction now it's less but still it's too expensive for daily transaction yeah for normal Builders so I would say that L1 is for um B like the more technical it's the basic infrastructure if I would say we have a lot of data models and a lot of engineering stuff yeah how we can explain that but still layer one is just the basics and layer twos now it's something there that people try to find the solution and I agree that it's like a decentralized computer like so we have decentralized Computing and every single chain is find their product Market fit and use case so I believe that it will be the same develop happened in the same direction so it will be layer twos and layer 1 and they will compete to each other and compared to and we will have more the use cases transition between level 1 and L2 yeah I agree with that perspective and I actually start to think of layer one more as securing the most important thing which usually are the crypto assets for example and I'm actually quite optimistic about the L2 approach no pun intended but I think optimism has a really cool idea and vision that I would like to subscribe to and hope it will come to fruition and and there I think the idea that all our assets are secured on a layer one and then they can be safely trustlessly bridged which is I guess the main advantage of an L2 is that you you don't need a multi-stick bridge you can have a trustless cryptographic bridge um so if if you have that world then I could see that applications become more centralized like you said the computer is on L2 more centralized again but the the assets aren't and I think that's really powerful maybe maybe not everything has to be decentralized to to reach these massively scalable applications that might not be possible even right so so I I'm more happy nowadays I definitely also was you know in the ethereum early camp and so on but but I changed my opinion a bit but I'm a bit more happy now especially with you optimistic um roadmap the super chain this idea that liquidity fragmentation first of all might not even really happen so badly because if you have super powerful Perpetual exchanges and so on built on L2 or L3 maybe they attract way more liquidity than they do today from C5 because suddenly they are much faster low latencies on way more Capital gets injected so maybe liquidity with fragmentation would be much better than it is today on an L2 Dex for example and and secondly um I I think that with you optimistic roadmap you you have some idea of the super chain shared sequencing so maybe cross L2 composability or atomicity becomes technically feasible and maybe this fragmentation technologically speaking isn't even so bad and can be abstracted in the future so I'm quite it's in the future but I'm it's nice to be excited about the future right so that's that's what I like yeah just to build on on the point you know layer twos uh have the chance to probably deliver better experiences for for people maybe for potential Perpetual dexes for example and you know from the Viewpoint of an oracle if you're needing to have a very high frequency trading experience you know if you're competing with binance let's say uh you need to have the data availability on Layer Two and the cost of delivering that on layer one at the moment is just it's just going to be unfeasible you're going to have so many overheads and gas fees Etc um and what layer two does is probably gives people the opportunity to have those use cases or experiences that they want to have and with the way that the layer to scale it just means it's more feasible for for the infrastructure to provide it um yeah nice cool thanks um so a little bit of like a quick fire um but before that I'd like to remind if anyone's wants to ask questions please do so on the slido um also one of the main organizers of the event is at home at the moment so I wanted to wave and say hello to Pascal if we can wave to Pascal we hope your leg gets better soon um so yeah so as a quick fire um what is the biggest challenge that you see with data and also then what is the biggest opportunity that you see with data to sell the data sorry to say that is that both the challenge and the opportunity both you know now when we have like kind of Boom of AI we have like a lot of okay for any kind of um Big Data you need to you need the data set to um you know to educate your model and at the same time people want to you know close the data surprisingly even for blockchains people want to close the data and say okay now it's my private data even in blockchain so the biggest opportunity so far I believe that is to sell data and at the same time as a second part is to keep privacy of the people so I know that if you trust someone to proceed your data to send the API request so it's important to make sure that there are no listeners who is go deeply into the body of your requests so and on the one hand you want to you know have a reliable infrastructure low latency and Global coverage or and another point you need to make sure that you can protect your data so and I believe that people will be more and more care about the big data sets especially especially for their client or for their debts and say okay guys please I I know that we have a blockchain and open data and at the same time we want to make sure that nobody use it in bad um in bad way I would say yeah interesting point um you touched on it before data is not data what we see is also like what data can we trust so sometimes when blockchain data is extracted especially if trade system RPC we saw like that sometimes data is missing and kind of knowing that that data that I'm presented is actually the truth although with blockchain we can verify basically but verification is a is a manual step or like at least we cannot automate it but it's not so easy so I think that's currently a big challenge um and a lot of data providers like um we know um are not the data is not super exact like it sometimes can have deviations if someone really dives into it and I think that's something that we also can collectively work on um one research there is verifiable queries so that we we know we've proved that the query that I sent that's actually it's true but I mean it's a hard topic and I'm not into the CK research deep into it but by myself I just um probably not too smart enough that's the problem but it's a cool thing to think about like yeah when I see data like what is true what is truth and can I know and can I trust the data provider oh yeah John King John cout I think the the biggest challenges with data are definitely uh right now um privacy is the biggest challenge I think because in blockchain's part of the design is that that it's super public and everybody knows this here like technically pretty much anyone can if they spend one hour track your nfts and so on find out who you are and then basically know what you're doing and that's that's not good privacy is normal and we want it so I think I think that's the biggest challenge and the other big challenge is um the the low latency quality of on-chain data um blockchains are too slow still and so on I think Solana is the biggest uh basically that's that's the idea idea right low latency blocks and so on so so they're the biggest mover there and ethereum has to follow suit uh in in terms of um opportunity though I think the biggest opportunity is Mev for sure um because um financialization is a big topic and still and Mev is massive and I would say another big opportunity is um um basically using these data availability layers ethereum or Celestia and so on to build whole new ecosystems around the data that you that you anchor and commit to on chain sorry that you anchor yes yeah for Solana yeah it's kind of uh yeah and another point is that engineering of the data operation for the node operation it's not easy anymore so it's the big Challenge from engineering point of view how you will proceed the archive data and how you can make sure that you can grab all the data from the blockchain it's another big issue especially for big blockchains like ethereum Solana is another point in another big chain and you need a lot of goods bare metal to to operate with Solana yeah so I'm kind of agree and it's boring we all agree to each other and we need to spiceness up a little bit um yeah so the challenge I'm going to say is I think transparency in the solutions or the architectures of the way data is being served this is from an oracle's Viewpoint as well I think at the moment There's an opportunity for people to to have more upfront honesty about the way that the oracles are working let's say and at the moment there hasn't needed to be there hasn't been a reason for for anybody to kind of put that step forward but I think when developers understand it with greater Clarity and then the end users understand it with greater Clarity will have an overall better data infrastructure ecosystem without specific to oracles or not and then opportunity wise yeah Louis I I think uh you know Mev and the relationship it has with oracles particularly Arbitrage um you know a lot of that is created by latency in the Oracle and there's probably going to be you know things like order flow auctions at the Oracle level start to be introduced so people can start to capture Oracle extractable value um which will provide ultimately better experiences for liquidity providers and the the services but that provides then a different source of Revenue that can perhaps go elsewhere other than the block producers let's say cool thanks a lot um going on to the audience questions was a surprisingly good so thanks a lot everyone um the uh I'd like you to give a short answer to this one which is we're at a privacy focused event what do you think of the seemingly tapped 22 of data collection and privacy I think it's already been touched on briefly but yeah if you could share your thoughts cap22 catch 22. I think um yeah yeah I think some of the Privacy Solutions um they're trying to basically solve it so that you can make claims about you without leaking all of the information right so I guess this is the promise of having data collection work and you as the user of the data can specify what you want to reveal about about yourself and what not and this is the power that these CK Solutions give to the users right so you can choose to share that you have more than 10 eth and you have a my lady or a remuillo preferably and yeah so zk3 is in the house I think sismore I got to know them for the first time here is in the house I think they're working on stuff like this so I guess that's maybe a solution to it although you know if you are like Cambridge analytica you will never be as happy with that as before right any other quick short answers on privacy and data in the space oh I have like not popular answer so we all want to believe that we can keep our privacy at the same time if you want to we want to use the modern product if we want to be included into the modern world we should you know the Privacy is a compromise so I believe that there are a lot of Brave guys who try try to find this compromise try to you know touch the water and uh like because in the modern world we have like not very easy let me put it that way not there easy problems to solve yeah and privacy is one of them so it's all the time the compromise and it's really hard to find the right way yeah which way is Right definitely we should not abuse the data that we have yeah as a any company any dab or whatever but on the other hand it's like one of the biggest question I believe is our own privacy any other thoughts of those who move on um all right one question and you mentioned zero knowledge proves just now um does the explosion of zero knowledge put a dampener on Data Solutions I think maybe reframing is like does it cause limitations in terms of the amount of data that can be aggregated and kind of how are you thinking about that I touched quickly before that that I know that there is a verifiable queries or a verifiable indexing in terms of the graph is something that's enabled with uh CK Technologies so that that helps there for um yeah for for knowing like is that data actually true but on the other hand side like um when we when it's not when it's obscure what what the data is then it's also like harder to reason about like if you don't know which transactions that are actually going on chain then we don't know how how much actually was going on yeah yeah I think um in general data will always flow and it will always be there and just grow and grow and grow I think I wouldn't say that zero knowledge proofs mean less data I think it's sort of like it's just that powerful technology to um compress data to perform computations over data to prove something about data I think it just basically just adds um adds to that actually it's not subtractive I would say cool yeah kind of building on again um you know you have these web 2 systems full of extensive amounts of data and you know the the typical example is medical records for example you don't want that information on chain but you can use their knowledge proofs to verify that off-chain data is is the most accurate way or is the right date of course um and yeah I think that's a very important primitive if you apply it to things like open banking or different parts Financial systems you can start to do credit checks personal checks very quickly and easily without you know making that data publicly available which will I think inevitably become part of applications and yeah it seems logical way to go yeah and you don't always need to have access to the underlying data set if you can verify kind of like a roll up of the underlying data where it's yeah you just get the graph at the end of it um not your graph um another question is um this is quite specific what do you think about erbit's approach to data making itself Sovereign any quick thoughts on that uh what do you guys think about erbit's approach to data do you know her a bit am I saying it right a bit orbit yeah yeah centralized cloud kind of centralized and centralized at the same time yeah you know I think in that it's the same issue as we have with any programming language because when we find out that each programming language contains some disadvantages we think okay let's create one more language that gonna be Universal and it's going to be one of the best and at the end of the day we have just plus one programming language you know so it's I I love that we have a lot of solution for this data but I believe there is no one solution yeah we have a good approach and I believe it is good to have one more but only after I don't know several years we can say is it good or bad is it solves the is it really solve the problem or not fair enough um yeah it's like the cloud Wars um I guess the last question that I have here that I'd like to ask and I'd like very short answers are there any data sets that are important that no one is currently paying attention to oh whoa no data about central banks and their real assets yeah I mean it was amazing how the banks in the US didn't have the data on the treasury purchases of SFS yeah Silicon Valley Bank for example like how is that data not available yeah however prayer is really centralized yeah okay that's that's fair I wish that data was accessible cool um well I think we can close it off at that um thank you so much for for attending this morning and thank you for for the great questions if there are a few more questions in here so if they weren't answered um please go and find these Charming humans afterwards to inquire um thanks a lot and please give a round thank you