ETHDay - An Introduction to Web3 data with Dune - Boxer

[Music] the talk of today will be open data for finance and i'm going to give you try to give you a brief introduction of web3 data and maybe a little words about me i have a business administration background so i'm not that technical i basically taught all of this myself using tune and you can exactly do the same things nowadays on june i'm basically i'm basically doing developer relations so i make sure that every one of you who tries to query stuff on dune can actually do that um so without further ado um what i want you to take away today is that web3 data is radically open it is fairly straightforward in my opinion and it really levels the playing fields and we will get into what each of these topics mean in a bit um so maybe who has traded on the decentralized exchange in the last month okay okay so you've all made it into this picture congratulations um so this is the total volume of decentralized exchanges over i think the last 30 days and what that shows you that on web 3 we can do very big data aggregation and we can break this down into uh individual projects wow the colors come out weird um so the yeah the purple bar is uni swap it's not the unison pink um well basically you can see that uni swap dominates um the entire decentralized exchanges market and then we can even go down to individual pools so this is only like it says in the top left corner all decentralized exchange trading pools and only using the usdc weather trading pair so we can really we can go all the way up and look at the entire decentralized exchange volume of all the different exchanges and we can go all the way down to just looking at the single pair and then even looking at things like hey which uni swap v3 um fee pair um so unisof v3 now has different fees how do you feed tears so obviously the 0.05 tier is high up there but surprisingly the zero points for the zero percent here is still ranking second overall on that training pair so that really shows us how dominant unison v3 really is in the decentralized exchange market and especially in these very liquid pairs so now um we can even go deeper than that and um this is basically just the individual traders over i think this is the last night pretty much um and yeah like if we want to do this we can really go all the way down in web 3 data and this is something that i can do for all different projects so that is pretty radical um so what that brings us to is that everything is super public you can't hide anything so um i can do this for unisop i can do this for curve i can make comparative analysis um yeah i can basically like take the values or take the different products and directly compare them to each other and there's nothing that my competitor can do to stop me from actually analyzing their data and that that's what i mean by everything super public if something fails you fail in public if something does well you can you can just like show your data and everyone can verify that the data is correct um yeah that means you can learn from your competitors so you can just see like hey um somebody has launched like a very similar product like how is this working out for them um you can also analyze your users so in i think in the crypto space in general we think of wallets as fairly impermanent and everybody is like switching their wallets all the time but actually if you like dig down into the into the actual data it turns out that uh well it's actually fairly permanent and we can make some big assumptions based on the wallets that are using my protocol am i attracting new users am i attracting uh advanced users am i attracting like first time defy users people who only come from the nft space before we can do all of that stuff with this public data and one very fun thing is if you don't deal with your own on-chain data someone else will i'll go into a little bit of a case study um so everybody knows looks very i guess for those who don't know looks rare is in unfc marketplace which is basically rivalling openc and looks very was incentivizing trading volume by subsidizing uh so each ethereum or each metric of volume that you traded on a platform basically got rewarded in luxury tokens and they were using this formula so basically um there's a set amount of tokens in a day and however much volume you make up of that day you get tokens in in that amount um so what then happens is we get something like this so this is the um daily nft so yeah the eight daily nft trading volume and if you look at this you would think that hey looks rare has is actually a valid competitor to uh to openc and they're really doing great and open sees open seas probably sweating the ass off and like this um this is a i don't know hairy situation for openstack or something but then um if we actually dive down into the data data and that's what i mean by you can't hide um we can we can take a look at uh a very simple query on dune which is basically uh looks rare zero royalty volume and what that means is um in nfts there's this concept of royalty which means that on every transfer of the nft after it has initially been minted the original creator can take a can take a cut off that of that transfer basically or of that cell event and um basically there's also collections which don't have this royalty and so if you want to watch trade on looks where so that means uh bringing a lot of volume through looks where um you would basically use these zero royalties zero royalty collections and as we can see so the red or orange bar [Music] is the zero royalty volume the blue the blue bar is the like all sales volume and the little uh gray bar here at the bottom is what's left so we can see that for most of the last month um the volume of luxury was actually just fake um so it's just uh wash trading volume or at least we can like there's still some cases there's very little cases where a zero royalty vol like zero royalty collection will actually be traded in a legit way um but just like a back of the envelope can calculation that we can make here um most of the volume uh was fake in the recent days it has actually gotten better but uh it's still mostly fake volume um so this also reflects in the users um so again you can't hide um openc has like 50k daily users and looks very sitting at like i don't know 500 to a thousand or something um and then if we actually apply that wash trading filter to the overall volume so uh we were looking at this earlier um you can see the uh yeah the relation of the of the bars of looks where in openc is um yeah it's a it's a joke right here so you can't hide um this was a very smart move by looks where um to i don't know if it was a bug or a feature or what they were thinking about this um but basically if you only so if we don't analyze the data in depth and we're just like yeah like looks very has this much volume now then if looks rare was the only owner of their data so how it is in web 2 most of the time um then like investors into luxury would probably be um on top of the moon um they would they'd be yeah they'd be very happy about the market share that luxery was able to take in such a short amount of time but then if we actually dive into the data and uh yeah filter out the wash training volume um yeah looks pretty shady um but this story is not over yet so um yeah this is a bit unrelated but basically there's this thesis that aggregators so things like gem and genie in the nft market could potentially lead to looks we're having a higher transaction count but if we look at the market share of so this is all the different transactions in the nft market normalized and we can see that yeah gem makes out like seven eight percent of the total uh nft yeah nft transactions at the moment um so yeah we'll see how that story plays out um but yeah uh web3 data is radically open you can't hide um that's basically what i want to show you with these slides and then in my opinion uh web3data is fairly straightforward so how does all of this work actually in the background so how do we how do we get to all of these cool charts um so there's like i'm gonna really butcher this and if there's any like proper developers in the room they'll probably hate me um but ethereum is a state machine every transaction has clear data elements um and smart contracts produce logs and that's basically like all we need to know we're just analyzing the outputs of the ethereum virtual machine and that is basically just a runtime environment which is in some ways a computer um so if we look at this so this relates to ethereum is the state machine um so basically what we're doing is so on ethereum so let's say i have twenty thousand die in my uh in my wallet and basically all all that is is somewhere in the die core uh in the die contract storage there's a um there's an entry with my wallet and um uh 20 000 in a non-decimal form so it would be like yeah i don't know how that number looks like but um basically it is nothing else and if we want to change that state so if i say uh i want to transact a uh die then i send a transaction to the to the die smart contract and i say um yeah please send my die to this other address and that's how we change the state of the system and state changes can only happen one by one um yeah transactions in ethereum are always run sequentially and um yeah you might now think uh it's pretty easy to analyze all the state data on june but we don't have state data yet we're working on it and there's other teams in the space who've actually done this but doing it right and doing it in a scalable way and so that our database infrastructure keeps running along is very hard so what we do instead is we can analyze the transactions so instead of analyzing the state we analyze transaction so a transaction always has input data and that input data basically makes some gears turn inside of the evm and then that might so so let's stay in the die example so if i successfully transfer my die then the die contract will spit out a event lock which is um which is just called uh event transfer and it basically says i've moved um this amount of die from this address to this address and using these locks we can actually replicate all of the states on ethereum sometimes easily sometimes it's a bit harder but most of the time it works and then there's also the thing which is called an internal transaction so that would be for example if i um if i send usdc the usdc contract will first query a blacklist so there's a blacklist contract and the usdc contract will call that blacklist contract and asked if one of the addresses um which is trying to transact here is blacklisted so um that's what happens during internal transactions and it's a very simple example sometimes there's like uh yeah literally hundreds of internal transactions that happen during one transaction but in general um this is just this is just a loop that can that can infinitely not infinitely for the amount of guests you have but it can go on for quite a while um so yeah so basically we have four data components and one is state so what's the current state of the system um then transactions so that's basically the first call i make into dm and that's who's trying to do what and then we have internal transactions so that's what happens inside of the transaction so if there's more steps um which which gears also start turning inside of the evm and um then we have event locks and the event logs are by far the easiest to analyze for for people on dune because they are very reliable every time something happens we can just use that event lock or look at what kind of locks the smart contract has emitted and then we can analyze that transaction using the locks so the locks are yeah they're designed um to be yeah kind of indicators of what just happened in that transaction so obviously that's going to be the easiest way um yeah so um how does all of this work on june and i don't know yeah i think i have time um so all the methods to get the data are public um once it runs and runs forever um so all of this is probably fairly opaque to somebody who is not familiar with june but standardization and abstraction happen on a macro level and social learning is absolutely okay op um so let's quickly go into this and basically what i've done here is i've written a short query and um so we're going to look at the unisra v3 factory event pool so unix swap v3 has a factory contact and that factory contract emits an event uh that is called pool created and um if we just run this query um so this is the dual interface for anyone who's not familiar on the left we have like a bunch of data tables this is where we input our queries and down here we can actually just get our results um so now i have um six 6.5 k rows which all would show me all the pools of unisof v3 uh that have been created and i can see the feet here i can see the tick spacing i can see um talk zero and one so that's the and the two components of the pool basically um i can see which address um the the the pool has and so on and so on and basically um so this is already pretty useful uh in some cases um but what's the cool thing about dune so this is basically we are just replicating etherscan here in the worst user interface right but what's the cool thing about unis that we can easily aggregate these oh how can i then we can just easily aggregate all of this oh that's very nice yeah life craving is always the worst guys oh come on i didn't save my query i'm sorry yeah there we go um so basically what this is giving us now is um how many uni-swap pools uh have been deployed over time um and this is also this this should just be a very uh easy example of what you can actually do with yoon um so we see that um on each um pool creation event um this uh yeah this event block uh gets emitted and we can just count them over time and the same we can do with trades the same we can do with if somebody deposits into a year pool the same we can do if somebody transfers something and basically the power of tune is that it just makes it very easy to analyze and aggregate these things so you can extract data with scale out of the blockchain and then maybe if we want to go one step deeper we can actually run this and so this is the different unisrop v3 pools by the by the fee percentage um so i don't know if there's any interesting data for someone in here but um basically the very low percentage fee pools um seem to be prettier so i guess it's just used for stable to stable swaps so it kind of makes sense um but yeah so so this is how dune works and um if we just go into [Music] some dashboard here um i just i had all of these things like once it runs it runs forever so that basically means once you've um once you've built the dashboard on june uh it just runs forever um until the the code in there breaks your query gets too long or um yeah but both of those things should optimally not happen um so basically we just provide you infrastructure um to build your own data hub for whatever you're trying to do so if you're trying to analyze your own wallet or your project or um i think in this case it's uh through validators or or ethereum staging deposits and um there was another point in there which is uh social learning so p so now um if i come on june and i'm fairly new and i want to know how all of this works and how do i actually work with the data i can just click into any query on june of course this is private now what are you doing to me adobe let's uh almost all of the queries on june are public um it's actually very rare that that things are private so usually i can just do this and in that way i can just look at what this creator on dune is doing and so i i can just focus query i can read through here and run individual parts so i can better understand how the whole system is actually working so that's what i mean by social learning is op so i learn best by just trying around and i think a lot of people work that way and on june you can literally you can go to world-class analysts and you can just go into the dashboards look how they work and that way you can very easily level up your own game um so yeah um so yeah that's basically my whole spiel about web3 data australia is fairly straightforward there's a lot of people in dune community who didn't know sql who were barely able to read ether scan and after two or three months um they were legit web 3 analysts and are getting job offers left and right so it's a very um yeah it's a very easy entry to a technical or kind of technical position in the space i would say yeah and my last point is basically web 3 data levels the playing field and it's it's really radical if you compare this to web 2 data um the amount of access we have with on-chain data into products and projects and companies uh is basically it's it's a whole new universe out there um all of this data is public um you can't hide like i said earlier um and if i make a private dashboard i can literally have better access to a project's data than they have themselves so so that's it's such a radical new way of of transacting and that everything is public and open um yeah it really yeah it really blows your mind if you think about it long enough so we have um equal access for everyone no pay walls so in the traditional world um if you want to have um yeah in-depth research access to a lot of the great data sources on this world you literally pay millions of dollars a year to have that access and um in blockchains it's all it's all out there it's all public and you don't need to pay anyone yeah gym creators are building public and private goods so we have a lot of dashboards which are just like general ecosystem statistics how's the ethereum doing how's how's the polygon chain doing how is optimism adoption growing along um those are all dashboards that are basically um they can be viewed as public good sites i think um but also a lot of projects or companies are using our platform to just take care of their own data um [Music] and then yeah all of us literally have better access to data than the ceos of web 2 companies and that's really mind-blowing we have live reporting it's not quarterly it's not locked in pdfs we can we can share the data between different analysts i can look at what other people are doing on june and slightly change their approach um it's really going from the silos web to data to web3 data if there's data scientists in the room who are not yet working in web 3 you should definitely come and join like it's it's so much more fun um so yeah that's it um the data must flow and uh this is our discord link we will have swag with us starting witness day still being flowing in from portugal but oh well yeah catch me catch me at the hackathon or at any of the side events and yeah [Applause] i love it the data must flow um again same question where what what are people going to get what's the merge like dude can you give us any uh should we hunt you down really seriously what is it aha the greatest socks in the world okay dune socks worth it worth it um okay opening the floor for questions raise your hand or not yes hey hi yeah quick one so speaking of competitors i think the most important you or the biggest one you got is the graph speaking of own experience so when we need to make this some visualization we do it in june when we need to build their own graph or something we use the graph because it has better api and stuff so how would you come well compare yourself to them and how do you plan to fight them if you want well there's enough space for everyone always but yeah how we see it is that graph is more geared towards developers at the moment so um and dune takes more of the data science and historical data role um but in the future we have announced that we will launch an api and yeah we'll see how how that plays out in the in the long run question here um so as the size of the data on doing gets bigger how do you plan on making it easier for people to do queries on the larger tables is there going to be partitioning or is it just going to be on the developer to select the correct materialized views i imagine you use for the protocols that they're querying so on on june we actually have an abstraction layer so we have tables that are called dexter traits and nft traits and in those tables we have basically standardized and normalized and joined joint prices in u.s dollar and all of that good stuff and those tables are fairly easy to use for basically everyone if you know a bit of sql you can work with that table because it's just it's another sql table it's all already normalized and standardized and in that way we hope to open the field to a lot more analysts out there who are not that familiar with the inner workings of verb 3. um so how the evm works and and all of that jazz even though i don't think it's not that hard to understand once you like get into a bit um but yeah basically we're we're we're abstracting away um complexity and complicated stuff um from our um yeah data scientists already and we hope to scale those efforts up in the future oh hi uh it's great talk um so i know i saw dune just just finished uh raising a 70 million dollar series b at a billion dollar evaluation why why does dune need to raise so much capital i saw that grin answer yeah great great question um really putting me on a spot here um no actually we are hiring um [Laughter] no um yeah building a great team with world-class people takes capital and um yeah we are just committed to being the web 3 data platform to query for any data across any of the chains and for that we massively need to scale up our uh yeah our our personnel um we raised 1 billion at only 16 people and now we are already 35 so we are scaling very very fast and we are hiring the best people in all in yeah basically in all of our um yeah positions that we that we have open um so go to june.

xyz careers i think um and yeah let's uh help us spend that money wow another question and actually it was 69.42 really nice how did you get involved what's your story when did you join dune um i actually um i recruited myself over discord i just hung out in discord all the time and bothered maths and frederick with questions and reported bucks and so on and after a while they were like hey do you actually want to work here um so that's pretty fun what were you doing before um i was supposed to write my bachelor thesis that did that okay that did not happen did you submit something like okay now i'm bi guys or no i'm i i i'm still and i'm still enrolled actually um at one point i might convince the professor to just um accept the dune dashboard as a paper um i hope that happens in my lifetime but yeah we'll see you should definitely any tips for any developers here i mean i think you're sort of your boldness of you said hired yourself uh in the discord um any tips for people that are starting and what should they do um just get involved um yeah the trends at the same time no um basically find something that you're good in um so for me i i was in the space since like 2017 and i had been on etherscan all the time and i kind of knew how blockchain works and how like how smart contracts work and all that all that stuff and then i also had like sql skills um which i picked up in some internship uh very random and then basically i just took those two skills um combined them and then basically there was very little other people who were as good as what i was doing uh at the time um and i i found my unique um position in the market basically that i could yeah that i could work with and i think everyone out here has some talent or at least i hope they have some talents that they can utilize and basically in web 3 you can almost create your own positions um just like show up somewhere um ask ask questions um do the do the like ground work answer questions in discord is not the most fun um task at the beginning but like actually it's it's kind of fun um like helping people out is uh is really something i like to do um but it's not like a position that you would initially think is super interesting um but just do like the groundwork and over time you'll um get access to to better working opportunities basically all right last question all right thank you so so much for this presentation it was the first i think to have a dune moment in amsterdam thank you [Music]