# Discovery - the tool at the core of L2BEAT

- Speakers: [Mateusz Radomski](https://streameth.org/speakers/mateusz-radomski)
- Channel: [Devcon 7 SEA](https://streameth.org/devcon_7_sea)
- Date: 2024-11-16
- Duration: 1:19:53
- Watch: https://streameth.org/watch/6738656320d1f9ac48c07177
- Download: https://vod-cdn.lp-playback.studio/raw/jxf4iblf6wlsyor6526t4tcmtmqa/catalyst-vod-com/hls/3d9fr0j4b82106m0/1080p0.mp4

## Description

Hands on workshop about how to use an L2BEAT tool called discovery for mapping, researching and monitoring all the contracts involved in a project. We'll start by introducing the problem that discovery tries to solve and after that we'll get into trying to understand the architecture of a real world project by using the avenues this tool gives us. After this session the participant should feel empowered to use discovery to deepen his knowledge about all on-chain deployments.

## About the speakers

### Mateusz Radomski

Trying to bring transparency to the game


## Transcript

Hi, hello. So I'm Mateusz, or you can call me Matt, like the short for Matthew. I'm from L2Beat. I'm half of the tooling team in the L2Beat. And my workshop is on the discovery. It's the tool we use, we build, to help us research projects, and it turned out that it is really useful at solving all of our problems. I mean, not all, but like majority of problems I'm going to later talk about. So, a funny story is that I almost lost the demo part of this presentation. And the only surviving copy of it was like some random Vim buffer. So I just wanted to point it out. Okay. So there is a lot of projects on L2Beat. You know, the amount of projects is growing. And we internally expect there to be at least 1,000 L2s. So currently you can see that there is 51 rollups and 66 validium and optimiums. So around let's call it 110 projects of only L2s. We also track bridges. Not to the same standard, but for all of those old tools, we have a minimum bar of data we want to show, and that would be the risk rosette. We want the risk rosette to be obviously correct, and the standard for this data is actually quite high. We want to show data that is correct correct so you don't lose your funds, you don't make a decision based on our data that is incorrect. So how to even maintain all of this, right? So just like a quick math lesson, right? So imagine that we have 110 projects, and every project has a single update every two months, right? So it's actually quite sparse. We see projects that are updating way more often. And even if every project had a single update every two months, we would have an 84% chance of seeing at least a single update every two months, we would have an 84% chance of seeing at least a single update in a day. So it's basically guaranteed that every single day something new will change on chain. We need to be able to detect it, see it, see what's changed and act upon it. And also, like, even if we assume that an update happens every six months, the chance is still 45%, right? So it's like a coin toss, basically. And these chances are basically always working against us. The more and more projects are going to be added, the higher those chances are basically to be 100%. So what is our needs, right? So like I said, there is a high chance of an update happening. And we need to be able to fastly see that it happened. And we need to be fast in reacting to it. If, for example, our optimism, right, any of the top docs has an update and we take like three weeks to process that update or to even see that an update happened, then we are showing incorrect data for three weeks, which is actually a quite long amount of time when it comes to Web 3.0. When an update happens, we need to know what changed. So it's not enough to see that something has changed. We also need to know what has changed, right? So did any of the risks, assumption that we had before, are now different? So also, we need to be able to look inside the project and compare it to the previous version. And as always, we want to automate as much as possible. You cannot get to 110 projects listed on a website and do all of them manually. It is possible with human effort, but we don't have the amount of resources to do it. So right now we have four researchers and we maintain all of the 110 projects with only four people and we also manage to do stuff that's additional. So discovery essentially, I think, is at the core of solving all of those problems. Of course, discovery is not a panacea. It does not solve all of the problems by itself. But of the problems that I listed, discovery is always at the core of the solution. So the mental model of discovery is like this. I like to think about discovery this way. And also I like understanding things from the low level. So let's try to understand the inputs and outputs of discovery. The input, of course, is the Ethereum state. We want to see what is the state of each project on a given, let's say, block number. So the input to discovery is that Ethereum state. Of course, we are not sending, like, a snapshot.zip to discovery. We are doing something like... I mean, we use RPC providers, which is essentially having the entire Ethereum state at your fingertips. You can do any call you would like. And to facilitate the querying the data from the state, we have a config JSON, which instructs the discovery program, which data we are interested in and how to process it. And after passing those two inputs to discovery, an output is generated, which is a discovery JSON file. Of course, it's all a simplification. The more, like, the actual mental model, which I don't even think is correct, and there's missing some parts. It's more something like this. So that's what I mean when I say discovery is at the core, right? All roads lead back to discovery. I'll try to, during this workshop, I'll try to touch upon all of the things that are listed here. So you'll be able to understand at least how this flowchart happened. So since I myself am a visual person, I want to create, I want to show you a demo of something nice to look at. So this is something we have been working on, which will enable us to move from command line to a graphical user interface. So right now we are calling it Discovery UI. And let's see, like like all of the projects that are listed here are projects that we are tracking with Discovery. So I have picked Zora for the project we are going to be using during this workshop. So let's see what kind of data we can expect to see in this tool for Zora. So of course there is like a lot of things to take in. But let's start from the left and keep going right. There are on the left, the thing is that they look like files, but they are actually contracts. They are kind of inspired by the file view in Visual Studio. And you can see, maybe, I don't know, the contrast is not the best, but I hope you'll see that. We have two contracts, which are the initial contracts. And by initial, I mean the contracts from which we are going to be starting and all of the contracts that are on the left have been found based on those two initial contracts. So it is, as you can see, like it is quite useful to be able to find two contracts used in a project and then basically find all other contracts used by that project and also this updates automatically so if anything new happens, new is added or is removed, they are automatically updated. Each of those contracts has some values, right? So each contract has an address, a name, a description given by a researcher. And let's focus on the fields for now. So fields are state variables variables that we have found in the ABI which are either public variables or functions which don't take any arguments and just return something so we are trying to build the state of the contract through these. There is also one more part which will be important later during the demo. I mean during the workshop part. Which is that we try to build arrays from functions which take a single argument which is of type integer. So we just assume that this function is like get something by index and we try to get all of those things. And you can see that there are actually addresses inside of those values. And those addresses lead to other addresses. And this is how discovery works. It gets all of those state variables, and if it finds an address, it assumes that this address is also connected and just keeps discovering on those addresses. And the third view here is like a graph view. So all of those projects here are used in Zora. And the way that it works is, for example, let's focus on the security council. Maybe that's not the best. But the system config, right? So you can see that there are some state variables and they point to other contracts. For this view, it is really easy to understand how the project is built, what contracts reference each other, and how the data flow inside the project happens. Of course, this is not the default view. Like the view allows you to select them, move them around, you can color these, whatever your heart desires to make it easier for you to understand. We have two layout algorithms since it's a graph view. You can use the D3. We called it slow because it is not that fast. It uses force simulation to lay out the graph. We also have more like a hierarchical view which just lays the graph from left to right. And the third thing is the code. So all contracts have source code. And let's go back to the L1 standard bridge example. And we want to show the code to the researchers because to understand what a contract actually does, you need to look at the source code. And you might be weirded out by the fact that there are only two files. I'm going to touch upon this a little bit later because it is actually quite important why there's only two files and not like more. So you can just view the code in here. The part that I want to also show is that we see that it is important to be able to switch between views, right? So I can click L2 output Oracle on the left in the list view and it is selected in the values and the nodes panel. Vice versa, I can select something in the values and it is selected in the list and the nodes view and I can select something in the nodes view, and it is selected on every single audit review. So this is something which is really graphical and nice to show. And it basically is only the look inside the Discover JSON. This is just a nice way to visualize what's inside the Discover JSON. But it doesn't touch upon the way of how we even got that Discover JSON. So it is something we are working towards. It's not yet ready. You can only view things. It's basically read only at this time. But in the future, we hope to make Discovery this, so you'll be able to do your research in a nice graphical environment and do anything you need. So yeah, let's get back to, yeah, go ahead. So you were showing the source code, but the source code is not on-chain, right? Yeah, so the question is, how do I get the source code since it's not on chain? So I kind of skipped one assumption. It's that by the Ethereum state, we also kind of consider the EtherScan source code database. Even if you go to L2Beat and look into the products that we list, if a product does not verify their source code on Etherscan, we give them a big red warning. We expect you to verify your source code to show transparency to the users and to the researchers so they can do their stuff. It is something that I omitted, but we do use Etherscan or Etherscan derivatives like BlockScout or stuff like that to get the source code, and we heavily rely on the source code because if we didn't get the source code, we wouldn't be able to call anything because we don't know what the ADBI is, right? So, yeah. If you have any questions, do please shut them up during the workshop. But I'll get back to the presentation. Yeah. So if you run into something where you can't get access to the source, you don't try any kind of decompilation? We don't try that. If you don't try decompile? No. I mean, there was one case where we knew what the source was, it just wasn't verified, so we like forcefully, like we verified the source code for the project because they didn't want to do it for some reason, so we just did it. And we are not trying to decompile the bytecode in any way. I mean, if it's something that we can't get the project to verify and we need to look inside, we might decompile it, but Discovery does not try to do anything like that. It just assumes the happy case where the source is on ETHESCAN or whatever explorer the chain is using and just goes from there. But yeah, if we hit snag, like there is no source, we just either accept it as we cannot look inside, or we talk with the team to verify the source code. Okay, if there are no more questions, I'm going to keep moving forward. So, going back to the mental model, because I think it is really important, like, if you leave this place with a single thing, it's just that discovery is just a program. It has some input, it has some output, right? So the output is this discover.json, and the input is the config.json.c. And the way you can think about discovery, if it makes thinking about it easier for you, is that it's like a scraper for a website. So scraping a website looks like you put some address of a website, it downloads the content of that website, it tries to find any links, and it follows those links recursively. Discovery essentially does the same thing, but it doesn't download websites, it downloads the state of contracts and doesn't follow links but follows addresses to other contracts. And of course, discovery is not just like a simple thing that is like a black box. There are things happening inside that we are going to be talking about a little bit. So discovery is able to detect whether a contract is a proxy. It does the source code flattening. I'm going to be talking about it later, like I said. It does template matching. It is something that also we are going to be touching upon during the demo part. It has custom handlers, which it executes. We are also going to be doing that during the demo. And it has typecasting. I left typecasting out because it is really, it will take like 20 minutes or 25 minutes to explain all of it. So I just left it out. If we have time or you are interested, you can hit me up after the workshop and I'll be glad to talk about it with you. And of course there is like an engine that orchestrates everything. But yeah, it's like there are things inside that black box. So I have a demo prepared. There's a QR code. If you want to follow along with me, please do. And if you get stuck, I'll be able to help you gladly. I'll be going over the same thing personally. So if you just want to sit and just listen, no problem. The website has instructions so you can do it now or later, whatever. And also I really hope that the website works for you because it's self-hosted. So if it doesn't work, try disabling your VPN, try a different country. I tried it like five times. It worked without any VPNs. So I'm going to give you like a minute to get to the website if you want to follow along. And yeah. All right. Okay. I expect everybody to be on the website right now. If you didn't manage to scan the QR code or type it through, just ask your neighbor for a nice deed so they will be able to show you the URL. So like I said, I will be going over the same thing. Personally, I have it on my iPad so I will be going over all of the things that are written in here and also I will be like adding some additional context for it. And also, this is the important part. I forked discovery from the L2B repository just for the purposes of this demonstration. If you actually want to use discovery yourself, please use the original repository because I'm not going to be maintaining this fork and it's going to get really stale really quick. And why did I fork it? It's just because we are not, no, it's an internal tool. We will try to make it more available to the public. It's just, it is in a rough stage right now. But it is still cool to show what this tool is able to do. OK. So is this visible? Like the contrast is okay? Okay. So I have downloaded the, the only thing that I have done is just npm install. And if you, if it worked, you should be able to just do npx discovery. And something along the lines of this should appear. So there are two subcommands. You don't really need to worry about them. So there is the single discovery. It's just for convenience. If you really need to discover something like a single address, you put it there. But if you want to build like a project, you'll be writing a config JSON either way. So there is also invert. Invert is like we used it to build those graphs you saw. We used mermaid before we got the protocol bit I was showing. So don't worry about those. The only thing that you're interested in is this cover. So the most boring part of doing anything fun is setting it up and configuring it. Unfortunately, I'll have to leave the part of configuration to you. You need to configure two things. You need to configure the Etherscan API key. So, you can actually call Etherscan. And you need to configure an RPC URL. And the sad part is that only few RPCs work that well with discovery. I'll say that the free tier of Alchemy worked for me without problems. So if you have Alchemy, do use it. There is like an asterisk where RPCs that only support block ranges of 10,000 when you're querying for logs do not work with discovery. For simplicity, we are essentially expecting you to have an RPC where the block range for log querying is basically infinite. So you will have to set this all up. I will just copy my end from the previous tries of that. So I have an end and you should probably do the same. You can always exploit the same variables if you want. So now we can get to the actual fun part. And before we actually configure anything, you know, we need an initial address. The address in the website is already there. But like this is also a good question. Like how do you actually get a hold of any address that belongs to a project? And I'm going to show you a simple example that you can find addresses belonging to a project. So we are going to be doing it this way. So like I said, I chose Zora for this presentation. So let's just go to the best website and find Zora. And how I do it is just go to their website of any project and try to bridge something. And also it is important there are multiple bridges for Zora, for example. But what you want is the official bridge. I know it's a touchy subject, but in L2B, we assume that the official bridge is part of the rollup. I don't want to get into it, but make sure you have the official canonical bridge to use. And we just want to bridge the smallest amount of if. do please don't bridge anything. We just want to get to the part where it says sign and don't sign, please. We just want to get to the part where it shows us the address we are going to be interacting with. So let me just bridge some if. Yeah, whatever. Let me just bridge some ease. Yeah, whatever. Okay, so this is the address of the contract we are interested in. And we can just copy it and store it for later. So now how do we configure the discovery to start at that address? So we need to create a folder structure that discovery is able to understand. So this folder structure looks like this. So just... I'll type it out here. So Discovery is the folder where all of the projects that you will use live, basically. And they're like configurations, results, flat files, source code, anything that pertains to a particular project lives in Discovery. And the actual project is like that. So anything related to Zora lives in there. But there's also one more level, which is Ethereum. This is the actual chain you're going to be doing Discovery on. Because we have the ability to do discovery on multiple chains, like I showed, maybe I'll