State of Python Tooling for Solidity Development - Michal Převrátil | Ackee Blockchain Security

Thanks for coming everyone and I think we can get started. So my name is Miklajil and I am the head of tooling at Ekki blockchain. We are smart contract auditors mainly uh focusing on contracts built in solidity for Ethereum but we also do have uh Sona team the guys who love just trust you know and yeah what's even more important is that we love building open source so there are two leading products uh we are developing at Ekki blockchain the first one is called wake it's a python based tool it's open source and so everyone can use And the second one is called striden. It's a fuzzer for Salana. But well, we will talk mainly about Python based tooling today.

So we will mainly talk about wake and also some other tooling. We also do onboard uh new people to the ecosystem to the blockchain through curses at the Czech technical university in Prague and also some Solana guys through through the the school of Sana and Sana programs. uh we have the pleasure to work with many exciting projects like Lido SAFE or for example AA and we've also received a couple of grants for for the tooling especially especially from Coinbase and Optimism. So but let's start with the topic. So why we should use Python?

First of all, Vitalik lives uh loves Python and I think this is like it's it's enough for us to also like Python. But except for that uh it it's got very powerful syntax. So well you can type just a few lines of code and still do very very interesting very very powerful things. So Python is just cool. It's very easy to read.

It's literally easy to read but also pleasure to write and also it's got a very strong debugging capabilities. There's interactive debugger so you can basically well do anything in the debugger not just list some info but also execute code so it's very powerful and it it's got also very strong testing capabilities because where there's pi test which is like testing framework for python so python is really good for testing for debugging and also for writing uh also another reason is the python is the most favorite language well after solidity obviously and the solidity developer developer survey from the last year that was published I think was one month ago. So solidity developers also love Python. So Python is well very favorite language and well let's move to the tooling itself. So I've got a couple of tools here.

There's Slitter uh there's Ape and so on. So I will start from the left and we are trying to compare those tools in in terms of their capabilities their features. So the first one is Slitter as I said it's obviously Python based it's open source everything here is actually open source and all of those tools are also frameworks and Slitter especially is a framework for static analysis. So well it basically can detect some issues in your codebase. There's also API for creating custom detectors.

So you can even design your own while it's still maintained and it unfortunately doesn't have a language server. So well you cannot use it for like writing your code and see the syntax highlighting and go to definition of a symbol and so on but there's still an extension so you can at least see the detection detections in in VS code. And then there's Ape which is especially focusing on testing and it's like let's say a follow-up of Brownie which is unfortunately no longer maintained. There are still comets from time to time but officially it's not maintained. So well uh Ape is like following uh following uh Brownie in in like the testing capabilities and well then there's finally Brownie.

Uh, Brownie actually supports fuzzing even though it doesn't support it native natively in a way that there there would be some let's say dedicated fuzzing engine but there's an extension a plug-in for for brownie hypothesis and it basically allows to do some kind of fuzzing. Uh then we've got hard hat for comparison here. So hard hat is still obviously maintained. it can be used for testing and it also has its own language server and also an extension for VS code. uh then there's uh there's foundry which is rust based and it can be used for testing for fuzzing and I think that they are actually making some efforts to also become a static analysis tool but as far as I know I don't think there's any like detector ready to be used as of today but I think there are some plans to to also turn it into a static analysis tool and maybe perhaps sometimes in the future also make a language server.

And finally, there's V, which has all the capabilities. It's like a Swiss knife for for slowity development. And what I think what's actually cool about this slide is that it's outdated because I'm very very happy to say that after a couple of months of development, we have open sourced a new testing engine written in Rust. So Py uh V is no longer Python based tool but it's actually hybrid tool with Python and Rust combined and so well all the capabilities remain the same. Even the tests remain the same.

So it's written in Python still well the syntax is completely the same but the testing core inside is in Rust. It's running within the same process and so it's very fast. I will think I will will tell you about that later on. So let's start with the static analysis and I've got an example here from Slitter. I believe a lot of you know Slitter.

This is the output uh on the console and well Slitter has kind of well a lot of detectors uh implemented. On the other hand, some of the detectors are not well up to date, let's say for the latest version of Solidity because well those bugs cannot can no longer be like written in in the latest versions of solidity or some det some some detections are actually covered by the solid compiler and as I said it also has an extension for VS code. So basically all of those detections can also be shown right in VS code. Then there's an example from wake and yes so this is again a console output and it also shows the code snippets for the relevant relevant like uh lines of code. And what I really like about wake is that if you can see like the relative path, you can actually click on that in your terminal and it will open up your VS code or whatever you have maybe cursor or anything and it will show you this this exact same line.

So it's quite easy to just check your detection right from the command line. As I said, there's VS Code integration which is basically the same for both Slitter and V because well it's VS code is VS VS codes API and so even how detections are shown in VS code. It's it's basically the same basically there's underlined line and there's also some information about the detection itself. But let's move on from the static analysis to the extension. And now you may wonder why I will show you an extension when this talk should be about P python based uh tooling and the reason is that well basically uh this is the only extension as far as I know that's Python based it's powered by Python so actually all those features that I will be talking about are powered by Python.

Uh the extension for wake or powered by wake is called solidity wake and if you want to check it out there's a QR code you can you can scan it and it will open up in the marketplace with the extensions it supports VS code cursor and so on and there are basically implemented basic features like go to definition syntax highlighting and well of also some interesting more interesting features like let's say you have a variable and there's a label shown above the variable with the number of references. You can click on it and it will it will show you all the references for the variable. Also, when you have a contract, you can basically see a label above the contract that's saying inheritance graph. When you click on that button, you you you can see the inheritance graph for a given contract. And even the nodes within the contract are actually linked.

So when you click on a note in in that graph you can see your code for for for the contract that's uh that's inherited from uh this is new addition to our VS code extension. It's called deploy and interact and it's something like remix. It's like remix like experience but within VS code. So you don't need to switch between those two tools. You can just choose VS Code and well when you have a project you can just compile.

I do think I don't think there's the compile button because we are on the second tab but there's a compile button when you can compile all of your contracts within the workspace. Yeah, there it is. So there's compile. Once you have compiled all the contracts you can see all of them. You can fill in some parameters for the deployment.

And once you fill in, you can just deploy the contract. You have the transaction with all the information, call trace, events and so on. And then when you have the contract deployed, you can see all the functions within the contract and you can just send transactions or perform calls and well yeah it's remix like experience but within VS code and it's all powered by Python. The next animation is well related to something called forking. So basically what you can do you can pass a URL to any node and you can fork the state the chain from that node into your local workspace and so you can do basically integration testing because well you have all the contracts from some external chain but you can also deploy your own contract to the same chain and do some interactions between them and uh there's scan and the reason for that is you can copy any address from etherscan or even sourceify and you can pass it into the UI and it will fetch the API and so well it's very easy because well you can just paste the address and see the interface and you can interact view even with some external contracts.

So yeah this is the deploy and interact UI within the extension. Let's move on to deployment and testing. And yeah, this is an example of how how tests can be written in the AP framework. And what's nice is the syntax is very easy to read as I said also to write because it's Python. And unfortunately within the AP framework there there's no support for typing.

So if you try to hover over the function create pool, you will see something like that which means it doesn't know what it is because well it's something dynamic created at the run runtime and so it won't help you to autocomplete and so on. Now if we move if we move on to brownie it's kind of the same basically almost the same syntax but well still there's no typing support. So if you can if you move on uh if you if you move your cursor above above the function you see basically the same information. It doesn't know about the function itself. Now if we check how it's done in wake there's a few more lines of code basically because of import and we also need to specify that we want to fork from some node.

But additionally if we check the create pool function it knows about the function because there's typing sport. So when you have a project you can compile it you can generate something called pi types which is like typing stops for your contract and this way you can write any code in python but still aware of the functions of the structures events and so on within your solidity codebase. So it helps you to write the code and it even helps like the let's say AI agents or whatever you want to call it basically basically the cursor and copilot let's say because they now can see the Python stops and they can help you write the code very effic efficiently. This is why we love Python because it's got very strong debugging capabilities. So this is an example where we can run a test and well basically the debugging console is the same in wake and also in brownie it's kind of the same and it's got a full a lot of colors and you can see well uh where your uh test failed.

You can print any information you can print call traces about the transactions you've performed. So you can check any transaction in the history. You can print code trace for such a transaction and you can even print local state within Python or interact with your contracts because the chain is still alive. So you can check any contract that's present on the chain and you can even send new transactions, deploy new contracts and so on. But Python is slow.

Well, every everyone claims that Python is slow. Well, sometimes it's slow. So what we did is we wanted to compare the performance of Python based tooling also with hardhead because it's also dynamic language and so we took UNISO v3 repository we took 271 tests that were originally written in hardhead in Typescript and we have reimplemented those tests into brownie ape and also wake we've also taken different like let's say testing environments or development chains. And we have left uh we we let all the tests run with this with the specific configuration and we've checked the execution times how long it took to execute those 271 tests. And these are the results the it's always in seconds.

And as you can see is pretty fast even though it's running with anvil. And even though it's still run, it was still running all within Python, it it it wasn't measured with the new Rust execution core and still it was very fast, especially with anvil. So Python can be still quite fast if you use it properly. But well there wasn't founder for example. So let's compare with founder.

But before we start comparing, let's let's move let's speak very quickly about fuzzing because I don't know how how many of you have ever heard about fuzzing. Yeah, that's quite a few of hands. Thank you. So well fuzzing is basically about generating random sequences of transactions also generating random parameters for such transactions and since we are generating random data we want to execute as many transactions as possible to make the the fuzzing efficient and to have quite high chance that we will find some issues. This is a very simple example where we are doing let's say a unit test where we just transfer a few tokens 1,00 tokens from Ellis to Bob and then we make assertion that the transfer was successful that we've actually transferred all those tokens and let's say that we want to re rewrite that simple unit test into a fast test flow into something that can be executed with fuzzing so we need to randomize the parameters and I I think this is how I would would do it probably.

So we choose a random account as a sender also a random account as a recipient and finally we also generate a random value for the amount of tokens we want to send and then we perform the transfer and finally check if the transfer was successful or not. And how this is more powerful than this. I think it may be obvious, maybe it's not. But we may introduce collisions in between sender and the recipient. So we will see how the contract uh behaves when we have the same recipient and sender.

And also we may generate zero here. So we can see how the contract behaves if we try to transfer zero amount of tokens. So this is like the basic idea behind fuzzing. But let's move on to the second benchmark we've performed that was especially focused on fuzzing. So we took Daido's benchmark originally developed by consensus and there are five mazes and you can imagine it's like a grid 7* 7 I will have an image there and when we have those uh mazes then we basically uh we basically assigned one CPU core for each each benchmark test and we left it running for 20 minutes and well I I show you the image.

So yeah, we started at the left bottom corner and we we can we can move in four different directions up down left right and each time we are moving we also need to specify eight 64 parameters and depending if we spec specify those parameters let's say correctly or like well yeah there there are some preconditions but if we specify those parameters correctly then we may find we may find an issue, a bug in in that maze. And well, there are some bugs as you can see. And the this whole benchmark is only about trying to find as many bugs as possible within those 20 minutes. So this is also a signature of a function. This is one of those four functions as we can move in four different directions.

And yeah, so this is the idea behind the benchmark. This is an example of one of those bugs that can be found or may maybe may be discovered by different tools uh for fuzzing. So the first condition is for the location within the grid. It's 7* 7. So these are like the coordinates and when we are moving to to this cell within the grid we need to specify all the parameters that uh uh P 0 to P7 parameters and depending on the values we may find or not find a bug and issue and some of those uh conditions are quite easy to satisfy like P2 must not be equal to P6 but some of those are quite like hard to satisfy like I don't know I think there's P3 must be less than equal uh less or equal to P 0 multiplied by P3 that's basically well we can use zero maybe one I guess that's maybe all so some of those uh some of those uh if conditions are not easy to satisfy and finally I think we can see the results so these are the results for different fuzzers out there.

As I said, there are five different mazes and those numbers are the numbers of issues of bugs found within the maze uh 7* 7 within those 20 minutes when assigned one CPU core and you can see akidna here probably many of you know what akidna is also foundry then there's wake but let's say the original wake written in python interacting with anvil and also the new wake with the rust testing core So well based on these results you can see that wake was the worst when you when we were using just Python with anvil and now it has become the best just because of the performance improvement. Otherwise the algorithm for generating random numbers is perfectly the same. It's completely the same but only the only difference is the execution speed which you can also see here. We've measured how many transactions per second on average we've achieved. And so it was like let's say around 145 transactions per second with V plus Anvil and almost 9,000 with the new RA testing core.

That's like 60x speed up. That's quite a lot. So well performance quite matters especially in fuzzing. So if you ask me if Python has some future like in solidity ecosystem, I think it definitely has and especially when it's combined with Rust. So there's like the best of both worlds which is user experience interactions with Python.

But when when we need some good performance, we should use like Rust for the background job. So it's best combined together. And since I cannot speak for other tools, I can only speak for Wake. So if you wonder what's coming in the next version of Wake. As I said, there's the new Rust testing core engine uh which is coming uh in Wake 5.

0. It's based on R EVM on the Rust EVM engine. Also, there's coming support for treasure for Ledger. also for EIP7702 which is like setting the code of externally owned account and also for example support for ARM Linux because well there are no official binaries from the solidity team so it's not possible to compile solid code on this platform but we will support it anyway and I think that's pretty much all from my side so if you want to uh well check more check more about wake you can scan the QR code it will lead to the landing page of Also the name of the package for Python is called eatwake and if you have any questions in the future you can also join the telegram group so you can ask the questions there but yeah I think that's all from my side and if you have any questions right here right now feel free to ask

security community you know what to do. Do we have any questions in the audience?

Here you go, sir. What's actually uh like the difference between fising in Python uh and against fising in uh for example solidity? Yeah. So, well, these are two different approaches and I think that Python is more powerful when it comes to debugging because you have the interactive debugger and you can stop at any point and well, you can execute new transactions, you can inspect anything and I do believe that it's also simpler to write tests in Python because it's more user friendly. It's very very easy to write and even there's like better support in IDE in VS Code and so on for Python than for Solidity.

So well it's ultimately it's the decision for users but well I think it's easier to write tests in Python.