Slang’s Query API: a better way to analyse Solidity code by Antony Blakey | Devcon SEA

Okay. Good night. My name is Tony. I work at the Nomicon Foundation in applied research. I was previously a co-lead on the Slang project and it's in this capacity that I'm going to speak to you today.

I have a fantastic job working with excellent people at Nomicon. Nomicon is a non-profit and our number one core value is kindness and we live this every day. And we are hiring. There's meant to be a QR code at the end of my slides, but I'm afraid it didn't make it. Uh and I want to give heartfelt thanks to the co-founders of Nomicon Pado and Fran for creating and maintaining such an amazing place to work.

So, today we're going to explore the Slang query API, which is a new feature which makes working with Solidity code surprisingly straightforward. And now this is not a tutorial, so some of the details have been omitted. You can't copy and paste any of the code that I'm going to be providing. It's just so that you can Well, I'm hoping to inspire you so that you go and find out some more details. Okay, so 20 minutes is a bit of a sprint.

So, I'm going to speed up. Let's go. Okay, so what is Slang? So, Slang's our Solidity compiler library. It's basically a developer tool that enables you to write better tools.

It parses and analyzes Solidity code from version 0.4.11 to the latest 0.8, which is basically all of the Solidity code that is live. Our focus is on correctness and convenience.

We're about to release version one, which covers the typical front-end features of a compiler. And our main objective, as I said, is to enable you to develop better tools. Slang has a radical open declarative meta architecture. Slang's a error correcting compiler, so it always produces output even if you've got source code errors. A key feature of Slang, which is what we're here to talk about, is its query API for code analysis.

So, compiler front ends fall into two main categories. There are those that produce what's called concrete syntax trees and then produce a an abstract syntax tree from that. And there are those that produce abstract syntax tree trees directly. So, what's the difference between concrete syntax and abstract syntax? Well, concrete syntax, as you can see on this slide, is a complete representation of the source code.

It includes every character, every bit of white space, every comment. It's like having, as I say, like having the full book. Like a PDF. Whereas an abstract syntax tree is a simplified tree representation. So, it doesn't include white space and equal signs and and punctuation.

It's just got the essential details. It's like if you were reading the the plot of a book, but you didn't actually have the text. So, here's an example of a contract of a concrete syntax tree. So, we've got a simple variable declaration here, and every single element of the original source is represented. Nothing is removed or simplified.

Now, this is different from an AST, which will eliminate some of these details. So, if we look what we've got here, it's a simple variable declaration. Now, it consists of a type name, which is uint, then some white space, an identifier, some white space, punctuation, the equal sign, some white space, a number literal, one, and then the semicolon. Now, what's crucial to understand is that we're preserving every single character. In fact, even if there's an error in the source code, you'll find it in the concrete syntax tree.

So, why is this important? Well, when we're developing building developer tools, or transforming code, we often need the complete information so we can round trip. If we were just concerned with the meaning of the code, then we'd use an abstract syntax tree. But for many tools, like formatters, linters, or refactoring tools, we often want the full syntactic details. You don't want to lose comments when you're refactoring.

And sometimes you want to know if you've left two blank lines. And you want to preserve that. This level of detail is great, but it's also challenging. It's quite difficult to process this. On the one hand, we have complete information, but working with detailed trees can be complex, and that's the challenge that the query API solves.

So, from now on, I'll admit trivia nodes and white space from the examples. They're still there, but I don't need to show them. So, let's look at a slightly more complex example. This definition shows how different elements nest within each other. And once again, notice how everything is preserved here.

We've got a function definition. We've got the the keyword function, the identifier. We've got a parameter list, which in this case is empty, but we still preserve it. And then nested in that is a block and a return statement, and so on. Now, this level of detail allows us to maintain complete source fidelity, as I said, if we're going to round trip, we can track precise code code locations.

So, if you want to do error reporting, uh it handles formatting preservation, and when you're building development tools this detailed structure is essential for certain class of development tools. So, to emphasize this point, I know I'm being repetitive here, I really want to make this point though. Why do we need a concrete syntax tree? Well, we really want to in a lot of tools, for example, in Prettier, uh which is now using Slang, by the way, using this technology, uh Prettier Solidity, we want to make sure that we can preserve every single character, especially comments. For example, you might have a tool that wants to deal with comments, wants to encode validation information in a special comment format.

Not like the documentation comment, but your own special comment format, so you want to preserve those comments. An AST, on the other hand, is focusing on the essential structure, which is good for semantic processing, doesn't contain the formatting information, simplifies expressions, and it omits all comments and white space. So, now that we understand what the CST is for, let's look at how you might traditionally process a CST. So, this is the kind of code that you'll often see a lot of. You know what it's like to traverse a tree, especially a very deep and complex tree, which is what you typically get out of a compiler.

Uh and it's not pretty. What we're trying to do is actually quite simple, we just want to find all the variable declarations in some Solidity code, but look at what we have to do. So, first of all, we're writing a recursive function. We need recursion because we don't know how deep in the tree it is. Could be at the contract, inside a function, inside a block, anywhere.

And then, when we find each node, we've got to manually check its kind, we've got to search through the children, we've got to handle all the edge cases, and we've got to manage the recursion stack, especially if you're threading state. And this is a simple example. In real life code, you'd have to handle error cases, deal with unexpected node types, which is what you can get if you've got a source code error, for example, manage your state during traversal, and if you've written a visitor pattern, you know that it's a pain in the ass to manage your state. You've got to handle parent references, and you've got to manage the position in the source code. Now, the problems with this approach are numerous.

First, it's incredibly verbose. The amount of code you need to write to even simple analysis is substantial. And the more code you write, the more you have to maintain, and the more places there are for bugs to hide. Uh second, it's error-prone, uh not only because of the volume of the code, but also it's quite tricky to write these when you're dealing with complicated trees. For example, in Solidity, variable declarations can appear in for loop initializers.

I don't know if this code would handle that. It's a question for the reader. It's hard to maintain. When the language evolves, and as you know, Solidity evolves significantly, uh in all sorts of interesting ways, like in one of the 0.5s where the exponentiation operator changed its associativity.

Uh so, when the language evolves, you need to update your traversal code. And that's because the traversal code is mixed with the analysis code. So, what you want to do for analysis is mixed in with the mechanics of traversal. To make changes, you're often going to break things. Finally, and more importantly, this forces you to think about what or how you want to do things rather than what you want to do.

And the intent of this code, finding variable declarations, is buried in here. You could look at a big bit of traversal code and you wouldn't know what it was meant to be doing. We need something better. So, we'll have a look at some specific challenges with manual traversal. So, recursion management.

You've got to handle deep nesting efficiently. Avoiding stack overflow. You've got to make sure you don't have any infinite loops. You've got to maintain your context, state management, and you've got to know when to stop recursing. Once again, if you use a visitor pattern where you've got an external visitor driving, external code driving your visitor, you often want to stop or you don't want to go into the children of this node.

How do you do that? That means you end up with a more complicated API. State tracking is quite complex. Need to keep track of the parent node when you're uh uh walking a tree. You need to maintain scope information in the case of a compiler.

Build up your composite results. Handle cross-node relationships. I.e., you want to link this node, which you visited you know, 30 minutes ago, with this other node which you're visiting now.

Now, error handling just multiplies the complexity. And edge cases keep appearing. For example, parenthesized expressions, nested structures, optional elements, and language-specific quirks, of which there are plenty in Solidity. And finally, you've got a big maintenance budget. Now, I know I'm repeating this point, but this is significant, as you'll see when we get to queries, which I think is the next slide.

So, this is where queries come in. So, let's look at how we solve some of those same problems using queries. So, this is our first example. We want to find all the uint variable all the uint variable declarations. It's a simple as saying in variable declarations where the type name is uint and it's got an identifier, find that and return it.

No traversal. No edge cases. Hardly any maintenance. No efficiency problems. So, the second example shows something a bit more powerful.

In this case, we want to find immediately nested function declarations, i.e. function inside another function. So, here we're saying in every contract definition, there are some contract members. In the contract members, there's a contract member.

For every function definition in there, find the block and look for the a statement in there that is a itself a function definition. Call that bind that to a variable called nested and return it. So, why is this so powerful? Well, first of all, it's declarative. We're saying what we want, not how we want to do it.

Makes our code easier to understand and maintain. And when you look at this 2 months later, it's obvious what it's doing. Whereas, I'd suggest that you'd have a couple of pages of code traversing your tree if you were using manual traversal and it wouldn't be obvious what it was actually meant to do. So, it's structure aware. So, the query understands Solidity's syntax structure.

It knows about scope and it knows about nesting. And also knows where type name, for example, in the in the first case here appears in the tree. These patterns are composable. So, we can build complex pattern matching from simple pattern matching. So, if you wanted to find uint variables within these nested function definitions, it'd would be as simple as putting that appending that first query below the second query.

But most importantly, this is focusing once again on what we're trying to achieve. We can think about code patterns and structure design without getting bogged bogged down in the implementation details. So, let's have a look at some more advanced query patterns. This one for well, this one isn't that advanced, but for example, in an unchecked block, you want to find all function calls. As easy as that, and it's obvious what it does.

You could write this on one line. The second example is a bit more complicated, but here we're looking for variable declarations uh state variable declarations that have a type that is either map a mapping or an array. And think about what this would take if you're doing manual traversal. You'd need scope tracking, type checking logic, complex conditional logic, and you'd need to hand carefully handle the nested structure here. So, what are the use cases for this?

Or some use cases, cuz I'm hoping that people come up with far more use cases than we know. We want sort of emergent benefits from this. Well, if you wanted to have custom coding standards, you want to enforce custom coding standards, you've got project-specific restrictions that you want to check, or you want to do version compatibility checks, so you want to make sure that you don't use certain features of Solidity. Uh style checking, well, if you've got specific naming patterns you want to enforce, uh specific structural conventions, and you can have context-aware rules. Now, you might be thinking, well, can't I use a linter to do this?

Well, yes, but this is meant to write the linter. This is designed for writing linters, so that's why these are the use cases. This is not an end user tool. This is a tool for developers to write tools. Uh so, you can do pattern detection, anti-patterns for example, uh optimization opportunities, which may not be immediately obvious if you've got a big library of things, uh detecting complex structural patterns that you might want to simplify or or mark.

So, once again, this is a kind of an ESLint uh equivalent. Code transformation, automated refactoring. Refactoring is transformations that um preserve the semantics. Uh code modernization, it's very easy to write um patterns that then transform into uh more modern code. Automated modifications, i.

e. things that aren't refactoring, so they actually change the code. And formatting. And as I said, uh prettier Solidity is already using uh Slang. Uh one important point I want to make here is that uh we support WASM, and we support in specific specifically the component spec and WIT.

So, for those who know what that means, this means that uh all of this technology works in the browser. It's also available as a Rust API or as TypeScript, which is our first target cuz most people uh are using TypeScript. So, it's more applications, code trans- uh I've already done that. Documentation generation. Uh it's easy to generate automatic documents by extracting function signatures, doing structure analysis, um checking for particular usage patterns and documenting them, and processing comments.

As I mentioned before, you might have validation uh information in a certain specialized comment comment format. Now, you can also use this technology in combination with AI tooling for example to produce diagrams from your code. You can use this with by encoding the slang API as a rag. So, what are the key benefits? Well, structural aware queries.

Your queries naturally align with the structure of the code that you're trying to match. You don't have to think about trees and nodes and and traversal. Complete syntax preservation. You never lose information. Perform transformations without losing the formatting and you can round trip from a CST back to source.

Efficient pattern matching. So, we can spend all the effort required to make this efficient. If you know anything about the state of the art in terms of tree pattern matching and in fact we're not a a search engine, we are a unifier. So, we use a Prolog/Datalog based mechanism which returns you all the potential match results, not just the first one. You can avoid unnecessary traversals.

We can cache results. We can index the syntax tree. Make it very efficient. It'd take a lot of effort to get the same result if you were doing it yourself. Composable syntax rules.

You can combine simple rules to make bigger ones. You can have a big library of these patterns and you can reuse them over your code. Share queries between your projects. You've got maintainable analysis code. So, the queries express your intent, not how you not what you want to do how you want to do it, but what you want to do.

Your code is shorter, it's easier to understand. Any changes to the language, we take care of that. Slang takes care of that so you don't have to. Less code means fewer bugs. And these benefits compound.

It's not 1 + 1 + 1, it's the the whole is far greater than the sum of the the parts. So, what impact does this have on your development? Well, before using this query API, you'd spend significant time writing and debugging the traversal code, you'd struggle with maintaining the analysis logic, and you'd face challenges when adding new features. After it, you end up with clear focused code, it's easy to maintain, you've got a robust implementation, and it's highly extensible. And that is it.

Any questions? Fantastic. Thank you, Anthony. All right, let's get with the let's get with the questions. So, folks, remember you can scan that QR code, add questions to that list, smash that upvote button so that the most interesting question gets asked.

Let's go with the first one. What are advantages to using Slang compared to Semgrep? Uh well, I must admit I'm not familiar with uh Semgrep, but I know the general uh concept. The specific thing here that this is a programmatic tool that you use to build other tools. Now, of course, you can combine things like Semgrep, uh I imagine, uh but I don't know if that is something that you would include in a tool for analyzing lots of different versions of Solidity.

That makes sense. Thank you. How does Slang differ from a fine-tuned LLM? Could Slang's modularity allow to use LLMs in the future? Uh this is I'm in applied research, and uh this is something that is very actively under research.

Uh I'm not making any commitments, uh but if you've used fine-tuned LLMs, uh and I've used them a lot, for example, a lot of the code that I write is actually written by Claude. Uh a lot of this presentation was written by Claude. Um amazing tool, but you always need a human in the loop. So, yes, you can do this using LLMs, but the thing with LLMs is that they are a human augmentation tool. Whereas, the approach that we're talking about here is exact and precise.

If you've used LLMs to do this, you know that you you spend a lot of time on prompt engineering, and then you cross your fingers, and something comes back, and it doesn't quite work, and you prompt it again, and so on and so forth. So, yes, this is certainly something that could be included in the future. There's a chance, but maybe not now. I mean, not immediately. Certainly not now.

Okay. Are the limitations of an of a tree visitor-based approach not solved by using an API with CFG and intermediate representation like Slither provides? Uh well, if if by CFG, well, you're either meaning two things. You're either meaning the control flow graph, which is typically something you'd find in an abstract syntax tree, or if you mean control flow as exposed by Rust, for example, and this is something where you reify your control flow. So, you return a token in your visitor which says, "Do you want to continue?

Do you want to go down the trees?" Now, this tree visitor uh these um solution these Slither may well solve the same kind of problem. We have a different set of constraints. For example, our query language is intended to be extended with semantic predicates and with the ability to do arbitrary recursion by skipping over arbitrary subtrees. So, there's a lot that we want to do there.

So, I wouldn't say that that in Slither you can or cannot do that. I'm not familiar enough with Slither. Perfect. Thank you. Is there a question you like most in there?

Uh well, with this work with different EVM versions? Yes, it will because we're dealing with source where well before EVM. Slang is a compiler and it'll produce bytecode. So, that's a question about the compiler, not about querying. Have we published the grammar for Slang?

Yes, we have. Uh Slang is actually a uh declarative meta project. So, you can use it for any programming language, not just Solidity. And we have um published that everything we do is open source. Uh Nomadic Foundation on GitHub.

All of our development is in the open. You can reuse all of this. Awesome. Uh how important? Well, uh so main How important is it to maintain code with different Solidity versions?

Why Why not just use one version? Because if you want to be able to provide tools that analyze a large number of contracts which may be an earlier version of Solidity than the latest one, for example, 0.4.11. There's a a contract on mainnet that is 0.

4.11. You want to get the source to that. You want to um provide analysis tools for it. That's why we want to put support all of these versions.

Now, we don't go back to 0.0.1 or whatever the first version was. So, we're pragmatic about it, but these are the versions that are live and our goal is to support all the versions that are live. How do you determine the cut-off version?

Uh What's pragmatic? Analysis. An- analysis of the of mainnet and all the contracts that are live. Okay, is it the number of contracts? Is it the value stored in those?

Is it the amount of transactions? no, the earliest the the um I didn't do this analysis, so I must admit I don't know what process we used to determine that, but it is the case that the earliest contract in use, apparently, is 0.4.11. Fantastic, thank you.

Um what's the best practice for using to analyze a large number of program? Where should the data be stored and in what format? Imagine this is something you run locally, correct? Yes, it is. Perfect.

Thank you. All right. What updating Solidity was the hardest to that for? You mentioned quirks and weird updates in Solidity. Which one was the hardest or It None of them was particularly hard.

Uh, the volume of quirks is what is the challenge. I mean, we've we've got hundreds of edge cases. We had to analyze Sol C because that's the only definition of the language. Uh, and go through the code and then test it and then go to sanctuary and look at massive uh, numbers of contracts. Uh, run uh, our compiler over them or run our parser over them.

What breaks, what doesn't. We've got extensive uh, test cases. So, that was the challenge. It's the number, not the not any particular difficulty. Fantastic.

Thank you. Do you dream in byte code? Uh, no no, I don't dream in I dream compilers, actually. All right, Anthony. Thank you a lot for your time and all the best to you.

Thank you. People, our next session will start in a few minutes. I am off for today. Thank you for those who spent a bit of time with me this morning. Um, and I'll see you soon.