Decentralized Threat Detection Bots by Jonathan Alexander | Devcon Bogotá

foreign [Music] so my name is Jonathan Alexander I am CTO at open Zeppelin I'm the co-founder of forta and in this talk we're going to be I'll be covering research and development on decentralized threat detection Bots there are a number of um contributors to this presentation whose work I'll be citing here um a lot of the information and research is been conducted by these individuals it includes Christian Seifert who is researcher and Residence at forta Mariko wakabashi who is lead machine learning engineer at open Zeppelin Dario laboulio who is security researcher at open Zeppelin we have a number of independent researchers and we also have teams at nethermind and lime chain who have contributed to some of the techniques that we'll be discussing here and uh so I'm going to start with a little bit of background we are starting to see increasing Acceptance in the space that security needs to be end to end and that part of end end is secure part of end and security includes runtime monitoring and threat detection we have multiple leading security audit firms uh open Zeppelin chain security hellborn mixed bytes who are beginning to make recommendations along with their audits for a smart contract monitoring that can be done post deployment the recommendations include covering protocol assumptions and invariants critical protocol variables known protocol risks that have been considered acceptable the use of the use of privileged functions and the transfer of privileges on chain and cross-chain synchronization and oracles and keeping track of the state of data that's meant to be kept in sync across these different environments and also external contracts and protocols that a that your protocol May rely on as well as identified attacks that may follow certain patterns that are already known and this monitoring is meant to cover the knowns the things like specific attack patterns that we might see so that's one thing when we might we want to monitor the known unknowns which would be places in the protocol where you know you're not sure what might happen and you might want to monitor that and then very importantly the unknown unknowns so we might have things in our protocols that we think are invariant and will not change and yet if something could happen that was unexpected we would certainly want to know about that so the challenge though with runtime monitoring is if you take that whole list it's very difficult to implement the full set of monitoring that you might want especially when you get into some of these areas that are not the responsibility of the protocol team themselves such as other protocol implementations and their risks and updates to those as well as things like the different attack patterns that have been seen in the ecosystem but also maybe new and emerging and we see new attack patterns emerging all the time so we uh at open Zeppelin we started working on um post deployment tools and specifically monitoring a few years ago and this led us to the realization of the challenges here and we decided to build and along with a number of Partners now have launched a decentralized network for runtime threat detection it's called forta I'm going to be citing some of it in this uh in this presentation and a lot of the research that's here has been done by people working in in and around Florida forta you can go check it out it is it's a decentralized network of scan nodes there's thousands of scan notes running now across number of uh number of main Nets and the way it works is that you build Bots threat detection Bots they're deployed as containers onto the network they're run on multiple scan nodes and it is those results then that are brought together to detect threats so um so and the idea here is that for protocol teams we um I mean sorry am I like literally standing right in front of the slides the um the uh for protocol teams back to the prior Slide the the challenge of trying to do sufficient and real end-to-end monitoring and threat detection now is not just the team doesn't just have to do it themselves they can take advantage of a community of threat detection we can work together to have a real complete solution so um first of all I'd like to share with you some observations from research that's been done on threat detection that hopefully you'll find a little bit interesting the first thing is that as we look at uh the history of attacks we begin to see a very common pattern of a set of stages that an attacker goes through and this starts with funding obtaining funds that will be used to carry out the attack um it moves to preparation and there's a number of steps that are common in different kinds of attacks it may involve deploying a smart contract which is going to be used to carry out the attack it may involve token impersonation uh act it may involve the transfer of privileges or making use of privileged functions and then there are different kinds of fraud techniques that we see being carried out to prepare for stealing funds and and we'll get into some of that a little bit later the next stage is exploitation that's where the actual attack occurs in this stage we may see flash loans we may see private transactions we may see re-entrancy minting anomalous transfers anomalous functions large balance changes things like that and finally we have the phase of laundering which is where the attacker takes the funds that they've obtained and they move them to try to uh you know hide the trail and uh and exit with funds one of the other interesting things in reese in the research is that looking at a quite a large number over 180 attacks over the last three years specifically in defy we see that over half of the attacks are not have been non-atomic and what that means is that in the exploitation phase it is it is more than one transaction and typically many transactions that's used to carry out the completion of the attack and why that's important is it means there is a time frame over which the exploit occurs and therefore early detection could be useful to taking action and possibly mitigating the full effect of the exploit so um so that's another interesting fact another observation is that attackers often use more than one account this may be obvious but as researchers have looked at this and you may have seen this in some other presentations we can use heuristics to associate attacker accounts uh through and then by associating attacker accounts into clusters we are able to track and attack through those stages as I just talked about even if many accounts are being used as part of that we have examples of using heuristic-based approaches such as a connected component graph algorithm which would allow us to kind of graph together the these connected accounts and then track them as a cluster another observation in over 40 percent of the attacks the attacker deploys a smart contract to execute the exploit these attack contracts differ from benign contracts in very clear ways and here I'm on on this slide I'm citing um a uh some research conducted where over ten thousand smart contracts were sourced from Lua base 155 external accounts were identified as tagged as having an exploit in ether scan and then those accounts were associated to a set of smart contracts that those accounts deployed so we had a set of benign contracts and we had a set of um uh malicious contracts using a classification of the top 100 op codes in each of those smart contracts we see incredible success in being able to identify contracts that are benign so in this case of the benign contracts night with 98 accuracy the classification algorithm is able to say that is not a malicious contract and in the case of the set of malicious contracts in this case 21 malicious contracts the classification algorithm was ident was able to identify 17 of those as suspicious or likely malicious so this shows promise in the ability to just identify among a smart contract if it falls into the category of potentially malicious taking that a step further we also see that when we get into the details not not just kind of like which uh op codes are used but we look at the patterns of the op codes of in use in these smart contracts that are used for attacks we see that they follow very similar patterns and so if we look deeper into the patterns in this case taking a even more advanced kind of approach to classification where using techniques from natural language processing grouping together op codes into groups collections in order and then analyzing the use of those different groups again with classification in this case the researchers looked at over 12 000 benign contracts over a hundred malicious contracts um and then using the uh the tfidff TF IDF um NLP technique of grouping feeding it into the classification now we get even a higher level of precision in this case 88 precision where the technique and the classification would predict that a contract was malicious when it predicted that it had a fairly small rate of error so 88 accuracy and also in this case the researchers went back to a number of known attacks that were that have been historically analyzed and carried out via um smart contracts and some of them are cited here the wintermute audience inverse Finance Finance exploits and in that case this technique was able to predict in almost 60 percent of the cases that the attacking contract was malicious so again promising areas of of research that we can identify malicious contracts um the last observation I want to share with you relates to fraud uh fraud is an attack um that is usually carried out with social engineering or some kind of web to attack where um in the case of as the examples given here ice fishing uh an attacker will get users will trick users into signing approval transactions that give the attacker control over their tokens these also produce detectable on-chain patterns and so in this case you can we can use heuristic based approaches just to look for these patterns of you know accumulation or repeated kind of actions that might be carried out by a single account or again a cluster of connected accounts that are um that are carrying out some kind of fraud attack on a lot of users and then and then you know going to steal those users tokens they might do it we've seen this happen in in certain cases we've seen attackers who do all the preparation up front and then they carry out the attack all at once we've seen some cases where the attack just continues over time but in this case using a heuristic based technique um in and then using public data about ice fishing attacks that occurred over a one week period in the month of September this technique was used was able to with 95 percent Precision predict that a uh that ice fishing an ice fishing attack was underway so in this case it made 12 predictions that there was a nice fishing attack and or I'm sorry it may have I probably have the data runs probably like 13 and um but it was 95 percent accurate it's a very very low number of false positives and of the 21 ice fishing attacks that were known publicly to avert that we could see historically had occurred that week it actually predicted 12 of those so again these are promising kind of this is kind of the research that's going on and some of the things we're seeing so now what I'd like to do is share with you some threat detection bot techniques that can be used to detect some of the various kinds of threats that we're seeing um there actually will be a workshop tomorrow on photobot development if you're interested I think it's tomorrow afternoon I don't have the exact details but it'll be useful for you to know just up front that because I'm going to reference it in a couple cases that photobots you implement handlers you can Implement in JavaScript or python one of the things you can do is initialize you have your own environment it's a doctor container and then you you kind of subscribe to either handle transaction or handle block and you can pretty much do whatever you want you can also look up other alerts that have been emitted by other Bots and ultimately what a bot does is it if it has a finding if it if it finds some kind of threat or something it one alerts on it shares that which then gets published and again other Bots can take advantage of that or users can take advantage the first technique that I want to discuss and on I have three techniques I'm going to talk about and in each of these I'm providing some examples of bots these examples if you go to the documentation on forta you'll be able to find references or if you go to the Ford Explorer you can also look up Bots by name and they most of them are open source and you can link there are templates that mostly are what I'm covering here that have been provided by either the Florida Network or nethermind or lime chain so the first technique I want to talk about is multiple Bots working together in a group to Track Attack stages and then to with increasing confidence as an attack goes through stages be able to to express that there is potentially attack underway going all the way to complete confidence that an attack has occurred this is done with having a set of bots that atomically detect some of those things that occur in funding in preparation in exploit in laundering you can have individual Bots looking for individual things like suspicious contract creation contract spoofing ice fishing large transfers money laundering you can also have Bots that identify clusters of accounts and we have there are example Bots now available where you can see the source and how to do that and finally once you have these Bots who are each kind of making an individual Discovery then you put that together with a pattern matching with pattern matching on the various alerts and being able to then declare with some level of confidence this is what's going on this is the possible attacker and eventually this is the party or parties under attack and so um this there were there are some Bots that you would be able to find the technique in the Florida Community has been referred to as a combiner bot the second technique that I want to discuss is simulation in this case you can have a bot that literally while it's executing forks mainnet with ganache with something else you can fork mainnet and then run simulations to detect issues there's kind of two techniques that have been used here one is if you have a known protocol and you know that there are if if certain things happen on the protocol then you would know that there must be something wrong perhaps an attack underway you could you could execute those in simulation after every single block that's been mined and alert if the if the transactions failed an example here is a liquidity tester this is a bot that's been written to take a defy system identify the top 10 current uh liquidity holders and simply execute their ability to exit their position after each block and in the case if any of them were not able to exit their their position it would alert the other technique here is to test contracts and we have a couple of Bot examples one that uses fuzzing one that doesn't but in this case the simulation is to take a deployed contract Fork the network and then basically try to execute every single method knowing nothing about the the contract itself just detecting it through code and trying to execute it and looking to see if any execution results in let's say a large funds transfer or some other kind of unusually anomalous occurrence that would say there is something this this contract is trying to do something or it seems to result in a big a big transfer of funds and therefore is suspicious the final technique I want to talk about is the use of machine learning in bots in the case in this case you can take a machine learning model you can serialize it to a file in the case of Florida you can deploy models with the bot and you can use the initialize function to load a model and then in your Bot you can execute the model this has been used successfully already in a number of bots that do anomaly detection uh and also a malicious contract detection so in this case for example there is a smart change detector bot that I've referred to here and a Time series analyzer that they use um they use machine learning algorithms I know one of them is a library that actually The Meta team developed that is very good at detecting outliers uh and um D deviation with a limited amount of false positives so it's pretty good at saying you know this is a real outlier and this is a you know there's a price change that occurs here if we're looking at on chain prices or there's uh if if you're looking at let's say the balances or certain uh values in your smart contract variables you know it can detect over over a period of time an outlier that is uh that is perhaps meaningful in something you should look at and then in the case of the um the machine learning for smart contracts there's a bot called the contract contract Destructor and some other Bots that do similar things where they literally take a deployed smart contract address and they um they decompile it to get the op codes for the smart contract and then they use these machine learning models kind of that I referred to before to pattern to analyze and classify that smart contract and and therefore predict whether the smart contract is malicious or benign so these are all techniques that are already now in use and further research is underway on how to improve these um these kinds of bots so I want to wrap up just with a couple of challenges um and you might have been thinking about some of these as I've been speaking there are challenges here that we're all dealing with um Atomic attacks are obviously a challenge so I said that well we have history to tell us that more than half the attacks are non-atomic but of course that means that 40 to 45 percent of the attacks are Atomic also private transactions can be used and so this is a challenge for threat detection and monitoring another challenge is that there are teams who want to monitor in secret that would prefer not to monitor in public and and they may have good reason for that for example if there is a known vulnerability and you want to you're you're in process of trying to deal with the non vulnerability you may want to do that privately and not on a public system and then finally we have the challenge of response latency with go which goes to how quickly can we alert but also can we respond and is there even anything we're able to do once we've detected um these are things that Ford is working on there are already some things available for private but these are also areas for future research um uh latency by the way on the Ford network is about 40 seconds from Mind Block to alert but things that are being looked at are possibility of trusted private scan pools for people who want to monitor in secret pre-submission or mempool transaction scanning on-chain alerts which would provide a possibility for on-chain reaction and other systems that can be used for automated action and follow-up open Zeppelins working on these things for the community and many of the contributors I mentioned are starting to work into these things and um and to wrap up I would say if you are interested bot Developers security researchers data scientists um please go to florida.org and you can learn more and find ways to get involved and there's various grants and incentives available for people to get involved and with that undone and if there's time I'll take questions [Applause] thank you for the sorry thank you for the talk today I find the Bots incredibly useful for teams but what I'm curious about is what what happens once it's once the bot alerts a team what does the team have to do to them like you know make a fast impact right yeah that's well I mean in this conver in this presentation I wasn't so much getting into that but that's obviously a big topic of conversation for everybody um I think we are starting to see um well well we have seen many attacks in the wild where teams have paused um their smart contracts and we even saw a chain pause the whole chain um we are seeing more discussion about that because I think we're we're definitely headed down the road where emergency response capabilities may be an acceptable thing to have on your protocol but it needs to be contained it needs to be limited we need to kind of have some level of Delegation to a set of responders so that they can maybe do fast response but with limited control right so we're starting to see protocols implement this and teams talk about it publicly um you know I think you may have heard some of the leading D5 Protocols are starting to discuss this so pausing and then having some capability to uh you know deal with the attack is is kind of a base point but there's a lot to get into there because uh you know even pausing depending on the nature of the attack may not solve the problem right and and I think the important thing here is that we all have to start thinking about this because if you go back to the you know the unknown unknowns it's just impossible that we're going to catch everything up front and so yeah but but it's not solved that's that's the true answer yeah right um thanks for your talk Jonathan um speaking of things that you do after the attack uh what is open sibling doing today with that information how does uh open sibling harness that information to transform that into knowledge that is the first question and the same question um is there anything to do besides a smart contract audit in terms of things that you can do before that attack happens like this scanning the the mempool or or having some heuristic in there well a couple two parts to your question um the first first one is that um uh open Zeppelin is in our audit practice we are now when we do an audit we typically also make recommendations because about what should be monitored because the Auditors have good insight into well we're training on our Auditors more and more in how monitoring can help so then the monitors also the Auditors also understand the protocol and they can say to a team you know these are things you should think about monitoring and production because if our audit didn't catch anything or these may be just live risks you're going to be exposed to open Zeppelin is also working on tooling to automate the creation of monitoring and we have like if you've seen our contracts wizard we're going to be starting to build in things that if you use open Zeppelin contracts will auto-generate monitoring templates for you and then we are working we have a product we have a product platform called Defender we're we're working on how to automate those responses that we were talking about like how you could at least get to where you could pause as quickly as possible as an example so we're doing all those things and we're collaborating I know Forte has many multiple I mentioned chain Security in halborn and mixed bytes are all doing some similar things I think yeah we're good thank you