# Auditors & Bounty Hunters: who should secure your bags? | Panel | ETHDam 2024

- Channel: [CryptoCanal](https://streameth.org/cryptocanal)
- Date: 2024-10-07
- Duration: 35:06
- Watch: https://streameth.org/watch/yt-tKGr6DqZlIc
- YouTube: https://www.youtube.com/watch?v=tKGr6DqZlIc

## Description

“Auditors & Bounty Hunters: who should secure your bags?” panel with Oliver Hörr from hats.finance, Gonçalo Magalhães from Immunefi, Erik Arfvidson from Euler Finance, and Josselin Feist from Trail of Bits at ETHDam 2024. Moderated by Matthias Egli from ChainSecurity.
https://twitter.com/0xAngler  https://twitter.com/realgmhacker https://www.linkedin.com/in/erik-arfvidson/ https://twitter.com/Montyly https://twitter.com/MatthiasEgli https://twitter.com/HatsFinance https://twitter.com/immunefi https://twitter.com/eulerfinance https://twitter.com/trailofbits https://twitter.com/chain_security 

Sterling Schuyler - MC of ETHDam, copy and content writer for emerging fund managers & crypto enthusiasts. 

ETHDam - a conference and hackathon held in the heart of Amsterdam, Netherlands from April 12th to 14th, 2024, celebrated its second edition, gathering more than 600 participants. 

In the dynamic space of ETHDam, privacy and security took center stage, featuring groundbreaking discussions on hacks, recovery, and the revolutionary work of figures like Pertsev. Privacy is dead in crypto, people that know, know. People who don’t know, should know. 
ETHDam is powered by CryptoCanal, an education and events platform growing in Amsterdam, spreading its roots to Rotterdam and Zürich.
Keep up with us to see updates on future events: https://www.cryptocanal.org/ 
Follow CryptoCanal on X: https://twitter.com/CryptoCanal
Join CryptoCanal TG Community: https://t.me/CryptoCanalCommunity 
Join CryptoCanal Discord: https://discord.com/invite/XJVjpCqQBz

We would like to thank our partners that made this event possible. 🌷
Battleship Partner 
🛳Oasis Network https://oasisprotocol.org/

Jet Ski Partner
🛩⛷  NEAR https://near.org/

Canoe Partners
🛶WAKU https://waku.org/
🛶Trail of Bits https://www.trailofbits.com/
🛶Avalanche https://www.avax.network/
🛶Privacy + Scaling Explorations https://pse.dev/en
🛶Threshold https://threshold.network/

Our Canoe Partner & Official Node Provider
🛶dRPC https://drpc.org/

Sponsor
🤝EF Ecosystem Support Program https://esp.ethereum.foundation/

Paddle Partners
🚣ChainSecurity https://chainsecurity.com/
🚣Lido https://lido.fi/
🚣Cyber Capital https://www.cyber.capital/
🚣Diva https://www.divastaking.net/
🚣Firn Protocol https://firn.cash/
🚣Beefy https://beefy.com/
🚣0xbow https://www.0xbow.io/
🚣Obscura https://obscura.build/
🚣Panther https://www.pantherprotocol.io/
🚣Maven 11 https://www.maven11.com/
🚣Zama https://www.zama.ai/
🚣zkSync https://zksync.io/
🚣Secret Network https://scrt.network/

ETHDam AfterParty Fren
🥳Bitvavo https://bitvavo.com/en

## Transcript

[Music] okay next up is um security panel um I'm Matias from train security and I'm very happy to be here together with a lot of my colleagues in the security industry so we going to discuss state of audits and especially focusing on the model which has come up which is um the crowd audits right we have uh two people we have some from bu bounties and so on but I'll have all of them introduced themselves first so you know uh who you are talking to today so once to start all right um yeah so I'm Oliver from heads Finance U we do crowdsourced audits completely on chain in a decentral way um have a global community of security researchers and since it's peer-to-peer we don't even know where they are often and yeah we observed a lot of um problems in the auditing Market um especially when um it's a crazy bull market and there's a lot of demand then we have seen that um Auditors have a great pain in scaling up and keeping audit quality high and so we thought that if we can completely uh crowdsource this and make it the global Community then that we can solve some of these pain points point and as well allow Builders to um obtain better audits cheaper hey I'm J from TR of bits uh if you don't know us we are a security company where we specialize in high hand security technology blockchain cryptography application security machine learning um one of the thing that I think def us from some of our competitor is that we always try to apply the latest research in our day-to-day application as a result we have buil a lot of Open Source tool you might know slitter Aina or Medusa um so are all kind of like the tools that we are trying to to bring to to everyone and to the [Music] community I'm gono uh GM Hecker online uh I'm head of security at immuni immuni is a web 3 BG boundy platform hosting some of the largest projects in defi some of the largest blockchain projects uh we facilitated uh the payment of over $95 million to White hats that BAS basically uncovered live um vulnerabilities in production code and protected funds from projects your user funds as well hi I'm Eric arvitson I'm the head of cyber security at Oiler Finance um we're a defi lending protocol um that is modular scalable um and uh we've been doing a lot of audits with this people over here and uh yeah thanks everyone so yeah um why am I moderating this I'm like I'm Matas from chain security and like kind of more on the older side chain security is an older audit company right and we had the pleasure to watch the space develop from its beginning till now um where historically you could say like okay there were the audits kind of the first thing during the big times of the Ico boom and then we started to see Buck bounties and then we started to see crowd audits right so this is kind of a little bit The Evolution uh and it keeps evolving right so my uh we'll have a poll for you to answer in the meantime which will help us a little bit um what we all do is like we uh we need talent right and um we kind of give that talent to uh use to secure ethereum to secure projects so what should projects expect of you like of the outcome you give like after an audit after a code review after a buck Bounty this goes first to the three non projects and then there is of course uh Eric here from the project side of things so uh what should the ones hire you expect of the outcome which parts and then might be missing from the point of view of an project so this goes into you okay I can start so um for us in general the whole system is incentive driven and uh you only pay per vulnerability that has been found so um it greatly depends on um how large is the the available pool that um the community can win for hunting vulnerabilities and um how large was the code but generally we give recommendations and if the customer followed the recommendations uh we have a very good experience of high audit quality um as well uh we we are doing multiple audits after the the first audit has already concluded and finding a lot of issues that have been missed by previous audits um which gave us a really uh good track record um but overall I think um there's always a tail risk that there's still um a hidden vulnerability in there and we try to um make a meeting with the customer after the competition and like discuss okay are we feeling confident in deploying this code do we believe there's additional security measures needed but overall of course the ambition is to have a very low risk that something is still in the code but it's never zero so from what kind of a security provider perspective what you get when you hire a FM like trof beats or CH security basically you are Lending uh security engineer so you have someone that is going to you know work with you for a couple of weeks to try to find vulnerability in your Cod base to try to understand what are like the different risk what are the different mitigation that you can put in place um it goes through like obviously fixing the bug but also discussing about like the architecture are you using like you know the proper Library are you using like the proper integration how are you doing the testing how are you kind of thinking about Security in the long term in your incident response plans in your monitoring uh so I think like really like doing a security overview is kind of compassing all the different aspect of security and having someone dedicated to help you grow in all the different aspect for like a a period of time yeah uh I think that projects would first of all expect um a security T partner on U from imuni from all of these colleagues as well uh we typically uh come in uh much later in the protocols security life cycle um I would say uh where we see the bug Bounty program as one last line of defense uh after going through audits Etc and well doing everything one can do to to harden your code never perfect but one does what it what it can do uh eventually in the end you you will have to deploy to production you will have to have uh real economic Val value inside your contracts so the bug Bounty will serve as a last economic incentive let's say much like block blockchain style um where you incentivize the security Community um to look at your code and potentially find vulnerabilities and responsibly disclose them and get rewarded for that and for securing um live funds at risk um so at Oiler Finance uh we think that like the strategy has been to do a like layer approach um we don't simply just do a single audit we do multiple Audits and actually we start during the development phase we do code reviews with security Auditors to help us give us feedback before we go into the audits um that way we are getting like from the beginning like not just like um finding vulnerabilities but we're also finding uh like improvements in gas op ation code improvements just in logic also readability uh which helps make a better codes and then the last thing to top it off is we do a code competition to kind of reassure of all the Great Value we've been getting across the different stages of audits and reviews and fussing and everything that we're trying to work with the security researchers on this like one thing we we always have to do except for buck bonies which usually are like uh always ongoing right we have to think about how long does it take how long do we want to spend on it right and this is very different between crowd Audits and traditional audits from my own experience I I have the feeling crowd audits are kind of often half as long as the time we would spend rough rule of thumb right can diverge some crowd audits are very long uh but most are very short um but we also have seen differences between uh traditional Auditors estimating this the length of uh a review and as you might know like it's time uh time is money also in this business um so how do you figure this out how long should it take and and the previous question was a little bit geared towards that right because uh if I recap Oliver's said like at the end of that you should come up with something where you feel you have a ideally high confidence in the security when you hand over your report it's a stand I share there is uh the other uh take of like you had a dedicated time for it right and you get that time but how do you estimate if you say like this is a project let's say like Oiler and you know in the end they need to be secure right they they are going to handle money which will make you head line if they get hacked right and how do you estimate how long does it take to to become secure um and again at the end to to you but so yeah I okay I can go first on this one um so like the question is how do we estimate how much time do we need for for like reviewing a cut base I think the first things to kind of acknowledge here is that security is not binary it's not it's secure it's not secure it's it's more granular right like it's how much can you spend versus how much you know can you can you achieve uh so when we have like a new client and they're like okay we have like this code base we want to understand how much time do you think do you feel you need to be confident uh we're going to look at the Cod base we are going to look at the size we're going to look at the complexity based on what we know from the previous audit uh usually we can do like a pretty good estimate on how much we need uh to get some confidence let's say it also depend on the complexity because if you are building something really new and it's like you know 10K line of code um we might not have the time to go in depth into all the components so what we can do is usually like a first review where we are going to get like a good understanding of the code base we are going to focus on some specific component and at the end of the overview we can also provide uh advice and recommendation such as like for example luk um this component was not totally cover we have found a couple of things but there are probably more to be found so we will recommend you to also focus a bit more time either do like additional review do a back context or even like do like an internal rview more focus on this or W in variant or all the type of things um so my deal here is we can do like an estimate on how much we need but there is also like a lot of back and forth that are going to go through the Cod review that might give you some insight into what should be the next step for yourself so for us it's different because we are not conducting the audit ourself but before we do a review of the code and then based on the complexity and the um pure lines of code we're giving a recommendation which usually is between 10 and 14 days but often as well if it's like 8,000 lines of code plus and complex then we will tell them like three weeks uh but honestly it's often very tough discussions with the customers because they say like no we only have two times you want to deploy afterwards and uh often good advice Falls really on deaf ears but what we try to do is um while the competition is running we're constantly talking to the um the good security researchers that are participating that we know and we ask him like hey what do you think of the code um are you in within time or um they will as well ask us like hey can you extend this and we often extend um the audit competitions uh for another week or so um sometimes we even increase the pool size and that do often me just being in the DMS of some founder and harassing him like Hey please we need more pool size we need more time you know like this not going to be the quality that we want to offer to you uh um most of the cases I'm successful sometimes I'm not um but in the end uh we try to be a permissionless protocol we try to support the approach of the customer and then try to guide him into doing what makes sense sometimes as well they have hard stop because um afterwards the next audit um is starting um then I worry less because I know okay there's like a next step and another audit and but uh yeah it's uh it's difficult and I I think it will become even more difficult if the market is getting more crazy uh in the bare Market it was easier to keep them you know like in line and remind them that security is important but if they feel pressure they need to deploy next week because someone could Fork their code and slap a token on top uh yeah it's going to be a tough time for us Security Professionals I think so yeah I totally understand what what Oliver is saying uh not just uh the rush to deploy but also even just the fact that perhaps the more time a protocol uh will have an audit going on maybe needs to pay more right so there's there's that constraint which is maybe the uh the biggest struggle of an audit will be that it is time bound right where basically as much as as a good talent as they can be they have a limited amount of time to review the code getting familiarized with it and and trying to find vulnerabilities essentially this this happens both in in traditional Audits and cross sourced audits uh it's pretty normal uh that's why I also very much enjoy the the like swiss cheese model like the layer model as as Eric was saying where you have multiple audits uh if possible but most importantly and hopefully the industry will will start growing towards that is that it is very important to have a security partner essentially as jlyn was mentioning it's not just the the audit report output that you have but also all these conversations and um all all those uh advising advising on what you maybe should be doing maybe this pattern will lead you eventually to to struggles Etc and of course at the end the book Bounty will will precisely uh meet in the middle with that point where basically you have some security researchers that don't have a time constraint so they can um they can be months looking into your code and getting specialized into it and potentially find more bugs um so um at oer what we found is that like one of the big indicators of when picking an auditor is like if they're evaluating the code that they're going to give us a quote is based on lines of code or actually understand the complexity because that really ties into the time and I've noticed that people that only quote by lines of code and don't really understand the codee and complexity usually are end up being bound like you say to the time constraint um but it's not like that easy cuz like you know you are getting a quote it's not like you're going to get extremely familiar with the code as you they try to do their best job giving you a quote but it's never perfect and we work with Auditors that they've asked for extensions uh just because like they can't get it all done within that time and it's nice when you get transparency from them too kind of what you said like if you know that it's not going to get fully covered you at least get visibility early on that like this probably needs more time to get audited so most of here never got an audit um oh it is but still still quite some got one or many right I'll get to an interesting question right after this one quick followup on this like you mentioned sometimes there's not enough time right um and then the same applies to everyone right so how should uh I as a user if I read a report uh from heads Finance from Trad off bits from chain security um where do I find this information that it's not enough right do you feel you are the one who has to tell the public we we know the client is kind of the project right but still it's your public report out there should it State this warning that this audit was like a limited review it's always limited but limited in a sense of please be careful here so on our side like all our report they have like a long coverage section theying where we think there like know Improvement to be made let's say in practice I don't think people read the report like there a lot of emphasis you know like you should have a public report whatever but I'm pretty sure even if we were having like a report a public report from us saying don't use this protocol it's trash I'm pretty sure no one will read it and no one will notice it so yes you can find this information I'm not sure people really will have any takeaway from it my perspective yeah so my view unfortunately until now is that security is the job of the team and the community and the user doesn't really care um exactly like like the users usually don't read audit ports um in very extreme cases for example we had one case where great team great project but they just didn't have the money for a decent pool size H and in the end we didn't take any money from them but we didn't give them an audit report because we just said like we don't consider this to be um an effective audit we don't have confidence in this we will not like make a report for this and publish it um but usually um well we are not auditing itself it's the community and all of it's happening out in public so the the way our audits work you have a GitHub repo and you see in real time all of the issues are getting created and you can even the discussion between the security researcher and the team is public as well in the comment section of those GitHub issues and um at the end of the day um we can only go through those issues and check like how many are there how was the quality how was the discussion how was the people that participated um if we have a really bad feeling about it we are not penalizing the team as of now maybe in the future we will do it but we will um very directly and clearly word it to the team and then in the end I I feel a little bit like a lawyer yeah I'm telling like hey this is the risk for you um and then the team has to make a decisions um there's some risk for our reputation in this um because if the customer completely ignore ignores our opinion um but so far we are not shaming customer publicly and um I think it's as well very difficult topic so yeah in very extreme cases we are refusing to audit um make an audit report and otherwise we're not but if you have an audit report out there in general you say like we have high confidence at this point high confidence um as much as possible yeah so okay and and also like I can imagine if uh Buck Bounty Hunter would see there is a report which says low confidence many things missing right then obviously they'll spend a lot more time on this one because it's just more likely that they'll find some so to me personally it's actually a service to the industry to be more transparent to be more clear and basically to say look this is what it is um and uh take it but not to hide this information which we have really as Auditors um give a public report but State what it is right be an independent third party but um yeah you were going to to talk first yeah yeah I just wanted to say like at least on our end like what we're doing is we're sharing the reports on our GitHub repo next to the code we have like a uh report section uh with Audits and um I guess I haven't seen Auditors actually like put a disclaimer like of what they didn't find or that their level of confidence but like they do leave like informational things that I think other Auditors can take a look and maybe explore in it deeper uh because they have more time or they're taking off since we like have like uh done audits one behind another like overlapped and I think like that sharing of information and transparency like think helps the other auditor find new things that the first auditor didn't find would you would you publish an audit which says low confidence um again they don't really say the level of confidence it it would be great if they put a level of confidence part of the report but I haven't seen reports actually including the confidence level our ours would but would you publish it then if it's low yeah I mean if it's going to help the other auditor do a better job like I think it's useful information it's always the scare of Auditors to basically say like hey the client is like not going to be happy with that like it's perhaps it's still their thing to like but I I think OA is not a great example here because one AIT yeah a they have the the motivation to be very secure and they have the financial means yes and um the I mean you know it very well right like your end goal is that there completely bulletproof as I mean it's never completely bullet but very very close to it and we're working with a lot of teams they have budget exactly for one audit and that's it and if you then give them a report with low confidence I think it will be a very different story than for like a team like yours yeah I completely agree if you can only do one audit like the confidence probably not look great if I guess you using a single audit to like bring uh confidence to the project that they did their diligence to do security but yeah it's a tough difficult question to answer if there's a single one audit and they put a low confidence like okay but maybe you can turn it into like a code competition off there like to kind of like explore it once you deploy and keep finding bugs and maybe have the contracts be upgradeable I guess it all depends right yeah and just to to add finally uh I I kind of agree with with JN that yeah he he's right most users won't read the the audit report or maybe not a single user I don't know uh and It ultimately did anyone read an audit report here you see this my exp I've read them but not your own not your own audit report yeah okay um quite some and at the end of it uh it's the job of the the project team to to convince the users how how secure they are um but I I do think it's actually pretty valuable uh for that transparency and because I see more and more that the security Community actually grabs that and and if if a project suddenly has a huge tvl and and and low a poor security stands typically gets called out we we see that in in the bug Bounty world as well where we don't we don't really need to shame um projects because um the hackers ultimately will will will kind of Shame the projects if they really misbehave there's a fine line there between whether that's actually true or not because we they shame the project but they also shame the users of the project right which trusted sure that's true in the rep it's kind of hard to to prove that there there was actually some uh something going on there essentially we cannot disclose typically so sometimes the whide is not right uh but yeah and on your final point that you mentioned that maybe the bug Bounty Hunter wants to to hunt on a project with with poor with a poor security sense and low quality reports uh we see it all basically uh some some Elite hackers actually want to take the challenge of of reviewing uh protols that that had an insane amount of of of audits just to yeah for that reputation that I I I got to find the uh this is yeah I I I think there's a problem with this yeah like the security researchers are do shaming and um I I know it's a fine line but actually Pro uh shaming uh but uh the problem there is I feel it always stays in our bubble like people like we will maybe see it but the average users they don't even see this kind of tweet or something so our self here in the security industry is not working very well yeah there's like Auditors that keep doing like lowcost Audits and then projects are getting hacked and I still have people coming up to me telling me very proudly I have gotten an audit from this company I'm like this is not a good thing you know like for me this is a negative thing but um yeah so I'm we need to somehow get this information as well to like the the user side I think in in this somehow we are not able to do so far unfortunately yeah yeah it falls back into the question of like uh one deep audit with one right versus many many many audits on the other side layered approach right pros and cons were already discussed on this um I I know this uh is a take where like uh based on the model it's different but would you say like over time a crowd audit is all you should get right is over time the thing where you basic basically say no go to the best auditor you can find get pay them for the amount of time they say be it two months so be it right or do it perhaps the oiler approach currently which is hire a lot of people follow it up with like what 1.2 million uh 1.25 million uh uh code competition like basically crowd audit right so what to do I think at the end of the day it's a question of budget like security is how much budget can you put and with infinite budget of course we can if you aren't you the bang for bucks right I think it's more about okay you have some budget for security how can you use it in an efficient ways and maybe doing five different audit at the end of the development is not the best usage of audit or back contest is not the best usage of your money maybe do one card review you know halfway through your development to get earlier feedback it's what we call early stage review where we review like proof of concept or working progress codebase maybe some component is not fully fleshed maybe they don't know about some architecture know and you can have like an earlier discussion just to get you know uh some of the things set done correctly and you have like a stronger Foundation but at the end maybe you don't need five different audit maybe you just need two because you started early on so it's really about you have a budget how can you first increase this budget and how can you use it efficiently we have we have seen also like you know some team having 10 marketing people in their team and no one with security in the title in the organization it's usually an indicator that maybe the priority is not on security and it's kind of like a red flag right I mean the budget is always important but I do recommend if your budget is limited to start with a smaller independent auditor I found like they're usually really great and you can cre a lot of value and then follow up maybe with a big name brand company to do a bigger audit but usually starting with a smaller audit auditor that has experience and participa in code competitions is a great way to get started at a low budget and also like bring a lot of value and you find people that have worked in similar projects to yours I think that's probably the best bet read their audit reports to pick your auditor um so I believe um you um always should get one more than one audit um and I think even though there are some uh uh companies that have earned the right maybe to claim we we can do it with one audit like for example you guys or open Zeppelin or chain security yeah that could even say like if you do one audit with us we are confident but even though I think no auditor should have this confidence um and this ego to say like just do one audit with us and you will be secure um and uh so I for my opinion the best approach right now is to doing one audit with a with a firm um and then as well like make sure to do like an Architectural Review um and like a best practice review with them and then do um audit competition but um it breaks down a little bit if you have very big pool sizes so for example if we have a $200,000 audit competition or 1.25 million um audit companies are participating in these big um audit competitions yeah we know it for our bigger Auditors and we even ping some of them and say like hey there's a very big pool coming up do you want to compete with your firm maybe as well if it's something um more Niche like rust then we have like two three like teams that we ping and they actually start to compete with their company so and then there the question really like how many audits did you get in this competition yeah because you have like 50 solo Auditors uh plus two companies or something are suddenly competing and uh if you give them a little uh extra there as well all make an audit report for you um so I think um the the the boring answer right it depends um I think best practice uh centralized audit audit competition together and maybe in the future only ordered competition but with big pools and maybe some base fee for firms to to make added services on top you have been a Hecker before right on immuni yes so how how did you choose in which projects to look was it related to how many audits they got or I I think that that that really is different for for every hacker there's a ton of different incentives that I think that's ultimately the goal how how can a protocol um attract uh talent and and get some good security talent to look at their code because that's the important part so very underrated is uh in my opinion get yourself a security partner or a security guy something that helps you look at your particular project particular budget and see what you can do uh and be as efficient as Joseline was was mentioning as for um what will incentivize the hackers there's a bunch of stuff obviously money um the more the bigger the Bounty uh the more likely you will be to attract good talent but also just um The Challenge the technical interest in a given uh protocol uh I think that security researchers are more and more getting specialized into a particular um protocol model or maybe a different a specific um for example fuzzing or or something and they do uh I'm glad they do that because that's that's the way they will be more and more valuable in the future um yeah very different uh set of skills for for each hacker for sure so I think we are almost out of time or over time we got time for maybe one or two questions yes yes so uh I already got some question questions but take a look at them also upload them right ask your own one um and um then we can go into one I'll start with the one which was already mentioned on the previous panel right skin in the game very simple like um should a project uh should like an auditor um put skin in the game in the form of like money right money which is being paid in case of a hack yeah um I would say that the it's it's not terribly needed I mean um I do enjoy those models and that that's seems like a competitive Advantage Sometimes some monitors will will put skin in the game Sherlock for example puts uh coverage in in bug bounties um and imuni I believe that you you could also put like money in in vaults on hats I think and Auditors sometimes will will actually enjoy the protocol um and and be confident in their security and put some some money right so I think that's a great signal um you're putting skin in the game not just your reputation but also money but um I'm not sure how how that will evolve over time cool thank you so much gentlemen for your for your very extensive for your very extensive answers let's give him a round of applause thank you so much [Applause] [Music]