2 reasons why your project is getting hacked - Oliver Hörr | Hats Finance

thank you um so I already told you the other guys if you have any questions or the talksporting just shout in like I'm happy to have a discussion as well we're not that many people um quick show of hands who of you is a security researcher or auditor one two three nice four people who is a builder who is like getting audits for his project one two all right okay so the provocative title of this is two reasons why your project is um keep getting hacked but what I want to talk about is there's like some easy incentive problems in web3 security and I I think we have the tools in web3 to to solve them so the first problem is really the auditing Market because when you are a young project the auditing market usually looks like this to you I don't know if this like feels familiar for you but like you know like some of the big names like trail of bits open Zeppelin some no depth Arc which is um the auditor of uni swap which is they don't even have a real website but they're like super famous and so you know that these guys are super good um but then you ask for an auditing slot and they say like yeah come back in four months and by the way bring 200k but the problem is that most projects cannot afford to pay 200k three times a year so it's it's a really bad situation so and and then below there's a tier 2 and tier three and in those tiers there's some really really good Auditors but some really really bad Auditors as well and so I'm who's like hanging out on Twitter hanging out and telegram all the time I by know now like okay this auditor has audited this project they got hacked afterwards and you know like the audit pod was a [ __ ] show but like even the builders most of the time don't know so you can either try to research it a little bit and gamble but it's really hard to research this because even if you look on like which projects got hacked and who got they audited by if the auditor is not on that list it doesn't mean it's a good auditor yeah so um and so the third thing is um you are lucky somehow and you know someone who uh knows good Auditors and he recommends you someone and you get an auditor at a reasonable price that it's really good so um what we ask ourselves is like okay what is the reason and how could we solve this and um why is it so hard to like find the auditor as a good fit and so the first problem is there's no risk sharing so if an auditor is auditing you and you're getting hacked the next day he will come to you and we'll say like well sorry for your loss but by the way we still like got already paid so you know like it's not our problem maybe there is some reputational damages but as we see every day that is not really working well like there's some Auditors in the market where I think like how the [ __ ] are they still getting customers it's insane so there's reputational uh kind of like self-cleaning of the market does not work really well um because there's apparently no reputational damage and yeah it's hard to verify the track record basically to check if an audit is good you would need to audit yourself and then prepare like compare it to audit reports or you would need to Source two Audits and then compare them which you know like usually you get one or two audits or even three and then you know it afterwards yeah like after you have done through audit cycle you know who you want to work with or not next time but yeah for a young project that's a problem then we believe there's like two solutions to this uh one which is um coming up recently and we think it's a beautiful system we call it skin in the uh skin in the game auditing so it's you you go to an auditor you source and audit but the order to take part of his fees and puts it into a bug Bounty for a certain period of time what does it mean that means that if someone if they miss a vulnerability and someone comes and claims it later they will as well lose money on like the auditing job so they basically share the risk with you and they have to prove that they're confident in their work so if they offer that to you like hey I'm less confident in my work I will like stake part of my my fees into your back Bounty you have a very higher level of trust to them and um yeah you you have as well the incentive of the auditor to stay engaged for the period they have locked in their money because even if something changes yeah like your project maybe as well leverage other technology and and let's say you use an Oracle and that Oracle changes their code the auditor suddenly has an incentive to come to you and say like hey guys there's something changing here please check it out what it means for you and we think that this is basically how auditing should work yeah like it should not be like this one-off thing where you do an audit and then buy and talk to you again in like a year when you need the next audit but it should be like a trust relationship the the second model and that is what we're doing personally and has Finance it's audit challenges or other competitions uh we personally run them in what we call the Cutthroat model which means you you make an open call to Auditors to come and look into your code before you launch it and anyone can compete so for us there's like not even like a signing up or you just need to have a telegram handle and you need to have a wallet address and then you can send in uh audit reports and pocs and um what we see is that regularly there's like more than 100 Auditors competing in these other competitions so in our last we had like 270 Auditors that competed and um even though quality of our quantity when it comes to Auditors but what we have seen is that um if you structure the incentives right you attract the right people as well so if you say like okay only the first one who find to find a severity a vulnerability will get the reward and you don't have to share it afterwards then suddenly like these audit firms come in and a day as well compete and experts that are specialized exactly on your product so if you're for example a bridge and there's an audio competition for a bridge then people that really familiar with that will come in and compete and so if you structured right there's no upfront cost for the project and um good Auditors earn a lot of money and the bad Auditors walk away then like empty-handed and I think this is really good because there's a lot of Auditors out there that are really good at writing beautiful PDF files I don't know if you had that yourself right just like the audit is complete you get this PDF and it's like looks amazing but then you look into it there's like one medium severity found and like three low or something like that and I believe that this is not really what you want to get an audit for right you don't like we had an auditor who would ping Us in Telegram and say like hey in line 27 in in this repo I found an issue and this is the issue this is how it works and he didn't even compile an audit report in the end and this was like the best thing so um we personally believe that it's more important to like get really the issues cleared out than like getting like a nasty compiled list the second big problem that we see in the auditing Market is it's actually pretty hard to be ethical as a hacker so in in 21 3.4 billion dollars were stolen in the whole D5 Market in 22 actually what's even higher is was um depending on the data source it was 3.7 or 3.8 billion dollars which is amazing if you consider that all the tvl went down by a lot a hex still went up so we actually got less secure from 21 to 22. um 23 let's see doesn't look that bad yet but let's see what happens and only 10 million dollars of back bounties were paid out so it's there's like a really really big gap and so there needs to be some kind of problem right so apparently disclosing vulnerability is is way less attractive than yeah stealing money and the problem here is the the web tool like uh process that we have in web3 because in web3 yeah like all the funds are on chain um there's like offshore exchanges where you can learn a launder a stolen money and so it's I mean it's not easy but it's not super hard either to like if you steal money to wash it and so what it typically how it typically works is you have a white attacker and he requests a back Bounty from from the Builder so he has information on the vulnerability and the team has money they put up either on the website or we do it on chain and say like hey if you want to give me um the information instead of exploiting my project and stealing my money I'm going to reward you but for that to happen the builder needs to have a look at the vulnerability because otherwise you cannot evaluate if the information is actually legit if it's a real vulnerability or just a Spam report and suddenly the Builder realizes wait a minute now I have all the information I need I still have my money so what happens very often is ciao the Builder becomes a ghost and this is happening like on the regular like uh hacker one did an uh um survey and 50 of the security researchers said they they found a vulnerability and they didn't disclose it because the project was hard to work with unresponsive there was no back bounty in place and uh yeah or they just simply ghosted them and then next time they said like well I'm not doing that [ __ ] again so this is like actually so easy to solve especially in web3 um that um you just need like a vault or like an escrow right that is holding the money so when the the white attacker requests a bounty from the Builder the Builder has to put the money into the Vault before and if the Builder is then looking into the vulnerability information they have the information then but the money is already in the world so if the Builder trying to become a ghost what we can do now is we can say like there can be like some kind of arbitration service in place right so we have a decentral court that will look at what has been reported by the hacker what has been the Bounty program and what has been the vulnerability and if they decide like wait this was a real one ability the watch attacker should be rewarded then they will make their ruling and the white attacker will get their money and would be very happy so maybe maybe you're building right now and you think like okay but why why is this important to me I'm giving up control and uh I don't want to give up control yeah like what what is in for me so um actually it's important that you get really like a level of security out of your back Bounty so we need to increase the effectiveness of those back bounties and we need to attract more security researchers to hunt bugs there are some uh groups out there that will never go down the ethical route I think you all know him like these North Korean State attackers you know like that are supposedly behind like this really big bridge hacks and what we have to do is like this is like a war between like black hat hackers and white attackers and we need to make sure that we have on the white attacker side like the best people as well so that the people that never will go down and get rude uh can be like defended off and so if we make the disclosure process more attractive um so they are not like okay I can either steal 100 million dollars right now or I have a 50 50 chance for like 200 000 but it's at least you know like either 100 million or 100 trends of like 200 000 then we already push the needle and make more people this loss and in the end it may shock you but a lot of these security researchers actually are coming from like top universities in the world like it's the meme that they're like some 14 year old kid in in some basement might be true sometimes but very often they just come from Top on universities and they want to have an ethical career yeah they don't necessarily want to live in the shadows and if we give them like a way to like earn really good money then um many many will choose that way and we'll do that yeah so this is my talk it was a rather short talk and we can have some discussion now as well I hope it was interesting if you want to check out what we are doing you can scan that QR code and yeah happy to like take questions or if you want to share your experience and like selecting Auditors or whatever I like super happy to talk about it [Applause] yeah I guess my question is um so there's a need we clearly see the need from projects to show the proof of attestation the proof of stem so there was an audit but is there really the uh push or the need for that audit to be qualified like you know quality high quality for the project because uh in the end if the project gets rugged or you know is exploited what is the real um what's the real outcome for the developers is there really um you know something happening so there's no there's no um there's no need for them to do and spend more money on high quality audits because all they need is just to attract the users and the users are attracted by the vision of the security right rather than the actual security so what's what's your take on that how can we solve that so it depends a little bit what you want to achieve as a team um so if you say like okay I launched this as an honor if it like gets hacked and I lose all that money I just gonna spin up the next project I don't give a damn I think then then it's problematic and I think that we really need to call out Auditors that do that like yesterday again someone told me like hey I can get audit from this project from this auditor that has a high name for five thousand dollars and they will just give me the stamp of approval and I'm like bro this cannot be like a real audit like no way and so I think in that case we need to uh um like make pressure on the Auditors to do like real jobs uh real work by the way the auditor I have to live up to my own Center the auditor static just if you were wondering yeah so I'm not a big fan of what they do right now um then um but if yeah if it's a project this is like like builders that want to build a real product that are successful in the long run maybe they are docs or at least they build up an unknown reputation that they want to keep in the long run then I think we need to Aid to some education right like if you would be like some manufacturer of cars right you would buy insurance on your factory because you understand very well that if that factory burns down and it's not ensured your business is dead there's no way of coming back from that and in in defy it's the same thing yeah if you are getting hacked and you lose your user funds it's super super hard to coming back from that yeah like Euler team might do that because they actually got like almost all of the money back and make the users whole for but still for them it's an uphill battle but I know many many projects that didn't get the money back from the hacker because the hacker was actually the Euler hacker seemed like very unsophisticated to be honest yeah he acted in a very strange way maybe he's very young or something like that but I know many projects that didn't come back like from the heck at all so the project is that and I think that is what people have to start to understand um is that yeah like this is this happens once your business is dead any more questions comments ideas uh so I have a question regarding the first problem the second solution when you told that uh you should do a sort of an open quality for Auditors to audit the project uh do you mean that also audit companies should take part in this qualis if you mean only independent Auditors what's the difference with bug Bounty that's the first question and the second is don't you think that the problem is and the feature in audit is not to find as many critical issues but not to miss the one that leads to a complete fund drainage and I don't see why audit companies should join the open quality for example there are huge projects with good code quality and why should they take Parts in the participation if there are no any bugs presented but they will spend time thanks um okay I try wait so I the question regarding um so first of all this other competition model we believe that it should be happen after a real audit like an audit firm audit because we still believe that it's super valuable to have this design partner that will help you make architectural choices will flush out um the uh like the common issues yeah um actually uh the model the competition model works best if actually has less issues still in the code um because then each individual payout is higher like you you have a pool of money and you share it between like the successful submissions and so if there's like um like a 50k medium severity budget and uh there's like 100 submissions for medium then it's less money than if there's 10.

um so and in terms of if if you believe that like the the what is the value of like security firms or freelancer audits it's that you only pay for like actual audit findings and um for you it doesn't really matter who who that is right I mean if you're sending in a POC and it's like hey this is how I can steal your money um then uh yeah you're just happy that happened and um you don't really care sorry I tried to remember all your questions I think one question if the code is really mature right is it worth my time to look into it um so you have to understand that human Auditors are just humans right like we we just finished an audio competition which was uh audited by trail of bids like arguably like the best Auditor in the market right now and we had three high severities that got found at trail of bits missed and every single of our like no one not everything but one other competition were done after an audit and every single time we found issues like there was one competition where you didn't find a high only medium and low but even in the others every time a high so I think we have to accept that one audit is usually not enough and um yeah I mean if if there's a competition and there's no issues back in the code uh okay that happened yeah then the security researcher didn't get paid for it for their time that's that's the risk they have in the competition model I think we have to openly say that but um most of the time they they are getting rewarded sorry what was the last question uh I think you answered everything okay I wanted to hear thank you uh one thing um there's um so there right now there's an incentive for projects to go into audits with shitty code because most of the time it's you know like costs them the same if they go into the audit with shitty code or with good code quality and I think this is as well an incentive we have to battle right Builders should have an incentive to have always the highest code quality even before going into audit because then the audit is more effective and this is as well like what you can still like in the competition model you can like design the game mechanics very carefully like what is the kind of behavior you want to incentivize from the security researcher to act and so for example um if you cap the maximum payout and there's like only one high severity then the project takes back the budget that is not used so actually the project now has an incentive to come in with as less severities as possible vulnerabilities as possible because then the competition costs the least money thank you more questions comments you can tell me as well if the talk sucked then you know like take the mic and tell me I know up there no okay all right thank you so much Oliver [Applause]