# Polygon | CvH - Effective Product Security: Lessons from bug bounties and audits | ETHDam 2024

- Channel: [CryptoCanal](https://streameth.org/cryptocanal)
- Date: 2024-10-07
- Duration: 23:56
- Watch: https://streameth.org/watch/yt-MkZ0cx4Wwj8
- YouTube: https://www.youtube.com/watch?v=MkZ0cx4Wwj8

## Description

Join CvH from Polygon for a talk about “Effective Product Security: Lessons from bug bounties and audits” recorded during ETHDam 2024. CvH will tell you more on how to manage security for your products and applications in a cost-effective way, and avoid making the mistakes newcomers do.
https://twitter.com/cvhessert https://twitter.com/0xPolygon https://polygon.technology/ 

James Campbell - MC of ETHDam, Hackathon Organiser, and Web3 Developer.

ETHDam - a conference and hackathon held in the heart of Amsterdam, Netherlands from April 12th to 14th, 2024, celebrated its second edition, gathering more than 600 participants. 

In the dynamic space of ETHDam, privacy and security took center stage, featuring groundbreaking discussions on hacks, recovery, and the revolutionary work of figures like Pertsev. Privacy is dead in crypto, people that know, know. People who don’t know, should know. 
ETHDam is powered by CryptoCanal, an education and events platform growing in Amsterdam, spreading its roots to Rotterdam and Zürich.
Keep up with us to see updates on future events: https://www.cryptocanal.org/ 
Follow CryptoCanal on X: https://twitter.com/CryptoCanal
Join CryptoCanal TG Community: https://t.me/CryptoCanalCommunity 
Join CryptoCanal Discord: https://discord.com/invite/XJVjpCqQBz

We would like to thank our partners that made this event possible. 🌷
Battleship Partner 
🛳Oasis Network https://oasisprotocol.org/

Jet Ski Partner
🛩⛷  NEAR https://near.org/

Canoe Partners
🛶WAKU https://waku.org/
🛶Trail of Bits https://www.trailofbits.com/
🛶Avalanche https://www.avax.network/
🛶Privacy + Scaling Explorations https://pse.dev/en
🛶Threshold https://threshold.network/

Our Canoe Partner & Official Node Provider
🛶dRPC https://drpc.org/

Sponsor
🤝EF Ecosystem Support Program https://esp.ethereum.foundation/

Paddle Partners
🚣ChainSecurity https://chainsecurity.com/
🚣Lido https://lido.fi/
🚣Cyber Capital https://www.cyber.capital/
🚣Diva https://www.divastaking.net/
🚣Firn Protocol https://firn.cash/
🚣Beefy https://beefy.com/
🚣0xbow https://www.0xbow.io/
🚣Obscura https://obscura.build/
🚣Panther https://www.pantherprotocol.io/
🚣Maven 11 https://www.maven11.com/
🚣Zama https://www.zama.ai/
🚣zkSync https://zksync.io/
🚣Secret Network https://scrt.network/

ETHDam AfterParty Fren
🥳Bitvavo https://bitvavo.com/en

Chapters:
00:00:00 - Security: Bug Bounties and Competitions
00:02:09 - Bug Bounty Programs and Audits at Polygon
00:04:22 - Bug Bounty Programs: Getting Started
00:06:16 - Hacker Incentives and Relationship Building
00:08:08 - Properly Setting Up the Commit
00:10:05 - The Limitations of Exploiting a Protected API
00:12:08 - Setting Clear Payment Ranges and Triage Management
00:14:07 - Urgent System Upgrade Required
00:15:58 - Budget and Scope in Audits
00:17:51 - Free Security Measures
00:19:44 - Simple Basic Security Checklist

## Transcript

[Music] all right I'd like to welcome Christopher from polygon who's talking about effective product security cool hi guys so I'm Christopher I work for polygon I'm a VP of security over there and uh basically my talk is about bashing audits bug bounties and uh yeah competitions and stuff like that I've done security for a very long time um you know before polygon I've did a lot of cloud software as a service type of things and also doing security pen testing I've done almost every normal security job you can imagine so um yeah I'm going to share some uh lessons hopefully um just to check here who you know who runs a bug Bounty program for the program any hackers that actually report stuff into it cool perfect uh anyway no so let's go into it so first of all uh you know I want to make sure that everybody understands that security in an application you know anything that you develop is not a checkbox it's actually a process no all that are literally a checkbox you want your product you want whatever is that you're developing you know either it is a client it is a smart contract or any type of product uh you want to make sure that um you know you do this over time because security is brought over time and it decreases over time and with work and that is is especially true in smart contracts now you've probably all interacted with a lot of the no typical um you know protocols that have been for ages maker Unis Swap and stuff like that those smart contracts have been there already for years and those years the amount of you know Buck bounties that they receive the amount of audits that they receive the amount of reviews that they've done made that product more secure and even more secure because if by now you know those smart contracts like Unis swap B2 hasn't been hacked the probabilities of it being hacked in the future are a little bit lower no let's not talk about AI or you know Quantum Computing you know that can change a little bit things but in general you know this is true for almost everything so um this is what the process looks at polygon no and and basically any company that has a security team that has a secure software development life cycle that's you know their Chron for it sdlc uh I'm not going to go through all of this no it's a pain in the ass and if you guys want actually you know learn a little bit more about how to do software you know how to deliver software in the most secure way possible you know you take a picture or reach out to me later I'm happy to help you out with that but uh you know it's it's a it's it's a long process it's it's difficult it's annoying you know and and basically where do buck bounties audits competitions fall into just here external assessments no that's kind of after the development and then the buck Bounty is kind of when you go live no it's just like one thing out of I don't know 20 that are over here and a lot more that are not mentioned in in this slide so um just to you know give a little bit of an introduction now so we at at polygon we handle we have four bug Bounty programs open right now no immun ify hack and proof hack one remedy um you know people also submit stuff directly to us at security at polygon technology um we have been the number one protocol that has paid out money in Buck bounties so I want you to remember these numbers so we've paid out almost 6 million so far in Buck bounties since 2001 and we spend 2 million on audits this is specifically for kind of like the POS chain now that I guess a lot of you guys know for ckvm we already spent 750,000 and it's been only live for basically a year and we spent also 2 million on audits so if you sum up no basically we've polygon since 2001 2002 has spent more than 4 million in Audits and has spent and more than sorry on Tero but on MAF like around 7 million in B bounties yeah so look at that comparison so we spend a lot of money in Audits and we still got a bunch of stuff in in our buck Bounty Pro like even more than what we spend in audits no at the same time so I'm not really saying one is better than the other but just think about that for a while so you can spend all this money and still have a ton of bugs that that come out into your product so and just to give you a you know little view of you know how how you know where do we pay a lot of them were medium types no so criticals thank God we had very Littles but most of them were medium type of reports now that have been paid out um so the polygon sh is done no just so this is now me no so disclaimer these are my own opinions this is not what polygon thinks even though I'm the VP of security so kind of but no these are my personal opinions on on Auditors competitions bug bounties and stuff like that so um yeah first of all don't wait for a buck Bounty program and it doesn't have to be expensive like at all like you can just start with an email address and tell everybody you got an issue just report it put a you know web page or stuff like that use a Google form your J I mean all of these are free tools this is how this that's about bounty pram you don't have to pay any of the big companies or big protocols or you know big Services out there a commission or you know a monthly fee just to run your buck Bounty program you can run it yourself like security researchers in generally don't care they just want to be able to report that no and you get you don't have to pay in money like even if you don't have enough money you haven't got enough investment or everything else you can pay with tokens your own tokens in the end you know you're the one minting them you know you can mint billion dollars and or billion tokens tokens you know just me more and give them tokens give them swag you know and and or or even a public Kudos a lot of security researchers that that work are basically you know they do it because they like what they do no it's not only about the money yeah it's also because they want to make the space safe they want to make sure the protocols are safe because they use them as well know and stuff like that so the only thing just make that clear if you're not going to pay them make it clear hey we cannot pay you we don't have money you know when when you put out your BG your web page or stuff like that we cannot pay you but we're definitely going to send you some swag we're definitely going to make a amazing Kudos Twitter post for you and stuff like that if you report anything to us now and for hackers if you're a hacker and you know you see a protocol that basically is not pain or anything like that you may actually get a deal after that maybe you report something you don't get paid you get 10 bucks you get a pizza you know whatever okay yeah thank you a Kudos but after that I'm sure that that protocol is going to call you again and say like hey you actually reported something really cool and we liked what you did can you do this again we actually found some money you know we got an investor we mined a bunch of tokens that are worth nothing we can give you some of them no you you you probably you know build a good relationship with a protocol if you do so so um a couple of actual lessons and oh I hate that I did this so first of all scope your assets properly no I've I've made this mistake many times when I built a bu Bounty program and I've put like you know oh yeah here's my GitHub just you know submit anything about it or here's my web page and then you receive a bunch of crap like a bunch of crap just just anything broken links uh you know stuff that makes absolutely no sense somebody in marketing putting out a website for you know some campaign that has some issues like who cares you know that's not going to really affect too much the company I don't want to spend my time over there know my time is limited the engineers time is limited they they're better developing stuff fixing stuff deploying stuff than having to triage and deal with stupid you know uh um reports that are being sent so oh yeah so what you should do is basically properly set up what the commit is no make sure another mistake that I see many times is they you know and and this is horrible for a hacker now that reports that spend some time looking at your code discovered something only that after it reported it you say oh but that's not the commit that is production and so your report is not valid like that's horrible I hate when that happens it has happened many times to me because you know you got to tell like it's invalid that's not what is in production so make sure you tell them like what is the commit you know put like you know hey just always look at the latest release no if you guys are using releases for your code uh just just release that and um yeah and then if you're using apis or stuff like that put things like Swagger and um you know so that people know what to look at for your API so make sure that your scope that properly no then learn how to score a severity it's this is the very important I've just hate dealing with people that don't understand that everything they put is critical oh a broken link it's critical because because of that broken link you can trick users into like come on dude like a broken link is not you in generally it's not a critical finding now critical finding is you stole my money know so make sure that you understand how to to you know from both sides from a hacker and from the project that you understand how to score the severity normally it's very easy it's the impact and the likelihood so what's the impact did you steal money did you to put down my servers um you know whatever the impact that it is and then what is the probability for that happening is it easy to do can can anybody do it do do I have to does a protocol need to allow me to do that no so even though you may be able to steal all the money yeah and that's the impact the probability or likelihood of that happening is probably is can be impossible maybe the you know the the product that you need to exploit or the API that you need to exploit is behind a white list it's protected by an IP then like you cannot really do it in theory you can if you were in the white list or if you're part of the you know IP access control but on the other side so um I know it's horrible sometimes because they're like yeah I found something it's critical and stuff like that and then I got to go tell them yeah but you cannot really do do it in theory you can so you that would be a medium in instead of being a a critical or a high no but it's not going to end up being a critical a critical really means like you and anybody else on the Internet is going to be able to exploit that yeah and if you need help use a c CVS CVSs calculator this is a very the one that gitlab exposes you don't have to copy a link just gitlab CVSs they have a really good one that just it's 10 questions and it asks you like is a is a code public can anybody do this you know and do they need to have permission blah blah blah blah blah and it will give you a score at the end and kind of guide you like okay yeah it is a critical it is a medium it is a high and that is a great way to work and discuss with with with Auditors or with um uh uh with protocols no and then very quick Buck Bounty reports and this is another issue that I've seen in in competitions or in audits or in bu bounties the severity that you put in a bug Bounty report not always is the same as the severity that you see in a bug Bounty or other reports so they're not always the same because in other report you're not really testing the live environment the other report is just testing you know something that is on GitHub the code that you are potentially going to put live so everybody should be able to have access while in a buck Bounty it's things that are live and many times when they're live you have other controls you have mitigating controls and maybe be an IP wh list the web page needs a username or a password in order to be able to access it so that helps lower a little bit the severity so now a little bit of some other some other thoughts uh don't set up ranges I you know hate when when protocols set up ranges or hackers you know or or you know try to negotiate the ranges but if you do make sure that it's clear what does it mean that you're going to pay like polygon we pay a critical starts at 100k if I'm not mistaken or 50k and it can go all the way to 1 million so how do I get the 1 million well it's quite easy you just need to steal all the money now if you cannot steal all the money then we're going to pay you 50 but if you can steal all the money we're going to pay you a million you know obviously I'm simplifying it you know there's there's some other variables that you need to have but in general that should be the the the thought process and especially in web 3 where you know bounties and aits are so expensive and people are paying you know Millions for you know for different things like make sure that you make that clear yeah uh manage triage I know a lot of buck Bounty programs are offering that not a fan to be honest uh I think that if you guys are engineers and building a protocol you can probably triage that faster than anybody else uh that does it externally in any of those protocols but if you have enough money and you don't have the time nor the engineers to do it it may be worth it uh just make sure that you understand they have a conflict of interest no because these managed triers get a commission for every confirmed B Bounty so it isn't the best interest to actually you know make sure that you know it is valid um yeah don't forget to disclose the issues to other protocols no I've seen this happening more than once where okay you receive a b Bounty an auditor find the critical issue or stuff like that and because of the nature of Open Source other people may be using your code make sure if you know of course many times you don't know make sure that you actually tell the people that are using your your code um that hey we found the critical please upgrade soon before you know somebody goes public with this so if you can and if you know how please and if you know who sorry please disclose that uh before going live with it um yeah work with a hacker over this work with a project they're going to be able to normally help you uh you know disclose this and make sure that you're not disclosing it if you're a hacker or if you're a project disclose it after it's fixed it's obvious sometimes it's not and sometimes hackers pressure you know or rush into disclosing some of this stuff um yeah mediations I don't like mediations again conflict of interest you know why why am I going to trust you if you get a 10% commission that uh you know if it's valid anyway and don't just accept the severity you know that whatever if if they tell you it's a critical always challenge even if you know it's true challenge it you know goes for hackers and for protocols challenge make sure you you engage in the discussion why why do you think it is um on Audits and competitions I think this is the last slide because I'm uh almost on time or I have five minutes I think I don't know um audits are very subjective like uh yeah they're great they're useful but they're expensive and they don't give you all the guarantees of Securities to be honest no and you also need to understand what you're getting for your money is it a automated one you know is it done by AI is it done by a bot is it a manual how many people are behind it what are the qualifications of the people now you can get Audits and you can get Audits and you've all seen audits from certain companies that get hacked all the time and all it's from certained companies that basically are not really on you know Twitter F and all those type of things ask you know how they do it what they do it and make sure that you understand what the money is and make and be clear with your budget like if you have 2K you have 2K if you have 20K you have 20K if you got 100K you don't say you have 100K then they're going to steal all your money but like if if you have limited budget make sure you make that clear with your auditor or or whatever it is to um you know so that they also know what to provide you know and what to scope you if not they're all going to go oh yeah I'm going to put you 10 people and it's going to cost you 150,000 and yeah whatever um if you're an investor I guess everybody's investor here no into meme coins nowadays definitely so um you know obviously it's gambling to a certain degree and I don't expect you to do thorough due diligence and look at all the other reports and stuff like that but if you're going to put serious money into a protocol maybe not a mem Cy but like a proper protocol you know just and especially if it's new just re you know review the scope review the methodology if you don't know how to read that ask somebody or look online because many times people you know all pro projects come in like oh I got an all that it's all good invest in me and when you look at the other report it says oh yeah we audited that chair all the other chairs we didn't look at it just that one and that chair is is is good so we believe that the rest of the chairs are good for you to sit on and that's not really true just because that chair is good doesn't mean that the rest of the chairs are not are good so review the scope what methodology you use like just ask for do you do you guys have a checklist like what do you check when you do an audit um yeah what else um I think mentioned this like audits are a point in time assessment and most likely when they did the audit by the time that they deploy it the code has completely changed so again audits are useless not really but you know kind of to a certain degree so web 3 moves very fast changes happen all the time you cannot expect companies to actually pay for every little change that they do no so um yeah audits are good but and necessary but they're not the only thing as I demonstrated at the beginning there's a lot of other things you can do for free to you know provide much better security than just an audit so if the only thing you can do is an audit then you know maybe don't do an audit hire a consultant to look at your development process um oh yeah don't get an audit again no um get a security review of your development process um what else hire uh hire an engineer instead like if you just got an investment and you're like okay I'm going to put a buck Bounty for or an AIT competition for a million dollars you know we're seeing that a lot lately where here a million dollars competition and stuff like that like I hope you're also spending a million dollars on your you know security team and your development process and everything else but I can assure you it's not the case they just threw a million dollars to you know to make some buzz and and then when you look at behind like there's no processes there's no sdlc there's no security scanning you know they they haven't even checked their you know they host on name chap still you know like what anyway and and the security guy will know how to hire also a security company like if you if you're you know an engineer or even if you're in the business side of things and you need to hire a security company just ask an auditor ask a security person to help you decide what is a good security company um audit is not the same as a pentest I think I mentioned that a couple of uh hours ago but like it doesn't check your website it doesn't check your DNS they don't check your GitHub repo if you know if it's good like there's so many other ways you're going to get hacked most likely that is not your smart contract that is audited no most likely it's going to be a fishing email most likely it's going to be a DNS change because you're a name cheap and uh last over here uh about competitions uh I like competitions no it gives you a very broad view of um you know of different people looking at your code and stuff like that but isn't just a competition at time box B Bounty like why am I going to pay a you know somebody else to do what I can do with an email address and just like here's 10 $10,000 go at it people will ape people you know security guys will go and you know hit your stuff regardless um was that it yeah that was it so um you want this presentation you got a QR over there also I just recently made a very small guide I advise a lot of little companies that don't know how to do security don't have security people so I made like a dummy version checklist like basic stuff check your email check your Google check your GitHub check all those type of things so if you want to know how to do very very simple basic security that will cover kind of like 80% of uh of what you should be doing then um you know look over there or find me on Twitter or foraster or anything else so I think we're done no right on time much one question maybe do we have yeah don't have any questions yeah no yeah it's always best I'm assuming so it's always best to engage with your Auditors even in like the the planning process right but um say you can't do that when's like when do you see the optimal window is to get the security Auditors looking at processes and code and so ideally what I like that Auditors are starting to do is just after you fin you done code complete and you're starting to test your things so then you can probably most like many Auditors actually offer for free they're like let me review your code just basically I'll spend an hour or two I'll do it for free just to make sure that you've done the basics and then once you're done finishing your testing uh no I already they're already going to give you some feedback on the testing and then once you're done the testing you're going to get that AIT honestly AIT is the last thing you kind of want to do I'm not going to go through all of this again sorry um all that is basically the last thing you want to do and it's really really a checkbox it's that's that's all how many audits don't get an audit spend that money on a security researcher that that works with you throughout the whole process like and all it is a check like I'm it does provide you some guarantees but in the end it's kind of a marketing thing as well it's just [Music] like weed I don't know we got time but possibly a stupid question but but audits in in the sense that someone reviews a piece of code uh is one thing but I'm also thinking audits could be someone reviewing uslc correct right and reviewing your process is more like a onetime or yearly thing yeah but that's not an all it what's that called then so that's that I mentioned that two hours ago a know it at least in traditional security is a check against a standard you know let's see if you comply to ISO 2701 let's see if you comply to you know this standard that's an audit we use this term horribly wrong in web 3 that what what you're asking is an assessment a Security review um yeah it's not an audit audit is really check the box oh you comply to it like you can do an audit if you comply to IR erc20 is your token irc2 compliant audit it it is compliant that's it but we're using the term wrong but yeah is an assessment then worth yeah I think an assessment is is the best um yeah is the best word to use it but we're not going to change you know people are going to call it whatever they want but is that worth it then if audit is that's why I I would prefer that auditing companies are actually reviewing your development process than actually doing the check and you should probably ask that to your auditor they're going to they're going to be even more happy because they know they're going to be able to upsell you later nice thank you very much [Applause] [Music]