# Designing Awesome Developer APIs for Protocols by Nikil Viswanathan (Devcon5)

- Channel: [Devcon](https://streameth.org/devcon)
- Date: 2020-10-09
- Duration: 20:37
- Topics: blockchain, smart contracts, distributed ledger, ethereum, Science & Technology
- Watch: https://streameth.org/watch/yt-AUFGcgJbAuI
- YouTube: https://www.youtube.com/watch?v=AUFGcgJbAuI

## Description

Visit the https://archive.devcon.org/ to gain access to the entire library of Devcon talks with the ease of filtering, playlists, personalized suggestions, decentralized access on IPFS and more.
https://archive.devcon.org/archive/watch/5/designing-awesome-developer-apis-for-protocols

The growth of your protocol ecosystem is rooted in the ease at which developers can build applications using your tools. How do you create intuitive, powerful developer experiences to encourage people to build a vibrant application ecosystem?We share the behind the scenes of what really goes into powering marquee developer communities. In this talk, we cover the design and implementation of custom APIs for some of the most popular applications: CheezeWizards by CryptoKitties, Kyber, and more.

Speaker(s): Nikil Viswanathan
Skill level: Intermediate
Track: Developer Infrastructure
Keywords: APIs, protocols, technical

Follow us: https://twitter.com/efdevcon, https://twitter.com/ethereum
Learn more about devcon: https://www.devcon.org/
Learn more about ethereum: https://ethereum.org/ 

Devcon is the Ethereum conference for developers, researchers, thinkers, and makers. 
Devcon 5 was held in Osaka, Japan on Oct 8 - 11, 2019.
Devcon is organized and presented by the Ethereum Foundation, with the support of our sponsors. To find out more, please visit https://ethereum.foundation/

## Transcript

okay I'm Nikhil I'm the CEO of alchemy hopefully you're here for this talk if not you should just stay cuz it's gonna be hopefully good we're gonna talk about designing protocols for api's today and it's gonna go into a couple details about aetherium and some of the projects that we've done so quick roadmap talk about the background a little bit about what we do why developer API sort of been important talk about how to enhance aetherium and then go into cheese wizards and a few design principles so a little bit a brief background of me we've been pretty quiet as a company so just want to share so my background is actually an artificial intelligence that Stanford did a bunch of research there and previously my co-founder Joe and I did consumer apps and one of the apps we built kind of for fun ended up being the gnome own app on the App Store so we did that for many years and then we started doing blockchain stuff so what alchemy does is we power infrastructure and developer tools for a bunch of different people across the space so we've we've kind of been in pseudo stealth but we're about to do kind of a big announcement and our goal as a company is we want to push blockchain development forward by we don't push the whole ecosystem forward by really really making it easy for people to build applications because ultimately what we think and what we believe is in order for blockchain to reach his full potential we really need to have users using real-world applications and getting value from that and what you need to do that is you need to make it really really easy for developers to create applications so our whole focus as a company is doing that and feel free to come by and ask me any questions afterwards about this so what is a developer API you and I know this sounds really basic but I just want to go back to the core of what it means to create an API for developers to build products right and ultimately what it means is it's just a way for programs to access data right and I think the thing that we need to really stress on as community is the goal is to make development easy right we need to make it as simple as possible and that should be our whole focus and today right now you know as aetherium is evolved people are building really incredible applications and those applications need more power than a theory was initially designed for and we'll go into a couple examples of what that means but basically and then actually the reason that this is so important is I alluded to the earlier but what happens is we actually want to have more people using real-world applications if we want billions of people to use blockchain when you have real concrete applications that provide tangible value to them in their daily lives in order to do that we need to make developers lives much easier so we can have more applications so kind of that is our whole focus as a company so to give you many people are developers in here ok ok I didn't expect everyone to be so this is probably a little a little obvious but the reason that good told people actually often underestimate how much good tooling matters and it's it matters a lot not because it helps you build products faster which is what it does but it actually lets you create new types of products you weren't able to before right so let's say you were baking your baking cake you have oven you have timer and measuring cup you can make like a really cool cake right but the challenge is imagine if you just had fire no timer you just like measuring stuff by hand you're kind of guessing on your time kind of like me right now seeing how much time I have left so you're gonna end up with something like this right so we really want to make it easy easy easy to create new products and also enable new products to happen and that's why developer api's are really important so let's go through an example about aetherium so right now one a very common use case for a lot of people's applications wallets games is you want to list all the transactions that an address has right and and this is actually very common we see this among a lot of our customers but it's actually very difficult to do in order to do this you need to go block by block and inspect every single transaction like let's say I had coinbase wallet right and I want to show my transaction history coinbase has to go block by block look through every single transaction in that they're in blockchain and see which one is relevant to mine so out of the millions and millions maybe I'd had 10 transactions right so you can imagine so it's not feasible to do and you actually can't build an application like that it would take like an hour to load to applicate the app so one of the things that we do is provide an enhanced developer API and what that does is it backs it indexes and stores the blockchain with a lot of infrastructure so then for the user experience it just turns into one API calls so transact get transaction history by address right and then you just get all your results immediately what this does is provides a building block so now as a developer you don't need to go build all of that from scratch so I give this example to show you this is almost like a building block so now when you look at an application like cheese wizards then you take these same concepts and apply it to making that development experience even better and you end up something really great so how many of you have heard of cheese wizards raise your hand okay cool so cheese wizards is a new game by crypto kitties how many people heard of crypto kitties okay so basically cryptic kitties is one of those popular games on aetherium about a year and half ago they came out with a new game and they came to us and said hey we really want a really really cool developer experience and a developer community and the reason for this and this is one of the cool things about blockchain right is since it's open people can build any kind of application they want with the crypto with the cheese wizards data so the goal in order to create a really flourishing community is you need to make development as easy as possible like we talked about before both on the infrastructure side and on the tool side and on the API side and back to kind of how we were talking about aetherium transaction history a lot of these things were not feasible to do if you're doing it on the standard aetherium API so if you're just trying to load all the cheese wizards and we'll go through a couple examples today it's actually you just can't do it unless you build your own database and infrastructure and all these things so that's kind of the motivation behind why they wanted to create a really great developer API so really quick highlight of what the game is what it does is it's kind of like a rock-paper-scissors game guys haven't played it definitely check it out and you have these wizards and they fight tools and basically it lets you you have these moves like wind fire and water and you fight each other it's just like tournament and you go up so and I'm just showing this so you can get a little bit understanding when we go through the examples so how do we think about the API so the first part is we mourn to be just really simple like when you look at it it's almost like deceptively simple the user experience right and that was on purpose and it's actually one thing is you guys know it's actually more difficult to design simple user experiences then to design complicated ones because you have to put in a lot of thought and a lot of infrastructure on the backend the other thing we wanted to do is have a powerful querying system so you can actually get the data you want so you can build applications really easily right and the third thing is we wanted to make get really easy so you'll have to go spin up your own node like if you're building a website you don't have to like spin up your own server and do all this awfully you just open your text editor and start typing so for blockchain we did for cheese moszer's we didn't want you to have to spin up a new ec2 instance and install a node and sync it from scratch and then do all this extra stuff so we're like we just wanna make it super simple web API so you can just prototype and build really quickly so under the hood what we had to do is actually build a ton of complicated indexing infrastructure that allows us to serve and query this data very very easily so let's go through a couple examples of what that looks like so you can think of it let's say we're building a leaderboard so we want to show all the cheese wizards and who's the most powerful one and who has been winning these fights which are called duels so this is what it would look like without the API and this is actually so this is actually a shortened version of the code we're writing this last night and we're magicians like actually I had to cut out like 2/3 of the code because it wouldn't fit on one screenshot but you get you get the idea right so don't try to run this so it won't actually work but basically like this you would have to write this much code just to get all the duels and you know that's actually not the hard part because you can write this code right but the thing is it takes like 10 minutes to load a page because again this is going through every block and every transaction in all the logs and figuring all that stuff out right so it's not feasible each query will take like about a minute so you need many queries on your page it's gonna take a long time so you can't build and no users gonna click a button and then wait 10 minutes for something to happen right so with backing with infrastructure you end up with something like this you have one API call that just immediately gets you back all the data that you need so in in one line of code and most importantly the application loads instantly so that's kind of one example and this is an example where the smart contract and this is a distinction I'm gonna make between this example in the next example is the smart contract actually doesn't have any methods for getting duels for wizard you actually have to go explore all the logs and the events and all that stuff so this is not built into the smart contract right so now you say ok what if it is built into the smart contract what does that look like alright so here's here's the API call that actually is built into the smart contract there's we want to do let's say we want to get all the wizards owned by like Joe Wright and we want to see is he does he have any cool ones Donna so the smart contract actually has a method that's called get wizard by ID but the challenge is we don't want it we don't know the idea of the wizard we want joe's wizards right and that small tweak in the smart contract is makes it so that you actually can't get the data you need so the challenge here is that in order there's no way the smart contract can have all of the methods to any kind of way that people want to build applications so if it's not exactly what you want you still end up with something like this right this this actually fit into one screenshot so we left it like this I think we cut out a few variable definitions and stuff but it's essentially like a bunch of code you have to write it's really complicated and most importantly you know it's the same thing the smart contracts method exists but it's not exactly what we need so you still have to go through all the logs and events and it takes about ten and a half minutes to load a page so now with the API you end up with something like this one line and it's loads instantly right so that's that's kind of a couple examples of why it's important so hopefully you can see that this is a really really huge difference in developer experience not only is it easier and you write less code but it's also enables you to do things that you couldn't before unless you're gonna go sit there and say I wouldn't build a cheese was a game but first let me spend like a week building out all the infrastructure to do all this right so let's go finally want to go over a couple principles API design so we we now power nobody can say publicly we now power billions of billions queries for hundreds companies around the world so a few things that we've learned from talking to hundreds of companies so the first thing is it's really important to understand your goals and this sounds very obvious but it's actually not and the reason is you kind of sit there and think okay cool like yeah we want developers use this let's just like put something together but I think there's actually like a very big trade-off around how easy is it to use how fast is it for developers how much time do you want to invest you know are you gonna be building out the full end-to-end experience and there's a lot of trade-offs here and really clearly defining your goals is really really important than that again something that as an engineer when I look at stuff like this you know I'm like ah this is obvious but then when you actually sit down and do it this is one of the really most important things know what you want the second thing in again something that sounds really obvious but may a huge huge huge difference really like talk to your users and understand what they want right you think that you know but you know you might know but you probably don't know all of it right so one of the examples was when we built this consumer app is called down to lunch we actually put wheat so it kind of start off this thing where we thought no one would use it and we put our phone number in the app and anybody could text us right we thought it's just gonna be our two friends ended up being Millie's people around the world we started getting 10,000 text messages a day literally broke the iMessage infrastructure I had to email Tim Cook the CEO of Apple and be like we were almost neighbors you have to help me I spent three days at Apple Store and they didn't help me right and the the result of this is like we were actually really really really in touch with our users so right now I see some of our customers are here you know we have a we have a telegram chat with every single of our customers real-time and you know it's super valuable for them but the most important thing is it's really valuable for us for getting feedback on what's good and what's not and how we can improve it so really you know I can't stress this enough the third thing is keep it simple one thing that we see in a lot of developer tools an API designs and documentation so that's really complicated like and you have to remember building if you let's say you're building cheese wizards when you have these developers coming to use your platform that is not their main objective in life their main objective in life is not how do I build the best cheese was there an application right there coming this thing they have 30 minutes you know free between before the game of Thrones episode and they just want to make it see if they can build something cool if you can't capture their attention and make it really easy to do you're gonna lose them and this is something that a bigger theme around building products for blockchain we need to approach it as a focus of hey we need to build we need to treat this as this is something we need to really engage people on and it's not something that deal we should just take for granted that they're gonna devote their whole life to blockchain the last thing is testing your API and this we this again all these things sounds super obvious but when you go to build an API people understand that this is really important so basically what we found is there a lot of edge cases especially with smart contracts and people querying stuff so having a tight feedback loop and testing a bunch of stuff before we've done this good and we've done this bad like sometimes you know there's been a couple times where we launch things we thought we had good coverage and people would have like a really crazy educate so I think this is this is really really important the other thing is we found oh my gosh creating good documents this is probably the highest leverage you can have on your own time because what happens if you don't create good documentation people ping you all the time of questions and super super super important not only for the developer experience but also to save your own time and then the last thing is our second glossing is getting feedback in the iterating when you ship something it's not done it's like you got to continually continually iterate on that and make it better and better and this is really what separates the good products from the great products right and the last thing I think to think about is really think about solving the users problem it's not just oh we built this API we're done it's like you need to like like literally have the whole experience covered right you need to like the best experience is like oh you know I don't have spin up a node I just have a Web API and then it's like oh you know I don't need to even like write my own HTTP wrappers I can just use like an SDK a JavaScript down to like oh you know the app just calls them an uber to get to their office in the morning like you need to make this like think about it end-to-end and this is what really makes a great user experience so in conclusion you know it is maybe not the key to life but it's but it is definitely the key to creating blockchain applications that people really use and ultimately like pushing for the success of our whole industry so I think we see this on every single this is like what we eat sleep breathe and live at alchemy so it's very it kind of ingrained in us but I want to emphasize how important it is for those you guys who are building developer products put a lot of thought and care and you'll get many many many times over reward in the dividends so thanks if I'll be around Zebedee have any quick questions great talk thanks happy birthday yeah yeah I have a question you said that you use the owner instead of tech G and there is some logic that you had to add do you process that but your team who builds the logic with the transformation like something like D graph protocol something like that yeah that that's a great question so the question was you know when we go back to this slide this one right so like it's slightly different than the smart contract and where does that data get processed and is that on our side is that on the user side so there's kind of two ways to handle this like one is we actually we actually provide a bunch of different transformations of the data so users can access in any way and then there's also there's we have plans for like a decentralized system but what we found is most of the time people just want the centralized version first and then you can make it decentralized so that can answer your question kind of in cheats in some of the things we do yes and cheese was particularly know there's like standard API and the reason for that is we basically thought through every single kind of use case that people would want and we provide like parameters so you can say for the API calls you can say query wizards based on fire type based on wind type or based on owner or based on duels so we basically think of everything you need and build a flexible system so yeah question great doc Thanks so I I saw that your API queries a lot of data yeah but do you also post transactions to the to the blockchain would you okay yes yes and do you use meta transactions or how do you do it what do you mean do we do we batch transactions together I mean as a developer I sign a transaction and send it through your API or others work yes so we don't we don't hold any private keys we don't so the question was how do we how do transactions work do we do transactions and how do we do transactions in the system the answer is we don't hold any private keys you kind of like the way to think about is you package this transaction and you hand it to us and we're just kind of like the pipe to the blockchain kind of like your internet provider it gives you internet we give you access to the watching so we put it in this little tube and that comes out on blockchain yeah so like this API that you build is it like you build a custom one for each one of your clients or is it like sort of an automated product where I give you my contracts and it sort of generates an API how does it work great question so the question was do we build one a custom one for every client or do we is it kind of like an automated one so typically we mostly focus on a theorem API so we build kind of a standard API and enhance so we support the standard JSON RPC API for theorem and then we build a suite of etherium api's this is actually the first API that we built for kind of a specific customer and typically we don't do these but short answer is this was custom but we do have kind of way to do automate if we wanted to but it's not kind of like a big area that we're focusing on okay so second question it's kind of it's not like a lot of the questions I just really want to try to understand so like pretty much every Deb if they want to have like good usability and good like um like fast responses what you do is you take your smart contracts you just listen for your own events you save them on your api's and then you mean decks all the stuff buy whatever you want then you expose your own API for your app is this the same thing you're doing or are you like taking a political a different approach yeah it's it's similar so I think for a theory I'm and I think this is probably like ask me afterwards I'm happy to go through it but basically what we do for aetherium is we basically take each piece of etherium data and it's not we just dump into a database we actually index and store it in a bunch of different ways so you can query it in really flexibly so for example the simple naive way of doing you just take all the data dump in a sequel database and you and you return it but you can only query certain things like that so we have like time series database we have caches we have Redis we have an emcee quote we have like all these different things that we use that enables you to query like really really flexibly so the short answer is like it's kind of like you know if you're bicycling is that the same thing as like a Ferrari it's like you know kind of but little different okay so in this case like to get the wizards by owner this was like your client that gave you like requirement fees I want to get them by the owner or it just like build something so generic that it also supports getting by the owner we it's great question we actually did the process I talked about earlier we sat down with our customer which sat down crypto kiddies and we brainstormed and we talked some developers and we said hey here's what we think it should do and it turns out in this case was actually pretty easy we did like three or four iterations and we're done because the game is a very small subset you can only do a few things you have wizards they fight and there's duels and that's about it and something like a theorem is much more complicated because you can query it in any way possible well basically took the approach they shared earlier thanks yeah all right cool I think we're times up but I'll be around I have stickers and I have answers some answers to some of the questions you'll probably ask but come find me thanks