Little-known Web3.py by Marc Garreau | Devcon Bogotá

foreign [Music] I think it's important to start with the why question so besides selling jpeg monkeys what are we all doing here I think we've reached a critical mass or we've gathered a critical mass of people here that I actually believe we stand a chance at re-architecting rethinking and even adopting some core level systems that we we rely on things that touch on uh our our governance systems our financial systems and some of our social coordination systems so how we how we work together um I think our window of opportunity is unknown so ultimately we should seize on that while we've got it the success of ethereum and decentralized Technologies is um or perhaps should be measured in terms of adoption and by adoption I think that at least a couple of the things that um move the needle most are number one that we're solving real problems and number two that we are providing powerful and accessible tools for people to solve real problems and my thesis here is uh that the latter comes first in in that it is unlikely that one dap will solve all the world's problems or one sphere of the world's problems right by putting powerful and accessible Tools in people's hands enables them to solve their problems locally and so that's where I think I can move the needle anyway so this is the sort of the framing for this talk my goal is to increase awareness of a set of python tools some best practices and some other projects that I think you might benefit from being aware of sort of the template for this talk is that I'll give you just enough information to know whether you want to learn more about it and point to somewhere where you can learn some more so this is a blog post I wrote about a year ago now that I think still relevant that digs a bit further into this call to action so this is meant to be beginner friendly sort of a series of lightning talks smushed together sound good my perspective is informed by five years at the ethereum foundation um thinking a lot about what it takes to get people to adopt this technology starting on the missed browser team where we produced one of the first uis for ethereum when we Sunset that the team moved over to do some UI experiments on configuring and running um ethereum clients call that ethereum grid and a couple years ago I pivoted to the snake Charmers team the ethereum python team where I write some code but also prioritize creating education content in the last year and a half my life has done some zigging and zagging my wife and I welcomed triplets into our family and then while I was on paternity leave I helped establish developer Dao which is a very reasonable thing to do I'm excited about developer Dow because it's it's sort of this hive mind of of bright and passionate people all sort of focused and digging into this this solving challenges of adoption I'm proud to be part of this ethereum ethereum Foundation python team we are a small team and we'll be most of us will be hanging out at an impact Booth hub for a lot of the day tomorrow so if you want to come say hey meet some of the team and get some very exclusive snakey mcnake face swag come find us there come hang out uh we just welcomed our fifth teammate uh this month Linda so hi Linda welcome to the Ruckus I'm going to take a moment to burn a couple seconds of time here that I don't really have to say that this is like a full circle moment for me um when I joined the EF in 2017 my first Devcon was uh in Cancun Devcon 3 and EV Fraga is in the audience who was on stage giving a shout out or like given the talk for the Miss browser team and he gave me a a welcome shout out so I am it brings me joy to make this a a newfound tradition quickly the EF python team manages or maintains about a dozen python libraries most notably web3pi and Pi VM the rest sort of roll up into those two libraries or help make those two libraries work a shout out to our open source community of contributors they also help things move along here and um shout out to our uh the snake Charmers who've come before us that kicked off essentially all of these libraries let's get to web3 pylon starting with the why question again why does web3pi need to exist and the same question is applicable to ethers JS web3.js and language you know libraries in other languages the short answer is because Json RPC exists or more specifically that the execution clients in ethereum speak Json RPC to if you want to communicate with them you have to speak their Json RPC standard effectively so in a command line if you want to make a an eth call request to an ethereum node might look like this where you've got a stringified uh Json object that you're passing along the ethereum client comes back to you with this um bit of this this blob of data which is then up to you to decode and make sense of web3pi and similar libraries give you like a human readable interface for this so you make the request for whatever it is you're looking for and ideally you get back exactly the data you're looking for for the Deep dive into what happens under the hood there I've got another blog post for you so what might you know web3pi for already um in order to communicate with an ethereum client you'll have to configure a provider this is [Music] this is your friendly reminder that the most secure way to do this is to run your own node on your own machine and connect to it via an IPC socket all know well that this is an unreasonable ask for most of the ethereum community so I'd like to plug here another project to keep an eye out on the portal network is there was a great talk given Yesterday by pipermerium about the portal Network what it is how it works what the progress is the idea here is that there is a fundamental redesign of the peer-to-peer protocols to actually allow light clients that are light enough to run in your consumer grade Hardware so your laptops your Raspberry Pi's even that's in Flight there are three clients that are being implemented and the spec is effectively done already so hopefully coming soon in the meantime use your HTTP or websockets providers as needed and once connected to your node you can do various things like look up account balances convert between whey and ether send ether from one account to another deploy contracts um interact with deployed contracts get block data or transaction data and then watch for events as they occur on chain that was intentionally quick if any of these concepts are new to you or you're new to ethereum in general then welcome to this weird and wonderful world and our docs are fairly good but I've also written a couple of introductory blog posts that will hopefully make these Concepts a bit easier so what's new a lot of work has gone into making async functionality just plain work in web3pi this year we've started with the HTTP provider because that's where the biggest bang for the buck is this is available today in beta versions of web3pi instantiating or like configuring these new providers is a little bit more verbose and that will improve over time but you're welcome to use it already uh so very quickly like an example of what some python or asynchronous python code might look like in this example we're just grabbing the first 50 blocks from the ethereum blockchain using async IOS as completed method so as each one of those requests goes to your remote node and comes back we just print out the block number that was received so you'll you'll get it in in this case a fairly random order uh the the takeaway here is that if you spend a lot of your application doing read operations um you in particular to and from a remote node uh you can see some massive [Music] performance improvements like in the neighborhood of 10x moving right along ens support is not new to web3pi but it's gotten some love in the last year so you might know it best for looking up the like it's read operations so looking up the in address and address for a name resolving a name from an address or getting some arbitrary text that ens allows you to store as part of your record new this year is async support for ens so if for whatever reason you want to fetch the address of Shaq vitalik and Paris Hilton in 30 times in Rapid succession you can see some massive gains in performance here uh same async i o as completed code sample there that's the takeaway your gains are might be even more significant than the standard web3 methods let's get a little weirder web3pi introduced support for ccaf ccip read functionality so this EIP was introduced by Nick Johnson of ens ccip stands for cross-chain improvement proposal and read alludes to your reading some data on off chain or otherwise not on ethereum mainnet so this EIP introduced a standard for contracts to let the user know it's going to fetch some data from somewhere else other than I mean ethereum the simple version of how this works or what this introduces is just a pattern for using a custom solidity error to return some metadata in of of note you can include one or more URLs where that off-chain data can be fetched from and then a callback function for where that off-chain data should be returned to to be verified as authentic essentially this diagram uh is included in the EIP and we're going to use it as a quick example of what this looks like in practice so the use case we'll chat through is uh ens placing some or there let's say ens is some sub domain is registered on on a layer two like optimism in our case the client is is your web 3 script or app we will be making some funk in this case is looking to resolve the domain or the subdomain of an address the contract is an ens resolver contract that says this value does not live on mainnet ethereum but you can find it on off chain somewhere at this optimism Gateway for example and that information is packaged up in this off-chain lookup revert message so under the hood without the user having to know this is even happening uh web3pi will go and make a get or post request to that optimism Gateway get that address back and then pass the relevant data to the specified callback function the Callback function within the same ens resolver contract then reads the signature verifies that this address came from a source that it trusts and sends back along the answer to web3pi so from a user standpoint this looks dead simple right you are just looking up the address for a name and you get a value back and this address might actually live on optimism on another L2 on somewhere else entirely so just for context here ens had to implement a protocol change of their own to support this functionality called wild card resolution if you're interested here they've they've got a full open source example of their resolver contract that supports this functionality also of note Nick is giving a talk directly after this one and I believe he's digging in a bit further into ccip read functionality cool little update from last week the lens protocol folks announced that they're going to be using the same feature set to supply [Music] social graph data from from their protocol via their own ens resolvers I guess they're running if for whatever reason this functionality introduces some security concern that is not compatible with your use case you can fundamentally disable that at the provider level level or at the Contra or the the call level and then again for more information go hang out with Nick or check out his blog post if you are looking to do something outside of the normal scope of what web3pi does you have a few options the first is middleware and this is the most common tool you'll reach for effectively what it does is it lets you inject some Behavior either just before a call goes out or just when it returns so you might be using this for some special logging Behavior some data visualization some additional like munging of of data points or whatever your use case might be it's pretty flexible and then you inject that into what we call our middleware onion your next option is a custom method so if you're using a a client like Aragon or autoscan one of these I have some non-standard RPC functions you can choose just to lump those directly into the the East module or whatever feels appropriate for you and you can use some of our data formatters and call that just like you would any other eth method also of note here is that you can use this to override any existing method in the in the module so if you want to change the way that any call or getting of Gas Works etc etc you can essentially replace it with your own version your third option is external modules this is intended to be plug-in support so if you want to include an entire API within web3pi a whole set of functions you can do that via external modules this is very flexible as well you can our only stipulation is that you create these classes and if you need to make use of the parent web3 module then you are or you have access to that via the the in-net method you can Nest these how you choose I think that's all there is to that one and fourth custom providers you're unlikely to reach for this unless you're creating something like a custom test harness or you just need to fundamentally alter the way that every request is made I'll leave that one there finally uh I don't recommend it but if you got a monkey patch things do what you got to do for more context done one you might reach for each of these got another blog post just for you are we good we're taking breaths out there all right let's talk about the merge as a app developer what do you need to care about the good news is uh that not much else like not not that much has changed here so a couple things uh we said goodbye to some test Nets Gourley is still a good choice sepolius still good as far as I know as well too block times changed there's maybe a subset of applications this might be relevant for pre-merge a new block was added to the chain every on average 13 seconds with high variability in our new world we have a new block every 12 seconds even with much less variability and I think it might be useful to understand where that variability comes from by differentiating slots and blocks if you're not familiar so every 12 seconds a new slot is made available and a randomly chosen validator can then propose a block to fill that slot and if a validator is offline for whatever reason then we might miss that slot and you're waiting for the next 12 seconds for the next slot to fill so that's where your variability comes from this happens very infrequently but it does happen next we've got block identifiers or sometimes called sometimes called block tags so you might be familiar with latest or pending or adding a specific block number if you want to specify when a particular East call is made we've got a couple new ones now safe and finalized and the short version of this is that safe is going to give you a result that is based on a period where it is very unlikely for a block reorg to occur and finalized is one in which it is extremely unlikely for a block reorg to occur So based on your use case explore those block tags and finally we've got the beacon API which has existed for a little while within web3pi but maybe it's more interesting to you now we've got there's nothing fancy here it's a very simple wrapper around the beacon node restful httpi endpoint HTTP endpoint I don't know where the eye came from that's about all there is to that a couple debugging tips these both revolve around East call I've prepared this little example scenario but in the interest of time I'm going to skip straight to the good stuff what does these call you might be familiar with it um you're probably more likely to use it and as attached to like the contract object so for example you if you're going to execute a function on a contract you effectively have two options to do so one is to call call on that method or transact transact will submit that application to the transaction pool to be to get picked up but if you call call that simulates the effect locally so that didn't feel like the perfect explanation but let's see if we can get there eth Cole simulates a transaction in your local environment we're going to use it in isolation here to like highlight a couple of debugging tips that might help you out the second parameter it takes the first is the transaction the second is that block identifier and the default is latest so you would play out a transaction in whatever the current state of the blockchain is let's say for example you wanted to find out why a transaction failed at some point in the history of of the chain so the revert reason is not something that's stored on chain but you can get it by essentially replaying a transaction at the time it originally occurred so in this example we've got a transaction we're interested in we build up a transaction a new transaction object to replay and then we call it at that block number -1 to replicate the state that it was in at the time it occurred and then you get that human readable revert message that you can then do whatever it is you need to do with for some more detail here another blog post but that's not all there is a third optional argument on each call at least within Geth and Aragon and possibly some others called State overrides this can save you some serious development Cycles if you need to get a contract in a specific state to perform whatever testing it is you need for example you need specific State values in that contract or you need the you want to even alter the bike code a little bit to or like the the operations within some of the contract itself or like undo a condition for which the contract would normally revert just to see what would happen otherwise you can override that in real time within the eth call method in this quick example I am telling I'm using so in this example we are we're telling each call that the contract that lives at this particular address I would like you to replace its runtime byte code with this altered version that I've made this version that I'm interested in testing and then we can run that and in this example we've got a successfully executing call in the last example that would have reverted for more detail here check out the blog finally last chapter we the python team uh know that as time goes on more and more of our users are going to be using our tools through one or more layers of abstraction and that's a good thing um specifically in mind are development Frameworks that sort of package together a bunch of functionality that let users be super productive in a much shorter time so I'd like to encourage you if you haven't yet to check out our friends over at ape they are they can be thought of as like the python version of hard hat and they are I think more and more in the future they will be our biggest or they will be the biggest consumers of web3pi as a vehicle for their users um so again this this gets us this enables our users to be more powerful to do more and to solve their local problems so that's a good thing so that's um that's what I've got for you we covered everything today uh show of hands make it a little interactive did we touch on anything you weren't already aware of is there anything that you're ready to dig into a little deeper that's like for the camera out there that's like 4 000 people that all raise their hands that's cool cool then mission accomplished um again the snakechummers.ethereum.org blog is where a lot of this lives if you'd like to catch up review on uh any of what we've chatted about today and yes we have room for some questions from the audience uh just a quick comment in terms of slots and block times there can be makes a missed slots where the slot occurs but there is no block so the block time doesn't necessarily increase every 12 seconds uh yeah thanks for clarifying though I'm not certain I got the clarification um so I block I think we're on the same page a block can fill a slot every 12 seconds but it doesn't always correct which one we'll have to