# Oracles for number values

- Speakers: [William George](https://streameth.org/speakers/william-george)
- Channel: [Devcon 7 SEA](https://streameth.org/devcon_7_sea)
- Date: 2024-11-15
- Duration: 25:39
- Watch: https://streameth.org/watch/67370d351b0f83434d27366f

## Description

We will overview the history and state of research on how to design a cryptoeconomic oracle that outputs a number value. One wants such tools for price oracles, but also for bringing other information on-chain, e.g. the damages to award from an on-chain insurance contract. We will look at approaches ranging from Vitalik's 2014 SchellingCoin proposal to ideas drawing from social choice theory, including based on recent research. We will explore tradeoffs including resistance to several attacks.

## About the speakers

### William George

William George has a PhD in mathematics from the University of Toronto. His thesis was on number theory problems related to elliptic curve cryptography. William began actively researching problems related to blockchains during postdoctoral fellowships at the University of Toronto and at École Polytechnique (Palaiseau, France). William has been doing crypto-economics research for the blockchain-based dispute resolution platform Kleros since 2018.


## Transcript

Hey everybody, so I'm going to talk about oracles for number values. My name is William George, I work for the Kleros cooperative. So what's the goal here? The district called a basic blockchain oracle problem. You've probably all heard about it many times. Blockchains don't have access to information about the off-chain world. If you want them to know something about the off-chain world, you have to tell them that information. A classic example is you might want an oracle for prices of DeFi, of assets for DeFi contracts. You might want other kinds of mechanisms to bring off-chain information onto the chain, maybe the amount of rainfall in some given location for some farm insurance contract. The mechanism used to do this is called an oracle. And sometimes the thing you want the oracle to output is a number, often, in the case of price oracles. Just to give a bit of my motivation and sort of how and why I'm thinking about this problem. So for those of you who aren't familiar, Kleros is a blockchain-based dispute resolution platform. Imagine Alice, the small business owner, hires Bob as the freelancer to provide her some service, build her a website, whatever. She puts Bob's payment in some smart contract escrow, and if Alice is happy with Bob's work, she just clicks a button and it's released. If Alice is unhappy, then she raises a dispute. That means that there's like some crowd of users of the Claris platform, a few of whom are drawn, and they decide who is right. And the right answer to this dispute, the correct resolution, may be a number. It may be some kind of partial settlement. How much should Bob be paid? If you have some kind of decentralized insurance contract, you might have disputes about how much of a compensation someone should get. And then basically you have this off-chain question, you know, like the answer to this off-chain dispute, and you want this, like, Oracle that's specialized in dispute resolution. And notice that these questions can be, you know, more subjective, more maybe individual than the kind of price Oracle question. And as a result, maybe your, like, oracle design might be slightly different for these slightly different questions. Okay. So now, like, digging really deep, how should we design an oracle? What are, like, the basic ingredients to go into an oracle? Well, on just, like, a fundamental level, you would ask, who is participating in the oracle? Obviously, you're bringing information from the off-chain world onto the chain. Somebody's got to be uploading that information. Who? What is the format of the information they provide? And assuming you have more than one person who's providing information, how do you aggregate their information if they don't provide exactly the same thing? How do you put it together into some kind of collective value that you use as the number in your system? And people probably aren't doing this altruistically. They're so, are they, what's in it for them? Like, are they being rewarded or penalized somehow? And as one makes these choices, what are you trying to optimize you for? Well, obviously, you want the Oracle to produce good information, fast, cheap. You want it to be attack resistant to whatever attacks are sort of relevant for your system. I'm going to talk about different choices people have made on these design choices over time, where some of the open like ideas are and like the attacks can vary from one idea to the other. I'll hit on a few. And so far all of this is relevant like everything I've said could be true for oracles whether they're outputting number values or not. I could have some discrete information that everything I've said so far would be relevant for. And where you really get into the number question, the specifics of the number question, is this question of how precise you want your information to be. How many decimals? And this is very relevant for incentive functions. If you have people, rewarding people to participate in your oracle, say you want a price oracle, and you say, tell me the price of ETH and USD. And I say, the price is like $3,200 more or less. And I say $3,201. Am I right? Should I be rewarded? Should I be penalized? So this is a question of, for a number, you can always have a more precise, closer answer. And there's this question of how close you need to be. And over the course of the talk, we'll hit on a few different notions of close that you can build into different kinds of incentive functions. Like maybe the most basic one would be to say, okay, you need to be within X percent of the output value. One percent, you're rewarded. Beyond one percent, you're penalized. And that's a simple thing you could do. There will be other ideas that people have done. Okay. Now to hit on, you know, the sort of next ingredient that goes into Oracle, this question of vote information and aggregation. So, again, I think a lot, spend a lot of time thinking about dispute resolution for the Kleros platform. Sometimes the disputes have binary answers or discrete answers that are non-binary. And as a result, I spent a lot of time in research thinking about how you aggregate kind of discrete non-numeric answers together. And this draws from the field of social choice theory frequently, this like academic field of how to design good voting systems. Voting systems that handle vote splitting well, that don't have too much tactical voting. There's all kinds of complicated questions when you're trying to define a voting system. And I bring this up because in the number case, you have similar problems. They have their own spin on these kind of vote aggregation questions. In some sense, the questions get easier of how to aggregate people's slightly different information together because now if your participants are submitting numbers, again, maybe the question is what is the price of ETH and USD, and everybody submits a number, now you can do number operations on those numbers. You can take the average, or better yet, you can take the median. So if you think of the numbers people provide to you, like lining them up from highest to lowest, you take the middle value, that's the median. And that's a particularly popular choice for people who have done a number of oracles because it's resistant to outlier effects. If somebody puts some crazy extreme value, that's not going to affect the median too much. So in some sense, this question of like vote aggregation is easier than in the discrete case, but also there are specific challenges like that sort of precision question I had from a slot a few ago. Okay, and now hitting back the sort of last ingredient before I get into the choices that different projects have made, this question of who is participating. And I would argue that there's like two basic models you could have here, and maybe you can do some combination of the two, but you can have either a crowd model or a delegate model. With the idea, and I would say that there are like parallels here to kind of different proof of stake systems, delegated proof of stake versus something that's more like Ethereum style proof of stake. So the question is, who is doing the thing? Who is providing the information? In the proof of stake systems and the consensus algorithms, who is producing blocks? In an Oracle question, who is producing Oracle values? With the delegated model sort of taking the point of view that these tasks are hard, maybe you want an Oracle that produces fast are hard. Maybe you want an oracle that produces fast values very frequently. You want zero downtime. And to have that kind of reliability, you want a beefy institutional actor that has really high performance hardware somewhere in some data center. And thus, maybe it's not appropriate for random people to be doing the task of providing the oracle information or producing the blocks in the consensus algorithm. So instead you have them vote on who you, like on delegates who then do the task for you. And, you know, then the crowd model is the opposite of that. You know, everybody's doing the task. I can get 32 ETH, I can spin up a validator, I can be part of proof of stake, or in an Oracleacle, I can be the person that's providing values for the oracle. And there are, of course, pluses and minuses. In the crowd model, you know, if you have, like, lots of people that are doing, like, a task constantly, that takes gas. Maybe, you know, like, normal hardware, they're not, like, super fast, as fast as an institutional actor. In a delegated model, there's some risk that the delegates could abuse their role. Sure, they can be voted out, but in any given question, they're delegated. And you can mitigate that somewhat by aggregating a bunch of delegates together, so that you'd have to have some collusion or something for them to crop an answer. And now, to sort of summarize some of the different choices people have made over time, I'm going to start with the beginning of the history of this problem, which is a blog post by Vitalik back in 2014. He called it Shelling Coin, so this is the dawn of Ethereum. This post was actually one of the big inspirations for what became Kleros, ultimately. And he was thinking about this sort of price oracle question again, where his proposal is, okay, I have people, they each submit a number value. You output the median. So far, that's kind of normal. And then his incentive function was that you reward people in the 25 to 75% range. So people who have middle values are rewarded, and people who have extreme values are not rewarded. Maybe they're penalized. He didn't explicitly say who the participant should be. He assumed that you had some kind of civil resistance tool. So this could be either delegated or crowd, depending on exactly what that civil resistance tool looks like. But you can certainly imagine a crowd version of this. And now here, note that this 25% to 70% rule, now the notion of being close to the answer, of being rewarded, close enough to be rewarded, is being closer than other people. Not just close to the answer, being rewarded, close enough to be rewarded, is being closer than other people. Not just close to the value, but like, I have to beat out somebody else. Everybody can be within 1%. Not everybody can be within the middle of 50 percentile. So, moving on in history, like, the big Oracle provider, at least for price Oracles now, is Chainlink. So, I'll summarize what they do and how that's kind of, you know, there's choices evolved from what SchellingCoin proposed. So here they have like a marketplace of nodes that provide Oracle information. Those nodes are supposed to take information from reliable sources, CoinGecko, Kaiko, whatever. And then a given price feed has a bunch of nodes that are sort of delegated into it. And you output the median of the different nodes. And those nodes are rewarded for their payment for services. So, in some sense, this is crowd-ish, because there's this marketplace. Everybody can participate. I, too, can spin up a Chainlink node. But in practice, it winds up being more like a delegated system, because any given price feed has some nodes selected. And then if you want to get rid of them, that goes back to this question of voting them out through a governance process. If I have a price feed that's used by a big DeFi application like Compound or something, if one of the nodes does a bad job, then you correct that by going on the Compound forum and having a proposal that looks kind of like this where you're like, please change the price feed to update and get rid of that node. like this, where you're like, please change the price feed to update and get rid of that node. So now to like, to like just wrap up this question of like delegated versus crowd, here's a sort of partial list of different projects over time. And there's kind of a spectrum where you can be sort of in the middle, but like more or less clustered into the two sides. Maker had an internal price Oracle for their, their DAI stable coin. And it's older than anything else on this table, so they had more of a delegated model. Really, all of the price oracle things have this delegated model, which is probably the only practical choice you could make, at least historically, because if you want a price oracle that updates in real time for DeFi applications, you can't have huge crowds of people voting constantly. The gas would be crazy. People aren't performant enough to do that. The stuff on the other side that's more crowd, Kleros and UMA, you know, have more like individual one-off cases involving human effort. So it's not super important that people are up to, you know, voting in real time. Nest has a sort of interesting thing where they are specific to price oracles because they use like an arbitration an arbitrage game to as part of their mechanism so it depends on being a price oracle okay now getting back to the other questions aggregation functions incentive functions and which ultimately comes down to what are the attacks if I'm trying to manipulate the system like what can I do and which systems are more robust? So an attack that was already highlighted by Vitalik in his shelling coin article is what he called micro-cheating. There's this long quote, I'm not going to read the whole thing. But basically the idea is that you can just nudge the value provided just a little bit. Like, you know what the true answer is, but you as an attacker provide a value that's just slightly to one side, whatever side you want to nudge the oracle. And if you're sophisticated, maybe you stay within the 25% to 75% range, so you're not even penalized. And maybe you can move the output a bit to give a bit of a more visual to this. There's some distribution of how honest people, people who are really trying, will provide answers. Some people are better at this task than others. So in the absence of attackers, it will kind of average out. Here the median answer is the spot on people. But if I have an attacker that has a few votes, not even like a majority, the votes only three out of I think 12 or nine or something here, then by sort of going to one side, you know, if they're like a sophisticated actor that can anticipate the distribution of the honest people's votes, they can kind of unwittingly wrap the people that are confused and off to one side into an attack coalition against their will. They can take the, like, confused people, join them together with the actual attack votes, and collectively they have a majority that moves the answer from spot on to okay. And then as the attacker gets more and more votes, you can drag the answer that much more. And if you think algebraically, how much can attacker with K votes move the answer like this? Well, if your aggregation function is taking the median, then an attacker with K votes can drag the result to the 0.5 minus k over 1 minus kth percentile of the distribution of honest participants. So this gives us a measure to judge, okay, this is the resistance of median as an aggregation tool against this kind of attack. And then we can compare that to other attacks and see which ones are more or less resistant. And a natural question is, does it ever make sense to do anything other than check the median? The median seems like a really robust mechanism. Lots of projects have used this. And I would say probably not if your voters only give you a single number. But if they give you multiple numbers, you can do something that's more interesting. And now, getting to research that I have done, I've thought a lot about how to have voters provide intervals of precision, where now they provide you some lower and upper range where they think the true value lies. And if everybody's intervals overlap, well, the answer should be somewhere in the overlap. If there's some point of conflict where there's one interval that's like the upper bound is strictly less than the lower bound of some other interval, then they're like they disagree. And you can essentially have the users vote on whether you're higher or lower than a point of disagreement. I will sort of quickly go through this because I don't have tons of time. But you know, you can think of if my upper bound is less than the point of conflict, I vote less. If it's my lower bound is higher than the point of conflict, I vote less. If my lower bound is higher than the point of conflict, I vote higher. And if my interval just contains the point of conflict, well, then I didn't vote at all. I gave you less precise information. Everybody can vote like this. You can kind of resolve the points of conflict and you can come up with a collective answer. So this is an aggregation mechanism that isn't just taking the median. This is based on an academic article I wrote with a co-author, Clement Lessage, several years ago at this point. That was based on a version of this that was slightly different because at the time, we wrote this with basically trying to be compatible with Kleros v1 as it existed at the time. As of a few days ago, Kleros 2.0 has been launched, which has much more flexible mechanisms for being able to encode things like this as modules. So now we have more flexibility to do things like this in the future. So how does that aggregation mechanism I just proposed, how does it compare to taking the meeting? Is it better or less attack resistant? Without going too much into the details, I will just say that it kind of depends on whether, on your distribution of the honest participants, like how they, how they act. Particularly if people who are confused know they're confused. If people who are like out on the edges of the distribution are providing improvised information, if they give you long intervals because they realized, oh, I don't really know the answer to this question, then this sort of voting by intervals performs better than just taking the median. If people that are wrong are really convinced that they're super confident of being wrong, then it performs not as well. There's slight effects either way, but this is the sort of analysis you can do of attack resistance for these different systems. Now, that was all about the aggregation rule. Even before you think about is the attacker going to be penalized or rewarded for doing an attack, or how much are they going to be penalized, this is the sort of basic question of how much can an attacker that's willing to sacrifice some amount of money drag the answer with some kind of minority attack coalition. When you get back to the incentive function, there are all kinds of interesting questions. And getting back to this question of how close do you need to be to be rewarded, ultimately every sort of incentive rule that you come up with encodes a notion of distance to say whether, like, a given participant is close or not. And for the voting by intervals thing, I have this really complicated formula that I use as an incentive rule, at least tentatively, that tries to balance the fact that you want to encourage people to submit really small intervals, which is the first term. And at the same time, you want to encourage people to be, you want to really reward people if they vote on the right side of the points of conflict. people to be, you want to like really reward people if they vote on the right side of the points of conflict. So that if there's like a point where the system could go a different way and choose a different answer, you want to like raise the stakes on people so that an attacker that winds up losing like loses a lot of money, you know, that much more. So if anybody's interested in that formula, feel free to talk to me. We can dig into it. And then like the different metrics that people have used, just to summarize the three that we've looked at. So there's this notion of being close if you're close to, like, as a percentage of the output. There is a notion of being close if you're closer than other people. These are different things. And then there's that crazy formula from the page, which is about being on the right side of points of conflict, so sort of being close when it matters. And if everybody kind of agreed it doesn't matter so much and the formula doesn't care as much about how you voted. Summarizing how different projects have taken the different things they've done on these choices. So, shelling coin and chain link, like the format of the vote is just a number. As such, the only real reasonable aggregation mechanism is just to take the median. For the interval approach, I have more complicated information. That means I can take more complicated aggregation rules. Some projects I didn't talk very much about. Other projects also have, you know, that have vote formats of numbers also take the median. Pyth, which is an interesting example, they also have something where you have something like an interval. They have a different aggregation mechanism. And I just want to say there's a lot of room for experiments here. So concluding, historically, on this delegated to crowd model, most people have been interested in price oracles. If you want a price oracle that updates really fast, you probably needed a price, like a delegated model, particularly in a high gas environment, maybe even in a low gas environment, just because you want people to be able to provide information with very low lag time. But as you consider more bespoke questions, subjective questions that you might have in a ClaroSelect platform, now you open up this design space and you get back to this question of digging in, like what kind of oracles can we design and does it make sense to go back to a crowd model? We've come up with measures to talk about how micro-cheating varies from one approach to another. The delegated approach, it leaves its incentives to just the threat of being thrown out of the platform. So they don't really have explicit incentive rules. But if you want a crowd model, you really have to think, like, what's my incentive role? And there's a lot of open, interesting research there. So if you're interested in that, you know, reach out. And I'm happy to take questions. happy to take questions. So we've got quite a few questions streaming in. As a wider variety of RWAs get tokenized, how concerned are you that oracles become a point of failure? Yeah, so like, the more things you have that are integrated in important ways in your system, the more attack service you have. You know, things you have that are integrated in important ways in your system, the more attack service you have. If you have real world assets that people are really engaged with and they're providing information, you have lots of watchers. A priori, it shouldn't be so bad, but we definitely want to think about how rigorous our oracles are. How do crowd oracles typically implement Sybil resistance? Yeah, so I mean, it depends. For Kleros, in Kleros 1.0, there is a token weighted drawing, so there's a token that participants have to have and you can't take over the system unless you have a large percentage of the tokens. In Kleros 2.0, we've considered additional social mechanisms that like that token mechanism is still there, but also you can use proof of personhood tools layered on top of that. And we've had interesting results that I can point you to if you're interested, such that you can design a system where you both have to break the proof of personhood and break the token way to draw to mount a large-scale Sybil attack. Is there an actual objectively right answer or truth for numbers, or is it something that's always subjective? I would argue, on some level, if you have a discrete question, you can talk truly about having a right answer, an answer that's better than any of the other answers. If you have a number question, people, if you zoom in enough, can always disagree. People might say, okay, yeah, we went up to the thousandth decimal place, but in the 10,000th decimal place, I'm right and you're wrong. So in some sense, there's always some amount of subjectivity in any kind of number question. How do Oracle implementations typically check that they have a sufficient quorum of answers? Yeah, so for the delegate systems, I mean, like, they have, like, some number of delegates that they think is appropriate. An individual system might have a quorum in case there's, like, some, like, big outage where, like, okay, I have nine delegates, at least four of them have to be online or something. So when you have a small number of delegates, it's straightforward. In a crowd system, you could build something like that in. In Kleros, we don't have a quorum system because there's an appeal process. So if for some reason everybody was censored, there was an outage, it was bad network connectivity, and nobody voted, you can just appeal and try again. But certainly something to think about in some systems. And the last one is a great question. If I'm a participant in the voting crowd, what's stopping me from just copying another answer as my vote and essentially free-riding? Yeah, so again, this will depend on the system. So in Kleros, again, there's this appeal mechanism which gives people extra rewards if they were on the right side of an answer that is, like, where they lost their voting round and ultimately, like, appealed and they were proven right by the appeal round. So that, you know, gives an incentive to people to be contrarian if they think that they're right. Other approaches can be taken. There are, you can use commit and reveal systems. There's still questions if you do that about what happens if someone pays you to reveal your vote even if the commit and reveal system allows you to technically hide your vote, you reveal anyway. There are lots of interesting sort of properly cryptographic questions there, but that's like a subject of research, but there are approaches. So I might require a fact check on this last question, that Polymarket has a highly centralized oracle and seems to be the largest. Do you think sentiment will drive crypto products to this model instead of algorithmic? Is it a centralized oracle first? I think they use Zuma. So, like, that's, I think, you know, like, people can fact-check me. But I believe they use, like, a crowd model. So, you know, if people are interested in that product, then there could be demand for that much more decentralization. If something takes off, people will have greater scrutiny, of course. But in general, what do you think about centralized oracles versus algorithmic or decentralized ones? It sort of depends on what you mean by centralized. Do you consider like a delegated model like centralized? I mean, it kind of depends. You know, it's not a question of like absolute centralization, it's about who has the power to do what. If you have just like a pure one actor responsible for providing you like the truth, you know, you don't have a bunch of delegates and you don't take the median of their answers. You don't have a marketplace to join or be kicked out as a delegate. And then, yeah, obviously, if you just have one absolute source of truth, that's not necessarily in the spirit of the ethos that we have. Thanks. Yeah, that was really good and quick.