# Privacy-preserving Smart Contracts at Scale by Noah Johnson (Devcon4)

- Channel: [Devcon](https://streameth.org/devcon)
- Date: 2019-10-09
- Duration: 30:12
- Topics: blockchain, smart contracts, distributed ledger, Science & Technology
- Watch: https://streameth.org/watch/yt-XmmdCP5lAUQ
- YouTube: https://www.youtube.com/watch?v=XmmdCP5lAUQ

## Description

Visit the https://archive.devcon.org/ to gain access to the entire library of Devcon talks with the ease of filtering, playlists, personalized suggestions, decentralized access on IPFS and more.
https://archive.devcon.org/archive/watch/4/privacy-preserving-smart-contracts-at-scale

In this talk we'll describe Oasis, a platform for privacy-preserving smart contracts at scale. Oasis addresses two critical issues of today's platforms: poor scalability and a requirement that all data is public. These platforms cannot support many exciting use-cases which have complex application logic (e.g. machine learning) or require protection of user or application secrets (e.g. data markets). Oasis is a layer 1 blockchain platform that scales to complex workloads such as machine learning and protects data via secure computing techniques. Oasis's scalability stems from the novel separation of computation and consensus in a layered design. This design allow transactions to execute in parallel before validation by the consensus layer, thus alleviating a major source of congestion and enabling new verifiable computing techniques that dramatically reduce replication needed to ensure integrity. Oasis uses a proof-of-stake consensus algorithm tailored for this architecture and supports multiple secure computing models (trusted hardware, multi-party computation, zero-knowledge proof) based on security and performance requirements. We'll summarize the Oasis protocol and discuss real-world applications built on Oasis in as credit scoring, medical data sharing, and blockchain-based games. We'll describe how Oasis enables these exciting applications to run directly on-chain, avoiding the need for off-chain computation.

Speaker(s): Noah Johnson
Skill level: Beginner
Track: Privacy
Keywords: oasis, contracts, scalability

Follow us: https://twitter.com/efdevcon, https://twitter.com/ethereum
Learn more about devcon: https://www.devcon.org/
Learn more about ethereum: https://ethereum.org/ 

Devcon is the Ethereum conference for developers, researchers, thinkers, and makers. 
Devcon 4 was held in Prague, Czech Republic on Oct 30 - Nov 2, 2018.
Devcon is organized and presented by the Ethereum Foundation, with the support of our sponsors. To find out more, please visit https://ethereum.foundation/

## Transcript

so I'm Noah Johnson I'm co-founder and chief product officer at Oasis labs and I'm gonna talk about privacy preserving smart contracts at scale so if you follow the news it seems like every article in the last three years or so has written about the huge potential of blockchain how blockchain can change the world how it can enable fundamentally new applications and unlock value in different verticals like healthcare now if you compare this to where we are today it's clear we're not there yet right what we have today are things like crypto kitties group do kitties are great but they're not gonna change the world and so an oasis what we're focused on is the question of how do we get there how do we build a computing platform that allows developers to build these sorts of applications to fulfill exactly these promises and so we're building a smart contract platform that can support real world applications and what makes our platform unique is that we're focused on both privacy and scalability together now by privacy I mean the ability to protect data and state of the smart contract as it runs on a decentralized network and for scalability we mean not just high TPS and low latency which is what most projects talk about when they discuss scalability what we mean specifically is the ability to scale to fundamentally more complex types of applications so we think these two things are absolutely necessary to support applications of the future so if we take a step back there's a reason people are so excited about blockchain in the first place blockchain has a number of really exciting properties like the fact that it's completely open and transparent doesn't rely on any central party and of course through smart contracts we can automatically enforce agreements even with people that we don't otherwise trust but blockchain by itself actually doesn't provide privacy so blockchain can provide anonymity in that I can have a wallet that's not linked to my identity but it doesn't protect the data and the computation that's running on the network in fact today's blockchain platforms are designed so that all data is public this makes the architecture a lot simpler and so this is a huge limitation if you think about all the applications on the internet that could benefit from these properties and then you think about which of those would still work if you required that all of the data was publicly accessible to everyone in the world right the list gets pretty small and so this is I think one of the reasons we haven't seen many of these breakthrough applications yet so what could you do if you were able to provide privacy here's a list of some of the applications that people are already building or excited to build on blockchain things ranging from credit scoring to private escrow to you know games and private tokens so what do all these things have in common well they all need privacy they all need the ability to protect data or protect state either because the data is sensitive for example it's user data like in the case of healthcare or you just need to be able to keep certain aspects of the application secret in order for the incentives to work out for example games I need to be able to keep the secret part of the game secret from the miners and the players to make sure that they don't cheat so these are all really exciting applications but if you wanted to run these applications on today's platforms you wouldn't be able to and to illustrate that let's take a simple example of credit scoring so I could go out and write a smart contract that does credit scoring right I could implement this in solidity the contract could take in some data from various institutions and output a model for predicting a credit score now if I try to take this and run it on today's platforms I'd face a number of issues that make this completely impractical the first issue is that if the output of my smart contract is derived from sensitive input data then I have to be very careful to make sure that the output doesn't leak anything about the inputs this might seem like this is intuitive if you're doing machine learning of course you're not leaking information it turns out actually that assumption is completely wrong and you need to employ solutions at the application layer to protect against information leakage in this way the other problem of course is that the workers that actually run the computation the workers that execute the smart contract will have access to all of this really sensitive data so if I don't trust those workers if this data is really sensitive they could trivially snoop on the data or leak the data so for that reason I wouldn't be able to run this on platform where all the data is public and then finally of course this is a relatively complex application everyone's familiar with the scalability challenges of today's platforms right now if I wanted to do anything even moderately sophisticated in a smart contract I would face really really poor performance and very high costs so that brings me to our solution so at Oasis we're building a new platform that tries to address all of these problems specifically we have an architecture that protects privacy both at the application layer as well as the platform layer and a new approach for scaling to more complex application workloads so I'll talk about each of these layers kind of independently I'll describe how our solution works and then how you can use them as developers so let's start with the platform layer so how do we protect data that is running on a network of nodes whom we may not trust so if you look at the model for smart contracts today the model for smart contracts of course is smart contract is a piece of code that takes in some input running the code produces some state transition that's recorded back to the ledger and on platforms like aetherium of course all the inputs and outputs are public so the concept of confidentiality preserving smart contract execution is that all of these inputs and outputs are encrypted that is they remain secret and nobody else in the world except for the smart contract can actually view their contents and this isn't just about encrypting the data the challenge is being able to still run computations on the data without allowing others to view the data and to protect data both during storage when the data is when the state for example stored on the ledger and when the data is actually being computed on so it turns out there are a number of different technologies for supporting confidentiality preserving smart contracts these range from the use of special hardware to cryptographic techniques like secure multi-party computation zero knowledge proof s-- and homomorphic encryption so each of this these techniques has different trade-offs in terms of the performance overhead in terms of the class of applications that they can support and in terms of the security model that is where are the security guarantees coming from so there's no single approach that is strictly better than all the others it depends on the application and the use case and so OSS will support all of these techniques and allow the developer to decide which one they want to use I'll talk briefly about secure hardware because it's the one that will support initially and have done the most work on so far so are people familiar with secure Hardware have you heard of trusted hardware great so for those who aren't familiar secure hardware is a technology that's built into processors that allows code to construct what's called the secure Enclave and the secure Enclave can run computations and store data in a way where it's completely isolated from the rest of the machine that is to say no other applications on the machine or the operating system or even the user of the machine is actually able to view the data or modify the data in the secure Enclave and this is enforced at the hardware level and so what this means is secure enclaves provide both integrity that is we have assurances that the computation result hasn't been tampered with and confidentiality that is we know that no one can actually view the data as long as we construct this unclaimed carefully the other thing that secure Hardware provides is the ability to generate a certificate that can prove to a remote party that doesn't have physical access to the machine that the code is running on correct hardware and that the contents of the Enclave are correct so in our previous work we publish an academic paper that shows how to use secure hardware to run smart contracts in an enclave to provide end-to-end privacy that papers called akedan there's a lot of interesting details to get this to work I encourage you to check out the paper we're also working with the Keystone project on a fully open source design for secure hardware so this is based on the risk 5 microarchitecture and this will be a design that anyone can audit anyone can verify and any chip maker can manufacture so in the meantime we're using the trusted Hardware technology that's available today including Intel SGX but in the future as new technologies become available they'll be exposed to developers through the Oasis platform tomorrow morning at 10:00 a.m. there's a workshop with with Intel and a few other companies that are using secure hardware so if you're interested in this subject I encourage you to check out that workshop okay so how do you actually use this feature so suppose we want to have an election through a smart contract so I'm gonna implement a ballot as a smart contract and typically today if you want to interact with a smart contract right you would have a DAP that includes a web 3 library and the DAP would call into web 3 that library would issue transactions to a web 3 gateway like in Fuhrer and those transactions would be relayed to the workers that would execute the transactions and of course in this model all of the transaction data and the state are public now this is a great design but it's designed specifically under the model where all of this transaction data is public so if we want to support the ability to keep state and data secret we need to make extensions to this web 3 interface and that's exactly what we've done so we have an extension to web 3 called confidential web 3 and what this does is allow callers locally to construct a secure channel to the contract and then send transactions to the contract in such a way where the contents of the transaction are encrypted so only the smart contract can actually decrypt them and view them so essentially we have an end-to-end encrypted channel to the smart contracts so no other machines on the network not the Gateway not the web 3 provider not even the workers are actually able to see the contents and the payloads of these transactions and of course the smart contract state is encrypted with a different key that's known only to the smart contract so that users can't decrypt the state but they can decrypt for example the responses from the smart contract so the the library that we're releasing is going to be available as a JavaScript library that you can integrate in your DAPs and the library handles all of these details under the hood establishing this channel and generating secure transactions so that the interface for calling in to confidential smart contracts is very similar to what you're used to today that is your code requires only very minimal changes in order to use the confidential version versus the existing version so I just described how os's can protect data at the platform layer from nodes and workers on the network this is essential of course for protecting privacy and supporting applications that need to protect data but it's actually not enough for certain applications and in certain cases you actually need to do something at the application level as well and so to motivate this let's ignore blockchain for a second let's just talk about the general problem of there's some data scientist that wants to do something useful on sensitive data maybe it's train and machine learning model so they would write some code that trains the model that code inputs sensitive training data and and outputs the model now even if we don't release the model if all we allow is for people to query the model right that's the whole point of training a model in the first place is so that I can serve inferences it turns out even through these inferences that are generated by the model it's possible to leak original sensitive data from the training data set and there's actually a recent work that shows that this happens even if you're not intentionally trying to leak the data even if this is a completely benign example and it's a pretty straightforward machine learning pipeline it's possible for the model to inadvertently leak for example credit card numbers and Social Security numbers this was shown in this paper and so clearly if we want to move this to blockchain and we want to provide end-to-end privacy guarantees we need to protect against these sorts of issues as well and this is actually a real-world problem that companies in industry face so in our previous work we collaborated with uber to help them solve a very similar problem their setting was of course they collect you know customer data about rides and destinations and so forth and the data is extremely valuable so they want to make it accessible to their data scientists internally to mine the data and and generate business insights on the other hand because the data is sensitive they want to make sure that they're protecting the privacy of individuals and so the the big challenge here is how do you allow kind of safe analytics things that don't violate privacy well disallowing things that do violate privacy so a good example to contrast this is you know something safe is asking a statistical query like how many people visited New York last year right clearly that's not going to violate anyone's privacy nobody in this data would object if this result was returned to the analyst on the other hand if the analyst asked this sort of question how many times did Joe go to Whole Foods last week that's a statistical question on exactly the same data this is a pretty serious privacy violation so how do you automatically guarantee that the system will allow things like this and disallow things like this this is a very hard problem in general so one of the techniques that we've used is a technique that's developed out of academia and it's called differential privacy by the way are people familiar with differential privacy so fewer people great so differential privacy is a formal definition of what privacy means it's a formal mathematical definition and intuitively it says that okay if I have any two databases that have sensitive information and one of those databases has a user and the other database is identical to the first one except it doesn't have that user right so database one has Jo database two has every record in database one except Jo differential privacy the definition says that if I take any query or I train any machine learning model or I do anything on both of these data sets independently then the answers the results of those computations have to be essentially the same whether it's run on this database or this database so what this means is that from the result there's no way to tell whether Jo was actually in the database because the result would look identical even if he wasn't and so what this means is given that I can't even tell whether a user's in the database I certainly can't learn their sensitive information and so it means the outcome is the same with or without Jo's data and the property says that this has to hold for every user and every database and so the nice thing about this definition is it provides very strong guarantees for individuals while still allowing those sorts of general-purpose analytics and machine learning which is the very reason often times we want to analyze data so if a user knows that the system enforces this property they don't need to worry about what the query is doing they don't need to inspect the query they have assurances that whatever is returned there's no way for the person that sees that result to know whether in the database so there's no reason for them to object on privacy grounds to being in the database in the first place so this is a very very nice property our work actually was focused on how to take this property and build real-world systems to automatically enforce differential privacy and other types of security policies for real world queries and in a real-world setting like ubers and in particular allow it to be used by non experts so hide all the complex details of how to actually get this to work so that analysts and developers have a very simple interface they don't need to make major changes to the way they write their code and the system automatically enforces differential privacy and can prove that it enforced a differential privacy so we have a number of papers that talk about this work and the the system that we built based on this research is actually now already deployed internally at uber so all queries that are submitted internally by uber analysts are routed through the system to enforce properties like differential privacy and other policies around compliance with gdpr and there are other pilots underway the tool itself is a veil above open-source we want to put this in the hands of smart contract developers so this has already been battle tested and enterprise we want smart contract developers to also benefit from this technology so that any developer can use it even if they're not privacy experts so we're gonna release what we call the Oasis privacy SDK that allows a developer to write a smart contract that includes some of these libraries and that means that the smart contract for example if it's training a machine learning model even if it's training on sensitive data there are provable guarantees that the output actually doesn't leak any information about inputs and so essentially what this means is that developers can demonstrate to users because smart contract code is auditable they can actually prove to users that they're not violating privacy the user can verify that their privacy is not being violated this is really an unprecedented property this is something that you don't have in traditional cloud applications the fact that the user doesn't need to take the developer's word for it they can prove to themselves that their privacy is not being leaked and we don't need terms of service we don't need privacy policies everything just happens automatically so example if you wanted to build a data market you would be able to use this SDK build a machine learning pipeline and have users that interact with that data market know that there's no way even if you wanted to to violate their privacy so in the future as privacy becomes more and more important for users features like this I think will be a huge competitive advantage for applications that are moving to blockchain versus traditional cloud applications okay so that's a glimpse of some of the solutions at the application layer finally I want to talk briefly about scalability and in particular how we can scale to more complex types of workloads so scalability is a complex subject I won't go into full depth in this but I'll kind of sketch out at least the intuition of some of our architecture so if we look at why scalability is important as these smart contract platforms continue to evolve people are building more and more complex applications and as the applications get more complex congestion on these networks increases so I'm sure everyone's familiar with crypto kitties when it became popular last year it you know it slowed down the entire aetherium network and it caused transaction fees to go through the roof so if we kind of extrapolate to the future where what we want to build is not just the next crypto kitties but rather a new paradigm for data sharing and doing machine learning on sensitive data these applications are significantly more complex than anything that is being deployed today and that means we need a new architecture that it can accommodate these sorts of applications now I want to emphasize that skill build is more than just high TPS so when when most platforms talk about scalability usually what they're referring to is the number of transactions that they can process in any given time period and this is an important metric but it actually doesn't capture the full story there are a number of variables that determine you know how well a platform performs in practice and one of the most important of them is what is the workload look like how complex are the computations there's a huge difference between a platform that can process you know thousands of very simple transactions like payments but completely fails to process anything that's even moderately more complex than the and so we architected in our system our system specifically to provide scalability for complex applications and those that have dependence on other smart contracts and so this is actually an architectural solution and to sketch why this is necessary if you look at the design for most existing platforms the reason these platforms don't work well for complex transactions is the fact that every node actually runs the same computation so that means there's an enormous amount of redundant work and there's no parallelism also the fact that every node has to do both execution that is to run the code as well as validating transactions and maintaining the ledger this means that if there's a slow transaction like a crypto kiddies breeding transaction that has to run and it will prevent even faster transactions that could have confirmed immediately if the machine was able to validate it those have to wait for the computation of the slower transaction to finish so this means fundamentally that slow transactions will cause congestion for faster transactions and this is true even for typical scaling solutions like sharding a simple shorting solution simply partitions the network into multiple groups of nodes that all have the same property so even with simple shorting you still have this congestion issue for transactions you know within the same shard so we need a fundamentally new approach one of the main intuitions behind our architecture is to separate out the computation that is you know the machines that actually run the code separate that from the machines that validate transactions and manage the ledger and so one of the you know one of the challenges of course is how do you do this in a scalable way how do you allow these nodes to verify that their transaction is correct even if they can't see the transaction data right earlier I showed the concept of confidentiality preserving smart contracts is that all of the outputs are secret so these nodes have to be able to verify that the transaction result is correct even if the data is encrypted so there's a lot of details like that that need to be worked out but fundamentally the advantage of this architecture is that these layers can be scaled and optimized completely independently so if there's a bottleneck because compute for example because there's a burst of transactions that require more computing power then it's very easy to scale up just the computing side of the network and conversely if there's congestion during consensus because there are too many transactions that need to be confirmed in any given time period you can add more workers easily to the consensus pool and so you have a lot of flexibility for for you know performance optimizations and scalability and so this is most beneficial for really any kind of complex application like machine learning and and and applications with mass adoption that required this sort of flexible scaling that you don't have with these bundled architectures today and we'll publish a paper soon that describes this architecture in more detail ok so let's put this all together and show how all these features work together to solve real-world problems and hopefully get you excited about the sorts of applications that you can build on this platform so I'll talk briefly about an application that's currently being developed on oasis which is called kora there's a collaboration between Berkeley ETH Zurich and Stanford and so cara is designed to solve this problem of data silos around medical data so medical data is very sensitive but it's also very valuable if researchers had better access to medical data they could develop better systems for personalized health and predicting you know health conditions and so forth so Cara implements a data market specifically around medical data so the way it works is the user inputs their data into a smart contract and a researcher that wants to train a model on that data can submit the model to the smart contract and pay a token or you know pay in terms of tokens in order to run that computation the entire computation runs directly in the smart contract and the users receive compensation for their data and the researchers receive the train model so this is ideal because on the one hand users write because this is implemented as a smart contract they don't need to trust researchers they don't need to vet the researchers they don't need to worry about exactly what the researchers are doing they have assurances that their data will only be used for this specific purpose of training a model and because they can verify actually that the model enforces differential privacy they know that while their data is going to be used for this analytics the output actually won't reveal anything about them individually so it's completely safe for them researchers on the other hand now have access to richer and broader data sets than they had access to before because users are more willing to contribute their data and the fact that the whole thing runs on Oasis means that even the workers that are running the machine learning actually can't see the data so we have end-to-end privacy guarantees for users and their data and of course you know you need scalability you need scalability to these complex types of computations in order to be able to run all of this directly on chain great so hopefully this gets you thinking about the sorts of things that you can do remember earlier the slide I talked about unleashing the value of medical data you know this is exactly an example where blockchain can can solve new problems and and bring value to places where right now the value is locked up specifically because of privacy concerns so we want to support and empower developers building on Oasis to develop these sorts of exciting applications we're releasing a number of tools and libraries that support you know diverse applications ranging from machine learning to data markets to games and even standard kind of aetherium style smart contracts we also have a startup hub which I'll discuss briefly to provide more direct support to teams that need it and we want to build a strong community together so we announced a few weeks ago this startup hub you can apply at the URL at the bottom this will give teams that are working on really complex applications and want a little more support and mentorship directly from the Oasis team this will give direct access to Oasis engineers and and access to our our list of top investors as well so if you're working on an exciting application please apply here we'd love to have you in the program and then finally so we launched our private test net a few months ago we've gotten a ton of excitement there's a lot of developers already building on it based on the feedback from these developers and our experience operating the test net we're really excited to announce the next milestone here today and we're calling it a dev net because the main goal is to allow developers to start building these sorts of privacy preserving applications so what it will include is the confidentiality libraries that I described earlier these extensions to web 3 for providing you know new ap is to protect data during computation provide a contract kit to allow you to develop and debug contracts locally we support EVM as well as web assembly so that means you can develop smart contracts in in rust and we have a set of tools for doing that we're backwards compatible with aetherium if you have an existing smart contract and you want to port it over to Oasis you can do so without any changes to the smart contract and your app immediately benefits from you know the scalability and privacy properties provided by Oasis we have a set of resources and tutorials including best practices for writing private smart contracts how to build you know secret voting apps for example how to build a game in order to protect the state of the game and a number of other tutorials so that will be available at launch and then finally just better reliability and performance and we'll continue to improve the performance as we implement more of our scalability architecture into the test net so we're really excited about this this will be available very very soon so if you're interested I encourage you to go to OSS labs calm we there's still be an application process but we've streamlined the process you should hear back from us probably within 24 hours and you can immediately start building your applications so that they'll be ready to go by the time we launch our main net so if you want to build an application you can go here if you're excited about this project or about confidentiality and secure hardware and machine learning in general we'd love to work with you either as researchers collaborators as well as team members so we're hiring you can go to this URL to see the job openings so that's it if you want to find out more go to always this lab calm and follow us on Twitter at Oasis labs thank you we have time for one question yes I I just wondered with your privacy-preserving smart contracts if the inputs and outputs are encrypted and I can't see it run how does consensus work on a blockchain right so we'll describe that in the paper that we'll publish about this new architecture conceptually the nodes that validate transactions the transactions still run on multiple workers the encrypted results are sent back and you can have the same sort of validation process that you would have in other systems where you compare the results from multiple independent workers and if there's any discrepancy in those results then you know that one of the workers actually produced the wrong result and there's ways of resolving that and punishing those workers so intuitively that's how it works there's a lot more details and those will all be in the paper good question I think we're out of time I'm gonna hang out here for a little while if you have other questions yeah this room is empty so if you guys have questions for them you can just come up and talk in person thank you guys