# Decentralized Federated Learning on the Oasis Network by Dawn Song (Devcon 5)

- Channel: [Devcon](https://streameth.org/devcon)
- Date: 2020-10-09
- Duration: 18:41
- Topics: blockchain, smart contracts, distributed ledger, ethereum, Science & Technology
- Watch: https://streameth.org/watch/yt-pZ3Oz7xfNAM
- YouTube: https://www.youtube.com/watch?v=pZ3Oz7xfNAM

## Description

Visit the https://archive.devcon.org/ to gain access to the entire library of Devcon talks with the ease of filtering, playlists, personalized suggestions, decentralized access on IPFS and more.
https://archive.devcon.org/archive/watch/5/decentralized-federated-learning-on-the-oasis-network

Machine learning is being adopted more and more broadly in technology. Such success is largely due to a combination of algorithmic breakthroughs, computation resource improvements, and the access to a large amount of diverse training data. The collection of data can raise concerns about siloing, security, and user privacy.
In this talk, I will highlight a new blockchain-based machine learning technology that allows users to share their data, train models in a fully decentralized way, and incentive end users to keep their data on the network using the Oasis network. This technology, called HiveMind, leverages a federated learning framework to reduce overhead both in communication and computation. In addition, the talk will highlight the benefits of a novel blockchain-based secure aggregation protocol that ensures client-level differential privacy, and thus prevents information leakage from trained model parameters.

Speaker(s): Dawn Song
Skill level: Advanced
Track: Developer Infrastructure
Keywords: academic

Follow us: https://twitter.com/efdevcon, https://twitter.com/ethereum
Learn more about devcon: https://www.devcon.org/
Learn more about ethereum: https://ethereum.org/ 

Devcon is the Ethereum conference for developers, researchers, thinkers, and makers. 
Devcon 5 was held in Osaka, Japan on Oct 8 - 11, 2019.
Devcon is organized and presented by the Ethereum Foundation, with the support of our sponsors. To find out more, please visit https://ethereum.foundation/

## Transcript

[Applause] and the funky CEO as Isis loves and also a professor at UC Berkeley and today I will talk about the decentralized petty tyranny so first as we all know Peter is the new oil this data is really helpful for us to make better decisions and give us a lot in fact hi a lot of this people is really is also very sensitive and had not any sensitive data protists a lot of challenges so for example we continue to see really large-scale data breaches where attackers compromised sensitive and still sensitive information about hundreds of millions of users being in a one in a single account and also we continue to see that users are using trusts incomplete as a lot bigger and maybe mishandles abuse and so as we handle sensitive data there are a number of challenges that we need to address one as the English actor maybe untrusted attackers may have compromised the computer infrastructure are the advocates not only be untrusted because the again attackers may have may have launched some malicious applications other applications may have poor abilities and also some applications may have undesirable deferred leakage so to address these problems at oasis labs we have been developing new solutions where we essentially builds the next generation Block in our contract platform with British scalability and privacy protection where we unique or skill a postmark on to execution and also enable for the first time confidential smart contracts and also on top of the platform we've built SDKs to enable privacy a machine so today I'll talk about some new research that we have done at every Clarion our voices with decentralized afetr children so Christmas book has a motivating example and again frantic action so one banks provides financial services for example immigrants the lands they need to detect fraud and typically how this works is that each Bank uses its own data to build for example some agility model for fraud detection poverty as well as we know that it would be much better if these banks can actually share that it had to build one traffic picture instead of the individual ones the case with Mozilla the machine imago can essentially be more practice I witnessed another today because there is a lot of privacy concerns because it is excessive and also there may be missed so luckily address this problem so there are alternative solutions that we can consider so II with one admissible solution we can use something called a feathery learning so our facility is mechanism we've done it's an approach that actually has seen some adoption deployments I could go and Apple for example we're essentially using fertility and usage data are stores locally on their mobile phones and then the central server at Google are a broken coordinates these different local devices to collectively train a machine any model in this distributed fashion so in this case with ready tyranny it takes an iterative approach where in run time it has very high bridge in a fleece global global model that's that is being trained and then this model is distributed to the local devices and and then each local device uses its own local data to try to update this model gives more we can't really update and then the sweetie updates is then sent back to the central server to then the listener terminal advocate studies with the updates to the create to the next version of the mother and then this continues so in this way the advantage is that the each user's data is stored locally is not actually uploaded to the cloud out to the central server but still in this case it still needs as inter server to coordinate this ferry terminal and there are of course a number of challenges about which might be use this type of approach to address the problem that I just mentioned for example in this case when the banks wants to collaborate the questions and one question is the who should actually hooks the center server ideally you don't want to rely on using be trusted so you don't want to actually have and you are hosting the central server and also this is a new survey by rely on accessibility and creates each year spreads at home if they became malicious are taken may also give the single server some some unfair advantage so in contrast to this family tyranny motto which realises that your server to coordinate distribute the devices to together train machine is our winner distributed way and that alternative solution is was had a peer-to-peer solution in this case essentially the different devices form a peer-to-peer network if there is no central server and essentially the the different devices will communicate through this peer-to-peer network with its neighbors and then through again an interesting approach to try to together updates this much mini model to utilize the local data on the different devices to train the to train the sandwich minimum I would advantage of this approach is that in the city we don't need to trust any central server because there's simply no central server however the challenge here is as in dispute appear the significant results and for our communication as on and especially when Anna Raph the client becomes flourished and also there is more privacy challenges Elizabeth so in our recent work here we proposed a new paradigm that we call decentralize the 2020 with where we combine is into the best of both worlds both giving and this virtualized setting while we don't rely on a nice central server but also at the same time we can achieve much better efficiency and dementia stronger privacy protection than the peer to peer said and the way we do that is the instead of using as intrusive a wastage we use a blockchain smacking telephone to coordinate this distribution meaning across the different local devices in different coins that would you host the the pocketing using this new paradise we achieve a number of advantages once again we ended with the centralized and the secularization so we do need to trust and listen to party for the operations and against much more efficient and community scale to large number of participants compared to the peer-to-peer solution and we can achieve we're high utility essentially the accuracy can be comparable to the centralized model and we achieve strong privacy where we can protect the clients data privacy and by using drugs an entire platform that wasn't me incentive method so now let's take a look at how this approach works so first let me give you a strongman solution whereas emissions want me to achieve this decentralize the federal sovereignty as we use of lotteries my contact upon to coordinates these different clients for training the marina model isn't a distributed decentralized way so one a stronger approach is that we can simply set have asthma contracts under party platform which performs a mother aggregation as I mentioned earlier in the feathers it's been exciting where essentially this a modification smart contract will perform what the central server would do down life but however so the child in this case it can be very efficient and also provides strong integrity guarantees even industry centralized the city where we don't need to actually rely trusts are at the center server Harvard the China here is they using this approach there can be privacy concerns because on a typical protein smacking her perform the protein dinner is public and even though in this case the the clients are now sending location to the got a platform but however even these screening others for it were updates in motion any models they can see okay information so the question is how can we do this in in a more appropriate way so to adjust these challenges we proposed pipelines to enable this new paradigm up he centralized effectively happened essentially you utilizes a number of private secretively technologies including adventure privacy where we actually differentially comments virtually numata and using secure aggregation where the mother updates has been sent to the blockchain smart contract upon actually will be in an encrypted form and the way a new secure application to do the mother aggregation and of course in this case also to protect privacy we we ensure that the mother is equal to so how this works is that again we take an iterative approach and in this case in particular we actually have we have enabled this as essential as a materially mother always is fourteen perform so when the animation are here oasis party platform actually may pose computational contract where the stock contracts States is actually even perform and we utilize secure computing to enable secure computation our execution is not contracts well the my contract state is stored in an encrypted form so essentially we kind of mind that we utilize this capital to smart contract primitives where again this iterative approach at each interior iteration the Machinima models are happening between if you view it as as part of the state's our confidential smart contracts well we can't hide mites and as each it erodes the the secret input the mother is sent to the local devices we do handle some key management so the the model then gives decrypted and the local device on the local devices and then the clients uses its own local data to update the local model and then it computes that really athletes and then it also has differential privacy in perturbation to ensure that they the nest authority creates evening activity trained as eventually private owner and then this Brittany updates with the differential privacy annoys Alice is then again at encrypted and the send to the also static route to the highlights of our contracts whether my smart contracts in then combines the screening updated that you measure different differentially planets perturbations and through the secure application and then compute the next iteration after update mode so again by taking this iterative approach within the news this combines combination is my contract to coordinates the distributed dimension and in our training across the different devices to enable decentralize the practice area so now I'm going to give you an example of actual application that we have in the domain of anomaly detection so in this case essentially we can update the head - my contract to enable especially to achieve commercial model for anomaly detection and in particular in this case for anomaly detection we look at the application of a system block another detection so as computers around throughout certain programs other times it produces the system loss and this is perhaps contains very useful information and one can try to train which million others to go over this system now so try to detect anomalies essentially one way to do system of anomaly detection and you think bringing is over here to between a sequence model and then using the sequence model we can then go this is not to try to detect anomalies so so in our case essentially were to use the liberal system lot and then Trinity through you see this decentralized fertility to actually train and update drama on our plasma and we achieve we enable strong privacy protection well as system achieve high utility and in many settings with utility extension which is close to the centralized model so to summarize we have proposed a new paradigm can't be centralized federal revenue where we don't need to Villa and the trust and incentive ready and we provide much British efficiency for training teacher with much mini model then the peer-to-peer setting now with the strong privacy protection and the beauty incentive mechanisms and this the sentiments friendly bringing has been enabled by the ISIS party platform with its unique capabilities to operate a skill ability and also privacy protection so right so with the Oasis Network a week here as I just demonstrated it's the first project smart contractor on that can enable this type of solutions and decentralized and we we hope encouraged everyone to to build the new applications and the Oasis Network and we recently launched the algorithm that 2.0 and also we release a number of new developer tools within with a new developer experience to make it really easy to build applications on top of the platform and in particular also equally new and as we are developing new the capacity is in particular to make it easier for a developer to build our secret applications and we are and we are inviting know the operatives and to join our and our and we'll be starting out a patent team and also we'll be launching a public destination and with the state competition so the speaking competition will be coming in a little interval and we really welcome everyone to to join the sticky competition and again we look forward to have people ask the network and to build new applications that couldn't be built before thank you [Applause]