NFT collections at scale - Marius Balaj + Vlad Temian | Sandbox

[Applause] hi hello everyone my name is Marius and together with my colleague Vlad we will present you how we scale nft collections when I first applied to eat well grad I made a promise to the organizer that I will wear a leper Brenner shirt which is a promise that I kept this is the leper Brena sandbox version yeah I have to admit that I feel really welcome here and it's not just because the organizer did such a great job but most of my memories as a kid are related to Serbia while Lepa Brena was always on the radio I really enjoyed the cheaper chocolate also I want to announce that two of the best questions from the Q a session will be rewarded with the same shirts as us a bit about us we both work as tech leads for sandbox and we also co-founding QED which is a software company that was trusted by sandbox to get things to the next level our journey started two years ago with a crew of four developers and now we are over 25 people in the team our office is located in Romania in the city of timisoara which is also the European capital of culture this year so if you plan to do a visit feel free to reach to us we can show you some cool places our the first nft collection that we created was the doggies which is the sandbox official collection for Snoop Dogg this was a real hit 10 000 nfts sold out in like 20 minutes this generated a huge demand for more and more sandbox official collection but also with the increasing popularity of nfts we had to integrate pre-existing nft collection into the ecosystem for example we had to integrate board days into the game so you can play with your avatar we we started with one collection every two months and we had to launch up to four collection per month however our previous infrastructure was not capable to handle this level of demand this presented three major issues but first let's take a brief moment and Define what uh an nft collection is in the context of sandbox an nft collection in the sandbox is made of three parts it has a web application which is the minting the app the smart contract which is an erc721 and the art integration which is a process that blood will further detail for all of this we hit some bottlenecks but let's take a little look into the web applications the web application for the nft collection is made from two parts one is the landing page and the second one is the minting page which sells solely for domaining purposes for the landing page uh we we have to the marketing team has to use it in order to promote the collection highlight the utilities and uniqueness of the collection and needs to be customizable by customizable I mean it should have the look and feel of the the client as you may see on the slides for example the Paris Hilton collection was uh very very bright and pink moving on forward as we creating more and more collections we begin to identify mechanics and components recurring on the pages as you might notice all of the sandbox collection follow the same structure all of them have characters on the page they have a section where we Define the trades they have a trailer and so on in order to speed up the development of these Pages we extracted all the recurring components into a small UI Library a component kit this component kit provides a set of components that are accessible and mobile ready and they can be used to bootstrap Pages really really fast the only thing we have to do is apply the Styles add the content and we are done also one good thing about this component kit is that ensures consistency between our collections okay now the problem with the development was fixed we could develop Pages really fast but we were still tied to the main sandbox application which is a huge monolith with a lot of Legacy as as you may expect it has its own lengthy release cycle so even though we managed to create the page is really really fast we could not release as we had to wait for the domain application to be released in order to fix that we created a micro front-end architecture so we extracted we extracted all the collection into micro front ends with its own CI so can be independently built they are autonomous and they build into a static files so that means they can be ready to deploy on ipfs okay now that we uh create really fast we can deploy every time we want we can release very often but that's that was not real fun as we were left to with duplicating code from previous collection when we wanted to create a new collection we just duplicate change some configuration file uh update the Styles update the content and we are ready for production so in order to to fix the duplication issue we created the CLI app called create sandbox collection which was highly inspired by the famous create react app which ties everything together and using this tool you can bootstrap collection in a matter of seconds as a side note this works only for the sandbox collection now the developers were happy we could we could release uh and create and build in a fun way but our users were not quite there as a data-driven team we place great emphasis on analyzing user behavior during the minting period by checking that data we managed to find some really interesting things for example we fans find some stuff that cause frustration at some point even the stoppage of the minting process one key feature that we develop in the new user experience was the transaction history list but to understand why this one was so important I need to give you some context during the early days of sandbox everything was deployed on the minute and it was the good old days everyone wanted to meet this caused the network to be congested and also the transaction resolution time to be very lengthy so let's take a look at the left GIF the user clicked the button to Mint they see a loading spinner and a loading message this could take up to 15 minutes then they don't know what happened they refreshed the page they see the button again they try to Mint again this caused unintentionally multiple means so we solved this problem with the transaction list when you create a transaction you you we see the transaction in the list with the status in real time even though you have a pending transaction and you refresh the page you could see a small warning that you have opening transaction and if you want to Mint again definitely you could do it another pain point for our user was the way we displayed the errors which was one at the time as an example let's uh say that we are on a wrong chain and we don't have enough metric rate to pay for the gas fee to mean the nft you will go to the page in the previous ux of course you'll try to to Mint you'll see that you have the wrong chain error you have to manually change the chain then refresh the page then try to Mint again and you see that you don't have enough magic to pay for the gas to fix this issue we proactively display the errors and also we added call to action to switch the chain automatically press a button you switch the chain and you're done okay now that the web application problem was fixed we encounter the same issues but on the smart contract area in order to create a new collection contract we had to duplicate some code from the previous collection so we duplicate the code we change the name we change the supply and then as we place great value on security we send the code to audit we wait for around two weeks we get response if they request changes we implement the changes we send back to audit and we have to wait this caused this delay caused us to miss almost Miss some of the deadlines in order to fix that as all of the sandbox Avatar collection are the same we extracted the code into a generic contract and when we create a new collection we just extend that generic contract we pass the correct arguments and we are ready to deploy the audit time was the auditing was made only once for the generic contract and of course if you want new feature on the contract we have to send it to the audit again but this is scaling really nicely but I will let Vlad tell you more about this thank you right so first I want to mention that everything we are doing on the smart contract development side is open source and it's free to use so you can go to our sandbox repository in the GitHub and take the the smart contract deploy it and you're gonna have a Sandbox collection now as a side note we also did some open source contribution for the openc registry filters so all of our generic contracts are compliant with the openc royalty systems this generic raffle generic contract approach works if you have two three four collection per year but if you want to deploy even more and more and more collections uh it's getting heavier and harder to manage you need a technical person to deploy the contract you need somebody to manage it so on and so forth so for that right now we are in audit with the third generation of our contract based on the factory pattern and Beacon pattern and the holidays that you're gonna have a UI a back office in our admin area and our non-technical staff can deploy create and manage the collections as they see fit the last problem that we encountered and try to solve is the one of Arc integration and art scaling in order to create three four collection per month and to bridge more than half a mill of avatars per year you need a system that can integrate generate and manage the 3D models 2D previews and the trades and the metadata of those avatars to give you a little bit of a context of why this problem is complex and complicated uh let's talk a little bit about about an avatar Anatomy inside the sandbox ecosystem inside sandbox we uh as an avatar is composed of three main components it has the 3D model which has two sub-components has the gltf and the glb files that are available on different model viewers in browsers in AR apps VR apps on and so forth but you also have our vxc proprietary model that is playable only in game and is highly optimized for voxels the second component is the 2D representation of that 3D model those two DRS the pfps the images the jpegs that you're going to see on openc on Twitter and use them as you see fit and the trades are the text representation of those visualizations when we started last year it was really a pretty brand new territory for most of us we didn't have the tools to create or Bridge avatars and the tools that existed they were like pretty generic doing pfps or doing all the type of collections but for us we have this hard dependency on the 3D models and the voxels themselves more specifically we have this problem in which you need to map a certain trait to the 3D models so to give you an example here you can see a golden bordeep and if you want to make that golden board deep in sandbox you need to Define uh skin trait with the value golden and somehow to map that that trait affects all the 3D nodes from the 3D model so in this case it's going to affect the head left right arm legs chest torso everything that's the first solution we implemented uh some stuff in Google spreadsheet tool in which you can Define all the traits you can Define their scarcity it has a randomization algorithm and it has this ability to map the trading nodes with the trades here is an example of the Playboy collection in which you can see the mapping is done really simple and basic with just some axes all the tokens generated and randomized their weights and much more features like description inclusion rules so on and so forth this tool outputs a package that is used by the art team in order to model your 3D collection so now that our trades metadata are solved the 3D our team they developed the collection it's time to generate the 3D models and we did that by using some glucose scripts using Vox edit blender mix and match of everything and the whole collection the first collection were generated on a laptop just clicking and running some some random commands now obviously this does not scale in in our our ecosystem so for that we moved everything that was locally running into the cloud and we did a cloud pipeline that is composed of multiple AWS step functions that underlying they use Lambda Dockers containers certain processes and those components you can mix and match them as you see fit in multiple workflows and those workflows are gated by some AWS Gateway endpoints API endpoints now on top of that we have a small UI so every single non-technical person from our staff can generate and operate a collection here is an example of Paris Hilton being generated first we receive the package from the r team we ingest that in the world pipeline then we generate the 3D models in parallel real efficient with the caching mechanism so if the r team wants to change just one trade or just one model of of the the collection only the models that are affected but by date that change is going to be generated so not the entire collection second step is aggregating the previous aggregating the models generating then the previous still in parallel still using the same mechanism this time using some more powerful gpus machine and finally packing everything together and inserting in our storage I'm glad to say that this tool was used by our non-technical staff to generate over half a million of avatars in less than two days without any technical interfere our next steps right now is is working are working on uh Solutions that's gonna bring everything together that I mentioned so I integrate that one into the sandbox so we're gonna be able to create your own collection Define the traits generate the models and have the entire collection in sandbox just by filling some forms and pressing one click this collection is going to be available for our landowners this tool is going to be available for for the landowners and yeah it's still in development is not and is not going to be just for Avatar is going to be for different kind of assets Like Houses cars whatever you you imagine right that was nft collection skills our goal is to build tools for our team to to handle collection a different skills so I think we are still working on on that but uh hopefully we're gonna Empower every single Creator to use these tools thank you everybody and now if you have questions let us know yep is this on can you hear me yeah okay um so I might have missed it but did you guys say where exactly you're storing the metadata so are you doing it it's off chain of course but is it on ipfs or your own servers or what yeah yeah so uh it's a good question for avatars right now in this moment is on S3 both and for assets is on ipfs but we have plans to migrate uh those to to ipfs as well we are doing that mostly because we were last year experimenting with Dynamic nfts and changing the metadata was trickier to do it last year hopefully this year we're going to be implemented also uh on Jing cool okay and then one more question on the same topic so it's really simple to write the metadata and then of course set your URI and chain but like State expiry is a core part of ethereum's roadmap and so more and more metadata for nfts is going to need to be off chain but inevitably we're going to encounter the need to consume this metadata on chain right it's like let's say example for this board a if it's got its gold glasses or whatever off chain have you guys done any work at all with pulling off chain metadata on chain so like using chain link or something uh not yet yeah it's pretty complex but it seems like everyone's gonna have to do it and no one's really talking about like it's kind of just hand wavy oh we're gonna put all this off chain on ipfs or whatever but if there's any hope of actually having utility for all these the meme pictures and it's inevitable yeah yeah well I'm glad to discuss more after that yeah I'm really really all right thanks for your time thank you hey how exactly do you go about securing the whole protocol like when you deploy a new collection yeah do you like have multiple rounds of like extensive auditing like which forums do you use and what is the whole process yes we use open sapling for auditing uh we do auditing for each contract we develop but at the moment our collection will run on the generic contract and this one was audited by open Zeppelin we implemented all the request changes and so on and yeah that's it and thank you hello great presentation by the way uh I have a quick question about so you said that you guys kept your like data in sheets before you guys kind of made those Cloud functions to kind of ingest that data so we're also having this problem where sheets is kind of unproductive but non-technical people it's just a lot of data and it's kind of messy so what did you guys find a way to like move this into like a better kind of way to ingest into these Cloud functions because that's the kind of similar thing that we're doing and having issues with uh so I don't have here this slide so yeah uh we are working on a solution to to do that to integrate everything uh but yeah it's quite in development right now so we don't have like a fully fully flight solution for for that yet Google spreadsheet was really easy because you can then go and have like this collaborative environment in which yeah somebody can redefine the trades can change something you see the previews we also have another system for our IPS to validate the the art so it was quite convenient but it's not as you as you mentioned it's not manageable and we are working on I'm glad to uh to show you after that cool thank you phase one in the back hi did you test the user experience for someone who doesn't own a wallet for like a normie to buy an nft yes that's the thing I uh completely missed so at the moment uh yeah we are we have a team of QA and also we are doing a b testing with web 2 users and we are experimenting right now with card payments and wallet-less accounts so that's the thing that we try to do in order to ease the on board of web 2 user into the web tree do you also have a user experience designer on board yes we do yes thank you you mentioned you have walletless users I'm curious about how you manage those is there a custodial or audio dimension sorry I didn't get the question so walletless users yes how do you manage those is it a email sign up do they have a custodial wallet yeah emails sign up that's definitely first you cannot meet without a Sandbox account which is usually linked with a wallet and an account but we are doing research in that area so we are using a provider so that someone that will create the wallet and then then you can claim your nfts and we're still in I think two or three providers we are testing right now so and for our sandbox account you can create the sandbox account without the wallet but you need to connect then and we have another providers for for that so we don't manage those data yes that's it we should pick the yeah we'll meet after after that around here and we have the t-shirts yeah I really like the security one and you should choose another question yeah I will I will figure it out okay right good thank you and see you around thank you thank you