Going on Safari: Researching Users in the Metaverse by Georgia Rakusen | Devcon Bogotá

foreign [Music] user research and has a specialized in the web 3 and crypto space for the last five years she works for tooling wallets D5 nfts dials and gaming clients to help to to help them understand their users and build better products and establish the user research Dao a collective of ux research experts dedicate to elevating the quality and quantity of user research being conducted in the web tree space so welcome and please give it up to the speaker [Applause] thanks for having me if you can hear me okay yeah cool all right I'm Georgia rackerson I'm a user researcher thank you for the great intro um I'd like to get a quick sense from those in the room who has played in the metaverse as a user oh quite a few okay and it's anyone building in the metaverse here good all right both excellent um so I'm going to share some of the research findings from a recent study that I've conducted into the experiences of early adopters in the metaverse and then what happened afterwards um and just for clarification when I say when I use the word metaverse and I did a quick word count I'm going to use it 61 times in this presentation I'm specifically meaning the web 3 enabled metaverse experiences that exist right now things like decentraland crypto voxels spatial the sandbox Etc not the non-web 3 enabled metaverses um things like Minecraft Roblox VR chat um metas Horizon as well okay great so a little bit of background for you um earlier this year protocol Labs commissioned a big study on the experiences of metaverse users and the objectives of This research were to uncover the characteristics of early metaverse users identify the meaningful experiences and user pains surface opportunities for platform improvements and deep dive on themes of identity ownership and portability so we interviewed 16 users all existing metaverse users in these kind of in-depth one-hour interviews and the research is now at the survey stage so and the full results will be made public available later this year but I'm going to share a few learnings from that study with you so from the research we mapped how people first encountered the metaverse and What affected whether they became a regular user or not and early adopters fall into like four main camps they're either just inquisitive people they want to get ahead of new tech Trends it's really up their street creatives people looking for new outlets for work investors those are folks who are maybe already in the nft space and people already working in web3 and the triggers for getting started in the metaverse really vary for these different people so it could be that they've introduced to it by another person that they know hearing about the term in the media or being driven by the opportunity to make money so though nfts go hand in hand with the metaverse um there were some who in in our pool of participants who came to it totally fresh they had no knowledge of web3 or nft and the metaverse was their first web3 experience and whether users continue once they try it depends on whether they experience a wow moment early on so wow moments might be social engagement or realizing the creative or earning opportunity these users move pretty quickly then from just trying it out to becoming quite active but some people have really negative first-time experiences they might be put off by the quality of the graphics if you've played it you may have a sense of that or from not discovering positive social engagement those users just don't come back so there was an assumption going into this research that um all of these early adopters were going to be Gamers right gaming is the really big use case for uh the metaverse um we're not there yet gaming in the metavers just can't compete compete with the web 2 experience um and early metaverse adopters are quite likely to be in a stage of life where they have less time for serious gaming so a lot of early adopters are millennials so some of them actually have children who are playing Minecraft right now give you a sense of who these people are and they have other priorities other than grinding in play to earn games so a lot of them had never played any play to learn games but their Nostalgia for that formative gaming experience and Nostalgia for the early internet does impact how they perceive and react to the metaverse so gaming in their youth during the emergence of um online forums in the early 2000s I'm sure some people can this resonates with them it's prepared them for what actually being in the metaverse is like now the convergence of 3D virtual worlds and social media and this user spells it out great to me the metaverse is the ultimate social platform it reminds me of AOL chat in the 90s but now you have visuals and tokens attached to it it's an improvement on life so those who are looking for AAA game quality are not going to find it right now and they are disappointed in the graphics it's enough of a turn off um and you know and the playability it's enough of a turn off for them just not uh not to come back ever again um so it's likely that committed gamers are just not going to be into this space until these um experiences vastly improve so from the pool of people that we spoke to if it's not gaming that brings them what is it the ability to find your tribe and create shared connections is really really powerful this user was a DJ who ended up quitting his full-time desk job to work in the metaverse he said I've spawned a lot of meaningful relationships it holds tremendous value to me it's stuff I wouldn't be doing in real life that I've always long to do it's hard to go out there and make friends in my job nobody shared the same interests as me the metaverse is an easy way to focus in on a group that has similar interests and then continue those relationships even outside of the metaverse so social connections make the metabus really sticky but hand-holding is essential for that positive first experience so we heard that if users have someone to show them the ropes right at the beginning they're more likely to work through the hassle of getting started which might be overcoming reservations that they have about it in the first place working out how to move their Avatar around all of that kind of The Growing Pains really of getting started and they end up on this fast track to those wow moments that keep them coming back again and again for people without um anyone they know to hand hold them they can feel lost and it's unclear to them where they should go and what activities match their interests so a question for those who are building metaverse experiences right now how can you fast track users to those meaningful experiences um and is the hand holding you're doing now optimal the path from Explorer to full-time Creator in the metaverse is really short I was surprised how far some of these people went from like zero to I'd quit my normal job and now I work only in the metaverse there's quite a lot of them and that's Testament to the success of web3 as a way for people to monetize new ways of being creative and it's also Testament to the fact that metaverse platforms are showcases for what can be achieved so this user was a professional artist in real life who's invite invited to display their work in the metaverse for the first time and he said I could see my art virtually on the gallery walls it looked stunning this would cost ten thousand dollars to display in a gallery in real life and here I am doing it for free a collector had questions and everyone in the space could read my answer I only had to answer one question once at Gallery shows I have to answer the same questions 70 times and still make it seem like it was the first time the question was asked that for me was like the golden wow moment but there is a challenge of course so many of these people have no prior web 3 knowledge so they have this really steep learning curve in order to onboard learning about wallets nfts a whole bunch of new Concepts so how are these platforms actually supporting web 3 newcomers with all of this learning so as I was writing up the findings for This research I reflected on my own personal experience of having used the metaverse and honestly I'd only used it once I'd dip my toes in I think it was for an East Global event like some years ago I tried it I remember feeling lost not really feeling like it was for me being unsure what the etiquette was and I I kind of found the experience a bit scary but after doing these interviews it got me wondering could you do research in the metaverse could you study user behaviors motivations and culture by immersing yourself into the virtual world so I had some initial reservations and for those of you who are researchers in the room you'll be thinking well usually when conducting research you try to take yourself the researcher out of the situation so that you are not influencing uh what's going on around you um thankfully a much bigger brain than mine has written a lot about this topic so I consulted the Bible on netnography it's called netnography by Robert kazinitz so netnography is qualitative social media research it's an adaptation of ethnography which is the study of people and their behaviors but instead of going to physically to where they are all the action happens online it's focused on human experiences and cultural understanding not product ux research is not the focus for it and it's informed by a sense of self-awareness of the researcher it's not spying on what your users are doing in public forums watching from a distance it's not that it's researching at the same time it's actually participating in the culture you can think of it like action research so it involves watching what people do online documenting what's going on collecting lots of data where that could be screenshots links anything video as well as immersion by the researcher into the space that they're researching and reflecting on their own role in that process as well and then interpreting all of that information and answering research questions now most of this book which I do recommend it's like quite a big book uh it is a good read though um most of it explores really this method around traditional social media there's very little reference to Virtual Worlds however someone has done their own netnographic study of Second Life um the author Tom bowlesdorf spent two years doing it I realized I didn't have that much time so I just wanted to get a taste for like what this method was like and whether it's useful for other people to do too so what does it look like in reality so the researcher keeps field notes like a journal and whenever they're observing or immersing they take notes with a focus on social behaviors and the researcher is writing what's happening and how they feel and they reflect on their own role in the experience as well now there are some very important ethical considerations for doing this type of study there's this concept known as the consent Gap um if you think about it all of us do it many people are always sharing information about themselves online publicly you know posting on social media but just because they do that doesn't mean that they're consenting for that their data to be used in research so um an ethical ethnographer needs to make their intentions known to the people that they're studying you can't go undercover basically another thing I needed to consider was that I'm studying web free people and I know that they're really privacy conscious so I needed to be extra careful about what I share in my research findings not everyone I interacted with said yes please share screenshots of the tweets I sent you so um so I'm being overly cautious on that front so with some trepidation I set up a page for myself on notion uh describing the re what the research was about I would then add the link to that in my bio in the in the metaverse platforms and direct people to that if they wanted to know more I updated my Twitter profile to say what I was doing and then I began observing and I started on Twitter Reddit um Discord following threads I dipped into different um metaverse platforms before finding any finally ending up in decentraland so I created my avatar there she is in order to um adhere to the netnographic method I made sure I looked somewhat like myself and I also called myself g-rack researcher just so there was no ambiguity where people would recognize me and I spent many hours doing this work I kept a journal and what I'm going to do now is read some excerpts for you I went to the decentraland events page chose a game and teleported there there are about 30 or so people their avatars all stood around in groups but not appearing to be doing anything I tried to work out how the game works which was tricky I wonder how everyone else Managed IT I felt a bit stupid and wondered whether I was the only new person to the game without any helpful prompts by others what is the experience like for other new users I engaged in the text chat it seemed people were up for surface level conversations but when I asked a researchery question I had no response which felt very disheartening it made me feel that my Approach of asking people upfront questions was wrong I noted how others were welcoming people they knew using web 3 signifiers like GM to one another looking around everyone else was in very flashy outfits and I began to feel self-conscious that my chosen normal looking Avatar was some sort of signifier to others later I followed the coordinates someone posted for a live music venue I teleported there and the atmosphere was immediately totally different I entered the venue and there was a live band I hadn't heard of playing streaming their performance into the venue my name g-rack researcher sparked some interest and people asked me what I was researching I noted in the text chat that some people share their real-life locations I think this is a this is a grounding to the experience and a wave of people to find commonality looking at what I'm wearing compared to everyone else it seems dressing to be a likeness of myself is unnecessary for netnography in web3 a player's name feels far more important for Avatar recognition just as in real life you can't just jump in with questions for people you don't know you have to do some social lubrication to get people to warn to you after participating in the rituals of wishing people GM even though it was my afternoon and welcoming others by name to the event space people began to ask me questions and this became a great opener when I explained what I was working on people were very positive towards it and offered to connect me to other people who might share their experiences in my earlier observations I saw how sharing screenshots of activities in the metaverse on social media is a really common activity while I was at the event the band encouraged people to tweet screenshots so I went ahead and did so after tweeting I got a vocal shout out from the band that felt very validating and enforced a sense of multi-platform immediacy I noted how this multi-platform experience requires a lot of multitasking I wonder how many browser tabs all the other users have open once the event finished someone in the text chat told me to check openc for a show app which I'd not heard of before thankfully they told me where to find it in the openc interface as otherwise I wouldn't have known I didn't ask this question they preempted it which suggests to me they've helped others with the same problem this person is a guide when I found the nft just there it was gratifying a souvenir from that moment that came to me automatically a question I began wondering how did the band experience the the events from where they were streaming live so as my emotion progressed I brought some manner I jazzed up my outfit a bit ready to fit in a bit more um I definitely upgraded the look and I did try a few different platforms but I kept coming back to decentralam because that's where most of the social engagement was and this study is about people so um that was why I came back to decentral and for the purpose of trying out this method so on to some later Journal excerpts a couple of days since spent playing in The Meta verse feels like a long time I wonder why that is I think I had such a welcoming experience from people here so I feel like I'm part of something that it is now for me when previously it wasn't I feel bold confident I visit a promotional bar there isn't much going on there it's sort of a liminal space with no obvious Focus there was a squid-like bartender which I interacted with but it was a chat bot and that was really disappointing to me interacting with it didn't really support my goals as a user but then I stumbled across another player whose first time it was and they were lost they wanted some help understanding what to do I told them about a music event coming up and Bam I felt like I'd instantly leveled up from newcomer to experience player The Apprentice had become the master except I wished I could take them around to venues with me the best I could do was post coordinates in the chat for the music venue wouldn't it be cool for us to tether together in order to travel as a group between locations I accidentally clicked on my own coordinates in the chat teleported off and then couldn't make my way back to them hopefully they found their own way around did I feel a tiny bit of responsibility for their experience I wonder how I'd feel if I were in their shoes and the person giving me advice just disappeared and never came back I went to the venue with the Dance Floor there was a DJ um live streaming I saw some friends and they called me out in the text chat it was nice to feel so welcomed I'm thinking back to other online music experiences I've had could this be better than watching a boiler room set at home you could just set up your avatar dancing with all these funky looking people and get live shout outs from the DJ and sit back and enjoy it I can imagine putting this on the television on a Friday night with some friends in the house instead of a Spotify playlist I wonder what everyone else is doing behind their screens at home whether much in-person social engagement happens simultaneously as in the metaverse the next morning I received a direct message on Twitter thanking me for my participation in the event and asking for a wallet address working in the space I get a lot of random DMS related to crypto and nfts but I checked out the profile picture of the person who messaged me and recognized them from their Avatar made sure I checked their profile I replied with my wallet address and I was excited to find what might be dropped to me it turned out to be a wearable it feels to me that participating in the metaverse is like a massive multi-platform treasure hunt so another time um I decided to begin my journey on a Twitter space the hosts were talking about spatial a platform I'd visited before but decided not to spend my research time in it because there wasn't a lot of social engagement going on so on Twitter two links were posted to a parcel of land in spatial on the left um that one of the speakers on the Twitter spaces had been working on and a venue in decentraland streaming the space in spatial So I entered both spaces and was able to have my avatar in decentraland watch my avatar in spatial whilst I was invited to speak on the Twitter space I was in three Worlds at once and it really got me thinking about interoperability of avatars and what use cases there might be for how wanting to have multiple versions of yourself to across different platforms it also really reinforced to me how early adoption is that so much of participation in these platforms appears to be about demonstrating what is possible so I conduct conducted my Explorations over about 10 days I spent about 15 hours trawling social media about 10 hours 15 hours in the metaverse itself and really only to test this method out to test the value of doing it I didn't expect to find out anything very certain in this period of time but I did come away with some reflections so you can learn stuff about people this way and I think it's really interesting that the engagement level on social media definitely feels more prominent it's busier than actual platform engagement and I know there's a lot of discussion about active daily users on these platforms being really low I did find lots of people I also found lots of the same people on these platforms um if anyone who works at decentraland can give us some insights on their daily users that would be interesting but I I think we have to see social media as an extension of the metaverse because it's part of the experience for it so Twitter twitch Discord openc telegram all of these form part of the metaverse experience and that poses a question is there value in folding in these social networks into metaverse platforms or should they exist outside of the 3D World so um as a place where culture can be observed tribes rituals and symbols are everywhere in the metaverse the sharing of screenshots seems to be its own art form there are people who are photographers who do this and that's what they um they get group photos of people together um there are so many rituals and symbols celebrating and welcoming one another the use of web3 language the symbols in wearables and even Avatar movements uh gifting items supporting new people with technical challenges all of this good stuff so if you're new to web3 culture immersion like this is a fast track way to get to know web 3 users but for the already initiated if you've been in the space for a long time like I have it may not be as fruitful it certainly reinforces what we know about web3 users um but I suspect that the culture in the metaverse will change I'm interested in the questions around how web3 culture now affects people who are outside of it what do they think when they join the space um and what new cultures will move into the space or emerge as adoption grows um I thought I thought I was being really clever and put loads of questions together that I was going to like Grill people about in the metaverse and that was totally the wrong approach to go for um they were all the wrong questions and it wasn't the right format good questions really only emerged after spending time in the metaverse and the netnographic method does cultivate very deep empathy with users sometimes it's hard to remember that you are studying other people and it's easy to slip into the mindset of I am the user um and you could end up just performing a ux audit you know just writing a journal of all the problems that you find as you go through but it's really it is about people um it's about observing people not just about the platform however I do think it can be used to identify opportunities for platform improvements the questions that I came up with as I was going as I was journaling on my Explorations point to those um certainly it can uncover good research questions for further inquiry so if you're building in the metaverse space I encourage you to get close to your users don't rely on just Discord signals to plan your feature roadmap like please don't do that method methodical there we go about your approach to collecting data right and triangulate with multiple methods that's really important interviews observations and immersion too and if you want to learn about emerging online communities in general virtual worlds are a great place to start so very lastly if you're a user in the metaverse then I really want to learn about your experiences to support this ongoing research um and the final report so please scan the QR code complete the six minute survey potentially win some usdc um and I really look forward to being able to share the full published results um gracias [Applause] I think we have time for one question oh excellent so proactive hi Georgia we talk I was wondering how could you keep your user in check your inner user in check and keep on being a researcher while you are immersing yourself and you were deep diving into this universe um I think it's the journaling I was doing I was doing the journaling live I wrote thousands of thousands of words as I went collecting like every data point I got like hundreds of screenshots as well that that keeps you on your toes to remember that you are observing the experience as a researcher and not just as a user otherwise you just go about you know you cover uncover a problem or maybe a question might enter your brain and then leave again because you're not documenting it so the the important bit is the field notes it's the documentation of the process