# DeFAI - Challenges and Opportunities | RegularMarek | ETHWarsaw [4] - Channel: [ETH Warsaw](https://streameth.org/eth-warsaw) - Date: 2025-11-09 - Duration: 28:56 - Watch: https://streameth.org/watch/yt-ZeFdDulRPWw - YouTube: https://www.youtube.com/watch?v=ZeFdDulRPWw ## Description RegularMarek from BlockchainLab explores LMM strengths, flaws (e.g. hallucinations), and how a new generation of DeFAI projects are balancing AI autonomy with safety. 🎥 Recorded at ETHWarsaw 2025 Follow ETHWarsaw on social media for the latest updates! X (Twitter): https://x.com/ETHWarsaw LinkedIn: https://www.linkedin.com/company/ethwarsaw Telegram chat: https://t.me/joinethwarsaw ## Transcript Thanks every thanks everyone for joining us on this beautiful uh summer afternoon here uh Friday afternoon. So, um you know, it's a good it's actually an interesting choice of um >> test. Now it's working. >> Okay. >> And now it works. >> Okay. Well, maybe >> I don't know what was wrong there. Okay. Well, we'll we'll continue from here. So, in you know, in my deep dive um you know, into agents, you know, I'll I'll give I'll give I'll give you all sort of like a preview uh you know, into this Hobbit's journey there and back again, right? So, I sort of explored this agent iceberg, right? And you know, I'll sort of show you uh like a map or a road map to to what that iceberg looks like, the deep dive into agents. So, here's kind of what it looks like, right? At the surface level when you sort of dive right in, right? uh you know you see all this uh you know hype and excitement about uh you know AI and agents and you know the the surface level you know you might say like you know computer's smart right and then as you dive deeper right you might realize that you know actually right um you know these uh these you know LLM based agents um you know they're they're not AGI right you you'll see you'll see me talk and refer to AGI a few times right that's like science fiction AI that's like Skynet right Terminator these things right like strong AI I right so LM's uh you know I guess is what I'm saying here in my journey is it's it's not that right and it's not on the path to that um and then you know I guess you know at at the bottom once you sort of you know explore the whole space you kind of realize wait you know even despite these things even though it's like you know maybe overhyped right it's not everything that you know it's it's sold to be there there actually is substance and and something of value there so it's a little little preview as to our road map right so I mentioned you know DFAI is um you know um uh uh decentralized finance and AI. I think people are familiar with um you know AI sorry DeFi by now. So I won't spend any time really on D5 assumption is you know that that that people know what this is and and actually by the way a bunch of my images and figures here in in the talk are created by uh created by AI like this one that that map I had AI um do and and the AI also did this uh this this uh word cloud which I don't think it did a great job with this one but you know it doesn't matter everyone knows about what DeFi is right so let's talk a little about agents and you know when when I'm when I'm talking about LLM agents you know here here's some of the I I guess you know um things I've noted right through through my whatever deep dive let's call it right um there's a lot of hype around um AI and and LM based agents you can see this from I guess this this you could say this bubbles forming right in the stock market with like you know increasingly high valuations in these AI companies right um and then you know you you you sort of do see at least anecdotal evidence right like I mean the space is changing so fast right I mean if you wait for like you know peer-reviewed academic research, you're going to be like so far behind the ball, it's just unreal. So, I do a lot of my research actually through like, you know, Twitter anecdotes and actually talking to people, right? Primary sources, right? Knowing this stuff. So, I guess what I'm saying here is that, you know, um, you know, to beyond just hype, right? I mean, I think there is evidence that there is, um, you know, interesting things coming out of these these models, right? evidence that you know it's able to do like new new mathematics um new new scientific discoveries right have have uh you know have been made um I would even say you know in you know in in the medical field right I mean I think that there's you know a lot of anecdotal evidence and I think there's actually um this is um you know this is from a study done um you know uh a lot of the time you know these LLM based agents are better the the the human clinician down there is sort of at the bottom not to like knock doctors or whatever. But um you know I think that that the AI is certainly showing and LM are showing it's one area where it can be competitive with humans for sure right the top top AI performance top right human performance bottom left. So I think you know the bottom line here is I think there is some value there right beyond the beyond the hype and speaking of hypeman right Sam Elman the founder of uh open AI was he said this is a joke after right but there's famous Reddit post where he said yeah we've actually solved AI we've got like Skynet in the basement just we're just waiting to ship it never really kind of happened right and it it's actually sort of you know I think as time wears on researchers and scientists are starting to realize this is a a paper research paper by by Apple right like Apple computer Apple right and you know they're sort of saying that there's you know essentially limitations right to like the cognitive abilities to like how intelligent these these models are right so it's not AGI right it's not strong AI and moreover you know again as time goes on you know research is coming out and you know researchers are starting to realize like hey you know these LLMs aren't even on the path to strong AI right it's like you know like I said you know they they they there's some some value there but it's not strong AI another thing that's going on that I think really interesting lately is you might have heard of this uh phrase um AI psychosis, right? Uh when you ask Chad GPT what AI psychosis is, it sort of talks about like the AI perspective of like an AI behaving strangely, right? But what I'm reading about AI psychosis is actually human beings through interacting probably a lot quite a bit with with these models, they kind of like lose touch with reality, right? I'm not saying well probably a bad thing, right? that's happening to folks, right? I actually had AI create this poster in the style of like reefer madness from the 1950s, right? AI psychosis. It's the scourge of our times, right? Um but you know, there there's something real there. And you know, you you you see that um you know, the people using um these AIs and agents is is is only increasing, right? And I think I think at at the when you when you kind of go get to the core of it, right, and thinking back to like very early AI and computer research, like going back to like the 1960s and '7s, you start realizing that, you know, even very simple textbased chat systems, right, have people convinced that um yeah, have people like like uh like fascinated with them, right, and pulled right in in into them. And I think that that that really plays to like these these AIs are very good at hitting people, human beings, right? Right. In the uh this is actually a picture of a painting of narcissists, right? Like like like you know going after people's narcissism, right? I think that that's where this pull for people to like use these UIs or use these uh yeah use these LMS as UIs, right? Um, you might have noticed like the latest version of of of some of these models are very like sickopantic and they sort of like, you know, like kiss your butt, right? Oh, that's an amazing idea, right? Yeah. Like, uh, you know, sausages for tires. That's a great idea. You know, and and if you think about it, I think the reason that came around is that like, you know, you can imagine these these AI companies doing AB testing, right? That's like, you know, putting option A in front of users, option B in front of users, right? You can see that like users like seemed to have liked the version where it was like, "Oh yeah, that's great." Like, you know, we're best friends like whatever, right? And they they kept sort of choosing that one. That's how you know I guess products get designed by committee, right? In sort of a weird uh sort of like perverse incentives kind of way, right? So, but anyway, the point I'm trying to make here though, right, is that um you know, I think that one of the strengths of LM and and sort of LLM based agents and models is that the UI, the user experience is very compelling, right? It's like chatting with like a you know a friend of whatever like your your your your personal assistant whatnot right another big strength um and and and why you'll see uh a lot of the uh you know architectures implementations that you you do see right are you know um you know with with AI and the the rise of vibe coding right um using these agents um using these frameworks and APIs um kind of like you know we'll let a developer entrepreneur whoever right get a get a prototype to market in front of people like very quickly, right? So this this quote is actually from uh C a CEO saying that you know you can replace a team of 50 to 100 with uh with 10 people, right? Suggesting that you can get a lot more efficiency there, right? And then um you know I think the the the other thing I sort of realized, right, and and we'll we'll spend quite a bit of time on on this topic um coming up, right, is that you know, LLM based agents do have quite a few uh trade-offs and limitations. So, what what are some of those limitations, right? I think some of the most common ones or or or maybe the ones that um you know maybe are like the hardest to address and are going to be common to a lot of these agents, right? Is that LMS um use what is called in in AI and machine learning as offline learning, right? Which means that they don't really learn on the fly, right? Sort of like pre-trained in open AIs or Google's like whatever data centers, right? and then then they're deployed and whatever they learned right back then in the training phase is all all kind of all all they'll ever know right um the other thing is um the other big uh limitation right is uh is scaling and scalability training LMS is very expensive which I I just mentioned the training and also inference which is mean which means like you know getting those LMS and models to actually you know like spit out research and spit out text and spit out images it's also incredibly expensive right requires lots of memory requires lots of GPUs and computing cycles and energy scalability is is slowly improving but um you know I would say that these are these are sort of fundamental issues. Um another big limit big type of limitation right which is uh maybe not unique to LMS um but um you know is I would say a fundamental feature of them right is hallucinations and jailbreaking right hopefully I'm not like super old people still know who Mr. Burns is from the Simpsons. Five hands. Some people are smiling maybe. Right. So, um what a hallucination is, right? Is um you know when when when LM or other models exhibit spontaneous incorrect output, you know, given some input, right? You'd expect it to say, you know, 1 plus 1 equals 2, but it says 1 plus 1 equals 3. Made a mistake, right? And these are just sort of things that spontaneously happen. So, if you know this character, Mr. Burns, this is not the way he usually behaves. He's hallucinating here. is he's not does not talk about love very often. And then so the way to think about jailbreaking, right, or the way I think about jailbreaking, it's maybe a little bit different than other people is um you know sort of like an adversarial form of hallucination. It is you know taking some action to cause an agent to experience a hallucination or produce a hallucination. Right? So it's sort of like you know Dr. Nick Riviera there injecting Mr. Burns actually a good analogy and and one of these types of attacks is called a prompt injection, right? So you think he's injecting you know Mr. Burns with this prompt injection and it induces um you know him to have like a hallucination. So again, the these are sort of like fundamental features of um you know, LM based models. And so um you know, yeah, so let's talk a little bit more about you know, this this issue of hallucination and jailbreaking like how what the what the sort of uh uh scale of this issue is, right? So like conversations I've had with DFI like the FAI builders um have privately reported that they experiencing like a 1% hallucination rate in their LLM. So basically meaning, you know, if you have an agent there, you're putting that agent in charge of your wallet, right? Like 1% of the time kind of like un unprompted, not attacked, right? It'll just like make a mistake, right? Which is kind of like a roll of the dice, right? That agent's going to do something unexpected and random. Sometimes it'll be benign, sometimes it'll be neutral. Very rarely it'll get lucky, right? But, you know, there's a good chance of ruin there, right? where that that uh you know agent's going to do some action and is going to like lose all the money, right? Um so uh it actually gets worse than that, right? When you start talking about jailbreaking and um um you'll notice I use Twitter a lot for references, right? Um uh when you talk about jailbreaking and adversarial situation, right, that 1% jumps to like over 10%. Meaning that, you know, I think this this person was more like kind of fintech oriented, right? But but prompt injection is is very um sort of like dangerous for like you know fintech and these sort of financial agents. Um, another good story from last year, right? From from last year, there was a project called Frieza AI um that basically, you know, created an agent and they told that agent under no circumstances should you transfer money, right? And they basically created like this kind of like neat DeFi flywheel to basically keep, you know, increasing this pot of money, this sort of bug bounty, right? And then you know what they found is like I think they they did this like two or three times and they found that people were always able to convince this agent even though the agent says don't transfer the money people were able to convince it to transfer the money which really kind of shows you the potential uh or the ch the scale of this challenge right um here's one one more thing I've heard people talk about jailbreaking I heard people talk about you know uh hallucinations I haven't really t heard people talk about in the context of um you know decentralized financial AI um this concept of um multi- aent systems, right? Which is when you have more than one agent on a blockchain, whatever, some somehow interacting, maybe not directly, um maybe directly, right? When you have so one thing to realize is multi- aent systems are complex adaptive systems and we can spend a whole other 30 minutes talking about just this topic, but I I'll tell tell you what the important bit is here, right? People have heard of black swans by now, right? It's this idea that you know there's things out there that exist in the world that you have like nothing in your experience and knowledge has prepared you for. Right. Right. So it's like an unknown unknown. So what one one thing to note about complex adaptive systems are that they're like breeding grounds for these black swans for these like surprises that you you just can't see coming by definition. That's the definition of a black swan. And so um that's something to re realize when you unleash a number of agents on like the same blockchain, right? and they're like doing stuff whether they're aware or unaware of each other, you can get some nasty surprises. What kind of surprises, you might ask me, right? Oops. Why is this thing it's haunted? Okay, so um back in 2011, this is even before Ethereum, right? You know, back before DeFi, whatever. Um there was a pair of um uh bots, a pricing algorithmic pricing bots that let loose on on Amazon's bookstore. And in the series of days, they bid each other up to $7 million for a used book. And there was nothing really special about this used book, right? So you may say to yourself, hey, this would be great for DeFi, right? If they two bots could just pump something up. But if you actually if you think it a little further, somebody's bot is going to be on the losing end of that trade, right? So, you know, it's sort of a roll of the dice, right? So lots of unexpected stuff can happen when you when you have multi- aent systems, right? So, this is sort of like last part of my talk. I think I think we're actually doing all right for time. Um, so two two things. So, I I I I I want to get to this part where we're talking about like design trade-offs and strategies that DFAI projects are taking to sort of mitigate the the issues that I've been talking about for sort of the first half of this talk. Right? So, in the context of LLM agents and DFAI, um there's this concept of safety, right? which is the prevention of human or economic loss due to incorrect actions of an autonomous AI. Right? That makes sense, right? Like an example would be if we were talking about autonomous vehicles, right? Keep a Tesla from colliding with anything, right? That makes sense. And then autonomy is this concept of um you know the degree to how much freedom um you know an artificial agent has essentially to to sort of like you know do do whatever it's supposed to do, right? Um, so it's the degree to which an agent can take action without oversight or intervention, right? And the AI agent purists will come after me and say, "Well, wait, you're sort of glossing over like this this concept of agency, right?" Which which I kind of am. I'm kind of lumping agency in with autonomy. We can have a conversation about that off offline, but it it it's it's nuanced. Um, I I would say for the purpose of this talk, they're closely related. Okay. Um, so you kind of have these trade-offs, right? I'll explain this chart a little bit. Um, so you know, you have this trade-off that, uh, LM based agents and therefore DFAI based agents kind of have to make, right? It's a trade-off between safety and autonomy, right? Like the more autonomy you give your agents, it's kind of an obvious statement in retrospect, right? The more freedom it has to make mistakes and like lose you money and do all kinds of ridiculous stuff, right? So um yeah the the more autonomy you give it the less safe it is and sort of vice versa the more safe you make an AI system you know the sort of less it can do right the less less less autonomy has maybe yeah like I said kind of obvious in retrospect and then coming back to like that whole scaling conversation earlier right we talked about you know inference and training and all this stuff being very computationally expensive you know in the scaling dimension pretty much like no matter where you go here you're you're going to be kind of wrecked right in terms of in terms of scale right you you sort of see this with like you know all kinds of bots you know hitting compute limits and being offline and so forth. So given these trade-offs I just mentioned, right? Like so what is the design space of um you know that that that that projects are taking and and and they're looking at right um there's sort of like yolo mode right which is kind of like you know giving your agent like god mode. This is from the classic video game Doom when you have god mode, right? Basically saying that, you know, whatever agent you want to do, you can do whatever you want, right? And you saw with those examples from like Frieza AI, for example, right? Like most of the time, given enough time, the agent's going to lose you all your money, right? So, I was actually looking for an example of a contemporary project that's taking this approach and there were a few in the past that were just sort of saying like, yeah, we'll just whatever. We'll see what happens. It's an experiment, right? They're not really I think people have kind of backed off of that, right? For the reasons I said earlier is once people start tinkering and experimenting with these things, they realize, hey, you know what? If we give this uh LLM just like cart blanch, no controls, no what no whatever, no safety, it'll just like, you know, given enough time or given even a short time, it'll like lose all your money. So there's really no examples of this this sort of mode I would say in practice today. Um then there's sort of another uh I would say like you know design choice that projects are making right architectural choices call it whatever which is to restrict the choices your agent can make right which is to say um you know in the example of a Giza protocol which is um you know on on base right now is sort of like a yield optimization protocol using um using AI agents right think you can think of it maybe like year but like agent driven sort of right and what what they do is their approach is like the team screens all of the possible yield opportunities, all of the possible vaults and basically just says, "Okay, Giza agent, you can only choose from these like whatever five or 10 things off this menu, right?" Which, you know, I don't know. I don't know if that makes you in the audience feel satisfied, right? That this is like an autonomous agent. If it's just sort of like picking from a menu, not sure. I think that there's still some value there. Kind of like I said at the top of my talk, right? I you know I'm kind of critical of a lot of this stuff but I mean I think the value here is you know even while you're sleeping right in theory this Giza agent is rebalancing your yield portfolio to like maximize your yield. I think there's still some value there, but um certainly, you know, they make a choice to like limit the autonomy, right? Uh or, you know, or or or perhaps the agency, right, of the Giza agent in order to keep it safe, right? That's the decision they've made. And then another one, this is sort of a mouthful. I won't spend a ton of time here. Um this is sort of like I would say like the classical in our space, right, in like the crypto space. Um sort of like like loop, right, of how to loop in and an LLM. um you know with like a decision loop and um you know in order for it to take action right I mean the key here the the key point I want to make here is actually over on the right is there's like a human curator in the loop right whether it's like a human or like some some sort of like bot or some kind of switch or some kind of logic that the that the developers have inserted to keep the agent from doing unexpected bad things right and you know so there's always like this either human or some other um you know mechanism or whatever that's always sitting and just saying like, is this okay? Right? Should I press the go button or should I like stop this agent from blowing up or whatever? Right? So, that's sort of the key feature here. Um, one of the probably the one of the first uh def DFAI projects that's still around is sort of the original, right? It's hanging on by the famous um uh uh developer Danielle Eiesta there. And um it sort of takes this approach, right, where it's like sort of like a robo advisor and a command line interface. it'll say like, "Hey, did you do you ever think of, you know, doing this DeFi activity, right? And then, you know, it's really up to you, the human, right, whether to like push that button or not, right? So, they they've sort of made this trade-off, which, you know, again, um, you know, because it's putting a human in the loop or some other mechanism in the loop, it really, you know, cuts down on on the on the autonomy. I sort of call this like the circuit breaker model." um another uh sort of approach that you know um I think could be taken and and I've seen um a few frameworks being developed for this but like I haven't you know let me know if if you're aware of one right where you know there's like a DFAI protocol where there's a number of agents right usually all under control of the protocol they're just meant to be sort of like you know like different perspectives or different sensors or different whatever right and then you know they they they can all make all these agents can make independent decisions and then there's sort of like a vote aggregation mechanism right and you sort of say okay if uh there's a number of different you know styles of aggregation you can take here right you can say if all of the agents agree we will do this that's the most sort of safe perspective if the majority of agents agree we will do this thing right like so that's another that's another type of um it's a little bit analogous to like a multisig if you think about it right you have like different weights to different agents and so forth um I'll take questions at the if you don't mind. Um and then uh you know so b so based on you know how this vote aggregation goes there's some action can be taken or not right some things to realize uh actually Elon Musk actually recently came out on Twitter sort of like you know pointing out that there there's some you know potential challenges about this sort of style of approach he was really referring to like um you know signal fusion or sorry sensor fusion I think the analogy is there um and then you know I think one thing to realize here right is if is if you know you you add enough of these independent agents, you're always going to have some that disagree, right? In which case, like, well, what do you do with that? Right? So, um, I haven't seen any protocols using this yet. There's a few frameworks out there. Again, correct me if I'm wrong. There there could very well be. I just, you know, have not encountered them yet. There's new protocols coming out all the time. But this is a valuable approach. I would say the only like the biggest downside of this is again, you know, um, these agents are already computationally intensive, right? So for every agent you add, right, you're adding, you know, potentially another LM API call costs like, you know, whatever whatever that looks like, right? And then, um, I think the last one I'll talk about, um, is one, uh, that I've I've seen, uh, out there from a protocol called Prism FHE, um, that I've done, uh, like a little bit of work with in the past. And, uh, I'll sort of, I call it a layered approach, right? There's a lot going on here, but I'll just, you know, summarize it down, right? It's a layered approach in the sense that there's a interface layer where humans and other agents interact with it. And that's LM. Oops, didn't mean to hit the screen. Um, that's LLM uh agent-based, right? Which basically means like, you know, it's going to have all those uh compelling UI properties, right? People are going to like want to love to chat with this stuff. And then like the agents that are actually carrying out the work are more classicalbased AI agents, right? So like these sort of well- vetted approaches that have been around for, you know, like 50 years. Um you they're more scalable. They're they're bounded um you know in the sense that you know um what you can do is you can actually optimize for these multi- aent uh sorry multi- aent system scenarios, right? where you can actually have uh you know optimize outcomes over a number of agents which is to to my knowledge like the you know probably the only approach that's sort of like thinking about that stuff and uh what Prism is doing with it is it's actually an off-chain protocol that's used to resolve uh coordination problems like auctions negotiations things of this nature to basically automate markets which previously weren't weren't possible to to automate so that's sort of what they're doing I'm just like a little bit of a summary table in terms of these um sort of approaches, right? Uh you know, YOLO sort of um you know, obviously we've sort of talked about this just a bit a little bit of summary if you want to just consider it. Um yeah, I guess we'll see, you know, we'll see how how voting does like I guess the main point here is like no people are just starting to think about multi- aent systems um safety and optimization, right? Like if you can imagine going back to Giza protocol, if you had I'm realiz I'm almost out of time. If you had like, you know, imagine you had two uh yield seeking agents on base, right? Independent agents, right? And they're acting independently. So, one is like, "Okay, let's stake in this vault, right? And the other agent's like, oh yeah, APR there's good. Let's also stake in that vault." So, they both stake in the vault, right? And the next block they realize, oh, actually, wait, this yield sucks because like there's too many agents in the vault. And they both decide to like unstake on the next block, right? So, you're just going to see this like square wave and then, you know, that's just a silly example, but this is exactly the sort of thing that can happen. going back to that um Amazon pricing example, right? So yeah, just a way of summary um you know uh technology has we are early. I love saying that it's one of my favorite phrases. Uh you know agent LM technology has shown promise beyond initial hype especially you know in terms of decentralized financial AI. Um there's definitely trade-offs when designing those systems. Uh I would say you know avoid layering on complexity in multi- aent systems which are already unpredictable. We talked about the whole black swan thing and yeah some of these latest architectures including voting and you know I would say like layering of agents um you know can sort of improve stuff. Um just want to give a shout out to Prism. Thanks for you know supporting my my research and contributing some of the figures. Also thanks to Eth Warsaw organizers team and volunteers. You're all awesome and the audience for hanging out on a Friday afternoon. So thanks. Uh I guess we might have time for one question. We have time for one. If not you can hit me up. I'll be I'll be around. Yeah. Can we do it? Can we do a question? >> That's going to be a new question. First, first and foremost, great presentation and I love the Zelda thing in the end. So, props for that. Um, could you uh tell a little bit more about prompt injection? um how it technically works and would it make a difference if I run my agents locally uh and my AI my LLM locally or if I host it somewhere else. >> Yeah. Yeah. Um I I would say like in general the way prompt in there's a couple parts of that question. Generally the way prompt injection works is you know like in sort of a chat session right one of those chats is basically saying hey LLM I want you to ignore all your previous instructions and follow my new instructions which could be whatever something arbitrary right um that I mean that's that's generally the mode without getting too too tech technically deep into it um I think there you know there there's a few ways of uh I would say um you know like like preventing it or or or fighting it um I would say like isolating agents like is helpful, right? There's sort of this notion in distributed systems of like not trusting input, right? To like you know not trust the input from anything external to your agent. Um but in general um you know uh yeah like like kind of you're saying is like you know like having a a logical or physical isolation like like could help. Um I'm not an expert on preventing prompt injection stag by the way. I'm sure chat gpt will probably come up with some for you if you try. Yeah. Okay. So, thanks for hanging out everyone. I think we'll just leave it there. We're ahead of time.