Kirill Balakhonov I Building Effective AI Agents Simple, Composable Patterns I Agentic Ethereum 2025

Hello everyone, my name is Keel. I have been in the AI ​​field since 2017 . I started out as an ML and AI researcher, then managed teams and products . Then in 2021, I entered the web3 space by building a graph-based indexing platform. I currently work at Nethermind , working on various projects at the intersection of AI and Web 3 .

I will mention some of them at the end . But one of the most notable and already in use is Audit Agent, which is an AI smart contract auditor. Through this example, we explored AI agents in Web 3 and how to build useful agents . So, today I would like to explore the topic of creating effective agents using LLMs . After the arrival of LLMs, I want to present a new and smarter way of building software .

LLMs have brought some new methods that help us create usable and reliable things, creating new values ​​for us as developers or users. So. Some insights. It is a consistent observation that successful agent operations in the real world are often not very complex. Simple, interoperable building blocks actually lead to the creation of more sophisticated, reliable, and usable agents than those that use complex architectures .

So, one of my observations is that starting from scratch is not a bad environment. Furthermore, when we keep it simple , we can measure the unmeasurable when using frameworks; Although it may seem easy to start with structures at first , many difficult issues arise during debugging. Definition of agents. The term agent has various meanings , with some considering agents as fully autonomous entities , while others see them as simple workflows that use LLMs , which are not as deterministic as we are used to in programming . We call these agentic systems , where agents make their own decisions .

Agentic systems have very powerful predefined workflows, and agents are programmed to make decisions on their own . This distinction helps determine when a model can be considered a reliable autonomous tool and when it can be trusted to find its own path . When considering agents, LM features naturally tend towards complexity. The key consideration is whether an agent's flexibility truly adds value or increases complexity for a particular application. Because agents typically involve higher computational costs, increased latency, difficult maintenance, and a higher risk of going away .

In contrast, structured workflows that are opposite to agents are more appropriate, predictable, and cost-effective for many tasks. Hmm. In terms of frameworks, several frameworks, such as Eliza, laid the foundation for chat agents in social environments and introduced many useful tools . However, this does not fully address the fundamentals of creating agents that align with the intentions of their creators. So, this raises questions about agent types, differences, and techniques, and how to implement what you need.

So. This is a kind of agent, structures like Elisa , which, in my view, are good for creating an agent that does what it wants. If you want to create an agent that only does what you want, you should start with something very simple or from scratch . There are other ready-to-use solutions such as LangGraph, Microsoft AutoGenAI, and OpenAI Swarm, which provide high-level abstractions for using multi- agent interactions and tools . These are examples of structures that can speed up development initially, but they can hide important details .

Debugging becomes more challenging when the visibility of the prompt and response is reduced . Starting with direct calls to the LLM API often provides the best understanding and results. The example on the screen is a poor way to use LLMs. This explains why we are starting to use LMS even when it is not necessary. Furthermore, Augmented LLM is a building block that is very easy to use and provides key value that we cannot get without an LMS.

Without an LMS. When LM emerged as a building block , we first started using it as a question-and- a question-and- answer system . But, if we start doing that, we quickly realize that it doesn't work that way because it creates hallucinations, right? We should consider this as processor for data and provide it as input . So if we control this data proliferation control this data proliferation , it becomes a tool that we can use predictably .

This is the slide I wanted to share for this section. So, this is the first workflow method. This presentation is structured to illustrate that method . In normal programming programming , if you start writing code without methods like functions, classes, object-oriented programming, or functional programming, it will lead to chaos that is difficult to control. This is even worse in the case of LLMs, as they create multiple pathways that yield unpredictable results .

That's why these methods are necessary necessary , and while some methods may be obvious how to use, others may be new. This applies to those who build artificial intelligence systems. The first example of a workflow is prompt chaining. We divide the work into several parts and provide the result of one LLM call as input to the next LLM call . For example , this includes applications such as content creation, translation, and advertising copywriting .

I will show you a simple example for each workflow. Here is an example of a marketing workflow for one input and one language, using an LLM like GPT-4. In this workflow, we use GPT-4 to first write marketing copy and then translate it into the target language . It is a very simple agentive system that can perform powerful tasks even though it is not a complete engine . The second method method is routing, which focuses on classifying inputs to send them to appropriate specialized processes .

This includes using different LLM prompts for different queries, or assigning efficient models to simple tasks and robust models to complex tasks . This is a very simple approach. This will provide a significant improvement in performance; This is like using the if-else method at the LLM level. In this example example , we... These are these two pieces of code.

This example contains a function to write a customer query. Let's see what it looks like when we call this. So, for each query, we'll take the customer query as input and return the result to the function . The query goes to the classification prompt, and we ask the linguistic model to classify it into one of three classes, answering in one word ; So we expect it to respond in a class . This is a simple approach approach .

In reality, it's best to use JSON format because it's more reliable, but it's okay to show this as an example . As a result, we get the type here. I will use JSON and extract data from it , then have triggers handlers for each specific class . Then we call GPT again with the type and trigger, and in return we get a response in return we get a response from LLM with the trigger chosen to generate this response . I think this was clear, next is parallelization.

There are more complex approaches , but in this parallel processing processing , there is nothing that complicated . The important thing is that tasks can be performed in parallel, and if the results of those tasks are used as separate steps with LLMs , they can be called parallel. Let's see what this example shows. Parallel code review, we do this in Python , but it can be done in any language. We run GPT calls in parallel and collect the results in a single array .

Let's see what's here, for example the code to be examined. We run a parallel code analysis for each feature concept, show an example, and that's it. So, we can do parallel processing for individual LLM calls , which sometimes makes sense . When performing a lot of different tasks, it can be batch processing or requests from different users . To increase performance, we will need to do them in parallel , because LLMs introduce high latency .

So, you might want to speed up your overall algorithm . Orchestration staff. This is a model in which a manager, or orchestrator orchestrator , divides and assigns tasks to different parts of the system. Unlike static methods such as prompt caching or separate operations, this approach allows tasks to be dynamically allocated according to needs . Therefore, this flexibility is very suitable for complex environments, such as code changes, where the scope of work is not predetermined .

Let's look at a code example. Hmm. This will take some time as I created these code examples . Checking Checking . We have a main function that has the task of input validation, which is provided to all functions .

These functions are given a list of files and the text in the files as input . This list can be of any length, and each file can request a different type of processing. But, in this example, we are trying to analyze it and implement it separately . First, let's look at this. So, for updated files files , we run the Orchestrator Code Update task and files.

Task is this prompt, and files is a list in dictionary form , where key is the file name, value is the text of the task. After doing this, we print it . What do we have in the orchestration code update ? Tasks and files are inputs. In the first step, the orchestrator creates a plan .

We create a separate prompt with a list containing the names of the files . We are not adding text here , we are creating a plan for file changes and writing it to respond in JSON format. Here is the example I mentioned earlier of how to properly format publications from an LLM. As a result, we run the GPT Plan prompt . As a result, a list of tasks to be executed will be available.

We convert this JSON into a variable and execute separate calls to GPT-4 for each file in the second step , which can be seen here. The new prompt is this: " Update this file according to the description ", which is already in the project , it was created was created , we put the current content here : text from the file. Provide only updated content, and we expect LM to follow our instructions and make changes to the file as a result . We will return the results, collect all the results in a list, then combine them and send them as output. The next pattern is called ' pattern is called ' evaluator evaluator - - optimizer' , which creates a feedback loop in which one LLM creates content, and another evaluates and critiques it .

Therefore, this cycle continues until the quality criteria are met. This approach mimics human editing and works best with clear evaluation parameters . Therefore, it is not necessary to use it only when there is a clear assessment. Sometimes, you can use some tricks to use this cycle in different situations. But its main value is that it is only useful when you have a testing function or environment where you can get feedback that is accountable for the results produced by the LLM .

This is a very powerful pattern, let's look at the example here. We have a translation development function here, which is as interesting as I can remember . Here is an example of use. We would like to translate the proverb 'The early bird catches the worm' into Spanish . We have set a maximum iterations (max iterations) , which imposes a constraint to prevent this loop from becoming infinite .

But, the most interesting part of this is how this function is structured . First, we translate into the target language with a simple GPT- simple GPT- 4 prompt , then we move on to subsequent cycles, in which we create the next prompt in this way . Please rate this translation . Here is the source text. Here is the translation the translation ; If this is correct, say ' If this is correct, say ' correct' , otherwise explain what needs to be improved; For this we are running GPT-4, I tested this example .

First, you can see that it says the translation is not correct. These are the specific problems with this translation . But after that, we take this rating I mentioned as feedback , put it in a separate prompt , and re- run it to improve this translation based on this feedback. It actually tries to improve the translation based on that feedback , because we have made it clear to it the key points where it should focus and what needs to be fixed . This can be a confusing subject , i.

e., if an LLM cannot produce a good result in the first place , how can it be improved by something that cannot produce it at all ? But it doesn't really work that way . So, when it produces one , it is just one data point in the set of possible results that the LLM can produce . We know that the relevant context guides it towards better results.

That's why when we run this separate evaluation step , we ask another LLM evaluator to focus solely on the evaluation task. It creates a perfect environment , and then we send that environment back to another LLM and use it to improve or refine the quality of the translation . So, it certainly doesn't mean that it can only be used for translations . As I said, if we have an environment where we can properly evaluate the results, it becomes very powerful. An example of this is when you create a code generation system and feed its results into a static analyzer or compiler, you can run the code and find errors.

These errors are actually very useful, valuable feedback feedback . You can put them in a feedback loop and improve the results . This is where we approach self- acting agents , which are not just agentic systems , but truly autonomous. Autonomous systems reflect the evolution of agentic capabilities . These agents start with a goal and complete the task autonomously through the use of tools, data collection, and continuous improvement .

While this autonomy capability is useful for open-source challenges, certain safeguards, such as testing protocols, security boundaries, and operational limitations, are necessary in production environments; This is because if an LLM and agent are left to their own devices , they are more likely to have unintended consequences likely to have unintended consequences . So, here is a useful example to understand all the aspects of this approach . Here is a small agent class in which we clearly define the tools define the tools . For example , there are some tools you can use , such as searching documents, calculating, and writing files . The names of the tools are very important; Because, the code and LM see them as text, so it has to decide which tool to select .

That's why you need to describe them correctly, and we also include a tool description to help with the best choice. Document search (Search docs): Available arguments include price and features; I'll describe them in plain text here , which will return the text . Calculate: The argument to this must be a numeric expression , which will be run in the Python interpreter; Since LLMs are prone to making mistakes in calculations , we define a separate calculator tool . Write file: The argument to this should be the text to be written to the file , similar to the regular GPT-4 function. We implement each tool individually ; For document search, a query is the input, which simulates the document search , and it acts as a key-value storage to return the answer.

Calculation is just a simple evaluation of the expression. So, if it asks for 2 + 2 , it evaluates and gets the answer four, file writing is a simulation of writing a file . We return it and print what it does if LM calls it . This is where the main cycle is set up is set up . The get next action function in this class takes the task and context as input and decides the next action based on the task and current context.

This is a separate function created to ask the LLM what action to take next function created to ask the LLM what action to take next . This should get all the relevant context . That's why we create a prompt that includes the task, the tools available, and the current context we're taking each step in . Here is the task, here is the prompt: Determine the next action to take based on the current context and available tools . Previous results can be accessed by specifying result X, where X is the step number.

Respond in JSON format as follows : tool, tool name, arguments. When the task is complete, respond with this . As you can see, we show the previous context and the previous steps that it has already executed, because this is its mode of operation. It must know what it did before. It must know what task it is currently performing .

It must know what tools it has. So, a separate function actually executes this action it has chosen executes this action it has chosen . In executing the operation , we have that specific action selected and the same context. So, we keep this context between function calls , and if the action is 'task completed', we stop and exit exit . Otherwise, we get this tool from the dictionary we chose.

If the tool is not there, we say it is a mistake . We show that the device is not detected, we take the arguments, and then we are basically in this loop. The thing is. If there are any context clues, change and implement them . Here it is.

So, I'm looking for tools and arguments. So, this is the actual function of this tool. So, all of these codes are to get to this one point of calling what we chose to do using LLM , and ' Solve the task' is the sum of all of these. So, this is the primary agent ring, right? In the next example, we will see that we can create tasks such as creating a price summary, searching for price information, calculating the annual cost for a basic plan, and writing the summary to a file.

So this is a very arbitrary way of defining the task. So, this cannot be done through normal programming without LLM and the agent approach, and we will see what we call 'Agent Task Solution' and what it does. Let's see, we print all of these . Here we are looking at the publications. It prints out what it does, and then it says ' then it says ' Planned Action Tool Search Arguments' .

So, it decided to choose this tool called ' search dogs' . Seeing as it was already defined to provide either 'pricing' or 'features' as input, it decided to post 'pricing' here. So, it decided to put 'pricing' as an argument . This is the result it got as a result. The result is that the basic plan is $10 per month .

It received a spontaneous release , which included a pro plan . But, knowing that it needed to find the underlying plan , it decided to take the number 10 dollars, which we see in the next line. So, the next thing it did was take the base plan price of 10 and multiply it by 12 . Obviously, it's 12 months, because we asked to calculate the annual price . So, LLM created this activity, and then it went into real action.

This is the result we got after the evaluation. Then it decided to move on to the next action , which was ' , which was ' write file'. So, it chose ' file'. So, it chose ' Write File' and entered the text "The annual cost for the basic plan is $120" as arguments. Result: Written successfully.

As the next action, it selected 'task complete' without any arguments, so the task was completed. This is how this tiny agent handled an arbitrary set of instructions . Here, I want to show how the coding agent we use in the cursor works, for example . If you use the cursor's 'composer' feature , you can set it up this way. Here is the interesting part I mentioned.

There is an environment there. So, it can be involved in searching files, returning results, writing code, viewing results and test results, etc. This is the complete agent algorithm with coding agents. Of course, you can combine all of these patterns to effectively create agents . So, you can decide which is best for any goal or purpose .

Start small and then expand it to the level you need . As I said , it becomes difficult to debug when problems arise . And, yes, I think I shared all the information I had. I would like to say a few words about the project I am working on , and I would be happy to answer your questions would be happy to answer your questions . Here you can find projects that I am personally involved in in some way.

I'm happy to talk about any of them in the Telegram chat . Okay, now if you have any questions, let's go to them. Okay, great. There are some questions in our chat , I think we can look at them . First, a question from Rashid.

Is this still a workflow workflow ? This is not a question about MAS. What is MAS? Sorry. Multi-agent systems.

Yes. The multi -agent system is a separate entity, a separate thing. So it's in a slightly different direction . For example , if you take an agent , it could be in the first example; In prompt chaining, you can say "You are a creative director" for the first part. Your assignment is to write the five best ideas for this topic as content .

After that you will have a separate prompt that will be described as "You are a copywriter" . This is a topic on which you should write a detailed training article . This is already a multi- already a multi- agent system, but you can easily create a multi- easily create a multi- agent system with frameworks like Crew AI , where you basically do the same things . But there are certain places where instead of writing "You are the Creative Director," you would write "Role: Creative Director." You write the task , you create the training, etc.

, and it's connected through this framework , but it doesn't provide much value to the results of your multi-agent system. What's valuable in multi-agent systems is that you can effectively divide those areas among agents. This is much easier to control than the scenario I showed in the last section. When you have an agent that can do anything you want , you expect it to control 100 tools and perform 100 different tasks . In this case , it will continue to fail to fail .

But if you divide it into five different agents, give each one 10 tools, and set up a clear workflow for accepting a task, you can make this system reliable. Yes, it is multi- it is multi- agent and controllable controllable , but there is nothing between multi- , but there is nothing between multi- agent and workflow , it is just a mechanism for exchanging data between components of the system.