{
  "video_id": "oaIBkEdITRQ",
  "channel_slug": "jamesbriggs",
  "channel_handle": "jamesbriggs",
  "title": "Mistral AI Agents | Full Walkthrough",
  "duration_seconds": 2585.0,
  "url": "https://www.youtube.com/watch?v=oaIBkEdITRQ",
  "upload_date": "",
  "transcript": "today we're going to be building a AI\nagent that allows us to chat with videos\nthis AI agent will be fully\nconversational we'll be using MOS embed\nand lemon points to build it we'll also\nbe using the aelia platforms video\nprocessing and chunking endpoints to\nprepare our data our agent will also use\nasyn and streaming so that we get a more\nscalable application that also has a\nnice userfriendly design of seeing\nstreaming of tokens and towards the end\nof the video we're going to be bringing\nall this together and seeing how we can\nbuild a significantly optimized agent\nthat will allow us to reduce our costs\npretty dramatically so let's jump\nstraight into it we're working through\nthis example notebook we first install\nour prerequisites we have the AIO SDK\nfor the video transcriptions and\nchunking we have YouTube DLP here which\nis going to download a YouTube video for\nus and Mr AI of course for the LM and\nembed end points so the first thing\nwe're going to do is download a YouTube\nvideo here this YouTube video is one\nthat I did before and it's essentially a\nvideo\nversion of this article here so going\nthrough this we see that I'm talking\nabout AI agents add neuros symbolic\nsystems and talking about the neural\nside of AI versus the symbolic side of\nAI a little bit of history there what\nthey both mean so on and so on so we'll\nbe able to ask a few questions around\nall of that now we're going to go ahead\nand first thing we need to do is\nactually get the orelio API key so for\nthat we need to go to platform.\nor. we log in at the top here and I'm\ngoing to use Google\nokay and because my first time logging\nwith this account it's going to take me\nthrough this little guide we can you can\ngo through it or you can skip it by\npressing the X here so I'm going to come\nover to here and we do need to add\ncredits before using the API but you can\nalso use coupon codes to get free\ncredits essentially so I'm going to show\nyou how to do that the coupon code is\nfor $5 so you'd go to here click\npurchase and what you need to do is come\nover to here where it says add promotion\ncode and type in JB video agent okay you\ncan apply that and you'll get $5 in\ncredits okay so now we can go over to\nAPI Keys we create a new API key you can\ncall it whatever you want I'm going to\ncall it video\nagent and I'm just going to copy that\nbring it over into my notebook here and\nthere we go so I've now authenticated\nwith the aelo client here and then what\nI'm going to do is send the video to the\nplatform for processing so I'm just\ngoing to process it as a single video\nright now I'm not going to chunk\nanything yet we're going to do chunking\na little bit later because I want to\nshow you how we use trunking to optimize\nour costs which are we'll see it's it's\nvery significant so that will take a\nlittle while to process so that's\nprocessing let me talk you through what\nthe first pipeline is that we'll be\nbuilding now to begin with our pipeline\nis going to be pretty simple so we're\ngoing to have our video coming in from\nYouTube we're going to be passing that\ninto the system prompt of the agent now\nthe system prompt is going to include\nsome additional Contex so it's going to\ninclude some instructions on how it\nshould be used initially but then we're\ngoing to insert the additional context\nwhich is our transcribed video into the\nsystem prompt here then this is going to\ncome down here and it's going to be fed\ninto an llm alongside a user query which\nis going to bring that in from over here\nokay so our user will probably ask a\nquestion and that will feed in just here\nnow our LM will then produce an answer\nbased on both the context that we we\nprovided up here and I'll use a query\nand it will return it back to us so it's\npretty straightforward this is the\ninitial pipeline we're going to be\nbuilding but we are going to make this\nslightly more sophisticated so what we\nwill be doing is removing this context\nfrom over here and also this video\ncomponent so those will now exist over\nhere in a separate part which is\nactually our tool okay so this is our\ntool here now what that tool is going to\ndo is given a llm generated query so the\nLM will generate a query like okay I\nneed to search based on the user's query\nI need to search for this and it might\neven do this multiple times it might try\nand answer multiple\nquestions so based on that it's going to\ncome over to here and an embedding model\nagain from mistro is going to convert\nthat into what we' call a query Vector\nwhich I would usually write as xq all\nright so that embed Endo tastes like and\nturns it into xq now what we've also\ndone in the middle of all this is we've\ntaken our article here and we've\nactually chunked it into many smaller\nparts now by chunking these into many\nsmaller parts we allow ourselves to\nperform what is essentially rag across\nthem so retrieval augment generation and\nthe way that we do that again this is\nbefore we come to the llm inference time\nthe way that we will have done that will\nset that up is that all of these chunks\nhere will have actually been passed\nthrough our embed endpoint and been used\nto\ngenerate a numpy array now once we have\nthat numpy array over here and then we\nhave our query Vector all we need to do\nis perform a simple NP dot across both\nour query vector and that array and what\nwe'll get from that is a similarity T it\ntells us which of those values are the\nmost relevant or the most similar to our\nquery Vector so we can essentially\ncancel out a few of those records and\nonly return the ones that are the most\nrelevant so in that whole process\nalthough it seems like a more\ncomplicated system and it is to some\ndegree more complicated system we\nactually massively reduce the number of\ntokens that we're sending to our LM and\nthis is with just one transcribed video\nwith more that would be even greater now\nonce that has finished processing we can\ncome down to here and check the content\nso you see there is essentially the full\ntransr video in there we can also count\na number of words from that trans video\nso it's just over 4,000 words now we can\nmove on to connecting that to our llm\nfor that first version of a simple video\nplus llm pipeline to do that we will\nneed a Myst API key we get those from\nconsole. m.ai API keys so You' come over\nhere create new key and I'm going to\ncall this video agent you then just copy\nyour key and throw it into your notebook\nhere now if you run this straight away\noccasionally you might see this error\nthat's unauthorized now if we go back to\nthe API key created box here it does\nknow that sometimes it can take a few\nminutes to be usable so we can we can\njust try\nagain okay so it seems like we're good\nnow and there we go so now we need to\nget our message content so what it was\nreturned based on this complete\nfrom mistro here so we want first things\nfirst we are using the mral large latest\nmodel and I just want to point out for\nthe system message here was is a pretty\ngeneric system message but then we're\nadding in the transcription and the\ncontent of that transcription which we\ngot earlier on from the video file\ncontent there and then as a user message\nI said hi can you summarize this for me\nokay so pretty straightforward and you\ncan see here that we have the assistant\nmessage with some content okay so that's\ncool one thing that we can also see in\nhere so if we go to response usage we\ncan see how many tokens we used we see\nhave should be pretty useful later on\nnow already looking at this see that we\nuse a ton of prom tokens the prom tokens\nare the input tokens completion tokens\nare what the llm generates back to us\nnow feeding the whole transcription in\nwith every single query that is just\nnaturally going to lead to a lot of\nprompt tokens being used which of course\ndoes add up to the cost and we'll see\nthe cost for that relatively soon but\nfor now let's just work on making this\npipeline that we have a lot more\nconversational so to do that we're going\nto Define this agent class now I\nwouldn't necessarily Define this as an\nagent just yet but later this class will\nbecome our agent class so right now I'm\njust wrapping what we did already so we\nhave our prompt and then we'll be\nfeeding in our user query when we hit\nthe chat method then we are hitting\nclient chat complete with those\nmessages and all I'm doing here is\nkeeping these self do messages so self.\nmessages is a simple list where we're\nstoring the messages as we create more\nand more interactions so let's see how\nthat works I'm going to display\neverything in markdown by the way as we\nsaw just up here the agent is using\nmarkdown so it's always nice to see the\nresponses in\nmarkdown so we have our response you can\nsee that the query here is can you\nsummarize the meaning of symbolic in\nthis\narticle and in the context of this\narticle symbolic refers to the\ntraditional approach to artificial\nintelligence that involves using\nhandwritten rules anies and logical\nfunctions so on and so on okay cool now\nlet's check that our conversational\nfeatures IE the self. messages attribute\nis working by asking a follow question\ncan you give me that but in short bullet\npoints if the chat history is not being\nstored and being sent to our LM the LM\nwill have no idea how to answer this it\nwill just see that we're asking can you\ngive me that it has no idea what that is\nbut Ure bullet points okay cool and we\nget some nice bullet points okay so we\nhave that built a llm pipeline it is\nconversational and it has input for\nvideo and we're able to ask it questions\nabout video and speak with it in a\nconversational way now what I want to\ntalk about next is using asynchronous\ncode and implementing streaming now why\ndo those two things matter well async\ncode is actually incredibly important\nespecially for AI applications the\nreason I say that is because with AI\napplications we tend to use a lot of AP\nAP calls more so than many other\napplications and those API calls not\nonly do we have a lot of them but they\nalso take a long time because they're\nwaiting for an llm to respond now if\nwe're writing synchronous code whilst we\nare waiting for a response from our LM\nAPI our python code is doing\nnothing is just waiting it's doing\nnothing if we write asynchronous\ncode um python code can be going and\ndoing other things whilst it is waiting\nand then once the response is received\nfrom the API our python code will see\nthat and it can then jump back into that\ntask and continue processing from there\nokay so it frees up a ton of compute\ntime if we write our code asynchronously\nespecially for AI applications then we\nhave streaming now streaming is more of\na preference so with\nstreaming let's say we get quite a long\nresponse from our\nllm without streaming that can feel\nquite painful to a user the user\nexperiences that you're just waiting and\nwaiting and waiting and then you just\nget this massive trun of text and\nthat's\nokay but it's it's it's not a great\nexperience and it's there are so many\napplications out there that use\nstreaming that most users of AI systems\nnow kind of expect streaming as a\nstandard so it's almost an essential\nfeature for most particularly chat\ninterfaces and another thing that we can\ndo really nicely which I'll use\nperplexity as example here when we use\nthe perplexity interface and perplexity\ngoes and searches for\nsomething the LM is generating that\nsearch quy and it's deciding what tool\nto use which is in that case a web surge\nif we stream our tokens we can actually\nImplement that in our own applications\nas well so whilst in this case as we'll\nsee later whilst our agent is going and\nlooking at our video transcription and\nperforming a search we can actually see\nthat our agent is doing that we can see\nhow is doing that we can see the the\nsearch queries that are happening which\nallows us to build far more interesting\nexperiences for users in my opinion okay\nso let's go ahead and see how we can\nrewrite this agent class to use async\nand also Implement streaming so we're\nnot changing anything about how we're\ninitializing our agent here the only\nthing we're really changing is we're now\ndefining our chat method here with async\nso this makes it a asynchronous method\nand we're switching from Mr Ro's\nsynchronous chat method to Mr Ro's\nstreaming and async chat method okay so\nasync because of this this means we will\nnow need to await the response and\nbecause we're streaming that means that\nwe will see just here we're actually\niterating through our response and\ngetting the tokens out from there okay\nso the chunks are coming through from\nour streaming response desription in a\nslightly different way to what they were\nbefore which was the we just got a\nsingle assistant message block and that\njust had everything inside it instead\nnow we're getting messages but for each\nchunk so we still need to go through we\nneed to go through into the chunks of\ndata we need to extract from the choices\nand we're now looking at the Delta which\ncontains any changes and inside the\nDelta we have our content so we're\nsaying if that content is not non okay\nif that content is not non we are going\nto go into this if statement here and\nwe're also just using the warus operator\nhere to pipe this information into the\ntoken variable here because that means\nwe don't have to well we don't have to\ntake this and put it into here which\nwould be just nice clean so we do that\nthen the final thing that we do is we\nalso create a list of all of our tokens\nthat we're receiving so what that allows\nus to do is on the next line down here\nit allows us to take all of those tokens\njoin them together and then use them to\nactually output a final assistant\nmessage object which contains everything\nso we do that there's also a slight\ndifference to our usage attribute here\nso we have to go chunk data\nusage all right so that is it we extract\nour assistant message from the messages\nthat we added here and let's go ahead\nand try that so I'm going to use the\nsame query again can you summarize the\nmeaning of symbolic in this article and\nwe should see a streamed\noutput Okay cool so we're now seeing a\nstreamed output there and one other\nthing that I should point out is that we\nhave to await our agent do chat method\nnow okay so that all looks pretty good\nwe can continue our conversation a\nlittle bit here and I want to do this so\nthat we can see the essentially the\nusage of our conversation over\ntime great so plenty of tokens output\nthere but then of course also plenty\ntokens input so we can see that even\nthough we output a ton of text here\nthey're pretty Leng the\nresponses that output is so not even\ncomparable to the inputs that we have\nwhich are huge so the number of inputs\nhere is of course all made up by us\nfeeding that article in every single\ntime so the question now is one how much\ndoes that cost and then two can we\noptimize it so we Define a cost\ncalculator here and we need to calculate\nbased on these prices here so input\ntokens which is our prompt tokens and\noutput tokens which is our completion\ntokens so I'm going to run that and\nlet's just see how much that\ncosts so looking at this we're paying\nthe these seem like pretty small numbers\nbut this is for each one of these is for\na single interaction so that actually\nwill add up quite quickly after just 100\ninteractions you're paying a dollar or\neven sorry even more than dollar\nprobably you're hitting a dollar within\nmaybe 70 interactions so depending on if\nthis is a single conversation that's\njust getting bigger and bigger or\nwhether it's multiple conversations over\ntime with smaller a smaller number of\ninteractions in there we can see how\nthat would add up very quickly now to\noptimize this what we're going to do is\nwhat I described here so this component\nhere where we're breaking up our\ntranscribed documents into smaller\npieces and then only using what we need\nfrom those chunts so how do we Implement\nthat there are a few steps like I said\nit's not necessarily A simpler system\nalthough it's not complicated but it is\na far more efficient and scalable and\ncost effective system so what will we do\nwe're going to break our transcribe\ndocuments into smaller chunks embed\nthose chunks into vector embeddings\nstore those Vector embeddings in a nump\narray then when querying our llm will\ntransform our question into a small\nquery we embed that query into a vector\nembedding that creates the xq query\nVector that I mentioned earlier then we\ncompare the semantic similarity between\nour query vector and trunk vectors to\nfind the most similar Trunks and return\nonly those most relevant ons to our LM\nfor the final\nresponse so let's start by truning our\ndocument now that we're using async\neverything we can go ahead and use the\nasync orelio chunky endpoint as well\nalthough given that this is\npre-processing we don't necessarily need\nto but in any case then we come down to\nhere and we first want to set up our\nchunking options for our semantic trunk\nso we're using a semantic trunker which\nmeans that our trunks are going to be\nproduced by looking at the semantic\nsimilarities between parallel components\nin the text we say that we want a\nmaximum chunk length of up to 500 tokens\nwhich is quite big to be honest I might\neven suggest going lower but it's\nokay and we use a window size of five\nthat is essentially what is the rolling\nwindow in which we are\ncomparing the similarities between our\nlike parallel components within the text\nso then we get our chunks you can see in\nthe first chunk I'm talking about the\nApple remote Pro program and the front\nrow program and then there's a couple of\ntruns in the middle here that are a\nlittle bit disjointed from the other\nparts it's mostly spoken audio I'm\nrambling a little bit and then we can\nsee down here that this final trunk here\nis focusing on the react\nagent and how it relates to a broader\ndefinition of\nAgents so we have\nthose and what we can do now is actually\ntake those chunks and we're going to\nembed them using mistro again again to\nkeep everything in line with the async\napproach we're using the async method\nfor embeddings here and we do actually\nneed to use this async method later on\nwhen we're producing our query embedding\nand we use the mystal embed model okay\nso we're just taking the content of our\nchunks out here and inputting them to\ncreate our embeddings and if we look in\nour embeddings response data we'll see\nthat we have 35 Chumps there we can also\ngo in and see the length of each one of\nthose embeddings so the embedding\ndimensionality of this mystal embed\nmodel is\n1024 which is a very typical\ndimensionality for most embedding models\nnow we take all those and we're just\ngoing to combine them all into a single\nnumpy array this will allow us to form\nthe dot product compar\nlater against it and let's see how we\nwould do that so I'm going to ask the\nfirst question of course later on I'm\ngoing to want to embed all this within\nour Search tool but first I want to ask\nthis first question of what is the\nrelationship between AI agents and good\noldfashioned AI I'm going to embed that\nquery using myal embed again and I'm\ngoing to convert into an array and\ncreate my query vector\nthe query Vector shape gives us the\ndimensionality of the model and with\nthat we're actually ready to calculate\nthe similarity specifically dot product\nsimilarity between our query vector xq\nand the precomputed document chunk\nvectors you can see in here these are\nall of the these are all the similarity\nscores so we if we're looking through\nthis we would see okay the like these\nvalues here these ones these are all\nkind of the higher ones around there and\nthey all tend to correlate into a\ncertain segment of the transcribed\ndocument and that is generally expected\nbecause when you're talking through\nsomething you're going to be switching\nfrom one topic to another topic to\nanother topic over time and those topics\nthat share the highest proximity within\nyour you know your your story or your\nvideo they're probably going to be more\nrelated than a random Chunk from you\nknow let's say the end of the video or\nstart the video okay so we can use NP AR\nsort with a top K value three to return\nthe top three most similar\nchunks okay so we see that that is 10 12\nand 13 which is probably if we look at\nthese I think it is this value\n10 12 and 13 so the ones we pointed out\nearlier great so we have those\nand then what we're going to do is get\nthose index values and we're going to\nuse them to get our most relevant trunks\nor the chunk content from those like\nso so looking at these we can see in\nhere I'm talking\nabout the symbolic AI stuff in the\nmiddle here we talk about R rules\nontologies and other logical functions\nokay which is the the good old fashion\nAI then here I actually mention good old\nfashioned AI directly at the end there\nbut I think I'm generally talking about\nthe same thing and then here I think\nI'll GOI probably I'm saying good old\nfashion AI I GOI there and yeah and I'm\ntalking about how that compares to\nconnectionism so all those seem relevant\nto the question now what we want to do\nis take what we've just done and\ncompress all that into to a single\nfunction okay that function again will\nbe async so in this function we are\nproviding a do string which we're going\nour llm is going to use this do string\nin order to understand how it should use\nthis function and when it should use\nthis function so I want to give it as\nmuch information as possible here that's\nwhy I'm saying use this tool to search a\nrelevant chunks of information from the\nprovided video and I'm saying how the LM\nshould use this tool as well so I'm\nsaying provid as much context possible\nto the query parameter ensuring to write\nyour search query in natural language\nthat's how you would get the best\nresults here then I also want to say\nokay if there are multiple questions\nbeing asked to just use this tool for\none of those questions at a time and the\nreason we do that is it can improve the\nretrieval quality because if you imagine\nyou're asking five questions all at once\nand your llm sends all that to your\nembedding model your embedding model is\nis essentially taking a query and it's\nplacing it at one point in Vector space\nand if you have five different meanings\nto your query and you're trying to\ncompress those five different meanings\ninto one point inv space you are\nessentially\ndiluting the quality of your embedding\nyou're you're trying to you're almost\naveraging out your your embedding\nbetween those five different meanings\nthat your query actually has so although\nit can it can still give you the results\nyou need it's generally better to try\nand separate out your query that can be\nmore concise when you're doing a search\nlike this so all we're doing here is\nwell what we just did okay we're\ncreating our query Vector using it to\nsearch across our chunks getting the\nhighest scoring similarity scores from\nthat and then using that to retrieve the\nmost relevant chunks and then I also\nmerg those into a single string for our\nLM to use okay so let's try that quickly\nour query here is I think it's so what\nwe asked before which is what is the\nrelationship between AI agents and GOI\nso we should see similar results well\nthe same results even and yes that is\nwhat we can see there cool so we now\nneed to redefine our agent and plug that\nnew function or tool into it now to do\nthat with the Myst API we actually need\nto create a function schemer object\nwhich is what we're doing here so I'm\ngetting the dock string with this from\nour search function which you can see\nhere I'm placing that doc string into\nthe description here of our function\nschema then I'm taking the name of our\nlet me even just copy this just the name\nof our search function which is\nobviously search then I'm also just\ninitializing these parameters now the\nreason I initialize these and then fill\nthem out directly is because we're going\nto iterate through all of the parameters\nof our function dynamically and set\nthose to do that we also need to be able\nto map from python types to the data\ntypes that mist understands and uses and\nthese are as far as I'm aware exactly\nthe same as the open AI mappings as well\nokay so we get the signature here I'm\njust show you what that looks like so we\nget the signature and then we're just\ngoing to go through those and let me\nagain just show you what those look like\nvery quickly I'll show you name and\ndype so what do we have here we have\neven let me print dtype there we go so\nwe have the name which is query and then\nwe have dtype which actually contains\nboth the name and also what type it is\nthe way that we would get the type only\nfrom that is we actually do D type\nannotation which you can see\nhere okay cool so that is what we're\nprocessing and we just do that quickly\nwith this Loop which is pretty\nstraightforward and that creates our\nfunction streamer that we can you know\nwe can apply this to any function and\nthen Metro will be able to use this\nfunction streamer and yeah use that when\nit's defining the tools and then all we\nneed to do is transform that into a\nmystery AI function object we could have\ndone this before but this is this is a a\ngeneral format so I do like to show this\nbut you can view this function as being\nthe same as this function sea that we\ndefined up here cool so now what do we\nwant to work through here we can\nactually remove this callable we don't\nuse that and we can go ahead and just\nwork through okay what are we what are\nwe doing so first we're adding in our\ntool\nsignatures so why do we add tool\nsignatures well that is so our LM knows\nwhat tools it has access to and also how\nto use them so we pass those in for that\nyou see that we also remove from here\nour transcribed content we don't need\nanymore we're going to use that or we're\ngoing to provide that via our Search\ntool and then we make some modifications\nto our chat method as well so we still\nusing stream async but now we've added\nthe tool signatures to the tools\nparameter and we're also we don't need\nto do this this is the default value but\nwe set tool Choice equal to Auto so we\nyeah we could just comment that out it\ndoesn't matter it will do the same thing\nbut I like to be explicit what you can\ndo if you want to force tool calling is\nyou can set any and that will\nessentially tell Mr you have to use a\ntool caller we're not doing that here\nbecause we don't want to force up all\nthe time okay then this bit here is kind\nof the same but also slightly different\nso we have two conditions here whereas\nbefore we just had one which was\nessentially this bit here which is\nsaying okay if there's some content\nwe're going to stream that content\ndirectly but now we also are getting\ntool cores and Tool calls are sent to a\ndifferent part of our chunk objects as\nyou can see here we're going Delta\ncontent for the content that we stream\nand we're doing Delta tool calls for\nTool calls so slightly different you\nwe're seeing those being returned in\nslightly different ways so let's run\nthis and see what happens I am removing\nthese for now because we're we're not\nquite done with cleaning up our agent\nfunction yet so I'm just removing those\nfor now and instead I want to return\nthat tool call so we can just have a a\ndeeper look at it in a moment so tool\ncall what do we have this is a tool call\nit's tool call object inside there we\nhave a function attribute which contains\nthe function call inside that function\ncall is the actual instructions from the\nLM on what tool or function to use so\nit's saying we need to use Search tool\nand these are the arguments so I want\nyou to provide this string into the\nquery parameter okay we also importantly\nyou need to pull out the tool call ID\nlater so just be aware of that so what\nwe've done here is our llm is now able\nto generate the instructions for what\ntool to use but it can't use them and\nthat is because we haven't written a\ncode that allows the execution of our\ntools so we need to do that to do so\nwe're going to create a tool execution\nfunction that tool execution function is\ngoing to take our tool call so that is\nthis object here so going to take that\ntool call we're going to get the tool\nname out from it which is search then\nwe're going to load the tool\nparameters which are from here right so\nit's just a query and that string and\nwe're also going to get the tool call ID\nwhich is exactly what we saw here so\npulling all that information out then\nwhat we're doing is we're using this\ntool map dictionary here so what is tool\nmap let me run this and I just show you\nvery quickly so tool map is just mapping\nus from a string which is our tool name\nto the actual function that it refers to\nokay that's all that's what we've done\nhere so that means that we can access\nour tool map as a dictionary and\nactually use it to execute the chosen\ntool in this case we just have one\nchosen tool but you could imagine if we\nhad multiple tools here we could have\nlike calculator code execution so on and\nso on this execute tool function would\nwork also for the multi-tool use\nscenario and from that we're going to\nreturn tool message containing all of\nthat useful information including the\noutput from us executing our function or\ntool great so we're going to take that\ntool call and we're going to just plug\nit into the your tool and see what\nhappens and you see that we get this so\nwe get this tool message and we get all\nthis content right so this content is\nexactly what we wrote before where we\nare providing the most relevant chunks\nbased on our user query so that's what\nwe're getting in the content there we're\nalso getting okay what tool is that and\nwe're also getting the tool call ID cool\nso we've done that now what I want to do\nis I want to extend the agent messages\nI'm doing this like from from outside\nthe agent class for now in a moment\nwe're going to implement this all within\nthe agent class but for now outside I'm\ngoing to go to agent send those messages\nand add an assistant message which is\nessentially the llm saying I want to use\nthis tool and this is how we're going to\nuse it followed by the tool output here\nokay and then if we look at our agent\nmessages after this we see that we have\na system message that's the you know the\nfirst one that we've predefined we have\nthe user message which is can you\nsummarize the meaning of symbolic in\nthis article then we have our assistant\nmessage which is the LM saying okay I\nwant you to use this query for this tool\nand then we get the response from that\ntool with all that information our LM\nnow has everything it needs in order to\nanswer our original question so let's\ntake what we would usually have in that\nagent chat method\nlet's extract that out and just run it\nagainst our new agent messages that we\nhave which include the the assistant\nmessage and Tool\nresponse okay so we can see that\nstreaming everything\nhere and saying okay symbolic hi refers\nto those written rules intelligence\nlogical functions so on and so on all\nright so definitely stuff that is coming\nfrom that transcribed document now how\ndo we take all of that and refactor our\nagent class for the final time to\ninclude all that additional tool\nexecution logic well we add one more\nattribute which is Max steps this is\nmore of a precaution than anything else\nso if our agent gets stuck in this Loop\nof iterating again and again and we're\nhitting the LM apis again and again that\ncan of course drive up cost pretty\nquickly so to put a limit on that we say\nI don't want to go above three\niterations of the agent saying I want to\nuse this tool and then this tool and\nthen this tool and then respond right it\nwill just have three\nopportunities and that should be all it\nneeds so let's go through our new logic\nwe have a while loop here which says\nokay we're going to go through a maximum\nof those MA steps we this it's pretty\nsimilar so we generate our response\nasynchronously we say okay if the tool\ncalls object is a list we are going to\nprint s of clean this up a little bit so\nit's a bit nicer to read we're going to\nprint the function name or the name of\nthe tool that we are calling and the\narguments being passed to that then\nwe're going to execute that tool here\nthat will give us our tool message and\nthen as we did before we're just\nextending our chat history with the\nassistant message message that told us\nwhat tool to use how to use it and the\ntool response and that is if we have a\ntool call otherwise the logic is exactly\nthe same for if we have just normal\ntokens being strewn back to us finally\nif we see that the length after going\nthrough this async for Loop here if we\nsee that we have all tokens here that\nmeans that our agent has responded to us\ndirectly using that content field and we\nshould break out of the loop and which\nis exactly what we do so let's run that\nand now let's try again so one thing\nthat we're doing here just to try or\ntest this out is I'm actually asking two\nquestions or I'm asking two questions\nthat should require the llm to search or\nuse the search tool and then I'm asking\na final question which is saying okay\nbring those both together and and\nexplain them to me now also throwing in\na little bit of a spanner here by asking\nabout deep seek the transcribe document\ndoesn't mention anything about deep seek\nso let's just see how our agent tackles\nthis problem okay so we have our outputs\nhere and you'll see that sometimes this\nwill work sometimes it will hallucinate\nparticularly with the Deep Seek part you\nsee here that the Search tool is used\ntwice wants to look for good\noldfashioned Ai and another time search\nfor deep seek then it just explained\nokay yes a document mentions go fashion\nAI it does not explicitly mention deep\nseek and then you it just continues\nanswering our questions so we can see\nthat yeah that agentic flow does work\nnow let's look at the usage for our\nquery now we can see that despite us\nhaving two tools being used right which\nis is probably the worst case scenario\nfor this sort of question despite that\nwhich we can see with the usage info for\nthese first two components here it\ndidn't necessarily add that many tokens\nespecially when compared to our original\ncosts so if we look at the original\ncosts from here so this is from the\nearlier execution that we performed we\nwe spent this much across this was\nacross three interactions so we could\nsay okay it was only this much for that\none single interaction now if we look at\nthe the cost for each one of these\nqueries that is pretty significantly\nlower looking at roughly half the price\nfor that first query despite asking a\nfar more complicated question as well\nnow I think this message this note was\nfrom my previous testing and I probably\ngot lucky one time but this gives you a\ngood idea of just how cheap it can get\nso I think the in this scenario the\ncompletion tokens were probably quite\nlow for my final response but yeah that\nis it so we've been through we've built\na fully functional conversational agent\nwith async and streaming using\nmistol we've implemented video with that\nusing the orelio\nSDK and we've also seen how we can then\noptimize further using chunking to\nessentially just reduce our costs pretty\nsignificantly and I believe probably if\nwe optimize this further we can\ndefinitely get that down even further so\nthat's it for this video I hope all this\nhas been useful and interesting for now\nI'll leave it there so thank you very\nmuch for watching and I will see you\nagain in the next one bye\n[Music]",
  "transcript_chars": 35340,
  "ingested_at": "2026-05-15T04:39:07.038165+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 8791,
    "like_count": 105,
    "channel_id": "UCv83tO5cePwHMt1952IVVHw",
    "categories": [
      "Science & Technology"
    ],
    "tags": [
      "python",
      "machine learning",
      "artificial intelligence",
      "natural language processing",
      "nlp",
      "semantic search",
      "vector similarity search",
      "vector search",
      "mistral ai",
      "mistral ai agent",
      "mistral ai tutorial",
      "mistral api",
      "mistral ai api",
      "chat with video",
      "video ai",
      "ai youtube",
      "mistral rag",
      "mistral rag python",
      "retrieval augmented generation",
      "llm"
    ]
  }
}