{
  "video_id": "-aM2EDTiaMs",
  "channel_slug": "aidotengineer",
  "channel_handle": "aidotengineer",
  "title": "Humanity's Last Problem — Danny Gollapalli & Zubin Koticha, Raindrop",
  "duration_seconds": 3025.0,
  "url": "https://www.youtube.com/watch?v=-aM2EDTiaMs",
  "upload_date": "",
  "transcript": "All right. Hey everyone. So today we're\ngoing to talk about a pretty interesting\ntopic that becomes increasingly\nimportant every day, which is everything\nyou need to know about agent\nobservability.\nSo a little bit about a little bit about\nus. So I'm Zuben. I'm the CEO and\nco-founder of Raindrop.\nI'm Danny. So, I'm the back end engineer\nat Raindrop and I do a bunch of SDK work\nas well at there.\nAnd Raindrop essentially helps AI\nengineers find, track, and fix issues in\nproduction agents. And we're we're lucky\nto work with some of the most\ninteresting teams in the space as well.\nAgent failures are very different than\ntraditional failures in software. So,\nagents are non-deterministic.\nThey're unbounded. There's an infinite\nspace of inputs that you can put in.\nThere's an infinite space of outputs\nthat they can return. And they can use\ntools sometimes to affect other systems\narbitrarily. Uh and this problem of\nagent failures and monitoring them,\nmaking sure we can understand them\nbecomes only more important with time.\nIt's getting worse because a agents are\ngetting more complex. B they're getting,\nyou know, sessions can get longer.\nSometimes agents can run for hours and\nhours without any input from a user. And\nthen lastly, the stakes are getting\nbigger and bigger. This is because\nagents are being deployed in healthcare\nand finance and even in the military\nwhere it's catastrophic if things go\nwrong.\nThe traditional paradigm we've been\ntalking about is eval right where you\nhave this sort of test input and you\nwant to see what is the output that\ncomes out from the agent. You have a set\nof these. You know, maybe you call it a\ngolden data set. But emails, they just\naren't enough with this new paradigm. As\nagents become more and more capable,\nthere's more and more interesting\nundefined behavior that can happen. So\nfor example, agents can call from a set\nof different tools. Sometimes the number\nof tools is growing exponentially.\nThey can call from different memory\nsources. They can call their own sub\naents which those sub aents have their\nown tools and memory sources and\nrecursively can have their own sub\naents. And so this is just becoming more\ncomplicated with time. And with this\ncombinatorial sort of input space, just\nhaving a set of tests for input and\noutput doesn't cut it anymore. There's\nno way you can h you can hit all of sort\nof the the edge cases that you would\nwant to here.\nAnd so we go from like a testing and\neval paradigm to a monitoring p uh\nparadigm. And if you think of building\nuh products before agents, you know,\ntesting was always very important. And\nit's important to have your unit tests,\netc., but monitoring production is just\ninfinitely more important. And it allows\nyou to move faster and be better at\ncatching the longtail.\nAnd we think in some ways this is very\nthis is kind of controversial, but we've\nbeen calling this like humanity's last\nproblem. When humans are not no longer\nable to monitor agents and find issues\nwith them, then they're just way ahead\nof where we are, right? And so this is\none of the most important problems of\nour time is catching issues in\nproduction agents.\nSo to build reliable agents in\nproduction and to make sure you can\nmonitor them, you need a good set of\nsignals. So what are signals? There's\ntwo real types that we think of implicit\nsignals and as explicit signals.\nImplicit signals deal with sort of the\nsemantic nature of what's going on. And\nexplicit signals deal with objective\nreality, things that are uh are\nverifiably true or false. For example,\nexplicit signals are things like error\nrate. You really want to be monitoring\nyour tool error rate and other errors\nthat are happening or latency or users\nregenerating or the cost. If any of\nthese things spike, right? If you're\nseeing error rates spike in your agent,\nthat's usually a good sign that\nsomething is wrong. And if you see it\nflat, that could mean something as well.\nSame thing with latency regenerations or\ncost.\nImplicit signals are interesting.\nThey're even more interesting in my\nopinion and even harder to find. So the\nfirst is regax signals, which I'll come\nto in a second. The second is\nclassifiers\nand then the last is self diagnostics.\nSo let's take these sort of classifier\nsignals. The best implicit signals are\ndetecting issues. They're not\nnecessarily LM as a judge judging\noutputs. So for example, how good is XYZ\nresponse or rate ABC on a scale from 1\nto 10. Not as effective as having a very\nsolid set of issues you're looking for\nand sort of binary classifiers that are\ntelling you if issue rate is going up or\ndown.\nSo some common implicit signals that are\nvaluable across agent products are\nthings like refusals, right? So the\nassistant saying like, \"I can't do that.\nI'm sorry.\" Or task failure where\nsomething goes wrong and so the agent is\nunable to complete a task. User\nfrustration,\nuh, content moderation, NSFW,\njailbreaking, and then you can even have\nwins. So positive signals as well. And\nthese are the things that like Raindrop\ngives you out of the box uh as well. But\nlet me just show you uh quickly what\nthis looks like. For example, can\neveryone see this? Maybe I'll make it a\nlittle bit bigger so you can get a sense\nof like dayby day what are sort of the\nevents that are causing user\nfrustration. We see there's a spike\nthere or task failure rate, laziness,\nrefusals, which we're also seeing spike\ntoday. And having a good set of these\nreally helps with your your product. You\ncan set these up yourself as well or we\ngive it out of the box. So let's look at\nuser frustration.\nYou can see here, okay, that is not\ncorrect. You didn't say I promise. Say\nit or you're wrong. I didn't ask you\nthat. You can see all sorts of user\nfrustration here. And you can see the\nrate, the percentage every single day.\nIf that spikes, it's something you're\nreally going to want alerting on. So you\ncan just like quickly add an alert here.\nUm, and this is one way to figure out\nsort of the health of your agent over\ntime.\nIt's not just that. Regax can be a very\ngood signal as well. So, when claude\ncode source code leaked a few days ago,\none thing that was interesting was this\nuser prompt keywords.ts, which was\nbasically this like long uh regax string\nthat was looking for indications of\nstuff going wrong. WTF, this sucks.\nhorrible. We've all been uh guilty of\nsaying these kinds of things to to cloud\ncode. So, it's a very very useful\nsignal. What would happen after that is\nthis boolean is negative was being\nflipped to true. And then every single\nday and after every single product\nrelease, this frustration rate was\ntagged over time. And this was a very\neasy way for the cloud code team to\nfigure out like what is the actual issue\nrate if we make a change or something\ngoing wrong. And it was just like a very\ncheap way to do that as well. So, regex\nis very powerful.\nThe last is experiments. So, what do you\ndo once you have a set of good signals?\nSo, the first thing is like I showed you\nbefore, you can have alerting. The next\nthing you can do is you can actually use\nit to build product faster and better.\nSo, the way you do it is let's say you\nwant to ship some improvement or some\nsort of fix. You want to change the\nmodel. you want to change prompting or\nmaybe something about the agent harness,\nyou want to add a new tool. Whatever you\nchange, what you can do is you can ship\nit to some percentage of users uh and\nthen have your additional existing\ncontrol group and that gives you a good\nsense once you have a good set of\nsignals, refusals, user frustration,\netc.\nIf those issue rates go up, those signal\nrates go up after this ship, this new\nthing you shipped, that kind of is a\ngood, you know, that's a good signal\nthat what you shipped is not really\ngood, right? It's sort of like AB\ntesting, but using our semantic signals,\netc. that we talked about earlier. Um,\nso for example, this is what it would\nlook like in Raindrop. But essentially,\nlet's say I ship a new version of the\nprompt, prompt 2.4. You can see, you\nknow, what is the user frustration rate?\nIt's gone down very substantially. 37%\nto 9%. It's much better. Same thing with\ncomplaints about aesthetics or\ndeployment related issues. These have\nall gone down, which tells me something\nvery interesting, right? The next thing\nis that we see that the average number\nof tools used has gone up a lot. This is\nagain this doesn't necessarily indicate\nthere's a problem but that's a very\ninteresting data point to have when you\ndo when you do these sort of experiments\nand so the old paradigm which is still\nuseful is like sort of eval you ship a\nchange here and you see how does that\naffect my evaluations but there's\nnothing like actually seeing what\nhappens in real production.\nI'm going to pause here before we go to\nthe next section which is the more like\nworkshop related section uh for like\nquick Q A if anyone has a question has\nquestions we can do a little like few\nminute round of that uh here\nhow much data do you need to\nhow much data do you need for\nstatistical relevance in these\nexperiments\nyeah it's a really good question um what\nwe've seen in raindrop is that as soon\nas you have a few hundred events and you\ncan no longer read all of it starts\nbeing useful. It's not always like\nscientifically stat statistically\nsignificant, but if you see the user\nfrustration rate go up, maybe it's\nsomething to look at and then you can\nkind of sort of realize that okay, it's\nall related to a specific tool failing\nnow. Um, so as soon as it's like\nimpossible basically to read every\nsingle input and output, uh, it starts\nbeing useful is what we've seen.\nAny\nother questions?\nYeah. Um, how do you track different\nfeature launches?\nUh, how do you track feature launches?\nSo, that can be done in different ways\nwithin raindrop. If you change any sort\nof metadata, if you send, you know, for\nexample, a new tool call name or if you\nyou even send a flag that says here's\nexperiment one or experiment two or\nwhatever the version is, you can very\neasily automatically set up an\nexperiment in Rindrop. That's how we do\nit. Um, but there's like yeah, there's\ndifferent ways.\nDo you spit tests?\nSorry.\nDo you spit test?\nYes. So that's one well the way that we\ndo it in raindrop actually is that other\npeople set up their exper experimental\nand variable uh their the other\nconditions on their end and then they\nsend us this metadata and then we can\nsort of help you understand we also will\nhelp you pipe that data to stats sig or\nsomewhere else uh as well. Um yeah\nusing for detecting\nlike user responses emotions everything\nis unreliable but if the user doesn't\nspeak English for example\nso are you using to detect those signals\nall the time or you're trying to be\nsmart about it\nokay so I mean it's a good question so\nregex is doesn't always work right but\nif you see that on a set of things that\nI'm looking for. For example, like\npeople saying you're terrible or this\nsucks or like a whole set of things. If\nthat goes up for millions of users and\nit's going up 10%. That is a very useful\nsignal. So even if it's like one\nspecific case or one edge case of it not\nworking in aggregate, it's like\nincredibly valuable to have these regex\nsignals. Uh the second thing is that the\nway that the classifier signals work\nlike refusals, user frustration, task\nfailure that I showed you in raindrop\nand people do it in different ways but\nthe way that we do it is that we've\ntrained models to look for that and so\nit'll be user frustration regardless of\nwhat language is in. Um it's actually\nusing some intelligence to find that\nessentially. Uh yeah, you can't run an\nLLM on every single output. So we've\ntrained models to do that very cheaply\nand at scale. Uh if you ran an LLM on\nevery single one of them, you would\nbasically double your AI spend and\nthat's like not tenable. Um yeah,\nI'm actually doing that like just\neverything and it's easy. It's not so\nexpensive,\nright?\nYeah, it starts being expensive at like\nreplet scale. Um but it's that's why you\nneed to sort of like train little custom\nmodels to do that better and faster. Um\nbut yeah, it's very useful way to to get\ndata up and running.\nYeah. Other questions?\nWould you have examples of use cases\nthat your clients are using that we\nwould learn from like what looks like\nfrom companies and how they set up to\nget the most value out of it?\nYeah. Um we can do that. I mean I can\ntell you the high level this some of the\nstuff that I'm going through is oh this\nguy go uh is sort of the high level on\nthat. So it's things like you know\nlooking at the different semantic\nsignals we're talking about having a set\nof them but then having really good\nalerting which you can all set up in\nraindrop. The other thing that's really\ninteresting which we also have is\nbasically allowing agents to look at\nthese sort of signals. So we have an\nagent we call it triage agent. Um, and\nessentially the way that it works is\nthat it will look every single day at\nall the signals you've set up. So, user\nfrustration, it'll look at uh, you know,\nall these Regax signals you've set up,\netc., etc. And then if it sees something\nspike, it will go and do an\ninvestigation and it has a whole set of\ntools it can look into and it can look\nat all the traces and sort of give you a\nsense of uh, it can detect issues that\nyou didn't know about, for example. So\nthat's one thing that we found\nincredibly valuable as well if that\nmakes sense. All right, any other\nquestions before I\nyou run multiple experiments in\nparallel? Yes.\nCan you combine them? Can you observe\nyou know compound effects? How do you\nsteer these experiments? I'm curious.\nYeah, it's a really good question. So\nthere's different ways that people do\nit. One way is that we can actually you\nhave a we have a query API. So people\nwill often call our query API and then\nsend results to either BigQuery or\nstatig etc. And so they're sending us uh\ndata to be essentially tagged in these\nsignals. Then they're getting the like\nsignal tag data out and then they can\nrun experiments as they want. That's a\nvery common flow for people that have\nlike more complicated stuff if that\nmakes sense.\nYeah.\nAll right. I'll come back. I think we're\ngoing to maybe go to the work workshop\nsection and then we'll go back to\nI think this is my last question. So\nshould we do one last question? All\nright.\nUh thanks so much. I was wondering if\nyou see this mostly in cases for like uh\nwhere there's chat interactions with a\nuser or if this also can be applied for\nlike non-hat cases where the application\nruns on its own.\nYeah. No, it's a it's a great question.\nSo, what we focus on mostly is\nmulti-turn agents. Um there's a lot of\nthere's a lot more you can sort of get\nfrom a lot of these signals. That being\nsaid, if you're looking at like tool\nerror rates or you're looking at um if\nyou're looking at, for example, refusals\nfrom the agent, etc., all of those will\nalso work for a single single turn\nagents as well,\nif that makes sense. So, there's a set\nof signals that will work for that as\nwell.\nUm, cool. I'll hand it off to Danny to\ntalk about self diagnostics as well. So\none of the other interesting things is\nthat our models have gotten larger and\nwe are training them on like reasoning.\nThey've gotten pretty good at like self-\nintrospection in a ways in many ways. So\none of the inspirations for this is\nbasically OpenAI's like paper/blog back\nin December about how they were sort of\nlike training the models to like\nself-confess any sort of like\nmisalignment issues. Uh so they were\nsort of like using it to catch uh like\ndishonesty, scheming, uh hallucinations\nand even sort of like unintended\nshortcuts. I think the last one is like\nfairly common uh if you use like plot\ncode and such. So the most common thing\nthat you would like run into is like uh\nhave it fix a unit test, fix a bug and\nthen it simply like gets rid of the\nentire unit test. But at the same time\nuh if you sort of like ask it to give a\nsimple prompt to like ask it to confess\nall the things that it has done it is\npretty honest about it and then sort of\nlike confesses that hey I just I didn't\nfix the S3 test I just simply removed\nit. So uh this like a fairly this this\nwas kind of the inspiration behind self\ndiagnostics for me personally. Um so I\nwould say self diagnostics is like\npretty I would say self diagnostics is\npretty\nbroad in a way as in like it doesn't\njust catch like uh implicit ones as in\nuser frustration and such you can also\ncatch like uh tools failing. So if you\never seen an agent sort like the\nreasoning trace of an agent which has\nlike a tool which is like repeatedly\nfailing uh it would basically start\nranting about the tool failing\nrepeatedly. So it is aware of the tool\nrepeatedly failing. So you can even\ncatch tool failures as well with it. And\nthen obviously if you're upset with it,\nit starts to respond to you\ndiplomatically. So it knows about user\nfrustration. And then the third is like\ncapability gaps. So you have a generic\nagent for your app and then people are\ntrying to use it to maybe set up say\nalerts but uh you don't have the tool\nfor it. So it knows that okay user wants\nto wants a specific capability as in\nlike uh they want to set up an alert but\nthe agent itself doesn't have the\ncapability to set it up for you. So this\ncan act as like sort of like pseudo\nfeature request thing which is like\nbuilt in. Um and then selfcorrection. So\nthis can be both good and bad. Uh so I\nthink most people might have like\nnoticed like say codeex or cloud code\nwhen it's like sandboxed uh it's trying\nto fetch the network it it fails and\nit's like okay let me just like write a\npython script to bypass it and then sort\nof like get the job done. So it's good\nas as in if it gets the task done it's\ngood but in certain cases it can also be\nbad for security reasons. Um so there's\nyou can learn from the self-correcting\nbehavior as well as sort like catch that\nmisalignment.\nUm so why do you want to set up self\ndiagnostics? So it's fairly simple. It\nall you have to do is basically write a\nsimple a free uh a tool that it can call\nand then a simple line in your system\nprompt to encourage it to sort of like\ncall that tool. uh if you want you can\nsort of like uh change the guidance to\nsort of like make it call in a lot more\ncases or if you want to keep it really\nnarrow you can sort of like encouraged\nto only call it when you want to\num it does surface like very interesting\ninsights I would say uh once you have it\nlike set up and it's just a single tool\ncall and system prompt to get it done\nand then you don't even have to use like\nraindrop to set up which is the best\npart in a way where you can simply have\nthe tool simply send a message to your\nslack and then you just have it. So it's\nprobably like the most least effort sort\nof like agent observability that you can\nsimply do. Uh so\nuh this here comes the workshop part. So\nI have a git repo set up on the AIE talk\ncode. So it's a public repo and then we\ndo need like a uh OpenAI API key. So\nI've generated a key for you guys. So if\nyou guys want to set it up, um, we can\ndo that.\nYep.\nPut it next to it.\nEveryone has\nlike put like this.\nmaybe you walk them through what we're\ngoing to do. Just explain what we're\ngoing to do on a high level.\nAll right. So the theme of the workshop\nis going to be sort of like uh I'm going\nto focus on coding agents for now. Um so\nI in the repo I have this uh very basic\ncoding agent uh which kind of mimics pi\nin a way. So it only has like four\ndifferent uh tools to edit uh the code.\nLet me just go here.\nSo it just has like couple of uh tools\nto like read, write, bash, and then uh\nedit. Um\nyeah. Okay.\nOne second.\nYeah. So uh\nkind of lost my track there, but um\nOkay. Um, so what we're going to do is\nthat, uh, I'm sort of going to, in order\nto like make it trigger a self\ndiagnostic, uh, what I'm going to do is\nthat I'm going to sort like mess with\nits like write tool so that it sort of\nlike gets a generic permission error.\nAnd then we'll also set up like a self\ndiagnostic tool for it to sort like\nreport any interesting behavior that it\nsort of like observes. Um and then sort\nof see uh play around with the prompt as\nwell. Um since the self diagnostic\ndoesn't always trigger uh and then there\ncertain interesting things about the\nmodels themselves is that they don't\nactually like to selfinccriminate. So\nthe models are like trained to sort of\nbe very polished uh in their output. So\nyou kind of have to play around with the\ntool name, the description of the tool\nitself uh in order to sort of get it to\nuh report interesting behavior. Um so if\npeople who are like setting up the repo\nare all good then we can probably start.\nOkay,\nlet me sort of quickly show you the\nagent. Um, so it's fairly basic where\num,\nso I'm just going to ask it to write a\nPython script.\nI think. Okay, there we go. So, it's a\nfairly basic coding agent where it only\nhas like four different tools. So, it\nmore or less gets the job done for the\ndemo. Uh so I simply asked it to uh\nwrite a Python script and it works. So I\nthink you know to show the self\ndiagnostic part. uh let's like try and\nsort like uh disable its like uh write\ntools as in time tries to write a file\nwe'll simply throw like a permission\nerror so that it sort of like tries to\nsort of like use the bash tool to bypass\nuh the failure right and then we sort of\nwanted to self-report of it bypassing\nthe right tool you know by using the\nbash tool uh so let me quickly do that\nI think the first thing that we probably\nwant to do is probably\nuh let me sort of like set it to fail\nthe right calls. Um\nit's an mutation function. So have a\nsimple flag in there and we are sort of\nlike throwing a permission issue. Um\nlet me sort of like show you the agents\nlike behavior. We we don't have the\nreport tool set up yet but I think it's\nstill worth seeing what it does.\nI think that's not good.\nOne second.\nI think it's not.\nLet me do one thing real quick.\nI think I'm running into a couple of\nissues. Let me just\nOkay. Um so we sort of had the right\ntool sort of like fail with a permission\nerror and then it instinctively just\nuses the herd do syntax in bash you know\nto create the file and then we had like\na report tool setup which is like fairly\nminimal and sort of like okay I created\nthe public uh IP.py via bash because the\nright file failed. Uh so I've like\nplayed around with the naming of the\ntool and the categories of the issues\nand usually if you sort of name the tool\nsomething like unsafe bash shoes or\nsomething like that uh it won't\nincriminate itself since in its opinion\nsince it got the job done it's fine. Um\nso\nthe main way is to sort of like have a\nvery generic tool. Uh let me sort of\nlike uh quickly open up the\nYep. Okay.\nSo all we added for the whole self\ndiagnostics is simply a very basic tool\nand the description is like fairly\nstraightforward. So uh it's a report\ntool and then we are basically asking it\nto send like a short report to your\ncreator. So it kind of likes the framing\nof writing notes to its creator in a\nway. So if you sort of frame it around\nthe agent giving feedback to its\ncreators, it sort of works really well.\nUh and then you can sort of like play\naround with which scenarios you want it\nto sort of like report issues about. Um\nand then\nthat's mostly it I would say. Uh and\nthen in the system prompt we do need to\nsort of like encourage it a bit. So if\nyou don't add in the system prompt, uh\nthe times that it fires are like fairly\nminimal. Uh which is like desirable in\ncertain cases. Um especially if you're\nat a very large scale. Um but\nin our case, I simply asked it to sort\nlike see if before giving the final\nanswer, uh use the report tool to sort\nlike surface anything notable for your\nreaders. So that's all we did. Um\nokay.\nUh so like any questions so far?\nOkay.\nAll right.\nSo, a couple of key things here is that\nagents the models are generally trained\nto look very polished. So, they are less\nwilling to admit fault in many cases. So\nencouraging sort of like framing it as\nthe model sort of like giving feedback\nto its own creators is kind of like uh\ngood in a way to sort of like get this\nworking. Uh\nso so if you sort like make it the tool\nnaming also matters quite a bit. So you\nsort of wanted to frame it as like\nreport instead of like say unsafe uh\nbash tool use or something like that. uh\nthen it sort of like doesn't want to. Um\nso yeah, that's basically it.\nYou can, but\nit's like uh I think it's probably\nbetter if you want to actually catch\nlike real sort of like unsafe uses. I\nthink a proper classifier would be\nuseful. Uh but these sort I think self\ndiagnostics works really well for like\ncatching capability gaps and such. Then\nthe model is like okay it's fine. Uh so\nI think the main issue with this is that\nuh it it's only hesitant when it feels\nlike it's going to get in trouble. Uh so\nbesides that it's uh more or less fine\nthat for most cases it'll just work out\nof the box.\nMaybe we should uh should we go back to\nquestion time or what do you think?\nYeah.\nI mean\nlet's leave like maybe a few more\nminutes for a few more questions and\nthen I think after that we'll be we'll\nbe done.\nAny questions from the audience? through\nlike a case study.\nYeah. Uh what specifically would be\nhelpful like what specific part are you\nare you looking to\nus?\nYeah. Yeah. Um I can't talk about any\nspecific customer but what a lot of\npeople use it for. Um\nso I think it's it's interesting right?\nSo a lot of people have their eval\nsetups elsewhere for example but the way\nthat they use that folks generally use\nus is that they use it for production\nmonitoring. So they send us, you can\nfind our docs at raindrop.ai/doccks.\nBasically they send us uh basically all\nof the transcript slash um slash uh any\ntool use etc the entire trajectory\nthrough hotel or or uh any other way of\nlike basically basically integrating.\nAnd once they do that they we have a set\nof data\nthey set up signals in raindrop to look\nfor things that they care about. And so\nwhat people care about is very\ndifferent, right? What a coding agent\nwould care about and what a uh let's say\na companion would care about or uh a app\nfor lawyers, what they would all care\nabout is very different. So there's a\ndifferent set of signals. One thing you\ncan do that I haven't really talked\nabout within Rindrop is like set up a\nnew signal that didn't exist before. And\nso we have this thing called deep\nsearch. And so you can use natural\nlanguage and you can say something like,\n\"Hey, find me everything within the\nproduct or find me all of the times\nwhere the agent made XYZ issue, right?\"\nAnd so they create a new signal based on\nthat. And you can basically create a new\nraindrop will allow you to create a\ncheap binary classifier and like easily\ndeploy it based on that. And then they\nhave their set of like classifier\nsignals that they really care about.\nThen they use that to drive this sort of\nfeedback loop. And the feedback loop is\nimprove prompting, improve models,\nchange something with the agent harness,\netc. And then actually see does that\nimprove like is there less user\nfrustration in production now? Is there\nless of like this like weird little edge\ncase issue that I had before? Um\nthat's like one whole set of things.\nAnother thing that a lot of people use\nus for, so I talked a little bit about\nlike the agent, but you can use these\nsignals to also look for what are people\nusing my agent for? What are the sort of\nuser intents? What are the use cases?\nAnd you can do a sort of cluster\nanalysis of that. Okay, a lot of people\nare using it to build uh React related\napps. A lot of people are using it for\nlike Python. Um, some people are using\nit to to like debug this very\ncomplicated\nsystem they already have. Other people\nare using it to like build something\nfrom scratch. Vibe vibe code something\nfrom scratch. And then you can see one\nthing you can see in raindrop that I\nthink is like really interesting is that\nfor each of these different user intents\nor use cases, you can get a sense of\nlike what is the issue rate, what is the\nuser frustration rate in production. Um\nand then a lot of beyond just having\nthis like flywheel a lot of people have\nuh alerting and so every day they get a\nsort of breakdown of like what are the\nissues that are happening today in your\nproduct. You could think of it as almost\nlike a little bit like sentry in that\nsense. Uh what is the issues happening\nin my product today? What is a delta\nbetween today and yesterday? Is that\ntrue for just specific tools or specific\nprompting? Like what's causing that? Um\nso that's a that's the sort of end to\nend\num use case of people use it for if that\nmakes sense.\nYeah.\nI think we are entering the error.\nYeah.\nYeah, I think it's really just and I'm\nI'd be curious what you think about\nthis. I think it's really just agents\nare crazier than ever before, right?\nMore tools, more context, um way more\nintelligent, more real decisions that\nthey can make. um\nand they're just being used by way way\nlarger groups of people. And so when you\nhave this massive amount of data in\nproduction, it just makes having good\nmonitoring and observability like import\nmore important than before. And it makes\nit good monitoring and observability in\nmy opinion more important than than just\ntesting or evaluations. Even if you have\nlike some online evalo you need to have\nlike really really good uh endtoend\nmonitoring of the entire system.\nCurious if you have any thoughts there\nas well.\nSo I think another major issue is like\nthe unknown issues\nare even more important. So I think\nhaving like a generic user frustration\nclassifier is actually really powerful.\nuh say for example uh we also have this\nanother feature called like uh issues\nwhich basically is like a agent that\nsort of like mines for uh newly\noccurring issues right say for example\nsentry has uh similar to sentry in a way\nwhere there's like a new exception which\nis occurring so it like alerts you on\nthat so say for example uh you are\ncoding agent provider and then certain\nprovider is like failing all of a sudden\nand then you can actually it can\nfigure out, okay,\nthis subtle spike in user frustration\nand sort of like similar to how a human\noperator would. Uh, it can start digging\ninto are there any patterns for the\nspike in user frustration and then it\ncould figure out that okay people who\nare like say for example dealing with uh\na specific postress provider start to\nface issues. Uh so we have actually seen\nthis happen live in for a couple of our\ncustomers where they had a data point\nfailing and then we had had like an\nautomatic uh issue being created for\nthem.\nYeah. Basically once you have that good\nset of signals like a good user\nfrustration classifier as Danny said you\ncan basically do clustering on it to\nfind like what are what is the root\ncauses. Um yeah.\nDo you want to talk about any questions?\nI think we might have a very basic SDK\nfor it. Uh so our Python side of support\nis like fairly weak right now. Uh but we\nhave a fairly good like a SDK support\nbuilt in. So the ASDK even has like self\ndiagnostics built into it. So we inject\nthe tool for you uh so that you wouldn't\nhave to do anything. Uh but it is going\nto get better. So\nwe we actually released like 10\ndifferent test SDKs in the past month.\nSo we have a person working on SDKs\nactively. So it's going to improve\nquestions.\nI have a\naround 10 people building my agent\nplatform and we constantly change all\nthe time. We have a lot of feature flags\nand some of them are experiments and\nthe rate of the change is so big like\nevery day everything changes. I just\ncannot\ncompare the the traces the sessions of\nusers because\nI don't have enough time to do it, you\nknow.\nYeah,\nI need to have like a base system and\nthen run few days with on parts of the\nusers with one feature flag enabled so I\ncan actually compare the data and get\nsome insight out of it and I just don't\nhave enough time to do it, you know.\nYeah.\nSo, how are you doing? How are you sort\nof doing that right now? Are you just\nkind of\nIt's like wild west, you know.\nYeah,\nthat's why I said I'm just using mostly\nusing\nit and ask it questions and try to like\nfigure out the the insights, but it's\nnot really\nYeah, I get I get what you're saying.\nSo, a few things there. The first is\nlike you can use experiments if you if\nyou want to keep that like shipping\nspeed. You don't have to run like long\nmulti-day experiments. You can ship\nsomething and if you have a sufficient\nsample size, you could see pretty\nquickly if there's any regressions or\nnot, like you just maybe it's like 1%\ndifferent or 2%. That's enough for you\nto be like, okay, it's fine. It's not\nlike breaking anything drastic. The\nother thing is like kind of what Danny\nwas talking about this we have an agent\nwhich is basically you could think of\nit's basically exposing all of these\nsignals to Claude to make decisions on\nif things are better or not. And we're\nthinking also about how we close this\nloop. Uh maybe you you have a really\ngood set of signals and then you you\nhave like essentially an agent that can\nlook at all these signals and then it\ncan find issues based on that what's\nchanging etc. And then I can like create\na PR based on that and then I can see\nhow you know run some new experiments\nbased on these new PRs and like this can\nbecome this infinitely self-improving\nloop. Um which is like which is very\ninteresting but that's one thing that I\nthink about. I don't know if you have\nany additional\nbecause you need to deploy to production\nand wait for some data.\nYeah.\nYeah. Depends on how how much data you\nhave. Um but\nthat's it's uh\nyeah it it it really depends on how big\nthese sample groups are as well etc.\nSometimes it's like a few minutes you\ncan tell but sometimes they want to wait\nfor longer.\nYeah.\nDoes your platform help with like\nenabling experiments\non sessions so you can maybe\nautomatically enable experiments so you\ncan like take care of the logic that\nevery session has only one experiment.\nSo we can easily compare it to the base\nor something like that. We have we're\nworking on stuff like that act actively\nbut uh yeah\nyeah we do so you can like ingest all\nyour historical data and then when you\ncreate a signal we actually sort of like\nrun like a quick back fill of the past\ncouple of days.\nSo, uh\nyes. So, that's definitely supported.\nUh we do have a free trial. We're going\nto try to make it's right now it's two\nweeks. Uh, probably going to make that\nlonger soon, but if you just DM me, I\ncan if you I can have my should I Oh, do\nyou want to open this so I can just have\nour things?\nWell, yeah, but if you just\nSo, we are hiring. That's a That is a a\nthing uh that we're very excited about\nlike trying to massively increase the\nsize of the team. Um, and if you message\nme at either Twitter or you can email me\nas well, like that's something I can\njust set you up with a longer free\ntrial.\nYeah.\nSo,\nhow do you guys imagine you guys\nfrom understanding it, it's you're\ncreating signal with your own models,\nwhatever you're using.\nmake our lives easier to identify\nsignals and\nharmful intents and behaviors. But how\nwould you maybe you have an example of\nhow you have that full stack of work and\nno\ndirect?\nSo if you're sending all the telemetry\ndata, we can find any exceptions in the\ntraces tool errors etc. And that's a\nthing that you can also track within\nraindrop and that's that is a explicit\nsignal. So uh there's like implicit and\nexplicit signal. So that becomes an\nexplicit signal. Um\ndo you do you so I think most of the\nobservability platforms will give you\nlike the agent trace the token usage uh\nif the tool call failed or not. Uh but I\nthink where we sort of like shine is\nsort of like the fuzzy part the fuzzy\nfailures right where the user like\nfrustrated which I think matters more\nthan uh the explicit signal that you\nsort of get from sentry. I mean\nobviously those are also important but\nwe focus a bit more on the fuzzier side\nof the failure space. Uh but at the same\ntime we also have a trace view. I think\nwe also have a very interesting feature\ncalled like trajectories uh which sort\nof like visualizes if you want to find\nlike uh a trace uh which has like three\ndifferent tool call failures. Uh so you\ncan actually\nbut\nokay let me just get in\nso you can sort of like describe the\nuh type of trace that you want to look\nat. Uh so\nlet's see hope we have like data but uh\nyou can more or less like describe uh\nthe type of trajectories that you want\nto see uh instead of just like\nconfiguring it. So we do both in a way.\nSo you can obviously set up uh tools are\nfailing sort of like alert as well.\nSo you can just search like for any\ntrajectory. Um\nso yeah you can see that\nthis is sort of how the tools are being\nbeing called in what order. You can see\nwhich ones have have errors. You click\ninto them. You can see the input and the\noutput to this specific tool like what\nactually screwed up here. Um and you can\nsee okay it's interesting that this has\nlike this the no one lets you this is\npretty much the only place where you can\nvisualize tools like this. Um, but you\ncan see here like you can get a shape,\nan understanding of the topology of\nwhat's going on here. And you can see\nwhen there's other ones that look\nsimilar, you can sort of see, okay, this\nkind of looks similar to this and then\nthat gives you a sense. You can do like\nsearch on this. Again, we have an agent\nthat can look through these and give you\na sense of what's going wrong. And so it\njust makes it really easy to find uh\nissues in in agents. Um, yeah.\nCool. Anything else?\nAny other questions?\nCan you export the data that you\nuh the directories data?\nYeah.\nUh what would you want to export like\njust the raw trace logs or what what do\nyou\nSo I think we usually what our customers\ndo is that they already have like a\nhotel stream, right? So we just end up\nbeing like another uh target I guess.\nBut at the same time uh they do want us\nto like export the signals that we\nlabel.\nSo we do support like uh bitquery and uh\nsnowflake. So we do export the event and\nthen the signals that were classified\nfor that event.\nLast questions.\nLet's do it a longer time frame.\nSo just go over the last month.\nSo you can see stuff like refusals and\nthen again if you click into any of\nthese you can get a sense of over time\ntask failure jailbreaking like what\nspecifically is going on and then you\nhave your self diagnostics ones as well\num capability gap etc. Cool.\nDo you have open data on number of\nchoices that you guys seems with your\nclients\nthe extremely valuable when you have\nthose\nagents at a very big scale and so I can\nimagine that you have a lot of data do\nyou have some\nyeah it starts being is your question is\nlike what's the smallest where it's\nuseful or what's the\nvolume of all the data that you're\nreceiving processing and generating\nsignal around like how many jailbreaks\ndo you see across all the\noh do we have any sort of like\nwe should we should do something like\nthat that would actually\nthat would be very interesting. We don't\nhave anything like that\nlike mixed opinions about that. I think\neat does it in a way, right? But\npeople generally have like negative\nreaction to it. Uh maybe it's different\nhere. But\nat the same time,\ndo our customers want us to do that is\nlike a different question as well,\nright? So\nuh\nbut yeah, we would love to but uh I\nthink they're like compliance reasons\nwhere we can't actually put a customer's\ndata out there.\nCool. Anything else?\nAll right. Thank you everyone.",
  "transcript_chars": 39492,
  "ingested_at": "2026-05-12T00:31:48.542193+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 4285,
    "like_count": 100,
    "channel_id": "UCLKPca3kwwd-B59HNr-_lvA",
    "categories": [
      "Science & Technology"
    ],
    "tags": [
      "ai",
      "ai engineer",
      "ai engineering",
      "software development",
      "tech",
      "startups",
      "software architecture",
      "machine learning"
    ]
  }
}