{
  "video_id": "5ID22ACI7IM",
  "channel_slug": "aidotengineer",
  "channel_handle": "aidotengineer",
  "title": "Mergeable by default: Building the context engine to save time and tokens — Peter Werry, Unblocked",
  "duration_seconds": 6085.0,
  "url": "https://www.youtube.com/watch?v=5ID22ACI7IM",
  "upload_date": "",
  "transcript": "All\nright, thanks everyone. Sorry about the\nwait. Um, this is going to be a a bit of\na strange session because um there is a\nworkshop component to this. So, uh I\nguess everyone will be coding on their\nlaps. Sorry. Um but anyways, sorry. I'm\nPeter and uh this is my colleague\nBrandon. Um so we're we're going to\nbreak this this session into two\ndifferent parts. One is a um sort of a\ntalk that I'm going to give about what\ncontext engines are are useful for and\nand how you might go about building one,\nwhat to think about. Um and then we'll\nwe'll launch into the u the second part\nof it. So um just briefly\num quick quick agenda. We're going to\ntalk about three myths that are\ncirculating uh right now about about\ncontext engines and then I'll go over a\ncouple of less or a few lessons uh that\nwe learned along the way building one of\nthese things. Um and then finally we'll\ndo this. So we're going to build a\nsocial engineering graph. Uh this is a\ncomponent that is super useful in a\ncontext engine. And first just a show of\nhands. Does everyone know what I mean by\ncontext engine or does anyone want\nclarification on that?\nOkay. So uh in the world of AI agents\nuh you have agents that uh when you\nstart off and you start coding they are\nbasically at ground zero. They have no\ncontext about your code, your\norganization, nothing. Okay. So\ntypically what happens is the first\nthing they do is they start to rip\naround your codebase uh based on the\ntask that you give them to try to gain\nsome understanding uh sort of background\nunderstanding before they start to do\ntheir task. So context engineering is is\nkind of the art of supplying uh all the\ncontext that you need and most\nimportantly none of the context that you\ndon't need in a highly optimized way so\nthat when the agent starts to run it\nexecutes the task uh in a streamlined\nway that's in line with your\norganization's best practices and\nexpectations and so on. Okay, so we'll\nget to this after.\nSo, not long ago,\nas in like four years ago or less, uh\nyou were the context engine. Okay? So,\nwhen when your agent needed something,\num you would prompt it, you'd grab the\nthe issue ticket, you'd hand it all of\nthe information that it needed to start\nits task. And in many cases, even when\nit was ripping around getting background\ncontexts, when it got to the end of its\ntask, sometimes it it got it wrong. In\nfact, many times it did. and you'd have\nto kind of like reset it. Um, reguide it\ntowards the solution that you were\nthinking of. Um, or if it completely\nmissed the mark, you'd have to be like,\n\"No, not the not the JavaScript dummy.\nIt's the Python source code that I want\nyou to look at.\" Um, so\nlet's just remember how you built\ncontext in an organization. Uh, so we're\ntaking AI out of the picture for it for\nfor just a sec. And I just want you to\npretend that pre- AAI uh you just joined\nan organization, let's remember how we\nbuilt it up. So\nover time, you would accumulate this\nkind of context through experience,\nright? You would start a job, maybe\nstart code splunking a little bit to\nfigure out um uh how things work. You'd\nmaybe latch on to a mentor. Um, and\neventually you'd you'd experience real\nthings like incidents and outages and\nthings like that. Those are sort of the\nthe pain things that stick with you.\nThose are the battle scars, right? And\nthat is what constitutes organizational\ncontext. It's the um it's the learnings\nalong the way, the why did we do things\nthe way we did it.\nAnd now you're good at your job because\nuh after all of that experience of pain,\nnow you know what questions to ask. You\nknow where to look when an incident\nhappens. And this is the goal. This is\nwhat we want to get for our AI agents.\nSo um I'm going to just lift this\nadoption curve from Vimath. And uh I I\nmay have butchered his last name, but\nsorry uh Vim if you see this. Um so\nlet's let's start at the beginning here.\nThis was like four years ago in 2022.\nEveryone everyone remembers fancy\nautocomplete, right? Um so back in those\ndays context windows and AI were pretty\nlimited. I'm not sure if everyone even\nremembers this, but it was like 8\nkilobytes or or 8k tokens I should say.\nAnd that's not a ton. And so tokens were\nhighly optimized and um agentic idees\nlike cursor focused just on the code\nthat surrounded uh the code that you\nwanted to go in and autocomplete. So\nbasically they took some code before\nthey took some code after they put it\ninto a model and they said this user is\nworking on this piece of code what's the\nmost likely next thing and that's what\nwas printed out. Um it got progressively\nbetter than that. as were uh were\nintegrated uh language servers and then\nit you you were able to basically pull\nlike collers of source code and pull all\nthat into context and then the LLMs were\nreally good at at completing uh code. Um\nso\nat those levels you were the context\nengine and uh in in many in many\ncircumstances here this is kind of where\nmost people are here. They're at the uh\nuh parallel agents hooked up with MCP\nand skills. Okay. Just super curious,\nhas anyone gone beyond curated context\ninto the the last uh few degrees of of\nagentic freedom, shall we say, where you\nhave background agents running in the\ncloud doing stuff in YOLO mode. Is\nanyone anyone experimenting with that?\nOkay, cool. That's that's very cool.\nThat's bleeding edge. Um, but let's just\ntake a moment to recognize that bleeding\nedge today is like yesterday's news in\nsix months. Okay. So, the the puck I'm\nCanadian, so I'm going to say this, the\npuck is going down down the line towards\nbackground agents for sure. Um, and one\nof the things that we run into right now\nis this. Uh, we're becoming the\nbottleneck as humans, right? I'm not\nsure if if people have tried managing\nparallel agents and uh working on\nseveral tasks at once, but everyone's\nstarting to feel this like cognitive\ndisconnect because you're context\nswitching all the time and it's just\nit's just really really painful. Um, it\nis very difficult to move from that mode\nwhere you're the human managing context\ninto the background agents mode unless\nyou have some kind of context engine\nthat knows how your code operates, how\nyour organization works and understands\nthe motivations for historical changes\nand things like that.\nSo, Andrew, Andre, he nailed it. Um,\nsystems are intelligent. We're reaching\nthe exponential on on intelligence for\ncode pretty soon. Everyone's seen the\nthe release about Mythos. Um even though\nwe all haven't had a chance to really\ntry it out yet. Uh the promise is that\nfrom a code intelligence perspective,\nthis thing is like pretty much close to\nto perfect. Um but so now the bottleneck\nis context. Of course,\nwithout um without context, I'm just\ngoing to re-emphasize this point. you'll\nprobably end up in doom loops. Does\neveryone know what a doom loop is? A\ndoom loop is like when you're uh you're\nstruggling with the agent. It's it's not\nquite doing what you want and you have\nto keep iterating on it. The worst case\nscenario is you run this thing in yolo\nmode and it finishes the entire task and\nit's completely wrong. You have to go\nback and correct, you know, various\nstages. Um so when you have a context\nengine, you can get there faster. The\nproblem is that access doesn't equal\nunderstanding.\nSo we have customers that are on various\nparts of I'm just going to go back to\nhere. We have customers that are on\nvarious parts of this journey.\nUm and one of the one of the interesting\nthings that we've noted is that uh\npeople feel that you know they're\nthey're they understand their\norganization best. So when it comes to\nfeeding the right context to these\nagents, people will try to build some\nsemblance of what a context engine\nactually is. They'll maybe build a rag\nsystem or they'll build some way to like\nfeed organizational data to an agent. Um\nso unfortunately though, uh access\ndoesn't mean understanding. So, what\nthat means is you could just wire up a\nbunch of MCP servers. Um, and it's not\ngoing to be able to understand what the\nrelationships are uh between all that\ndata, how it was, how it got there, and\nhow why it is the way it is. Um, and\nthen there's another problem which I'll\ntalk about a little bit later called\nsatisfaction of search. So, just\nremember that term. I'll come back to\nit. Um, okay.\nSo I just wanted to show you this. Um\nthis was something that uh we actually\nimplemented and we did it in in two\nparts. One was just without any context\nengine but wired up to a bunch of MCP\nservers. It did a pretty good job. Um\nbut then when we reached the end um it\nit it missed the fact that we had some\nlegacy stuff that depended on this old\num method of um of intelligence size to\nto anthropics. So they have adaptive\nthinking now but it used to be you had\nto supply a a token budget and that's\nhow like you could increase the size of\nthe thinking window. Um, so we we had\nsome code that kind of like depended on\nthis and there were reasons for that um\nthat the agent didn't understand or see\nand so it just basically clogged all\nthat code. Um, but when we added the\ncontext engine then it saw all those\nreasons and implemented it the right\nway. So it made the appropriate changes\nin the right places, included backwards\ncompatibility for the code that was\nusing the old method.\nOkay, so now for the myths.\nMyth one, naive rag over my docs is a\ncontext engine. Um, so\nif you implement um say like vector\nsearch um or just a couple of search\nmethods\nuh you're going to run into this you're\ngoing to run into a few issues. One is\nthis satisfaction of search problem\nwhere uh the agent will search like\ncrazy consume your tokens and then in\nthe worst case you'll reach compaction.\nOkay. So, um without being able to find\nthe the endgame,\num there are a few different other\ntechniques like you you need to have\npersonalization when you build a\nretrieval system because if you just rag\nall your data, especially for very large\norganizations, there's going to be\nthings like conflicts that you have to\nresolve in the data. Um it won't be\nfocused on the task that you're trying\nto perform. it might pull in, you know,\nrelevant code from other parts of your\norganization that especially if if you\nhave a really big org and you've got\ntons of different repos. Um, it's it's\njust going to create a huge mess. So,\nyou need to have some element of\npersonalization.\nAnd then here again, connect a bunch of\nMCPs. I'm just going to reiterate this\npoint. I'm done. Nope, definitely not.\nUm, so that that is the thing that that\nreally um puts an emphasis on the\nsatisfaction of search point. And I'll\nexplain that in a sec.\nAnd finally, a bigger context window\nwill solve this. Um, so way back, you\nknow, when the models were starting to\nget big, people were really excited\nabout a million tokens in your context\nwindow. Uh, the first models that tried\nthis, I think it might have been Claude,\nactually. Was it Claude?\nI think it was\nor OpenAI. Okay. Okay. Gemini.\nGemini. Yes. Sorry. I'm so sorry. Um, so\nyeah, Gemini first first model to try\nthis and it was really good at finding\nneedle in the hay stack. So you could\nfeed like a huge document to it and as\nlong as you you knew what you were\nlooking for ahead of time, it could find\nit. Um, but it wasn't good at all at\nreasoning across different data sources,\num, understanding the real meaning\nbehind a problem and then recommending\nthe appropriate solutions. So none of\nthat was possible. Um, obviously things\nhave things have gotten much better.\nNow, the problem is most organizations\nhave more than a million tokens worth of\ncontext. So, trying to fit all that into\nthe context window isn't going to work\nanyways. Let's project out to the future\nand imagine that you could fit like 10\nmillion tokens, 50 million tokens. Um,\nat the current uh rate of memory\nconsumption um just to operate the\nmodels, that's not going to be possible\nfor a really long time. even if it was\nand you fit all that context in your\ncontext window, you're still going to\nrun into problems with understanding\nwhat's true, what's false, um how to\nselect the right information. Okay, so\nnow I'm going to come back to this\nsecond point here, satisfaction of\nsearch. This is a a term that actually\ncomes out of uh the medical field in\nradiology. And the idea is that um when\ntechs are looking at x-rays uh and\nthey're looking for uh the cause of of\nof symptoms, they might find something\non the x-ray that explains those\nsymptoms and then they stop. Um and\nthat's that's kind of like a dangerous\nthing medically because uh there might\nbe other indicators for things like\ncancer that get missed. So, uh,\nsatisfaction of search is a is a real\nproblem in radiology and there's lots of\nprotocols to prevent just stopping as\nsoon as you find the first thing. Um,\nthis is what happens with agents when\nthey search around in say uh, notion and\nyour code uh, confluence, they'll\nstumble across what looks like the the\nthing they're looking for and they'll\nstop and then they'll they'll proceed.\nBut the the real like golden nuggets of\ninformation might be in a different\nplace that the agent wouldn't think to\nlook like in a in a past Slack\nconversation or in an incident report,\nsomething like that.\nSo here's the the classic iceberg meme.\nUm\ncode that compiles. That's like the\nbaseline. Does the agent produce code\nthat compiles? Um but everything that\nthat is actually important is happens\nunderneath here. So understanding the\nuser's original intent um what was\nrejected in the past by the team and\ntried before but failed. Uh how are you\ngoing to surface that kind of content\njust by looking at docs and and code and\nstuff? Um so you need to understand that\nsomehow.\nUm and even worse like it it's it's\nsometimes hard to know uh when things\nwere deleted like in the absence of\ninformation. So you need history as well\nleading up to decisions.\nSo this is why we think you need a\ncontext engine. Uh a context engine\nunderstands who you are, what team you\nwork on, who you work with, who the\nexperts are in your organization,\num and and what the decisions were that\nled up to the current iteration of your\ncodebase. it's able to resolve\nconflicts. Uh so this is like a truth\nand false type situation. What's true,\nwhat's not. Um sometimes that truthiness\nis a gray area, right? So the context\nengine needs to also understand when to\ninstruct the agent um that it wasn't\nable to resolve a conflict and then uh\nlearn from additional user input.\nUm this third point is super important\nof course in any large organization or\nenterprise. Um there's often you know\nrepositories that not everybody can\naccess secret projects that sort of\nthing. So uh it's really important that\nyou flow the access controls up. We have\nI'll give you an example that everyone\nwill appreciate which is Slack. Um our\ncontact engine integrates with Slack or\nMicrosoft Teams. Um, and when you have\nuh private channels that that's really\nhighly sensitive, right? Like you could\nbe discussing HR information or uh maybe\nsomething that you just really don't\nwant um everyone else to see. And so\nwhen when unblocked answers questions,\nuh it will use private channel\ninformation, but it won't it it will\nonly use that information if the person\nthat's asking the question has access to\nit. And then those answers are not\npublic. Okay? So they're they're private\nto you.\nUm, and then finally, of course,\ndelivering the right contest at the\nright time. And this is about token\nefficiency. It's about getting to the\nanswer as quickly as possible.\nSo, here's a kind of a highlevel\noverview of how how a context engine\nmight work. Um, on the left we've got\ndata source inputs. So, things like\nplanning tools, docs, conversations,\ncode, PRs, basically like anything\nthat's relevant to getting work done at\nthe engineering level. Um and then on\nthe right side we have the outputs. So\nyou know th this all can flow to coding\nagents CP or CLI tools. Um you can\ncustom build apps through the API. We've\ngot a we have a code review uh component\nthat just plugs right into your SCM and\nprovides code reviews and of course uh\nintegrations with social messaging apps.\nSo the these are the kind of like broad\nsix requirements that we think are\nimportant. There's actually much more\nthan this but these are the highle\nthings. So again unified system contexts\num this is about\nbuilding relationships between data.\nOkay. But it's more than just um\nrecognizing when uh one piece of data is\nrelated to another. Like for example, in\nin Slack, you might have conversations\nabout PRs. That's an easy linkage\nbecause posting links back and forth. So\nthat's easy. Um what's less easy is\nunderstanding uh the reason why\ndecisions were made or your\norganization's best practices, right? So\nto understand that you have to go a\nlittle deeper. um do do things like\ndistill um historical poll request\ncomments on PRs and uh try to distill\nthose down to the their core essence and\nthen when you see repeated patterns uh\nyou can pull those patterns together and\nstore them as you know quote unquote\nmemories so that when uh someone is\nworking on a similar piece of code you\ncan load those memories and then the\nagent can see that and go oh yeah right\nuh this is the way this organization\ndoes this particular thing.\nUm, conflict resolution super important.\nUm, we took a initially kind of a naive\napproach to this at first and based it\njust on recency, right? So, we we would\nbias towards newer stuff. Uh,\nunfortunately, when you have in the\nfullness of all your context, recency is\nnot enough. Um, often you have people\num, writing documents or chatting in in\nin their messaging platforms and they\nmight be saying things that are not like\ncompletely aligned with uh, how the the\nsystem works. Um, uh, so you know then\nwe started to bias towards code. So, we\nhad recency and we're like the main\nbranch is definitely your source of\ntruth, but not always because sometimes\num what's important is what happens\nnext, not the way a system currently\nworks. Like when when you're working on\na task um what you really want is for\nthe agent to understand where you're\ngoing, not necessarily where you've\nbeen. Where you've been helps it\nunderstand what not to do. where you're\ngoing helps it understand what you\nshould do. Okay. So, in in the Slack\ncase, looking at the conversations that\nyour organization's experts are having\nis more important than just\nunderstanding what you know, every\nrandom engineer is talking about.\nUm, targeted retrieval and personal\nrelevance are very related. So, I'll\njust talk about them uh together\nbriefly. So, um, again, like when you're\npulling context in, it's important that,\nuh, you're only pulling context in for\nthe relevant task at hand and probably\nrelevant to you. So, here's a technique\nthat's kind of interesting. um you can\nunderstand what repos a person works on\nmost by the number of PRs they submit\ncontributions and then if you do a if\nyou're doing vector retrieval you can do\na uh deep retrieval on those focused\nrepositories and then a wider retrieval\non you know the rest of the source code\nand then sort of bias the the selection\ntowards uh the focused repositories\nbecause that's more likely where\nsomeone's going to be working and\nspending their\nUm, and then you know, we've talked\nabout data governance, so I don't think\nI need to go over that again. Super\nimportant though.\nThis was just a a little experiment that\nwe ran uh with a larger task. Um, I'm I\nfully admit that some of these numbers\nare a bit wonky. This is basically like\nClaude outputting numbers. So, don't\ndon't trust it. Just trust the the vibe\nof the thing and not necessarily the\nnumbers. Um, basically what it's saying\nis that when we started out uh without\nthe MCP server act or sorry without the\ncontext engine active um it it really\nmissed the mark on a lot of stuff. Uh\nand that's just because it didn't\nunderstand how um the existing\nimplementation really worked and why it\nwas the way it was, what was tried\nbefore and failed. Um and so it made a\nlot of those same mistakes. uh with a\ncontext engine turned turned on\nobviously it it um it nailed it. The the\nkey numbers though are the the time and\nthe tokens that it took. So without um\nthe context engine took two and a half\nhours to finish this task with 21\nmillion tokens which is a lot of tokens.\nUm but with the context engine it took\nonly 25 minutes and 10 million tokens.\nSo it's it's a pretty dramatic\ndifference.\nUm okay so the hard lessons these are\njust samples by the way but the these\nare ones that we thought were kind of\ninteresting. So first of all uh\ninitially we optimized for access not\nunderstanding. So we our our first\npremise was if we just wire up a bunch\nof tools\num and provide a a knowledge graph it\nwill be able to traverse the knowledge\ngraph and uh execute a bunch of\nretrieval specific tools for particular\nintegrations and so on and figure\neverything out. Um that does not work.\nSo uh you'll have to go a little bit\ndeeper than that.\nUh second one is we hid conflicts\ninstead of surfacing them. So um by con\nby hiding conflicts I don't mean that we\njust ignored the conflicts. What we did\ninstead was we tried to resolve those\nconflicts using those naive strategies\nand we didn't surface the conflicts that\nwe weren't able to resolve. So this was\na really good learning is that um a\ncontext engine I mean we'll get there\neventually probably but uh it can't\nalways tell uh what the truth elements\nare and when it can't you should surface\nthat and learn from it. That's the key\nthing.\nAnd then finally I think a lot of folks\ntried this. This is a really bad idea.\nSo when when a context engine supplies\nan answer um do not cache the answer and\ntry to serve that same answer up again\nuh to a similar question. The reason is\nobvious is is fairly obvious in\nretrospect but um everything changes\nconstantly right code changes docs\nchange the reason for things change. So\nthis just doesn't work. Um the other\nthing is if you try to uh use the the\nprevious answers as context for new\nanswers, you regress towards a mean. So\nif the model is like misbehaving or\ndoing something bad and you continuously\nbring that into context, you're\nobviously going to pollute uh the\ncontext.\nAnd this is what happens.\nOkay. So let's now talk about where AI\nforward teams like like those that are\ndoing this like cloud-based agent thing\nare are using and taking advantage of\ncontext engines.\nUm definitely and especially during the\nplanning phase. Okay, this is where you\nget the biggest bang for buck\nunquestionably. Um get the context\nengine involved, use a skill to bring it\nin. Um connect it to the MCP server and\nand watch it do its thing. it. This is\nwhere you get the biggest bang for buck.\nIt's also useful to do this during\nreview. So you get planning and review\nat the end. Um because you know if if\nyou get an agent to do review, it's\nbasically just going to pay attention to\nthe code and try to understand where the\nbreak points are um security concerns,\nthat kind of thing. But without the\norganizational context, it doesn't\nunderstand the motivation for it. So\nthat's the really important thing.\nPick enrichment. Um, this is a a super\ncool use case. So, you create a ticket\nfor a new feature and then you just ask\nthe agent that's connected to a context\nengine to fill in the blanks. Works.\nTriage. Uh, I use this all the time.\nWhen I see an issue in production, I\njust whack it into an agent connect to\nthe context engine and it just like\ninstantly brings up all the past issues\nrelated to this and um, starts operating\nright away.\nIncreasingly we're seeing this one\nincident management. Okay. So we we just\nuh wired up data dog and this sorry\nsentry and data dog sorry. Um and this\nis already proving like super cool use\ncase. It uh it can see the signals and\nthen it can act on all the signals and\nrelate that to code uh relate it to past\nincidents that you and discussions that\nyou've had in Slack. Having all those\nthings come together at once is is\nalmost like magical. And finally, I\nthink this one's actually my favorite\none and it's the one that customers use\nthe most is uh customer success and\nsales and engineering support. So what\nwhat a lot of big teams do is they have\nengineering support channels where other\nteams can come in and ask questions. If\nyou put a context engine into one of\nthese things, you can have it\nautomatically answer a lot of questions\nand save engineers a ton of time.\nAll right. So, how teams make a context\nengine their own skills. So, definitely\nbuild uh skills that you can use to\ncurate context in a GitHub repo.\nUm and you can build other skills around\nit like typing ticket enrich give it the\nissue ID and then it it can use the\ncontext engine to build the enrichment.\nuh workflows like this one prepare\nprepare an incident timeline um and then\nyou can just send it off to your agent\nagain context engine blah blah blah\nbrings everything together magical\nand this thing here um you can wire this\nup to all kinds of agents I've got um uh\none one of the things that a lot of\ncustomers like to do is wire this up to\nclaude code in their CI system um we\nactually do have a code review component\nso you don't have to do this if you're\nusing unblocked Um, but people use this\nfor other things, not just code review.\nAs soon as you wire up a context engine\nin the background, give it an API key,\nlet it let it run on its own, it it can\ndo some pretty insane stuff. Um, so I'm\njust going to show a quick\num\nexample of what wiring up a context\nengine can do. So this is a PR that uh\nmy colleague wrote and uh it it\nunblocked like went through and provided\na a kind of review to this thing and at\nthe bottom of this review here's the\nreview part. Um you can see that Richie\nwho was the author of this PR was like\nvery cool this is something I would say.\nUh now the reason for the comment which\nwas you've basically duplicated a bunch\nof tests you can you can kind of dry\nthat up a little bit is because um this\nwas a best practice that was distilled\nfrom a bunch of other PRs and the the\nfunny part is that the author of those\nPRs was Richie. So he's the one that\nactually instilled the best practice in\nthe organization. Uh so that was that\nwas just a cool little moment when we\ndiscovered that. Um, here's another\nexample. So, this was a it's a fairly\nlong transcript. I'm not going to like\nshow the whole thing, but we we sent it\non a on a mission to do a big large\ntask. Without uh unblocked, it it uh\ntook quite a while. Like you can see\ntranscripts quite long. Um, and it it\nmissed a whole bunch of stuff. With\nunblocked, uh, it was a lot more\ncompact. It it got to the answer like\nvery quickly and correctly. And just\nbecause we're now AI forward and lazy,\num, we took both of those transcripts\nand ran them into Claude and just said,\n\"Hey, Claude, why don't you just do a an\nanalysis of both these things and give\nus your give us your result.\" Um, so it\nit went through I won't, you know, bore\nyou with the details, but just to say\nthat at the end, the verdict is that uh\nthe context engine plan is is what I'd\nship with. This other one is good for a\nprototype, but it's missing a whole\nbunch of stuff that is important to this\norganization. It was previously\ndiscussed. Okay,\nso this is essentially what what I've\nbeen trying to say. Uh AI generated code\nshould just feel like it was written by\nsomeone that's been in your team for\nlike 20 years. Okay. Um it doesn't if it\ndoesn't yet, that's fine. Um it will um\nyou're if you wire up unblocked you'll\nyou'll see like a a huge difference in\nperformance of agents and if you're\nbuilding one of these things absolutely\nlike take all these things and and build\nand and let's see where that goes.\nSo just before we get into the workshop\ncomponent um maybe we'll just have like\nfive 10 minutes of Q A.\nI'm Brandon\nand this is Brandon. So he'll he'll help\nwith\nthis.\nThanks. Um, so it's clear what it does\nfor you and what kind of problems it\nsolves? But to me, a big question mark\nis what is the thing? What is the\nartifact that that fits the bill? Is it\nlike a program you install, an API\nthat's hosted remotely, or an MCP\nserver? What is it?\nIt's it's all of those things. So, a a\ncontext engine, I'll explain what\nunblocked is. Maybe I can just show a\nquick demo of it. Um, so broadly\nspeaking, there's a bunch of different\nsurfaces to a context engine. You want\nto get it into your agent flow, and you\ncan do that with an MCP server. You can\ndo that with a CLI tool, for example.\nUm, we also have this dashboard surface\nwhere you can ask questions about your\ncode. Um, this is a pretty basic one,\nbut you can see it understands who I am\nand what I've been working on. Um, and\nthen, uh, we have a Slack, we have Slack\nconnectivity as well. So, you can bring\nunblocked into Slack. Um, drive it in\nconversations and have it auto answer\nthings. Does that make sense? Did I\nanswer your question or\nOkay.\nYes. API, CLI, MC.\nYeah.\nYeah,\nsorry.\nThank you. Um, so my question is, so as\nfar as I understand is like a knowledge\nmanagement and retrieval um application.\nYeah. And does this relate somehow to\nthings like um LLM wiki like it was made\npopular recently by Andre Karpati or the\ndecision traces and context graphs\nwhich was discussed a lot a few months\nago.\nYeah. So you can think of all of those\nthings as kind of uh useful components\nto a context engine. A context engine\nhas to do much more than that because um\nso agents are really good at recursing\nthrough a wiki for example. Depends on\nhow you build this wiki because there's\na bunch of things like organizational\nmemories, best practices, you know,\nexperts in your organization and that\nare used as pivot points for context\nretrieval. So a a wiki doesn't solve\nthose problems unless it has like a you\nknow you could build a structure with\nit. And I think uh Carpathy discovered\nthat if you treat a wiki as um kind of\nlike a file system, you can break it\ndown and have the agent uh whack through\nit like a file system. They're by the\nway, agents are like highly optimized\nfor file system traversal.\nYeah. The compilation step. Exactly.\nYeah. Yeah, sorry.\nSorry, maybe the same question, but is\nit is it a general purpose context\nengine or is it targeted against uh code\nbecause will it be useful as a say uh as\na business domain expert uh or sort of\nbuilding up a business domain and then\nhaving this context engine use my so I\ncould all my other I agents could use\nthis as context for the business. uh or\nwould you say that is more like just for\nthe code part of it?\nUh so it it's definitely engineering\nfocused the the integrations are focused\non engineering activities. So you know\nSCM integrations and other other tools\nthat engineers use um we are\nincreasingly seeing customers using it\nfor other purposes. So business\nintelligence is a key thing. Uh and\nthat's usually useful when uh people in\nin business functions are trying to get\nan understanding of the product and its\nfunction. Um we don't have uh like say\nSalesforce integrations wired up for\nthat. So you couldn't use it to\nunderstand um you know any anything\nthat's salesreated. It's it's really\nprimarily an engineering focused context\nengine. That's not to say that that\nwon't change.\nYeah,\non the governance thing, if you're\num respecting access rights, how can it\ndo sort of synthesis across stuff and\nthen develop new knowledge inter\ninternally that it could then surface to\npeople?\nSo that yes, you're correct to point\nthat out. The the synthesis um is\ncompartmentalized.\nSo there are, you know, places that are\ncompartmentalized like individual\nrepositories. That's kind of the level\nof access. So if you can synthesize uh\nhistorical data based off of that um and\nthen correlate that with public Slack\ninformation, then that's that's one way\nto do synthesis without crossing the the\norganizational boundaries. Um so\nuh the you know the other way is to look\nat and tag when um synthesized\ninformation crosses those organizational\nboundaries and you can take something\nlike a group ID approach to that problem\nwhere you attach group ID tags to the\nsynthesized information and then only\nretrieve it if the person that uh has\naccess to that can can build it out. So\nfirst take the compartmentalized\napproach because that's the where you'll\nget the the most mileage and then you\nkind of build up from there. I mean this\nis the core problem with using a\ntechnology like graph rag right because\ngraph rag is like a pyramid where it\nbuilds up in layers and then basically\nsummarizes each layer but that like\nunavoidably crosses uh permissions\nboundaries. So you have to be you have\nto create compartmentalized pockets.\nYeah.\nIt's a good question.\nYeah.\nYes. You've talked a lot about like all\nthe different sources of information\nthat you consume and putting them all\ntogether. When it's like synthesizing\nthose down, is that still sort of like\nnaive rag, vector search, all that stuff\nunder the hood? Or is it like agents\ndeciding what is appropriate? like what\nwhat or probably like combinations of\nall of them, but what is that sort of\nstep?\nUm yeah, you're right. It is a\ncombination of all of them. So knowledge\ngraph like knowledge graph buildup\nhappens in a bunch of different ways.\nUm the the PR thing that I showed you\nfor example is like a first you build a\na naive knowledge graph procedurally and\nthen from there you can use an LLM to\ndistill down and summarize and build up\num those types of techniques. Um our\ncontext engine builds first like a\nknowledge graph from the base uh using\ntrying to leverage like all the\ndifferent entities. It's kind of like a\npage rank thing where it builds up the\nrelationships procedurally and then of\ncourse it vectorizes data. Um and then\nthere are procedural tools that fetch\ndata at runtime. Um a lot of the\ndistillation for uh you know conflict\nresolution happens in two places. So one\nis like during data ingestion time\nthere's there are tags that relate data\nto each other so that we can see if we\ncan deconlict at that level and then\nlike rank against each other at that\nlevel and then of course at runtime you\nhave to pass the things to a judge with\nthe criteria um and then it does\nadditional deconliction in real time.\nDoes that make sense?\nYeah.\nOkay.\nOne more question. So uh I was curious\nyou said conflicts but at some point you\nget conflicts that something means\nrevenue for one company and means\nrevenue for another company isn't a\ntotally different meaning how you can\nrecognize that so how do you get humans\nin the loop how how do you use their\nontologies and how can you do you use it\nwhen you run into it so I'm very curious\nabout that actually how how\nyeah so if you I can show you just a\nquick thing here so um you'll notice\nthat at the bottom the the references\nthat were used for answers are delivered\nboth like to the human in this interface\nbut also to the agent. So um if the\nagent if the context engine isn't able\nto do the deconliction then at this\npoint here the human can step in and\nguide the agent when there are enough s\nso you can you can literally just reply\nand say like that's not correct or you\ncan come here\nand\noh yeah sorry\nyeah or you can you can do this like not\nhelpful and and give the reason why um\nlike it is a bit of a manual process at\nthis stage, but the signals that build\nup over time,\nit's funny, right?\nYou might have catch a lot of human\nintelligence by this, right?\nYeah, that's amazing.\nYeah, for a typical customer, how much\ndo you have that metric?\nOh, it's huge. It's it's it's amazing.\nLike I I was actually really surprised\nby how willing people are to give\nfeedback. Um, yeah. No, it's\ncan you\nthousands or hundreds\nof\nI mean at at small team size it's you\nknow in the hundreds at so small team\nsize being like 20 30 people at large\nteam size 100 to 200 people it's like\nhundreds and hundreds of\noh wow\nof feedback. Yeah\npeople just really like to interact with\nagents and tell them in natural language\nwhat's wrong. It's It's just a totally\nnatural thing to do.\nYeah.\nCool. All right. Um are we are we good\nfor Q A\nand then we can get on\nask questions as we hack. But\nyeah, let's let's get on to the let's\nget on to the workshop part of this. So\num we have created a um for actually\nwhat I'll do is I'll just do this first.\nSo you can do this now if you'd like. Um\nI will come back to this slide in a sec.\nSo the idea here is we're going to get\neveryone to join a Slack workspace that\nwe created and then we're going to get\nuh everyone into a repo where this um\nwhere this sample code lives and then\nwe'll just start hacking away on it\ntogether. Okay.\nYeah.\nI got some people coming.\nNice.\nWhen you drop in, you'll see an AI\nengineering London channel. Hopefully\nthere's a link to\nthis drop\nand\nthe unblocked link will not work until\nyou do step two. Yes.\nTo get into the GitHub or\nOh, no.\nI've got I've got many people coming in.\nSo, I'm really hopeful that\nis it network? Yeah.\nWe will find out.\nUm, okay. While while folks are doing\nthat, I'm just going to show you what\nwe're getting into here. So, this is the\nuh GitHub organization. Um, what we're\nwhat we're working on is a social graph\nbuilder. So, what this is going to do is\nlook at a source code repository. So,\nyou can run this on your own repo. It's\nnot going to upload anything. It's all\nlocal. Um, so that you can see this\nthing building up against your own\norganization.\nUm, and it's going to do a bunch of\nthings. We're going to get basically a\nsocial graph out of it, and I'll show\nyou what that looks like. And we're\ngoing to understand who the experts are\nand which parts of the code they work\non. Um, and then there's going to be a\nlittle like interactive visualization\nthing. So, what what the goal of this\nexercise is is to get this thing up and\nrunning and start just start hacking\naway on it. So, like start submitting\nPRs as soon as uh we get this going. Um,\nso this is what it looks like.\nThis graph here is our organization\nunblocked. And uh what you're seeing\nhere is a a relationship graph that\nshows who's reviewing whose PRs um and\nwho's who's getting reviewed.\nEssentially\nthe uh this thing is\na distillation of all the different\nteams within on blocks. So this is\nroughly accurate actually. Well, not\nroughly, it is pretty accurate. Um,\nwe've got I I did this all the way back\nto the start of 25, 2025. When you run\nthe thing, I'd recommend maybe doing it\nfor a shorter timeline because it will\nbe a little bit slow uh if you go all\nthe way back to 25. Could take like 15\nminutes. Um, but it's effectively\ndistilled who the teams are and um you\nthe only AI step in this is to label the\nteams. You don't have to run the AI step\nif you don't want to. it'll just use the\nthe parts of the code that people work\non the most. Um, this tab here will show\nthe experts in the organization and what\nthey work on. So, this is just broken\ndown by uh project area and path um and\nshows like what areas of the code have\ngood coverage. coverage is defined\nmostly by whether a a high contributing\norganizational expert is present and\nwhether uh it's it's an actively\ncontributed to part of the code.\nAnd then finally uh we'll have this\ninteractive graph that um breaks things\ndown by team area and we'll show like\nyou know who the major contributors are.\nI'm over here on the AI team. Um yeah,\nso that's it. Let's get everybody in and\nwe'll start hacking away at this.\nYes, absolutely.\nYeah,\nmany of you should have an invite who\nhave put your GitHub already in. So,\nplease give it a check. GitHub is the\nworst.\nYeah,\nwe will. It is an MIT license. We will\nbe making it public later, but for now,\nwe needed it locked down.\nOh, did you still\nI'll show you.\nWhat have you done?\nI also slight.\nSo, I think the the rest of this session\nis going to be now just like hacking\naway. So, um in a sec here, I think I'll\nI'll take this this down if everyone's\ngot it. Um so, so that Brian and I can\nconcentrate on working with you guys to\nbuild features.\nOh, when you submit PRs, by the way,\nyou'll notice that unblocked is sitting\nthere as a code reviewer. So, don't\ndon't feel badly if it uh sprays on your\nPR a little bit.\nDepends on how much\nYou got one heck of username. Good work.\nIs Is everyone good with this? I take it\ndown. Okay, cool.\nThat is it.\nBrandon, you're you're on top of the\ninvites. Okay, cool.\nThere's a few more. I'm on to\nChris\nhas given two.\nThat's okay. I'm gonna send both. Don't\nworry.\nThat's just where I am in this list.\nOh,\nforgot to mention a couple of things\nhere actually.\nYeah.\nComing back live. Yeah, good. Um, just a\ncouple of things. So if if uh you're\nlooking for something to implement and\nstarting with with any with coming up\nwith ideas and stuff, there is a uh a\nset of sort of predefined issues that\nyou can hack away on. So you can just\ngrab one of these, whack it into Claude\nand see how it does when it's connected\nto the context engine.\nUm, the MCP server for unblocked is\nhere. So, if you want instructions on\nhow to wire this up to uh claw code or\nanother agent, then you can grab it from\nthe instructions from here.\nAll right. So, I'm at Lars. There's two\nmore in here. So, I'm still going, by\nthe way, for those just adding\nWhat's going on, Brandon?\nOh, sorry. Just one of the usernames is\nOh, okay.\nYou should have\njust behind\nChristopher, did you not get invited\nyet?\nNo, I didn't.\nThat's weird.\nLet me double check. You should have\none, but\nyeah, I should. You should have an\nemail. I'm up to like one of you. So\nhard.\nIt's like five clicks to add a member.\nI'm like,\nI was going to say should be able to use\nthe CLI for this. What's going on?\nThat's right.\nNo, they keep putting it in my PR and I\ndon't want it there. Copilot's going to\nreview for me\nvery poorly, but it will\nquestion.\nYeah, for sure.\nDo Hold on. Let me grab you the mic.\nHopefully that's on.\nDoes it work? Yes. Um, so I guess that\ncontext engine works very well for\nasynchronous agents so that you don't\nneed to specify things on your keyboard\nbecause they can fetch what they need.\nThat's one of the main uses I I guess.\nAnd\num so it plays very well uh I think with\nagents like Copilot on GitHub. Do do you\nsee uh if you can share it uh which\nagents are used most with unblocked\nwhether it's more because on the wild as\na developers with our laptops I think CL\ncode is much more used than copilot but\nmaybe you see a different picture.\nOkay so I'm going to take this off the\nscreen for a secure\nand try to see if I can pull that up for\nyou.\nUm, but the answer is yes, we do know\nroughly what that breakdown looks like.\nSo, let me grab that.\nOkay.\nI think this gives you kind of the rough\npicture.\nOkay. So, this is kind of the rough the\nrough picture here. Um,\nunfortunately because of the way that\nthis is I should probably like\nextend the screen, but I'll just step\nover here. So, uh, cloud code is by far\nthe most used. Um, followed this is the\nthe next one is cursor. So, that that\nseems fairly obvious. This last one here\nis kind of a catch-all, but what's\nreally interesting is that a lot of\npeople use cloud desktop, which which\nwas very unexpected, but this is the\ncase. Um, so, and then VS Code and\nCodeex account for a much smaller\ncomponent, but yeah, it seems like\neveryone's using either cursor clog\ncode. I would have expected more of, you\nknow, totally a synchronous agent, like\nsomething that people would just run\nfrom a PR. Okay, you can run code from a\nPR, but it's less common. Maybe\nsometimes you use copilot because it's\nbuilt in.\nYeah, actually this this one here, cloud\ncode, um may may capture some of that\ntraffic. So that that's probably what\nyou're seeing because people will wire\nup cloud code in CI\nand do things like that.\nThanks.\nNo problem.\nI've got a potentially dumb question.\nThere's no dumb questions\nthis. Well, we'll see.\nActually, you know, you know,\nyou soon.\nI I I had a teacher in grade three that\nused to tell me, \"There are no dumb\nquestions, only dumb people.\" Go on. I\ncould I could be one of them. Um, how\nlike from from your point of view,\nright, you've got you you can use like\nsub agents from like an exploratory\nstandpoint.\nYeah.\nHow how how does like that plus memory\nplus just like storing snippets of\ninformation that might be able I I'm\nthinking of the like social graph that\nyou just showed, right?\nYeah.\nEven in an organization that's like\nseveral thousand people, you would be\nable to store that in a very small file.\nNo.\nUm you you would as the graph that you\nshowed.\nUh oh, I see the social graph component.\nYes. Yeah, it can be compact. I'm\ntrying to understand how this compares\nlike what's the kind of like USP\ncompared to the exploratory agents and\nrepeating that.\nI I see what you're saying. Okay. Um\nso there there are two there are two\ncomponents to that. One is that uh an\nexploratory agent would have to do this\nevery time. So when it starts from\nground zero, yes, it might be possible\nfor it to reconstitute\na sort of social graph hierarchy, but it\nwould have to do two things in order to\ndo that. One is it would actually have\nto write code in order to constitute the\nthe graph, at least the way that agents\nare today or the way that the models are\ntoday. You wouldn't be able to just have\nit like run basic tools around um the\nthe organization and figure out the\nwho's who. um it would have to write\nkind of like what that social graph\nalgorithm is, run it and then get the\ndistillation out the back end. So um at\nthat point you're basically getting\nclose to that component. Anyways, so\nthat you short circuit it and just run\nit and use it. Um maybe I should explain\nsome of the motivation for that thing.\nActually I I realized now that I may not\nhave done that effectively. Um, social\ngraph is not just about conveying\ninformation about who the experts are.\nIt's used within the context engine as a\npivot point um into more like important\ncontext. So understanding who the\nexperts are in a particular code area\nacts as a jump point because um another\npart of a context engine which happens\nat the ingestion and processing layer is\num distilling the um we call it bottling\nthe expert but it's essentially\ndistilling what that individual has\nworked on in the past. uh where they sit\nin the in the kind of hierarchy of the\norganization um the the decisions that\nthey've made based on Slack\nconversations that they've had based on\ntheir PR comments all this kind of stuff\num when you distill that down it's and\nyou pass it to the agent then what\nhappens is like let's say that I'm a new\nemployee and I'm coming to work on a\nparticular area of code um there are a\nbunch of different ways of loading\ncontext for that code one is you know\nsemantic search via vector vector\nsearch. Right? So that's kind of layer\none. Another layer is uh pre-built\nmemories. And then the the third layer\nis bottling unbottling the expert for\nthat area of code. And getting that\nexpert's learnings into context is is a\nreally powerful mechanism. It helps\ndrive the rest of the retrieval in an\nagentic loop and it helps um the agent\nuh directionally like where to go next.\nDoes that make sense?\nRight. I think everybody's in now.\nAwesome.\nSo, let's uh\nOkay. So, I think we're if we're all in\nthen\nuh the next thing here is\nonce I get this back up on the screen.\nI'm still I'm still sending invites. I\nsaw someone just So, please keep coming\nand we can keep going.\nYep. So, um, feel free to basically just\nfire this repo at your agent and get it\nto like run it. If you if you literally\njust say to Cloud Code, run this against\nmy repo, um, be sure to give it a time\nrange or a PR limit, otherwise it'll go\noff the rails and take a really long\ntime to finish. So just say like process\nthe last like 300 PRs or process up till\nyou know September 2025 or something\nlike that. Um there's enough information\nin the readme that it should be able to\njust do it and just run it against your\nrepo.\nI get cloned said read the read me and\nmake it happen.\nYeah.\nCan I ask another what's your\nWhat's your plans for the coming year or\nsomething\nfor for unblocked\nis it is it about unblocked or or about\nthis this sort of side project\nthis\num so I mean I've I've sort of alluded\nto this before but like where the puck\nis going is with fully autonomous agents\nSo we're very focused on making sure\nthat autonomous agent flows are highly\noptimized. You, as I was saying at the\nbeginning of the conversation, you\ncannot run those things effectively\nwithout like um very finely tuned\ncontext.\nYeah. So when you think about it\nat some point\nI read things like tracing what what do\nagents and you get run books out of\nthose is that is that the path you're\nyou're investing in or what what is it\nretrieval what what\nare you talking specifically about\nincident management then or\nsorry\nare you are you speaking specifically\nabout incident management management,\nthat sort of thing.\nNo, I'm I'm speaking about your I'm\nthinking actually more from a business\nperspective. How can we extract business\nknowledge that's really deeply embedded\ninto systems nobody knows anymore and\nsome people know think they know but\nthey don't know.\nYeah.\nUh and documents, human knowledge,\nright? Tested knowledge.\nYep. So, I mean there's there's two ways\nof servicing that either at the product\nlevel or um through the context engine\nitself. And increasingly what we see is\nthat people leverage uh agents to do\ntheir work even at that level. So\nthey'll they'll go to cloud code,\nthey'll connect the unblock context\nengine, it'll be like do this thing for\nme and then the context engine will find\nall the things that it needs to do that\ntask and it it'll surface that data.\nYeah.\nFor us that means the first near-term\nroad map is API.\nYes. It's like CLI\nCLI API\nquestion\nor it's just good that\nCool. I'm going to lift this off again.\nHopefully people start submitting some\nPRs and we can\nYeah,\nyou're in that GitHub. Let me actually\nrepost it in the Slack channel\nbecause that link will\nso this this org will stay up until the\nend of the week. Um at which point we'll\nbasically bring it down and um release\nthis uh as open source and uh everyone\nthat contributes obviously is going to\nget credited. So um you your name will\nbe on it.\nShould\nwe like\nset up the repo locally and then start\ndoing what's basically? So I just\nfinished setting up\nYeah, just just clone the repo. Um you\nthe easiest thing to do is to take uh an\nagent like Claude and point it at um\njust launch it from that repo from that\ndirectory and just say please uh\nbootstrap and launch this this product\nand away it will go.\nIf if you guys run into any kind of\ntechnical things, we'll we're here\nobviously. Yeah.\nLet's hold on. Let's get you the the\nmic. Oh, you got it. I've got a I've got\na lapel now. So\nawesome. Cool.\nYeah.\nCan you hear me? Yeah. Perfect. Um, so\non the on the slide where you had like\nthe performance and you guys were like\n80% and without unblocked it was 20%.\nYeah.\nUm, and now I see that well you are\nbasically hooking up like unblocked to\ncloud cut. So in a way is it a fair\ncomparison to say\nI will use vanilla cloud code with\naccess to the MCP and to the skills.\nYeah.\nAnd then I will use clo codes hooked\nwith unblocked with the same MCPS and\nthe same skills.\nYeah.\nAnd here you can do the performance\ncomparison. And here you still have a\nlot of alpha from I I guess whatever you\nare cooking inside unblocked. Is was it\nthe comparison that was done or was it\ndone without\nwas it done with a vanilla cloud code\nbut without context?\nNo, it was done with MCP servers like\nGitHub and Slack wired up.\nI see.\nYeah, cool.\nWe we basically got parody with all the\nMCP servers of every SAS vendor in one.\nIt was like vanilla clawed all MCPS and\nthe other one was clawed with unblocked\nonly\nand then do the task and\nsame context like the same context file\nsame same prompt\nand same access. Yeah. Yeah.\nIt's it's pretty fun. Yeah. Oh, thank\nyou.\nUm maybe two questions. So, one is uh I\nsee that like a lot of these like social\ngraphs are built with like the\ntraditional network uh kind of\ncalculation and statistical aspects of\nnetworks. Um, is this like the approach\nthat you began with and it already\nworked the best or uh did you like\nbecause most of memory systems you work\nmore on like filtering out like episodic\nmemory something else something else\nsomething else and this is like really\nscoring really nice scoring system\nuh that's first question is it like also\nwith the unblocked second question\num you mentioned that it works with\nteams uh Microsoft environment I wonder\nwhat the differences did you observe\nbetween building social graphs for\ndifferent environments because on GitHub\nI imagine it's very different than on\nSharePoint teams etc etc is it also like\nthese network stats based or is it\nsomething different\num so I mean our first implementation\nwas was incredibly naive right it was\njust using uh the numbers of PR\ncontributions and comparing that\ndirectly with uh the number of PRs\nreviewed by each person so just a simple\nlike numbers game um with that that\ndidn't produce accurate team clusters.\nSo then we we got on to um the\nalgorithms that you see here. Um Unblock\ndoes a little bit more than than this.\nSo this is kind of like a middle road.\nUm another strategy that Unblock uses is\num like experts by by vector clusters.\nSo when we ingest the source code and\nvectorize it um we understand like who\nthe the most contributors are for that\npiece of source code. So when we look up\nindividuals, we can see what they've\nbeen working on and what the um the\nclusters in proximity are and then\nrelate people based on their their\ncluster proximity. So that's more of\nlike an ML type approach. And then\nthere's a final layer which is um uh an\nsort of AI LLM heavy layer that does\ndistillations of uh a whole bunch of\ndifferent context elements, things that\npeople have worked on in the past,\nconversations that they've been having\nin Slack. Um and then when when you take\nall that and you weigh it against uh the\nlike procedurally generated graph, you\nget a much more accurate distillation\nthere. This one here, you'll notice like\nsome some people will get pulled into\nteam clusters that you know are you know\noperating across many different teams\nfor example and this won't account for\nthat\ndifference\nalgorithms different let's say that you\nI don't want to take out\nthis so no this algorithm is like purely\nSEM based. So the algorithms for you're\nyou're right like um Slack teams they're\nquite a bit different because you don't\nhave these review points.\nSo then it becomes you know who's the\nmost active in particular channels and\nthen you need a distillation or a\nsummary of what that channel is about\nand you need to vectorize that and then\nyou need to score it against the the\nmost frequent contributors. Um, but it's\nnot enough. You have to relate that back\nto the SCM data in order to figure out\nwho the real experts are. One one of the\nproblems that I I've personally\nexperienced in some organizations I've\nworked at is that you get like the noisy\njunior engineer, right? So, they're\nthey're very noisy. They love to talk,\nbut the signal to noise ratio is not\ngreat. And uh just because someone's not\nsaying a lot of things doesn't mean that\ntheir messages are not impactful.\nSo part of this game is about assessing\nthe impact of uh when people say certain\nthings, you know, how does that relate\nto the PRs that get spawned off as a\nconsequence? How many of those PRs get\nmerged? You know, that sort of thing.\nYeah.\nOh, is there not?\nThere should be. Okay, check that.\nWell, you should be able to open a pull\nrequest. You can't push to main.\nOkay.\nSo, if that if that's the situ we But I\nmean, we'll check.\nYeah, you should you should be able to\ncreate a branch.\nOh, uh, no. No. Can he can't fork the\nrepo either.\nOh. Um, yeah, forks might be disabled.\nThis will be open source like at the end\nof the week. Um, and your all your\ncontributions will be on it.\nWhat's really fun is using that social\ngraph tool later against your own repo\nand like showing your team.\nYeah.\nOh, sorry.\nOh, I'll\ncome.\nI like that. Unblock tried to answer you\nfor that question.\nOh,\nyou see that the Slack auto response?\nSorry.\nAre you in here?\nThat's a camera. Sorry.\nOh, it's okay. I was just\nOh, okay. Um, let me check to see. That\nshould not be the case.\nOkay.\nLet me know if you still need a GitHub\ninvite.\nJust check the members. I think there\nmight Yeah, there might be an issue\nhere. Just a second.\nOh, these were direct assignments. So, I\nthink we have to like pull people into\nthe whole project because they're not\nthey're not org assigned.\nOh, GitHub, I love you.\n09s of uptime.\nYeah, we'll fix this one here. Yeah,\nslam everybody in.\nCome on.\nYou got it up. I'm trying to because now\nwe just need to add people.\nGo to settings collaborators.\nunblocked. You all have right access. It\nis the name of the company.\nJust just validate that for us if you\nwould.\nYeah, please let me know.\nPerfect.\nAll right.\nAll right, we're getting real PRs now.\nThere we go. Nice. Nice.\nHell yeah.\nNow let's do fun things.\nOkay, nice. Looks good to me.\nWhat?\nI think we we have our our first\napproved PR.\nI'm send I'm just sending ridiculous\nchats to unblocked so you can see it try\nto answer questions in Slack as as PRs\ncome up.\nI'm going to see what it says about\nthis.\nAsk it to\nIt's like Oh, let me think about it.\nOh, did you ask it about the PR?\nYeah, but the PR I think you accepted.\nSo, we'll see what happens.\nYeah, I mean it did it did approve it.\nSo, you know, unblocked was\nunblocked like this looks good to me,\nman. blocked was down.\nOnly visible to you. Oh no.\nWhat was such a good answer though?\nNice PR. Good job on block. Great\nanswer.\nYep. Oh,\nyeah. Yeah. I'll put it back up. I'll\nput it back up. One sec.\nUh, where did it go? Actually, I lost\nthe\nover here.\nOh, yeah.\nhandle the sources or maybe something.\nDo you want the app?\nYeah, for sure. Yeah.\nOh, yeah. Yeah.\nYeah. Of course. We were focused on you\nbuilding, but Yeah.\nWhat am I supposed to do?\nOh, no. It's okay. I mean, let's go.\nOh. Oh,\nI sorry. So this this uh this thing that\nI showed before it it is the project\nthat exists in that repo.\nSo the\noh so the idea is like um think think\nabout features that you want to add or\nthings that you want to to fix or like\nnew components and then just hack away\nat it and submit a PR.\nSorry.\nYeah, my bad.\nUm do you want to open up like a\nterminal session and show the MCP?\nOh, sure. Yeah, because I'm like people\ncan obviously use it but they don't have\nall our source. Yeah.\nConsultant\nwanted to try to propose it to a client.\nI cannot show the\ncontext or maybe get an ide.\nWell, I mean like one thing that you\ncould do um if you're visiting clients\nis u you can ask them if they run the\ntool on their\nuh on their um repo and then it will\ngenerate this result for them so they\ncan see on their own project what the\nvalue is. Right.\nI think Peter I think he's just asking\nabout our product specifically not this.\nOh unblocks. You're asking about\nunblocked.\nMy bad man.\nWe're driving this way.\nSorry. Sorry, single track mind. Um,\nokay. So your your question is how can\nyou demonstrate the value of unblock to\ncustomers or\nsee the value?\nYeah,\nsorry.\nYeah,\nyou can make conflicts emerge in in your\napp, but um and then there is the\ncompliance layer which is very\ninteresting for corporate clients.\nI was thinking how this um is translated\nto a UX because you know\nmany people are known I understand it's\nmainly for coding. Yeah.\nAnd whether this is for technical people\nor maybe you know people overseeing some\nengineers or the engineer itself I mean\njust see how your platform works. But if\nit is is out of context I mean I it's\nit's okay. I\nno no that's that's totally fine. So\nthis this dashboard is kind of like the\num the sort of front-end customer\ninterface to the product. So you know\nyou come in here and you can ask any\nquestion about your codebase or your or\nyour organization and get an answer for\nit here. Um this is right now you know\nattached to sorry I lost my cursor. This\nis attached to um this test or that we\nhave but I could use it against\nunblocked and I could say like you know\num I have a little hot thing here that I\ncan show.\nOops.\nSo, the source mark engine is an\ninternal component that we use to track\nsource code changes through time,\nincluding like where\num you know, changes move between files\nand so on. Um, so as a demonstration,\nyou know, you can show off, I mean, you\ncan book your your customers into a demo\nwith us and we can demonstrate this or\nyou can wire it up to your own\norganization and demonstrate this flow\nto customers um, and try to find, you\nknow, use cases where data sources\nconflict and demonstrate that that the\nchallenge with context engines is that\nit's really hard to demonstrate the\nvalue to someone without actually wiring\nit up. So there there is a little bit of\noverhead there where people have to\nconnect it to all their integrations.\nNow the good thing is um unblocked has a\nfree enterprise trial period so people\ncan try out the product in its fullest\nform before um uh paying for it. Yeah.\nSo if some of that information is\nincorrect, you can just reply in the\nchatbot or flag it in the references.\nExactly. Yeah.\nSo you can just you can reply here or\nyou can say not helpful and explain why\nand then uh it will distill it for the\nnext the next round.\nSo it will adjust some weights or\nconfidence scores internally.\nUh well internally what it does is it\nconstructs task memory.\nSo um it looks for those kind of\nrepeated signals and uh it this is\nactually where the experts graph comes\nin. It's used a lot. Um the experts\ngraph provides like weight. So when an\nexpert comes in and says that's not\ncorrect, it's going to get some some\nmore weight and distill a memory for it.\nUm if uh if it's just a new engineer\nthat says that's not right, then that's\nnot really a trustworthy source yet. So\nuh you have to have a trustworthy source\nto to base that on. Does that make\nsense?\nYeah, it makes a lot of sense. It's like\nsocial network.\nExactly. somehow.\nYeah. Yeah.\nThanks.\nNo problem.\nCool.\nOh, under the hood. Um, well, when it's\npresented to the AI, it's presented as\nas files. Um but under the hood we store\nit in you know database tables and\nstuff. Um like the memories are are are\nconstituted from a bunch of different\nsources. So they're not just like flat\nfile based you know they'll be the whole\nmemory construct will be hydrated at\nruntime. So\nwill you just give your tools to your\ndatabase based on whatever criteria\nusers?\nYeah. Well for Yeah. So yes, um there\nare a bunch of tools for data retrieval.\nFor memory specifically, um you can't\nreally leave it up to the agent to do\nmemory hydration because that's kind of\nlike part of the seed context. In order\nto get the agent to go in the right\ndirection, you have to seed it with the\nappropriate data and experts context is\na good jump off point for the agent. So\nyeah.\nYep.\nUh is there any official benchmark that\nkind of track the type of value you try\nto bring like um yeah because I feel\nlike it's not exactly coding or it is\nbut yeah I'm curious if there's any uh\npublic things that you're tracking\nyourself against.\nSo we we we do have some internal\nbenchmarks. Um you're right it's a\nlittle bit squishy.\nUm so anthro have you have you heard\nBoris Churnney talk um at cloud code?\nIt's like the creator of cloud code.\nThe creator of cloud code. Yeah. So he\num did this interview where they were\ntalking about like how they measure\nsuccess uh for cloud code internally.\nThis may have changed because there's a\nlot of benchmarks now that they have\nlike they they have like the the\ntalk benchmark. You guys have probably\nseen that one. Um but it but what that\nreally distills down to is vibes. And so\nthe most important thing in uh systems\nlike this is to capture sentiment. And\nso if your sentiment is uh is trending\nupwards then um that's a good thing. Our\nour sentiment right now is uh on a scale\nof minus 100 to 100 somewhere around 60\nuh 60 score. So on a normalized scale\nthat's like 0.75\nto 0.8. So, so the vibe would be\ncaptured by something like maybe less\nback and forth on the PRs or maybe um I\ndon't know you having less back and\nforth with clothes to get your stuff\ndone.\nYeah. So the the vibes are like they're\nthey're people satisfied, right? So\nsatisfaction can come from a lot of\ndifferent sources and dissatisfaction\ncan come from a lot of different\nsources. So the way to think about that\nis that it encodes all of those things.\nUm, but you can capture specific metrics\nand we do how long things take and we're\nactually currently working really hard\nto bring the uh response times down\nbecause um\nuh you know even though agents are um\nhere's the interesting thing as we move\ntowards a more autonomous universe\nresponse times for MCP servers are\nactually less and less important. The\nmore important thing is that they get\nthe answer absolutely bang on.\nYeah.\nAnd the reason is because um the the\namount of time that a context engine\nspends collecting all that information\nand distilling it is a microcosm of what\nthe full task takes to implement and to\nand to traverse. So if you can spend a\nlittle bit more time and cut the\nimplementation down by like 60 70 80%\nthat's a huge win, right? And go ahead.\nSorry, very small followup. Actually,\nI'm curious. Do you have any uh rough uh\nnumbers on how much time does it spend\nretrieving context versus executing the\ntask to your point? Like is it 10% right\nnow, 90%, or is it I have no idea. I\nmean, I have my own experience.\nIt's like yeah um agent context\ncollection is probably close to that\nnumber. It's like 90%. Um the actual\ncode writing part is really really fast.\nIf if you can even just watch what an\nagent is doing. Um when it writes the\ncode that output tokens are by the way\nthe the thing that drags down um the the\nperformance. Everyone used to think it\nwas input tokens. We've run tons of\nexperiments with this. You can bring the\ninput token size up and you know time to\nfirst output token now is is pretty\npretty good. Like it's pretty highly\noptimized. The thing that really impacts\nperformance is output tokens. So um you\nhave to be like judicious with the way\nthat you collect and supply context back\nto the agent uh so that it remains tight\non its output loops as well.\nFor um for one benchmark that Peter\nmentioned in the talk we we gave an\nambitious task because obviously it's\nprompt dependent how much time you're\nadding and like with a context engine.\nUm but the ambitious task we gave was to\nimplement the new adaptive thinking mode\nin anthropics tool chain when they\nintroduced that which as mentioned it\nwent from a 25m minute wall clock time\nto with with unblock with a context\nengine. The other case without was 2 and\na half hours. It was 2 hours and 25\nminutes. But the main reason for that\nwas we gave it all the data. We ran the\nprompt and then its first output was\nlike totally wrong. So you had to the\nhuman had to loop again and be like no\nno no this this this and the next output\nwas wrong and the next output. So once\nyou do four loops you have like a two\nand a half hour wall clock time versus\nobviously the 25minut when it did not\nmeet that when there's no corrections\nrequired.\nUm so as mentioned it's think of it as a\nwaterfall. The more high quality correct\nlike high signal context you have up\nfront the better every single thing the\nagent's going to do until it says it's\ndone whether it got it right or not.\nYeah.\nIt's got it.\nYou also mentioned that uh the token\nusage on tool calls and like just\ninformation search really decreased. So\nI know that a lot of these tools that\nprovide uh or aggregators for tool use\nthey have insane like uh token usage. So\nmaybe have like some estimations on how\nlike let's say I need a slack\nconversation some summary from one\nconversation to another or like how\npeople interact there would be like 60k\ntokens on composio I wonder how many\ntokens it would be like using unblocked\nyeah\nlower we're still very vibes there like\nit's hard to get real data from other\ncustomer or people in the market um But\nthe again with that same I'm going to\nkeep talking to the same task as easy.\nThat one went from 21 million token\ntotal usage to 10 million token with the\ncontext engine. So a part of that though\nis because you didn't have to doom loop.\nSo when when the of course like that\nincreased a lot of the tokens expense\nlike so we did drop it by 50% on a large\ntask. Again obviously if you're like yo\ncenter a div you're not going to get a\nlot of gain. It's like probably in the\ntraining data. Um, but yeah, like any\nfeature uh fix like so a lot of like\nagain a lot of what people are putting\nthrough unblocked are what an engineer\nis doing every day. It's very rare that\nyou're doing a task that's like so I\ndon't know minor that like I mean then\nagain I've asked I've asked Claude to do\ngit push so I'm not the only one I bet I\nwas like you do it. It's like why did\nthat cost me 30 cents?\nI don't know.\nYeah, I put I did all the effort to put\nmy GBG keys in the right place so I'm\nlike cloud\nGo.\nAny more questions while you all ship?\nAny confusion? Anything I can unblock\nfor you? It's my purpose in life.\nSorry, you may have answered this\nquestion already, but um so you're are\nyou using knowledge b knowledgebased rag\non in unblocked or what exactly is the\ntech that you are surfacing?\nOh, so many things. Uh I can come talk\nto you at the side. I'll take my mic\noff. I'm just gonna answer that\nquestion.\nSure.\nThat was just\nthis\ntalented\nYeah.\nOh, it's it's real time basically. So,\num there I guess there's there's two\nparts to that question. one is like how\nmuch or how frequently unblocked updates\nthe data on the back end. Um so it's\nit's real time for many of the\nintegrations and then on a a cron job\nfor others because for those for those\nparticular integrations they don't have\nweb hooks basically.\nYeah. But the the dis so that means that\nrebuilding the graph data has to happen\non a on a very frequent basis.\nYeah.\nNo, it's it's incremental.\nSo our our like you know social graph\nbuilder algorithm has an incremental\ncomponent to it. So we don't have to\nrerun the whole thing. Um but also uh\nsocial graphs are less sensitive to\nfrequent changes in data because it's\nunlikely that you know a single change\nis going to make a huge impact on the\nexperts graph unless your organization\nis brand new. So for\nYeah.\nYes. Yeah. So, as an example, um we do\nbest practices distillation on a much\nlower cadence like basically uh week by\nweek because uh yeah, it just doesn't\nchange that much.\nYeah.\nUm, well, the Oh, yeah.\nRepeat your question. That's a good\nquestion. So, I want to make sure we get\nthat one down.\nOh,\nin in terms of customer privacy, data\nretention\num kind of\nyeah from from my point of view I'm\nthinking of like enterprise SAS or even\nlike on premise type deployments which\nI'm I'm not suggesting that you I'm just\nthinking of that customer kind of\nmodality.\nUm\nyeah, do you get do you get push back?\nDo you how do they feel about you\nholding data? It's another processor in\nthe loop.\nUm well so the the the privacy\ndiscussions happen at the organizational\nlevel. So it um uh we don't actually run\ninto a lot of friction. Um there are\ndefinitely environments like in\ngovernment and at banks that have uh\nsuper sensitive needs and so for those\nneeds we have an on-prem solution but\nit's definitely not the path that I\nwould recommend like staying cloud-based\nlike we we have very large enterprise\norganizations\nuh that are entirely cloud-based, like\nfully cloud-based. Um the you know the\nthe secret sauce is kind of like less\nencoded in source code now and more\nencoded in um uh the reasoning. So,\norganizations tend to be a little bit\nmore sensitive around things like Slack\ndata for instance, but uh the way that\nwe store uh data like we have a whole\nwhite paper about how we protect\ncustomer data um and it's never been a\nproblem.\nYeah.\nPardon me. Can you run on prem?\nYes, we we do have an on-prem solution,\nbut as I say, like it's it's not the\nrecommended approach, but for sensitive\nenvironments, for sure. Yeah.\nOh, why it's not recommended? Um, well,\nthe cloud-based integrations um,\nyou know, get updated more frequently\nand so there's software patches. It's a\nlittle bit harder to maintain within an\norganization. Uh there's there's one\ncustomer it's a bank um where\nadministering\nuh the platform becomes quite difficult\nbecause they have network isolation and\nso like now one of us has to you know\nsit within that network and administer\nthe platform or we have to train uh\nindividuals within the company to\nadminister the platform. So it's just\nit's more of a a maintenance and um\nhandholding exercise.\nBut yeah.\nYeah, exactly. That's exactly right.\nYeah. Thank you. Thanks for coming.\nvery much.",
  "transcript_chars": 72828,
  "ingested_at": "2026-05-12T00:33:04.588441+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 12085,
    "like_count": 246,
    "channel_id": "UCLKPca3kwwd-B59HNr-_lvA",
    "categories": [
      "Science & Technology"
    ],
    "tags": [
      "ai",
      "ai engineer",
      "ai engineering",
      "software development",
      "tech",
      "startups",
      "software architecture",
      "machine learning"
    ]
  }
}