{
  "video_id": "fCHe_fOqlYA",
  "channel_slug": "ibmtechnology",
  "channel_handle": "ibmtechnology",
  "title": "Building AI Agent Systems and Scaling Challenges in Agentic AI",
  "duration_seconds": 785.0,
  "url": "https://www.youtube.com/watch?v=fCHe_fOqlYA",
  "upload_date": "",
  "transcript": "Agents are easy to demo and surprisingly hard to scale.\nIt's now easier than ever to build a working demoable agent that can complete meaningful tasks end to end,\nwhich naturally leads to the next obvious question, why not just scale it?\nMore steps, more tasks, less supervision.\nBut before we push that further, we need to take a closer look at what actually changes when you start scaling agentic systems.\nAnd how scaling agents is not quite as simple as it seems.\nIn traditional software systems, scaling is a well-understood problem.\nAs demand grows with more users, more requests, more data, you add.\nThis can happen in many ways, such as horizontally by adding machines or\ncontainers, or vertically by increasing CPU, memory, and storage.\nBut fundamentally, more users means more infrastructure, which results in the same behavior.\nAgentic systems break this pattern.\nYes, they require infrastructure scaling, but when people talk about scaling agents, they're often mixing two different ideas,\ntraditional scaling of handling more requests and expanding capabilities to enable the system to do more.\nAnd it's the second one that changes everything.\nThe scaling we're going to talk about here is making these AI systems work\nreliably across wider scopes and more complex tasks.\nTo understand why this matters, let's look at how agents actually operate.\nMost agents follow a simple loop.\nThey plan the tasks into steps, execute by using tools to act, remember and store relevant context to memory.\nAnd reflect on any actions to evaluate what worked and what didn't.\nFor narrowly-scoped tasks, this works remarkably well.\nThe problem is bounded.\nThe system makes a few decisions and completes the task and stops.\nWith this success, we naturally decide to scale it.\nMaybe you want to expand into another domain or to a new suite of features users have been requesting.\nAt first glance, this seems like a straightforward extension.\nJust give the agent more tools, more knowledge, and broader responsibilities.\nThat's where we hit the first large challenge.\nWhile the agent loop doesn't change, the cost of each execution does.\nFor a narrowly-scoped task, the agent might plan a few steps, make some tool calls, and complete in a handful of seconds.\nToken usage is small, and latency is not very noticeable.\nBut as you scale, planning takes longer.\nExecution becomes more demanding as the agent has to decide between more possible tools and actions.\nMemory grows, increasing the context passed into every step and requiring more effort to fight through the noise.\nReflection also becomes more expensive and less reliable as more context begins to dilute useful signals.\nWhat used to be quick, cheap interactions no longer scale cleanly.\nLatency and costs scale non-linearly, as each decision requires more context,\nmore reasoning, and more careful selection between actions.\nIt's not just that we have added more features, we've multiplied the complexity\nof decisions the agent has to make to complete even simple tasks.\nThe immediate consequence is simple.\nScaling agentic systems increases the cost per decision, and ultimately, the cost per successful.\noutcome.\nNow let's assume you're willing to pay these costs.\nYou are still not in the clear.\nSomething more subtle and more dangerous happens next.\nLet's illustrate this with a simple example of a travel agent.\nYou say, book me a trip to Washington.\nThe agent gets started by building its plan for your upcoming trip to Washington, DC.\nIt executes tools to find flights, book hotels.\nAnd organize transportation.\nAnd all of these execute successfully.\nA few minutes later, we have this great trip fully planned and ready to go.\nBut the initial assumption was wrong.\nThe model misinterpreted the request Washington as Washington, DC when\nyou actually met Washington State nearly 3,000 miles away.\nAnd now that assumption drives the plan, influences the execution, and gets written into memory.\nThis tiny error...\npoisoned the entire interaction, not just wasting money, but wasting your time.\nThis is the key shift.\nFailures are not isolated, they propagate.\nThe system didn't just make a silly little mistake, it spread that mistake across time.\nThis is dangerous because as agents scale, they make more decisions under uncertainty, not less.\nAnd because the system is operating autonomously, there may be no natural\ncheckpoint where a user could come in and easily correct it.\nSo let's take a step back.\nAs we've discussed, scaling agents is not something we can treat as simple extension.\nIt requires architectural changes.\nA single agent doesn't scale well because it owns everything, every decision, all memory.\nAs that scope grows, the context becomes noisy, state becomes hard to\nmanage, failures cascade easily and per task cost continues to rise.\nThis is not a model limitation, but rather a consequence of how responsibility is distributed.\nThe core issue here is ownership.\nWhen a single agent is responsible for everything, every decision becomes\nmore expensive, more complex and more fragile.\nThere are no clear boundaries or separation of concerns.\nThe limiting factor is less the capability of the model.\nAnd more how much each agent is responsible for.\nThat's what determines whether the system scales.\nIn other words, it's a systems design problem, not a model capability problem.\nImagine a company where every single decision, let's say engineering,\nmarketing, hiring, support, all has to go through one person.\nAs the company grows, even simple decisions take longer and longer because the person has to understand more context,\nconsider more factors, and switch between specialized domains.\nAgents are the same way.\nWhen responsibility is centralized, the bottleneck isn't the effort but the growing cost of making each decision.\nSo what do we actually do about this?\nMoving away from a single agent, We decompose the system.\nInto multiple components with bounded and distributed responsibility.\nEach component operates with less context, makes fewer decisions, and has a narrower scope.\nTogether, they form a system where individual decisions are cheaper, faster, and easier to reason about,\nwhile complexity and failures are contained rather than compounded.\nThis is where multi-agent systems begin, as a consequence of scaling correctly.\nBy distributing responsibility and decomposing components, we begin to\nregain control over decision size, cost, latency, and failure propagation.\nOnce we move into the multi-agent design space,\nwe introduce a central challenge of managing how agents coordinate, share work, and manage dependencies.\nAs systems grow and evolve, you must decide how to scale their capabilities.\nOne path is horizontal, introducing new agents to take on distinct responsibilities.\nThis makes new capability easier to access and reuse,\nbut as the system grows, coordination becomes the limiting factor and communication overhead increases quickly.\nThe other path is vertical, increasing the capability of individual agents through additional tools or subagents.\nThis reduces the need for coordination but can increase latency and complexity concentrated in each agent.\nRealistically, this shows up as a question of capability placement.\nShould a new capability live as its own agent or be embedded within an existing one?\nLet's consider a research assistant agentic system.\nWe have a central coordinator agent and sub-agents for retrieving documents,\nrefining search queries, and finally for synthesizing the results.\nIf we want to introduce fact checking, one option is a dedicated agent that evaluates outputs across the system.\nThis works well because fact checking is a distinct reusable capability with its own logic and policies.\nSeparating it keeps responsibilities clear, but requires an additional coordination step.\nIn contrast, consider adding the capability rank and filter retrieved results to get more relevant documents.\nThis is best embedded within the existing retrieval agent because the\ncapability is tightly coupled to the existing agent's retrieval process and depends on shared context across steps.\nSplitting it into a separate agent would introduce unnecessary coordination and kind of fragment the decision process.\nSo there's a trade-off.\nSystems that scale more horizontally must invest more effort at the coordination layer.\nSystems that scale vertically must manage growing complexity and cost of these individual agents.\nIn both cases, complexity from scaling to new capabilities is shifted.\nThe decision really comes down to how expensive coordination will be versus\nhow much complexity an agent can reasonably absorb.\nA useful rule of thumb is to split capabilities when they are reusable and independent,\nand embed them when they're tightly coupled and context-dependent.\nIn practice, agentic systems that will actually scale are those that balance these forces\nand deliberately choose where the complexity accumulates in coordination,\nin individual agents, or in the structure that connects them.\nAt every stage, scaling introduces a new constraint.\nCost rises, latency increases, failures propagate, and coordination becomes harder.\nScaling AI agents doesn't just amplify capability, it amplifies everything in the system at once.\nThe teams that succeed are those who understand these challenges and\nconstraints and make deliberate architectural decisions about what is allowed to scale and what is kept bounded.\nAll of this might sound like a lot of problems.\nBut it's actually where the opportunity lies because once you understand how decisions flow through your system,\nyou can shape how those decisions behave at scale.\nThe teams that win won't be those with the most capable agents.\nThey'll be the ones that design systems where decisions are bounded, costs are\nintentional, and intelligence compounds instead of collapsing.\nThe goal in scaling agentic AI is to design systems that can survive.\nAnd benefit from their own successes.",
  "transcript_chars": 9969,
  "ingested_at": "2026-06-17T04:32:31.776287+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": null,
    "like_count": null,
    "channel_id": null,
    "categories": null,
    "tags": null
  }
}