{
  "video_id": "pqlWNihgdjI",
  "channel_slug": "aidotengineer",
  "channel_handle": "aidotengineer",
  "title": "From AI-Assisted to AI-Native: Building a Frontier Development Team — Clare Liguori, AWS",
  "duration_seconds": 1257.0,
  "url": "https://www.youtube.com/watch?v=pqlWNihgdjI",
  "upload_date": "",
  "transcript": "[music]\n>> My name is Claire La Gory and I'm a\nsenior principal engineer at AWS. I\nmostly work on Kuro, our agent encoding\nassistant, but today I want to talk\nabout some of the practices we've been\nseeing inside of Amazon and Amazon teams\nwhere we've been seeing really exciting\nresults of productivity increases that\nare step function improvements since\nwhat what we've been seeing with AI so\nfar.\nSo, I've been working on agentic AI for\nover 3 years now and I've kind of seen\nthe evolution that's happened in our\nindustry when it comes to coding\nassistance with AI. First, we had this\ninline code completion helping us to\nwrite the next line, maybe the next\nfunction. We moved on to chat, asking\nquestions about our code. Everybody\nstarted doing vibe coding sometime last\nyear, but now we're starting to see kind\nof an early adopter phase of what we've\nbeen calling frontier development.\nAnd completely anecdotally, based on my\nown experience, I've really only felt\nmaybe 10 to 20% more productive with all\nof these phases that have come before.\nBut now inside of Amazon, we've been\nrunning pilots with different teams\nacross the company and we've been seeing\na median of 4.5x productivity\nimprovement and sometimes more than 10x.\nSo, something has really changed here\nnow that we're seeing these step\nfunction improvements in productivity.\nAnd I like to\ndefine what we've been calling frontier\ndevelopers inside of Amazon by three\nbehaviors that I've been seeing. One is\nhands-off coding. Frontier developers\nwrite maybe 1 to 2% of the code that\nthey produce. The rest is agents.\nThe second is that they interact with\ntheir agents infrequently. They'll aim\nto get their coding assistant to run for\nup to hours at a time without their\nintervention.\nAnd third is that they minimize idle\ntime.\nThese frontier developers tend to run\nmultiple agents in parallel churning\nthrough a backlog of tasks.\nThe first time that I saw a frontier\ndeveloper team was the Bedrock Mantle\nteam. Bedrock is our model hosting\nservice.\nHosts LLMs like Claude and GPT. And\nsometime last year we knew or I say we\nbut the Bedrock team\nknew that they were going to need to\nbuild a new inference data plane. But\nthey had estimated it at 30 people over\n18 months. This is a big big service and\nit was going to take time to build the\nnew one, migrate customers over, migrate\nmodels over. They decided to take a step\nback. They took six people and they\nbuilt [snorts] it in 76 days with Kiro.\nSo this was a huge achievement. This was\nthe first time we've we'd seen anything\nof the kind inside of Amazon. So this\nwas truly the pathfinder team that\nproved that it was possible to get up to\n20X improvement. Now they looked at\ncommits and I'll talk about a couple of\nother ways that we are uh measuring\nproductivity improvements.\nBut there was one problem with this\nstory which was that yes, it was built\nwith six people. It was built with some\nof the top engineers literally in the\ncompany including two distinguished\nengineers. So this was not just any team\nof six people. These were experts in\ndistributed systems, experts at LLMs and\ntheir architecture.\nSo this the story was amazing and it\nkind of spread like wildfire across\nAmazon, but it was also very\nunachievable for a lot of teams. There\nwere a lot of questions about can this\nactually be reproduced on another team?\nSo, another experiment that I want to\ntalk about is an experimental sprint\nthat was done in the Prime Video\norganization.\nThey took a 10-day sprint and they did\nan experiment where they put, again, six\nengineers in a room and they let them go\nwild with Kiro.\nUh they brought down the project\ndelivery time estimate from what was\ngoing to be 90 weeks down to 24 based on\nall of the progress they had made in\nthis 10-day sprint. And they they looked\nat their commit history and they looked\nat what did they used to do prior to\nthis 10-day sprint and how many commits\ndid they produce just in this 10 days.\nAnd so, this sprint really proved that\nwe can achieve, again, at least\nsomething close to what the Bedrock\nMantle team had uh had achieved with a\ndifferent set of engineers.\nBut again, there was a challenge with\nthis story, which was it was six\nengineers in a room, but they had no\non-call duties, limited meetings, very\nfew distractions, which we all know are\nregular in the lives of an engineer.\nAnd the senior engineer on the team had\nspent the previous 3 weeks creating very\ndetailed, small, well-scoped tasks with\ndetailed requirements for these\nsix [clears throat] engineers to just go\nchurn on for those 2 weeks.\nSo, this was again not necessarily real\nlife. This was a structured sprint, uh a\na point in time that they were able to\nachieve this, but again, the question is\nis this achievable on real teams on\nday-to-day\nwork?\nSo, Amazon stores which encompasses\namazon.com, all of our retail websites,\nas well as our physical stores,\ndid a more structured pilot. They\nwatched 50 teams that were totally\nnormal normal distribution of um early\ncareer folks, mid-career, senior\nengineers, and that worked on existing\nsystems. Nothing green field like the\nmantle team got to build from the ground\nup, but existing systems with existing\ncode bases.\nAnd they they watched them for the\nbetter part of last year, and they found\nsomething super interesting.\nThey found that there was a big\ndifference in the productivity gains\nthat they saw between half of the teams\nand the other half.\nAnd in this case, they used a\nproductivity metric of deployment\nvelocity to production. So, not just\ncommits, how many commits are they\nproducing, but how quickly are we\ngetting changes out to customers? How\nhow quickly are we able to ship things?\nAnd they saw that for half of the teams,\nthey achieved less than 3x increase.\nAnd what they found that was the\ndifference between seeing less than 3x\nproductivity increase, these teams that\nsaw a median of 4.5x, and and in some\ncases more than 10,\nwas how they used the tools. 90% of\nthese teams used Kiro, among other\ninternal tools that we have, and what\nthey found was it wasn't about the\ntools, it was about the way that they\nworked.\nThe teams that achieved step function\nimprovements\nintentionally changed the way that they\nworked, and the other simply kind of\nsprinkled Kiro and some of the other\ntools that we have on top of their\nexisting way of working. And for me at\nleast, this was the big aha moment. That\nwhy I hadn't been feeling potentially\nthe massive gains that productive that\nin in productivity that AI has promised.\nIt's about changing the way that we\nwork.\nSo, across this pilot, they went and\ninterviewed uh the teams that were\ninvolved in the pilot as well as some of\nthese other teams on the Bedrock mantel\nteam, on uh Prime Video, and they found\nfive habits. And and I use the word\nhabits very specifically because again,\nit's not about that one sprint. It's\nabout doing this day-to-day. And it And\nwhat they found when they interviewed\nwith these teams was that it really was\nhabits that they had to build\nday-to-day. When we change our way of\nworking, it's it's hard to build these\nhabits. It takes time to build these\nhabits.\nSo, let's go through each of these one\nby one.\nHabit number one is investing in agent\ncontext. We have a lot of stuff in our\nhead. We tend to transfer all of that\nstuff in our head to other people\nthrough Slack conversations, through\nonboarding, mentors, things like that,\nthrough code reviews, through\nstand-ups and sprint planning, and they\nhad to write all of that down. And the\nhabit that they built was every time the\nagent makes a mistake or does something\nnot the way that you would have done it,\nwhat am I missing in my skills files?\nWhat am I missing in my steering files\nthat the agent needed?\nBut then, as we know, across last year,\nwe saw leaps and bounds in models'\nabilities and their behaviors.\nUh the Sonnet 3.7 in the middle of last\nyear had a lot of quirks that we had to\nput a lot of do nots in our uh in our\nsteering files, and now we don't have to\ndo that as much with Opus 4.5 as of last\nNovember, and then we've had 6 months\nmore than 6 months of improvement since\nthen\nuh with all of the new versions of\nmodels that have come out since then.\nAnd so, the question, the new habit,\nagain, is do I still need this in my\nsteering files or is this just bloating\ncontext?\nThe second one is slowing down to speed\nup. In almost every team that was\ninterviewed, they reported that their\nproductivity actually went down as they\nintentionally adopted a new way of\nworking.\nThat's counterintuitive, right? You have\nto do intentional engineering work\nbefore you're going to see that hockey\nstick curve in productivity improvement.\nBecause we have to do real work in our\ncode base first for agents to be\nsuccessful there, especially in\nbrownfield existing code bases. So they\nhad to build that agent context up. They\nhad to improve existing tools error\nmessages so that the model knew what was\ngoing on when it failed. They built new\ntools, new MCP servers for helping that\nmodel to actually get done what it\nneeded to get done. A lot of teams ended\nup restructuring their code base so that\nagents could actually navigate it more\neasily. And I've even seen drastic\nchanges like changing the programming\nlanguage of the code base.\nUm often I've seen teams struggle with\nPython, with JavaScript because they're\nuntyped languages. It's hard to test.\nThere's no compiler errors. So the model\nkind of guesses and give it gives it\nback to you. And so I've seen teams\nmoving to TypeScript. Um Rust has become\nvery popular inside of Amazon. The\ncompiler gives great error messages.\nUm you don't have to do that, but I've\nseen a lot of teams making those\nintentional changes for the productivity\ngains that they're able to see.\nThe third one is feeding agents, not\nbabysitting agents. And for me this was\none of those aha moments of why we're\nseeing this step function improvement in\nproductivity.\nIf you are vibe coding, if you are\nhaving a back-and-forth conversation\nwith your agent all day long, of course\nyou're not going to see four to five x\nproductivity improvements because you\nare in the loop the entire time. You're\nprobably sitting there for 30 seconds to\na minute waiting for it to generate code\nand come back to you with with the code\nto review.\nIf you're sitting there waiting for it,\nthen you can't go off and do other\nstuff. It's really difficult to run\nagents in parallel. It's very difficult\nto get to to clone yourself into\nmultiple agents. And so if your\nconversations look a bit like this on\nthe left, then you're babysitting that\nagent. As opposed to the right side\nwhere you're feeding it what it needs to\ndo and how it can self-validate. And\nthat's really the key so that agents can\nself-correct and only come back to you\nwhen it meets a certain quality bar,\nwhen it when it actually runs and\ncompiles and passes tests, when it's\ntestable, when it it actually has high\ncoverage. And of course the next level\nis put all of this content into your\nsteering file so it does it every time\nwithout you having to prompt it.\nThe fourth habit is to make intent\nexplicit. At Amazon we practice a lot of\nbehavior-driven development. We've built\nthat into the Q product and so it's very\nnatural for Amazon engineers to adopt it\nin Q. Um what what I've typically seen\nwith live coding as opposed to frontier\nengineering is giving a very high-level\nprompt, letting the agent generate a ton\nof code, and then having a\nback-and-forth conversation saying, \"Oh,\nthat's not really what I meant. That you\nhaven't you haven't exactly gotten the\nthe requirements right. No, I didn't\nactually want to build it that way.\nHere's a technical design.\" And it is\nless I find less productive to iterate\nwith the agent on code when the intent\nitself was incorrect. So often will have\nwill see Amazon engineers go through\nthis process for for ambiguous complex\nfeatures of writing the specification.\nAnd in Kiro, of course, you don't have\nto write this whole specification. You\ncan have the model generate it, but it's\na lot easier to to iterate with the\nmodel in kind of a back and forth\nconversation about a document than it is\nabout code that's code changes that are\nspread across a code base.\nThe fifth one is shift testing left. One\nof the keys here is to give the agent\nthat fast feedback loop.\nBecause that's what lets it go off for\nhours at a time and self-correct. The\nagent is going to make mistakes and\nthat's fine. But if you give it the\nright signals, it can self-correct and\nit can spend a while doing that.\nSo, I've seen teams adding linters,\nadding unit tests, integration tests,\nperformance tests, security tests. These\nare all things we all know we should\nhave been doing all along. This is good\nengineering hygiene and practices. But\nnow the ROI is, I think, finally high\nenough for actually us to actually\ninvest in it. Um one thing that I've\nbeen seeing a lot of teams do is mock\nout services. Often with integration\ntests, we would test kind of end-to-end\nan entire system including live\nservices. But we've been investing a lot\nin in mock services that run entirely\nlocally with deterministic responses\nbecause it lets the agent do everything\nlocally. Um doing everything on your\nlaptop without having to spin up a bunch\nof other services and and connect to\ncloud services makes everything a lot\nfaster because the the more that your\nagent can get fast feedback means the\nmore loops that it can can do and the\nmore productive your own agent can be.\nSo, across all of these, these are some\nof the habits we've seen, but of course\nI would be remiss if I would tell you if\nyou adopt all of these habits, you will\nachieve nirvana. You will be the most\nproductive engineering organization the\nworld has ever seen. Things are still\nhard. We are still very much in an early\nadopter phase and teams are still\nfiguring it out.\nSo, one thing that we've been seeing\nacross our teams just organizationally\nis the risk of burnout. I did not coin\nthis term. I forget who did at what\nconference, but flow mat is real. We've\nbeen seeing engineers staying up late\nlate at night\ntrying to get that perfect prompt that's\ngoing to make their agent run for hours\novernight so that they wake up in the\nmorning with a code change ready.\nThe cognitive load increases as you run\nthese multiple agents in parallel.\nYou're constantly shifting between\nterminal tabs.\nAnd then we do see that reviewing AI\noutput is often harder for some than\nthan actually writing it, especially\nearly in career.\nSenior engineers have have already spent\na large portion of their career\nreviewing others code.\nBut early career engineers don't have\nthat muscle yet and so reviewing it can\ncan feel like a lot more cognitive load\nthan they're used to and actually\nwriting it.\nThe other one is organizational change.\nSo, it's already hard to change the way\nwe work as engineers. The way that we\nspend our entire day completely changes\nwhen we're frontier engineers, but also\norganizations have to change to enable\nfrontier engineering teams.\nOne that I've seen very commonly is\naccepting slowing down to speed up.\nAnd I've been guilty of this myself. My\nmy fellow leaders have been guilty of of\nthis of saying, \"Well, you have the AI\ntools now and the models are so amazing\nnow. Why are you not going faster?\nUm and that's because you have to take\nthose two months to invest in your code\nbase, to figure out the best practices\nfor your team, to make hard habit\nchanges on your team.\nUm and and if you're constantly\nexpecting\nshipping features every month because\nnow we have these amazing models and\nwe're seeing um all of these these\ncompanies on X saying how they're\nshipping 20 PRs a day, um we have to\nslow down to speed up.\nThe second one is actually going too\nbroad in the organization too fast. I\nthink that if we had um expected all\nteams in massive organizations to be\nfrontier teams immediately, we would not\nhave had the learnings that we had from\nthe Pathfinder, from the from the sprint\nexperiment, from the pilot uh teams\nwithin Amazon. And now the challenge for\nus is how do we scale it out? And that's\nwhat 2026 is about for Amazon is how do\nwe scale this out to more and more\nteams, to the next uh 2,000 teams\ninstead of uh 50 teams.\nUm and so I think that when you roll it\nout too quickly, you have a lot of teams\nwho don't know what they're doing. You\nhaven't had time to find the best\npractices for your own organizations,\nthe the context that your organization\nneeds.\nAnd the last one is that you're going to\nfind new bottlenecks.\nPreviously, code writing code manually\nwas the bottleneck. Um I find that\nwithin Amazon, we've found um the speed\nof decision-making becomes a new\nbottleneck. Um the more that you spend\nreviewing the decision to actually build\na new product, the slower it is to build\nthe product now because the code only\ntakes 1 to two months to write.\n>> [snorts]\n>> Um all of the review processes\nassociated with the launch of a product\nbecome the bottleneck. When it used to\ntake 9 to 12 months to build a new\nproduct, it didn't matter so much in the\nin the overall wash of things if it took\ntwo months to make the decision to build\nthe product and then two months to\napprove the launch. But now those are\nthe bottlenecks. Those are the long\npole. And so you find all of these all\nof these things that slow you down.\nOften I find that frontier engineering\nteams spend more time making decisions\nthan they do writing code. And so the\nmore that you can make fast decisions,\nespecially ones that are easy to be\nreversed, the better.\nSo my one big takeaway for for everyone\nhere is that\nfrontier engineering is about\nintentionally changing the way that you\nwork. And that is difficult. That takes\ntime. It is forming new habits and a new\nway of working.\nAnd that goes across any engineering\nteam as well as your organization. Um so\nI encourage you to think about\num how you're interacting with AI tools\nand how that can change to free yourself\nup from being in the loop.\nUm thanks. I'm going to I'll hang out uh\na little bit if anyone has questions in\nthe back. Um but thanks for the time\ntoday.\n>> [music]",
  "transcript_chars": 18148,
  "ingested_at": "2026-09-03T10:30:44.099230+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": null,
    "like_count": null,
    "channel_id": null,
    "categories": null,
    "tags": null
  }
}