{
  "video_id": "Z6sqEbg2ETo",
  "channel_slug": "huggingface",
  "channel_handle": "huggingface",
  "title": "From AI Agents to Faster Kernels: Ben Burtenshaw & Felix LeClair (AI Plumbers #2)",
  "duration_seconds": 613.0,
  "url": "https://www.youtube.com/watch?v=Z6sqEbg2ETo",
  "upload_date": "",
  "transcript": "Hi and welcome to Fostum Fringe for AI Plumbers, edition number two.\nI'm Felix from Aneko and I'm sitting here with...\nBen from Hugging Face.\nBen just did a really awesome talk earlier today on kernelize and that like really cool project\nbut I think one of the cool things we want to do is engineers are people right so what kind of got\nyou been into writing high performance kernels okay yeah interesting question I don't write\nhigh performance kernels I am a user of high performance kernels so I train machine learning\nmodels I'm mainly in a post-training setting but most of all I work in the advocacy team at\nSo I'm not a daily trainer or builder of models.\nI use them on the side for specific use cases for specific domains.\nAnd in that, I'm a huge beneficiary of optimized kernels like Flash Attention 3 and 4, etc.\nAnd I mainly use those on single instances.\nI don't have a kind of mega cluster that I use.\nI'm usually using them to kind of reduce costs or save me time.\nand up until recently they've taken a number of hours to install from source so Hugging Face has\nreleased the kernels hub a library a client library and a building library to go with that\nand with that we can basically get those kernels in a couple of seconds on most instances and we\ncan save ourselves a lot of time so yeah I'm a beneficiary of that work basically that's really\ncool so then in terms of like using the models themselves how are you are are there any ideas\naround how do I use models to help me do the kernel work?\nOr is this still like a very human process\nin terms of like supplying those kernels to the community?\nSo that's an interesting question.\nJust last week, actually, we wrote a blog post.\nWe collaborated with some engineers from the diffusers team,\nfrom the kernels team, and from the agentic team.\nAnd we got Claude code to generate and write kernels.\nTo be honest, when we first did this,\nClaude was extremely bad.\nEven Opus 4.5 really struggled at this use case.\nIt took us a number of hours of back and forth to get Opus to do this And so we used a skill which is a new format that most agentic libraries are working with now\nSo you define like a skills.md,\nyou say this is how you write kernels,\nyou define references files with all the best practices,\nand then you also give like example scripts\nof how you do it.\nAnd Claude was with that was able to get through it\nand it could write kernels.\nWe wrote a kernel for diffusers and H100,\nwhich was really nice.\nI think it was about 50% faster than the baseline.\nBut what was really cool is that we also generated a set of evals for it,\na test set.\nSo we measured and benchmarked the kernel.\nSo we said, okay, Claude says he did it, but they said they did it.\nHow much faster is it?\nIs it actually faster?\nDoes it actually work?\nSo, okay, then we had this base setup.\nAnd then with that skill, we took it over to other models.\nSo this was this kind of like Robin Hood moment,\nwhere we took it over to like first of all to gpc 5.2 to benchmark that and what we found was like\nthe skill doesn't just translate like so you can't just say okay great claude did it now they all do\nit actually we saw that some models didn't really even get that much better so gpc 5.2 didn't\nbenefit so much from the skill as claude did but open weight models like kimmy 2.5 got way better\nand they also used a lot less tokens.\nSo in the baseline setting, they were doing it,\nbut they were really consuming tokens in order to do it, right?\nThey're on this kind of Ralph loop.\nBut when we gave it the skill, it was like,\nokay, it's just going straight to it and it's doing it.\nSo yeah, that's a bit of a tangent into the agentic world.\nThat's super cool.\nBeing able to like bring that,\nbecause the whole point of Hugging Face I see is like,\nhow do I go and expose so much more of how the models work?\nPresent so many different models.\nWhat's the infrastructure behind it?\nHow do I serve that?\nHow do I empower the people working on all aspects of the flow?\nAnd integrating agents is like the big thing right now.\nAnd on the performance side, it's fascinating because what happened this week was the big\nanthropic performance challenge, right?\nCan you beat this many cycles?\nCan you beat Claude?\nAnd we've seen a lot of people beat Claude.\nBut now we're also at the point where as a performance, my back is performance engineering.\nLike, how do I go and use these kernels and generate these kernels?\nIt's like, wait, I don't have to worry about this stuff.\nI can let Claude or KimiK2.5, incredible model,\nin the open source community, I can use 5.2 and expose that.\nSo then the question for me becomes,\nhow does the kernelized community, as part of Hugging Face,\nthen go and look at which kernels it makes sense to bring in And how do you approach that from a developer empowerment standpoint of we to have a lot of kernels how do i empower the new developers who haven had that start in doing a low kernel design to still be contributing to the community yeah so that\nreally the most important question for us so if you look at optimized kernels and you see\nnew model releases and kind of new hardware releases typically what you'll see is like a\nreally nice pairing at the top of that. The latest models are paired to the latest hardware,\nand you'll see quite nice optimized kernels that are really well integrated. But generally,\nas you go down the spectrum of hardware into the older and cheaper categories, you don't see the\nsame support. And it's actually, that's the area where most of the community is sitting, most of us\nare sitting, right? People that just want to run a model, just want to run it locally on hardware\nthat we can actually afford, or these kind of things, old hardware that we don't want to\nnecessarily replace. And so inside the Kernels community, you'll see a lot of support for NVIDIA,\nAMD, Intel hardware as it comes out, growing and growing, which is great for users of that hardware.\nBut building off of that, we can see that the Kernels community goes into older AMD hardware,\nor older NVIDIA hardware, getting things to run on T4s and things like this. So it's like, okay,\nnow we've got it on Colab for free. And that's a huge unlock for people, right?\nis Colab is like the big unlock in the community because it's like now I can do whatever it is for\nfree just on the free tier it's really cool that's awesome that's really really cool and then from\nthere that kind of begs the question as we see the models getting better at generating these kernels\nand people working on these kernels and we see this sort of give and take and working together\ndo you think that we're going to be able to generate more kernels for that older hardware\nto help those users that do have, are limited or only have access to that older, cheaper hardware?\nYes.\nHow does that change over time?\nI think, in essence, the biggest part of the problem is probably not the agents being able to do it,\nbut more the infrastructure around the community.\nSo the Kernels Hub, having hermetically sealed, reproducible kernels,\nbecause in many ways it's probably possible now to get an agent to write kernels based off flash\nattention 3 in that similar format but it doesn't necessarily mean that they can be shared widely\nacross the community right it's the the kernels libraries contribution that unlocks the community\nto get access to that so i would say um yeah the agents definitely kind of supercharge it but without the kernels hub there really wouldn be any usage layer for those kernels that the agents generate\nThat would be my expectation.\nOkay, and then you also have a huge diversity of hardware you're supporting, right, as you mentioned.\nSo with that, is there an effort to make sure, like,\nyeah, I can't reuse this exact optimized kernel,\nbut this quantization of this model is using similar sub-quantization techniques\nto fit on the specific hardware,\nand I can take those learnings and actually expose those learnings to the user.\nBecause to your point, a lot of these techniques,\nyou can beg, borrow, steal from different models,\ndifferent optimizations of how do I go and translate this\nand make that information available to normal everyday developers.\nYeah, so the kernel's hub is extremely educational\nbecause of the fact that it's declarative and standardized.\nSo you can go to a kernel on the hub and you can go to the,\nfirstly to the readme and the readme is generated\nand it will have a standardized set of build options that it's built for.\nThen you can go to the build.toml and you can look through the build.toml and say,\nokay, this works for this hardware in this kind of situation.\nAnd then from there, you can go to transformers and you can look at the modular declaration of a model.\nSo like GPT OSS.\nAnd you can say, okay, this is where that kernel was used for that model.\nAnd then from there, you can go to the modeling.py and you can look at how that ends up as modeling code.\nAnd you can see other models that are potentially used within this model and how they use kernels.\nSo we kind of like unlock the single source of truth from this one place, right from the kernel.\nSo it's tethered together all the way back through the ecosystem, basically.\nSpeaking of that ecosystem, how does someone get involved?\nSo if you want to get involved with Hugging Face, the best way really is just to start pushing models to the hub or pulling them and just trying to use them.\nIf you want to contribute to a library like Kernels, the best thing to do is to go to the GitHub repo and the Hugging Face org.\nSo to join and follow the Hugging Face org, if you do that, you'll get updates about new things that arrive.\nIf you go to the GitHub repo, you'll find good first issues and you can start to build yourself there.\nand probably just to try kernels.\nIt's one line of code.\nYou can still get in CoLab\nand you'll just kind of see the gains.\nJust feel it.\nThat's awesome.\nHopefully a lot of people get involved.\nLooking forward to hearing more.\nAwesome talk earlier today.\nThank you.\nThanks so much, man.\nThanks.\nThanks so much, everybody.\nThank you.",
  "transcript_chars": 10114,
  "ingested_at": "2026-05-15T10:40:26.562479+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 1388,
    "like_count": 54,
    "channel_id": "UCHlNU7kIZhRgSbhHvFoy72w",
    "categories": [
      "Education"
    ],
    "tags": []
  }
}