{
  "video_id": "T-MusIIayAE",
  "channel_slug": "weightsbiases",
  "channel_handle": "weightsbiases",
  "title": "Accelerate LLM post training with W&B Serverless SFT ",
  "duration_seconds": 663.0,
  "url": "https://www.youtube.com/watch?v=T-MusIIayAE",
  "upload_date": "",
  "transcript": "Hi, I'm Russ from Weights   Biases, and\ntoday I'll be discussing how W B\ntraining serverless SFT, powered by\nCoreWeave, helps AI engineers use\ncurated data sets for model\ndistillation, customize model output\nformat and style, and warm models up for\nRL training so they start from a\nstronger baseline.\nAs the barrier to entry for building AI\napplication systems diminished over the\npast several months, both the number of\nAI agents and the number of developers\nbuilding AI agents have grown\nexponentially.\nWe're now in a world where more often\nthan not, agents are building other\nagents during interactive vibe coding\nsessions or much longer\nset-it-and-forget-it runs.\nBut even as the proliferation of agents\nhas exploded, the exact same challenge\nremains.\nIt's easier than ever to build demos or\nprototypes, but hard to productionize.\nEven after debugging and ironing out\ncoding issues, agent performance must be\noptimized along multiple dimensions,\nincluding accuracy, latency, cost, and\nsafety.\nThere are a number of ways to optimize\nagents, including prompt engineering,\nretrieval augmented generation or RAG,\nand LLM slopping.\nMore resource-intensive post-training\ntechniques include reinforcement\nlearning and supervised fine-tuning.\nPre-trained large language models are\nstrong next token predictors, but\ndepending on what purpose they're\nserving, a customer support agent, a\nresearch agent, a coding agent,\nthey usually benefit from fine-tuning to\nhandle specific tasks like question\nanswering, reasoning, and other agentic\nactions.\nSupervised fine-tuning, or SFT, is one\nof the most efficient ways to teach LLMs\nhow to perform specific tasks.\nAnd combining SFT with reinforcement\nlearning, or RL, has become a trusted\napproach for producing reliable,\nproduction-ready agents.\nIn fact, it's become common for teams to\nrepeat the cycle between SFT and RL,\nrefining performance until the agent\nmeets deployment standards.\nBut executing this loop between SFT and\nRL is easier said than done.\nThe problem is that alternating between\nSFT and RL means potentially moving\nmodel checkpoints and artifacts back and\nforth between different systems,\nor storing and managing model weights\nbetween runs depending on evaluation and\ndecision-making schedules.\nThese challenges impede rapid iteration\nand can delay optimization progress and\nultimately time to market.\nW B training serverless SFT removes the\nobstacles associated with post-training\nagents, allowing AI engineers to switch\nquickly between SFT and RL as often as\nneeded without shuttling model artifacts\nacross different systems.\nWith W B training, engineers receive\ninstant, seamless access to CoreWeave\nGPU capacity.\nThis means no infrastructure headaches\nas provisioning, scaling, and optimizing\nare all handled for you automatically.\nTo get started, you just call the\nopen-source agent reinforcement trainer,\nor ART, API, specifying your data set\nand a base model.\nDuring and after the SFT run, resulting\nLoRA adapters are saved directly to W B\nartifacts.\nServe up these model weights using W B\ninference, collect traces, and then run\nserverless RL to improve reliability on\nagentic tasks.\nRepeat the cycle as many times as needed\nto reach the performance levels required\nto move your agent into production.\nTo demonstrate serverless SFT, we're\ngoing to use a simple coding agent that\nwe've developed and post-trained using\nW B training.\nOur agent can handle application\nimprovement requests and general coding\nquestions.\nInputs are delivered to the planner\nagent, which has a couple of tools at\nits disposal for considering and acting\nupon the request.\nAnd then the planner agent output, often\nconsisting of code changes or\nrecommendations,\nis sent to a review agent for a quality\nassessment and approval or rejection\nbefore sending a response back to the\nuser.\nWe've run through some initial\nevaluations using proprietary and\nopen-source base models.\nLet's take a look at the results using\nweave before we jump into some\npost-training. Weave helps developers\nevaluate, monitor, and iterate\ncontinuously to deliver AI agents with\nconfidence.\nI've run a bunch of evaluations on our\ncoding agent, specifically examining\nperformance across multiple dimensions\nusing different planner agent LLMs. In\nthis case, all of them base models with\nno fine-tuning.\nRight off the bat, our Qwen 3 model\nlooks decent, but still didn't score\nquite as high in terms of accuracy as\nour GPT models.\nHowever, as to be expected with our\nopen-source model here, latency\nand cost look great.\nLet's jump over to W B training to see\nhow we can improve our agent model,\nstarting with serverless SFT. And then\nalso take a look at how easy it is to\ntransition between serverless SFT and\nserverless RL to optimize model\nperformance and reliability. To get\nstarted quickly with serverless SFT,\njust check out the W B training docs at\ndocs.wandb.ai/training.\nHere you'll find a product overview and\na link\nthat'll take you directly to the\nserverless SFT page with detailed\nexamples and some sample code for common\nSFT use cases, including distillation,\nwhere a smaller model learns from the\noutputs of a larger teacher model,\nand RL warm-up, where you use SFT to\ngive the model a head start, then switch\nto RL for further improvement.\nComing back to our coding agent, our\ngoal with serverless SFT is to teach our\nLLM a specific output style or format to\nuse when responding to developer\nqueries.\nOur desire for consistency and brevity\nin the output is twofold.\nOne, we expect our users to expect and\nappreciate concise, to-the-point\nresponses.\nAnd two, less tokens, lower cost.\nFor our training data set, we're using\nthe Code Alpaca 20k data set.\nFor fine-tuning an LLM, this data set\ncan help the model become better at\nfollowing code-related instructions,\nrecognizing the difference between a\ncode question and an application coding\nrequest,\nand producing responses in a more\nuseful, developer-oriented format.\nAnd for our base model, we've started\noff with Qwen 34B,\nserved up using W B inference powered by\nCoreWeave.\nOf course, W B inference offers a wide\nselection of open-source AI models, so\nas we progress, we'll plan to give a few\nothers a try as well.\nYou can see all of our training runs\nhere on the left,\nand our results here on the right.\nAgain, we're looking for that sweet spot\ncombination of accuracy plus minimal\ntokens, and we don't mind low latency,\neither.\nIn addition to the standard training and\nvalidation metrics that we're\ncollecting, we're also running weave\nevaluations on our agent using the\nfine-tuned LoRA weights after every\nepoch.\nThe ability to include weave panels\ndirectly in our W B workspace and build\ncharts using our evaluation results\ngives us all of the data points we need\nto track the progress of our serverless\nSFT run and decide on our next steps.\nThese fine-tuned models are certainly\nlooking promising.\nAre they ready for some final testing\nand then deployment?\nOr should we kick off another run\nadjusting configuration parameters and\nmaybe trying out a different base model?\nAnother option is to pursue our\nserverless SFT serverless RL training\nloop to see whether that might further\nimprove performance. Because we've been\nrunning evaluations on models after\nevery epoch,\nwe know at which steps they've been\ndemonstrating the best performance.\nUsing W B training, we can fork and\nstart our serverless RL runs from the\noptimal serverless SFT checkpoints.\nServerless SFT has gotten us the output\nstyle and formatting that we want.\nAnd now we can use serverless RL to\nachieve our ideal agent behavior\nproviding the best possible experience\nfor our end user by generating accurate\ncode and concise easy-to-understand\nJust as before in our serverless SFT\nworkspace, I'm also consistently running\nweave evaluations during my RL runs so I\ncan continually track model progress\nalong all of my important performance\nmetrics.\nAnd once we've got Laura weights that\nare good production candidates, we can\nserve them up for final testing and\nproduction deployment using W B\ninference serverless Laura inference.\nThis run here seems to produce the best\nresults.\nTo get instructions for accessing the\nLaura's, I simply click on the run name\nand And click the artifacts link up top.\nAnd then click on the final set of\nweights here.\nScroll down to the inference access\nsection.\nAnd I've got sample Python and curl code\nsyntax to get me started.\nI can also click the try in playground\nbutton and test the model out in the\nweave playground for a quick interactive\ntest.\nJust ask a coding question right here.\nGreat to get an initial sanity check.\nBut of course, we may want to run this\none through a final round of thorough\nrigorous e-vals before we make it\navailable to our end users. Combining\nW B training and W B inference offers\nthe opportunity to achieve\nstate-of-the-art performance with lower\nlatency and that a fraction of the cost.\nThe Weights   Biases AI developer\nplatform eliminates the barriers\nassociated with infrastructure\nrequirements and post-training\nexpertise, making it easier to build\nproduction-ready models.\nThanks for your time. And if you haven't\nalready, head on over to wandb.ai,\nsign up for a free account, and give the\nWeights   Biases AI developer platform a\ntry today.",
  "transcript_chars": 9260,
  "ingested_at": "2026-05-15T10:41:46.292410+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": 186,
    "like_count": 7,
    "channel_id": "UCBp3w4DCEC64FZr4k9ROxig",
    "categories": [
      "Science & Technology"
    ],
    "tags": []
  }
}