{
  "video_id": "hfba9dAT6xE",
  "channel_slug": "techwithtim",
  "channel_handle": "techwithtim",
  "title": "The Best LOCAL Agentic Coding Workflow (Complete Guide)",
  "duration_seconds": 2031.0,
  "url": "https://www.youtube.com/watch?v=hfba9dAT6xE",
  "upload_date": "",
  "transcript": "In the past few months, local models\nhave gotten incredibly good and\nrelatively easy to run. So much so that\nyou can now use them for full local\ncoding. That means you don't need\ninternet connection and you can do full\nagentic coding on your own computer\nwithout spending any money or\nsubscribing to any platform just by\nhaving the right model and configuring\nyour environment properly. So in this\nvideo, I'm going to give you a tutorial\non exactly how to do that. We're going\nto talk about model selection, how local\nmodels actually work, and more\nimportantly, how you can configure them\nand set them up to do agentic coding.\nIt's one thing to get a local model on\nyour computer. It's another to get it\nworking inside of a coding environment\nso it can actually execute tasks, write\nfiles, write code, do bash commands. You\nget the idea. So, with all that said,\nlet's get started. So, I just want to\nspend 30 seconds here talking about how\na local model works. Then, we're going\nto get into model selection, which is\nsuper important. So whenever you're\ndoing a gentic coding, you're\nessentially just using someone else's\ncomputer for your autocompletes or for\nyour, you know, agent tasks or code\ngeneration or whatever. Now, that other\ncomputer that you're using is extremely\npowerful. Typically, it has like a\nterabyte of RAM, a bunch of graphics\ncards, and it can run really powerful\nmodels. Then you can just essentially\ncall that computer and get the response\nback. Now, when you're using a local\nmodel, everything's on your own\ncomputer. So, you're not calling out on\nthe internet. you're going you're not\ngoing to some kind of server and more\nimportantly you're not paying for any\ncompute. If I'm using cursor or I'm\nusing cloud code I'm using cursor's\nmodel and I'm paying their fees. If I'm\nusing cloud code I'm using anthropics\nmodel and I'm paying their fees. If I'm\nusing a local model I'm just running it\non my own machine and it doesn't cost me\nanything. So I can run this you know an\ninfinite amount of time and there's no\ncost whatsoever. It's just going to be\nusing my own hardware. Now, that means\nthat the type of model you can run is\nfully dependent on what type of hardware\nyou have. And we're not yet capable of\nrunning, you know, Opus level models on\nour own computer. We can run ones that\nare pretty good that are close to maybe\nlike a Sonnet or a Haiku. And depending\non the type of computer you have,\nobviously your kind of mileage and\nperformance is going to vary. So, keep\nthat in mind. If you're on a really old\ncomputer, this video is probably not\ngoing to be that useful. But if you're\non a newer computer, if you have a\nhigh-end graphics card, you're going to\nsee that you can get some really good\nperformance to the point where you can\nactually just use this fully all the\ntime and only go to a high-end model if\nyou really need something complex done.\nOkay, so that said, let's talk about\nselecting the model because that's super\nimportant. And then after we do the\nselection, we'll go into all of the\nsetup. So, in terms of selecting the\nmodel, the type or more importantly the\nsize of the model that you can run is\ngoing to be dictated by pretty much one\nnumber and that's your video memory or\nyour video RAM. Okay? Otherwise known as\nVRAM. Now, in order to find that number,\nit really depends what operating system\nyou're on. So, if you're on a Windows\ncomputer, you hopefully know the\nhardware that you're running. If not,\nyou should be able to locate that\nrelatively quickly. I believe you can go\nto your task manager and you should be\nable to see what your graphics card is.\nand that graphics card is going to have\na certain amount of VRAM. So, if you're\nrunning an Nvidia GPU, for example,\nmaybe you're running a 4090 or a 5090 or\na 3060 Ti or whatever the value is. Just\nfigure out what the VRAM is, the amount\nof video memory. For most modern\ngraphics cards, you're going to be\nlooking at 8 GB, 12 GB, 16 GB, sometimes\n24 GB, depending on the model, and of\ncourse, you know, how expensive it was\nand what you paid for. Now, that number\nis effectively going to dictate the\nmaximum size of model that you can run\non your computer. I'll talk more about\nthat later, but if you have a 16 GB VR\nRAM graphics card, you can effectively\nrun a model that's up to 16 GB in size.\nShould be a little bit smaller, but\nabout that size. If we talk about Mac,\nthings change a little bit because on\nMac, especially the newer Macs, any M\nseries Mac, you have something called\nunified memory. Now, what that means is\nthat the RAM on your computer. So, in my\ncase, I have a M5 Max with 64 GB of RAM\nis actually shared between the CPU and\nthe GPU. So, I have 64 GB of RAM, just\nnormal unified memory on my Mac\ncomputer. And that technically means\nthat my GPU could use up to 64 GB of\nvideo memory or of this unified memory.\nSo, if you're on Windows, you're looking\nspecifically at your graphics card and\nhow much VRAM is there. If you're on\nMac, you're looking at how much unified\nmemory you have. And if you want to find\nthis on Mac, it's really easy. You can\ngo to the Apple icon. You can go to\nabout this Mac and just look for memory\nhere. And you can see that I have 64\ngigs. Now, this is only going to be\napplicable on the M series Mac. If\nyou're running an Intel Mac, then you're\ngoing to want to look at how much VRAM\nis on the integrated graphics card\nthat's on your computer. But if you're\non an Intel Mac, then this video is\nprobably not going to work that uh very\nwell for you and you're only going to be\nable to run really kind of local models\nbecause those are just very old machines\nat this point. Now, one thing to be\naware of is that while you have unified\nmemory and it technically all could be\nused by the uh what do you call GPU in\ntheory, it's not all accessible. There's\nsome memory that's required to run the\noperating system, other processes that\nyou have. So, while you might have 64 GB\nof RAM like me, maybe only 55 GB of that\ncan actually be used to run the model.\nSo essentially just figure out okay how\nmuch memory do I have? How much VRAM do\nI have on Windows? And how much unified\nmemory do I have on Mac and then we're\ngoing to subtract about 10 to 15% of\nthat and that's about the maximum size\nmodel that we'll be able to run with\ndecent performance. Okay. Now you can\nactually see I made a slide for this and\neffectively on Mac yeah 75 to 80% of\nyour unified memory is probably usable\nand if you're on Windows you can use\neffectively all of it because you have\nall of that RAM specifically for the\nGPU. It's not shared like it is on a\nMac. Now, one thing to note is that if\nyou are running on a Windows computer,\nyour memory throughput is going to be a\nlot faster than on a Mac computer. So,\nfor example, if we look at an RTX 4090,\nwe have 1,08 GB per second in terms of\nthe speed of the memory. If we're\nlooking at M4 Max, we have 546 GB per\nsecond. Don't worry too much about this,\nbut this is effectively the speed at\nwhich the memory can read and write\ndata. So, while you might be able to\nload a bigger model on Mac, having a\ndedicated GPU, especially a high-end\nGPU, usually you're going to get more\ntokens per second or a faster response\nfrom the model because it's just faster.\nOkay? So, there's a difference between\nspeed and kind of capability. And just\nbecause a model is bigger doesn't\nnecessarily mean it's better. And same\nthing, just because it's smaller doesn't\nmean it's worse. The speed also is\nfactored in based on the hardware that\nyou're using. So with Mac, you can have\nbigger models generally, but if you have\na dedicated GPU, while the models you\ncan load are typically a little bit\nsmaller, they're going to be a lot\nfaster because the memory bandwidth is\ntypically quicker. Okay, so let's keep\ngoing and let's just quickly go through\nkind of a cheat sheet of what the size\nmodel you'll be able to run based on the\ncomputer that you're on. So models have\na number of parameters and you're going\nto see this advertised when you start\nlooking at them. You'll see 7 billion,\n14 billion, 32 billion, 70 billion, 480\nbillion. And the number of parameters\nwill dictate the size of the model. So\ntypically speaking, if you have 8 GB of\nVRAM, you can run a 7 billion parameter\nmodel. If you have 12 to 16 gigs, you\nshould be able to run a 14 billion one.\nIf you have 24 gigs, you should be able\nto run a 32 billion one. And if you have\n64 gigs or more, you should be able to\nrun a 70 billion model. Now, this is not\nalways the case, and we're going to look\nat this later, but this is kind of a\ngeneral guideline for the size of the\nmodel that you can run. And some models\nwill be smaller or larger with the same\nnumber of parameters based on kind of\nhow they're tuned, the quantitization,\nwhich we'll look at later. I'm just\ntrying to give you a highle\nunderstanding because the model\nselection is by far the most important\npart. Okay, now let's just quickly have\na look at what model you should be\nrunning based on the hardware tier that\nyou have. Now, if you want to look for\nmodels, you can go to hugging face.\nHugging Face has a ton of different\nmodels that you can scroll through here.\nYou can filter and you can find all\ndifferent types of models that you can\nrun on your machine, but these are the\nones that I'm going to recommend. If you\nfollow along, this is just going to be\nsuper easy because I've tested a lot of\nthese different models. Now, usually\nwe're going to use two models when we're\nsetting up um kind of local development.\nWe're going to have one really small,\nreally fast model, which we use for\nautocomplete if we're actually going to\nbe typing and, you know, getting\nautocomplete from code. And then we're\ngoing to use one much larger, more\ncapable model, which is what's going to\nbe used for chatting, editing, agentic\nmode, whatever. So, you're going to have\ntwo different models. For most of us,\nwe're just going to be using this Gwen\n2.5 coder with 1.5 billion parameters\nfor our autocomplete model. Pretty much\nany computer that you have is going to\nbe capable of running this model. Um,\neven if you don't have a graphics card,\nbut if you don't have a graphics card,\nit's going to be fairly slow. So,\nanyways, for autocomplete, that's pretty\nmuch just what we're going to use. And\nthen for our chat and edit model, this\nis really going to depend on the amount\nof VRAMm that you have. Now, there's a\nfamily of models that are very good for\nrunning locally and doing agentic\ncoding, and it's the Gwen family of\nmodels. So, you'll see Gwen 2.5, Gwen\n3.6, Gwen 3.5, Gwen Coder next. Okay, so\nthere's a bunch of different models with\ndifferent sizes. We'll look at them here\nin a second. But for example, if you're\nrunning on a 16 GB unified memory laptop\nwith Apple or 8 GB of VRAM, uh for a\nnormal PC, you should be able to run\nthis Gwen 2.5 coder with 7 billion\nparameters. Not a ton of parameters.\nIt's going to take up a few gigabytes of\nspace. Should be possible to run and run\nwith decent performance. Next, if we go\nup in performance here and we have 24 to\n32 GB of RAM on Apple or 12 to 16 on a\ndedicated GPU, we should be able to run\nthis 14 billion parameter model or this\n30 billion parameter Gwen 3 coder model.\nOkay, so again, I'm just putting a cheat\nsheet here so you guys can see them\nlater. If we have more than 24 GB of\nVRAM or on a really high-end machine\nlike I am, you should be able to run\nthis Gwen 3 coder model. If we keep\ngoing, you should be able to run the\nGwen 3 coder next model if you have 64\nGB or more of memory. And then if you\ndon't have any GPU at all, you're going\nto use the same model that you'd use for\nautocomplete as your main completion\nmodel, which is Glen 2.5 coder with 1.5\nto 3 billion parameters. Again, we'll\nlook at this in a minute. I'll go into\nit in more detail, but I'm just giving\nyou a quick cheat sheet. And I made even\na larger table here with exact models in\nterms of GPUs and max uh specs, just a\nfew of them that are pretty common, so\nyou can see which model I would\nrecommend. Okay, so have a quick look at\nthis. See if any of this applies to you.\nAnd now we're going to get into the\nactual setup. We'll look at a few models\nspecifically and this will start to make\na little bit more sense. And one last\nthing to be aware of, while you can run\nany of these models on your computer,\nassuming you have enough hard drive\nspace, they're really only going to be\nefficient to run if they fit into the\namount of VRAM that you have. So, this\nis why I focus so much on the VRAM. If a\nmodel is 10 gigabytes, but you only have\n8 gigabytes of VRAM, the model can still\nbe loaded and it can still run on your\ncomputer, but the overflowing amounts,\nlike you have 8 gigabytes in VRAM, the\nother two GB is going to go in your\nnormal system memory or maybe even on\nthe disc depending on kind of how things\nare loaded, which is going to make it\nlike 100 times slower than if it was\nfully loaded in your video memory. So,\npretty much just only run models that\nare smaller than the amount of VRAM that\nyou have if you want them to run in any\nreasonable amount of time. Again, you\ncan always run these models, but they\ncan just be extremely slow if you can't\nload them in your VRAM. Okay, so we're\ngoing to keep going and get into the\nsetup. But one thing I want to make you\naware of is that this presentation is\navailable from the link in the\ndescription in case you want to check it\nout for yourself. And you'll notice that\nI actually have it deployed to this\nhere.now domain. Now, here is actually a\nlong-term partner of the channel. They\nare completely free to use and they are\na way that you can deploy sites really\neasily, especially for demo purposes, by\nliterally just copying one single\nprompt. Let me show you what I mean. If\nyou just go to here.now, I'll leave a\nlink to it in the description. You can\ncopy their agent instructions. And this\nis actually exactly what I did when I\nwas building this presentation using\ncloud code. You can just paste the\ninstructions here and say, \"Hey, I want\nyou to set up here.now and I want you to\ndeploy this site.\" And you literally\njust paste it inside of here. I already\ndid this, so you can see it does it. And\nit will just deploy this to a website\nfor you automatically. So you see it\ndeployed to this site that I was\nloading, right? You guys can now view\nit. And then if you want to claim the\nsite and have it be available for more\nthan 24 hours. You just need to create a\nfree account and sign in just so you can\nsave the site to a particular account.\nYou can see if I go to my dashboard\nhere, I have a bunch of sites that have\nalready been deployed. This is\ncompletely free. You don't need to pay\nfor anything. You actually don't even\nneed to make an account unless you want\nto claim the site. And then of course,\nif you do want to upgrade, you can pay\n$4 per month, $20 per month if you want\nmore sites, a bunch of storage, but I\nhaven't had to do that yet and I use\nthis a ton. Anyways, massive shout out\nto her here. now for sponsoring this\nvideo and providing this service because\nI use it all the time and I genuinely\nthink it is great. And with that said,\nlet's go to the next part of the video.\nOkay, so now that we have kind of a\nbasis and understanding, we can start\nactually setting this up. Now, there's\ntwo pieces of software we're going to\nuse in this video. The first is LM\nStudio. The second is Visual Studio\nCode. So, download both of these if you\nwant to follow along exactly. You can\nuse other code editors as well, but VS\nCode recently added a feature that makes\nit really easy to run these local\nmodels. So I'm going to suggest that you\ngo with that and it's just kind of the\nbest experience for local model\ndevelopment. Trust me, I have tried a\nlot. Okay, so download Visual Studio\nCode, download LM Studio. Again, both\nthese are free. You don't need to pay\nfor anything. And then what we're going\nto do is we're going to open LM Studio.\nAnd I'm just going to walk you through\nkind of the basic user interface here.\nNow LM Studio allows you to download and\nmanage with some more precision than\nsomething like O Lama local models. Now\nwhat you can do is you can go to the\nchat, which is kind of in the top left\nhand corner here. You can go to the\ndeveloper view, which allows you to run\na server, which we're going to look at\nin a second. And you can go to the model\nview, which is what we're going to want\nto go on here, where we can search for\nmodels, and we can see our current LLM.\nAnd you can see I have some downloaded.\nAnd then what we're going to want to do\nis go to, sorry, actually this last\nbutton here, kind of the robot button,\nwhere we can start searching for the LLM\nthat we want to download and use. Now,\nif you're unsure which one to use, feel\nfree to ask Claude Code, look at my\npresentation, or just copy exactly what\nI use in this video. Again, really\ndepends on the amount of VRAM that you\nhave. So, in my situation, I have 64 GB\nof RAM on this computer. I'm using an M5\nMax, very high-end machine, which means\nI can run some pretty powerful models.\nNow, the models that I'm going to be\nrunning in this video are going to be\nthe Gwen 3.6 35 billion parameter model\nand the Gwen 3 coder Next model. Now,\nI'm going to talk to you about how I\ndecided to pick these and then I'll go\nthrough a few smaller ones that you can\nuse because I know a lot of you don't\nhave the same level of hardware that I\ndo. So, the reason that I decided to\npick these models is first of all, their\ncapabilities. Now, when you're looking\nat models, you want to make sure that\nthey have the ability to do tool use.\nOkay? So, this is going to be for your\nmain chat model. Okay? The autocomplete\none is basic. For the main chat and edit\nmodel, it needs to have tool use. If it\ndoesn't have tool use, it's not going to\nbe able to actually call the tools to\ncreate the files. It's just going to be\nable to give you a chat response. So,\nyeah, it can tell you, you know, what\nthe code is, but it can't actually edit\nit, modify it, which is what makes these\nuseful. So, first of all, you need tool\nuse, right? Okay. Now, for this one\nhere, the reason I picked it is because\nit's 22 GB. So, I know for sure I can\nload it in my VRAM because I have 64 GB\nof RAM. And I'm using this Q4 version,\nwhich is what's called quantitization.\nNow effectively when you pick these\nmodels you can have the entire model\nokay which is all of the parameters\nnothing shortened nothing fine-tuned\nwhatever which is going to be quite\nlarge and that this case it's 37 GB for\nthe 35 parameter billion for the 35\nbillion parameter model we have Q6 which\nis just a smaller more condensed version\nof this and then we have Q4 which is an\neven smaller version of this you have\ndifferent levels of what's called\nquantitization effectively you lose a\ntiny bit of performance but it shrinks\nthe size of the model drastically. So,\nit's almost always best to just go with\nkind of the lowest quantitization\nversion of the model that you're capable\nof running on your machine. And this is\nalso going to make the model a lot\nfaster. The smaller the model is, the\nfaster it's going to run. So, I decided\nto go with this one cuz I'm like, okay,\n35 billion parameters, that's a lot.\nIt's still super capable. It has tool\nuse. It's going to easily fit in my\nmemory and it's going to run relatively\nquickly rather than me trying to run a\nsuper powerful, really big model that's\ngoing to be a lot slower. And I'll show\nthis to you in a second. Okay, so that's\none model that's a great choice. Another\none is the Gwen 3 Coder Next. Now, this\nis a much larger model. I'm going to\nshow it to you in a second. It's about\n44 GB on my computer. Uh, but the reason\nI picked it again is because it has tool\nuse and because this is kind of one of\nthe state-of-the-art local models that\nyou can be running. And fortunately, it\njust fits on my machine. I tested both\nGwen 3 coder Next and this 35 billion\nparameter Gwen 3.6 on my machine. Both\nwork really well. This one is just quite\na bit slower. So, typically I prefer to\nuse this Gwen 3.6. Okay, that's for me,\nright? Because I'm running a pretty\nhigh-end machine. For most of you,\nyou're probably going to want to look at\nthe Gwen 3.6\n14 billion parameter model. Okay, so you\ncan scroll through here. You'll see that\nthere's this 14 billion parameter model\nwith this A3B. What A3B stands for is\nactive 3 billion parameters. Don't worry\ntoo much about that, but effectively it\nmeans that what's actually going to be\nrunning on your graphics card is a\nsmaller number of parameters than the\nfull size of the model, which allows you\nto run larger models um on worse\nhardware. So, if you see these A3B, A6B,\nA9B, whatever, that's saying the number\nof active parameters, and that means the\nmodel's typically going to be more\nperformant on your machine, even if it's\na little bit larger. So if you look\nthrough the options here, you can see\nthat there's a Q4, a Q3, a Q2. So\ndepending on again the amount of VRAM\nthat you have, pick the quantitization\nand try to pick something that is maybe\nlike 50 60% of the VRAM that you have.\nSo it runs pretty quickly. So for most\nof you, if you have 16 GB of VRAM, you\nshould be able to run this model no\nproblem on your machine. There's a few\nother ones like there's a Deep Seek\nmodel that's capable of being ran. You\ncan see this one is pretty small. We're\nlooking at like uh what do you call it?\n4 6 whatever gigabytes with 8 billion\nparameters. So have a look. do some\nresearch, see which one you want to try\nto run. The Gemma 4 models are also\npretty good. They're very new, but\nthey're not as good at coding. Uh, and\nagain, the Gwen ones are the ones that I\nwould probably recommend to pick. I know\nI'm rambling a lot, but I'm trying to\ngive you as much info as I can because\nthis model selection is really going to\ndictate the performance, and you're\nprobably going to want to play around\nwith a few of them and see the best\nresults that you get. Okay, so that is\nfor just downloading your main kind of\nchat model. Once you find the one you\nwant, download it, wait for it to\ndownload, and then make sure that we get\nour autocomplete model. The one we're\nall probably going to want here is the\nGwen Coder Next 2.5 with 1.5 billion\nparameters. Uh, I'm typing it\nincorrectly, but if we scroll through\nhere, we should be able to find it. Uh,\nokay, let me just find it from here.\nWhat is it called? It is, let's see, my\nLLMs. Let's make this a little bit\nlarger. Okay, Gwen 2.5 coder 1.5 billion\ninstruct. I just downloaded the largest\nversion of it cuz I have a really\nhigh-end machine, but you can download\nthe smaller version, which is 900\nmegabytes. Okay, so these are the models\nthat we want. Now, once we have these\nmodels, if we just want to test them and\nmake sure they're working, we can just\ngo to the chat window here. We can\nselect the model from the top of the\nscreen. So, we can go, you know, Gwen\n3.6 and we can adjust a bunch of\nsettings for this model. Now, what we're\ngoing to want to do is just make sure\nthat our GPU offload is as high as\npossible. We're going to set the context\nlength. So, the more that you set this,\nso you can see it goes up to 262,000\num tokens here, obviously, the more room\nit's going to take up on your computer\nbecause this also needs to be loaded in\nmemory. And you can see at the top that\nit shows you the amount of memory that's\ngoing to be used here as you scroll\nthrough and set the context. So, you can\ncheck versus your VRAM to see what you\nshould set this at. There's a bunch of\nother settings that you don't really\nneed to go through. You just want to\nmake sure GPU offload is as high as\npossible so it uses the full GPU. And\nthen you set the context length. Then\nyou can load this model. It's going to\ntake a second to load. You can see that\nat the top. And then you can start\nchatting with it just to make sure it's\nworking. And you should see that you\nstart getting tokens outputed. Now, you\ncan see for me I was getting 100 tokens\nper second. That's quite fast. And\nyou're not going to feel like it's a\nslow model if you're running more than\nlike 50 60 tokens per second. But you\ncan again play with it and see what\nspeed you're getting. Let's quickly\nchange over to the Gwen 3 coder next.\nAnd you're going to see that in some\nsituations it's going to tell you, hey,\nlike you cannot load this model. It's\ngoing to take up too much room on your\ncomputer. I found found that this is\npretty aggressive and that most of the\ntimes the model will actually load. So\nif it's giving you this issue, don't\nworry about that too much. Feel free\njust to crank up the context. I'm going\nto go here and just press load anyways\nand just wait and see if it loads. If it\nloads, you should be good to go. Then\nyou can go ahead and use it and see the\nperformance that you're actually\ngetting. Keep in mind that if you're on\nMac, if you have a lot of other things\nopen, you're probably going to want to\nclose those when you're doing this so\nthat your memor is freed up so that you\ncan do this. So now we've switched the\nmodel. Um, let me just go hello. And now\nyou can see we're using Gwen 3 coder\nnext. I believe this should work. Uh,\nand let's see if we get a response.\nOkay, now it did give me a response\nhere, but this one took 25 seconds even\nthough we had 82 tokens per second. So\nsometimes on Mac, if your other\nprocesses are using the memory, you're\ngoing to have a delay and it's kind of\nfighting for the space. I believe\nbecause I'm screen recording right now,\nI'm just at that limit where I'm not\nable to uh, you know, run this\nefficiently. So for this video, I'll\njust use the Gwen uh, what is it? 3.635\n635 billion parameter model, but you\nguys get the idea. Okay, so just test\nthem, make sure they work, and then what\nwe're going to do is go over here to\nthis developer section. Now, from this\ndeveloper section, what we need to do,\njust going to eject this for right now,\nis we need to preload the models that we\nactually want to use when we're coding.\nWe're going to need to do this every\ntime that we're setting up coding. And\nwe're going to need to start this\ndevelopment server. Now, starting this\ndevelopment server is going to expose\nthese models to our local machine, which\nmeans VS Code will be able to use them.\nand then we can see all of the logs and\neverything that's going on in here. So,\nI'm just going to turn on the server and\nthen we're going to go load model and\nwe're going to load our autocomplete\nmodel and we're going to load the chat\nmodel that we're using. So, I'm going to\nuse Gwen 3.6, right? Like I said, so I'm\ngoing to load that up here. You can\nchange the settings again. Make sure GPU\noffload is as high as possible and then\nadjust the context based on how much\nmemory you have here. And you're going\nto have to play with these settings a\nlittle bit to see what works best for\nyou. So, that's loading. While it's\nloading, we're going to load the\nautocomplete model as well. Same thing.\nI'll just set the context really large\neven though we probably don't need that\nmuch. Let's set the GPU offload and then\njust load this up. And then both of\nthese are now ready. Okay, so both of\nthese models are now loaded, meaning\nthey'll be accessible when we start\nusing them from VS Code. And the next\nstep is to set this up in VS Code. So in\norder to set this up in VS Code, first\nof all, make sure you have the newest\nversion. If you have the newest version\nof VS Code, they have a feature that\nallows you without signing in or being\nconnected to the internet to manage\nlocal models. So, first thing I did here\nis I just went to file, open folder. I\njust opened a new folder on my desktop.\nSo, you just make a new folder and open\nit just so you're inside of some kind of\nproject. And then what you're going to\ndo is open what's called the command\npallet. So, you're going to hit control\nshift and then P on your keyboard or\ncommand shiftp if you're on Mac. This is\ngoing to open what's called the command\npallet. From here, you're going to type\nmanage and you're going to look for\nmanage. It's like models or something.\nYeah, manage language models. Okay, this\nis the new feature that they've added\nlike a week ago or something. Now, from\nhere, what we can do is simply press on\nadd models. When we press on add models,\nwe're going to go custom endpoint. And\nfrom custom endpoint, we're just going\nto give this a name. So, the name, you\ncould just call this LM Studio. I\nalready have one called LM Studio, so\nI'm just going to call it LM Studio 2.\ndoesn't really matter. This is just\nwhat's going to show up in the list. So,\nwe're going to uh press that. For the\nAPI key, just put any random values in.\nYou just need something. So, just put\nrandom string and hit enter. And then\nwe're going to go with chat completions.\nOkay. So, select chat completions. And\nthen what you're going to see is that it\nopens up this config file for you. Now,\nin this config file, what we need to do\nis just fill in three values here. The\nID, name, and URL. It's super basic.\nWhat we're going to do is we're going to\ncopy this URL right here. It'll be the\nsame that I'm typing in. So, you can\njust copy it from what you see me uh\ntyping, but it's also available here in\nLM Studio. And this is the URL to the LM\nStudio uh what do you call it? Um API.\nOkay. So, we're going to paste that URL.\nAnd then for the ID, we're going to\nselect uh the ID of the model that we\nwant to use. Now, the ID is available\nright here in LM Studio. So you can just\npress this copy button and it will\ndirectly copy it for you and just paste\nthat ID. And then for the name, you just\nput the exact same thing. Okay. Now\nyou'll see that for the capabilities it\nshows tool calling and then vision. You\nonly want to select vision if vision is\na capability of the model. For Gwen 3.6\nit is. So we'll just leave vision as\ntrue. And then you can adjust the max\ninput and output tokens. So in my case,\nI believe we can see the context of\nthis. Yeah, it's this. So, I'm just\ngoing to copy this number here and\nadjust the max input tokens. And then\nfor the output tokens, you can set out\nwhatever you want. I'm going to crank\nthis up to 64,000.\nOkay. So, what this has now done is it's\nsimply added a model. And if we want to\ntest if this is working, let me just\ndelete my other uh settings here so that\nwe don't mess anything up. We can just\nsave by hitting Ctrl S on the keyboard\nand then exit out of this. In order to\nsee if this is working, what we'll do\nagain is go can go command shiftp or\ncontrol shiftp. So open up the command\npallet again. Same thing, manage\nlanguage models. And we're just going to\nsee if this model is appearing. And you\ncan see that it is. It says Gwen Gwen\n3.6 35b a3b. Now, if we want to adjust\nthis, we can press this button right\nhere, which is opening the language\nmodels JSON. So it's going to bring us\nright back here. And then we can add\nanother model as well. So I'm just going\nto add my next model. The way I can do\nthat is just copy this and put it inside\nof LM Studio here. So notice there's\nthis kind of models array. For this next\none, I'm just going to say vision is\nfalse.\nOkay. And for the max input tokens,\nwe're going to adjust this. It's going\nto be a lot smaller. We're just going to\ngo like 32,000. And then for the ID\nname, and the URL, we can put the\nautocomplete model. So I'm just going to\ncopy this. Okay. And just change the\nmodel here for our autocomplete. Okay.\nCool. Go ahead. And now we've added that\nand we have this other model added as\nwell. Although for the autocomplete\nwe're going to have to do something a\nlittle bit different which I'll show you\nin a second. But this is how you can add\nmultiple models in case you want to\nswitch between them. So now that we have\nthis, what we want to do is we want to\nopen the chat window in VS Code. So to\nget to chat I believe you can toggle it\nfrom the top somehow or you can hit\ncontrol L or command L on your keyboard.\nNow you just want to go into this chat\nwindow. Should be open by default. I\nbelieve you can also go view appearance.\nYou should be able to find chat\nsomewhere. I don't know exactly how you\nopen it usually in VS Code. Here you go.\nChat in the setting. It should open it\nup. That just unopened it, of course,\ncuz it was already open. But from chat,\nwe can make a new chat. And then we\nshould be able to select our model. So\nif we go to select model and we go to\nother models, we should see the models\nwe've added. So we have this Gwen 3.6\nmodel. And then we can say something\nlike, \"Hey, can you make me a game of\nchess using JavaScript and React?\" Okay,\ngo ahead and press enter. By the way,\nyou guys always ask me what I'm using to\ndictate here. It's a really cool tool\ncalled Whisper Flow. I'm an absolute\npower user of the tool. You can see I\nhave over 133,000 words. It's free to\nuse. It's the best AI voice dictation.\nIt works right inside of these code\neditors. You can also like directly tag\nfiles. And I have a long-term\npartnership with them. So, if you guys\nwant to download it and support me, I\nhave a link below that you can use.\nAgain, totally free. And then you can\nupgrade if you want to get more limits,\nwhatever, all the features that they\nhave. Okay, cool. So, you can see this\nis now working, right? It's functioning.\nIt's actually using the model, although\nit seems to be a little bit slow. And if\nI go here, you can now see in LM Studio\nthat it's generating tokens. And if we\ngo to this developer logs at the bottom\nof the screen here, you'll be able to\nsee all of the requests that are being\nsent back and forth, the speed, what's\ngoing on, the responses, any errors, and\nwe've set this up. And now we have\nfunctioning local models inside of\nVisual Studio Code, which is super cool.\nSo, it's going to take a second here to\ndo this. My fan is now spinning up on my\ncomputer and my performance is going to\nbe a little bit degraded because I am\nrecording my screen. So, it might be a\nlittle bit laggy here for you guys. Uh,\nbut overall, this should function. I'll\nwait for it to finish. Then, we'll come\nright back and see if this is working.\nOkay, so it actually just completed the\ngame here. Now, there was a little bit\nof a glitch where it was trying to\naccess like an integrated browser, which\ndidn't work, and I gave you some bug,\nbut it's now functioning. It's created\nthis huge index file, this script.js,\nJS, which is like 600 lines. The\nstyle.css. And while it didn't make it\nin React like I asked for, it did make\nthe fully functioning game. And now if I\nwant to just open this. Let's just test\nit out and see if this actually works.\nUm, okay. So, clearly there's some bug\nor something in the chess game because\nit's not loading. But the point is it\ndid some coding. So, you know, I'm going\nto have to uh debug that. And maybe I\nshould have given it a simpler task to\nbegin. But it is working. Okay. So\nthat's how you set up the chat, right?\nAnd this now is going to allow us to do\nagentic coding. We can change the mode,\nask, agent, plan, whatever. Set up\ndifferent agents, test them, see which\none is going to work. We can tag\ndifferent files and just use this like\nwe would use any agentic code editor.\nHowever, if we want autocomplete to\nfunction locally as well. So if we don't\nhave an internet that this will work,\nwhat we can do is install an extension\nin VS Code and this extension is called\ncontinue. So to install the extension,\nyou can go to this little puzzle piece\nicon here. You can search for continue.\nYou can open this up. And continue\nessentially will do the same thing that\nwe've already done inside of VS Code,\nbut it will just allow us to uh set the\nautocomplete, which is I don't believe a\nnative setting right now in VS Code. So\nyou can install continue. There should\nbe a button here that says install. And\nonce you install that, you should see an\nicon on the bottom left hand side of\nyour screen that looks uh sorry, that's\nnot what I meant to type. looks\nsomething like, let me just put it here,\nlike this. Okay, it should show this\nweird kind of hexagonal thing. And now\nthis is going to open up essentially a\nchat window, which is a clone of what we\njust saw in VS Code. So what we can do\nis we can configure models inside of\nthis continue extension. They're going\nto work the same way that they would\nwork natively in VS Code. And we can\nconfigure the autocomplete. So, what I'm\ngoing to do is just use continue to\nconfigure the autocomplete, but you also\ncould just fully use this configuration\nuh for everything else as well. So, I\nknow it might be a little bit confusing.\nLet me just show you what I mean. What\nwe're going to do is go to settings\nhere. From settings, if we go to models,\nyou should see that it says like setup\nmodel. It should show you some default\noption or something. What we can do is\njust simply press this add button, we\ncan change the provider to say LM\nStudio. And then for the model, we can\njust pick anything random. It doesn't\nhave to be the one that we actually\ninstalled. If it's there, obviously\nselect it, but if not, just pick\nanything and just press connect. From\nhere, it's going to bring you to this\nconfig file. And at the bottom, it's\ngoing to have this kind of connected\nmodel, um, which is the one that you\njust added. What we want to do here is\nwe just want to change this model. So,\nwhat it shows right here to be the name\nof the model that we have in LM Studio.\nSo, what I can do is just select this\nright here, the Gwen 2.5 coder. I can\nchange the model like that and now it's\ngood and we've simply just added this\nand of course we can change the name\nGwen 2.5 coder. However, if we want this\nto work for autocomplete, let me just\ndelete this cuz I'm going to show you a\nfinished version. We need to one second\nhere. Let me get rid of this. Add the\nRO. Okay, so ROS is just an array here\nthat says autocomplete. So effectively\nwhat you can do is just copy what I have\nhere if you use the same model. put the\nname, put the provider, the model, the\nAPI base, which is just this, which will\nalready be there. And then the rules-a\nautocomplete. Once you add that, you can\njust save this file s. And if you come\nback here now, you can set this\nautocomplete model. And if you set this\nautocomplete model, now when you're\ntyping inside of VS Code, it's going to\nuse that for the automatic completions.\nFor the chat, you can also add your own\nchat model here. And then just make sure\nthat you add these roles and\ncapabilities like I have here. chat edit\napply and then tool use. Again, you can\njust literally copy the configuration\nthat I have and then that will work in\nthis chat window in the continue\nextension. I kind of prefer to use the\ndefault chat in VS Code. I find it works\na little bit better and it edits the\nfiles better. I'm just showing you this\nbecause this is how we set the\nautocomplete. But if you want to use\nthis chat window, you can and you can\nmess with it and see. Okay, so now if I\ngo to like index.html,\nactually let's go to script. Let's just\nwrite like if or something. You should\nsee that we start getting some\nautocomplete. Let's just go like\nconsole. Okay, whatever. And if I go\nback to LM Studio and I click into the\ncoder model. Um, we should see that\nwe're getting some token streamed. And\nregardless, it should be working in that\nuh LM Studio view. Sometimes you need to\nrestart VS Code in order for it to work,\nbut for me, that's pretty much all I\ndid. And now the autocomplete is\nfunctioning. And again, you can go\nsettings. There's a bunch of other stuff\nthat you can adjust and change here.\nOverall, I don't change anything there.\nI just set the autocomplete model, set\nthe chat model stuff and we are pretty\nmuch good to go and that is going to\ncomplete the setup. I mean that gives\nyou the autocomplete model, that gives\nyou the chat, the edit, whatever. And\nnow you can just do full local coding on\nyour machine. Now again, the results are\ngoing to vary here. This is not as good\nas Opus. It's not as good as using these\ncloud providers. But if you're on a\nplane and you need to use this for a few\nhours, if you're out of credits and you\nwant to do something small, if you're\ndoing, you know, smaller inline edits,\ncreating some functions, not trying to\ndo like super complex prompts, this\nworks and it works pretty well. Now,\nlater I will do a video going through\nthe performance of these local models\nand talk about that more in depth. But\ngenerally speaking, I just wanted to\nshow you guys how to set this up. If you\nfound this useful, make sure leave a\nlike, subscribe to the channel, and I\nwill see you guys in another [music]\nvideo.",
  "transcript_chars": 40659,
  "ingested_at": "2026-06-17T16:31:29.152116+00:00",
  "source": "channel",
  "yt_meta": {
    "view_count": null,
    "like_count": null,
    "channel_id": null,
    "categories": null,
    "tags": null
  }
}