{
  "video_id": "reddit_1txro73",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "PSA: Gemma 4 12B is NOT completely broken for coding and tool calling, you need a special chat template",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1txro73/psa_gemma_4_12b_is_not_completely_broken_for/",
  "external_url": null,
  "upload_date": "20260605",
  "published_at": "2026-06-05T17:31:53+00:00",
  "transcript": "This is a PSA for people like me who tried it and hit the wall with tool calls failing left and right, so much so that harnesses like OpenCode just didn't work:\n\nThere is a fix for that. You need to pass a better chat template file, [which is available](https://gist.github.com/jscott3201/ad69c4ffbd79f18b11a0f6a94c94fadf) (I did not write it). [See also this comment.](https://www.reddit.com/r/LocalLLaMA/comments/1twmw4o/comment/oppmvdg/)\n\nTo actually use it with llama.cpp, **first compile llama.cpp from source,** then download the chat template file I linked above, then try this (8 bit quant in this case):\n\n    ./build/bin/llama-server -hf unsloth/gemma-4-12b-it-GGUF:UD-Q8_K_XL --host 127.0.0.1 --port 8899 --jinja --chat-template-file ./custom-pub-chat-template-gemma4.jinja\n\nI'm not saying the results are great, or good, or better or worse than Qwen 3 9B or any other model! But with this setting, the tool calling bugs go away and you can genuinely evaluate its capabilities in opencode.\n\nSo, please do that before forming a judgement of the model's coding ability.\n\nBut once you've done that, judge away 😀\n\nI'm posting because I see so many \"I can't code with Gemma 4 12B, tool calls never work\" comments that it's tough to cut through the noise when discussing the model.\n\nThanks to u/HVACcontrolsGuru for bringing the solution to my attention. I hope I'm not stealing their thunder, just thought it was time to call more eyeballs to this.\n\n\n\n--- Top Comments ---\n\n\n[44 upvotes] Haha that’s my template! Enjoy! I have another small patch to add later!\n\n[7 upvotes] Weird post. What does the chat-template have to do with llama.cpp built from source vs using the binaries? Absolutely nothing.\n\n[7 upvotes] Is this not helping anyone else? I dont see any change. Its still saying its going to do stuff, then does nothing, over and over.\n\n[6 upvotes] In addition to picking which quant to run, folks now also need to choose the right chat template? I remember similar thing happened to Qwen 3.6 too.",
  "transcript_chars": 2010,
  "ingested_at": "2026-06-06T01:30:01.880015+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 66,
    "upvote_ratio": 0.92,
    "num_comments": 28,
    "author": "boutell",
    "is_self": true
  }
}