{
  "video_id": "reddit_1tagtpv",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "I catalogued every way local models break JSON output and built a repair library, here's what I found across 288 model calls",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1tagtpv/i_catalogued_every_way_local_models_break_json/",
  "external_url": null,
  "upload_date": "20260511",
  "published_at": "2026-05-11T21:17:18+00:00",
  "transcript": "I've been running structured output prompts through a bunch of models on OpenRouter for the past few months — Llama 3, Mistral, Command R, DeepSeek, Qwen, and every other model on OpenRouter — alongside the usual closed-source suspects. 288 calls total. I wanted to know what actually breaks, how often, and whether open models fail differently from the API-only ones.\n\nShort answer: not really. The failure modes are almost identical across the board. The *rate* varies — some models hit you with markdown fences on nearly every call, others only when you phrase the prompt a certain way; but the categories of breakage are the same everywhere.\n\nWhat I saw most, roughly in order:\n\n1. Markdown fences wrapping the JSON (the model thinks it's being helpful)\n2. Trailing commas (JS habits from training data)\n3. Python `True`/`False`/`None` instead of JSON `true`/`false`/`null`\n4. Truncated objects from running out of tokens mid-response\n5. Unescaped quotes inside string values\n6. `//` or `#` comments inside JSON\n7. Literal `...` where the model got lazy and didn't generate all the data\n\nThe reason I'm posting here specifically: most of the advice I see for handling this is \"just use JSON mode\" or \"use a constrained grammar.\" And yeah, those help when they're available. But a lot of what people run locally doesn't have reliable JSON mode, grammar-based generation has its own tradeoffs (speed, compatibility), and even when you do get syntactically valid JSON you can still get schema violations and truncation.\n\nI ended up building a Python library ([outputguard](https://github.com/ndcorder/outputguard)) that validates against JSON Schema and runs 15 repair strategies in a specific order when things break. The ordering part turned out to be more important than I expected: fixing encoding before structure, and re-parsing between each strategy so later fixes don't undo earlier ones.\n\nAlso handles YAML, TOML, and Python literals, which came up more than I thought it would once I started working with models that don't have a JSON mode and just output whatever format they feel like.\n\nWrote up the full findings in a blog post if anyone wants the details: [What Breaks When You Ask an LLM for JSON](https://thecrosswalk.news/what-breaks-when-you-ask-an-llm-for-json)\n\n2,001 tests, MIT licensed, no LLM provider dependencies. `pip install outputguard`\n\nCurious what other people's experience has been — are you seeing the same failure patterns, or are there models/quants that behave differently than what I'm describing?\n\n\n\n--- Top Comments ---\n\n\n[9 upvotes] How does this compare to json-repair?\n\nhttps://github.com/mangiucugna/json_repair\n\n[7 upvotes] gg for citing anticient model, it almost seems as if an AI wrote this post 🧐",
  "transcript_chars": 2745,
  "ingested_at": "2026-05-12T11:04:32.404199+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 51,
    "upvote_ratio": 0.8,
    "num_comments": 46,
    "author": "kexxty",
    "is_self": true
  }
}