{
  "video_id": "reddit_1tyb3np",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "DeepSeek V4 Flash is amazing! (WIP llama.cpp PR #24162)",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1tyb3np/deepseek_v4_flash_is_amazing_wip_llamacpp_pr_24162/",
  "external_url": null,
  "upload_date": "20260606",
  "published_at": "2026-06-06T07:56:42+00:00",
  "transcript": "In case you're not aware already, the DeepSeek V4 series is finally getting supported on llama.cpp [with this PR](https://github.com/ggml-org/llama.cpp/pull/24162)!\n\nThe PR is at a very early stage right now, so only try it if you're consciously willing to experiment out of curiosity and accept severe stability/performance tradeoffs. It runs very slow (5-6 tps), GPU and FA support need work, etc., but it is reliable-enough already for correctness.\n\nThis is my most anticipated model and I had some time to spare, so I ended up downloading the HF model for DS-V4-Flash and quantizing it myself using the PR(Made a custom 3-bit quant to mimic the full-sized model's tensor layout). And wow!\n\nThe model perfectly addresses the crucial three pillars for local inference IMO:\n\n- The model's intelligence is amazing for its size. First time a local model in this size range actually feels comparable to frontier models, and I'm not exaggerating.\n- Fares a lot better against quantization since it's natively an FP4-FP8 hybrid. This is crucial for local deployment and is my primary problem with models like MiniMax M2.7, where I'm not happy even with UD-Q4_K_XL.\n- Incredibly efficient with context window scaling. Consumes way less KV cache size with no flash attention!\n\nQwen 3.5/3.6 series is also a huge hit amongst the local community since it addresses the three pillars above way better than its competitors. However, I feel the DeepSeek model has levelled it up even further, and I predict it will easily dominate the 80-140GB model space for many more months to come.\n\nHuge shoutout and thanks to fairydreaming [for their relentless work on getting DSA implemented](https://github.com/ggml-org/llama.cpp/pull/21149), and to am17an and pwilkin for taking this up! Really looking forward to this PR getting merged!\n\n\n\n--- Top Comments ---\n\n\n[36 upvotes] The llama.cpp team is amazing, thanks for your work, looking forward to this model!\n\n[20 upvotes] 100 gb of vram seems to be quite a high spec for the majority of users. \n\n",
  "transcript_chars": 2031,
  "ingested_at": "2026-06-06T13:30:01.871860+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 102,
    "upvote_ratio": 0.97,
    "num_comments": 63,
    "author": "Lowkey_LokiSN",
    "is_self": true
  }
}