{
  "video_id": "reddit_1vyq2v4",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "[Megathread] Qwen3.8-Flash-Next - Release Day",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1vyq2v4/megathread_qwen38flashnext_release_day/",
  "external_url": null,
  "upload_date": "20260826",
  "published_at": "2026-08-26T07:26:50+00:00",
  "transcript": "Megathread for discussing the release of Qwen 3.8 Flash Next.\n\n* Quants\n* Fine-Tunes & Abliterations\n* Chat Templates\n* Inference Server Support & Configuration\n* Experiences, Benchmarks & Model Comparisons\n\nWe'll try to clean up future duplicates around the release and point them here.\n\n# Highlights\n\nThe first open-weight release under this architecture is Qwen3.8-Flash-Next, which introduces:\n\n* **Hybrid Attention with QSA**: The Gated DeltaNet and Gated Attention pairing has been reworked into Gated DeltaNet and Qwen Sparse Attention (QSA). Rather than selecting individual tokens for processing, QSA operates at the micro-block level. This cuts long-context latency significantly, a critical gain as agentic workloads increasingly dominate real-world usage.\n* **Gated Residual**: Residual streams with normalisation are what make deep LLM training manageable. Gated Residual modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and a per-branch scalar write gate. This brings finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low.\n* **N-gram Embedding**: Embeddings provide a unique axis for parameter scaling that requires less computation and is more amenable to offloading than Mixture-of-Experts (MoE). By indexing with short n-grams, this approach makes parameter scaling highly efficient for memory-constrained accelerators without sacrificing quality.\n* **Tailored Training Recipe**: The Muon and AdamW optimisers are applied to specific weight categories to maximise efficiency. Guided by refitted scaling laws, we eliminate traditional batch-size warmups and start directly at the target batch size, substantially reducing total optimiser steps while safely supporting larger learning rates for robust convergence.\n\n# Model Overview\n\n* Type: Causal Language Model with Vision Encoder\n* Training Stage: Pre-training & Post-training\n* Language Model\n   * Number of Parameters: 125B with 6B activated, plus 51B n-gram embedding and 4B MTP\n   * Hidden Dimension: 2560\n   * Token Embedding: 248320 (Padded)\n   * N-gram Embedding: 20,000,000 (bigrams/trigrams at layer 2)\n   * Number of Layers: 48\n   * Hidden Layout: 12 × (3 × (Gated DeltaNet → MoE) → 1 × (Qwen Sparse Attention → MoE))\n   * Gated DeltaNet:\n      * Number of Linear Attention Heads: 48 for V and 16 for QK\n      * Head Dimension: 128\n   * Qwen Sparse Attention:\n      * Number of Attention Heads: 24 for Q and 2 for KV\n      * Head Dimension: 256\n      * Rotary Position Embedding Dimension: 64\n      * Indexer Structure: MQA with 4 Query Heads and 1 Shared Key Head\n      * Indexer Head Dimension: 128\n      * Budget: 512 blocks or 2048 tokens\n   * Mixture Of Experts\n      * Number of Experts: 512\n      * Number of Activated Experts: 10 Routed + 1 Shared\n      * Expert Intermediate Dimension: 640\n   * Gated Residual:\n      * Number of Branches: 4\n      * Bottleneck Rank: 320\n   * LM Output: 248320 (Padded)\n   * MTP: 1 layer, trained with multi-steps\n* Context Length: 262,144 natively and extensible up to 1,000,000 tokens.\n\nhttps://preview.redd.it/d94jf1p3tplh1.png?width=2885&format=png&auto=webp&s=8af470ae8b2c93e0427e3f6d335faafcf8356fcc\n\nRecommended sampling parameters for generation:\n\n* Thinking Mode: `temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`, `presence_penalty=0.0`, `repetition_penalty=1.0`\n* Instruct (or non-thinking) mode: `temperature=0.7`, `top_p=0.80`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`, `repetition_penalty=1.0`\n\nOfficial Links:\n\n* HF: [https://huggingface.co/Qwen/Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)\n* MS: [https://modelscope.cn/models/Qwen/Qwen3.8-Flash-Next](https://modelscope.cn/models/Qwen/Qwen3.8-Flash-Next)\n* Repo: [https://github.com/QwenLM/Qwen3.8-Flash-Next](https://github.com/QwenLM/Qwen3.8-Flash-Next)\n* Blog: [https://qwen.ai/blog?id=qwen3.8-flash-next](https://qwen.ai/blog?id=qwen3.8-flash-next)\n* Technical Report: [https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech\\_report.pdf](https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf)\n* vLLM: [https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next](https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next)\n* SGLang: [https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next)\n\nPopular:\n\n* Unsloth GGUF: [https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)\n\n\n\n--- Top Comments ---\n\n\n[1 upvotes] I'm off for the night now. I'll update the post again in the morning (AEST) with any new official links etc. Hopefully this post will help prevent the incredible flood of duplicated posts we see for major model releases. Feel free to report any duplicates over the next day or so.\n\n[68 upvotes] I really hope QSA (qwen sparse attention) will have much less compute headroom or the ability to offload kv cache to SSD without the massive bandwidth bottleneck of the ssd since it's sparse. I was able to do it it for deepseek v4 and it worked well enough. I hope llama.cpp adds features that allow more freedom with where we want to load our models and parts of it, e.g. n-cpu-moe could be extended to n-ssd-ffn or n-cpu-ffn or n ssd kv or engram etc\n\n\n\n[53 upvotes] Existing large pre-release threads on it with some more information and discussion:\n\n* The model is around [125B A6B with 51B n-gram](https://www.reddit.com/r/LocalLLaMA/comments/1vy6smx/qwen38flashnext_this_architecture_could_be/) which can be offloaded.\n* There should be day-0 Unsloth (so probably llama.cpp) [support](https://www.reddit.com/r/LocalLLaMA/comments/1vxybmy/qwen_38_flash_next_day_0_support_from_unsloth/) for it.\n* It's sort of a [Qwen 4 preview](https://www.reddit.com/r/LocalLLaMA/comments/1vxwtyd/qwen38flashnext_tomorrow/) (thread also has modelscope link and way more discussion)\n\n[13 upvotes] Can ngrams be offloaded to disk without much slowdown? I only have 76gb of ram so I was thinking of loading the main weights into memory in q4 and keep the ngrams on the SSD.\n\n[13 upvotes] if QSA cuts the kv reads enough then SSD offload stops being bandwith bound and a 125b A6B n-gram offloaded gets pretty runnable on modest ram. really hoping n-ssd-kv / n-cou-ffn flags land in the llama.cpp soon enough . sooner the better ",
  "transcript_chars": 6411,
  "ingested_at": "2026-08-26T13:30:05.159430+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 237,
    "upvote_ratio": 0.97,
    "num_comments": 146,
    "author": "sammcj",
    "is_self": true
  }
}