{
  "video_id": "reddit_1ud5wx2",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "100+ t/s on Qwen3.6-27B Q8 across a 5090 + 3090 Ti — switching to tensor split-mode got me from 70 to 100+",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1ud5wx2/100_ts_on_qwen3627b_q8_across_a_5090_3090_ti/",
  "external_url": null,
  "upload_date": "20260623",
  "published_at": "2026-06-23T03:29:12+00:00",
  "transcript": "Wanted to share a setup that's been working great for me. Running Qwen3.6-27B at Q8\\_0 across two GPUs (RTX 5090 + RTX 3090 Ti) and getting \\~100 t/s.\n\nThe big jump came from switching `--split-mode` to `tensor`. I was sitting at 70+ t/s on layer split before that. Tensor split keeps both cards busy on the same tensors instead of handing whole layers back and forth, and with a fast/slow pairing like this it made a real difference. Pairing it with a 70/30 tensor split (favoring the 5090) to match the relative compute.\n\nFair warning: this thing turns into a proper space heater under load. During decoding both GPUs pull hard the entire time — 750W+ from the cards alone.\n\nThroughput depends on the prompt as well, with some reaching up to 130 t/s.\n\nFull llama.cpp server command:\n\nbash\n\n    llama-server \\\n    -m Qwen3.6-27B-Q8_0.gguf \\\n    -fa 1 \\\n    --n-gpu-layers 99 \\\n    --tensor-split 70,30 \\\n    --fit off \\\n    --main-gpu 0 \\\n    --split-mode tensor \\\n    --no-mmap \\\n    --mlock \\\n    --cpu-range 0-23 \\\n    --cpu-range-batch 0-7 \\\n    --ctx-size 196608 \\\n    --parallel 2 \\\n    --kv-unified \\\n    --jinja --no-warmup --threads 24 --numa isolate \\\n    --batch-size 2048 --ubatch-size 2048 --threads-batch 8 \\\n    --chat-template-kwargs '{\"preserve_thinking\": false}' \\\n    -cms 24000 \\\n    -ctxcp 5 \\\n    --alias qwen.3.6-27b.q8 \\\n    --spec-type draft-mtp --spec-draft-n-max 3 \\\n    --reasoning-budget 12288 \\\n    --reasoning-budget-message \"Wrap up your reasoning and give the final answer.\" \\\n    --host 0.0.0.0 --port 8080\n\nHappy to answer questions about the config.\n\nP.s. If you want to understand how tensor splitting works, you can find more information in the llama.cpp documentation here: [https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md](https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md)\n\n\n\n--- Top Comments ---\n\n\n[14 upvotes] You can reduce the power limit by probably 100W per card with surprisingly little impact on token generation speed.\n\nPrompt processing is slowed down fairly proportinally by power limits, but not eval -- it is memory bound rather than compute bound.\n\n[7 upvotes] Me with 5 tks on 4080\n\n[4 upvotes] Try to disable MTP, when paired with tensor mode the MTP backend is moved to CPU.\n\n[3 upvotes] Yup. Same gpus. Pushing 110-130 tks when coding with MTP and tensor split",
  "transcript_chars": 2358,
  "ingested_at": "2026-06-23T13:30:04.214420+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 57,
    "upvote_ratio": 0.94,
    "num_comments": 45,
    "author": "Shoddy_Bed3240",
    "is_self": true
  }
}