{
  "video_id": "reddit_1ub69xu",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "GLM 5.2, what speeds are we getting locally?",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1ub69xu/glm_52_what_speeds_are_we_getting_locally/",
  "external_url": null,
  "upload_date": "20260620",
  "published_at": "2026-06-20T20:11:58+00:00",
  "transcript": "Can everyone that is able to run GLM 5.2 locally report what their inference engine, system specs, quantization, context size, and tokens/sec?  If you're getting great numbers expect follow-up questions.  I'll start:\n\nllamma.cpp, 6x RTX 3090, 128 DDR5, i7-13700K, unsloth UD-IQ2_M, 90K context @ Q8_0 KV: 7.8 tokens/sec generation, prompt processing was roughly 40 tokens/sec\n\n\n\n--- Top Comments ---\n\n\n[88 upvotes] watching this thread with intrigue and a heavy dose of envy\n\n[54 upvotes] M3 ultra 512GB running mxfp4 at 24tok/s 16bit kv on a slightly customised mlx vlm with mtp enabled. Havent checked PP yet but its in the 150 ish region. Well i also have disaggregated prefill on a 5090 in another machine that does about 400 and passes kv over but thats only for long prompts\n\n[30 upvotes] With 4x3090 GPUs, 64-core EPYC 7763 CPU and 1 TB of 8-channel DDR4 3200 MHz RAM, running IQ4 quant and F16 KV cache, using llama.cpp, I get about 6 tokens/s generation (drops to 5 tokens/s around 90K context filled), 174 tokens/s prefill (it varies mostly in 150-200 range, but gradually decreases as context fills).\n\n[24 upvotes] M3 Ultra 512GB running on a slightly customized MLX 4.5-bit~ quant I built myself but haven't enabled MTP to any effect on yet - sitting at about 150tok/s PP and 18.5tok/s TG at an input tokens of ~128 and output tokens in the ~4096 range.\n\nIt's not amazing, but it's not unusable either.\n\n[25 upvotes] 8x RTX 6000 Pro max-q's with vllm and the b6k setup here: [https://github.com/local-inference-lab/rtx6kpro/blob/master/models/glm5.2\\_v11.md](https://github.com/local-inference-lab/rtx6kpro/blob/master/models/glm5.2_v11.md)\n\nWas getting about 40tk/s single-request on Luke's NVFP4, max of \\~2.2M ctx at FP8 KV cache. Batched throughput was \\~200tk/s:\n\n    ============ Serving Benchmark Result ============\n    Successful requests:                     256       \n    Failed requests:                         0         \n    Maximum request concurrency:             64        \n    Benchmark duration (s):                  327.28    \n    Total input tokens:                      262144    \n    Total generated tokens:                  65536     \n    Request throughput (req/s):              0.78      \n    Output token throughput (tok/s):         200.25    \n    Peak output token throughput (tok/s):    384.00    \n    Peak concurrent requests:                79.00     \n    Total token throughput (tok/s):          1001.23   \n    ---------------Time to First Token----------------\n    Mean TTFT (ms):                          44239.69  \n    Median TTFT (ms):                        49077.28  \n    P99 TT",
  "transcript_chars": 2629,
  "ingested_at": "2026-06-21T01:30:02.892066+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 58,
    "upvote_ratio": 0.92,
    "num_comments": 68,
    "author": "neverbyte",
    "is_self": true
  }
}