{
  "video_id": "reddit_1ufd4g8",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "GLM 5.2 on consumer hardware",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1ufd4g8/glm_52_on_consumer_hardware/",
  "external_url": null,
  "upload_date": "20260625",
  "published_at": "2026-06-25T15:22:55+00:00",
  "transcript": "I tried out the unsloth quants of GLM 5.2 on still \"consumer-ish\" hardware:\n\n32C Zen5 Threadripper Pro 9975 WX, Asus WRX90E-SAGE-SE PCIe Gen5, 512GB DDR5 ECC RAM @ 4800MHz, dual RTX 5090.\n\nThis machine was put together pre-RAMpocalypse, and by then not exceedingly expensive compared to today's grotesque prices.\n\nThe quant I used was unsloth/GLM-5.2-GGUF, UD-Q5_K_S (492GB of weights).\n\nI used a freshly compiled (cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=\"120f\" -DGGML_CUDA_FA_ALL_QUANTS=ON -DGGML_CUDA_FORCE_MMQ=ON -DGGML_SCHED_MAX_COPIES=1 -DGGML_CUDA_GRAPHS=ON -DGGML_CCACHE=OFF -DGGML_CUDA_ENABLE_UNIFIED_MEMORY=0; cmake --build build --config Release -j 64) llama.cpp with the following invocation:\n\n    CUDA_VISIBLE_DEVICES=0,1 numactl --physcpubind=0-31 --localalloc llama.cpp/build/bin/llama-server \\\n    --model ./GLM-5.2-UD-Q5_K_S-00001-of-00012.gguf \\\n    --temp 1.0 \\\n    --top-p 0.95 \\\n    --min-p 0.01 \\\n    --fit on --no-mmap  --flash-attn on --ctx-size 32768 --no-warmup --prio 3 \\\n    --threads 32 --threads-batch 32 --numa isolate --log-verbosity 4 --split-mode layer --direct-io --jinja\n\nWith this I get consistently 12t/s. I just tried chatting, no agentic stuff.\n\nThere is very little to none variation of speed by omitting or using last line's llama.cpp options; same applies to the numa stuff.\n\n\n\n--- Top Comments ---\n\n\n[16 upvotes] Prompt processing speed is the biggest concern here, do you know how much do you get?\n\n[15 upvotes] https://preview.redd.it/daeikij4sg9h1.png?width=640&format=png&auto=webp&s=4a8c049366ef87e66e6f286b5b869d4816658d78\n\n[10 upvotes] I also tried it. Its very good, but prompt-processing is super slow, around 30 tok/s. This is a problem of llama.cpp as VLLM and even DS4 have prompt-processing speeds of >2000 tok/s on the same hardware with similar sized models.\n\n[11 upvotes] What is prompt processing speed?",
  "transcript_chars": 1880,
  "ingested_at": "2026-06-26T01:30:04.538516+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 53,
    "upvote_ratio": 0.91,
    "num_comments": 66,
    "author": "phwlarxoc",
    "is_self": true
  }
}