{
  "video_id": "reddit_1tmf63y",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "BitCPM-CANN: Native 1.58-Bit Large Language Model Training on Ascend NPU",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1tmf63y/bitcpmcann_native_158bit_large_language_model/",
  "external_url": null,
  "upload_date": "20260524",
  "published_at": "2026-05-24T15:24:26+00:00",
  "transcript": "Paper: https://github.com/OpenBMB/MiniCPM/blob/main/docs/BitCPM_CANN.pdf\n\n### Abstract\n\n>We present BitCPM-CANN, a systematic family-level study of 1.58-bit (ternary)\nquantization-aware training (QAT) on the Huawei Ascend NPU platform. To address\ntwo practical gaps for extreme low-bit LLMs—whether ternary weights preserve capabili-\nties on complex reasoning tasks at on-device scales, and how to make end-to-end 1.58-bit\ntraining natively available outside the CUDA ecosystem—we port our prior GPU-based\npipeline to CANN, MindSpeed, and Megatron-LM, and train four models (BitCPM-\nCANN-0.5B/1B/3B/8B) strictly aligned with their full-precision MiniCPM4 counterparts\nin architecture and pre-training data. Across 11 benchmarks spanning commonsense\nreasoning, domain knowledge, and mathematics & reasoning, the 1B, 3B, and 8B variants\nretain 95.7%–97.2% of full-precision performance, with the 3B variant achieving parity on\nBBH and the 3B/8B variants recovering nearly all of GSM8K. The 0.5B variant retains\n90.1%, with the residual gap concentrated on mathematics, indicating that capacity—not\nthe quantizer—is the bottleneck at sub-billion scales. Our QAT integration adds only\na 4.5% training throughput overhead (148 vs. 155 TFLOP/s per NPU), making ternary\ntraining viable as a default configuration, while enabling up to an 8× weight memory\nreduction (approximately 6× end-to-end including scaling factors) at inference. To our\nknowledge, this is the first end-to-end 1.58-bit training system on a domestic NPU scaled\nup to 8B parameters, providing a reusable low-bit training infrastructure for the Ascend\necosystem\n\nBitCPM-CANN was trained in ternary from scratch with the same data as MiniCPM4. MiniCPM4 8B achieves comparable performance with Qwen3-8B trained with 36 trillion tokens using only 8 trillion tokens. (MiniCPM4 was released last year: https://arxiv.org/abs/2506.07900)\n\n- https://github.com/OpenBMB/MiniCPM\n\n- https://huggingface.co/collections/openbmb/bitcpm-cann\n\n\n\n--- Top Comments ---\n\n\n[10 upvotes] 4.5% training overhead for 6-8x memory savings at inference is wild if it actually holds... The fact that their 0.5B struggles on math while the bigger ones don't is a good sign they're being honest about where ternary breaks down. Curious if anyone independently reproduces the 8B numbers.\n\n[6 upvotes] I tried their 0.5b and 1b models in tq2_0 with llama cpp on CPU in termux. Worked without any fuss and as fast as you'd expect.\n\n\nI do have to say though,in my testing they weren't very smart. Maybe I'm doing something wrong but I got more mileage in question/answer and world knowledge with lfm 2.5 350m ,than I did with their 1b\n\n\nStill, it's stupid cool this is a thing now. I've been dreaming of the mythical bitnet since that Microsoft paper. Double so they can do it without Nvidia at all.\n\n[3 upvotes] I'm eagerly waiting for 20+B models(Ex: 27B, 31B, 35B-A3B, 26B-A4B, etc.,) in this format ASAP.\n\n[2 upvotes] theory in paper is interesting but the performance im still waiting for real usable model. But this new approach is exciting.",
  "transcript_chars": 3075,
  "ingested_at": "2026-05-25T01:30:01.658746+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 52,
    "upvote_ratio": 1.0,
    "num_comments": 16,
    "author": "Aaaaaaaaaeeeee",
    "is_self": true
  }
}