{
  "video_id": "reddit_1vkrnb5",
  "channel_slug": "MachineLearning",
  "channel_handle": "r/MachineLearning",
  "title": "Transformers are famously bad at arithmetic, so I set one's weights by hand (no training) and it multiplies with 100% accuracy [P]",
  "url": "https://www.reddit.com/r/MachineLearning/comments/1vkrnb5/transformers_are_famously_bad_at_arithmetic_so_i/",
  "external_url": null,
  "upload_date": "20260810",
  "published_at": "2026-08-10T17:37:35+00:00",
  "transcript": "Obviously nobody needs a transformer that's good at multiplication. I wanted to know whether a stock transformer could do exact arithmetic if I chose its weights directly.\n\nI implemented the grade-school algorithm as a computation graph and compiled it into an ordinary Phi-3 Hugging Face checkpoint using Torchwright, a compiler I wrote. No training. The three-digit calculator gets all 3,000,000 supported expressions right. I've published checkpoints to Hugging Face that support up to 12 digit x 12 digit multiplication.\n\nFor fun, I also disabled reasoning and tested six frontier models. Accuracy falls off a cliff as the numbers get longer; at seven digits, five scored 0/500. Mine stays at 100%, although it has the considerable advantage that I put the multiplication algorithm directly into its weights.\n\nI ended up building four versions: grade-school, hardware-style, scratchpad, and brute-force memorization. They compute the same function while spending layers, width, generated tokens, and parameters very differently.\n\nWrite-up: https://ood.dev/posts/calculator/\nRepo: https://github.com/physicsrob/torchwright\nCheckpoint: https://huggingface.co/physicsrob/torchwright-calculator-simple-max-digits-3\n\n\n\n--- Top Comments ---\n\n\n[99 upvotes] This reminds me of the `It's Hard for Neural Networks To Learn the Game of Life` paper\n The authors crafted a hand made network that could take a game of life at step N and prodice step N+3. Then they tried to train a bunch of different network sizes with different initialized weights, and found that very few managed to achieve the goal, and the none managed to do so in the same small size that the hand made version had.\n\nThere is likely multiple things that we could handcode to speed up the trainability of these networks.\n\n[65 upvotes] this is actually a really cool way to get around the arithmetic weakness. putting the alogirthm directly into the weights is kinda wild lol\n\n[14 upvotes] kinda wild that this is less about teaching the model math and more about turning the model itself into a calculator. The fact that it works without any training is probably the most interesting  part ",
  "transcript_chars": 2152,
  "ingested_at": "2026-08-11T13:30:10.939204+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 187,
    "upvote_ratio": 0.88,
    "num_comments": 39,
    "author": "notforrob",
    "is_self": true
  }
}