{
  "video_id": "reddit_1upl93b",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "Qwen's J-Space - Anthropic's discovery of an internal model Global Workspace",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1upl93b/qwens_jspace_anthropics_discovery_of_an_internal/",
  "external_url": null,
  "upload_date": "20260707",
  "published_at": "2026-07-07T05:03:20+00:00",
  "transcript": "[Anthropic published research today](https://www.anthropic.com/research/global-workspace) into what a model is thinking behind the scenes while it is deciding what to actually write. \n\nMore importantly, they [released the J-Space lens code ](https://github.com/anthropics/jacobian-lens)and their partner put together a [demonstration of Qwen 3.6 27B J-Space](http://neuronpedia.org/jlens).\n\n\n\n--- Top Comments ---\n\n\n[38 upvotes] [https://transformer-circuits.pub/2026/workspace/index.html](https://transformer-circuits.pub/2026/workspace/index.html)\n\nFull write-up is in Transformer Circuits.\n\nI don't know, this whole thing sounds like more formal rediscovery of (well known) intermediate layer abstract representation.\n\nI have been doing mid-training by upscaling for years and I think in a more naive sense, one could think of the thinking path of those LLMs as a marble falling down n-layer maze (but with geometry more akin to \"basins\" rather than tight labyrinths). In those intermediate layers the basins are not as defined as final layers, and the marble can easily flip into the opposite side with various momentum. But the final few layers there tend to be two wholly separate basins (base-model like corpus and instruct-model like corpus; at the end of the day, your model is a classifier as well) and the marble will continue to fall into the same side (along with a bit of nudge in each layer). If you duplicate intermediate layers, the model tends to just \"work\" with a bit of CPT, but it was very difficult to make it work by duplicating the final layers because you need very defined basins in those final layers,\n\n[26 upvotes] this confuses me like nothing else\n\ni only asked about maths, didn't change anything\n\nwtf is qwen thinking about man 😭\n\nhttps://reddit.com/link/ow163fv/video/yv5cvryzxqbh1/player\n\n[15 upvotes] I haven't looked into it yet just watched their video they put up on youtube and idk why but I feel like this J-Space consciousness-thingy thing is bullshit. I know I'm jumping to this conclusion way too early that too without looking at the code and other things.\n\nThe reason why I'm feeling this way is that in their video they explained that the model \"thinks\" of different words and all internally but I feel like it's more because of the Attention mechanism playing around. In attention we have multiple heads. Each head explores different words & concepts so that the final transformed hidden state is as useful to the FFN as possible.\n\nI believe that in that process attention heads in Claude (and this sound very plausible to me cuz Claude definitely has 100s of heads) process different concepts which obv results in improved quality of our hidden state. After that when that transformed hidden state is passed to the FFN, it \"thinks\" of concepts that Anthropic might be talking about.\n\nIn very oversimplified terms we can say that Attention takes dot-product of input tokens with each other and FFN takes dot-product of those transformed tokens with all the tokens that are stored in the FF",
  "transcript_chars": 3037,
  "ingested_at": "2026-07-07T13:30:14.613959+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 75,
    "upvote_ratio": 0.93,
    "num_comments": 43,
    "author": "AutomataManifold",
    "is_self": true
  }
}