{
  "video_id": "reddit_1ttvpmt",
  "channel_slug": "LocalLLaMA",
  "channel_handle": "r/LocalLLaMA",
  "title": "llama: limit max outputs of `llama_context` by am17an · Pull Request #23861 · ggml-org/llama.cpp",
  "url": "https://www.reddit.com/r/LocalLLaMA/comments/1ttvpmt/llama_limit_max_outputs_of_llama_context_by/",
  "external_url": "https://github.com/ggml-org/llama.cpp/pull/23861",
  "upload_date": "20260601",
  "published_at": "2026-06-01T15:29:02+00:00",
  "transcript": "# Overview\n\ncontinue [\\#23764](https://github.com/ggml-org/llama.cpp/pull/23764), this PR only reserves logits space for `n_seqs` when possible. With `-ub 2048` and MTP, **this saves another 1.2GB of VRAM** for me. I've tested `llama-perplexity` also and it seems to work fine. But maybe there is a better API, putting up as a draft for now According to me an API in llama-context is a good solution for this, by default it will reserve all tokens but specifically in server-context we can set it to 1 whenever possible.\n\n\\- u/am17an\n\n\n\n--- Top Comments ---\n\n\n[44 upvotes] 1.2gb saved just by not being dumb about logits allocation. everyone chases bigger models while this kind of low level optimization makes local inference actually usable on consumer gpus. more of this please\n\n[10 upvotes] 7900XTX on Vulkan running Gemma4 26B-A4B, this does save 1.2GB for me, enabling running a higher quant and more context. Awesome to see\n\n[7 upvotes] Can we merge this and your previous VRAM-saving PR into your Gemma 4 MTP PR branch? (merging Master into \"am17an:gemma4-mtp\" branch)\n\n I use that branch and i'm a bit too lazy and underskilled to merge every time your nice optimizations gets into master\n\nThanks man",
  "transcript_chars": 1209,
  "ingested_at": "2026-06-02T01:30:30.684905+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 65,
    "upvote_ratio": 0.97,
    "num_comments": 28,
    "author": "pmttyji",
    "is_self": false
  }
}