{
  "video_id": "reddit_1the441",
  "channel_slug": "MachineLearning",
  "channel_handle": "r/MachineLearning",
  "title": "A Simple Solution to Improve Broken Peer Review System at AI Conferences [R]",
  "url": "https://www.reddit.com/r/MachineLearning/comments/1the441/a_simple_solution_to_improve_broken_peer_review/",
  "external_url": null,
  "upload_date": "20260519",
  "published_at": "2026-05-19T06:12:02+00:00",
  "transcript": "An issue with the peer review system is reciprocal reviewing, which incentivizes reviewers to unfairly reject good papers to increase their own papers' chances of acceptance.\n\nMy proposed solution is that the conference should divide the authors/papers into 2 halves (A and B). If you are an author in half A, then you will only be a reviewer in half B.  All papers by the same author, their coauthors, and coauthors of coauthors should be in the same half.\n\nEach AC/SAC can only serve in one half and acceptance decisions for the two halves would be independent. So reciprocal reviewers will not have incentive to reject good papers to serve themselves.\n\nFurthermore, the discussion period for the two halves should not be concurrent. This way the reciprocal reviewer will have sufficient time to discuss author rebuttals as they will not have to deal with their own papers concurrently. Maybe the first 2 weeks can be the discussion period for half A, and the next two weeks for half B.\n\nI don't think conference organizers have thought of this solution, because if they have, there is no excuse for not trying to implement it because it does not hurt the conference's self-interest in any way.\n\nDoes anyone think this will work? If so, I hope someone of more power than me might ask the conferences to implement it.\n\n\n\n--- Top Comments ---\n\n\n[51 upvotes] Interesting approach to address the direct bias in reviewing of competitive work. However, it does not address the indirect bias: as a reviewer in, let's say, half A, I am still incentivized to rate a competing paper lower to lower its chances to get published during the decision process in half B.\n\nMoreover, I disagree with your premise:\n\n>The biggest issue with the peer review system is reciprocal reviewing,\n\nThat is one the many problem but not the biggest issue in my opinion. From the perspective of an author, bigger problems are, for example, the high variance and low reliability of peer review and the game of \"who-cite-who\". And from the perspective of the conferences probably the biggest problem right now  is how to deal with AI generated papers and reviews.\n\nNevertheless, would be interesting to see your suggestion get implemented as a pilot at a conference or at ACL ARR.\n\n[33 upvotes] I think there is still no publication about this but I personally doubt the problem is really that they want to reject to eliminate the competition.\n\n\nThe problem is that they don't genuinely want to review, so they just say to the llm \"review this paper rejecting it\" just to not have their own papers desk rejected.\n\n\nReciprocal reviews are a cancer, but there is none that can be done now because for some reason many conference organizers now put in their heads it's something that is needed.\n\n[9 upvotes] the clustering by coauthorship is where this gets tricky. ML is a small world and if you trace coauthors of coauthors you end up with massive overlapping clusters that basically make clean halves impossible at top conferences like NeurIPS or ICML\n\n[7 upvotes] how about we just move back to journals like every other field",
  "transcript_chars": 3097,
  "ingested_at": "2026-05-21T19:45:47.206014+00:00",
  "source": "reddit",
  "yt_meta": {
    "score": 63,
    "upvote_ratio": 0.85,
    "num_comments": 23,
    "author": "isentropiccombustor",
    "is_self": true
  }
}