The main objective of paper review process is not paper selection. Spoiler alert: It is rather providing an objective technical review of a paper from the viewpoint of an expert. The discussion between reviewers should also be primarily about this. The decision making process

should be deferred to the latest stage. Unfortunately, the current practice combines the paper review and paper selection, which leads to all subjective non-constructive reviews where reviewers try to justify their accept/reject decision instead of writing the *review*. In the

Shadow PC of @IEEESSP this year, we completely removed the “decision” field from the review forms, and constantly nudged young reviewers to focus on the paper instead of expressing their subjective views on whether the paper *deserves* to be presented at S&P. My observation is

that some reviewers are doing an outstanding job in discussing each others’ understanding of the paper, and the research results which are reported in the paper. However, we have to nudge many reviewers to get used to this style (and avoid generic comments on novelty, depth, etc

without being specific about their claims). The job of discussion leads is not to see if reviewers agree or not on accept/reject, and write a meta-review which is just compilation of all negative points! It is rather to *challenge* reviewers to be specific, clear, and consistent.

After all the technical discussions, and updated reviews, we open the system for “decisions”, and we discuss paper selection.

I hope many conferences adopt this approach...

@acm_ccs @USENIXSecurity @NDSSSymposium @PET_Symposium @icmlconf @iclr_conf @NeurIPSConf @aistats_conf