Reviewer policies are specified via "Setup Reviewer Policies" on the START console. Here are some standard policies for us, and the reasons behind them. The ones marked with * differ from the START defaults. ---------------------------------------------------------------------- *Track members can view the list of submissions in the track (including the short summaries), but papers are restricted to reviewers. Reason: Abstracts are necessary for informed bidding. But don't let everyone see all papers: for reasons of confidentiality, papers should be read only by their 3 assigned reviewers (and the area chair & program chair). Otherwise, if a rejected author is scooped by a later paper by one of our reviewers, the author might suspect (correctly or incorrectly) that the reviewer saw the work and borrowed key ideas (consciously or unconsciously). We protect both authors and reviewers when we reduce the opportunities for this. *Enable Bids Reason: More qualified reviewers; happier reviewers. A submission can only be reviewed by an assigned reviewer. Reason: Confidentiality (see above). Allow members to recruit their external reviewers? No Reason: The way we organize reviewing, each reviewer gets only a few assignments, so it is not necessary for them to invite external reviewers. (They can use a secondary reviewer on a given review.) When accessing somebody else's review, include the reviewer's name. Reason: Promotes discussion among the reviewers of a paper. And one incentive to submit good reviews, rather than shoddy ones, is the knowledge that other reviewers will see what you did! *After somebody completes a review for a submission, he/she can see the other reviews for the submission. Reason: See both reasons above. Publish no summaries at all. Reason: Summary reports are not very useful to reviewers. Scores mean nothing without the comments. So keep it simple. *Only allow reviewers to see [comprehensive] reports for submissions which they reviewed. Reason: Reviewers can already see other reviews of each paper they reviewed, by the policies above. I think this feature just lets them get a combined report for ALL papers they reviewed (useful?). Also, giving them access to the comprehensive report gives them access to the message board. But don't let them see reviews or participate in message boards for other papers. Showing the reviews might disclose some ideas from the paper (see confidentiality concern above). It might also unnecessarily air dirty laundry (especially a concern if authors' identities can be guessed). *Allow message boards: Yes Reason: Although reviewers can discuss by email, the message boards provide a better record of the discussion. *Email forwarding: Email messages to everyone allowed to access forum. [this policy can be set at conference manager level only, for some reason] This policy ensures that reviewers and the area chair (and the program chair?) will actually see new postings on the message boards. (Unfortunately, email replies are not cc'd to START -- a message board does not currently have an address to serve as a gateway for incoming email.) ---------------------------------------------------------------------- An administrative annoyance is that the policies must be specified for every track separately, either by the program chair or by individual area chairs. (Possibly if the program chair sets up conference-wide policies and emails BEFORE creating the tracks, they will be copied to the new tracks?) The same is true for the standard emails to reviewers. However, the message-board-to-email gateway must be turned on at the conference-wide level, not in any specific track.