Context

• SIGCSE symposia and conferences follow approach for many conferences and journals
  • Want sessions interesting, stimulating, high quality
  • Depend on contributed papers for various technical sessions
  • Selection depends on a viewing process

Natural Questions Regarding Ratings by Reviewers

1. To what extent do some reviewers give consistently high or consistently low ratings?
2. Are ratings affected by such factors as referee gender, nationality, or expertise, or by the paper's format?
3. Do authors of one paper give consistently high or low ratings to other papers?
4. How does an overall rating correlate with subscores in areas of technical content, writing quality, originality, and significance?
5. How does a joint review by a large committee working collaboratively affect the ratings given by that committee?

Some Hoped-for Insights

• Original hopes for conference leadership:
  • Factors to consider in assigning reviewers
  • How to interpret ratings
• Possible hopes for authors:
  • Clarify fairness of the process
  • Identify what factors contribute to making papers "good"

created September 2, 2003 last revised October 22, 2003

previous   next