This project is a collection of code and data to experiment with different ways in which language models can be used in the reviewing process. The experiments are organized into three main categories:
- Review Generation: Given a paper, generate a review.
- Review Quality Assessment: Given a review, assess its quality.
- Discussion Facilitation: Given a paper, a set of reviews, author responses, and (optionally) previous discussion threads, summarize the consensus among reviewers, the points of disagreement, and either make a recommendation for acceptance or rejection or suggest points to be discussed to reach a consensus.
All requirements are listed in requirements.txt. To install them, run:
pip install -r requirements.txt- PDF Text to Review: Given a PDF of a paper, convert it to text and generate a review using a single prompt with the GPT4o model.
- PDF Text to Review, Step-by-Step: Same as above, but using prescribed steps that ask specific questions (see
review_generation/1_pdf_text_step_review.pyfor details). - O1 Review: Same as 0, but using the O1 model.