Does Your Paper Really Suck?
Key takeaways
- Oded Rechavi, at QED Science, believes that if your paper is not in the top 1% of their QED score then it "sucks".
- These are important questions because scientists are increasingly overwhelmed with the volume of new work posted on preprint servers and published in journals.
- QED recently released a white paper that goes one step further and describes the "QED Score", a single number that is intended to measure a paper's quality.
Oded Rechavi, at QED Science, believes that if your paper is not in the top 1% of their QED score then it "sucks". But what is this QED score and what is its purpose? Does it really measure scientific quality? If a paper is not in the 1% does it really suck?
These are important questions because scientists are increasingly overwhelmed with the volume of new work posted on preprint servers and published in journals. As a result, traditional quality signals used for triaging papers, such as journal, conference venue, and institution, are becoming less reliable. AI further compounds this problem by making it easy to produce plausible scientific writing at scale. Papers are longer, figures are denser, and the existence of a paper is no longer sufficient evidence that it represents substantial scientific work.
In response, companies like QED Science are building AI tools to help scientists identify quality work. QED uses Large Language Models (LLMs) to review scientific papers and provide AI feedback. Many scientists report that the feedback is useful and often resembles comments received during human peer review.