Table 1 An example table of one of 37 policy/practices from the full guide

What:

Registered Reports (RRs) are “a scientific publishing format that includes an initial round of peer review of the background and methods (study design, measurement, and analysis plan); sufficiently high quality manuscripts are accepted for in-principle acceptance (IPA) at this stage…Following data analyses and write up of results and discussion sections, the stage 2 review assesses whether authors sufficiently followed their study plan and reported deviations from it (and remains indifferent to the results).” [13]

RRs have two main features: 1) peer review before data collection, and 2) acceptance regardless of the results obtained [51]

Why:

The RR format redirects the review's focus toward the proposed research question and methodology, rather than the anticipated results of the study [13]

The RR format eliminates several questionable research practices, such as low statistical power, selective reporting of results, and publication bias, while providing the flexibility to report any unexpected findings (https://www.cos.io/initiatives/registered-reports)

Because RR reviewers evaluate a Stage 1 study proposal and then evaluate the final Stage 2 manuscript, any deviations can be spotted clearly and reported in a more transparent way. Instead, with study preregistration, discrepancies between the preregistration and the final article may be harder to spot [52]

With Registered Reports reviews have added value, as the feedback and suggestions can still be incorporated into the study, rather than addressed afterwards

How:

The Center for Open Science provides resources for editors (see “Resources for Editors” and “FAQ” tabs https://www.cos.io/initiatives/registered-reports). This includes email templates for all key sections, submission templates and journal policy guidelines

Worries:

RRs are not necessary or relevant for my discipline

• RRs can be conducted in any field that follows a research workflow which begins with study planning and design

• RRs are especially helpful in any discipline where publication bias and questionable research practices exist

We work on rapid/fast-paced science for immediate impact – RRs are too slow

• RRs can be achieved in a short time scale. Journals can offer 'rapid response' RRs for time-sensitive projects (for example research in response to the COVID pandemic [53]

• Research that is fast-paced may lead to more errors, and so the RR format ensures that the study design can be reviewed before data collection commences to reduce the likelihood of errors at the design stage

I only want to publish significant results at my journal, because these are the results that will be cited more

• A well designed study should lead to informative results regardless of the outcome

• The increase of “null” results in RRs [54] may be a better representation of the research being conducted as a whole

• RRs are cited equivalently to, or at a slightly higher rate than, ‘traditional’ articles [55]

• RRs are judged (through masked peer-review) as being higher in rigor, quality, and detail, as well as comparable in creativity and importance [56]

RRs create more administrative burden, due to reviewers being required at Stage 1 and Stage 2

• Compared to a ‘traditional’ manuscript, the RR manuscript is split into two stages: Stage 1, which focuses on the Introduction, Methods, and Analysis Plan, and Stage 2, which focuses on the Results and Discussion [57]. The amount of total article to be reviewed for an RR is therefore the same as a traditional manuscript, but of course the burden on reviewers and editors can be larger (especially because of the need to compare the Stage 1 and Stage 2 reports). As with everything else in this guide, editors will have to weigh up the pros and cons of adding additional steps to the review process

It will be harder to find reviewers

• There is no evidence that this is the case, but you can outsource peer review to PCI-RR if you’re worried about this (see next box in full guide)

• Most RR reviewers are more motivated because their feedback can directly impact the work at a crucial time (before data collection)

I might lose the reviewer of a Stage 1 manuscript at Stage 2

• While peer-reviewers are invited to be reviewers for both stages of the RR review process, there may sometimes be difficulties in retaining reviewers for both the Stage 1 and Stage 2 reviews. In this case, a new reviewer should be sought at Stage 2. This is not too different from a revision of a traditional article format except that the Stage 2 reviewer can still refer back to the Stage 1 submission

RRs are not a suitable format for qualitative research

• RRs are not just for quantitative research; they are also suitable also for qualitative studies. For authors, being able to receive feedback and transparently work toward agreed research questions can be valuable. For editors, being able to ensure the quality of the research plan and data/materials sharing can increase the quality of the publications

Authors may not want to use the RR format

• Adding Registered Reports as an option does not require authors to use it, they can simply continue to submit traditional reports if they so choose. It does not require an extra load of resources to implement, so it is not a problem even if it is seldomly used

• Most journals offering the RR format also offer the “‘traditional” publishing track

The process of adding RRs to a journal is complicated and arduous

• Installing RRs has become increasingly easy over time. With around 300 journals now offering them, all major publishers have at least one adopter under their umbrella. In many cases, the format and workflow can be imported very easily between journals and all central resources/templates required are openly available (see “How” section). The increased frequency of adoption means that major publishers are generally familiar with how to implement them in the manuscript systems

Resources:

COS Registered Reports: https://www.cos.io/initiatives/registered-reports

[58]: Registered Reports: A Method to Increase the Credibility of Published Results

[57]: The past, present and future of Registered Reports

[51]: Opening the Door to Registered Reports: Census of Journals Publishing Registered Reports (2013–2020)

[59]: Registered reports for qualitative research

Registered Report Census Database: https://datastudio.google.com/u/0/reporting/95aff8ff-7ec6-4363-bf05-f81f61215bd3/page/shIqB