Trial registration

Registration of a study creates a record about the trial in a public trial registry. Registry entries can serve several purposes.

First and foremost, a registry facilitates the discovery of published and unpublished research. Though trial registration is still relatively new in the social sciences, there is potential in the future for these registries to tell us more about the true extent of publication bias, which is bias due to papers with significant results being more likely to be published (see e.g., Rosenthal 1979 or Andrews & Kasy 2019).

If widely used, registries can be used to track all studies in a given field, which can facilitate comprehensive meta-analysis. They can give an overview of the body of work on a given topic, so that others can better understand gaps in the literature or future research needs. A comprehensive registry also allows the universe of existing studies in a given discipline to be more visible and searchable.

When the trial is registered before the trial intervention begins, we speak of pre-registration. Several registries have badges or icons to distinguish pre-registered trials from other trials. Pre-registration carries the additional benefit of creating a record of researchers' initial intentions (see also the section on Pre-analysis plans). By allowing researchers to declare their study design and outcomes, trial (pre-)registration can thus provide transparency on researchers' plans and intended analysis methods. 

As such, a trial is ideally registered before it begins, though it may be registered at any point in time (as a practical matter, many pieces of information can be copied over from the IRB application into the registry, so it may be convenient for researchers to register right after receiving IRB approval). If necessary, a trial registration can typically be amended after it is submitted. A trial registry entry can, but does not have to, include a separate pre-analysis plan document. More on pre-registration and pre-analysis plans is covered below. 

This resource describes common funder and journal requirements, considerations to make when registering a study, and the process for registering a trial in the AEA RCT Registry.

There are a number of registries where researchers can register studies in the social sciences. The different social science registries have varying eligibility requirements. For example, the AEA RCT Registry permits experiments conducted anywhere in the world, while 3ie's Registry of International Development Impact Evaluations (RIDIE) allows all types of impact evaluation methods (regression discontinuity, natural experiment, etc.) but is restricted to studies related to development in low and middle income countries. 

• American Economic Association (AEA) Randomized Controlled Trial Registry: This trial registry is probably the relevant one for most economists. It is for any studies that involve an RCT, but imposes no restrictions on study location or topic matter.
• Experiments in Governance and Politics (EGAP): The EGAP registry includes experiments and observational studies in governance and politics.
• The Registry of Efficacy and Effectiveness Studies (REES): This registry is run by the Society for Research in Educational Effectiveness and covers causal inference studies (randomized trials, quasi-experimental designs, regression discontinuity designs, and single case designs) in education and related fields.
• The Registry of International Development Impact Evaluations (RIDIE): Run by 3ie, this registry is for experimental and observational studies assessing the impact of development programs in low and middle income countries.
• Open Science Framework (OSF): The registry of the Center for Open Science offers various registration form types, enabling researchers to register their study according to discipline norms, study design, or stage in research. 

Though not widely used by economists or political scientists, As Predicted is another registry, primarily used in psychology. Their website provides a template tool that allows people to register a simple outline of their study design by answering a standard set of questions, and then prints a time-stamped PDF of researcher responses. 

Clinical Registries

IMPORTANT: The AEA RCT Registry is not an approved clinical trial registry. Health and medical journals that follow the International Committee of Medical Journal Editors (ICMJE) recommendations require pre-registration in an approved clinical trial registry (e.g., Clinicaltrials.gov, or another government-sponsored clinical trial registry). The registration process may take several days to a few weeks. These journals include Science, The British Medical Journal Open, The Lancet, The Journal of the American Medical Association, the Journal of Health Economics, New England Journal of Medicine, etc.  The full list of approved clinical trial registries for ICMJE journals can be found at the International Clinical Trials Registry Platform. This requirement applies even when the research is within the social sciences or another non-biomedical discipline and even when the trial is already registered in another (unapproved) registry.

• The World Health Organization (WHO) registry network is an international network for clinical registries. The WHO registry network designates official clinical trials registries around the world. This includes the US-based ClinicalTrials.gov as well as other national clinical trial registries. Visit the WHO registry network website if you are looking for a clinical trial registry in a particular country. 

In what follows, we provide advice that is specific to the AEA RCT Registry, as well as considerations that apply regardless of the registry used. 

The registration process for the AEA RCT Registry has been designed to be as easy as possible. The mandatory registry fields are: 

• Title, country, keyword(s), abstract
• Current status of the trial, trial and intervention start and end dates
• Primary outcomes, experimental design, randomization method, randomization unit, clustering, sample size (total number, number of clusters, and units per treatment arm)
• IRB approval information. 

Please consult the AEA's Registration Guidelines for a full list of fields and the Field definitions guide for other information about registering. The submission to the AEA registry is time-stamped, as are any subsequent modifications.

The researchers can choose to hide some fields until the project is completed. There is also opportunity to include optional supporting materials such as survey instruments, proposal(s), etc. There is also a place to upload a separate Pre-analysis plan document, which may remain hidden until the trial is complete.

The completed study Health Care Hotspotting: A Randomized Controlled Trial (Doyle, Finkelstein, Taubman, and Zhou, 2014) is a good example of a registration that other researchers can follow.

Pre-registration and pre-analysis plans

Pre-registration on the AEA RCT Registry refers to the act of registering the trial prior to the intervention start date. The AEA RCT registry has fields to describe many key parameters of the trial and additionally allows the upload of a pre-analysis plan (PAP) document that can remain hidden until the trial is complete. 

A PAP is simply a plan that is filed publicly, typically before the intervention starts but at the latest before data analysis begins, that describes how the researchers plan to conduct the study and analyze the resulting data. In many disciplines, such as psychology¹, the term pre-registration is synonymous with filing a PAP. 

In the same manner, filling the form fields on the AEA RCT registry and pre-registering the trial allows researchers to create a public record of key information about their planned trial, including the study objective, outcomes of interest, and sample size. As Banerjee et al. (2020) point out, the information in the registry fields may be sufficient for recording researchers' initial intentions for the study, although they also have the option to file a separate document. The section on pre-analysis plans describes situations in which pre-specifying additional information about the intended analysis approach may be beneficial.

All members of the research community can access AEA trial registry information, although researchers may choose to keep specific fields private until the study has been completed.

Updating registry entries

Field research might encounter unforeseen circumstances that lead to changes to initially planned research protocols. Researchers should update their trial registration to record changes to key trial details, such as sample sizes or outcomes. A change log keeps track of  modifications to the initially specified research plan. Before making any edits, please be sure to review the version control system of the registry and follow the instructions for clearly documenting changes. 

Changes that concern the research design or analysis approach made to a trial pre-registration are best done before the analysis stage, but modifications can be made at any point in time. Multiple amendments and edits can make it difficult to understand what was registered. Banerjee et al. (2020) therefore recommend that researchers limit themselves to one or two milestones at which the registration is edited. 

All researchers should return to the trial registration when their study is complete and fill in post-trial information about the completed study, such as the study status, completion dates, and some basic study details, including a link to any academic papers and linking or uploading related reports and documents, such as a "populated PAP" (Banerjee et al. 2020). Filling post-trial information is very important for maintaining a record of the complete body of work on a given topic, especially and including any unpublished trials. 

Content and level of detail

In general, trial (pre-)registration should make clear what the researchers' original intentions were with respect to the design and analysis of the field experiment. This is especially important for variables or decisions that require a lot of discretion in their construction and are critical to the analysis, such as primary outcomes or subgroups for heterogeneity analysis. For these variables, the description should be sufficiently detailed that it is clear to others how they will be measured or constructed. However, the registry entry should also be concise and specific and need not describe all secondary outcomes or conditional analyses. Researchers can also admit uncertainty when registering studies and modify registrations when changes arise. 

Initial review process: The AEA Registry focuses on randomized evaluations in social sciences; therefore studies must meet the eligibility criteria to register (the criteria are not intended to reflect the quality or the ethics of the research). Researchers should review the criteria and thoroughly inspect their entry before submitting their trial registration. 

Most registries are reviewed and posted within 1-2 business days of submission. Registered draft trials are stamped with their original submission date and time. Once the registration is approved, an auto-generated citation will appear on the public registered trial. 

Citing the registration: The registration entry can be cited with its DOI, for instance in the academic paper reporting the trial results. For example, a footnote might say “This study is registered in the AEA RCT Registry under ID number doi.org/10.1257/rct.2549.” All versions of AEA RCT Registry entries have DOIs which unambiguously identify them; see the AEA Registry FAQ for more information. 

Creating an account: To register a trial, first create an account at https://www.socialscienceregistry.org/users/sign_up, filling in the required information (name, organization name, etc.). Anyone can create an account from any email address. Follow the automated prompt to confirm your e-mail. Once confirmed, you are able to login and start drafting trial registrations. 

Registering a trial on behalf of someone else: If you are not the “Lead” Investigator or Primary Investigator (PI), but are asked to register on behalf of one:

1. Log in from your own account
2. Create a trial from the main page
3. Fill in fields with as much information as you wish to include.
4. When you are ready for the PI to review the trial, you will need to add them as a collaborator. From the draft trial main page, there is an orange button in the upper right hand corner labeled "Manage Collaborators." This gives you the option to add people. You will need to use the e-mail address that they used to create an account. Note that a collaborator must have an account in the registry system before being added to the trial.
5. Search the “Add Collaborator” field by the collaborator's e-mail. Once you have added it and saved the draft, a drop-down menu with the option to change the primary PI will appear. Change the PI from yourself to the designated project PI. The Trial Information page will automatically update to show the newly assigned PI.
6. Submitting the trial for review/publishing: the primary PI should review the draft registration for accuracy and completeness, and then click the "Register Trial" button to complete the registration. 

Note: if you follow this procedure to register on behalf of someone else, your name will not be visible anywhere on the public version of the trial. If you would like to be included as a PI in the registry entry, you need to add the former primary PI manually as an "Other PI". 

Downloading data from the AEA RCT Registry: The registry database is available for download as a comma-separated values (CSV) file that compiles all current public information from each registered trial. There are two options for downloading the data:

1. Downloading historic monthly snapshots of the database from the AEA RCT Registry Dataverse
2. Downloading a current snapshot of the database directly from the website.

Instructions can be found on the Trial Data Access page.

Pre-registration versus registration: Projects that register before the start of their intervention receive an orange clock icon on the site indicating that they have registered before the trial start date. (Psychology rewards researchers with pre-registration badges.)

Insights from the AEA RCT Registry data: AEA RCT Registry data is available for the analysis of existing and emerging trends in the experimental research literature as well as of the underlying researcher population. For instance, a World Bank Development Impact guest blog by J-PAL staff explores how the COVID-19 pandemic changed the research pipeline and initiation of new research projects by using registration and pre-registration data from the AEA RCT Registry. This is one example of what can be done with this data, and many other avenues for research and analysis remain open. The AEA RCT Registry metadata can be accessed by following these instructions. 

J-PAL’s requirements: All projects funded or implemented by J-PAL must be registered, and pre-registration is recommended (i.e., registration before the intervention begins). Specifically, J-PAL’s registration requirement applies to all randomized evaluations that: 

• Are not pilots
• Have received or will receive funding from a J-PAL research initiative, or are implemented/run through a J-PAL office

Research staff may be assigned to initiate registration and fill in the study information. However, the Principal Investigator (PI) must create an account and approve the registration.

Donor and journal requirements: Donors and journals are increasingly mandating study registration. 

• Six AEA journals (American Economic Review, AER: Insights, AEJ: Applied Economics, AEJ: Economic Policy, AEJ: Macroeconomics, AEJ: Microeconomics): as of 2018, all publication submissions are required to register field experiments in the AEA RCT Registry. Registration can be done at any time prior to submission of the study for publication. 
• The National Bureau of Economic Research (NBER) encourages registration. They provide a reminder about the registry when RCT-based studies are submitted for review by their Institutional Review Board, and also when RCT-based working papers are submitted through their site (if they haven’t been registered already). 
• Individual Initiatives at J-PAL North America require registration in the Open Science Framework. 
• Arnold Ventures (formerly the Laura and John Arnold Foundation) requires any empirical study that involves statistical inference to be pre-registered before the start of intervention or data collection. Studies must be registered on the Open Science Framework, but can also be registered elsewhere (e.g., the AEA RCT Registry). Arnold provides a guide for what information should be registered based on the type of study. 
• The Global Innovation Fund (GIF) requires pre-registration (i.e., before the intervention begins) of all impact evaluations either on the AEA RCT Registry or the 3ie registry (which includes both experimental and non-experimental studies). On a case-by-case basis, other registries (for instance, EGAP’s) may be used to reach the right audience. Clinical trials should follow standards associated with leading journals in the appropriate field. Pre-registration should generally occur before the intervention begins. It may precede or follow GIF funding decisions.
• The Innovation Growth Lab (IGL) requires registration for all studies that they fund.

The Center for Open Science has a helpful wiki page of funding agencies policies, including language about data, materials, or code sharing, reporting guidelines, pre-analysis plans (pre-registration), and replication.

In addition to requiring trial registration for studies J-PAL funds and implements, J-PAL's policies state that all other randomized experiments run by J-PAL affiliates should be ideally registered before the intervention begins. However, if that cutoff is missed, a project should still be registered as soon as possible after that. 

We provide our network of affiliated professors assistance in registering their projects on the AEA RCT Registry. This assistance includes creating draft registrations based on information in a research proposal, applications to Institutional Review Boards (IRB), narrative reports, etc. In order to increase the comprehensiveness of the registry, we also retroactively register completed projects of J-PAL affiliates. 

To learn more about J-PAL's work to increase transparency in research, please visit our website.

Last updated July 2023.

These resources are a collaborative effort. If you notice a bug or have a suggestion for additional content, please fill out this form.