Resources for researchers new to randomized evaluations

A project's life cycle can be split into four primary phases: inception, planning, implementation, and wrap-up. Key activities to undertake at each stage are shown in the diagram below. 

These activities are distilled into "minimum must dos" and best practices, specified in our Research Protocols, to ensure ethical conduct of research, data security, and data quality.¹ All projects funded or carried out by J-PAL must adhere to these guidelines.

Funding: If you receive funding from a J-PAL initiative, the J-PAL Global Finance team or regional initiative staff will be your main point of contact for grant deadlines and deliverables such as the initial IRB approval for the project. PI compliance with grant requirements helps J-PAL fulfill its obligations towards its own funders and its host universities.

Field implementation: PIs may work with J-PAL regional offices to carry out field projects. Each office is a research center within a university. All research staff are part of the reporting structure within J-PAL (from Associates to Managers to Associate Directors and Executive Director). The host university’s HR rules apply (for vacation, holidays, salary bands, work time etc.). A project may have one or more RAs and Senior RAs, as well as a Research Manager (RM) (who often splits time between several projects). Offices provide shared resources such as office space, survey tablets, and financial coordinators who help manage project funds. Policy & Training staff can support partnership and capacity building. All project needs should be discussed with the regional office and built into the budget.

J-PAL support for (new) PIs: Ask your regional office about what exactly J-PAL can offer.

• J-PAL hosts regular trainings for new research staff, and PIs can re-use slides, exercises, etc. from these trainings. Many other resources were built over time, such as pre-programmed survey modules and guides, e.g., on data de-identification. Many are available on our website or on our internal Google drive for PIs (see the Information for Affiliates page)
• J-PAL offices have many pre-existing policy and government partnerships, which may give researchers access to administrative data or the possibility to collaborate on impact evaluations. They can connect PIs with local academic researchers as well as specialists such as translators, programmers, videographers, etc.
• Several regional offices will submit IRB applications on the PI’s behalf and handle permits and contracting with local providers or partners, such as data use agreements, sub-awards, or contracting out survey work.
• PIs can draw on advice from experienced research staff (e.g. Associate Director of Research; Director of Research, Education, and Training). 

More information on J-PAL's important policies and services for affiliates can be found on our Information for Affiliates page.

The PIs need to keep sight of some high-priority tasks where issues can jeopardize months of work. Data may be lost, survey permits withdrawn, treatments not rolled out, partners alienated, funding rescinded, etc. In multi-PI teams, one PI can be assigned to sign off on each key task; this division of responsibilities should be shared with the research team. J-PAL research staff can provide support, especially on local compliance, but final responsibility lies with the PIs.

Some simple principles and tips can prevent errors and make sure that projects run smoothly.

Timeline management: A shared project calendar helps to keep track of team meetings and important deadlines, such as survey code freezes, surveyor training, PI field visits, and grant deadlines. Plan buffers for IRB turnaround and partner or government approvals. It’s also useful to include public holidays, trips, or training/onboarding of team members. 

Communication: Plan at least weekly hour-long team calls. RAs should be instructed to carefully document all relevant decisions, problems, etc. as they occur. A “running agenda” attached to the calendar invite, where the latest meeting is added to the top of a shared Google Doc or Dropbox document, serves as a record of discussions and decisions. Agree on only one “permanent” channel (typically email) for important communications such as task delegation, and one instant messaging channel, e.g. Slack. It is useful to set a norm for all team members of replying within, say, two business days. Avoid WhatsApp or text messaging linked to personal phones to retain a project record when staff leave the team.

In-person field visits: Many project tasks can be accomplished remotely, but there are key times to be on the ground. PIs should meet key partners in person. The best way to learn whether the survey flow and individual modules work as expected is to accompany a survey team on pilot visits. The period shortly before survey start, including final pilots and surveyor training, is extremely valuable for catching errors and motivating the research and field team.

Research team capacity: PIs are responsible for hiring, training, and evaluating research team members. Hiring the right person can be the single most significant contribution to the success of a research project. The best way for research teams to learn best practices, such as how to structure field teams, is to observe other projects or learn from experienced research staff. All new research staff should attend the week-long J-PAL/IPA Research Staff Training and receive onboarding from a senior research staff member. Explicitly plan for professional development for new tasks, such as staff attending the surveyor training on another project or watching relevant lectures from one of J-PAL's online courses. 

Partner management: Plan regular project updates and in-depth conversations with implementing partners to convey the importance of randomization and treatment arm adherence and dispel any concerns (ethical, financial, logistical…). It builds trust and engagement when PIs attend key meetings and communicate project milestones directly.

Managing a field team may be one of the most complex management jobs you have had – PIs are often remote and in a different time zone, there are language and connectivity barriers, and projects are often resource constrained; especially during crunch times, stress levels can be high. We can here only give an incomplete overview of some points to consider.

Motivation and priorities: PIs are delegating many key components of the implementation of their research project to junior staff who are often quite junior. Spending some time explaining the logic of the research design and the underlying economic model, as well as the rationale for making changes later, e.g., after piloting, will help the team understand why what they do is important and what elements are key. Going over the priorities list above together will help the team focus. The PI’s attitude towards IRB and compliance informs how much responsibility research staff feel as well. 

Delegation: Staff may not always express when they did not hear or understand a task, or do not see its relevance. For important tasks it may help to let the staff member describe the steps they plan on taking, and how long they will take, back to the PI in their own words. All tasks should have a concrete timeline and a clear deliverable attached to them.

Feedback: The most conscientious staff will most take feedback to heart. PIs should remember to acknowledge when things went well, and think about how to respond if they go wrong. A constructive response focused on addressing the problem can promote learning and avoid future errors. It is useful to set up regular check-ins (once every 1-2 months) where the focus is the team’s working relationship and what might need changing. Some projects have a bi-annual feedback conversation between the RA and their J-PAL supervisor (e.g. a Senior RM). They discuss the PIs feedback with the RA, and then in turn debrief with the PI afterwards. This can be an invaluable mechanism to receive feedback.

Power differential: A joint research project can start a rewarding mentor-mentee relationship between PI and RA. RAs see PIs as role models. However, PIs, especially new ones, are often not aware of the significant power imbalance. RAs may depend on letters of reference or feel intimidated. In this situation, only the PI may feel like they are speaking as equals. Cultural, language, and technological communication barriers can compound the issue. 

Work-life balance: As a result, PIs cannot rely on RAs to tell them when they are overtaxed. Research teams should schedule regular meetings during the RA’s normal work hours, to avoid conflicts with family obligations or compromising their safety. Outside work hours, instant channels such as chat, cell phone, and messaging should be reserved for emergencies; weekend work should be compensated with off days. The PIs should signal their interest in the safety and well-being of field and research staff e.g. during survey work and travel and ensure there is a budget for safe transport.

Last updated April 2023.

These resources are a collaborative effort. If you notice a bug or have a suggestion for additional content, please fill out this form.