Neha Sharma, J-PAL ‘12, on evaluating the impact of climate investments

The Alumni Spotlight series highlights J-PAL alumni who are making an impact across industries and around the world. Formerly a policy and training associate at J-PAL South Asia, Neha Sharma ‘12 now leads the Evaluation and Learning Unit at the Climate Investment Funds, hosted by the World Bank. She describes her journey from J-PAL to the World Bank in this post in the Alumni Voices series.

What inspired you to build a career in the development and public policy space and how did you first come across J-PAL?

My study of economics and long stints with nonprofits in India reinforced the need for good policies to make progress on development goals. I was reading and witnessing that the things that I was most curious about and cared about could be answered through development and policy, so a career in this field felt inevitable. 

I discovered J-PAL in 2011 while I was searching for an organization that truly cared about impact. At the time, I was following many different government initiatives, particularly on infrastructure and urban development, and was relying quite heavily on monitoring data, which was often inaccurate or incomplete. One day, somewhat out of frustration, I typed into the search bar, “organizations that care about impact,” and J-PAL popped up! It led me down the rabbit hole of some excellent J-PAL papers and I ended up ordering Poor Economics, which had just been published. Soon after, I dropped my CV into the general application pool for J-PAL and heard back from the amazing Shobhini Mukerji, J-PAL South Asia’s Executive Director. 

Policy & Training staff at J-PAL inherently wear many hats. What aspect of the job did you find most challenging and/or interesting?

I enjoyed working closely with clients and testing innovative approaches to monitoring and evaluation (M&E). I had a chance to do this in Kerala and Haryana—two very different capacity development projects but both, I found, at the cutting edge of what was happening in M&E in India. In Kerala, we were building a bottom-up monitoring tool to understand gender-based violence. In Haryana, we were establishing an innovative research center in the Department of Education. One of the greatest challenges in this, and capacity development work in general, was speaking the clients’ language versus enforcing the more standard M&E vocabulary. Evaluation can be rather technical, so simplifying messages without losing the rigor and accuracy was a fine line to walk.

In my experience, evaluation is a normative, value-driven field. It’s a powerful type of research that is multi-disciplinary and requires an understanding of people’s motivations, goals, incentives, in addition to the rigor of tracking what actually happened. The opportunities I had to understand these motivations and work across sectors during my time at J-PAL South Asia were the most interesting to me.  

You now lead evaluations at the World Bank for the Climate Investment Funds. What is that like and how has your work at J-PAL influenced what you do today? 

I get to apply what I learned at J-PAL and at past academic and professional experiences on a daily basis at the Climate Investment Funds. Climate is going to be the defining challenge for humanity in this century and we have a very small window of opportunity to make an impact, so I am deeply motivated to try and contribute to finding solutions. I believe that evaluations can help answer the pressing questions that need to be tackled to make sure we maximize impact and understand both the benefits and losses resulting from different interventions. 

At the same time, I think the toolbox needed to understand something as complex as climate needs to incorporate many evaluation approaches and methods, not just RCTs. When we are examining complex systems that are dynamic and nonlinear, that are capable of being self-sustaining and self-organizing, there is no one, direct path to answers. We need multiple tools to evaluate climate policies and track impacts at a large scale, in a sustainable way, and at the urgent pace required. In particular, bringing a justice lens to climate action, both for people and the planet, adds a layer of complexity, because you have to look at interconnected systems, power imbalances, and deep political economy challenges across very long-time horizons. J-PAL instilled in me both an openness to new methods and an analytical rigor that I believe has been strengthened over time. 

Since you’ve mentioned other ways to evaluate policies besides RCTs, could you walk us through an example of an evaluation you’ve worked on?

We are about to release an evaluation of the development impacts of climate finance, which is essentially looking at how countries can achieve the dual goals of development and climate. To do this, we first had to think about all the potential pathways for change. For example, if you have a renewable energy plant leading to reduction in fossil fuel emissions, you also may have local economic development, job creation, and positive health impacts. These can subsequently lead to gender impacts. How does one map out all the potential impact pathways from climate finance, and then focus on a few that will have the highest impact? And, how does one do this when these impacts were not identified upfront and/or monitored during implementation?  

With that in mind, we first determined potential impact pathways and then looked at various modeling approaches that could be reliably used for those impacts, say health impacts from emission reduction. We examined in detail what data is needed to run these models, what were the caveats and limitations to them, and in which context they could be used. Lastly, we conducted case studies looking at projects in specific contexts to understand intended and unintended impacts. For these case studies, we used a realist evaluation approach to look at what works, for whom, how, and in which circumstances. 

Ultimately, we ended up with a theory-based model that paired impact pathways with quantitative modeling, which included computational general equilibrium models and input and output models, along with a qualitative case study approach. 

What skills or experiences that you gained from working at J-PAL have you found to be useful in your own career?

I think one should be open to changing one’s mind, which can be a difficult lesson to learn. Allowing new information and experiences to change the way one previously thought, to me, is a sign of growth. At J-PAL, through the many, very talented people I met, I was often forced to reconsider my perspectives and assumptions. I learned so much from the people and their drive to make a positive impact. Reflecting upon this now, I feel inspired yet again!