Community-Led Transparency and Accountability for Maternal and Child Health in Indonesia and Tanzania
- Children under five
- Mothers and pregnant women
- Empowerment
- Sexual and reproductive health
- Social service delivery
- Transparency and accountability
- Provider Performance
- Community participation
- Information
While the expansion of public services continues across many low- and middle-income countries, the quality of and access to these services is often low, limiting their potential benefits. In Indonesia and Tanzania, researchers conducted a mixed methods randomized evaluation of a transparency and accountability program designed to improve maternal and newborn health (MNH) outcomes, known as the Transparency for Development (T4D) program. The program was community-led and non-prescriptive, aiming to encourage community participation to address local barriers in access to quality care for pregnant women and infants. While the T4D program drew strong participation, it had no impact on the use or content of MNH services, perceptions of civic efficacy, or participation among recent mothers in communities offered the program. Although participants attempted some of what they planned, they reported tangible changes from their efforts in only one-third of communities.
Policy issue
While the expansion of public services continues across many low- and middle-income countries, the quality of and access to these services is often low, limiting their potential benefits. In recent years, programs that try to increase transparency and accountability have been widely explored for their potential to improve access to and quality of public services. One premise of these programs is that information empowers citizens to improve the responsiveness, accountability, and ultimately the effectiveness of their public services. While a number of studies have evaluated various programs that aim to strengthen the transparency and accountability of public healthcare systems through providing information on provider performance to communities—such as community scorecards in Uganda and Sierra Leone and monitoring absenteeism among providers in India—evidence of their effectiveness remains mixed.
Context of the evaluation
Indonesia and Tanzania, where two joint randomized evaluations of the Transparency for Development (T4D) program were conducted, differ substantially in terms of access to quality healthcare, choice among health service providers, as well as general levels of economic development.
In Indonesia, despite major reductions in poverty in the past decades, access to high quality health care remains low. In 2015, the maternal mortality rate was 192 deaths per 100,000 births. In the provinces of Banten and South Sulawesi, where the evaluation took place in Indonesia, nearly all villages had access to a “pukesmas,” or public health center with birthing services. However, prior to the intervention, only half of women reported giving birth at a facility and only a fifth reported receiving complete antenatal care.
In Tanzania, the maternal mortality rate was 556 deaths per 100,000 live births at the time of the evaluation. Relative to Indonesia, Tanzania’s health care system has with fewer providers, in a lower resource setting, offering less choice among providers for those seeking MNH care. Fewer than two-thirds of villages in the regions of Dodoma and Tanga, where the evaluation took place in Tanzania, had a health facility located in the village. Similar to Indonesia, only half of women reported giving birth at a public health facility, while slightly under a third reported receiving complete antenatal care.
Details of the intervention
Researchers partnered with PATTIRO, the Clinton Health Access Initiative (CHAI), and Results for Development to co-design and evaluate the T4D program, an adaptation of a community scorecard program that aimed to improve community-level maternal and neonatal care. T4D is co-designed with local partner organizations (PATTIRO in Indonesia and the country office of CHAI in Tanzania) to be community-led and non-prescriptive.
Among 400 total communities in Banten and South Sulawesi, Indonesia, and Dodoma and Tanga, Tanzania, researchers randomly assigned half to be offered this program. The other half formed a comparison group where the program was not offered.
To increase transparency in communities where the program was offered, facilitators from PATTIRO (Indonesia) or CHAI (Tanzania) gathered information in each community about obstacles to maternal and neonatal care in communities, including issues of access, demand, and quality, and consolidated this information into a community scorecard. The scorecard included information on health indicators in the community, such as how often women gave birth in health facilities and sought postnatal care services and obstacles that women faced when seeking access to high-quality health services.
In each community, the facilitator then led six community meetings over approximately three months. In the first two meetings, the facilitator shared the scorecard with approximately 15 community members whom they had identified as interested in participating in the meetings and to represent a broad swath of the village, particularly non-elites, including women and men of a range of ages and with varying previous leadership experience.
In subsequent meetings, the facilitator encouraged participants to discuss and formulate plans of activities that they could undertake to address the problems related to maternal and newborn health in their communities, present and discuss these plans with their broader communities, attempt their plans, reflect on their progress and make adjustments, and plan for the sustainability of their progress beyond program completion.
This program was designed not to be prescriptive in terms of the problems that communities chose to focus on or in the action plans that they designed. In addition, the program did not provide outside resources such as financial payments or brokering of relationships to community participants. The program took place over the course of seven months in Indonesia and ten months in Tanzania from 2015-2016, with follow-up data collection occurring a year and a half after the program.
Researchers collected both quantitative and qualitative data, including on the content and use of maternal and newborn health care services as well as child health outcomes. They also measured outcomes related to civic participation and perceptions of civic empowerment among recently pregnant women in the broader community.
Results and policy lessons
While the program resulted in broad, deep, and independent participation within communities and participants tried the action plans they designed, on average the intervention had no significant impact on the outcomes of interest: the quality or use of maternal and newborn health care services, infant health outcomes, and civic participation and perceptions of civic efficacy among recently pregnant women in the broader community.
Participation: Program meetings occurred largely as planned, with an average of 12 participants (out of 15-16 invited) in attendance in Indonesia at the initial meeting and an average of 9 at the final meeting. In Tanzania, the average was 15 at the initial meeting and 12 at the final. The majority of those who participated in the meetings were women, and most participants were consistently engaged in the discussion.
Action plans: Participants planned a large number of actions, an average of seven actions per community in Indonesia and four actions per community in Tanzania. These were designed to alleviate a wide range of issues, in most cases including actions to raise expectant mothers’ awareness of resources and best practices in seeking care, improve access to health facilities, and improve the affordability of care. Interviews suggest that participants attempted many of their planned activities, and researchers estimate that by the final program meeting they completed 36 percent of all their planned actions in Indonesia and 52 percent in Tanzania.
Maternal and newborn health: In both Indonesia and Tanzania, the program on average had no significant impact on utilization of health care, quality of health care, or health outcomes relative to comparison communities. Eighteen months after the program ended, mothers in program communities were no more likely to give birth at a health facility or seek postnatal care, and were no less likely to experience maternal depression than mothers in the comparison group. There were no significant average differences in stunting or the proportion of underweight infants in program communities relative to comparison communities.
Empowerment and civic engagement: Similarly, this program had no impact in either country on civic participation or perceptions of empowerment among recently pregnant women in communities where the program was offered, relative to comparison communities. However, for those who participated directly in program meetings and activities, researchers find evidence that most perceived their involvement as helpful for improving their health care and that the experience maintained or increased their confidence in their ability to improve their communities.
Understanding lack of impact: Given that participation in meetings and planned activities was high, a likely explanation for the lack of significant impacts on average outcomes is that participants in few communities were able to fully make the connection between planning actions to accomplishing tangible improvements that could improve community-level health outcomes. In only one-third of communities, participants and others in their communities saw and later recalled seeing participants’ efforts translate into a tangible improvement, such as an ambulance or a new or improved facility or pharmacy, new staff, or more privacy. Many of the kinds of activities that participants tried were also common in these regions and countries and were also occurring in comparison communities. Additionally, many of the completed activities may have had only indirect or weak links with improved health outcomes. For example, some planned activities were cosmetic changes at the health facility such as planting a garden or cleaning the premises, rather than more systemic changes with greater potential to lead to transformative improvements.
Following preliminary results from the work in Indonesia and Tanzania, the researchers implemented a second phase of the program in Ghana, Malawi, and Sierra Leone, where they adapted the intervention to include 1) the identification of sub-national government champions and 2) meetings between community representatives and the government champions. A qualitative analysis of these pilots (comprised of five communities in each country) found some concrete responses on the part of the government champions to a subset of community actions in Ghana and Sierra Leone. These results suggest the potential of integrating community action with government engagement in social accountability approaches.
Lessons and results from this evaluation informed the work of partner CSOs, in particular PATTIRO, to develop other social accountability programs. Future research into improving well-being or public services through community-led transparency and accountability programs may benefit from exploring the generalizability of these effects to other contexts as well as alterations to this program’s design. Possible improvements could include changing the way participants are selected, the length of the program, or offering more resources, such as facilitating relationships with others in and outside the community or providing material or financial resources of other kinds to support more effective actions.