The All Babies Count Initiative: Impact of a Health System Improvement Approach on Neonatal Care and Outcomes in Rwanda

Implementation Successes

We found that the successful implementation of a multilevel intervention combining facility readiness, clinical mentoring, and district-wide improvement collaboratives increased QI capacity, improved quality of care, and was temporally associated with reduced neonatal mortality overall and among preterm/LBW infants—a high-risk subpopulation—in the intervention districts.²⁰ Maternal and neonatal quality of care in low- and middle-income countries has gained attention as increases in service coverage have not been met with anticipated mortality reduction.⁶ Attention is now drawn to health system improvement as a strategy to improve outcomes.¹

We approached neonatal mortality reduction as a district-wide endeavor—a factor that we believe facilitated impact. We aimed to align facility readiness and care provision with national standards and strengthen quality across all facilities acting as a network of care to serve a catchment population. We leveraged the standard improvement collaborative approach—which typically focuses on facilities through a defined learning network¹⁴—to create a district-wide learning system to accelerate improvement.

Although improvement collaboratives have gained popularity based on success in high-income settings,²¹ evidence of impact in low- and middle-income countries has been mixed.^22,23 However, to our knowledge, evidence has not been published regarding collaboratives in low- and middle-income countries adapted to facilitate a district-wide health system approach. We used LSs to convene key stakeholders, from community representatives to district leaders, to facilitate problem solving across traditional hierarchies. MNH quality gaps that require multilevel solutions, such as infrastructure, referral systems, and data quality, were tackled in improvement projects.¹⁶

The ABC initiative, as with all QI efforts, required ownership by leaders to achieve impact. We achieved this through intensive and proactive stakeholder engagement. It included partnering with national and district leadership to determine what they believed would be necessary to achieve high-quality MNH care, ensure district hospital capacity to manage neonatal complications, and create functioning district-wide referral systems. Evidence of leadership engagement was seen through the prioritization of addressing newborn health in a coordinated manner. Both districts’ leaders included neonatal quality of care and mortality into their public performance contracts (imihigo) to which they were formally accountable to the government, and they led efforts to improve the culture of data reporting and use. We found the inclusion of MNH goals in the district leadership imihigo to be a way to help engage leaders throughout the duration of program implementation and to jointly solve challenges encountered. We saw reduced discrepancies in the HMIS of neonatal deaths over time, indicating some degree of success of these efforts.^16,24

Within the collaboratives, we saw broad QI adoption reflected in activated facility teams with QI capability and a high degree of QI activities across facilities. QI is often found to be focused on microlevel facility improvements, resource intensive, and of questionable impact¹; however, we found this district-wide system approach feasible and effective.

Implementation Challenges

Furthermore, the QI work endured despite challenges. Illustrative examples include staff turnover, a data culture of “blame,” and changing population needs. First, staff at health centers moved frequently—as is commonly faced in low- and middle-income countries. Although overall staff number remained stable, changes in individuals required an adaptive process to bring new staff in QI teams up to date. We found the improvement collaborative design, when integrated into routine structures, provided a scaffolding for quickly bringing new staff into facility and district QI efforts and enabling them to be clinically mentored and learn QI methods.

Another challenge encountered early in the program related to leadership support at health centers. Leaders were engaged in stakeholder meetings; however, once QI teams began working to address system problems, some participants reported resistance from their supervisors attributed to lack of familiarity with QI methods. As in many settings, creating a culture of data use requires shifting from norms of blaming individuals, to norms of understanding and improving systems. Therefore, mentors and district supervisors trained district and facility leaders in QI methods and included these leaders in LSs to build their QI capability and data fluency.

Contextual changes in the population and health system also posed a challenge. In Kirehe, given the large population size of some facility catchment areas, 3 new health centers were built during the initiative and had to be incorporated into coaching and LSs. Additionally, the district received a large influx of returning Rwandans from Tanzania in 2013. The improvement collaborative design allowed for natural integration of new facilities into LSs, coaching visits, and QI initiatives, as well as customized support to respond to these changes.

More generally, the multilevel design facilitated system gap identification necessary to provide quality care and the integration of multiple methodologies enabled flexible solutions. District mentors grounded QI coaching and built trust with providers through clinical observation and support.¹⁶ A qualitative study of participating QI teams found that the nontraditional collaborative components—equipment and clinical support and the combined mentorship approach—were factors related to high impact QI initiatives.¹⁶ Many improvement programs focus on narrow interventions, without taking a comprehensive view of the mortality drivers of MNH care quality across a health system.^3,7 Consistent with other research, the Better Birth Trial—a study of the World Health Organization Safe Childbirth Checklist—found that despite improved care practices at individual facilities, there was no impact on mortality, concluding that a greater system improvement focus could be required.^23,25

Variation in Changes Between Districts

We saw district-level variability in the baseline and change associated with the intervention in some quality measures, consistent with prior study in the region.¹⁵ SK had more facilities for the population than Kirehe, with a better developed road system to facilitate access. These factors could have contributed to lower baseline levels in Kirehe for service utilization indicators. We saw rapid change in ANC in Kirehe, which had low documented coverage at baseline and greater room for improvement. However, although Kirehe had a lower mentorship dose due to higher numbers of facilities for the single mentor than in SK, the improvement seen in Kirehe was evidence that ABC could still be associated with meaningful improvement in different contexts.

For immediate skin-to-skin after birth, the baselines were reportedly starkly different between districts. Although this indicator existed in the HMIS before ABC, the register data source was not consistently available at baseline in Kirehe District, which may have contributed to the lower documentation of the practice at the start.

Management of preterm labor with antenatal corticosteroids was a new initiative and began from a low baseline in both district hospitals. Much of the initial work focused on district leadership advocacy to enable dexamethasone procurement by health centers, combined with clinical mentorship to align practice with emerging evidence on safety of steroid administration.²⁶ Therefore, mentors guided clinical providers to provide steroids only when gestational age was known by last menstrual period, which was consistent with national protocol.

Surprisingly, we saw a significant increase in stillbirths in SK in the mixed-effects model. It is possible that despite improvements in target indicators related to improved labor management, stillbirths increased in the face of reduced neonatal deaths if access to quality postnatal care and complications management drove mortality reduction, and quality gaps in antenatal or labor management not captured in our performance measures remained. This possibility is the subject of further investigation.

Despite improved MNH quality of care and neonatal mortality, we found limited change in patient satisfaction. Mutaganzwa et al.¹⁹ described the baseline results in detail and found that patient-centeredness of care (including interpersonal relationships, respect, and privacy) and organizational factors such as cleanliness, comfort, and equipment/commodity availability were associated with higher satisfaction with care. Similar to other studies, one explanation for the lack of increased satisfaction corresponding with improved quality of care and outcomes could be that women were reluctant to share negative views or had low expectations of facility care and were satisfied with poor quality at baseline.²⁷ It is also possible that the intervention did not have adequate focus directly on improving dignity and respect during patient contact care in the intervention, indicating that future programs may need intentional focus on patient-centered care as part of holistic quality to improve satisfaction.

Finally, despite the district-level variation in process level impacts, we found reductions in hospital case fatality and district neonatal mortality in the intervention area. The highest impact was seen among preterm/LBW infants, which has been a difficult area for improvement globally.^7,28,29 The district-wide approach included supporting specialized care at the district hospital, which likely facilitated this finding and perhaps distinguishing the current study from other studies.^7,30 Importantly, Rwanda has experienced health sector improvements over the past decade^8,31 and would be expected to see some reduction in neonatal mortality; however, analysis of available data from the rest of rural Rwanda did not find a similar reduction.

The ABC approach was designed in collaboration with the MOH national and district leadership to enhance scalability and sustainability, which led to the integration of the improvement collaborative components into existing district structures. Based on the results of this program, MOH has continued scaling the approach into non-PIH-supported districts with funding from a Saving Lives at Birth award, with an impact evaluation underway. Furthermore, the program design worked to increase sustainability by building capability of health system actors at multiple levels—district leadership to oversee the approach and integration into routine district systems, and district supervisors to incorporate the mentorship and improvement methodology into their routine work. A sustainability study has been completed to understand system performance 1 year after the end of the intensive period and results will be published separately. As with all health system interventions, integration of effective approaches into routine administrative budgets can be a challenge. Demonstration of the technical impact and program ownership by local leaders were important to support incorporation of the core components (supervisor transport, review meeting costs, clinical and QI continuous learning) into district budgets, and they are ongoing efforts in financially constrained systems.

Limitations

Our study had some limitations. Using HMIS and programmatic data meant that we were limited by the quality of the data available. We accounted for this by register review for all possible measures; however, this step would not address issues of accuracy or availability of paper registers. Facilities needed variable lead time to achieve consistent documentation of newly introduced indicators in preterm delivery management, and such documentation was sometimes incomplete. That said, given the growing global call to measure preterm labor management indicators, we hope this practical experience will have global relevance.³²

In addition, mentors coached QI teams to document change ideas and proximal process data rigorously to determine “success” of a given change. However, we cannot be certain that all successful changes were included in the published change package to fully explain the causal pathway to measured improvement in quality of care and mortality.¹⁶ Some other time-intensive activities were difficult to capture in program documentation. These included documenting the specific focus of clinical mentorship activities at facilities and the work targeting coordination and communication within and across facilities to strengthen complications management.

Importantly, we did not have a comparison area with measurement and register comparison of mortality; therefore, we used the national HMIS data for rural districts. It is also possible that unknown differences were present between the intervention area and the rest of rural Rwanda. The intervention baseline mortality appears starkly higher than the mortality in the comparison districts, which could be easier to reduce. However, sensitivity analyses using HMIS routinely reported data from the intervention area showed a more comparable baseline mortality, and based on 2010 DHS analyses, the neonatal mortality rates in SK and Kirehe were similar to the rest of rural Rwanda.¹² We believe the baseline mortality reported in the comparison districts to be low at least in part due to consistent underreporting in the HMIS, which was overcome in the intervention districts by extracting data from the register records.

Furthermore, improving data quality was part of the QI process, so the difference in differences results could be confounded by change in data quality over time.³³ Routine review of HMIS mortality data compared with registers demonstrated underreporting of poor outcomes at baseline, which decreased throughout the intervention period. If a similar improvement in data quality occurred in the comparison area, it could have reduced the effect size. However, to our knowledge, no concurrent major data quality initiatives were occurring elsewhere in the country.

Mentors triangulated data across registers to use the highest quality data possible for QI and to improve the HMIS data quality. For example, birth weight and gestational age were recorded in the birth register. Mentors compared these data with the counts listed in the HMIS report tally for the same data elements to provide feedback on HMIS data quality to the facility data officer and to build awareness and data quality capability among facility staff in the process. HMIS data QI would not influence the reported quality measures; however, if reporting in the paper registers improved, it would likely lead to underestimation of effect size because it would bias results towards the null in most cases.

Finally, with the introduction of 3 new maternity units in Kirehe District, it is possible that the opening of the 3 new health centers has allowed a decrease of number of patients seeking care in other HCs, which could have contributed to improve the quality of care in the facilities surrounding them by allowing more time per patient; however, we did not track the differences in patient flow.