The Costs and Cost-Effectiveness of a District-Strengthening Strategy to Mitigate the 3 Delays to Quality Maternal Health Care: Results From Uganda and Zambia

Study Design

We calculated the costs per maternal death averted and life-year gained by combining data on intervention costs that we compiled with direct outcome evaluation data from studies that previously documented maternal and newborn mortality associated with the SMGL approach. Health impact data in the SMGL-supported districts were collected in a separate evaluation of SMGL.^30,31 The impact evaluation used a before-after design comparing selected health indicators and outcomes in 2012 (baseline) and 2016 (endline). These evaluations, including the data sources and the impact results, are described elsewhere in this special supplement.³⁰

The estimated costs of MNH interventions were assessed in a subset of the districts where the SMGL approach was implemented and compared to estimated costs in 2012, prior to SMGL interventions. Since costs were not directly collected in the SMGL-supported districts prior to SMGL implementation in 2012, we derived comparison costs from the 2016 unit costs (e.g., cost per antenatal care visit, cost per vaginal delivery, cost per cesarean delivery) in neighboring districts where MNH programs were chiefly supported by country government efforts alone, to be consistent with the time frame for the effectiveness evaluation (end year as 2016). Table 2 lists the variables used in the cost-effectiveness analysis, along with the sources of data.

Because cost data prior to SMGL implementation were not available, we assumed that SMGL would affect the unit cost of health services through both the scale (quantity utilized) and quality of services.³² Thus, we selected comparison districts in the costing analysis for proxy measures of costs before SMGL, assuming that the unit prices of health services in these districts were similar to those in the SMGL-supported districts prior to interventions. We also assumed that unit costs in the comparison districts did not change substantially during the 2012–2016 time period, and we conducted sensitivity analyses to explore this assumption. We used districts as the unit of analysis because the SMGL approach was implemented at the district level, and many of the costs were incurred at the district level and could not be easily attributed to specific health facilities. This study assessed costs associated with provision of MNH care retrospectively in the SMGL intervention districts for the year 2016, including annualized start-up costs and capital costs over the 2012–2016 period. Start-up costs are defined as the costs for activities needed to establish interventions that are not incurred on an annual basis, while capital costs include the purchase of durable goods that are used over multiple years. Thus, we assumed effects are not cumulative across years except to the extent that continued capacity building, which is captured in start-up costs, allowed for increasing the effectiveness of the SMGL approach over time. We also assessed unit costs in comparison districts for the year 2016. We then used these unit costs together with 2012 utilization data from SMGL intervention districts to estimate costs in 2012 in the SMGL-supported districts (before SMGL started).

Selection of Study Areas

Planned data collection included 2 of the 4 learning districts in each country. In Zambia, we randomly selected Mansa and Nyimba from the 4 SMGL-supported districts for inclusion in these analyses. Mansa became 2 separate districts (Mansa and Chembe) in 2012, and data were collected from both. We also selected 2 districts, Kapiri Mposhi and Mbala, for comparative purposes for the costing analyses. The intent was to conduct the costing study in districts used in an external evaluation of SMGL at the end of Phase 1, where 2 comparison districts were selected to be similar to SMGL-supported districts across a number of factors (including health infrastructure, geography and climate, health utilization, morbidity and mortality, and socioeconomic context) that would also likely influence costs.²⁹ However, one of the comparison districts (Kabwe) later received extensive donor support for MNH programs and was excluded. We decided to select a second comparison district from the Northern Province. We randomly selected Mbala district after excluding districts participating in the World Bank's results-based financing project in the province.

In Uganda, we purposively included the greater Kibaale district (now existing as 3 districts; data were collected from all 3 districts) in the study because it was the only SMGL district to receive extensive support from one of the 2 main implementing partners. From the other 3 SMGL-supported districts, we selected Kyenjojo as the remaining rural district with a district hospital. Both districts received similar SMGL-supported interventions, with the exception of transport vouchers, which were implemented in Kyenjojo only.³³ We also included the Fort Portal Referral Hospital, which received referrals from both Kibaale and Kyenjojo (and is located in a third SMGL district). For comparison purposes, we selected Masindi district, which served as a comparison district in an early evaluation of SMGL because it is located in the Western region and has a population and health system similar to the SMGL-supported districts. Due to limited time and budget, only one comparison district was included in Uganda.

In each of the selected districts of Uganda and Zambia, we collected data from the district health office, the government hospital in that district, and 2 randomly selected government health centers. Overall, we collected data from 5 districts, 4 hospitals, and 6 health centers in Uganda (3 health centers level III and 3 health centers level IV—one of each type in each of 2 SMGL-supported districts and in the comparison district), and 5 districts, 4 hospitals, and 8 health centers in Zambia. Data relating to MNH care activities in these districts were collected from district and provincial health offices (where appropriate) at the national level and from implementing partners, including 3 implementing partners in Uganda and over 30 in Zambia involved in MNH care, in the districts included in this analysis.

Ethical Approval

The data collection specific to this study was exempted from the need for ethical approval by Abt Associates Institutional Review Board and from the University of Zambia Biomedical Research Ethics Committee in July 2017 because it did not include research on human subjects. The study received approval from the Makerere University of Public Health Higher Degrees, Research, and Ethics Committee in January 2018 and the Uganda National Council for Science and Technology (approval number SS 4511) in February 2018.

Data Collection

Data collection at health facilities occurred in July–August 2017 in Zambia and February–March 2018 in Uganda. Trained data collection teams extracted information on health facility area (square meters), staffing, service utilization, vehicles, and consumption of commodities from these facilities. Data were entered into Microsoft Excel templates designed for the study and were reviewed daily by data collection supervisors and again by the research team. Questions were sent to data collectors to verify information, and facilities were contacted again to clarify ambiguous information as needed. Similarly, structured templates were used to capture data at district health offices and, where appropriate, provincial health offices related to overall district health statistics (e.g., number of deliveries, number of health facilities) and activities related to MNH (e.g., training, health systems strengthening, mentoring, supervision, community outreach) during 2012–2016 for annualized start-up costs.

Data collection templates were constructed based on past analyses of expenditures in SMGL areas^34,35 and sent to implementing partners. Data collectors then visited these partners to provide support for extracting the necessary data. Implementing partners provided data on all relevant start-up activity costs, capital expenditures, and routine activities for 2012–2016. Costs for national-level activities were not included unless the activity specifically focused on one of the districts included in these analyses; thus, for example, these analyses did not include costs for international staff and national staff working on multiple projects in addition to SMGL or costs for offices outside the SMGL districts.

Data were collected for all activities supporting MNH, whether or not they were “officially” part of SMGL. However, some activities were not assessed as part of the SMGL evaluation, including HIV/AIDS care or prevention for pregnant women and postpartum family planning outside the MNH clinics of health facilities, unless the SMGL program specifically included them.

Data Analysis Methods

We used a 1-year analytic horizon for deaths averted (i.e., the difference between deaths occurring in 2012 and in 2016) and a lifetime analytic horizon for life-years gained. We included costs from the health system perspective—that is, costs incurred by the health system and implementing partners. Costs incurred by patients and volunteers' time were not captured or included in the analysis. Capital items and start-up costs were converted into annual equivalent costs using standard formulas; we assumed a 3% discount rate³⁶ but explored locally published discount rates in sensitivity analyses.^37,38 All costs were inflated to 2016 based on local GDP inflators and are presented in US dollars.³⁹

We used a financial approach to estimate the costs of activities presented in Table 1. Costs of activities targeting communities included the support of volunteer community groups (Village Health Teams in Uganda and Safe Motherhood Action Groups in Zambia), including costs to train, equip, and organize group meetings.

We employed a mix of top-down and bottom-up costing methods to estimate health facility costs.⁴⁰ Costs of administrative and support services (e.g., cleaning, maintenance) were allocated to maternal, newborn, and antenatal wards or clinics based on number of staff, size, number of prescriptions, or service utilization in a top-down manner. Costs for ambulances were allocated to MNH services based on a review of ambulance logs. Whenever possible, quantities of consumables used directly in provision of maternal and newborn care were estimated from existing registers and stock cards specifying the amounts issued to a ward or clinic. If these data were not available, we relied on either allocation based on utilization (for general drugs and supplies) or health facility staff opinion (for drugs used specifically for maternal health). Quantities of consumables were multiplied by their unit prices, which were collected at the national level. Staff costs, inclusive of salary and benefits, were allocated to MNH services based on assigned duty stations, opening hours, work patterns, and service utilization. In Uganda, costs for utilities and building costs in public facilities were estimated based on implementing partners' accounts of costs for similar items; in Zambia, costs for utilities and buildings were estimated from previous costing exercises (R Homan, FHI 360, written communication, January 2018). Almost 40% of delivering facilities in Uganda SMGL-supported districts and 9% in Zambia are private.³⁰ Costs for maternal and newborn services at nongovernmental health facilities in Uganda were based on a previous study carried out in the same districts.⁴¹ Costs incurred at health facilities and reported by implementing partners were cross-checked to ensure that items were not double counted.

Total costs for MNH services for entire districts were estimated using the average unit costs from sampled facilities for different types of services (e.g., antenatal care, vaginal delivery, cesarean delivery) and multiplying the results by the total utilization of these services in a district. These figures include costs for inpatient admissions. For estimates of costs before the start of SMGL, we used utilization numbers from the SMGL-supported districts in 2012 and unit costs from comparison districts, while for 2016 we used utilization numbers and unit costs from SMGL-supported districts in 2016. We disaggregated these calculations by type of health facility. We then added costs incurred at the community level and “above service delivery costs” (e.g., costs for offices located in districts, general and office support staff, program vehicles, and other general management and planning activities) to the facility-based costs. For SMGL districts, the programmatic costs and facility costs to address the first, second, and third delay total costs were added to derive the total costs for the districts. To convert costs incurred outside health facilities in comparison districts to costs incurred in 2012, we divided these costs by the number of facility deliveries in the comparison districts and then multiplied the results by the number of facility deliveries estimated to have occurred without SMGL in the SMGL districts. This calculation assumes that costs outside facilities varied directly with the number of deliveries at facilities; however, costs outside facilities in the comparison districts were a small proportion of all costs. For the 2012 cost estimates, we included observed community-level and above service delivery level costs from the comparison districts, under the assumption that these activities also likely existed in SMGL-supported districts before the start of SMGL. We calculated costs per facility delivery in 2012 (i.e., baseline facility delivery costs) and in 2016 (defined as the costs of improved facility delivery, which included all costs associated with delivery, demand generation, and transport). Incremental costs were calculated by taking the difference between the estimated total costs in 2016 and those in 2012. Sources of financing (donor, government, and private) were tracked throughout this exercise.

To estimate the health impact, we used the facility-based maternal mortality ratios and perinatal death rates in 2016 in SMGL-supported districts multiplied by the reported number of facility-based deliveries for the cohort of women giving birth in 2016 to determine the number of deaths in 2016.³¹ To estimate the number of deaths that would have occurred in the same districts in the absence of SMGL, we started with the number of deliveries for SMGL districts in 2016 multiplied by the institutional delivery rate in 2012 to estimate the number of facility-based deliveries that would have occurred without SMGL. To account for secular trends in maternal mortality and perinatal deaths, we adjusted the facility-based maternal mortality ratios and perinatal death rates from 2012 by subtracting the change in these indicators at a national level from 2012 to 2016 from the SMGL district-specific 2012 figures (see Supplement 1).^27,28,42,43 To reflect a generalizable cost-effectiveness applicable as broadly as possible, we used the facility-based death ratios/rates from all SMGL areas. However, since we collected costs in only half of the SMGL-supported districts, this approach assumes that costs in the districts included in the costing data collection did not differ substantially from those in the SMGL-supported districts where cost data were not collected. We subsequently explored this assumption in a sensitivity analysis (described below).

The number of deaths averted was estimated by subtracting the number of deaths in 2012 from the number of deaths in 2016. We estimated the incremental cost-effectiveness by dividing the incremental costs by the number of deaths averted in SMGL areas. Additionally, we estimated the potential remaining years of life for pregnant women and newborns at the time of death and calculated the cost per life-year gained using national life-expectancy data.^44,45 For averted perinatal deaths, we calculated the cost per life-year gained assuming that stillborn infants and newborns would have had a full life expectancy.⁴⁶

Sensitivity Analyses

We conducted sensitivity analyses to explore the potential impact of our assumptions on the results. Sensitivity analyses were done by changing input variable amounts and assessing how results were altered. The following scenarios and variables were considered for sensitivity analyses:

1. We used locally published discount rates to calculate annual equivalent costs.

2. We re-estimated the increased number of deliveries at health facilities using data from all 4 SMGL-supported districts rather than the 2 districts included in the costing. Chance variation in the increase in the number of deliveries at health facilities between districts may change the results.

3. We re-estimated the proportion of facility deliveries by cesarean delivery, for the same reason as above and using the same method.

4. We re-estimated incremental costs by considering all donor-supported costs as incremental costs (as opposed to using estimated 2012 costs). Although donor funds may displace some other sources of funding, this provides an upper-end estimation of the incremental costs in the absence of other district data.

5. We re-estimated the cost of preventing a year of lost life considering a discount rate of 3% for future life-years, as suggested by the WHO-CHOICE guidance.³⁶

Applying each of these 5 scenarios, we also calculated a “combined-case” scenario in which all the above scenarios were included at the least favorable value. Finally, we re-ran the analyses using mortality rates/ratios specific to the 2 SMGL-supported districts included in the costing.