http://orcid.org/0000-0002-3417-1049
Figures
Abstract
Strengthening district-level management may be an important lever for improving key public health outcomes in low-income settings; however, previous studies have not established the statistical associations between better management and primary healthcare system performance in such settings. To explore this gap, we conducted a cross-sectional study of 36 rural districts and 226 health centers in Ethiopia, a country which has made ambitious investment in expanding access to primary care over the last decade. We employed quantitative measure of management capacity at both the district health office and health center levels and used multiple regression models, accounting for clustering of health centers within districts, to estimate the statistical association between management capacity and a key performance indicator (KPI) summary score based on antenatal care coverage, contraception use, skilled birth attendance, infant immunization, and availability of essential medications. In districts with above median district management capacity, health center management capacity was strongly associated (p < 0.05) with KPI performance. In districts with below median management capacity, health center management capacity was not associated with KPI performance. Having more staff at the district health office was also associated with better KPI performance (p < 0.05) but only in districts with above median management capacity. The results suggest that district-level management may provide an opportunity for improving health system performance in low-income country settings.
Citation: Fetene N, Canavan ME, Megentta A, Linnander E, Tan AX, Nadew K, et al. (2019) District-level health management and health system performance. PLoS ONE 14(2): e0210624. https://doi.org/10.1371/journal.pone.0210624
Editor: Elizeus Rutebemberwa, Makerere University, UGANDA
Received: July 14, 2017; Accepted: December 28, 2018; Published: February 1, 2019
Copyright: © 2019 Fetene et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data is available in the Dryad Repository at doi:10.5061/dryad.gc974r7.
Funding: This work was supported by Bill & Melinda Gates Foundation.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Developing management and problem-solving capacity at all levels is fundamental to health systems strengthening. Competencies of health management include strategic thinking and problem solving, human resource management, financial management, operations management, performance management and accountability, governance and leadership, political analysis and dialogue, and customer and community assessment and engagement [1]. Effective management is particularly critical in low-income settings where scarce resources must be carefully stewarded and efficiently deployed to meet the substantial health needs of the population. To date, much of the literature on improving health management has been conducted at the level of health centers or hospitals and has largely demonstrated that practical education and mentoring in management can promote significant improvements in the quality and consistency of health service delivery [2–12]. These studies have tracked a variety of health service measures such as waiting times [2, 5], pharmacy stock outs [4, 6], human resource management [6, 7], infection control processes [6], medical record availability [4, 9], and information system implementation and data monitoring [10–12].
Despite the plethora of studies examining facility-level (e.g., health center and hospital) management capacity and health service delivery outcomes, we could find only a handful of studies that assessed the relationship between district-level health management and health system performance. Three case studies [3, 13, 14] used the Challenge Model as part of the Leadership Development Program (LDP) developed by Management Sciences for Health (MSH) in Ghana, Egypt, and Mozambique, respectively. In these case studies, the LDP intervention was linked to improved skilled birth attendance and fewer still births in Ghana [3], improved antenatal and postnatal care visits and reductions in maternal mortality in Egypt [13], and increased percentage of attended births in Mozambique [14]. An additional pre-post interventional study [15] indicated a positive association between the LDP intervention and improved antenatal care visits, skilled birth attendant deliveries, and fully-immunized children in Kenya. Last, using a mixed methods approach, researchers documented increased antenatal care coverage and skilled birth attendant rates attributed in part to increased district management capacity as part of the Ethiopian Millennium Rural Initiative [8].
While these studies are helpful, the literature examining the influence of district-level management on population outcomes remains limited. First, few studies exist. Second, most are case studies, limiting their generalizability. Third, most commonly, the studies use exposure to the LDP model as the indicator of management capacity rather than employing a quantifiable and more precise measure of management capacity. Fourth, the studies lack statistical analysis to establish the significance of the relationships between management capacity and population-based outcomes. Last, none of the studies examines the impact of improved district-level management in conjunction with health center-level management. As a result, the association between management capacity at the district health office level and how it may interact with management capacity at the health center level to produce population-level health outcomes remains largely unknown.
Accordingly, we sought to examine the variation in district level health management capacity and examine its association with health system performance including contraception use, antenatal care, skilled birth attendance, immunization rates, essential drug availability. We also examined the potentially interaction effects of district-level and health center-level management capacity in their association with population-based health outcomes. We chose to pursue this inquiry in Ethiopia where the Ministry of Health has endorsed a national plan to enhance district health office and health center capacity as key to a robust primary care system. Findings from this study can be useful in understanding the potential leverage of improving management capacity, as well as the comparative importance of targeting management capacity building efforts at district health offices, health centers, or both.
Methods
Study design and sample
We conducted a cross-sectional analysis of data collected during the baseline assessment for a 3-year interventional study to improve targeted district-level health outcomes in Ethiopia, the Primary Healthcare Transformative Initiative (PTI). PTI is led by the Federal Ministry of Health with support from Yale University and The Bill & Melinda Gates Foundation to improve management capacity and performance within district health offices. Districts are called woredas in Ethiopia, and are the third level administrative division of the country, following regions and zones. Woreda health offices oversee and coordinate primary care services for catchment areas of approximately 200,000 population, including oversight of 4–5 health centers, 20–30 health extension workers, and, in some cases, a primary hospital. In collaboration with the Federal Ministry of Health and Regional Health Bureaus, 36 woredas from 4 regions were selected as PTI sites (9 in Amhara, 15 in Oromia, 9 in SNNPR, and 3 in Tigray). These woredas include 226 health centers covering a catchment area of more than 5.5 million population. Five health centers (3 in Amhara and 2 in Tigray) were dropped from the statistical analyses due to missing data (97.8% health center inclusion rate). Formal institutional board review approval was deemed unnecessary by the Yale Human Subjects Committee because we collected health center and district-level data in aggregate. No individual-level health information was obtained.
Data collection
Our analysis used data collected as part of the baseline measures for PTI, to examine the association between management capacity at the woreda health office and health center levels and health system performance at the start of PTI. In each woreda and health center, data were collected by PTI management mentors, who were assigned to each woreda and trained to promote data reliability and quality assurance, for the quarter spanning October to December 2015 from the district health office directors and their delegates, including health center directors, health center department heads, and other relevant woreda and health center staff. The data collection process included review of relevant official documents and records, and direct observations of the availability of the required standards or services across the woreda health offices and all health centers within study woredas.
Dependent variables
The dependent variable was a key performance indicator (KPI) summary score calculated from the performance on 5 indicators, which were prioritized by the Federal Ministry of Health and Regional Health Bureaus, a subset of the 18 KPIs prioritized by the Federal Ministry of Health and Regional Health Bureaus as part of the Health Services, Development, and Planning (HSDP) national planning efforts because they were most consistently reported with reliable data quality [16]. The five indicators were: 1) antenatal care coverage (ANC), i.e. the number of women having 4 or more antenatal care visits divided by the number of expected births in the health center catchment area; 2) contraceptive acceptance rate (CAR), i.e., or the number of women reporting the use of modern contraception divided by the estimated number of women in child bearing years who are not pregnant in the health center catchment area; 3) skilled birth attendance rate (SBA), i.e., the number of women who give birth in a health facility divided by expected number of births in the health center catchment area; 4) the percent of 1-year old children who have received all recommended immunizations in the health center catchment arear; and 5) essential drug availability, i.e., the average percentage of 22 essential drugs to be found in stock per month at health centers. Performance on each of the 5 indicators, reported as a percentage from 0–100%, was normally distributed. To create a summary KPI score for each health center, performance on the 5 indicators was averaged to create a KPI summary score that could range from 0–100% [17]. We had no apriori reason to weight the 5 indicators. Whether the five indicators were summed or averaged did not affect the results, and we found averaging to be the most intuitive for policy makers and practitioners.
Independent variables
Our primary independent variable was management capacity, measured both at the woreda health office and health center levels. At the woreda health office level, management capacity was ascertained using adherence to the woreda management standards (WMS), which is a regionally endorsed composite of 26 standards organized in 5 domains: governance and organizational capacity, service delivery, collaboration with other sectors, community engagement, and performance management (S1 and S2 Tables). For each woreda health office, adherence to WMS was calculated as the percent of the 26 standards that were fully met. Management capacity at the health center level was measured using the Ethiopia Health Center Reform Implementation Guidelines (EHCRIG, October 2015, current at time of study) (S3 Table), a nationally endorsed set of 99 standards for health center management in 11 domains: leadership and governance, health post support, patient flow, medical records management, pharmacy services, laboratory services, infection prevention safety, medical equipment management, human resource management, performance quality improvement, and financial management. For each health center, adherence to EHCRIG was measured as the percent of the 99 standards that were met. Last, we included variables representing the number of staff employed within the woreda health office and the regional location (Amhara, Oromia, Tigray, or SNNP).
Statistical analysis
We used standard descriptive statistics to characterize health centers and woredas in terms of region, staffing, management capacity at health centers (measured by EHCRIG) and at woreda health offices (measured by WMS), and the KPI summary score. We estimated the unadjusted association between the KPI summary score and management capacity at the health center and woreda health office, total staffing in the woreda, health center catchment area population, and region. We used generalized linear regression models that accounted for the cluster design of sampling of health centers clustered within woredas within regions. We used survey regression analysis with a cluster statement to ensure that the standard deviation of the regression coefficients was correctly estimated to account for the distribution of health centers within woredas. Because the measures of management capacity at the health center level and at the woreda health office level were strongly correlated (Pearson correlation coefficient: 0.56, p < 0.001), we tested a model with the interaction of these variables and found the interaction to be statistically significant (p = 0.03). As a result, we reported adjusted associations between the KPI summary score and independent variables overall and stratified by above and below median woreda management capacity. All analyses were conducted accounting for the clustering of health centers within a woreda. We employed p < 0.05 as the threshold for statistical significance for all analyses. We used SAS software, version 9.2 (SAS institute, Cary, NC) for statistical analysis. All underlying deidentified data are available from the Dryad repository at doi:10.5061/dryad.gc974r7.
Results
Description of health centers and woredas
Our sample included 221 health centers, representing 97.8% of all health centers approached. A total of 33.5% (n = 74) of the health centers were located in Amhara; 37.6% (n = 83) in Oromia, 22.6% (n = 50) in SNNPR, and 6.3% (n = 14) in Tigray region. Of the 36 woredas, 25.0% (n = 9) were located in Amhara, 41.7% (n = 15) in Oromia, 25.0% (n = 9) in SNNPR, and 8.3% (n = 3) in Tigray. The number of staff in the woreda health office averaged 23.3 with SD of 4.7.
KPI summary scores varied (Table 1), with a mean of 66% and standard deviation (SD) of 21.2 percentage points. The mean across the health centers for ANC coverage was 64% (SD 28.1 percentage points); mean CAR was 58% (SD 30.1 percentage points); the mean SBA rate was 64% (SD 28.2 percentage points), the mean percent of 1-year olds fully immunized was 71% (SD 29.8 percentage points), and the mean percent of essential drugs available was 73 (SD 29.6 percentage points). The health center EHCRIG scores also varied, with mean 41%, median 39%, and SD 15.2. The woreda WMS score also varied across woredas, with mean 43%, median 42%, and SD 15.1 percentage points.
Association between management and key performance outcomes
In unadjusted analysis, health centers with above median management capacity had significantly better performance on the KPI summary score (KPI scores 73% versus 60%, respectively, or 13 percentage point difference; p <0.01) (Table 2). Health centers located in woredas with above median management capacity also had higher KPI summary scores (KPI scores 70% versus 62%, or 8 percentage point difference), but this difference was not statistically significant (p = 0.15). Neither region nor level of staffing in the woreda were significantly associated with KPI performance (p > 0.05).
In adjusted models, the associations between health center management capacity and KPI summary scores differed significantly by woreda management capacity, as shown in stratified analysis (Table 3). Among woredas with above median management capacity (measured by WMS), health center management capacity (measured by EHCRIG) was significantly associated with better KPI performance (p = 0.03) whereas this association was not significant in woredas with low management capacity (p = 0.96). Higher staffing levels were also associated with better KPI performance but only in woredas with above median management capacity (p = 0.01); regional differences were pronounced, particularly among woredas with poor management capacity.
Discussion
In this study, we found that woredas with better management capacity at both the woreda health office and the health center levels achieved significantly better performance in a summary score of key performance indicators including ANC, CAR, SBA, immunization rates, and availability of essential medicines. Importantly, the management capacity at the woreda health office level modified the effect of management capacity on KPI performance at the health center level; we found that having stronger management at the woreda health office magnified the positive effects of strong management at the health center level. We also found that having more staff employed at the woreda health office was associated with better KPI performance but only among those woredas with stronger management capacity. In both cases, this likely because added staffing, coordination, and accountability structures—all fundamental to management capacity—may prompt greater productivity, collaboration, and focus on performance than in woredas with lower management capacity. Previous work in low-income countries has suggested the importance of management on the functioning of health centers and hospitals [2–12], but fewer have demonstrated the link between district-level management and health indicators in the population the district health office serves.
Policymakers have called for greater attention on the district-level as part of health system strengthening, but much of this work has focused on district-level planning and data monitoring. We have argued that management skills–such as strategic problem solving, human resource management, financial management, and operations management–are fundamental to health system strengthening [1, 18]. In contrast to a health facility, which delivers healthcare services, the district level health office focuses on enabling facilities to deliver services through regulatory, monitoring, and oversight functions. Thus, management skills at the woreda health office—including governance, performance data review, collaboration with non-health sectors, community engagement, and performance management—are essential to planning, ensuring resources are available, and holding staff accountable to provide and monitor services. Furthermore, this study demonstrates management capacity can be feasibly measured with quantitative data, providing a potentially powerful lever to target for improvement as part of health system strengthening efforts in low-income settings.
The results of the study should be interpreted in light of its limitations. First, this was a cross-sectional study demonstrating a statistical association between management capacity and primary health system performance. We are unable to establish causality from this observation study; however, these early results suggest that longitudinal studies to examine changes over time are warranted. Second, the sample size is relatively limited and the study was conducted in a single country. Although we were able to detect statistically significant differences in this sample, larger studies with more statistical power as well as greater generalizability would be helpful. Third, we were unable to examine differences in key performance indicators based on the sub-components of management capacity. Future larger studies with greater variation across sub-components may reveal such patterns. Notably, longitudinal evaluation of changes in management practices and performance is across Ethiopia is ongoing and may allow such analysis. Fourth, we had limited data on sociodemographic and economic factors across woredas. Although we knew the region of the woreda, we did not have data in its wealth or education, both of which may be important influences on key performance indicators. Last, the quality of health data in low-income countries has been shown to be limited [19, 20]. To address data quality issues, we used explicit protocols and conducted substantial training and supervision of data collectors. Nevertheless, we anticipate that the misclassification bias arising from errors in data collection are likely non-differential, thus leading to a conservative bias toward the null. Advances underway in Ethiopia in public health data collection and reporting will likely improve the reliability of future such studies.
In summary, this is one of the first studies of which we know to examine the association between a detailed measures of management capacity at the district level and outcomes in the population served by that district. We find that although the management at the health center level is critical, its effect is strengthened by management capacity at the district health office level. The work highlights a key, often neglected leverage point in health system strengthening: management capacity. Policymakers might focus greater attention on the development of management capacity, particularly at the district level; management capacity can be reliably measured and can an additional input with which to achieve better district-level coverage and health facility readiness. The work in Ethiopia provides encouraging evidence supporting the feasibility of measuring and reporting on management capacity and its potential to promote public health outcomes in resource limited settings.
Supporting information
S2 Table. Woreda management standards (detailed).
https://doi.org/10.1371/journal.pone.0210624.s002
(PDF)
S3 Table. Ethiopian Health Center Reform Implementation Guidelines (EHCRIG).
https://doi.org/10.1371/journal.pone.0210624.s003
(PDF)
References
- 1. Bradley EH, Taylor LA, Cuellar CJ. Management Matters: A Leverage Point for Health Systems Strengthening in Global Health. Int J Health Policy Manag. 2015;4(7):411–5. pmid:26188805.
- 2. Atnafu A, Haile Mariam D, Wong R, Awoke T, Wondimeneh Y. Improving Adult ART Clinic Patient Waiting Time by Implementing an Appointment System at Gondar University Teaching Hospital, Northwest Ethiopia. Advances in Public Health. 2015;2015:1–5.
- 3. Kwamie A, van Dijk H, Agyepong IA. Advancing the application of systems thinking in health: realist evaluation of the Leadership Development Programme for district manager decision-making in Ghana. Health Research Policy and Systems. 2014;12(1):29.
- 4. Kebede S, Mantopoulos J, Ramanadhan S, Cherlin E, Gebeyehu M, Lawson R, et al. Educating leaders in hospital management: a pre-post study in Ethiopian hospitals. Glob Public Health. 2012;7(2):164–74. pmid:21259143.
- 5. Wong R, Hathi S, Linnander EL, El Banna A, El Maraghi M, El Din RZ, et al. Building hospital management capacity to improve patient flow for cardiac catheterization at a cardiovascular hospital in Egypt. The Joint Commission Journal on Quality and Patient Safety. 2012;38(4):147–53. pmid:22533126
- 6. Mutale W, Stringer J, Chintu N, Chilengi R, Mwanamwenge MT, Kasese N, et al. Application of balanced scorecard in the evaluation of a complex health system intervention: 12 months post intervention findings from the BHOMA intervention: a cluster randomised trial in Zambia. PLoS One. 2014;9(4):e93977. pmid:24751780
- 7. Edwards LJ, Moises A, Nzaramba M, Cassimo A, Silva L, Mauricio J, et al. Implementation of a health management mentoring program: year-1 evaluation of its impact on health system strengthening in Zambezia Province, Mozambique. Int J Health Policy Manag. 2015;4(6):353–61. pmid:26029894.
- 8. Bradley E, Thompson JW, Byam P, Webster TR, Zerihun A, Alpern R, et al. Access and quality of rural healthcare: Ethiopian Millennium Rural Initiative. International Journal for Quality in Health Care. 2011:mzr013.
- 9. Wong R, Bradley EH. Developing patient registration and medical records management system in Ethiopia. International journal for quality in health care. 2009:mzp026.
- 10. Kmanzi J, Megentta A, Nsabiyumva W, Sendegeya A, Wong R. Improving clinical documentation through monthly audits in Butare teaching hospital, Rwanda. Journal of Service Science and Management. 2015;8(06):86.
- 11. Ibyimana C, Wong R, Adomako E, Lukas S, Birungi F, Munyanshongore C. Establishing a tracking system in human resources department to improve the completeness of personnel files at a district hospital in Rwanda. Journal of Hospital Administration. 2016;5(5).
- 12. Lukas S, Hogan U, Muhirwa V, Davis C, Nyiligira J, Ogbuagu O, et al. Establishment of a hospital-acquired infection surveillance system in a teaching hospital in Rwanda. International Journal of Infection Control. 2016;12(3).
- 13. Mansour M, Mansour JB, El Swesy AH. Scaling up proven public health interventions through a locally owned and sustained leadership development programme in rural Upper Egypt. Human Resources for Health. 2010;8(1):1.
- 14. Perry C. Empowering primary care workers to improve health services: results from Mozambique’s leadership and management development program. Human resources for health. 2008;6(1):14.
- 15. La Rue KS, Alegre JC, Murei L, Bragar J, Thatte N, Kibunga P, et al. Strengthening management and leadership practices to increase health-service delivery in Kenya: an evidence-based approach. Human resources for health. 2012;10(1):25.
- 16.
Ethiopia FDRo. Woreda Implementation Manual. In: Health Mo, editor. 2017.
- 17.
Casella G, Berger R. Statistical Inference. 2002. p. 272.
- 18. Fetene N, Linnander E, Fekadu B, Alemu H, Omer H, Canavan M, et al. The Ethiopian Health Extension Program and Variation in Health Systems Performance: What Matters? PloS one. 2016;11(5):e0156438. pmid:27227972
- 19. Mavimbe JC, Braa J, Bjune G. Assessing immunization data quality from routine reports in Mozambique. BMC public health. 2005;5(1):108.
- 20. Makombe SD, Hochgesang M, Jahn A, Tweya H, Hedt B, Chuka S, et al. Assessing the quality of data aggregated by antiretroviral treatment clinics in Malawi. Bulletin of the World Health Organization. 2008;86(4):310–4. pmid:18438520