Improving State Government's Responsiveness to Family Planning Interventions in Nigeria Using an Innovative Reflection and Action Tool

BACKGROUND

Nigeria is undergoing rapid urbanization with a growing population that is especially concentrated in urban centers. At the current growth rate of about 2.8%–3.0% a year, Nigeria's urban population is expected to double over the next 2 decades.¹ Health outcomes of urban women living in poverty are considerably worse than those of their wealthier urban counterparts and, in some cases, of rural women.^2–5 Although women in urban areas have fewer children (4.5 children per woman) compared to those in rural areas (5.9 children per woman), the fertility rate is impacted by education and wealth quintile in Nigeria. For example, urban poor women in Nigeria often experience similar fertility rates as women in rural areas.⁶

As a result, it is essential to reach urban women and girls of reproductive age, particularly those living in poor urban settlements, with health and family planning (FP) services and to empower them to decide and act freely about whether and when to have children. Contraceptive use improves women's and children's health in many ways, including reducing maternal mortality risks and improving child survival.⁷ There is a solid evidence base for what works in FP programming.^8,9 Despite this knowledge, evidence-based interventions have not been effectively scaled widely, including in Nigeria. The Federal Ministry of Health set a goal to improve the modern contraceptive prevalence rate (mCPR) to 27% by 2020; in 2018, the mCPR was 17% among currently married women aged 15–49 years, and unmet need for FP among currently married women was 19%.⁶ In addition, mCPR was lowest among poor women: 4% among those in the lowest wealth quintile compared to 22% among the highest quintile.⁶ Considering that Nigeria did not meet this set target by 2020, a review of progress and restructure of a timeline for targets was conducted as part of the Nigeria FP2030 commitment, with a renewed commitment to achieve 27% mCPR by 2030.¹⁰

Ownership and sustainability of health programs remain challenges in low- and middle-income countries, including Nigeria, with local governments depending heavily on international donors and partners for support. Challenges in Nigeria include inadequate domestic funding for FP programs, limited human resource and monitoring and evaluation capacity, and weak health systems.^11–15 State governments in Nigeria have the autonomy to decide how to manage their health budget annually; prioritization of FP in those budgets varies between states depending on other competing priorities, government capacity, and program implementation. Suboptimal state FP program response is further compounded by weak government capacity to institutionalize and scale up proven FP interventions.

To address these challenges, it is imperative that Nigerian state governments and local government authorities actively lead and coordinate their family planning and reproductive health (FP/RH) programs. Traditionally, global health programming has been top-down, driven by major donors and governments of high-income countries.¹⁶ However, the 2005 Paris Declaration on Aid Effectiveness, followed by the Accra Agenda for Action (2008) and the Busan Agenda (2011), underscore the importance of country “ownership,” increased reliance on “country systems,” and “enhanced developing country capacities” tailored to country-specific situations and needs for development effectiveness.¹⁷ Both the Paris Declaration and the Accra Agenda included recommendations that technical assistance (TA) providers make capacity-building a fundamental feature of TA to enable countries to operate their health systems effectively and autonomously in the future,¹⁸ while the Busan Declaration placed explicit emphasis on learning as 1 of the core aspects of enhanced capacity.¹⁷

Local government ownership and capacity strengthening that is tailored to local needs is at the heart of The Challenge Initiative (TCI)'s “business unusual” approach, as outlined in Mwaikambo et al.² TCI is an urban FP/RH program with a goal of achieving greater local government self-reliance to scale up evidence-based FP and adolescent and youth sexual and reproductive health (AYSRH) interventions, leading to sustained improvements in FP service delivery and increased use of modern contraception, especially among the urban poor. Funded by the Bill & Melinda Gates Foundation, Bayer AG, Comic Relief, and others, TCI provides coaching and technical and financial assistance to state governments to enable them to lead implementation of evidence-based FP and AYSRH interventions and ultimately sustain the positive outcomes of a government-driven FP/RH program.¹⁹

TCI puts state government leadership at the center of implementing, managing, and monitoring the proven FP and AYSRH interventions. States self-select to become a part of TCI, bringing their own financial and human resources to bear in scaling the proven interventions with the knowledge of their graduation from direct TCI support within 3–4 years. Led globally by the Bill & Melinda Gates Institute for Population and Reproductive Health (Gates Institute) and implemented by the Johns Hopkins Center for Communication Programs in Nigeria, TCI partnered with 13 states in its first phase, including Ogun, Delta, Kano, Niger, Bauchi, Anambra, Plateau, Taraba, Abia, Rivers, Nasarawa, Lagos, and Gombe states.

While the overarching purpose of TCI is to rapidly meet women's and couples' unmet need for modern contraception, it aims to effect changes in the health system along the way to ensure the results are sustainable, even after a state graduates from TCI. Therefore, in addition to measuring scale-up in the traditional sense of the number of states engaged and population reached, TCI developed the Reflection and Action to Improve Self-reliance and Effectiveness (RAISE) tool, which monitors a state's progress toward achieving sustainable scale-up of high-quality FP programming and outcomes in the following areas.

• Increased political and financial commitment for FP among local government

• Capacity strengthening of local government stakeholders at the systems, organizational, and individual levels

• Institutionalization of TCI high-impact interventions and processes

• Sustained demand for FP services, including increased agency of women and supportive community norms

TCI designed RAISE as a responsive feedback (RF) tool based on empowerment evaluation theoretical principles and successful experiences in implementing organizational capacity assessment tools, with the intention to go beyond measuring and monitoring capacity development to institutionalizing a process for organizational reflection, learning, and problem-solving.^20–23 Through quarterly assessments, state government teams reflect upon the progress and challenges encountered in financing, implementing, and managing their FP programs and ultimately arrive at actionable feedback and steps to ensure the program stays on track and achieves its intended outcomes. RAISE also fosters best practice sharing and enables state governments to lead and facilitate implementation of their improvement plans. The RAISE tool is used by state governments in partnership with TCI, therefore increasing the likelihood that it will be sustained beyond TCI's engagement period. It is currently being used by all 13 TCI-supported states in Nigeria as well as 97 TCI-supported cities across East Africa, Francophone West Africa, India, and the Philippines.²⁴ The aim is that state and city governments will continue to use this RF tool and process themselves following graduation.

This article aims to investigate how RAISE has been used as an innovative RF mechanism to periodically evaluate a government's readiness to lead and implement evidence-based FP and AYSRH interventions. The tool was developed to help TCI explore the following questions: (1) How can TCI and government determine when a state is ready to graduate from TCI's direct support? (2) How can TCI effectively monitor a state's capacity to finance, implement, and monitor the outcomes from evidence-based FP and AYSRH interventions, with incorporation of feedback to adapt the program for improvement in real time? In this article, we review the RAISE assessment findings in Nigeria and explore trends in use of the RAISE tool for RF to improve FP/RH programs. We also reflect upon lessons learned from conducting 5 rounds of assessments with the tool and highlight adaptations in its use, as well as successes and ongoing challenges.

RAISE TOOL

RAISE is a management and organization tool that uses an RF approach to assess and improve state government's self-reliance in FP/RH programming. RF is an integrated framework that incorporates elements of feedback, continuous learning, and engagement, premised on using foresight and learning to iterate programs for better results. The RAISE approach encompasses and incorporates the 5 key elements of RF in its implementation (Box 1).

The RAISE tool comprises 4 pillars of measurement: (1) political and financial commitment, (2) capacity strengthening, (3) institutionalization of proven approaches, and (4) sustained demand (Figure 1). Each domain has indicators that the assessors score from 1 (minimum) to 4 (maximum). The domains are weighted percentages to allow for easy aggregation of final scores.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1

RAISE Sustainability Pillars

Abbreviations: FP, family planning; RAISE, Reflection and Action to Improve Self-reliance and Effectiveness. Source: The Challenge Initiative.

In June 2020, TCI Nigeria adapted the global TCI RAISE tool to suit the country's context and respond to key issues limiting government self-reliance in FP/RH programming, specifically noting the challenge related to limited health expenditure and, more specifically, domestic financing of FP/RH programming. This adaptation process was done at the national level in Abuja in consultation with the TCI state and government teams to ensure that the state-specific context was considered while standardizing the indicators and assessment approach. This adaptation occurred at 2 levels: (1) an initial joint review of the tool by the state government and state TCI team members, and (2) a second review between the TCI teams at the state and national levels to aggregate feedback and considerations from the state teams. Consensus from the review of the tool led to weighting the 4 domains as follows, resulting in a total possible score of 100% for the Nigeria RAISE assessment.

• Domain 1, political and financial commitment: 30%, was deemed the most important by the government to achieve self-reliance in programming based on current realities.

• Domain 2, government capacity: 20%, was reported to be the least important considering the number of trainings and capacity-building exercises that have happened at the state level, with states having moderately acceptable numbers of trained FP providers.

• Domain 3, institutionalization of the proven interventions: 25%, was given equal importance as domain 4 and retained its original weight.

• Domain 4, sustained demand: 25%, was given equal importance as domain 3 and retained its original weight.

However, the Nigeria hub maintained the original overall categories of government maturity or capacity levels based on the aggregated final score (Figure 2). This categorization was originally developed in the global tool by a team of experts who initially piloted the tool, including members from the Gates Institute, TCI East Africa, and government teams from Kenya and Uganda. The categories of state government maturity stages toward self-reliance are as follows.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 2

RAISE Ranking Levels

Abbreviation: RAISE, Reflection and Action to Improve Self-reliance and Effectiveness. Source: The Challenge Initiative.

• “Beginning” (54% and below), showing increased capacity needed by government

• “Developing” (55%–69%), depicting basic government capacity

• “Expanding” (70%–84%), depicting moderate government capacity

• “Mature” (85% and above), showing a mature FP/RH program owned and led by government

TCI uses the RAISE tool to gauge state governments' self-reliance of their FP/RH programs and as 1 indicator among 2 others to measure state governments' progress toward graduation from the TCI program and its direct support. The 2 other graduation indicators, alongside a mature stage of RAISE (85% and above), include level of government funding for FP/RH programs and modern contraceptive uptake trends, as described by Finkle et al.¹⁹

Implementation of the RAISE assessment is coordinated and led by a designated government staff member in the FP/RH unit, usually the state FP coordinator, who ensures that relevant government staff is available to participate in the assessment. Critical to the assessment is the use of government data sources to complement and validate the scores given to the indicators measured in the RAISE tool. Some of the data sources used include the health management information system (HMIS) for review of the reproductive, maternal, newborn, and child health indicators; program data related to the proven interventions being implemented; and financial reports to assess the level of domestic funding. The HMIS data are reviewed for the quarter being assessed against the indicators in the RAISE tool to confirm if there was an increase or a dip in the number of new FP acceptors. Similarly, the state annual operational plans are reviewed for scoring domain 3—institutionalization of proven interventions—to assess the number of proven interventions adopted for the reporting quarter. The state TCI and government teams use a funding tracker to monitor the funding amount released by the state government and TCI to support program implementation on a quarterly basis. The government assessors develop an action or remediation plan to address suboptimal areas, with responsible persons identified to lead the corrective actions planned and set timelines for implementing the action plan. TCI typically participates in the first few assessments to provide coaching and guidance on the use of the tool before states fully take over the process. The government's leadership of this process is critical to its ownership of the actions associated with learnings that emerge from the assessment reflections and discussions.

After adapting the tool, TCI Nigeria subsequently conducted an orientation for all TCI staff to ensure uniform understanding of the tool and seamless rollout across the supported states. TCI teams across the supported states, in turn, developed a RAISE implementation schedule that articulated quarterly assessment plans working with relevant government teams. Using the guidance provided in the tool, the TCI state teams worked with their state government counterparts to identify relevant participants for the RAISE assessment. The assessment teams, which include FP/RH policymakers, program managers, implementers, and implementing partners, differed slightly in composition based on the state context. These sets of participants are the target for adaptive management of health programs due to the nature of their work, which encompasses provision of TA and implementation of proven FP/RH interventions. The program managers and implementers work closely together to ensure that remediation plans from the assessments are implemented to cause the desired change and improvement.

The RAISE Assessment Process

The RAISE assessment is conducted on a quarterly basis using a multistep process (Figure 3).

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 3

RAISE Assessment Process

Abbreviation: RAISE, Reflection and Action to Improve Self-reliance and Effectiveness. Source: The Challenge Initiative.

Pre-workshop review. Individual participants listed in Box 2 are asked to review and familiarize themselves with the RAISE checklist (Supplement) ahead of the assessment workshop and come to the workshop with questions on areas or sections that are unclear. This step is skipped for future assessments except when there are new participants.

BOX 2

Participants in the Reflection and Action to Improve Self-reliance and Effectiveness (RAISE) Assessment Teams in Nigeria

Policymakers

• Honorable commissioner for health

• Executive secretary, State primary health care development agency

• Director, Public health

• Director, Planning, research, and statistics

• Director, Community services

Program managers and implementers

• State reproductive health coordinator

• State family planning coordinator

• State health educator

• State monitoring and evaluation officer

• State logistics officer

• Adolescent health desk officer

• State finance focal person

Partners

• The Challenge Initiative team

• Representatives from family planning/reproductive health implementing partners

Small group assessment. Participants are divided into small groups to build consensus on scoring the checklist indicators based on available evidence from the data sources mentioned earlier.

Group workshop. The small groups convene as 1 large group to review the small-group scores using available evidence and come to a consensus on the final scores per domain for the state.

Local government action plan development. The team jointly develops an action or remediation plan—including corrective actions, responsible persons, and timelines (usually within 2 months to allow for implementation before the next quarterly assessment)—to address gaps identified during the assessment.

The assessment team jointly develops an action or remediation plan to address gaps identified during the assessment.

The workshop is facilitated by a trained state government staff member, usually the state FP coordinator, with TA from the TCI state team. This facilitator works with the assessment team members to plan and conduct a dissemination and feedback meeting with the extended FP/RH team across relevant ministries, departments, and agencies while working together to implement the solutions proffered in the action plan. The RAISE assessment process is not complete until the assessment team has provided feedback on their findings and planned actions to relevant policymakers and other technocrats who will support the implementation of corrective actions to close identified gaps. The feedback summarizes components for which the state is performing optimally but focuses more on the challenges or issues leading to low scores that need to be addressed.

Policy and funding issues requiring attention are assigned to relevant directors who are part of the assessment team. The directors advocate for policy change or enforcement with the executive secretaries of the state primary health care development agencies and honorable commissioners for health of the state ministries of health and secure additional funding for the FP program when necessary. Once these changes are approved by the policymakers and translated into action, program managers and technocrats are duly notified, enabling them to implement the policies across implementing geographies in the state and request funds for relevant activities. Implementation of these interventions with attendant results empowers them to adjust the RAISE scores upward during the next assessment based on guidance.

Technical and quality-related gaps are assigned to program managers or technical leads on the assessment team to facilitate planned corrective actions. Examples of these gaps include the lack of FP supportive supervision systems, inadequate capacity of health providers to administer long-acting contraceptive methods, provider bias, poor documentation of services provided, and weak commodity logistics systems. To address the challenges and contribute to improved scores in subsequent assessments, the team organizes and includes quarterly FP supportive supervision in their workplans and plans coaching and mentoring sessions for providers in local government areas reporting gaps in service, commodities, and documentation, using available domestic funding and leveraging partner funding. Implementation of the action plan items on the remediation plan before the next assessment usually leads to improvement in the RAISE scores, thus moving the state toward program maturity. However, states are not always able to make equal progress on all items noted in the remediation plans before the next assessment. As a result, the items that are not completely resolved move to the following quarter's remediation plan.

The assessment team uses a tracker to measure progress and make further course corrections based on quarterly assessments implemented. TCI Nigeria also maintains a digital RAISE dashboard at the national level that tracks progress across all supported states. This allows TCI Nigeria to see at a glance how states are performing and follow up with the assessment teams to provide TA to any states with stagnant indicators. This is an important way in which the RAISE tool is used as an RF approach to elicit real-time improvement in FP/RH programming.

TCI Nigeria uses a digital RAISE dashboard as an RF mechanism to monitor performance across all states and follow up with teams to elicit real-time improvements in FP/RH programming.

LESSONS LEARNED

The use of the RAISE tool as an RF mechanism has enabled each TCI-supported state to function as a learning organization, intentionally pausing quarterly to assess progress and identify challenges to program implementation. It has required a certain level of adaptation to the Nigerian context and nuance in interpreting the results. Buy-in of key government counterparts regarding the use of the RAISE tool and development of a RAISE implementation schedule enabled state teams to have relatively uninterrupted RAISE assessments quarterly. Assessment findings are triangulated with other sources of data, including data from the government's HMIS and program records data, to ensure objectivity of scores assigned.

Joint development of the action plan and the provision of feedback from assessment findings to key government policymakers are critical components of the RAISE process, enabling policymakers to make commitments and act. Policymakers have acted to enforce policies and secure funding because of these RAISE feedback sessions.

Supporting state and local government teams to own the RAISE tool by contributing to its adaptation to the Nigerian context and leading its implementation on a quarterly basis has improved their understanding of adaptive program management and led to numerous positive changes. This is reflected in the following comments from various government team members.

I appreciate The Challenge Initiative for this assessment process, and I have come to realize that the assessment has kept us on track with regards to FP and the development of the remediation/action plan has assisted us in also tracking the level of implementation of our AOP [annual operational plan] with particular emphasis to the FP component. I want to recommend that similar assessment checklist should be adopted for use in other areas of health interventions like malaria, nutrition, etc. —Permanent Secretary, Taraba State Ministry of Health

The first assessment was an eye opener to the state in areas that require strong advocacy, such as budget for adolescent health and increase funding for FP. —Head of Program Unit, Ogun State Primary Health Care Development Agency

I initially thought of the RAISE process as cumbersome; however, as we progressed with the assessment seeing the final product, I was very impressed by the expository nature of the findings. They were succinct and empirical, and RAISE is indeed an eye opener. We will need TCI's support to adopt this tool for other [health] programs. —Head of Planning, Research & Statistics, Anambra State Ministry of Health

The things we knew are what we were working on before, but now, we know more on areas that really require our attention, and this has improved the way we work. RAISE has opened our eyes. —State FP Coordinator, Ogun State Primary Health Care Development Agency

We describe key lessons learned and observations from implementing the tool and how challenges have been addressed.

CHALLENGES

Although the RAISE assessment contributed to improved responsiveness of governments to their FP/RH programs across focus states, there have been challenges with implementing and institutionalizing this RF mechanism and process. Challenges faced in introducing and implementing the RAISE tool are similar to the known obstacles in trying to promote a “test and learn” culture within well-established, hierarchical organizations, including acceptability, institutionalization, and scale-up.