Figures & Tables
Figures
Community health service outreach session in Ghana with a Matlab family welfare assistant. © 1981 James F. Phillips/Population CouncilThe initial official reaction to the Matlab Family Planning Health Services Project results was decidedly negative, despite its impressive statistical and demographic evidence of success. In the view of stakeholders, statistical results provided no evidence that its operational staffing design and management system could be replicated by the Bangladesh Government program. To the official audience for Matlab results, its scientific results initially mattered less than the Maternal-Child Health-Family Planning Extension Project demonstration of the feasibility of replicating its strategies.
International Centre for Diarrheal Disease scientists in Bangladesh hosting Navrongo Health Research Centre counterparts in Matlab. © 1992 James F. Phillips/Population CouncilA 1993 exchange between the Navrongo service implementation and research teams and Matlab counterparts aimed to transfer the Matlab system to northern Ghana. Since social institutions, the health systems context, and the resource base in Navrongo were fundamentally different from the Matlab context, transferring the strategic details of the Matlab program to Africa did not make sense. What was transferable, however, was the process of developing a program that would work. In both examples, a diagnostic phase was followed by a plausibility trial and then by replication research. In both settings, this phased approach to systems development set the stage for scaling up evidence-based systems improvements.
Ifakara Health Institute demographic surveillance system interview, Rufiji District, Tanzania. © 2012 James F. Phillips/Columbia UniversityThe Tanzania Connect experiment was a randomized controlled trial that conformed to conventional standards of statistical rigor. Its use of demographic surveillance permitted community randomization of the assignment of community health workers (CHWs). While this was statistically appropriate, the relevant unit of program governance was the hierarchy defined by dispensary catchment area, ward, and district. Lack of congruence of randomization with bureaucratic context spuriously weakened prospects that the investigation of the CHW deployment hypothesis would succeed.
- FIGURE 1
Research Phases Associated With Developing Community-Based Primary Health Care in Bangladesh and Ghana
Adapted from Nyonator et al.78 and Awoonor-Williams et al.79
- FIGURE 2
The Agile Science Process of Health Systems Strengthening
Tables
- TABLE 1.
Contradictions Associated With the Planning of Embedded Implementation Science and Case Study Examples of Strategies for Resolving Contradictions
Attribute Core Strategies of Implementation Science Strategic Adjustments of Embedded Science Contradictions Associated With Strategic Adjustments Case Study Resolution of Contradictions Goal Problems are identified a priori and resolved through researcher controlled hypothesis testing and dissemination. Organizational change and development requires joint researcher and host agency goal setting. Goals are defined in terms of endpoint hypotheses to be tested rather than host agency goals for testing means of achieving system change. Retain, but subordinate, primary health and demographic impact research to implementation research as an integrated and continuous process.4 Outcome evaluation Statistical inference is based on observation of treatment and counterfactual endpoints, with units of observation conforming to power requirements. Improved host agency functionality and impact Protocols define project start and end dates, endpoints, and hypotheses, whereas organizational change is a continuous, open-ended, and multi-faceted process.10 Phase in research as a process that fosters continuous utilization and action.44
Avoid ending learning processes just because a protocol has been completed.2Leadership Researchers in directive, independent, and autonomous roles with outreach to decision makers and managers at the end of investigation. Collaboration of host agency and research partner leadership Researchers assume directive, independent, and autonomous roles and episodically communicate health and demographic outcomes to host agency counterparts.
Managers' roles are defined by bureaucratic and organizational norms.Host agency managers representing each level of the investigative process are appropriately teamed with research counterparts at each system level. Ownership Host agency audience through “steering committees” and end of the project dissemination. Subordination of research leadership to host agency governance.
Joint dissemination.Leadership malaise in the host agency can permeate an embedded research system, diluting rigor and compromising research implementation.54,80 Develop a partnership of research leadership with host agency institutional structures, but maintain an autonomous research operation. Scientific rigor Study designs conform to conventional criteria for statistical inference. Studies embrace process research, mixed methods research designs, and multilevel analyses in concert with the norms of statistical inference. Constructing the counterfactual is essential but inconsistent with management operations that span all organizational levels.90
Acquiring meaningful numbers of randomized organizational units for observation and statistical inference is impossible.92Intervene with treatment and counterfactual conditions that conform to the host organizational structure.
Use plausibility trials56,85 with statistical methods for non-experimental designs.60System relevance Systems thinking provides frameworks for data capture and analysis.
Optimize costing for achieving clear and unequivocal results for hypothesis testing.Systems thinking includes partnership arrangements and research activities that reflect units of the host agency organization.
Establish researcher and host agency collaboration on operational costing and costing research.Contexts where implementation science is needed most are settings where systems research is most challenging to conduct.92
Systems research requires multilevel longitudinal data that are complex to capture, manage, and analyze.90,92
Prospects for utilization are enhanced if costs are clear, but organizational change often incurs costs that are impossible to predict.Utilize replication studies to disperse research in all relevant cultural and ecological contexts.
Configure learning localities that are consistent with program organizational units at each level of the system.44
Restrict implementation financing to affordable and replicable activities.
Prioritize costing analyses for replication and scale-up phases.100,101 - TABLE 2.
Contradictions Associated With the Process of Conducting Embedded Implementation Science and Case Study Examples of Strategies for Resolving Contradictions
Attribute Core Strategies of Implementation Science Strategic Adjustments of Embedded Science Contradictions Encountered by Embedded Implementation Science Implications for Resolving Contradictions Teamwork Constitute teams according to technical functions. Delineate implementation and research teams. Research teams and implementation teams have contrasting skills, orientations, and roles. Configure at each level of the system “learning localities” where the pursuit of excellence is a collaborative endeavor that integrates implementation with investigation. Simplicity Develop measureable indicators of endpoints and possible confounders. Focus on indicators that are commensurate with host organizational data capture, analysis, and communication capabilities. Research and implementation integration is complex to undertake, but simplicity is often essential for fostering organizational change.105,106 Employ mixed methods research and knowledge management to promote understanding of essential processes and outcomes.44,45,98 Replicability End of project terminates further research on replication or scale-up. Design projects to facilitate subsequent replication and scale-up. Developing learning systems requires focused inquiry in localities where interventions can be tractably managed. Managers often seek investigation that is immediately relevant to large-scale operations.44,45 Plan phases in advance99 that (i) diagnose systems requirements, (ii) test impact, (iii) test replication, and (iv) scale up based on replication lessons. Fidelity Fidelity of interventions to themes appearing in the scientific literature. For longitudinal research on scaling up, develop communication mechanisms that ensure widespread host agency understanding of the evidence justifying change. Primary science generates knowledge about impact without providing knowledge about change processes.90 Adapting to unanticipated changes is essential to scale-up.100
Fidelity to research outcomes is often incompatible with flexibility.Develop “learning localities” for catalyzing the geographic spread of implementation.
Integrate learning into national systems planning processes.
Avoid advocacy focusing solely on “success” without also publicizing challenges and failure.88 - TABLE 3.
Contradictions Associated With Utilizing Embedded Implementation Science for Policy and Action
Attribute Core Strategies of Implementation Science Strategic Adjustments of Embedded Science Contradictions Encountered by Embedded Implementation Science Implications for Resolving Contradictions Curation of knowledge Publish results and disseminate findings to host agency and research audiences. Develop knowledge-sharing mechanisms. Science is disseminated by modes of communication that have limited currency among donors, decision makers, implementers, and managers. Develop a multimethod knowledge management system for research advocacy,105 and build participatory learning and exchanges into research operations.71,74 Sustainability Recommend utilization of research findings in the course of end-of-project dissemination activities. Collaboration of researchers and host agency counterparts on research utilization strategic planning. Planning research utilization is challenged by the institutionalization of dysfunction. Failure is therefore more sustainable than improvement.
Research results may contradict existing organizational norms and policies.Utilize research phase 3 replication research to investigate the determinants of sustainability. - TABLE 4.
Implications of Lessons From the Principles of Agile Science and Case Example for an Agile Paradigm for Embedded Implementation Research
Attribute The Agile Working Group's 12 Principles of Agile Science101,a Agile Embedded Science Implications Goal “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software [program improvements].” Problem identification is a continuous process. Owing to contextual complexity and uncertainty, problem details and solutions cannot always be identified in advance. Outcome evaluation “Continuous attention to technical excellence and good design enhances agility.” Monitor compliance with implementation goals continuously with evaluation criteria that continuously shift, as needed.
Subordinate demographic and health hypothesis testing to implementation process evaluation.
Leadership “The best architectures, requirements, and designs [research strategies] emerge from self-organizing teams.” Problem identification and candidate solutions can be defined by anyone in the research or host agency teams.
Peer leadership is encouraged.
Project leadership is systemic and multileveled, and it is the outcome of collaborative investigation of appropriate system development needs.
Ownership “Business people [Host agency participants] and developers must work together daily throughout the project.” Establish host agency and research joint ownership.
Participatory decision making throughout the process of organizational development.
Scientific rigor [Not relevant] Develop credible results that focus on implementation processes and outcomes. System relevance Working software is the primary measure of progress
“Deliver working software [or research products] frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.”Achieve concordance of research operations with host agency structure and functions.
Assess costs and design research to demonstrate affordability.Open-ended, iterative, and continuous sharing of information and review of progress.
Timing of phases governed by host agency planning and decision processes.
Teamwork “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior [or strategies] accordingly.” At regular intervals, program managers review feedback to implementers and researchers to detect departures from quality or the need to adjust research or implementation strategy.
Roles are integrated for research and host agency counterparts by implementation function.
“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.” Build teams around champions who are successful communicators of innovation.
Foster peer leadership through exchanges.
Simplicity “Simplicity—the art of maximizing the amount of work not done—is essential.” Simple solutions are preferred over more complex interventions.
Complexity determined by host agency targeted changes to be investigated.
Replicability “Welcome changing requirements, even late in development.” Intervention targets, processes for monitoring, and evaluation procedures can be changed by evolving host agency priorities. Fidelity [Not relevant] Intervention targets, processes for monitoring, and evaluation procedures can be changed by evolving host agency priorities. Curation of knowledge “The most efficient and effective method of conveying information to and within a [software] development team is face-to-face conversation.” Direct communication between host agency and research team is essential.
Integrate the process of generating evidence and outcomes with the process of utilizing evidence for decision making.
Sustainability “Agile processes promote sustainable [software] development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.” Research activities and processes are pursued at a pace that can be maintained indefinitely.
Outcomes are delivered continuously as a regular part of research operations.
Investigation is embedded in change processes that are continuous and never ending.
Adapted from similar tables by Nerur et al.107 and by Flood et al.109
