Many Cooks in the Kitchen: Iterating a Qualitative Analysis Process Across Multiple Countries, Sites, and Teams

Stage 1: A Structured, Phased Approach to Data Collection and Preliminary Analysis

From February 2021 to January 2022, we conducted data collection with concurrent preliminary analysis in Kenya, Malawi, and Uganda through a systematic approach to iterative data collection and analysis that divided data collection into 4 phases with structured analysis pauses between each phase. The Table summarizes the 4 phases and a fifth phase during which the Nigeria ICAN team conducted data collection without pausing for analysis (described further later). A research director from the U.S.-based team worked with leads from each data collection site to coordinate this initial stage. The number of researchers involved in data collection and preliminary analysis during this phase differed by country, ranging from 5 to 7. Each Africa-based team worked to code and memo about the data from their own country; the U.S. ICAN team provided technical support for and contributed to each country-based process and facilitated cross-country analysis.

In Phase 1 of Stage 1, we started data collection slowly to facilitate initial review of the data. We knew from previous experience that processing data would be relatively slow due to the need to simultaneously transcribe and translate audio recordings of each interview. So, we designed the slow start partially to give transcriptionists adequate time to do their work. Once transcripts were finalized, they were uploaded to a shared cloud storage drive that all team members could access. Phases 2–4 of Stage 1 then moved more quickly to scaffold data collection so that we had time to review data, monitor quality, and conduct preliminary analyses. Keeping with our adapted grounded theory approach, we built in regular pauses throughout. These pauses gave teams time to assess whether data collectors should probe participants further on key themes that required additional evidence or focus on types of women (e.g., adolescents and married women) who were yielding especially interesting or surprising data.

To bypass the extra time needed for transcription at this stage, we drew on the literature on rapid qualitative analysis,¹⁷ which suggests time-saving strategies for eliminating the use of transcripts¹⁸ or speeding up transcript production.¹⁹ In response, we developed a short form that each interviewer could complete after an interview to capture key data points (Supplement 1). These post-interview report forms were programmed into REDCap data collection software. We chose REDCap because it allowed interviewers to complete the forms remotely offline—a critical consideration when conducting data collection in areas with low connectivity—and then immediately transmit their data back to the full team once they were able to connect to the Internet. The simultaneous data collection and analysis process is illustrated in the Figure.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE

Simultaneous Data Collection and Rapid Analysis Process as Part of Stage 1 in Kenya, Malawi, and Uganda

Abbreviations: ICAN, Innovations for Choice and Autonomy; IDI, in-depth interview.

During each analysis pause, team members (including members from U.S. ICAN) were assigned 1–2 full transcripts to review and open code.²⁰ Team members also skimmed the post-interview report forms collected since the last pause. In some study countries, ICAN researchers not participating in the qualitative analysis teams also gave input through weekly country team meetings. After this individual and group review, each Africa-based team was joined by U.S. ICAN researchers to discuss preliminary findings and jointly complete a table of initial findings similar to the “data reduction” process described by Watkins.²¹ These tables addressed priority study questions, such as “How do women form contraceptive preferences?” An example table can be found in Supplement 2.

During analysis meetings, key themes began to emerge, such as the critical role that male partners play in women's contraceptive preferences and the importance of accessing health care providers for trustworthy information. Group discussion included a deep exploration of the nuances of, for example, couple dynamics that inform contraceptive decision-making. Identifying these key themes and issues during data collection allowed research assistants to probe more strategically when they returned to data collection after an analysis pause. Analytic teams also used these meetings to discuss and agree on any adjustments to data collection instruments before fieldwork resumed and to begin codesigning a codebook. Notes from the analysis meetings and the post-interview report forms were also used in adjacent ICAN study activities, which are detailed elsewhere.¹⁶

Data collection and analysis were slightly staggered across countries due to logistical issues and to enable the U.S. ICAN researchers, as facilitators of the cross-country analysis, to participate in each country's preliminary analysis. This staggering also allowed country teams to learn from one another and was particularly beneficial for teams rolling out data collection later to refine field procedures and tools based on prior learnings. Although preliminary analyses were conducted separately for each set of country study data, the U.S.-based researchers facilitated sharing of cross-country findings. To keep meeting times manageable for all, these researchers relayed cross-country findings during individual country team meetings and referred teams to each other to discuss key themes or challenges where appropriate. U.S. ICAN researchers also ensured that edits made to the data collection instruments were updated across countries.

The Nigeria ICAN team chose a modified approach to increase efficiency and began data collection several months after the Kenya, Uganda, and Malawi ICAN teams. While following the same general methodology as other country teams, the ICAN Nigeria team chose to compress data management, consolidation, and preliminary analysis after learning from the experiences of other country teams. They also were able to work with instruments that had gone through several iterations and improvements. With careful planning and piloting, Nigeria ICAN researchers collected data over 2 weeks in September 2021.

Stage 2: Standardizing Coding Across Teams

Once data collection was complete in each country, all teams elected to follow a similar process to code the full set of interview transcripts. First, qualitative team researchers participated in an in-depth qualitative analysis training. These trainings were facilitated in each country by team members with expertise in the field with virtual cofacilitation by U.S. ICAN researchers where necessary. This training included an introductory tutorial in the qualitative analysis software package Dedoose. The U.S. ICAN researcher who served as technical lead on the cross-country analysis chose Dedoose for the cross-country analysis because, unlike most qualitative coding programs, it allows for simultaneous coding in real time without the need for manual file exchange and syncing. In this regard, using Dedoose made central management of files relatively straightforward and reduced concerns about version control. However, few others on the research team had experience with Dedoose. During the coding process, some discovered that using a web-based tool was cumbersome for country teams experiencing unstable Internet connectivity and electricity, most notably in Malawi and Nigeria. These factors required researchers to constantly restart the software and slowed progress while adding to frustration.

Each of the coding teams based in Kenya, Uganda, Malawi, and Nigeria standardized coding using a codebook codeveloped across all teams by inductively drawing on findings from the simultaneous data collection and analysis and deductively using the available literature (the final codebook is available in Supplement 3). Although grounded theory typically relies on an inductive approach to analysis, we took this combined approach to meet the applied needs of the study. For example, when developing deductive codes, we mainly relied on the literature related to reproductive empowerment and autonomy^22,23 to ensure findings could inform our main analyses of contraceptive decision-making and the relationship between self-injection and contraceptive agency. Although the codebook was standardized as much as possible to allow for cross-national comparison of the data,²⁴ it remained flexible to allow for local context to be taken into account.²⁵ The codebook largely included index codes to facilitate coding in a large group and some analytic codes designed to address key areas of inquiry as well.

For this stage of the analysis, the Kenya ICAN team developed a process whereby the entire team met as a group and coded a full transcript together over a series of 2-hour virtual sessions. This process was subsequently adopted by the Uganda, Malawi, and Nigeria ICAN teams. U.S. ICAN team members joined these initial virtual sessions to ensure that codes were applied consistently across countries. Although this process was especially time intensive, it allowed researchers to have nuanced conversations about the codebook while standardizing code application across countries. In this way, all team members brought their unique experiences to the analytic process and contributed to a richer, more accurate analysis. After completing the group coding process for 1 full transcript, team members worked in pairs to code 1 additional transcript and compare their coding choices. During this process, teams were able to consult internally, as well as with the ICAN lead of qualitative research, if questions arose.

Editing the codebook and standardizing code application took 2–4 months for each country team to complete, with some teams greatly delayed in their coding due to COVID-related emergencies. In addition, other components of the larger ICAN study were intensifying in Kenya, Uganda, and Malawi during this period, requiring researchers to divert some time to other study activities. Because adjacent ICAN activities followed a different timeline in Nigeria, the team there finished qualitative data collection and analysis in between other study activities; with all team members dedicated to this activity, coding was completed at the same time as the other country teams despite collecting data much later.

Stage 3: Coding Independently and Writing Analytic Memos

Once teams had standardized coding, researchers from each team completed coding a set of individually assigned transcripts using Dedoose. In some cases, such as ICAN Nigeria, the entire team contributed to independent coding. In others, such as ICAN Kenya, additional staff with expertise in qualitative coding were brought in to complete this exercise so that core ICAN researchers could focus their time on other study activities. When additional staff were brought in, teams developed locally appropriate processes to establish standardized code application before coding individually. For example, the ICAN Kenya team worked both internally and with the ICAN qualitative research lead to ensure external coders understood how to use the codebook and to establish standardized code application. Across teams, coders worked on their own timeline with meetings scheduled as needed when questions arose, rather than meeting regularly as in earlier phases of the analysis.

Whether using a version of grounded theory or another approach to qualitative analysis, researchers typically write analytic memos during and after data review. These memos are meant to capture the researcher's interpretation of the data by connecting themes arising from transcript and code review and relating them back to the key research questions.^26,27 Although we initially planned to incorporate this type of memo-writing into our coding process using a standardized template in Dedoose, coders found it cumbersome and impractical to switch back and forth between the templates and transcripts and thus rarely used the templates. Instead, some researchers kept their own memos in Microsoft Word documents, which they saved to a shared cloud folder. The benefit of this approach was that it allowed for more flexibility and ease in keeping memos and resulted in accessible notes that other ICAN researchers could easily digest and repurpose for their own analyses without having to download and refer to multiple code reports themselves.

Stage 4: Drawing on Code Output and Memos to Answer Multiple Research Questions

Members of the U.S. ICAN team are taking the lead on synthesizing Stage 1 rapid analysis findings and Stage 3 code output into results for our 2 primary cross-country manuscripts on contraceptive decision-making and identifying the relationship between self-injection and contraceptive agency (results forthcoming). For secondary analyses, the Stage 4 analysis process is decentralized to allow multiple members of the consortia to identify and lead individual analyses to address additional research questions for which we collected data. This decentralization is meant to allow us to conduct multiple analyses more efficiently, develop more diverse findings, and disseminate findings more broadly while also giving all team members the opportunity to take leadership roles and experience qualitative analysis from start to finish. Earlier in the ICAN study, all teams contributed to a list of potential manuscripts that they expected might follow from the final analysis. This list was derived from both key study questions and team members' individual research interests. With data fully coded, teams revisited this list, consolidated the list to reduce overlap, and identified priority analyses. To promote participation in writing and leading manuscripts by the full consortium—in particular, early career researchers from each team—ICAN has an authorship secretariat that provides structure and transparency in the process of proposing and leading manuscripts. The secretariat also offers a mentorship program and writing workshops for those seeking writing support. The structure and dynamics of the ICAN authorship secretariat will be explored in a future manuscript.