Limited Effectiveness of a Skills and Drills Intervention to Improve Emergency Obstetric and Newborn Care in Karnataka, India: A Proof-of-Concept Study

Study Setting and Design

The study was conducted in 2 northern districts (Bagallkot and Koppal) of Karnataka, India. Among the southern states of India, Karnataka reports the highest maternal mortality ratio, at 133 per 100,000 live births (2012–2013), and the second highest infant mortality rate, at 31 per 1,000 live births (2013).^2,3 Within Karnataka, the 8 northern districts are economically disadvantaged and have significant disparities in health infrastructure and service delivery compared with the rest of the state.^10,11

We used a quasi-experimental design with 4 intervention and 4 comparison facilities within these 2 districts over a 14-month time period (July 2013 to September 2014) to test the effectiveness of the skills and drills intervention. The facilities were sub-district-level government health facilities with basic emergency obstetric and newborn capabilities, with 2 facilities equipped to handle cesarean deliveries. The intervention and comparison facilities were comparable in terms of delivery load, type of deliveries, infrastructure, and human resource capacity (Table 1). We hypothesized that the planned intervention would improve the skill and knowledge levels of providers, improve emergency preparedness, and strengthen team work, resulting in improved diagnosis and management of selected maternal and newborn complications at the intervention facilities.

Intervention

The intervention (at the 4 intervention facilities) included a skills refresher session and emergency drills held every 2 months combined with supportive supervision provided by an external team of trainers who were obstetric and pediatric specialists and nurses. This external team consisted of 10 obstetrics specialists and 10 pediatric specialists along with 8 nurses selected from medical colleges across 3 cities or towns of Karnataka (Bangalore and Mysore cities and Bagalkot town). Each of the selected specialists had more than 10 years of clinical experience in their respective fields and most of them were professors or associate professors at medical colleges; the obstetric and pediatric nurses had more than 5 years of experience in their respective fields and had some public health experience, either serving at public facilities or working in the past with research teams. They were all apprised of the travel requirement to study facilities, which required an overnight journey for all except for those from Bagalkot. The team of trainers received an orientation over 2.5 days prior to the intervention; the orientation covered disease management protocols, skill stations demonstrations, and drill exercises. The Institute for Clinical Effectiveness and Health Policy, Argentina, provided the training on the emergency drill exercises.

The intervention activities consisted of:

1. Clinical skills refresher session: The skills refresher training session (conducted by the external team of obstetricians and pediatricians) was a one-time activity held over 2 days at the 4 intervention facilities. The session focused on technical skills and competence building through didactic sessions and skill stations, covering topics related to physical examination of a woman in labor, active management of third stage of labor (AMTSL), emergency preparedness for complications, immediate newborn care, warm chain and feeding of the newborn, neonatal resuscitation, and supportive care for a sick newborn. Participants included all nurses in the labor rooms as well as medical doctors and specialists (if any were available at these facilities).

2. Emergency drills: Once every 2 months, emergency drills for one obstetric complication (either PPH or eclampsia) and one neonatal complication (birth asphyxia) were simulated using a prewritten script; the drills were conducted by a team of 3 trainers (from the external team) comprising an obstetric or pediatric specialist and 2 nurses. The specialist facilitated the drill exercise by providing clinical information relevant to the enacted case scenario while one of the nurses acted as the patient and the other nurse as the patient's relative; health facility staff members were instructed to respond to the simulated emergency situation as close to the real-life local context as possible. The facilities did not receive any prior notice of when the drills would be conducted. The simulation exercises related to birth asphyxia used a NeoNatalie simulator from Laerdal—an inflatable simulator designed to teach basic neonatal resuscitation skills. Each drill, lasting about 45 minutes, was videotaped and played back to the facility staff. The trainers conducted a detailed debriefing to identify both health systems issues (such as availability of drugs and supplies) and gaps in skills and team preparedness (such as communication or team rapport).

3. Supportive supervision: While the team of trainers conducted the drills, a second team of trainers provided supportive supervision to the facility staff on duty by observing practices such as deliveries, newborn feeding, infection control, and availability of critical drugs and supplies, and reviewing delivery and complication case sheets. Based on observations made during the drills and the supportive supervision, the trainer team then debriefed the facility staff and developed a joint action plan for the next cycle. Each subsequent visit started with a review of the action plan and gaps identified during the previous visit.

Outcome Measures

The primary outcome measure was the timely and correct diagnosis and management of direct obstetric complications (PPH and pre-eclampsia/eclampsia) and the newborn complication of birth asphyxia. PPH was defined as the loss of 500 mL or more of blood during or within 24 hours of childbirth¹²; pre-eclampsia was defined as systolic blood pressure ≥140 mm Hg or diastolic blood pressure ≥90 mm Hg with proteinuria after 20 weeks gestation in women with a previously normal blood pressure; eclampsia was defined as the presence of convulsions with signs of pre-eclampsia¹²; and birth asphyxia was defined as the failure to establish breathing at birth or not crying at birth.¹³

Secondary outcomes included knowledge and skill levels of the providers related to maternal and newborn care. For obstetrics, we focused on assessment of labor and initial management of hypertensive disorders of pregnancy and postpartum hemorrhage; for newborn care, we focused on infection control, neonatal resuscitation, and care of low birthweight babies. In addition, we measured the acceptability and feasibility of the intervention as perceived by facility providers.

Data Collection and Data Analysis

Prior to the start of the intervention, a team of specialists along with the project staff introduced a new case sheet in all 8 facilities through a one-day on-site training. This team provided a brief refresher on skilled birth attendance as part of the orientation process and focused on the use of the case sheet as a job aid and for documentation of the delivery process. The case sheet had 2 components: (1) delivery record providing clinical information relevant to delivery care, and (2) complication case sheets providing information on diagnosis and management of maternal and newborn complications, specifically focused on PPH, PIH/eclampsia, and birth asphyxia. Photocopies of the completed case sheets from all 8 study facilities were sent monthly to the project office, which remained the primary source of data on maternal and newborn care practices. Independent obstetric and neonatal experts who were oriented about the study protocols reviewed these case sheets and marked the diagnosis (per study definition) and management (per national health mission protocols) of a complication (primary outcome) as correct, incorrect, or incomplete.

For the secondary outcome of provider knowledge, the providers completed a knowledge questionnaire before and after the intervention. The questionnaire consisted of 21 items on obstetric content (initial assessment and labor, PIH, and PPH) and 15 items on newborn content (infection control, neonatal resuscitation, routine care, and care of low birthweight babies). The maximum score that providers could obtain was 50 (29 points from the obstetric component and 21 points from the newborn component), and they had up to 30 minutes to complete each section. Specialists involved in the development of the training content validated the questionnaire. In the intervention facilities, all participants completed the pretest before the skills refresher session (n = 73). The post-intervention test was restricted to those who had taken the pretest and had attended one of the drills sessions (n = 50). In the comparison facilities, all nurse providers and medical doctors associated with maternal and newborn care (n = 36) took the knowledge tests at the beginning and at the end of the study.

To assess provider skills, we used an objective structured clinical examination (OSCE). For obstetric content, we had 6 observed (32 points) and 4 unobserved (18 points) stations focused on assessing the ability of the staff nurses to identify maternal complications and to begin initial management based on guidelines. For newborn skills, there were 5 observed (36 points) and 5 unobserved stations (14 points) that focused on assessment of a newborn, resuscitation steps, and care of a low birthweight baby. Participants were expected to complete the tasks in 4 minutes. A team of 4 assessors conducted the skills assessment: at the intervention facilities, 50 staff members who had attended the skills session and at least one drill session took the test; in the comparison facilities, 35 staff associated with labor and newborn care took the test. All assessors (obstetricians, pediatricians, and nurses) attended a standardization workshop and received a checklist so that observations were reliable.

At the end of the intervention period, a descriptive qualitative study involving in-depth interviews (one-on-one and group-based) with the health care providers was carried out at 3 intervention and 2 comparison facilities. The interviews helped understand the experiences of these providers, capture best practices, and highlight some of the main challenges with the intervention. The team of interviewers (4 nurses along with project staff) underwent a 2-day orientation and training process on qualitative methods of data collection and note taking. Teams of 2 interviewers plus 1 project staff member (who served as an observer and note taker) conducted in total 12 semi-structured interviews with 19 participants comprising medical doctors (one-on-one interviews) and staff nurses (group interviews). Informed written consent was obtained from all providers with clear information and instructions provided regarding the objectives and their rights of participation in the interviews. All interviews were recorded except interviews with 2 medical doctors who did not want to be recorded. Most interviews with the doctors were in English while those with the nurses were in the local language of Kannada.

All data from cases sheets were entered into a database using Epi Info (version 3.5.2, 2008). For evaluation purposes, data from November 2013 to October 2014 were analyzed. Descriptive statistics were reported using mean and standard deviation for all the continuous variables and number and percentages for categorical variables. Cross-tabulations were used to compare the categorical variables. For all outcomes of interest, numbers and percentages were compared between 2 groups using z test of proportion. A P value less than .05 was considered statistically significant. All the analyses were performed using SPSS 18.

Completed knowledge questionnaires were scored by project staff using an answer key, and completed OSCE sheets were collated and individual scores compiled. The individual knowledge scores and OSCE scores were then entered into Microsoft Excel.

The recordings of the interviews were translated and transcribed; the transcriptions along with the interview notes were analyzed to create topical summaries of the discussion with a special focus on the feasibility and acceptability of the intervention as well as documentation of any changes in obstetric and newborn care practices in the intervention facilities compared with comparison facilities. Figure 1 provides a summary of the overall study design and timeline.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1.

Flow Chart Depicting Study Design, Timeline, Tools, and Study Outcomes

Abbreviations: OSCE, objective structured clinical examination; PIH, pregnancy-induced hypertension; PPH, postpartum hemorrhage.

Approval for this study was obtained from the Institutional Ethics Committee at St. Johns Medical College and from the Department of Health and Family Welfare, Karnataka, India. Consent was obtained from each facility in-charge prior to the start of the intervention at each facility.