TY - JOUR T1 - Measuring Knowledge of Community Health Workers at the Last Mile in Liberia: Feasibility and Results of Clinical Vignette Assessments JF - Global Health: Science and Practice JO - GLOB HEALTH SCI PRACT SP - S111 LP - S121 DO - 10.9745/GHSP-D-20-00380 VL - 9 IS - Supplement 1 AU - Jordan Downey AU - Anne H. McKenna AU - Savior Flomo Mendin AU - Ami Waters AU - Nelson Dunbar AU - Lekilay G. Tehmeh AU - Emily E. White AU - Mark J. Siedner AU - Raj Panjabi AU - John D. Kraemer AU - Avi Kenny AU - E. John Ly AU - Jennifer Bass AU - Kuang-Ning Huang AU - M. Shoaib Khan AU - Nathan Uchtmann AU - Anup Agarwal AU - Lisa R. Hirschhorn Y1 - 2021/03/15 UR - http://www.ghspjournal.org/content/9/Supplement_1/S111.abstract N2 - Key FindingsAdministration of vignettes in a community setting in rural Liberia was feasible and effective for measuring community health worker (CHW) knowledge of case management for the 3 major causes of child mortality addressed by integrated management of childhood illness.Correct diagnosis and lifesaving treatment for uncomplicated malaria were high, but correct diagnosis and treatment rates for complicated conditions including pneumonia with a danger sign were low.Although use of existing digital and paper-based clinical decision support job aids was inconsistent, CHWs who used them had statistically significant higher rates of correct diagnosis and treatment.Key ImplicationsImproving the quality of CHW-delivered care is an important component of achieving universal health coverage, but gaps exist in how to measure knowledge and quality of delivery in remote settings.Clinical vignettes offer a potential approach for program implementers and policy makers to measure CHW knowledge, as a first step to incorporating assessments of the quality of CHW-delivered care into national CHW programs.Introduction:Community health workers (CHWs) can provide lifesaving treatment for children in remote areas, but high-quality care is essential for effective delivery. Measuring the quality of community-based care in remote areas is logistically challenging. Clinical vignettes have been validated in facility settings as a proxy for competency. We assessed feasibility and effectiveness of clinical vignettes to measure CHW knowledge of integrated community case management (iCCM) in Liberia's national CHW program.Methods:We developed 3 vignettes to measure knowledge of iCCM illnesses (malaria, diarrhea, and pneumonia) in 4 main areas: assessment, diagnosis, treatment, and caregiver instructions. Trained nurse supervisors administered the vignettes to CHWs in 3 counties in rural Liberia as part of routine program supervision between January and May 2019, collected data on CHW knowledge using a standardized checklist tool, and provided feedback and coaching to CHWs in real time after vignette administration. Proportions of vignettes correctly managed, including illness classification, treatment, and referral where necessary, were calculated. We assessed feasibility, defined as the ability of clinical supervisors to administer the vignettes integrated into their routine activities once per year for each CHW, and effectiveness, defined as the ability of the vignettes to measure the primary outcomes of CHW knowledge of diagnosis and treatment including referrals.Results:We were able to integrate this assessment into routine supervision, facilitate real-time coaching, and collect data on iCCM knowledge among 155 CHWs through delivery of 465 vignettes. Diagnosis including severity was correct in 65%–82% of vignettes. CHWs correctly identified danger signs in 44%–50% of vignettes, correctly proposed referral to the facility in 63% of vignettes including danger signs, and chose correct lifesaving treatment in 23%–65% of vignettes. Both diagnosis and lifesaving treatment rates were highest for malaria and lowest for severe pneumonia.Conclusion:Administration of vignettes to assess knowledge of correct iCCM case management was feasible and effective in producing results in this setting. Proportions of correct diagnosis and lifesaving treatment varied, with high proportions for uncomplicated disease, but lower for more severe cases, with accurate recognition of danger signs posing a challenge. Future work includes validation of vignettes for use with CHWs through direct observation, strengthening supportive supervision, and program interventions to address identified knowledge gaps. ER -