Skip to main content

Main menu

  • Content
    • Current Issue
    • Advance Access
    • Archive
    • Supplements
    • Special Collections
    • Topic Collections
  • For Authors
    • Instructions for Authors
    • Tips for Writing About Programs in GHSP
      • Local Voices Webinar
      • Connecting Creators and Users of Knowledge
    • Submit Manuscript
    • Publish a Supplement
    • Promote Your Article
    • Resources for Writing Journal Articles
  • About
    • About GHSP
    • Editorial Team
    • Advisory Board
    • FAQs
    • Instructions for Reviewers

User menu

  • My Alerts

Search

  • Advanced search
Global Health: Science and Practice
  • My Alerts

Global Health: Science and Practice

Dedicated to what works in global health programs

Advanced Search

  • Content
    • Current Issue
    • Advance Access
    • Archive
    • Supplements
    • Special Collections
    • Topic Collections
  • For Authors
    • Instructions for Authors
    • Tips for Writing About Programs in GHSP
    • Submit Manuscript
    • Publish a Supplement
    • Promote Your Article
    • Resources for Writing Journal Articles
  • About
    • About GHSP
    • Editorial Team
    • Advisory Board
    • FAQs
    • Instructions for Reviewers
  • Alerts
  • Find GHSP on LinkedIn
  • Visit GHSP on Facebook
  • RSS
ORIGINAL ARTICLE
Open Access

Unpacking the “Black Box”: How an SMS-Based Continuing Medical Education Intervention Improved Medical Knowledge Among HIV Clinicians in Vietnam

Maia R. Nofal, Nafisa Halim, Bao Ngoc Le, Lora L. Sabin, Anna Larson Williams, Rachael Bonawitz, Ha Viet Nguyen, Tam Thi Thanh Nguyen and Christopher J. Gill
Global Health: Science and Practice December 2018, 6(4):668-679; https://doi.org/10.9745/GHSP-D-18-00298
Maia R. Nofal
aDepartment of Global Health, Boston University School of Public Health, Boston, MA, USA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nafisa Halim
aDepartment of Global Health, Boston University School of Public Health, Boston, MA, USA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: nhalim@bu.edu
Bao Ngoc Le
bConsulting, Researching Community Development (CRCD), Hanoi, Vietnam.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lora L. Sabin
aDepartment of Global Health, Boston University School of Public Health, Boston, MA, USA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Anna Larson Williams
aDepartment of Global Health, Boston University School of Public Health, Boston, MA, USA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rachael Bonawitz
aDepartment of Global Health, Boston University School of Public Health, Boston, MA, USA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ha Viet Nguyen
bConsulting, Researching Community Development (CRCD), Hanoi, Vietnam.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tam Thi Thanh Nguyen
cCenter for Population Research Information and Databases (CPRID), General Office for Population and Family Planning, Ministry of Health, Hanoi, Vietnam.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christopher J. Gill
aDepartment of Global Health, Boston University School of Public Health, Boston, MA, USA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
PreviousNext
  • Article
  • Figures & Tables
  • Info & Metrics
  • Comments
  • PDF
Loading

Figures & Tables

Figures

  • Tables
  • FIGURE
    • Download figure
    • Open in new tab
    • Download powerpoint
    FIGURE

    Logic Model of Lateral Learning for the mCME Intervention in Vietnam

    Abbreviations: HMU, Hanoi Medical University; mCME, mobile continuing medical education; SMS, short message service.

    The mCME intervention implemented in Vietnam was hypothesized to function as a pedagogical analog to the Health Belief Model. In the intervention, the stimuli were the daily text messages and the desired initial outputs were a change in self-study behaviors, which were also a hypothesized mediator of acquired medical knowledge. As in the Health Belief Model, the potency of the stimuli were contingent on the willingness of a given individual to respond by investing their time and energy to self-study.

Tables

  • Figures
    • View popup
    TABLE 1.

    Sociodemographic Characteristics and Test Scores of Study Participants (N=106).

    Value
    Female, %56.0
    Age, years, mean (SD)41.2 (9.5)
    Highest clinical degree, No. (%)
    CBPA3 (2.8)
    CK157 (53.8)
    CK21 (0.9)
    MD45 (42.5)
    Years of experience providing HIV care to patients, mean (SD)4.2 (4.9)
    Number of patients typically seen each daily, No. (%)
    1–262 (58.5)
    3–413 (12.3)
    5–712 (11.3)
    8–118 (7.6)
    12+11 (10.4)
    Baseline scores, mean (SD)46.4 (11.9)
    Intervention group44.6 (12.4)
    Control group48.2 (11.2)
    Endline scores,a mean (SD)54.2 (12.1)
    Intervention group55.0 (11.7)
    Control group53.4 (12.5)
    % change in scores between baseline and endline,a mean (SD)19.6 (30.9)
    Intervention group25.6 (32.4)
    Control group13.5 (28.2)
    • Abbreviations: CBPA, community-based physician's assistants; CK1, first-level specialization/specialist; CK2, second-level specialization/specialist, SD, standard deviation.

    • ↵a Sample size at endline consisted of 95 individuals (n=48 in the intervention group, n=47 in the control group).

    • View popup
    TABLE 2.

    HIV Clinicians' Participation in Intervention-Prompted and Independent Study Behaviors

    Value
    Intervention-Prompted Study Behaviors Among Intervention Group (n=53)
    % of SMS quizzes answered, mean (SD)81.9 (23.1)
    % of SMS quizzes correctly answered, mean (SD)52.0 (20.5)
    % of daily hyperlinks accessed, mean (SD)18.1 (21.8)
    Ever accessed online CME courses,a % (SD)60.4 (49.4)
    Independent Study Behaviors Among Intervention and Control Groups Combinedb
    Used medical textbooks (ref: 1–3 times/month or <1 time/month)61.7 (48.9)
    Consulted with colleagues (ref: 1–3 times/month or <1 time/month)44.7 (50.0)
    Researched online (ref: 1–3 times/month or <1 time/month)56.8 (49.8)
    Researched website for medical professionals (ref: 1–3 times/month or <1 time/month)55.3 (50.0)
    Reviewed HIV guidelines (ref: 1–3 times/month or <1 time/month)61.1 (49.0)
    Reviewed scientific papers (ref: 1–3 times/month or <1 time/month)42.6 (49.7)
    More use of medical textbooks than before (ref: the same or less)62.8 (48.6)
    More consultation with colleagues than before (ref: the same or less)43.2 (49.8)
    More research online than before (ref: the same or less)60.2 (49.2)
    More visits to researched medical websites than before (ref: the same or less)56.8 (49.8)
    More frequent review of HIV guidelines than before (ref: the same or less)62.4 (48.7)
    More frequent reviews of scientific papers than before (ref: the same or less)50.5 (50.3)
    Total number of study resources used 1–3 or 4 or more times/week, mean (SD)3.6 (2.7)
    • Abbreviations: CME, continuing medical education; SD, standard deviation; SMS, short message service.

    • ↵a Among the entire study group (intervention and control group combined), 43.4% (SD=49.8) ever accessed the online CME courses.

    • ↵b Sample size for the intervention and control group varied slightly for each indicator, from 93 to 95 individuals, depending on how many respondents answered each question.

    • View popup
    TABLE 3.

    Correlation Coefficients for Intervention-Prompted Study Behaviors and Baseline and Endline Test Scores (n=48)

    (1)(2)(3)(4)(5)(6)(7)(8)(9)
    Endline Test ScoresBaseline Test ScoresAverage % of SMS Quizzes AnsweredAverage % of SMS Quizzes Correctly AnsweredAverage % of Daily Readings AccessedEver Accessed Online CME CoursesYears of Experience in HIV Care ProvisionAgeMale
    rPrPrPrPrPrPrPrPrP
    (1)Endline test scores1.00
    (2)Baseline test scores0.59<.0011.00
    (3)Average % of SMS quizzes answered0.17.260.04.811.00
    (4)Average % of SMS quizzes correctly answered0.36.010.11.470.83<.0011.00
    (5)Average % of daily readings accessed0.20.18−0.004.980.38.010.56<.0011.00
    (6)Ever accessed online CME courses0.04.81−0.08.570.45.0020.35.010.30.041.00
    (7)Years of experience in HIV care provision0.16.28−0.07.65−0.16.28-0.09.56−0.002.99−0.19.201.00
    (8)Age−0.13.39-0.09.530.13.360.26.080.14.360.07.630.33.021.00
    (9)Male−0.12.41-0.11.450.19.200.16.27-0.17.260.06.660.15.300.23.111.00
    • Abbreviations: CME, continuing medical education; SMS, short message service.

    • View popup
    TABLE 4.

    Associations Between Intervention-Prompted Study Behaviors and Total Endline Test Scores (n=48)

    Model 1: Predictor, Quiz ParticipationModel 2: Predictor, Quiz PerformanceModel 3: Predictor, Daily ReadingsModel 4: Predictor, Online CME CoursesModel 5: All Predictors Except Quiz PerformanceaModel 6: All Predictors Except Quiz Participationa
    ß (SE)Pß (SE)Pß (SE)Pß (SE)Pß (SE)Pß (SE)P
    Average % of SMS quizzes answered0.24 (0.06).050.16 (0.07).27
    Average % of SMS quizzes correctly answered0.42 (0.06).0040.43 (0.08).005
    Average % of daily readings accessed0.22 (0.06).060.14 (0.07).28−0.03 (0.07).81
    Accessed online CME courses (ref: never)0.16 (2.89).190.05 (3.16).710.03 (2.78).79
    Male (ref: female)−0.11 (2.81).38−0.12 (2.51).28−0.01 (2.83).92−0.07 (2.84).57−0.06 (2.97)0.62−0.13 (2.74).28
    Age−0.18 (0.16).15−0.27 (0.15).02−0.19 (0.17).14−0.16 (0.17).21−0.20 (0.17)0.11−0.27 (0.16).03
    Years of HIV care provision0.31 (0.30).020.34 (0.27).0040.26 (0.29).040.29 (0.31).030.31 (0.31).020.34 (0.28).005
    Baseline test score0.57 (0.11)<.0010.53 (0.10)<.0010.59 (0.11)<.0010.60 (0.12)<.0010.58 (0.11)<.0010.53 (0.11)<.001
    Adjusted R20.400.510.400.370.390.49
    • Abbreviations: CME, continuing medical education; SE, standard error; SMS, short message service.

    • ↵a Model 5 includes Quiz Participation, Daily Readings, Online CME Courses, and controls while Model 6 includes Quiz Performance, Daily Readings, Online CME Courses, and controls. Quiz Participation and Quiz Performance were not included in the same model together due to high collinearity.

    • View popup
    TABLE 5.

    Associations Between Independent Study Behaviors and Total Endline Scores

    Model 1 Predictor, Intervention Status (n=95)Model 2 Independent Study Behaviors as Potential Mediators of Intervention Effect (n=91)Model 3 Independent Study Behaviors as Predictors of Endline Scores Among Intervention Group (n=47)Model 4 Independent Study Behaviors as Predictors of Endline Scores Among Control Group (n=44)
    ß (SE)Pß (SE)Pß (SE)Pß (SE)P
    Treatment status (ref: control)0.11 (2.08).220.15 (2.09).11
    Used medical textbooks (ref: 1–3 times/month or <1 time/month)−0.11 (3.11).410.07 (3.95).69−0.35 (5.47).16
    Consulted with colleagues (ref: 1–3 times/month or <1 time/month)−0.12 (2.57).28−0.23 (3.18).110.18 (4.56).35
    Researched online (ref: 1–3 times/month or <1 time/month)0.20 (3.32).190.26 (3.89).110.12 (6.56).69
    Researched website for medical professionals (ref: 1–3 times/month or <1 time/month)−0.04 (3.11).80−0.14 (3.84).410.40 (6.07).14
    Reviewed HIV guidelines (ref: 1–3 times/month or <1 time/month)−0.09 (2.70).45−0.002 (3.81).99−0.15 (4.37).45
    Reviewed scientific papers (ref: 1–3 times/month or <1 time/month)0.17 (2.46).120.21 (3.13).15−0.03 (4.56).89
    Male (ref: female)−0.25 (2.12).01−0.28 (2.16).005−0.15 (2.71).23−0.55 (4.67).01
    Age−0.02 (0.12).860.11 (0.13).32−0.07 (0.19).660.23 (0.19).19
    Years of HIV care provision0.15 (0.24).120.12 (0.23).240.23 (0.33).110.04 (0.37).83
    Baseline test score0.50 (0.09)<.0010.55 (0.09)<.0010.62 (0.12)<.0010.47 (0.16).01
    Adjusted R20.310.340.420.26
    • Abbreviation: SE, standard error.

    • View popup
    TABLE 6.

    Associations of Interactions Between Quiz Performance and Study Behaviors With Total Endline Scores

    Model 1 (n=48)Model 2 (n=48)
    ß (SE)Pß (SE)P
    % of SMS quizzes correctly answered0.18 (0.08).040.11 (0.09).24
    % of daily readings accessed−0.64 (0.37).09
    Accessed online CME courses (ref: never)−13.81 (8.06).09
    % of SMS quizzes correctly answered * % of daily readings accessed0.87 (0.48).08
    % of SMS quizzes correctly answered * accessed online CME courses0.25 (0.15).09
    Age−1.88 (2.85).51−0.92 (2.76).74
    Years of HIV care provision−0.28 (0.16).09−0.30 (0.16).08
    Baseline test score0.49 (0.11)<.0010.52 (0.11)<.001
    Intercept36.67 (9.15)<.00137.69 (9.51)<.001
    Adjusted R20.440.43
    • Abbreviations: CME, continuing medical education; SE, standard error; SMS, short message service.

    • View popup
    TABLE 7.

    Percentage of Total Explained Variances Attributable to Intervention-Prompted Study Behaviors (n=48)

    Adjusted R2% of Adjusted R2 Attributable to:
    % of SMS Quizzes Correctly Answered% of Daily Readings AccessedAccessed Online CME Courses
    Model 1. Gender, age, years of HIV care provision, baseline test scores0.36
    Model 2. % of SMS quizzes correctly answered, gender, age, years of HIV care provision, baseline test scores0.5143
    Model 3. % of daily readings accessed, gender, age, years of HIV care provision, baseline test scores0.4010
    Model 4. Accessed online CME courses, gender, age, years of HIV care provision, baseline test scores0.373
    • Abbreviations: CME, continuing medical education; SMS, short message service.

PreviousNext
Back to top

In this issue

Global Health: Science and Practice: 6 (4)
Global Health: Science and Practice
Vol. 6, No. 4
December 27, 2018
  • Table of Contents
  • About the Cover
  • Index by Author
  • Complete Issue (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word about Global Health: Science and Practice.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Unpacking the “Black Box”: How an SMS-Based Continuing Medical Education Intervention Improved Medical Knowledge Among HIV Clinicians in Vietnam
(Your Name) has forwarded a page to you from Global Health: Science and Practice
(Your Name) thought you would like to see this page from the Global Health: Science and Practice web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Unpacking the “Black Box”: How an SMS-Based Continuing Medical Education Intervention Improved Medical Knowledge Among HIV Clinicians in Vietnam
Maia R. Nofal, Nafisa Halim, Bao Ngoc Le, Lora L. Sabin, Anna Larson Williams, Rachael Bonawitz, Ha Viet Nguyen, Tam Thi Thanh Nguyen, Christopher J. Gill
Global Health: Science and Practice Dec 2018, 6 (4) 668-679; DOI: 10.9745/GHSP-D-18-00298

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Unpacking the “Black Box”: How an SMS-Based Continuing Medical Education Intervention Improved Medical Knowledge Among HIV Clinicians in Vietnam
Maia R. Nofal, Nafisa Halim, Bao Ngoc Le, Lora L. Sabin, Anna Larson Williams, Rachael Bonawitz, Ha Viet Nguyen, Tam Thi Thanh Nguyen, Christopher J. Gill
Global Health: Science and Practice Dec 2018, 6 (4) 668-679; DOI: 10.9745/GHSP-D-18-00298
del.icio.us logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Statistics from Altmetric.com

Jump to section

  • Article
    • ABSTRACT
    • INTRODUCTION
    • METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • Acknowledgments
    • Notes
    • REFERENCES
  • Figures & Tables
  • Info & Metrics
  • Comments
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Costs and Cost-Effectiveness of mCME Version 2.0: An SMS-Based Continuing Medical Education Program for HIV Clinicians in Vietnam
  • Google Scholar

More in this TOC Section

  • People that Deliver Theory of Change for Building Human Resources for Supply Chain Management: Applications in sub-Saharan Africa and Southeast Asia
  • Exploring the Role of Gender in the Public Health Supply Chain Workforce in Low- and Middle-Income Countries
  • Applying a Theory of Change for Human Resources Development in Public Health Supply Chains in Rwanda
Show more ORIGINAL ARTICLE

Similar Articles

Johns Hopkins Center for Communication Programs

Follow Us On

  • LinkedIn
  • Facebook
  • RSS

Articles

  • Current Issue
  • Advance Access Articles
  • Past Issues
  • Topic Collections
  • Most Read Articles
  • Supplements

More Information

  • Submit a Paper
  • Instructions for Authors
  • Instructions for Reviewers

About

  • About GHSP
  • Advisory Board
  • FAQs
  • Privacy Policy
  • Contact Us

© 2025 Creative Commons Attribution 4.0 International License. ISSN: 2169-575X

Powered by HighWire