Effectiveness of SMS Technology on Timely Community Health Worker Follow-Up for Childhood Malnutrition: A Retrospective Cohort Study in sub-Saharan Africa

In Ghana, Rwanda, Senegal, and Uganda, we found positive association between community health workers (CHWs) using SMS data entry with reminder alerts and timely follow-up for childhood malnutrition screening visits compared with paper forms. This association was strongest when CHWs used SMS data entry consecutively over multiple visits than when they switched between SMS and paper forms.


INTRODUCTION
I n 2000, the world committed to achieving the measurable targets set by the Millennium Development Goals (MDGs) to combat extreme poverty by 2015. Goal No. 1 targeted halving the proportion of people suffering from hunger globally 1 and Goal No. 4 aimed to cut the global under-5 mortality rate by two-thirds. 2 In 2015, more than 99 million children under 5 years of age remained undernourished, with two-thirds of this number in Asia and one-third in Africa. 3 The pace of progress has not been consistent or equal across regions. The global prevalence of underweight children decreased from 25% (1990) to 15% (2013), 3 but Africa experienced the smallest relative decrease compared with other regions. 3 These regional inequalities persist due to complex and multifaceted socioeconomic and political variables. Child health remains a priority in the post-2015 agenda, as evidenced by the targets included in the Sustainable Development Goals (SDGs) 1, 2, and 3 to end poverty, prevent hunger, and improve health, respectively. 4 The Millennium Villages Project (MVP) was initiated in 2005 to meet these MDGs in rural sub-Saharan Africa with a series of integrated interventions spanning multiple sectors inclu-ding health, education, agriculture, and infrastructure. 5 By 2006, MVP operated across 10 countries and 14 community sites in sub-Saharan Africa, covering approximately 500,000 people. 5,6 An opensource information and communication system called the Millennium Villages Global Network (MVG-Net) was designed and deployed to assist with health systems monitoring and evaluation and to communicate between sites. 7 The open-source components of this system used for community health worker (CHW) support were comprised of ChildCountþ, a mobile platform used to collect data, and OpenMRS, an electronic medical record in which patient health information was stored. 7,8 These 2 components of MVG-Net interacted with each other to collect health information at household visits, store data in a central system, and send CHWs follow-up reminders for pending visits (Figure). 7 Included in these interventions was the equipping of CHWs with mobile technology (SMS, or short message service) for data collection, reporting, communications, and point-of-care support.
A cluster randomized control trial (RCT) of mobile text message reminders in Kenya evaluated health worker adherence to malaria treatment and counseling. 9 When health workers received daily text message reminders, there was a 23.7% improvement in correct management immediately and 24.5% improvement at 6 months. 9 Additional evidence from Western Kenya's MVP site found improved adherence to antenatal care and postpartum visits among CHWs receiving ChildCareþ text reminders. 10 Similarly, mobile phone use had a positive effect on malaria surveillance, case reporting, and follow-up in rural South Africa. 11 Quality of SMS reporting by health workers remains a concern; the SMS for Life study across 5 rural Kenyan districts showed variable accuracy of surveillance data. 12 The field of mHealth has itself burgeoned recently and the use of mobile phones for health interventions has become increasingly sophisticated. The World Health Organization (WHO) describes mHealth as "the practice of medicine and public health assisted by mobile technologies, such as mobile phones, patient monitoring personal digital assistants (PDAs) and other wireless technologies. 13 In high-income countries, examples include point-of-care monitoring for blood glucose levels in diabetes management 14 or running a 2-lead electrocardiogram (ECG) for point-of-care cardiac investigations. 15,16 In lowand middle-income countries, mobile phones offer a low-cost tool for data collection. The WHO Global Observatory for eHealth (electronic health) reports that the majority (83%) of WHO member states offer at least 1 type of mHealth service within their countries and 70% of mobile phone users reside in low-and middle-income countries. 13 A systematic review on the use of SMS in health programs identified 31 projects in developing countries in 2012. 17 The majority of these programs were in Africa (Kenya and South Africa) followed by Asia (India). Within Africa, programs were concentrated in the Great Lakes region, particularly Kenya, Tanzania, and Uganda, where the mobile phone market has recently expanded. 17 The purpose of these interventions included disease prevention, surveillance, self-management, treatment adherence, 17 and improved health worker behavior. 18 Despite this interest, academic research in the field is minimal and mixed, 19,20 particularly in Africa. 21 Studies are mainly single observational data points rather than multinational or regional research. 22 The purpose of this article is to examine whether SMS patient data entry with text message reminders influenced CHW follow-up visits compared with paper data entry. Our study uniquely shows evidence from multiple MVP country sites across sub-Saharan Africa.

INTERVENTION
The MVG-Net system's purpose was to gather health information in order to provide a feedback mechanism to inform patient management. 7 Standard mobile phones with SIM cards were provided to CHWs. 8 These phones included the ChildCountþ platform, an open-source system built on Django web framework and Python language. 8 CHWs used these phones to send SMS formatted texts to register patients during home visits and enter their individual and household data into a central database. 8 Household heads, children under 5 years, and all pregnant women in a household were registered using a unique patient identifier (Patient ID) with a linked household identifier (Household Head ID) at the first home visit. Malnutrition screening for children under 5 years occurred at 90-day intervals. 23 After performing a nutritional screen that included checking mid-upper arm circumference (MUAC) and signs of edema for all under-5 children in a household, CHWs sent a formatted SMS text message into the ChildCountþ system. 8 The ChildCountþ system then sent a reminder text message to the CHW's phone 75 days after the previous MUAC screen. 23 If the CHW did not conduct a follow-up MUAC screen within 90 days, then the system would send weekly text reminders until follow-up was achieved. 23 CHWs using paper forms for data entry did not receive these reminder alerts. CHWs were also responsible for ad hoc visits to follow-up if a child was malnourished or after a recent discharge from clinic. These ad hoc visits were prioritized over regular follow-ups when needed.

CHW Training
Integral to MVP's focus on health systems was integrated community case management. CHWs played a large role in regular home visits and the patient referral process. CHWs were generally chosen by the communities and given initial training on community health including maternal and child health. 24 Scope of practice involved clear tasks during household visits including registering individuals, identifying sick individuals, following up on young children and pregnant women, and reporting visits to CHW supervisors. A defined management structure with regular remuneration was in place, including necessary resources such as medical supplies, mobile phones, and bicycles, among others. 24 As per the Earth Institute's Technical Task Force Report on CHW scale-up and management, the average estimated cost of funding CHWs per head per year for the covered rural population was $6.56. 25 CHWs scheduled a total of 30 home visits per week with 5 visits per day over a 6-day work week. These visits included a mix of follow-up and regular visits, which prioritized pregnant women and infants. Sick individuals were referred to the nearby primary health care facility. At the end of each month, weekly logbooks outlining each visit were presented to CHW supervisors. The supervisor analyzed visit history and redistributed caseloads as necessary. CHW supervisors were generally experienced senior CHWs with supervisory training, who then reported to CHW managers at the primary health care facility. There was an approximate minimum ratio of 150 households per CHW across MVP sites. 25

Implementation Challenges
There were 2 components to data collection involved in this study: CHWs entered patient and household data and tracked their home visits during an initial 3-to 6-month period using only paper forms followed by 3 to 6 months of using only SMS entries. During the transition from paper forms to mobile phones, data collection was validated using both paper and SMS for at least a 1-month period in order to ensure operational effectiveness and data accuracy. The date of visit on the paper forms was the actual date visited, whereas the date of visit on SMS entries was not entered but assumed to be the date of SMS transmission. There were no major server outages or SMS coverage disturbances during this study period that could have offset consistent reporting.
The purpose of this article is to examine whether SMS patient data entry with text message reminders influenced CHW follow-up visits compared with paper data entry.
For 3 to 6 months, CHWs entered data using only paper forms, followed by using only SMS entry for another 3 to 6 months but some CHWs switched between paper and SMS for the same patients. Variations in implementation did occur across sites and was likely due to unfamiliarity with the tool or CHW-related factors. These variations allowed for concomitant use of paper forms and SMS data entry outside the prescribed overlap period. CHWs switched between paper and SMS entries for follow-up visits for the same patients. This switching created 4 comparison groups for evaluation over consecutive visits: 1. SMS entry at initial visit followed by paper form at follow-up visit 2. Paper form at initial visit followed by SMS entry at follow-up visit 3. SMS entries at both visits

Paper forms at both visits
When CHWs used SMS at the first of 2 consecutive visits (groups 1 and 3), a reminder text would still be sent after 75 days. This text would not be sent when using paper forms at the first visit (groups 2 and 4).

Study Design
The study design is a retrospective observational study with follow-up over 30 months, from February 1, 2010, through July 31, 2012. The 2 comparison groups were SMS for data entry versus paper forms.

Study Sites
Four MVP sites were selected for this study: Ruhiira (Uganda), Bonsaaso (Ghana), Potou (Senegal), and Mayange (Rwanda). These sites were chosen based on the continuity of accurate data available over the study period. These sites represent a collection of villages with a total catchment population of approximately 155,740 people. 6 Ruhiira is the largest of the 4 sites with 8 villages and more than 10,000 households (Table 1). Bonsaaso is a group of 6 villages with about 5,700 households. Potou, similarly, has 6 villages with approximately 3,200 households. Lastly, Mayange is made up of 4 villages with about 5,500 households.

Study Population and Inclusion Criteria
The study population across the 4 sites included children aged 6-60 months 26 who received an MUAC screen from a CHW between February 1, 2010, and July 31, 2012. Only patient data entries that could be linked to MUAC readings were included. Participants with missing MUAC readings or missing information on type of data entry (SMS or paper) were excluded. Children with MUAC screens who turned 5 during the course of follow-up were included. Children under 6 months of age were excluded because they do not receive regular MUAC screens as per WHO growth monitoring standards. 27

Study Follow-Up Period
Appropriate follow-up by CHWs was defined as an MUAC reading within 90 days from previous measurement (during the relevant age range). Prior research demonstrated that MUAC measurements were independently comparable predictors of child mortality compared with weight-for-age z score indices. 28 Risk of childhood mortality was shown to significantly increase when MUAC measurements fell below 115 mm and it is one of the WHO diagnostic criteria for severe acute malnutrition. 26, 29 The 90 days maximum for followup MUAC readings was chosen since MUAC measurements were previously shown to predict mortality within 30-and 90-day intervals. 30

Exposure Variable
The exposure variable was a dichotomous variable representing MUAC data entries using either SMS (1) or paper forms (0) for children aged 6-60 months over the entire study period.

Outcome Variable
The outcome variable captured each follow-up MUAC entry after the first MUAC entry for a unique child. The outcome variable was coded as a dichotomous variable: proportion of followup MUAC entries within 90 days (1) or after 90 days (0).

Covariates
Six explanatory variables were selected a priori and included age of child, gender of child, age of household head, gender of household head, number of children per household, and number of children per CHW. Each of these covariates was created as a continuous variable initially. In order to assess subgroup differences, the variables were then transformed into categorical variables.

Data Selection
The method of selecting study participants was non-randomized; there was no participant sampling because the entire study site was evaluated. All children between 6-60 months who received a household visit between February 1, 2010, and July 31, 2012, and an MUAC assessment at any point in this duration were automatically enrolled.

Power and Sample Size Calculation
We conducted post-hoc power and sample size calculations to guide interpretation of our findings (Supplement 1). 31 We used 5% alpha level and a 2-sample comparison of proportion of timely follow-up visits between data and SMS entry.

Data Analysis
Household-level data were reformatted and merged to patient-level data by matching Household Head IDs with Patient IDs in order to group patients who belonged to a unique household together. Duplicates were removed from each variable separately before merging all observations together by Patient ID. Chi-square tests at 5% significance level were conducted between exposure and outcome variables, and also between all covariates and exposure and outcome variables. Logistic regression was used to examine the strength of the association between exposure and outcome in both crude and adjusted models. All covariates were included in the multivariate regression models regardless of associated P values. The final adjusted multivariate model included analysis of the switch between paper and SMS entries to elicit whether using SMS even at 1 visit had a benefit in timely follow-up due to the 75-day text reminder. This relationship between type of data entry switching and proportion of timely visits was coded as following:

Patient Confidentiality and Ethics
All MVG-Net personnel received training on patient confidentiality and data access. Only health care providers had access to primary identified data. The data were de-identified and stored in the OpenMRS system using secure, confidential, password-protected means. The de-identified data were retrospectively accessed by the research analysis team at the Earth Institute at Columbia University. All de-identified retrospective raw data are available in  This study was approved by the Columbia University Institutional Review Board (IRB) under protocol number IRB AAAF1647.

Descriptive Statistics
The number of children with MUAC data entries ranged across the study sites from 1,970 in Ghana, to 10,256 in Uganda ( Table 2). The median number of under-5 children per household was 1 in Ghana and Rwanda, and 2 in Uganda and Senegal. CHWs in Uganda and Senegal saw more children (median of 177 and 162, respectively) than CHWs in Ghana and Rwanda (median of 60 and 35, respectively).

Bivariate Analyses
Two-sided bivariate analyses with chi-square tests of SMS versus paper entry including all covariates was conducted at 5% level of significance ( significant association with the outcome except gender of the child and of the household head. The percentage of follow-up visits occurring within 90 days by SMS entry and paper forms is displayed in Table 4. In all sites, a greater proportion of follow-up visits occurred within 90 days when CHWs entered data through SMS versus paper forms. For example, in Uganda 92% of follow-up visits occurred within 90 days when data were entered through SMS compared with 78% when data were entered using paper forms. Similarly, in Ghana the percentages were 85% and 46%, respectively. Associated chisquare tests demonstrated P values <.001 across all sites.

Ruhiira, Uganda
At the Uganda site, there was a strong positive crude association between SMS data entry and timely MUAC follow-up visits within 90 days of

DISCUSSION
The results of our study show a clear positive association between SMS data entry with reminder alerts and timely CHW follow-up for malnutrition screening visits. This finding is consistent with other studies that show improvement in case reporting, follow-up, and treatment by CHWs using SMS text reminders. Our study focused on the process indicator of proportion of timely malnutrition follow-ups using SMS versus paper data entry. This analysis does not comment on what program evaluation literature describes as outcome or impact indicators to describe adequacy of child nutritional support. 32 Mitchell et al. recently published the endline evaluation of 40 outcomes across 10 MVP-scaled sites in categories including nutrition and child health. 5 We refer readers to this evaluation for end outcomes that are overall positive for the study duration. Across all study sites, the association with timely follow-up was strongest when SMS data entry was used consecutively over multiple visits compared with switching between SMS and paper entry. Using SMS entry at the first of 2 consecutive visits still showed benefits, likely due to the reminder alerts sent at 75 days. The poorer results in some sites are difficult to explain without qualitative field data. Future qualitative research including interviews with CHWs and site administrative staff would help to explain differences in SMS implementation. This knowledge would help improve implementation in the future and Abbreviations: CI, confidence interval; OR, odds ratio; SMS, short message service.
Between all 4 study sites, the association between SMS vs. paper entry and timely follow-up was strongest for Rwanda.
Across all study sites, the association with timely follow-up was strongest when SMS data entry was used consecutively than when CHWs switched between SMS and paper. prevent switching between SMS and paper entry.
Our analysis reflects what occurred on the ground across sites during program implementation and shows benefit of using SMS consistently over follow-up visits.
Additionally, SMS use may be largely dependent on individual CHW characteristics and patient demographics. In our study, CHWs were more likely to use SMS entry when there were more than 200 children per CHW (Senegal and Uganda). Similarly, other factors may influence CHW adherence to mobile phones including CHW age, gender, and training duration. Future analyses could explore patient and CHW characteristics associated with SMS use behavior.

Limitations
According to post-hoc power calculations, the study was highly powered across all 4 sites (Supplement 1), which could reflect the large effect sizes. A multilevel regression model combining all 4 sites was initially considered, but this analysis was underpowered and difficult to achieve due to variations in site implementation. The lack of a clear comparison group in this study makes it difficult to control for unknown factors that influence patient follow-up. Without qualitative data, it is difficult to know why CHWs sometimes switched between paper and SMS data entry. This switching could occur due to various reasons such as poor cellular reception, short battery life, or lack of adequate training. More qualitative data about the study sites may help understand why Uganda had such a low proportion of SMS use compared with Ghana, Rwanda, and Senegal. There were no reported power outages or natural reasons causing poor SMS use in Uganda, but it is possible that an administrative challenge led to disruptions in service. Delays in follow-up visits could also be related to difficult and hilly terrains specifically in Ruhiira, Uganda, impacting both SMS and paper entry groups.

CONCLUSION
Using SMS for patient data entry with reminder alerts led to more timely CHW follow-up visits for malnutrition screening across multiple countries. The study highlights the importance of SMS technology in improving community-based health care delivery in low-income countries.