Skip to main content

Main menu

  • Content
    • Current Issue
    • Advance Access
    • Archive
    • Supplements
      • The Challenge Initiative Platform
      • Call for Abstracts
      • The Responsive Feedback Approach
    • Topic Collections
  • For Authors
    • Instructions for Authors
    • Submit Manuscript
    • Publish a Supplement
    • Promote Your Article
    • Resources for Writing Journal Articles
  • About
    • About GHSP
    • Editorial Team
    • Advisory Board
    • FAQs
    • Instructions for Reviewers
  • Webinars
    • Local Voices Webinar
    • Connecting Creators and Users of Knowledge
    • Publishing About Programs in GHSP
  • Other Useful Sites
    • GH eLearning
    • GHJournal Search

User menu

  • My Alerts

Search

  • Advanced search
Global Health: Science and Practice
  • Other Useful Sites
    • GH eLearning
    • GHJournal Search
  • My Alerts

Global Health: Science and Practice

Dedicated to what works in global health programs

Advanced Search

  • Content
    • Current Issue
    • Advance Access
    • Archive
    • Supplements
    • Topic Collections
  • For Authors
    • Instructions for Authors
    • Submit Manuscript
    • Publish a Supplement
    • Promote Your Article
    • Resources for Writing Journal Articles
  • About
    • About GHSP
    • Editorial Team
    • Advisory Board
    • FAQs
    • Instructions for Reviewers
  • Webinars
    • Local Voices Webinar
    • Connecting Creators and Users of Knowledge
    • Publishing About Programs in GHSP
  • Alerts
  • Visit GHSP on Facebook
  • Follow GHSP on Twitter
  • RSS
  • Find GHSP on LinkedIn
Original Articles
Open Access

SMS versus voice messaging to deliver MNCH communication in rural Malawi: assessment of delivery success and user experience

Jessica Crawford, Erin Larsen-Cooper, Zachariah Jezman, Stacey C Cunningham and Emily Bancroft
Global Health: Science and Practice February 2014, 2(1):35-46; https://doi.org/10.9745/GHSP-D-13-00155
Jessica Crawford
aVillageReach, Balaka, Malawi
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: jessica.crawford@villagereach.org
Erin Larsen-Cooper
bVillageReach, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Zachariah Jezman
aVillageReach, Balaka, Malawi
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stacey C Cunningham
cVillageReach, Seattle, WA, USA. Now with Ipas, Chapel Hill, NC, USA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Emily Bancroft
bVillageReach, Seattle, WA, USA
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
PreviousNext
  • Article
  • Figures & Tables
  • Info & Metrics
  • Comments
  • PDF
Loading

Mobile SMS health messages had higher successful delivery and led to higher intended or actual behavior change among subscribers than voice messages. Providing multiple delivery modalities led to greater overall access.

Abstract

Objective: To determine the difference in delivery success of health messages delivered through pushed SMS, pushed voice messages sent to personal phones, and voice messages retrieved from a community phone (“retrieved voice messaging”), as well as the difference in quality of the user experience.

Methods: We analyzed the project's electronic monitoring data between September 2011 and June 2013, including demographics, enrollment data, and messages sent and successfully delivered. We also collected and analyzed information from quarterly phone-based surveys with users to assess quality of the user experience, including acceptability, comprehension, new information learned, and reported behavior change.

Results: More than half of subscribers enrolled in the retrieved voice messaging service while nearly one-third enrolled in the pushed SMS service and less than 10% in pushed voice messaging. Message delivery success was highest among pushed SMS subscribers and lowest among retrieved voice subscribers. Overall, 99% of survey respondents reported trusting messages they received, and about 75% of respondents recalled the last message they received and learned something new. Almost 75% of respondents reported that they had already changed or intended to change their behavior based on received messages. Intended or actual behavior change was significantly higher among pushed SMS enrollees than among pushed or retrieved voice messaging enrollees (P = .01).

Conclusion: All message modalities led to high levels of satisfaction, comprehension, and new information learned. Due to lower cost, higher delivery success, and higher levels of intended or actual behavior change, SMS is the preferred delivery modality. However, the majority of users included in this study did not have access to a personal phone, and retrieved voice messages provided an opportunity to access a population that otherwise could not be served. Providing multiple methods by which users could access the service was crucial in extending reach beyond literate personal phone owners.

BACKGROUND

Despite significant progress toward improving maternal and child health, Malawi still has high maternal and infant mortality; the maternal mortality ratio is 675 maternal deaths per 100,000 live births and the under-5 mortality rate is 112 deaths per 1,000 births.1 Underlying causes of poor health for women and children include limited availability of timely and reliable health information for decision-making and poor access to and use of health facilities.2 Knowing when and where to go for care is integral to maximizing health care access and use and to reducing maternal and child mortality.

With increased availability and use of mobile technology, mHealth is becoming a widely used strategy to address barriers to accessing health information and care.3–5 mHealth is the use of mobile phones to promote healthy behavior, increase use of health services, improve adherence to health advice, and increase access to health information. One growing application of mHealth is mobile messaging, whereby health information and promotion messages are sent directly to clients. Previous studies have highlighted ways in which mobile messaging can be successful in smoking cessation, weight loss, diet and physical activity, treatment adherence, and disease management.6,7

Mobile health messaging has been successful in public health programs, such as for smoking cessation and weight loss.

Mobile messaging is increasingly being applied to improve reproductive, maternal, neonatal, and child health (RMNCH), and there is growing evidence of its effectiveness.8–11 For example, a U.S.-based program, text4baby, sent text messages to traditionally underserved pregnant women encouraging them to adopt healthy attitudes, beliefs, and behaviors during pregnancy. The program was effective in increasing agreement with the statement, “I am prepared to be a new mother.” However, it did not lead to significant behavior change among enrollees.10 The effects of the intervention were greater among enrollees with more education, suggesting that literacy and comprehension of messages may play crucial roles in the effectiveness of mHealth projects.

Literacy may be a critical factor in the effectiveness of mHealth projects.

An mHealth project in Zanzibar called Wired Mothers sent pregnant women text messages with appointment reminders and health education information, and it also gave enrollees phone vouchers to call health care providers directly with questions. The project significantly increased facility-based births among urban women but had no significant effect on rural women.11

In Thailand, the “Better Border and Healthcare Program” found that sending users text messages regarding antenatal care visits and immunizations increased the number of mothers and children attending scheduled appointments.12

Mobile messages are generally delivered by either short message service (SMS) or voice messaging; both offer distinct advantages and disadvantages. SMS is available on an estimated 98% of mobile phones, does not require technical expertise to use, and is adaptable to multiple mHealth purposes.13 SMS messages can be accessed at user convenience and can be delivered to phones that are turned off or have flat batteries. In addition, telecom costs for SMS are generally less expensive than costs for voice communication. Given these advantages, it is not surprising that SMS is the most common delivery method for mobile development services.13

Voice messages, although used less often in mHealth projects, offer the advantages of accessibility to illiterate populations, ability to contain more information per message than SMS, and ability to be recorded in any language (not all languages/characters are supported by SMS).14 In addition, recorded voice messages allow information to be conveyed by a clinical “character,” as used in Bangladesh by the Mobile Alliance for Maternal Action (MAMA), to build rapport and trust over time.15

SMS has several advantages over voice communication, but voice messages may provide accessibility to illiterate populations.

Research comparing the outcomes of voice versus SMS messaging is limited in the mHealth literature, although user preference has been examined in a small number of studies. A U.S.-based study among literate English speakers found that 72% of participants in an mHealth program targeting exercise, diet, and smoking preferred to enroll in automated voice messages over SMS. Those who preferred SMS tended to be younger and to have higher levels of comfort with computers and higher levels of SMS use.16

The Mobile Technology for Community Health (MOTECH) initiative in Ghana used a “Mobile Midwife” application to provide pregnant women with health education and reminders to access necessary medical services. Messages could be received in voice or SMS format, depending on the enrollees' preference; 99% of enrollees chose voice.17

Questionnaires administered to potential mHealth users in Argentina, however, found that the users were similarly open to receiving SMS and voice messages during pregnancy, with 96% of participants willing to receive SMS messages and 87% willing to receive voice messages.18 To our knowledge, no studies have been conducted to determine the difference in acceptability, comprehension, or behavior change between users receiving messages through voice compared with SMS formats.

This article describes the messaging component of an mHealth pilot project in Malawi aimed at improving knowledge and uptake of home- and facility-based MNCH care practices using both SMS and voice message delivery modalities. We report findings from the pilot examining differences between SMS and voice messaging with regard to delivery success and quality of user experience, including acceptability, comprehension, new information learned, and reported behavior change.

PROJECT DESCRIPTION

Chipatala Cha Pa Foni (CCPF) (“Health Center by Phone”) is a mobile health project of Concern Worldwide's Innovations for Maternal, Newborn & Child Health initiative, and its partners VillageReach and the Malawi Ministry of Health (MOH). A nationwide campaign generated more than 6,000 ideas of how to improve quality of health services from community members in Malawi; one of the winning ideas became CCPF.

Figure
  • Download figure
  • Open in new tab
  • Download powerpoint

A communication poster encourages caregivers of children to call the Chipatala Cha Pa Foni hotline to get answers to their health-related questions.

CCPF consists of a toll-free hotline offering protocol-based health information, advice, and referrals as well as a mobile messaging service offering automated tips and reminders for pregnant women, guardians of young children, and women of childbearing age. Starting in 2011, CCPF was piloted in 4 rural health center catchment areas in Balaka District in Southern Malawi, representing a population of 32,000 women of childbearing age, 24,000 children under 5, and 7,000 expected pregnancies per year. The pilot project aimed to improve the coverage and quality of RMNCH in Balaka District. Early success and interest has resulted in scale up to an additional 3 districts.

The hotline component of CCPF is staffed by individuals trained in RMNCH using modules from the MOH's curriculum for community health workers (CHWs). When answering calls, hotline workers are prompted by software on touch screen devices to identify clients' symptoms and/or information needs. They answer questions, offer health advice, and provide information on when and where to seek care if clients have symptoms or danger signs that cannot be treated safely at home.

Figure
  • Download figure
  • Open in new tab
  • Download powerpoint

A hotline worker answers an incoming calling using a touch screen device that prompts the worker to identify the client's symptoms and information needs.

Hotline workers also offer eligible clients the opportunity to enroll in the tips and reminders mobile messaging service where they can access voice or SMS messages on a weekly basis. Messages offer timely information and reminders about important health services, based on the estimated date of delivery for pregnant women or the child's age for caregivers of children under 1. The tips and reminders are intended to elicit behavior change by helping clients understand their susceptibility to and seriousness of common maternal and child health illness, and they provide cues to action, reminding pregnant women and caregivers of children to seek timely preventive care. Messages for women of childbearing age were added later in implementation.

During project design, we found that low literacy and phone ownership among the target population may limit access to the service. Although an estimated 75% of women over 15 years old in Balaka District are literate,1 poor women and women living in rural areas are less likely to be literate than their wealthier and urban counterparts.19 Furthermore, a baseline assessment found that only approximately one-third of women of childbearing age in the catchment area owned a mobile phone.20 Thus, we developed 3 message delivery modalities in order to better accommodate individuals with low literacy or without access to personal phones:

  1. Pushed SMS Messages: Clients with access to a personal or household mobile phone can automatically receive SMS messages on their mobile phones.

  2. Pushed Voice Messages: Low-literacy clients with access to a personal or household phone can receive weekly recorded voice messages that are sent to the phone at a specific time each week.

  3. Retrieved Voice Messages: Clients without access to a personal phone can access their weekly messages by calling the toll-free hotline from any phone and entering an appropriate code to hear their message.

Voice messages are more expensive to deliver so clients with personal phones who identified as being literate were generally encouraged to register for SMS. Low-literacy clients were offered the option of registering for voice messages. Prior to launch, each message delivery option was pilot tested among a small group of users in Balaka to ensure delivery and accessibility.

Due to low phone ownership in Balaka, we provided an additional point of access to CCPF. We originally planned to distribute phones to Malawi's paid cadre of CHWs. However, given the large number of potential users without access to a personal phone, community volunteers from each village were instead recruited in consultation with traditional leaders in the area and provided with a low-cost phone. The volunteers were oriented to the program and provided with small incentives throughout the pilot period to encourage their continued participation. In order to provide phones to all volunteers, we sought partnerships that would allow us to stretch the budget to provide phones to a larger number of people than originally anticipated. Our telecom partner agreed to provide a discount on their basic low-cost promotional phones, which were ultimately purchased for the project.

Figure
  • Download figure
  • Open in new tab
  • Download powerpoint

A woman uses a community volunteer's mobile phone to access the health message delivered by the Chipatala Cha Pa Foni project.

Message Development and Content

Messages were developed in collaboration with MOH staff with different areas of expertise, with technical support from PATH, the Grameen Foundation, and BabyCenter. The content was designed to complement national policies and to address local myths and traditions around pregnancy and child rearing. The final content was reviewed and revised by a group of women from Balaka with experience working on RMNCH issues at the community level. Messages were initially developed in English and later translated to Chichewa and Chiyao, the primary local languages spoken in Balaka.

The final content was the same for both voice and SMS messages. However, SMS messages were delivered more frequently than voice messages because SMS messages were shorter than voice messages. Messages were delivered once or twice per week depending on the stage of pregnancy or age of the child. Table 1 provides examples of SMS and voice messages.

SMS messages were delivered more frequently than voice messages because they were shorter than the voice messages.

View this table:
  • View inline
  • View popup
TABLE 1. Sample SMS and Voice Messages

Content was reviewed periodically and updates were made to include new information. For example, new information was added in mid-2012 about the launch of a new pneumococcal vaccine in Malawi.

Originally, CCPF relied on volunteers to advertise the service in the community. Later, CCPF staff started to attend antenatal care clinics to inform pregnant women about CCPF and to help them sign up for the service. In addition, large community events were held to inform community members about CCPF and how to access the service.

DATA AND METHODS

To understand the difference in delivery success and quality between the 3 different messaging modalities (pushed SMS, pushed voice messages, retrieved voice messages), we collected and analyzed information from routine monthly monitoring data and quarterly phone-based surveys with users.

Routine Monthly Monitoring

Data from the messaging service were monitored monthly from 3 technology sources built to deliver the service:

  • Hotline Software. Built by Baobab Health, a Malawian nongovernmental organization providing eHealth solutions, the hotline software is protocol-based and guides hotline workers through each call according to information collected through interactions with the client. The software includes demographic information and tips and reminders enrollment.

  • Notification Application. Built by VillageReach, the application connects the hotline software to a communications server and pushes voice and SMS messages to enrollees with personal phones. This application records data on messages sent and successfully delivered.

  • INTELLIVR (IVR) Software. Built by Yo! Uganda, a software company in Uganda, IVR hosts incoming calls and also serves as the outgoing-voice gateway for the tips and reminders service.

Monthly data for all enrolled clients from September 2011 through June 2013 were analyzed to describe the number and characteristics of enrollees by delivery modality as well as delivery success rates.

Quarterly Phone-Based Surveys

Starting roughly 3 months following the launch of CCPF, periodic phone-based surveys were conducted among a sample of enrollees. Clients enrolled in pushed SMS and voice messaging services were randomly selected from a list of all enrollees and were contacted directly using the phone number for message delivery. Because we did not have phone numbers for clients enrolled in the retrieved voice message service, random selection was not feasible. Instead, we sent community volunteers an SMS asking them to identify a woman in their area who had used the service and to notify hotline staff once they gave the community phone over to the client. At that time, the client was contacted directly for the survey.

For all participants, a hotline worker or volunteer administered the questionnaire, which collected information on frequency of messages received and quality of the experience. Quality of user experience was measured as:

  • Acceptability: proportion of respondents who indicated they trust the messages

  • Comprehension: proportion of respondents who could describe the last message they received; the description given by the respondent was then checked in the analysis phase against the messages the respondent should have received

  • New information learned: proportion of respondents who indicated they learned new information about their pregnancy or child's health and could list topics they deemed to be new information

  • Reported behavior change: proportion of respondents indicating a health behavior change or intent to change behavior based on the messages received. Behavior change intent was scored according to a series of questions regarding specific behaviors targeted in the messaging. For pregnant women, these behaviors include number of antenatal care visits, food and medicine consumption, and place of delivery. For caregivers of children, these behaviors include medicine and vaccinations for the child, exclusive breastfeeding, and number of postnatal care visits.

The survey was implemented at 3, 6, 12, and 18 months following the launch of the service. Data from 266 questionnaires were entered into an Excel spreadsheet and analyzed using STATA version 10.

RESULTS

Routine Monthly Monitoring

During the first 2 years of implementation (through June 2013), more than 5,000 pregnant women and caregivers of children under 1 registered for the tips and reminders service. Client registration data were collected starting in July 2011, when clients could begin registering for the service, while monitoring of delivery success rates started in September 2011 when the messaging service was officially launched.

Table 2 displays the number of registered clients and demographics by content type and delivery method. More than half of clients enrolled in the retrieved voice service, nearly one-third enrolled in the pushed SMS service, and less than 10% enrolled in the pushed voice message service. The mean age of pregnant enrollees was 25, and they registered, on average, 16 weeks before their estimated due date. Caregivers of children under 1 registered, on average, 36 weeks before their child's first birthday.

The majority of subscribers enrolled in the “retrieved voice messaging” service because they did not own a personal phone.

View this table:
  • View inline
  • View popup
TABLE 2. Demographics of Registered Users, by Subscriber Type and Delivery Method (July 2011–June 2013)

Characteristics of clients were similar across service types with the exception of occupation. Pregnant women who registered for pushed SMS messages were more likely to be employed in the formal sector than those who registered for (pushed or retrieved) voice messages. Formal sector includes those who reported their occupation as “business,” “health care worker,” or “teacher.” Informal sector includes those who reported their occupation as “farmer,” “housewife,” “student,” or “other.”

The proportion of messages successfully pushed and retrieved varies by month. For pushed messages (SMS and voice), success rates represent the number of messages successfully received after 3 attempts divided by the number of messages attempted. For retrieved voice messages, success rates represent the number of messages successfully retrieved from the IVR menu divided by the number of messages we estimate should have been retrieved each month. Overall success rates for pushed SMS and voice messages ranged from 54% to 64% and were higher for pushed SMS than for pushed voice messages (Table 3). Success rates for retrieved voice messages were lower, at 27% to 38%.

The SMS service had higher delivery success rates than the pushed or retrieved voice messaging services.

View this table:
  • View inline
  • View popup
TABLE 3. Delivery Success of Messages, by Delivery Method and Subscriber Type (September 2011–June 2013)

Delivery success changed over time, with the proportion of pushed voice messages successfully delivered to a personal phone increasing slightly over time, and the proportion of retrieved voice messages successfully delivered decreasing over time (Figure 1). Pushed SMS delivery success rates remained relatively stable over time, with a decline in the beginning of 2013, as a result of a number of technical issues. For example, configuration changes by the telecom provider unintentionally reduced the number of times our system attempted to send messages, resulting in messages being attempted less than 3 times. The issue has since been rectified, but it affected success rates for January–May 2013.

FIGURE 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 1.

Delivery Success Rates of Messages Over Time, by Delivery Method, September 2011–June 2013

Success rate for pushed SMS and voice messages is the percent of messages successfully received within 3 attempts; for retrieved voice messages, the success rate is the percent of expected messages successfully retrieved.

To better understand voice message retrieval rates, we also calculated the proportion of messages successfully played based on the number of attempts (Figure 2). This proportion is calculated from the number of times the message menu is reached (that is, a person calls CCPF's toll-free number and enters “2” to hear a message and then “1” to hear a pregnancy message) and how many times messages are played. Using this definition of delivery success for retrieved voice messages, messages were successfully played in 56% of the instances when the message menu was reached, and success was slightly higher for messages played from the child message menu than messages played from the pregnancy message menu (60% versus 54%).

FIGURE 2.
  • Download figure
  • Open in new tab
  • Download powerpoint
FIGURE 2.

Percentage of Successful Message Retrieval Attempts Among Retrieved Voice Message Subscribers, by Type of Subscriber, September 2011–June 2013

Phone-Based Surveys

Hotline workers and volunteers were able to reach 266 total clients for the phone survey. Of those, 96 were enrolled in the pushed SMS service, 30 in the pushed voice service, and 140 in the retrieved voice service (Table 4).

View this table:
  • View inline
  • View popup
TABLE 4. Phone-Based Survey Sample Characteristics, by Delivery Method

Slightly more clients enrolled to receive messages about child health than about pregnancy were included in the survey. At the time of the survey, SMS enrollees reported the highest number of messages received, at an average of 6.9 messages. Pushed voice and retrieved voice enrollees included in the survey had received an average of 3.1 and 3.7 messages, respectively, at the time of survey. Overall, 19% (51) of respondents indicated that they had not yet received any messages, the majority of which (29) were among pushed voice enrollees.

Overall, 99% of respondents reported trusting messages they received (acceptability), and 75% recalled the last message they received (comprehension) (Table 5). Pushed SMS and retrieved voice enrollees were more likely than pushed voice enrollees to recall the last message received, but the observed differences were not statistically significant.

View this table:
  • View inline
  • View popup
TABLE 5. Quality of User Experience, by Delivery Method

Just over three-quarters of enrollees indicated that they had learned something new from the messaging service and were able to describe the new information. Pushed SMS enrollees were more likely to respond that they had learned something new.

Almost three-quarters of respondents reported that they had already changed or intended to change their behavior based on the messages they received. Pushed SMS enrollees were significantly more likely to report intended or actual behavior change than voice enrollees (P = .01). See box for examples of behaviors that respondents said they intended to change.

SMS subscribers were significantly more likely to report intended or actual behavior change than pushed or retrieved voice messaging subscribers.

BOX. Quality of User Experience: Selected Examples Reported by Survey Respondents

Recalled messages:

  • I should breastfeed my child up to 6 months without giving supplementary foods.

  • If the child has shown signs of drying in the mouth, I should run to the hospital.

New information:

  • I have learnt how to solve small problems without going to hospital.

  • The child is supposed to receive measles vaccine.

Intended or actual behavior change themes:

  • The number of antenatal care visits I plan to attend.

    The types of foods I am eating.

    The age I will give my baby food other than breast milk.

    How often my child sleeps under a bed net.

DISCUSSION

Understanding the effects of message modality is important when deciding between SMS or voice message services in mHealth programs. Although SMS is a common and effective method, voice messages may be a desirable option based on literacy levels and mobile phone access.

The data presented in this article reveal that delivery success rates for all message modalities are less than ideal. Phone network challenges in rural Balaka were commonly reported by community volunteers and clients throughout the pilot period and likely contributed to the overall low success rates.

Message delivery rates are far more successful among SMS than voice enrollees. Pushing voice messages to clients with personal phones is a complex process, requiring the client to answer the phone at the time of delivery, whereas SMS messages can be delivered at any time, including when the phone is turned off. We believe this is the major reason that SMS messages pushed to personal phones have a higher delivery success rate than voice messages.

During the first few months of implementation, we varied the times that we pushed voice messages to users to find times that clients were most likely to be available to answer the phone. When tips and reminders were launched, voice messages were attempted hourly between 12:00 pm and 6:00 pm local time (CAT) because anecdotal reports indicated that phones were most likely to be charged and switched on during daylight hours. In November 2011, we pushed the voice messages between 2:00 pm and 6:00 pm because we noted more successful attempts occurred during that timeframe. In early 2012, we discovered a configuration problem between the telephone card and the telecom provider that was also contributing to the low delivery success rate. After changing the timing that we pushed the voice messages and fixing the configuration problem, delivery success rates for pushed voice messages increased, but the delivery success rate remained highest for pushed SMS messages.

Retrieved voice messages have the lowest delivery success rate, and this rate decreased over time. Retrieving voice messages from the IVR menu using a community phone requires initiative on behalf of the client who must seek out a volunteer in order to access the community phone. In addition, it requires the volunteer to be available. There were some instances where volunteers moved away from their assigned village or became inactive over time. It is also possible that some community volunteers acted as gatekeepers to the service. We heard anecdotal reports of a small number of volunteers refusing to give out the CCPF telephone number or telling clients that the service must be accessed through a community phone rather than a personal phone.

Furthermore, many community phones broke during the project period, limiting access to the service for community phone users. The phones given to community volunteers were low cost, and approximately 60% of the phones distributed to volunteers were not working 2 years after the original implementation began. It is likely that the unreliability of community phones played a role in the low delivery success of retrieved voice messages in 2 ways:

  • Malfunctioning keypads made it difficult to enter an accurate estimated due date or child's birthday in order to hear a message. This is a major reason stated by both volunteers and users for why numerous calls that reached the voice message menu never successfully heard a message.

  • Clients who signed up for voice messages in a village where the phone eventually broke were unlikely to find another way to access the service. Some users stated that after the community phone in their village broke, they no longer accessed the CCPF hotline or voice messages.

The data also reveal that delivery success for retrieved voice messages is significantly higher among users subscribed to pregnancy messages than users subscribed to child messages (based on the definition of success used in Table 3). Pregnant users registered an average of 16 weeks before their estimated due date while caregivers of children registered an average of 36 weeks before their child's first birthday. It is possible that the average pregnancy-message enrollee was more likely to take the initiative to visit a community volunteer because the eligibility period was shorter than for the average child-message enrollee. Similarly, it is less likely that a community volunteer would become inactive or the community phone would break during a shorter eligibility period. For pushed message enrollees, there is no clear relationship between message type and success rates because delivery success of voice messages was significantly higher among pregnancy-message enrollees than child-message enrollees, but the opposite was true for pushed SMS messages.

Although delivery success was higher among pushed SMS messages compared with other delivery modalities, the data reveal that clients were highly satisfied with and trusted the voice and SMS messages equally and were likely to learn new information and recall messages regardless of message delivery modality. SMS enrollees, however, were significantly more likely to report intended or actual behavior change than pushed voice enrollees. We suspect that this may be because voice messages can only be heard once while SMS messages can be saved in the client's inbox and read multiple times, and even shared with others. It is also possible that shorter and more frequent messages in the form of SMS are more effective at eliciting intended behavior change than longer voice messages that are delivered once per week. Finally, the differing characteristics of individuals who sign up for SMS messages might make them more willing or able to change their behavior.

Shorter and more frequent messages in the form of SMS may be more effective at prompting behavior change than longer voice messages.

At baseline, this mHealth project discovered that most women in the project area did not own a mobile phone. To address this problem, we provided an additional access point by providing community volunteers with mobile phones. Although the solution was not perfect, our study demonstrates that it is feasible to provide low-literate clients without a personal phone with mobile health information and to stimulate behavior change. Although delivery success was the lowest for retrieved voice message subscribers, approximately 1,200 voice messages were successfully retrieved each month. A similar number of pushed SMS and voice messages were successfully delivered each month. This finding emphasizes the importance of recognizing that mobile health messaging can potentially marginalize populations without access to mobile phones if efforts are not made to adapt mHealth projects to meet the needs of the population. Not surprisingly, phone ownership is inversely correlated with poverty,21 and women and people living in rural areas are also less likely to own mobile phones than their male and urban counterparts.19 Thus, offering a retrieved voice message option for those without personal phones is an important strategy to deliver health messages to an otherwise hard-to-reach population.

Limitations

There are a number of important limitations to consider when interpreting the data. First, data from the hotline software (such as demographic data and tips and reminders enrollment status) were entered by hotline workers during calls with clients, and mistakes were common. Implausible data points, such as estimated due dates that were more than 9 months from the call date, had to be excluded. In addition, the data available from the IVR software are by call and cannot be grouped by unique caller, making it difficult to draw conclusions about individual behavior.

Second, our calculation for the number of messages retrieved by voice message subscribers provides only an estimate and is not completely analogous to the success rate for pushed messages. Pushed message success rates are based on 3 attempts while the retrieved message success rate is based on an unknown number of attempts since a subscriber can attempt to retrieve a message an unlimited number of times. Furthermore, some subscribers likely do not try at all. The proportion of message retrieval attempts where a message is successfully played (Figure 2), however, is based on only 1 call attempt and cannot tell us about individual behavior. For example, if 50% of message retrieval attempts are successfully played, we cannot determine whether this translates to 10 unique callers attempting to retrieve a message and only half are successful, or 5 unique callers each trying to retrieve a message twice and failing once and successful on their second try.

There were also a number of limitations to the type of study design used for the phone-based surveys. First, the characteristics of enrollees that largely determined their enrollment type, such as literacy level and access to a personal phone, could be associated with their level of trust, comprehension, and knowledge, and/or behavior change reported during the survey. Thus, any differences observed in survey responses between the different delivery methods may be attributable to characteristics of the sample and not necessarily the delivery method. In addition, among retrieved voice message enrollees, the sample was not randomly selected. Although the community volunteers did not know the purpose of their assignment to identify a client, it is possible that they chose women according to certain characteristics that may bias the sample. We did not collect enough demographic information during the phone surveys to be able to control for potential confounding characteristics. Also, information collected about the enrollees' maternal and child health knowledge and/or behavior change intent relied on self-report, which may present an additional form of bias. Because surveys were conducted by CCPF staff or volunteers, respondents may have wanted to provide answers that they felt were acceptable to the surveyor. Thus, satisfaction, trust, and behavior change indicators may be inflated. On the other hand, respondents were asked to describe their last message and new information learned. Because the information provided was cross-checked against messages sent to the respondent, these indicators are less susceptible to bias. Finally, the sample size is small, making detection of small differences in outcomes difficult.

CONCLUSION

Mobile technology, specifically SMS and voices messages, can be successfully used to extend health information services to pregnant women and caregivers of young children in rural Malawi. All 3 message modalities led to high levels of satisfaction, comprehension, and new information learned. Due to lower cost, higher delivery success, and higher levels of intent to change behavior, SMS is the preferred delivery modality when possible.

Implementing mHealth projects in areas with low phone penetration creates unique challenges in ensuring access and generating demand. By adapting the service to the context—allowing users to retrieve voice messages from the IVR system without a personal phone and providing community volunteers with phones to serve as access points to the service—this rural population gained access to the mHealth service that would not otherwise have been possible. Although these adaptations presented their own challenges, particularly apparent in the low success rate of retrieved messages, availability of community volunteers, and the difficulty sustaining the volunteer model over time as phones began to break, providing multiple methods by which users could access the service was crucial in extending reach beyond literate personal phone owners.

Further research to understand potential models for increasing access to mobile phones among low-literate, rural populations is warranted. Prior to implementing a similar service, it is important to understand the cost-effectiveness of the intervention and cost implications at scale. At the time of this study, an economic evaluation of CCPF was underway. Finally, further research examining the translation of health information transmitted through mobile phones into healthy behaviors is needed to understand the potential impact of mHealth interventions on their intended health outcomes.

Acknowledgments

This project was funded by Concern Worldwide's Innovations for Maternal, Newborn & Child Health initiative.

Notes

Competing Interests: None declared.

Peer Reviewed

Cite this article as: Crawford J, Larsen-Cooper E, Jezman Z, CunninghamSC, Bancroft E. SMS versus voice messaging to deliverMNCH communication in rural Malawi: assessment of delivery success and user experience. Glob Health Sci Pract. 2014;2(1):35-46. http://dx.doi.org/10.9745/GHSP-D-13-00155

  • Received: 2013 Oct 17.
  • Accepted: 2013 Dec 17.
  • Published: 2014 Feb 1.
  • © Crawford et al.

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/

REFERENCES

  1. ↵
    National Statistical Office (NSO) [Malawi]; ICF Macro. Malawi demographic and health survey 2010. Zomba (Malawi): NSO; 2011. Co-published by ICF Macro. Available from: http://www.measuredhs.com/publications/publication-FR247-DHS-Final-Reports.cfm
  2. ↵
    1. Combs Thorsen V,
    2. Sundby J,
    3. Malata A
    . Piecing together the maternal death puzzle through narratives: the three delays model revisited. PLoS One. 2012;7(12): e52090. doi:10.1371/journal.pone.0052090
    OpenUrlCrossRefPubMed
  3. ↵
    Vital Wave Consulting. mHealth in the global south: landscape analysis. Washington (DC): Vodafone Group Foundation; 2008. Co-published by the United Nations Foundation Partnership. Available from: http://www.foresightfordevelopment.org/sobipro/download-file/46-448/54
    1. Cole-Lewis H,
    2. Kershaw T
    . Text messaging as a tool for behavior change in disease prevention and management. Epidemiol Rev. 2010;32(1): 56–69. doi:10.1093/epirev/mxq004
    OpenUrlAbstract/FREE Full Text
  4. ↵
    1. Sloninsky D,
    2. Mechael PN
    . Towards the development of an mHealth strategy: a literature review. New York: Millennium Villages Project; 2008. Available from: http://www.who.int/goe/mobile_health/mHealthReview_Aug09.pdf
  5. ↵
    1. Free C,
    2. Phillips G,
    3. Felix L,
    4. Galli L,
    5. Patel V,
    6. Edwards P
    . The effectiveness of M-health technologies for improving health and health services: a systematic review protocol. BMC Res Notes. 2010;3:250. doi:10.1186/1756-0500-3-250
    OpenUrlCrossRefPubMed
  6. ↵
    1. Free C,
    2. Phillips G,
    3. Galli L,
    4. Watson L,
    5. Felix L,
    6. Edwards P,
    7. et al
    . The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med. 2013;10(1): e1001362. doi:10.1371/journal.pmed.1001362
    OpenUrlCrossRefPubMed
  7. ↵
    1. Sharma R,
    2. Hebbal M,
    3. Ankola AV,
    4. Murugabupathy V
    . Mobile-phone text messaging (SMS) for providing oral health education to mothers of preschool children in Belgaum City. J Telemed Telecare. 2011;17(8): 432–436. doi:10.1258/jtt.2011.110416
    OpenUrlAbstract/FREE Full Text
    1. Tamrat T,
    2. Kachnowski S
    . Special delivery: an analysis of mHealth in maternal and newborn health programs and their outcomes around the world. Matern Child Health J. 2012;16(5): 1092–1101. doi:10.1007/s10995-011-0836-3
    OpenUrlCrossRefPubMed
  8. ↵
    1. Evans WD,
    2. Wallace JL,
    3. Snider J
    . Pilot evaluation of the text4baby mobile health program. BMC Public Health. 2012;12:1031. doi:10.1186/1471-2458-12-1031
    OpenUrlCrossRefPubMed
  9. ↵
    1. Lund S,
    2. Hemed M,
    3. Nielsen BB,
    4. Said A,
    5. Said K,
    6. Makungu MH,
    7. et al
    . Mobile phones as a health communication tool to improve skilled attendance at delivery in Zanzibar: a cluster-randomised controlled trial. BJOG. 2012;119(10): 1256–1264. doi:10.1111/j.1471-0528.2012.03413.x
    OpenUrlCrossRefPubMed
  10. ↵
    1. Kaewkungwal J,
    2. Singhasivanon P,
    3. Khamsiriwatchara A,
    4. Sawang S,
    5. Meankaew P,
    6. Wechsart A
    . Application of smart phone in “Better Border Healthcare Program”: a module for mother and child care. BMC Med Inform Decis Mak. 2010;10:69. doi:10.1186/1472-6947-10-69
    OpenUrlCrossRefPubMed
  11. ↵
    GSMA Mobile for Development Intelligence (MDI). Scaling mobile for development: a developing world opportunity. London: MDI; 2013. Available from: https://d2sjbwdjcc5vgl.cloudfront.net/Scaling_Mobile_-_interim_publish_MDI_template.pdf
  12. ↵
    GSMA Mobile for Development Intelligence (MDI). Value of voice – an unsung opportunity. London: MDI; [2013]. Available from: https://wiki.gsmaintelligence.com/gsma_kb/images/2/20/Value_of_Voice_-_an_unsung_opportunity.pdf
  13. ↵
    MAMA: Mobile Alliance for Maternal Action: the power of health in every mama's hand [Internet]. Washington (DC): United Nations Foundation; c2013. MAMA Bangladesh; [cited 2013 Jan 7]; [about 3 screens]. Available from: http://www.mobilemamaalliance.org/mama-bangladesh
  14. ↵
    1. Greaney ML,
    2. Puleo E,
    3. Sprunck-Harrild K,
    4. Bennett GG,
    5. Cunningham MA,
    6. Gillman MW,
    7. et al
    . Electronic reminders for cancer prevention: factors associated with preference for automated voice reminders or text messages. Prev Med. 2012;55(2): 151–154. doi:10.1016/j.ypmed.2012.05.014
    OpenUrlCrossRefPubMed
  15. ↵
    Grameen Foundation. Mobile technology for community health in Ghana: what it is and what Grameen Foundation has learned so far. 2nd edition. Washington (DC): Grameen Foundation; 2012. Available from: http://www.grameenfoundation.org/sites/grameenfoundation.org/files/MOTECH-Lessons-Learned-Sept-2012.pdf
  16. ↵
    1. Cormick G,
    2. Kim NA,
    3. Rodgers A,
    4. Gibbons L,
    5. Buekens PM,
    6. Belizán JM,
    7. et al
    . Interest of pregnant women in the use of SMS (short message service) text messages for the improvement of perinatal and postnatal care. Reprod Health. 2012;9:9. doi:10.1186/1742-4755-9-9
    OpenUrlCrossRefPubMed
  17. ↵
    1. Wesolowski A,
    2. Eagle N,
    3. Noor AM,
    4. Snow RW,
    5. Buckee CO
    . Heterogeneous mobile phone ownership and usage patterns in Kenya. PLoS One. 2012;7(4): e35319. doi:10.1371/journal.pone.0035319
    OpenUrlCrossRefPubMed
  18. ↵
    Invest in Knowledge. Baseline report for an evaluation of the information and communications technology to improve health services for mothers and children project. Zomba (Malawi): Invest in Knowledge; 2012.
  19. ↵
    1. Noordam AC,
    2. Kuepper BM,
    3. Stekelenburg J,
    4. Milen A
    . Improvement of maternal health services through the use of mobile phones. Trop Med Int Health. 2011;16(5): 622–626. doi:10.1111/j.1365-3156.2011.02747.x
    OpenUrlCrossRefPubMed
PreviousNext
Back to top

In this issue

Global Health: Science and Practice: 2 (1)
Global Health: Science and Practice
Vol. 2, No. 1
February 01, 2014
  • Table of Contents
  • About the Cover
  • Index by Author
  • Complete Issue (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word about Global Health: Science and Practice.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
SMS versus voice messaging to deliver MNCH communication in rural Malawi: assessment of delivery success and user experience
(Your Name) has forwarded a page to you from Global Health: Science and Practice
(Your Name) thought you would like to see this page from the Global Health: Science and Practice web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
SMS versus voice messaging to deliver MNCH communication in rural Malawi: assessment of delivery success and user experience
Jessica Crawford, Erin Larsen-Cooper, Zachariah Jezman, Stacey C Cunningham, Emily Bancroft
Global Health: Science and Practice Feb 2014, 2 (1) 35-46; DOI: 10.9745/GHSP-D-13-00155

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
SMS versus voice messaging to deliver MNCH communication in rural Malawi: assessment of delivery success and user experience
Jessica Crawford, Erin Larsen-Cooper, Zachariah Jezman, Stacey C Cunningham, Emily Bancroft
Global Health: Science and Practice Feb 2014, 2 (1) 35-46; DOI: 10.9745/GHSP-D-13-00155
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Statistics from Altmetric.com

Jump to section

  • Article
    • Abstract
    • BACKGROUND
    • PROJECT DESCRIPTION
    • DATA AND METHODS
    • RESULTS
    • DISCUSSION
    • CONCLUSION
    • Acknowledgments
    • Notes
    • REFERENCES
  • Figures & Tables
  • Info & Metrics
  • Comments
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Efficacy of a Digital Health Tool on Contraceptive Ideation and Use in Nigeria: Results of a Cluster-Randomized Control Trial
  • Cell phones and CHWs: a transformational marriage?
  • Google Scholar

More in this TOC Section

  • Matching Intent With Intensity: Implementation Research on the Intensity of Health and Nutrition Programs With Women's Self-Help Groups in India
  • Baseline Assessment of Evidence-Based Intrapartum Care Practices in Medical Schools in 3 States in India: A Mixed-Methods Study
  • The Equity Tool for Valuing Global Health Partnerships
Show more ORIGINAL ARTICLES

Similar Articles

Subjects

  • Cross-Cutting Topics
    • Behavior Change Communication
    • Digital Health
  • Health Topics
    • Maternal, Newborn, and Child Health
US AIDJohns Hopkins Center for Communication ProgramsUniversity of Alberta

Follow Us On

  • Twitter
  • Facebook
  • LinkedIn
  • RSS

Articles

  • Current Issue
  • Advance Access Articles
  • Past Issues
  • Topic Collections
  • Most Read Articles
  • Supplements

More Information

  • Submit a Paper
  • Instructions for Authors
  • Instructions for Reviewers
  • GH Journals Database

About

  • About GHSP
  • Advisory Board
  • FAQs
  • Privacy Policy
  • Contact Us

© 2023 Creative Commons Attribution 4.0 International License. ISSN: 2169-575X

Powered by HighWire