SMS versus voice messaging to deliver MNCH communication in rural Malawi: assessment of delivery success and user experience

BACKGROUND

Despite significant progress toward improving maternal and child health, Malawi still has high maternal and infant mortality; the maternal mortality ratio is 675 maternal deaths per 100,000 live births and the under-5 mortality rate is 112 deaths per 1,000 births.¹ Underlying causes of poor health for women and children include limited availability of timely and reliable health information for decision-making and poor access to and use of health facilities.² Knowing when and where to go for care is integral to maximizing health care access and use and to reducing maternal and child mortality.

With increased availability and use of mobile technology, mHealth is becoming a widely used strategy to address barriers to accessing health information and care.^3–5 mHealth is the use of mobile phones to promote healthy behavior, increase use of health services, improve adherence to health advice, and increase access to health information. One growing application of mHealth is mobile messaging, whereby health information and promotion messages are sent directly to clients. Previous studies have highlighted ways in which mobile messaging can be successful in smoking cessation, weight loss, diet and physical activity, treatment adherence, and disease management.^6,7

Mobile messaging is increasingly being applied to improve reproductive, maternal, neonatal, and child health (RMNCH), and there is growing evidence of its effectiveness.^8–11 For example, a U.S.-based program, text4baby, sent text messages to traditionally underserved pregnant women encouraging them to adopt healthy attitudes, beliefs, and behaviors during pregnancy. The program was effective in increasing agreement with the statement, “I am prepared to be a new mother.” However, it did not lead to significant behavior change among enrollees.¹⁰ The effects of the intervention were greater among enrollees with more education, suggesting that literacy and comprehension of messages may play crucial roles in the effectiveness of mHealth projects.

An mHealth project in Zanzibar called Wired Mothers sent pregnant women text messages with appointment reminders and health education information, and it also gave enrollees phone vouchers to call health care providers directly with questions. The project significantly increased facility-based births among urban women but had no significant effect on rural women.¹¹

In Thailand, the “Better Border and Healthcare Program” found that sending users text messages regarding antenatal care visits and immunizations increased the number of mothers and children attending scheduled appointments.¹²

Mobile messages are generally delivered by either short message service (SMS) or voice messaging; both offer distinct advantages and disadvantages. SMS is available on an estimated 98% of mobile phones, does not require technical expertise to use, and is adaptable to multiple mHealth purposes.¹³ SMS messages can be accessed at user convenience and can be delivered to phones that are turned off or have flat batteries. In addition, telecom costs for SMS are generally less expensive than costs for voice communication. Given these advantages, it is not surprising that SMS is the most common delivery method for mobile development services.¹³

Voice messages, although used less often in mHealth projects, offer the advantages of accessibility to illiterate populations, ability to contain more information per message than SMS, and ability to be recorded in any language (not all languages/characters are supported by SMS).¹⁴ In addition, recorded voice messages allow information to be conveyed by a clinical “character,” as used in Bangladesh by the Mobile Alliance for Maternal Action (MAMA), to build rapport and trust over time.¹⁵

Research comparing the outcomes of voice versus SMS messaging is limited in the mHealth literature, although user preference has been examined in a small number of studies. A U.S.-based study among literate English speakers found that 72% of participants in an mHealth program targeting exercise, diet, and smoking preferred to enroll in automated voice messages over SMS. Those who preferred SMS tended to be younger and to have higher levels of comfort with computers and higher levels of SMS use.¹⁶

The Mobile Technology for Community Health (MOTECH) initiative in Ghana used a “Mobile Midwife” application to provide pregnant women with health education and reminders to access necessary medical services. Messages could be received in voice or SMS format, depending on the enrollees' preference; 99% of enrollees chose voice.¹⁷

Questionnaires administered to potential mHealth users in Argentina, however, found that the users were similarly open to receiving SMS and voice messages during pregnancy, with 96% of participants willing to receive SMS messages and 87% willing to receive voice messages.¹⁸ To our knowledge, no studies have been conducted to determine the difference in acceptability, comprehension, or behavior change between users receiving messages through voice compared with SMS formats.

This article describes the messaging component of an mHealth pilot project in Malawi aimed at improving knowledge and uptake of home- and facility-based MNCH care practices using both SMS and voice message delivery modalities. We report findings from the pilot examining differences between SMS and voice messaging with regard to delivery success and quality of user experience, including acceptability, comprehension, new information learned, and reported behavior change.

PROJECT DESCRIPTION

Chipatala Cha Pa Foni (CCPF) (“Health Center by Phone”) is a mobile health project of Concern Worldwide's Innovations for Maternal, Newborn & Child Health initiative, and its partners VillageReach and the Malawi Ministry of Health (MOH). A nationwide campaign generated more than 6,000 ideas of how to improve quality of health services from community members in Malawi; one of the winning ideas became CCPF.

• Download figure
• Open in new tab
• Download powerpoint

A communication poster encourages caregivers of children to call the Chipatala Cha Pa Foni hotline to get answers to their health-related questions.

CCPF consists of a toll-free hotline offering protocol-based health information, advice, and referrals as well as a mobile messaging service offering automated tips and reminders for pregnant women, guardians of young children, and women of childbearing age. Starting in 2011, CCPF was piloted in 4 rural health center catchment areas in Balaka District in Southern Malawi, representing a population of 32,000 women of childbearing age, 24,000 children under 5, and 7,000 expected pregnancies per year. The pilot project aimed to improve the coverage and quality of RMNCH in Balaka District. Early success and interest has resulted in scale up to an additional 3 districts.

The hotline component of CCPF is staffed by individuals trained in RMNCH using modules from the MOH's curriculum for community health workers (CHWs). When answering calls, hotline workers are prompted by software on touch screen devices to identify clients' symptoms and/or information needs. They answer questions, offer health advice, and provide information on when and where to seek care if clients have symptoms or danger signs that cannot be treated safely at home.

• Download figure
• Open in new tab
• Download powerpoint

A hotline worker answers an incoming calling using a touch screen device that prompts the worker to identify the client's symptoms and information needs.

Hotline workers also offer eligible clients the opportunity to enroll in the tips and reminders mobile messaging service where they can access voice or SMS messages on a weekly basis. Messages offer timely information and reminders about important health services, based on the estimated date of delivery for pregnant women or the child's age for caregivers of children under 1. The tips and reminders are intended to elicit behavior change by helping clients understand their susceptibility to and seriousness of common maternal and child health illness, and they provide cues to action, reminding pregnant women and caregivers of children to seek timely preventive care. Messages for women of childbearing age were added later in implementation.

During project design, we found that low literacy and phone ownership among the target population may limit access to the service. Although an estimated 75% of women over 15 years old in Balaka District are literate,¹ poor women and women living in rural areas are less likely to be literate than their wealthier and urban counterparts.¹⁹ Furthermore, a baseline assessment found that only approximately one-third of women of childbearing age in the catchment area owned a mobile phone.²⁰ Thus, we developed 3 message delivery modalities in order to better accommodate individuals with low literacy or without access to personal phones:

1. Pushed SMS Messages: Clients with access to a personal or household mobile phone can automatically receive SMS messages on their mobile phones.

2. Pushed Voice Messages: Low-literacy clients with access to a personal or household phone can receive weekly recorded voice messages that are sent to the phone at a specific time each week.

3. Retrieved Voice Messages: Clients without access to a personal phone can access their weekly messages by calling the toll-free hotline from any phone and entering an appropriate code to hear their message.

Voice messages are more expensive to deliver so clients with personal phones who identified as being literate were generally encouraged to register for SMS. Low-literacy clients were offered the option of registering for voice messages. Prior to launch, each message delivery option was pilot tested among a small group of users in Balaka to ensure delivery and accessibility.

Due to low phone ownership in Balaka, we provided an additional point of access to CCPF. We originally planned to distribute phones to Malawi's paid cadre of CHWs. However, given the large number of potential users without access to a personal phone, community volunteers from each village were instead recruited in consultation with traditional leaders in the area and provided with a low-cost phone. The volunteers were oriented to the program and provided with small incentives throughout the pilot period to encourage their continued participation. In order to provide phones to all volunteers, we sought partnerships that would allow us to stretch the budget to provide phones to a larger number of people than originally anticipated. Our telecom partner agreed to provide a discount on their basic low-cost promotional phones, which were ultimately purchased for the project.

• Download figure
• Open in new tab
• Download powerpoint

A woman uses a community volunteer's mobile phone to access the health message delivered by the Chipatala Cha Pa Foni project.

DATA AND METHODS

To understand the difference in delivery success and quality between the 3 different messaging modalities (pushed SMS, pushed voice messages, retrieved voice messages), we collected and analyzed information from routine monthly monitoring data and quarterly phone-based surveys with users.

RESULTS

Routine Monthly Monitoring

During the first 2 years of implementation (through June 2013), more than 5,000 pregnant women and caregivers of children under 1 registered for the tips and reminders service. Client registration data were collected starting in July 2011, when clients could begin registering for the service, while monitoring of delivery success rates started in September 2011 when the messaging service was officially launched.

Table 2 displays the number of registered clients and demographics by content type and delivery method. More than half of clients enrolled in the retrieved voice service, nearly one-third enrolled in the pushed SMS service, and less than 10% enrolled in the pushed voice message service. The mean age of pregnant enrollees was 25, and they registered, on average, 16 weeks before their estimated due date. Caregivers of children under 1 registered, on average, 36 weeks before their child's first birthday.

The majority of subscribers enrolled in the “retrieved voice messaging” service because they did not own a personal phone.

View this table:

• View inline
• View popup

TABLE 2. Demographics of Registered Users, by Subscriber Type and Delivery Method (July 2011–June 2013)

Characteristics of clients were similar across service types with the exception of occupation. Pregnant women who registered for pushed SMS messages were more likely to be employed in the formal sector than those who registered for (pushed or retrieved) voice messages. Formal sector includes those who reported their occupation as “business,” “health care worker,” or “teacher.” Informal sector includes those who reported their occupation as “farmer,” “housewife,” “student,” or “other.”

The proportion of messages successfully pushed and retrieved varies by month. For pushed messages (SMS and voice), success rates represent the number of messages successfully received after 3 attempts divided by the number of messages attempted. For retrieved voice messages, success rates represent the number of messages successfully retrieved from the IVR menu divided by the number of messages we estimate should have been retrieved each month. Overall success rates for pushed SMS and voice messages ranged from 54% to 64% and were higher for pushed SMS than for pushed voice messages (Table 3). Success rates for retrieved voice messages were lower, at 27% to 38%.

The SMS service had higher delivery success rates than the pushed or retrieved voice messaging services.

View this table:

• View inline
• View popup

TABLE 3. Delivery Success of Messages, by Delivery Method and Subscriber Type (September 2011–June 2013)

Delivery success changed over time, with the proportion of pushed voice messages successfully delivered to a personal phone increasing slightly over time, and the proportion of retrieved voice messages successfully delivered decreasing over time (Figure 1). Pushed SMS delivery success rates remained relatively stable over time, with a decline in the beginning of 2013, as a result of a number of technical issues. For example, configuration changes by the telecom provider unintentionally reduced the number of times our system attempted to send messages, resulting in messages being attempted less than 3 times. The issue has since been rectified, but it affected success rates for January–May 2013.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1.

Delivery Success Rates of Messages Over Time, by Delivery Method, September 2011–June 2013

Success rate for pushed SMS and voice messages is the percent of messages successfully received within 3 attempts; for retrieved voice messages, the success rate is the percent of expected messages successfully retrieved.

To better understand voice message retrieval rates, we also calculated the proportion of messages successfully played based on the number of attempts (Figure 2). This proportion is calculated from the number of times the message menu is reached (that is, a person calls CCPF's toll-free number and enters “2” to hear a message and then “1” to hear a pregnancy message) and how many times messages are played. Using this definition of delivery success for retrieved voice messages, messages were successfully played in 56% of the instances when the message menu was reached, and success was slightly higher for messages played from the child message menu than messages played from the pregnancy message menu (60% versus 54%).

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 2.

Percentage of Successful Message Retrieval Attempts Among Retrieved Voice Message Subscribers, by Type of Subscriber, September 2011–June 2013

DISCUSSION

Understanding the effects of message modality is important when deciding between SMS or voice message services in mHealth programs. Although SMS is a common and effective method, voice messages may be a desirable option based on literacy levels and mobile phone access.

The data presented in this article reveal that delivery success rates for all message modalities are less than ideal. Phone network challenges in rural Balaka were commonly reported by community volunteers and clients throughout the pilot period and likely contributed to the overall low success rates.

Message delivery rates are far more successful among SMS than voice enrollees. Pushing voice messages to clients with personal phones is a complex process, requiring the client to answer the phone at the time of delivery, whereas SMS messages can be delivered at any time, including when the phone is turned off. We believe this is the major reason that SMS messages pushed to personal phones have a higher delivery success rate than voice messages.

During the first few months of implementation, we varied the times that we pushed voice messages to users to find times that clients were most likely to be available to answer the phone. When tips and reminders were launched, voice messages were attempted hourly between 12:00 pm and 6:00 pm local time (CAT) because anecdotal reports indicated that phones were most likely to be charged and switched on during daylight hours. In November 2011, we pushed the voice messages between 2:00 pm and 6:00 pm because we noted more successful attempts occurred during that timeframe. In early 2012, we discovered a configuration problem between the telephone card and the telecom provider that was also contributing to the low delivery success rate. After changing the timing that we pushed the voice messages and fixing the configuration problem, delivery success rates for pushed voice messages increased, but the delivery success rate remained highest for pushed SMS messages.

Retrieved voice messages have the lowest delivery success rate, and this rate decreased over time. Retrieving voice messages from the IVR menu using a community phone requires initiative on behalf of the client who must seek out a volunteer in order to access the community phone. In addition, it requires the volunteer to be available. There were some instances where volunteers moved away from their assigned village or became inactive over time. It is also possible that some community volunteers acted as gatekeepers to the service. We heard anecdotal reports of a small number of volunteers refusing to give out the CCPF telephone number or telling clients that the service must be accessed through a community phone rather than a personal phone.

Furthermore, many community phones broke during the project period, limiting access to the service for community phone users. The phones given to community volunteers were low cost, and approximately 60% of the phones distributed to volunteers were not working 2 years after the original implementation began. It is likely that the unreliability of community phones played a role in the low delivery success of retrieved voice messages in 2 ways:

• Malfunctioning keypads made it difficult to enter an accurate estimated due date or child's birthday in order to hear a message. This is a major reason stated by both volunteers and users for why numerous calls that reached the voice message menu never successfully heard a message.

• Clients who signed up for voice messages in a village where the phone eventually broke were unlikely to find another way to access the service. Some users stated that after the community phone in their village broke, they no longer accessed the CCPF hotline or voice messages.

The data also reveal that delivery success for retrieved voice messages is significantly higher among users subscribed to pregnancy messages than users subscribed to child messages (based on the definition of success used in Table 3). Pregnant users registered an average of 16 weeks before their estimated due date while caregivers of children registered an average of 36 weeks before their child's first birthday. It is possible that the average pregnancy-message enrollee was more likely to take the initiative to visit a community volunteer because the eligibility period was shorter than for the average child-message enrollee. Similarly, it is less likely that a community volunteer would become inactive or the community phone would break during a shorter eligibility period. For pushed message enrollees, there is no clear relationship between message type and success rates because delivery success of voice messages was significantly higher among pregnancy-message enrollees than child-message enrollees, but the opposite was true for pushed SMS messages.

Although delivery success was higher among pushed SMS messages compared with other delivery modalities, the data reveal that clients were highly satisfied with and trusted the voice and SMS messages equally and were likely to learn new information and recall messages regardless of message delivery modality. SMS enrollees, however, were significantly more likely to report intended or actual behavior change than pushed voice enrollees. We suspect that this may be because voice messages can only be heard once while SMS messages can be saved in the client's inbox and read multiple times, and even shared with others. It is also possible that shorter and more frequent messages in the form of SMS are more effective at eliciting intended behavior change than longer voice messages that are delivered once per week. Finally, the differing characteristics of individuals who sign up for SMS messages might make them more willing or able to change their behavior.

At baseline, this mHealth project discovered that most women in the project area did not own a mobile phone. To address this problem, we provided an additional access point by providing community volunteers with mobile phones. Although the solution was not perfect, our study demonstrates that it is feasible to provide low-literate clients without a personal phone with mobile health information and to stimulate behavior change. Although delivery success was the lowest for retrieved voice message subscribers, approximately 1,200 voice messages were successfully retrieved each month. A similar number of pushed SMS and voice messages were successfully delivered each month. This finding emphasizes the importance of recognizing that mobile health messaging can potentially marginalize populations without access to mobile phones if efforts are not made to adapt mHealth projects to meet the needs of the population. Not surprisingly, phone ownership is inversely correlated with poverty,²¹ and women and people living in rural areas are also less likely to own mobile phones than their male and urban counterparts.¹⁹ Thus, offering a retrieved voice message option for those without personal phones is an important strategy to deliver health messages to an otherwise hard-to-reach population.

CONCLUSION

Mobile technology, specifically SMS and voices messages, can be successfully used to extend health information services to pregnant women and caregivers of young children in rural Malawi. All 3 message modalities led to high levels of satisfaction, comprehension, and new information learned. Due to lower cost, higher delivery success, and higher levels of intent to change behavior, SMS is the preferred delivery modality when possible.

Implementing mHealth projects in areas with low phone penetration creates unique challenges in ensuring access and generating demand. By adapting the service to the context—allowing users to retrieve voice messages from the IVR system without a personal phone and providing community volunteers with phones to serve as access points to the service—this rural population gained access to the mHealth service that would not otherwise have been possible. Although these adaptations presented their own challenges, particularly apparent in the low success rate of retrieved messages, availability of community volunteers, and the difficulty sustaining the volunteer model over time as phones began to break, providing multiple methods by which users could access the service was crucial in extending reach beyond literate personal phone owners.

Further research to understand potential models for increasing access to mobile phones among low-literate, rural populations is warranted. Prior to implementing a similar service, it is important to understand the cost-effectiveness of the intervention and cost implications at scale. At the time of this study, an economic evaluation of CCPF was underway. Finally, further research examining the translation of health information transmitted through mobile phones into healthy behaviors is needed to understand the potential impact of mHealth interventions on their intended health outcomes.