A randomized controlled trial to evaluate the Make Safe Happen® app—a mobile technology-based safety behavior change intervention for increasing parents’ safety knowledge and actions
Injury Epidemiology volume 5, Article number: 5 (2018)
Many unintentional injuries that occur in and around the home can be prevented through the use of safety equipment and by consistently following existing safety recommendations. Unfortunately, uptake of these safety behaviors is unacceptably low. This paper describes the design of the Make Safe Happen® smartphone application evaluation study, which aims to evaluate a mobile technology-based safety behavior change intervention on parents’ safety knowledge and actions.
Make Safe Happen® app evaluation study is a randomized controlled trial. Participants will be parents of children aged 0–12 years who are recruited from national consumer online survey panels. Parents will complete a pretest survey, and will be randomized to receive the Make Safe Happen® app or a non-injury-related app, and then complete a posttest follow-up survey after 1 week. Primary outcomes are: (1) safety knowledge; (2) safety behaviors; (3) safety device acquisition and use, and (4) behavioral intention to take safety actions.
Anticipated study results are presented.
Wide-reaching interventions, to reach substantial parent and caregiver audiences, to effectively reduce childhood injuries are needed. This study will contribute to the evidence-base about how to increase safety knowledge and actions to prevent home-related injuries in children.
Trial registration number
Unintentional injury is the leading cause of death for children in the United States (U.S.) and is responsible for more deaths than the next three causes combined, that is, homicide, suicide and cancer (Center for Disease Control and Prevention 2012). Each year approximately 9000 children die, 250,000 children are hospitalized and more than 9000,000 children are treated in emergency departments for preventable injuries (resulting in approximately $87 billion in medical and societal costs) (Center for Disease Control and Prevention 2012). Numbers of child and adolescent unintentional injury deaths have not declined to the same extent as some diseases have, and resources directed at reducing child injury are not commensurate with the burden it imposes.
More than 50% of these injuries occur in and around the home, where children spend most of their time (Bergen et al. 2008). Common causes of fatal and nonfatal unintentional childhood injuries include: falls, fires and burns, drowning, poisoning, and suffocation (10 Leading Causes of Injury Deaths by Age Group Highlighting Unintentional Injury Deaths, United States-2014 [Internet] 2014; National Estimates of the 10 Leading Causes of Nonfatal Injuries Treated in Hospital Emergency Departments, United States-2013 [Internet] 2013). For example, in the U.S., residential fires account for nearly 90% of all fire-related deaths in children younger than 15 years (Baker and Waller 1989). almost 13,000 children are injured by TV tip-overs each year (De Roo et al. 2013) and medications are the leading causes of child poisoning (Baker and Mickalide 2012). Other home hazards include bunk beds (D’Souza et al. 2008). high chairs (Kurinsky et al. 2014), windows (Harris et al. 2011), stoves and ovens (D’Souza et al. 2009), button batteries (Sharpe et al. 2012), laundry packets (Davis et al. 2016; Valdez et al. 2014), and toys (Abraham et al. 2015)—ubiquitous products and home features present in nearly every U.S. home.
Many of these injuries in and around the home can be prevented through the use of safety equipment and by consistently following existing safety recommendations. Using known, effective countermeasures, such as smoke alarms, carbon monoxide detectors, stair gates, cabinet locks and latches, and anchoring TVs and furniture, can prevent or reduce the consequences of a child being injured (Kendrick et al. 2005).
The overall rates of these safety behaviors are unacceptably low; a 2015 survey of more than 1000 parents of children aged 12 and under found that 48% of parents surveyed had not secured TVs and furniture, 73% reported placing items like blankets, bumpers, and stuffed animals in the crib with their baby, 12.5% have left their young child alone in a bathtub for 5 min or longer, 30% of parents with a toddler report keeping medicines and cleaning products on a low shelf or unlocked cabinet, 14% report never checking their smoke alarm batteries (Report to the Nation: Protecting Children in Your Home 2015). Child and adolescent unintentional fatalities and injuries are a significant public health problem and resources directed at reducing child injury are not commensurate with the burden it poses. Parents and caregivers often have difficulty identifying hazards in their home (Gaines and Schwebel 2009), finding credible information and recommendations, and obtaining the safety products best-suited to their home. Interventions for increasing home safety behaviors which have ranged from the provision of educational materials, to health care provider counseling, safety product distribution, hands-on experiential learning provided in safety resource centers have been evaluated (with varying degrees of effectiveness) (Kendrick et al. 2014). However, wide-reaching, effective and readily available interventions may be needed to reach substantial parent and caregiver audiences.
Smartphone mobile applications are particularly promising in this area as they offer an efficient and cost-effective medium to deliver health-related programs and interventions. Offering targeted safety information on multiple topics via a single platform, combined with the ability to acquire child safety devices, may be a more efficient means to reach a large segment of the parent and caregiver population and may help reduce the aforementioned barriers. Unfortunately, few injury-related apps exist and fewer still have been systematically evaluated (Omaki et al. 2016).
The aim of the study described in this protocol is to evaluate the impact of and determine whether a mobile technology-based health behavior change intervention, the Make Safe Happen® app, is effective in increasing (1) safety knowledge; (2) safety actions/behaviors; and (3) safety device acquisition and use; and (4) behavioral intention (planning to take safety actions or acquire devices) for the prevention of unintentional home-related injuries, compared with no intervention.
Study design and setting
The Make Safe Happen® app (Additional file 1) evaluation study will be a two-group, randomized controlled trial with pre- and posttest elements (no blinding), and will involve 1200 parents of children aged 0–12 years. These data will be collected in the U.S. The intervention group (IG) will use the Make Safe Happen® app and receive information about the prevention of unintentional child home-related injuries. The “attention-matched” control group (CG) will use an app to receive information about dinner recipes. Surveys measuring safety knowledge, behaviors and behavioral intentions, and safety device acquisition and use will be administered before and after app use (Fig. 1). This study was approved by the Institutional Review Board (IRB) at Nationwide Children’s Hospital. Plans for protocol modifications or for communications to relevant parties will be made to the study investigators, IRB, trial participants if necessary. Study investigators will have access to the final dataset. See Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Checklist (Research Checklist) (Additional file 2).
Screening, enrollment and randomization
Parents or legal guardians of at least one child aged 0–12 years will be recruited to participate in this study. The study participants will be members of a pre-existing, national online survey panel. This panel recruits people by a variety of methods, including random-digit-dial telephone calls, USPS mailings, and advertising on social networking websites. After individuals join the panel and participate in surveys, quality checks of their responses are performed. We will employ a standard practice recommended by the online survey company, that is, for participants who complete surveys too rapidly (i.e., one-third the median length of the survey response time) will be disqualified from participation in the study because of a concern that they will not be carefully processing the questions’ language and intent, nor accurately formulating their responses. These respondents will be treated as “incomplete” surveys and will not be included in the analyses.
For the Make Safe Happen® app evaluation study, panel members will receive an email inviting them to complete a brief survey that screens for eligibility. Inclusion criteria will require that participants: (1) have an Android phone or Apple smartphone (i.e., iOS or Android); (2) be a parent or legal guardian of a child (“index child”) aged 0–12 years; and (3) this child must live with them at least “most of the time.” Parents will be excluded if they have previously downloaded or used the Make Safe Happen® app or the control app. An “index child” will be selected via a least filled quota protocol, meaning a respondent will be assigned to a child age group depending on the needs, or quotas, for each age group for the entire sample and treatment group. When a new participant with more than one child between 0 and 12 years of age completes a survey, that participant will be assigned to a quota age group based on whichever has the smallest number of completed participant surveys at that time. The selection of an index child is only relevant for assigning participants to child age groups for recruitment to prevent unequal child age groups in the treatment groups. Birth order is not an issue in the proposed study.
Parents will indicate their consent to participate after reading and marking their consent to participate on an online consent form (consent language available from authors upon request). Participants will then be randomly assigned by a computer-generated roster in a 5:1 ratio to receive the Make Safe Happen® app Version 1.0.3 (IG) or Allrecipes Dinner Spinner app (CG). Randomization will be blocked in groups of five based on an index child’s age in age subgroups. After completing the pretest survey participants will be emailed and asked to download the Make Safe Happen® app or Allrecipes Dinner Spinner app from iTunes or GooglePlay app stores for free onto their smartphone. IG participants will be asked to enter a participant identification (ID) number into the Make Safe Happen® app which will be assigned to them after the completion of the pretest survey (Fig. 2a and b). Entering a participant ID into the Make Safe Happen® app will establish a link between each study participant assigned to the IG and the unique actions they take in the app over the duration of the study. This linkage will also allow participant’s app data to be connected to the participant’s pretest and posttest survey data. Participants in the IG will receive email reminders about downloading the Make Safe Happen® app at 24 and 72 h after they complete the pretest survey. Intervention participants who do not download the app or enter their participant ID will not be eligible to complete the posttest survey. Participants will be compensated by a point rewards system (equivalent to $8–10) when they complete all parts of the study.
Intervention and control group content
CG Content. Allrecipes Dinner Spinner (version 6.1). It is described as “the most popular food—focused social app” that helps “cooks discover and share the joy of home cooking” (App Store, Food & Drink, All Recipes, Inc. Allrecipes.com). Content will vary by user preference and choices. Download of and use of the CG content will not be confirmed by the study team. This app was selected to be used for the control condition because it does not include information about home safety and does not require that users to log in or set up an account and is available for free in iTunes and Google Play. The recipe app was selected for use by the CG participants so that they would have a similar download and app use experience to the IG group. Respondents will be asked if they downloaded Dinner Spinner when they complete the posttest survey. Participants in the CG will also receive information inviting them to download and use the Make Safe Happen® app after completing the posttest survey.
Results and discussion
Outcomes and measures
Primary outcomes will include: (1) safety knowledge; (2) safety behaviors; (3) device acquisition and use; and (4) behavioral intention to take safety actions. Participants will self-report these at baseline (pretest) and at follow-up (posttest). A combination of yes-no, multiple choice and Likert scale response options will be used. A sample of questions from the pretest and posttest surveys will be vetted for coherent meaning and level of difficulty via an hour-long cognitive interview with 20 parents prior to the launch of the study. A pilot study with 200 participants will be completed prior to the study launch to examine the distribution of participants’ responses and to address any technical difficulties with the pretest survey instrument.
Safety knowledge will be measured by responses to 17 questions that were created specifically for this study and are based on safety content that is delivered in the Make Safe Happen® app. One point will be given for each correct answer and correct responses will be summed to determine a total knowledge score for each participant. The total safety knowledge score percent will be derived by dividing the total number of correct responses by 17. The mean total safety knowledge score and the mean total safety knowledge score percent will be calculated for pretest intervention condition, pretest control condition, posttest intervention condition and posttest control condition participants.
Safety behaviors and behavioral intentions
Safety behaviors (either repeatedly done or one-time behaviors) and behavioral intention (planning to perform safety behaviors in the future) will be measured by responses to 31 questions. Participants will be asked about 14 safety behaviors that are recommended to repeatedly do, e.g., turning pot handles away from the stove when cooking, and 15 safety behaviors that are typically completed once, e.g., buying and installing a carbon monoxide detector. For example, participants will read the question stem, “I repeatedly take the following safety actions in my home…” and will be presented with several statements such as, “Turn pot handles to the back of the stove when I cook.” Response choices will include the following options: always, sometimes, never, and not applicable. Participants who report never doing any of these behaviors, will be asked whether they intend to in the future (yes, no, not sure).
Additional measures that will be collected include a count of safety actions completed (or “checked off”) within the Make Safe Happen® app.
Safety device acquisition and use
Participants will be asked whether they have any of the 22 possible types of safety devices installed in their homes, e.g., fire escape ladder, medicine lock box, etc. Response choices will include the following options: yes, no, not sure, and not applicable. Participants who report no to one or more safety devices will be asked whether they intend to purchase and install these devices in the future (yes, no, not sure).
Data analysis plan
Data analysis will be primarily comprised of two activities: 1) the generation of descriptive information summarizing the sample and its characteristics; and 2) the analyses related to the aims and hypotheses. The central hypothesis is that parents in the IG will have higher safety knowledge, do more safety behaviors, have increased behavioral intention to take safety actions, and be more likely to acquire and use safety devices posttest, as compared to the CG. Linear mixed model regression will be used to analyze safety knowledge outcome and longitudinal logistic regression models will be run for behavioral intention and safety actions. Primary outcomes for the safety behaviors and device use will be assessed while stratifying on the index child age group. Analyses will be undertaken to examine the hypotheses that the IG will have higher change at posttest from baseline in safety knowledge, safety behaviors, behavioral intention, safety device use and acquisition than the CG.
Besides the intent-to-treat analyses of safety knowledge, safety behaviors and behavioral intention, safety device use and acquisition, we will also conduct exposure analyses to examine the relationship between outcomes and exposure to the intervention. For the IG participants those who enter a participant ID in the Make Safe Happen® app will be asked to complete a posttest survey. For the CG participants, those who complete the pretest survey will be asked to complete a posttest survey since we cannot confirm download and use of the control condition app. We plan to link app analytics for IG participants to their pretest and posttest survey responses, for example, if a participant uses the Make Safe Happen® app to add the poison control center emergency number to their contacts we will check to see whether that participant got the poison control center emergency number safety knowledge question correct in the pre- and posttest surveys. All study data will be stored on password protected computers of the study personnel. The study will be monitored by the study investigators.
We plan to enroll 5000 participants, and we estimate a 40% loss-to-follow-up, which will provide us with the needed sample size of 1200 in the posttest sample for the planned analyses. A power analysis indicated it would not be necessary to have equal numbers of participants assigned to the IG and the CG. Therefore, participants will be randomized to the treatment groups using a 5:1 ratio, allowing efficient allocation of participants to the IG or CG (respectively) within the online survey group. Focus group participants and home visit participants will not be randomized, although all focus group and home visit participants will receive the Make Safe Happen app intervention. The randomization will be completed by the use of a computer program embedded within the online survey, which randomly generates a number between 1 and 120 for each participant. Numbers between 1 and 100 will be assigned to the IG, while numbers between 101 and 120 will be assigned to the CG. Our proposed sample size reflects consideration of our proposed attrition rate and is based on previous research using online survey panels. Additionally, we will aim to recruit a diverse sample of parents ≥18 years old based on sex (30% male, 70% female), ethnicity (maximum of 66% White, non-Hispanic), and age of index child (~ 20% in each of the five age brackets).
Home-related unintentional injuries in children remain unacceptably high in the U.S. despite the availability of proven and effective strategies for reducing or preventing these injuries. Developing and widely disseminating effective means to communicate these strategies with parents and caregivers is a research and public health priority. This study will contribute to the evidence about how to promote the life-saving and injury-reducing benefits of home safety actions and practices.
Home safety smartphone apps are ideal for parents and caregivers. Apps can be used on parents’ own time and in the comfort of their home. Because safety recommendations are usually provided as a one-size-fits-all recommendation, parents may need information that is customized to the features in their home and to the ages of their children. Thus, this research seeks to provide home safety information, using innovative communication technology. The app can be used on their own time, in their own home, and does not rely on or involve clinicians or a medical setting and is widely available. The use of a mobile app improves upon the “one-time” nature of some brief interventions by allowing the participant or app user to revisit the app and complete safety actions as frequently as desired and to set reminders for themselves for future actions. The information provided can change as the child ages in order to be relevant for the child’s age and development, or addition of new children to the family, as well as protecting persons of all ages residing in the home. New content and features can be added in order that the Make Safe Happen® app can function as a virtual safety expert offering ongoing education on home safety or related topics and posts notifications about new or seasonal safety topics. Given the high rates of smartphone app use [roughly three-quarters of Americans (77%) own a smartphone] (Smith 2017),—these results should have widespread applicability.
The pretest and posttest survey data will be limited due to the nature of self-reported data and may also be subject to reporting bias. Further, the surveys will be conducted close together in time (a span of 7–10 days apart) therefore we may see a learning effect of having recently taken the survey. A follow-up period of 7 to 10 days was used in order to allow recruited participants time to download and use the app. We do not expect that participants will complete all of the recommended safety actions or safety device installations in this limited time period. We opted to keep the follow-up period relatively short in an effort to prevent loss to follow-up. While we will have actual app data to report on what users completed within the app, at best these data will be a proxy for real behaviors or safety actions. Further, the requirement for a smartphone Internet connection in order to transmit the data from the Make Safe Happen® app to the online platform that stores these data as well as the app data processing latency of 24—48 h may both cause delays in distribution of posttest surveys to IG participants. An additional limitation is that we will not be able to confirm or track the CG recipe app usage.
This work will contribute to reducing child injury by providing an evaluation of an innovative, targeted, free and widely available smartphone app that delivers focused home safety information to parents and caregivers. With the wide availability of smartphone apps, the results of this work will have utility for disseminating behavior change programs to a broad audience, and the results of this study will advance the application of behavior change interventions in a new area—injury prevention. Further, the systematic evaluation of apps is rare in the published literature, so this protocol may help demonstrate how others (even those outside the injury prevention field) can accomplish this type of rigorous evaluation.
10 Leading Causes of Injury Deaths by Age Group Highlighting Unintentional Injury Deaths, United States-2014 [Internet]. CDC using WISQARS. 2014. Available from: https://www.cdc.gov/injury/images/lc-charts/leading_causes_of_injury_deaths_unintentional_injury_2014_1040w740h.gif. Cited 4 Jan 2017.
Abraham VM, Gaw CE, Chounthirath T, et al. Toy-related injuries among children treated in US emergency departments, 1990-2011. Clin Pediatr (Phila). 2015;54(2):127–37. https://doi.org/10.1177/0009922814561353. Epub 2014 Nov 30
Baker JM, Mickalide AD. Safe storage, safe dosing, safe kids: a report to the nation on safe medication. Washington, D.C.: Safe Kids Worldwide; 2012.
Baker SP, Waller AE. Childhood injury state-by-state mortality facts. Baltimore: Johns Hopkins Injury Prevention Center; 1989.
Bergen G, Chen LH, Warner M, Fingerhut LA. Injury in the United States: 2007 Chartbook. Hyattsville: National Center for Health Statistics; 2008.
Center for Disease Control and Prevention. National action plan for child injury prevention. Atlanta: National Center for Injury Prevention and Control; 2012.
D’Souza AL, Nelson NG, McKenzie LB. Pediatric burn injuries treated in US emergency departments between 1990 and 2006. Pediatrics. 2009;124(5):1424–30. https://doi.org/10.1542/peds.2008-2802.
D’Souza AL, Smith GA, McKenzie LB. Bunk bed-related injuries among children and adolescents treated in emergency departments in the United States, 1990-2005. Pediatrics. 2008 Jun;121(6):e1696–702. https://doi.org/10.1542/peds.2007-2555.
Davis MG, Casavant MJ, Spiller HA, et al. Pediatric exposures to laundry and dishwasher detergents in the United States: 2013-2014. Pediatrics. 2016;137(5) https://doi.org/10.1542/peds.2015-4529.
De Roo AC, Chounthirath T, Smith GA. Television-related injuries to children in the United States, 1990-2011. Pediatrics. 2013;132(2):267–74. https://doi.org/10.1542/peds.2013-1086.
Gaines J, Schwebel DC. Recognition of home injury risks by novice parents of toddlers. Accid Anal Prev. 2009;41:1070–4.
Harris VA, Rochette LM, Smith GA. Pediatric injuries attributable to falls from windows in the United States in 1990-2008. Pediatrics. 2011;128(3):455–62. https://doi.org/10.1542/peds.2010-2687. Epub 2011 Aug 22
Kendrick D, Watson M, Mulvaney C, et al. How useful are home safety behaviours for predicting childhood injury? A cohort study. Health Educ Res. 2005;20(6):709–18. Epub 2005 Mar 8
Kendrick D, Young B, Mason-Jones AJ, Ilyas N, Achana FA, Cooper NJ, Hubbard SJ, Sutton AJ, Smith S, Wynn P, Mulvaney C, Watson MC, Coupland C. Home safety education and provision of safety equipment for injury prevention (review). 2014;8(3):761–939. https://doi.org/10.1002/ebch.1911.
Kurinsky RM, Rochette LM, Smith GA. Pediatric injuries associated with high chairs and chairs in the United States, 2003-2010. Clin Pediatr (Phila). 2014;53(4):372–9. https://doi.org/10.1177/0009922813510599. Epub 2013 Dec 9
National Cancer Institute. Theory at a glance: a guide for health promotion practice. Second ed: U.S. Department of Health and Human Services. Bethesda: National Institutes of Health; 2005.
National Estimates of the 10 Leading Causes of Nonfatal Injuries Treated in Hospital Emergency Departments, United States-2013 [Internet]. CDC Using WISQARS. 2013. Available from: https://www.cdc.gov/injury/wisqars/pdf/leading_causes_of_nonfatal_injury_2013-a.pdf. Cited 4 Jan 2016.
Omaki E, Rizzutti N, Shields W, Zhu J, McDonald E, Stevens MW, et al. A systematic review of technology-based interventions for unintentional injury prevention education and behaviour change. Inj Prev. Published Online First: 19 January 2016. https://doi.org/10.1136/injuryprev-2015-041740.
Report to the Nation: Protecting Children in Your Home. Washington, DC: Safe Kids Worldwide. 2015. https://www.safekids.org/sites/default/files/documents/ResearchReports/report_to_the_nation_protecting_children_in_your_home.pdf. Accessed 30 Jan 2018.
Rimal RN, Adkins AD. Using computers to narrowcast health messages: the role of audience segmentation, targeting, and tailoring in health promotion. In: Thompson TL, Dorsey AM, Miller KI, Parrott R, editors. Handbook of health communication. Mahwah: Lawrence Erlbaum Associates; 2003. p. 497–513.
Sharpe SJ, Rochette LM, Smith GA. Pediatric battery-related emergency department visits in the United States, 1990-2009. Pediatrics. 2012;129(6):1111–7. https://doi.org/10.1542/peds.2011-0012. Epub 2012 May 14
Smith A. Record shares of Americans now own smartphones, have home broadband: Pew Research Center; 2017. Available from: http://www.pewresearch.org/fact-tank/2017/01/12/evolution-of-technology/. Accessed 30 Jan 2018.
Valdez AL, Casavant MJ, Spiller HA, et al. Pediatric exposure to laundry detergent pods. Pediatrics. 2014;134(6):1127–35. https://doi.org/10.1542/peds.2014-0057. Epub 2014 Nov 10
The authors would like to thank the following individuals and teams, Patty Yoder formerly of Nationwide, who provided content writing help for the Make Safe Happen app; Matthew Bailey and Jeremy Patterson, of the Research Institute at Nationwide Children’s Hospital for their technical assistance with app development, testing and Google Analytics; and Nationwide® and the Make Safe Happen® program for their unwavering commitment to this important cause.
This study protocol and evaluation trial was supported by funding from the Nationwide Children’s Hospital Foundation. The development of the Make Safe Happen app was made possible by funding from Nationwide Children’s Hospital Foundation’s Innovation Fund and Nationwide Mutual Insurance.
The study was approved by the Institutional Review Board at Nationwide Children’s Hospital.
The authors declare that they have no competing interests.
About this article
Cite this article
McKenzie, L.B., Roberts, K.J., Clark, R. et al. A randomized controlled trial to evaluate the Make Safe Happen® app—a mobile technology-based safety behavior change intervention for increasing parents’ safety knowledge and actions. Inj. Epidemiol. 5, 5 (2018). https://doi.org/10.1186/s40621-018-0133-3