Safety Concerns in Mobility-Assistive Products for Older Adults: Content Analysis of Online Reviews

Introduction

Background

The Consumer Product Safety Commission (CPSC) has reported that from 2016 to 2020 there were approximately 14.6 million emergency department (ED) cases associated with older adults using consumer products [1]. Of these, nearly two-thirds were due to falls [2], indicating that the dominant context of injury is during motion or transfer of older adults. Mobility devices such as canes and walkers are among consumer products frequently implicated in such injuries. The development of interventions to prevent potential future injuries among older adults using mobility-assistive devices requires a better understanding of equipment limitations or failures that precipitate injuries from using such devices. A substantive curated data set of safety concerns regarding mobility-assistive devices would address this gap in knowledge.

The adoption of these devices is relatively common, with approximately 10% of community-living adults aged ≥65 years adopting them each year to help with their mobility limitations [3]. Given that older adults are among the fastest growing demographic groups in the United States [4], annual injury figures are expected to continue increasing within the next decade. In fact, the Centers for Disease Control and Prevention anticipates that by 2030 there will be 7 older adult fall-related fatalities per hour [5]. When compared with an estimated 4.17 fall-related fatalities per hour in 2020 [6], this calls for expanded injury prevention mechanisms capable of rapidly detecting safety concerns regarding products targeted toward older adults.

Regulatory agency injury prevention efforts often rely on customers and businesses to report unsafe products because it is logistically impossible to inspect every product in the market. It should be noted that safety hazards are generally reported after consumer harm, failing to prevent injury; for example, between 2014 and 2021, the CPSC issued recalls for hundreds of thousands of units of 4 bed handle brands but only after multiple deaths were attributed to these products [7-11]. Although existing databases such as the CPSC’s National Electronic Injury Surveillance System (NEISS) and the Food and Drug Administration’s Manufacturer and User Facility Device Experience (MAUDE) contain reports of consumer harm caused by product failure, these are generally abstracted from a third party, not the consumer itself, which poses several problems. In the case of NEISS, because data are abstracted from ED visit records, the content is highly skewed toward injury description, and context regarding how the device failed is scarce and ambiguous (eg, “the walker broke”). Regarding MAUDE, although it does abstract from various sources (hospitals, device user facilities, voluntary reports, and manufacturers), report-processing times may result in untimely reports of product malfunctions. Moreover, it should be noted that consumers may often forgo voluntary reporting to the Food and Drug Administration because they may believe that the issue is “not that serious,” when it could, in fact, be a malfunction capable of causing serious injury. Regarding MAUDE manufacturer reports, these are only required if there is reasonable information suggesting that the product may contribute to consumer death or serious injury, and, as a result, such reports are rare. Thus, given the long delays in reporting, investigating, and initiating regulatory action for product safety hazards, as well as existing database content tending to focus on circumstances other than device malfunction context, additional resources are needed to inform older adults and their caregivers about the safety of mobility-assistive devices.

Although medical device stores, specialized pharmacies, and physiotherapists may provide warnings or recommendations on specific devices, access to these may not be straightforward for every older adult. Location or health concerns may prevent access, leading to an older adult demographic that relies on online retailers for purchase of these devices. In this regard, online reviews are often used by consumers and regulators to assess the safety of products. Studies show that 0.2% to 4% of online reviews contain evidence of a product safety concern [12-15] that cannot be found in reports by regulatory agencies. The early work by Restrepo et al [16] indicates that online product reviews present a novel and untapped source of safety concerns for mobility-assistive devices for older adults. As the thousands of available reviews for any given device may be overwhelming and difficult to navigate for an older adult, further work is needed to substantially expand upon, analyze, and compile consumer-reported safety concerns of mobility-assistive devices.

Objectives

The aims of this study were to investigate older adult injuries stemming from the use of mobility-assistive devices, as reported by older adults or their caregivers. We not only identify injury severities and mobility-assistive device failure pathways but also shed light on the development of safety information and protocols for these devices. To do so, we developed and used an expansive, manually curated data set of mobility-assistive device safety concerns from consumer reviews on Amazon’s US website. On the basis of the findings from our prior studies, we expect that the curated data set of online reviews for mobility-assistive devices will provide valuable insight into the types of injuries that are associated with the use of mobility-assistive devices and the pathways by which mobility-assistive device use precipitates injuries.

Methods

Data Source

Initial data were extracted from Amazon’s US website through the use of an automated script [16]. The script extracted 633,141 reviews, distributed across thousands of unique products pertaining to “assistive aid” categories typically intended for older adults. The reviews were collected from September 12, 2017, to September 16, 2017. The reviews span the time range from February 28, 2002, to September 15, 2017.

Exploratory Data Coding

Large-scale data coding (labeling) efforts were undertaken as volunteers became available, resulting in 2 separate exploratory coding phases (I and II), described later in this section, and visualized in Figure S1 and Table S1 in Multimedia Appendix 1 [17]. Coding categories were iteratively refined by having senior raters, who were the lead authors (all having graduate student or faculty investigator status), code random samples and discuss discrepancies across both phases. Cohen κ [17] was used to verify interrater reliability (scores presented in Table S1 in Multimedia Appendix 1). Throughout both phases, junior raters, who were undergraduate student volunteers who had completed the qualification examination verifying that they were able to code safety concerns with an accuracy of ≥80%, were asked to code reviews according to injury type:

• Major injury occurred: someone was seriously hurt by the product, as described in the following review, and required a physician or hospital visit (medical professional intervention) or died:

Very unstable as wheelbase is way too short...product [went] from under me walking down a ramp at least 6 times...broke my thumb.

• Minor injury occurred: someone was actually hurt by the product, but it was a minor, self-treatable incident, and no physician visit or hospital visit was explicitly mentioned:

Seems to be not very stable, my mother fell with it and she really scraped her leg badly...she doesn’t want to use it.

• Potential future injury: injury could possibly occur; thus, the review writer is cautious about using the product:

Not even 3 months old and already the wheels are falling off! my husband is handicapped and would not be good if the walker collapsed when he tried to sit because the wheel came off!

• No injury occurred: there is no indication of an actual or potential future injury as a result of product use:

Sturdy cane. The stands by itself feature, very nice.

Phase I, described in Restrepo et al [16], encompassed the coding and examination of 50,000 randomly selected reviews, which resulted in the identification of 18 major product categories (Table S2 in Multimedia Appendix 1) and 3100 (6.2%) shortlisted reviews of potential safety concerns. Shortlisted concerns pertaining to categories unrelated to mobility-assistive devices were removed, and of the 3100 reviews, we retained only 983 (31.7%). Of the 18 major product categories, we retained only 8 (44%; Table S2 in Multimedia Appendix 1).

Before phase II coding, the 8 retained categories were further narrowed down to 5 (63%): canes, gait and transfer belts, ramps, walkers and rollators, and wheelchairs and transport chairs. Crutches as well as knee walkers and scooters were determined to be frequently used by younger adults (aged 25-64 years), rather than predominantly older adults (aged ≥65 years), and car assistive devices were re-examined and found to be unrelated to actual mobility assistance, resulting in the elimination of these 3 categories. Phase I verified concerns were filtered accordingly: of the initial 983 reviews, we preserved 685 (69.7%). In addition, the 633,141 review sample pool was reduced to 44,119 (7%), solely keeping reviews (not previously labeled in phase I) belonging to the 5 retained mobility-assistive device categories. Table 1 displays mobility-assistive device review distributions for each of the phases.

The 44,119 retained reviews were coded by 190 undergraduate student volunteers (junior raters) from a triple crown–accredited university in Thailand and an R1 public land-grant university in the United States, which resulted in 3300 (7.48%) shortlisted online reviews that mentioned potential safety concerns. Volunteer work was validated by methods described in phase I [16], in which 2 of the authors and a doctoral student (senior raters) verified both whether the concern was relevant and what the nature of the concern was, resulting in, out of the 3300 shortlisted online reviews, a final list of 2203 (66.76%) online reviews that mentioned safety concerns. The Cohen κ [17] value among the senior raters for the final list was 0.84.

A mobility-assistive device safety concern data set comprising 2888 reviews was created by consolidating the validated concerns of both phase II (n=2203, 76.28%) and phase I (n=685, 23.72%).

Additional Data Coding

The manual validation efforts of phases I and II led to the discovery of 7 device failure mechanisms (injury pathways), resulting in additional coding (of the 2888 reviews) by the authors. Coding disagreements were discussed by the authors to refine the 7 identified injury pathways, leaving 6 (86%) discrete categories (pathways):

• Critical component breakage or decoupling: defined as the device or a part or parts of it breaking, falling off, frequently requiring adjusting or retightening (with the exception of brakes), or collapsing; also used when the device was described as being cheaply made, “unsturdy,” or flimsy
• Unintended movement: defined as the device or a part or parts of it moving without the user intending it to, although the part itself is not generating instability or at risk of decoupling; also used when the device’s brakes were said to require frequent adjusting or retightening
• Instability: defined as the device or a part or parts of it causing instability or unsteadiness while being used; also used when the device was described as wobbly, tippy, off-balance, insecure, or unable to provide adequate support
• Poor, uneven surface handling: defined as the device or a part or parts of it frequently getting caught or stuck in small cracks or seams, tangled in grass or gravel, or unable to cross small bumps or thresholds; also used when the device was explicitly described as unsafe when used outside
• Trip hazards: defined as the device or a part or parts of it being prone to tripping the user; also used when the device was described as frequently catching the user’s feet
• Design failure: defined as the device or a part or parts of it failing to safely assist the user because of a clearly identifiable design flaw not described by the previous 5 pathways (eg, poor ergonomic design, brake accessibility, or absence of brakes [by design]).

Of the 2888 reviews, 32 (1.11%) were deemed to provide insufficient context surrounding the injury to be accurately classified, and 128 (4.43%) had concerns related to shipping and inspection quality (missing parts, broken upon arrival, etc) rather than true safety concerns. Thus, of the 2888 reviews, 160 (5.54%) were removed, leaving 2728 (94.46%).

Statistical Methods

No hypotheses were specified a priori; therefore, no statistical tests were performed. The values presented in the following tables represent the normalized count per 10,000 reviews of the relevant product categories.

Ethical Considerations

Our research collected and analyzed secondary data from publicly available reviews on Amazon’s US website. The reviewers’ names were not included in the analysis. Reviews are posted on a public forum, where reviewers do not have an expectation of privacy. This type of research was therefore classified as exempt regarding institutional review board considerations. Our research did not involve direct personal interaction with any human participants.

Results

Injury-Type Distributions

Mobility-assistive device injury–type distributions and totals as well as sample cases are presented in Table 2 and Table 3, respectively. Major-injury distributions ranged from 0 per 10,000 reviews (ramps and wheelchairs or transport chairs) to a high of 10 (0.1%) per 10,000 reviews (walkers or rollators). Minor-injury distributions ranged from 22 (0.22%) per 10,000 reviews (ramps) to 57 (0.57%) per 10,000 reviews (wheelchairs or transport chairs). When combined with major injuries, per 10,000 reviews, 240 (2.4%) voice hazardous experiences entailing actual user injury. As for potential future injury, gait or transfer belts had the lowest count (344 instances per 10,000 reviews, 3.44%), suggesting that users may be at increased risk of actual injury while using a defective gait belt. By contrast, canes had the highest potential future injury count (614 instances per 10,000 reviews, 6.14%).

Injury Pathway Distributions

Injury pathway by mobility-assistive device–type distributions and totals are displayed in Table 4, and sample cases for each pathway are presented in Textbox 1. Canes had the highest Critical component breakage or decoupling rate (240 mentions per 10,000 reviews, 2.4%), closely followed by wheelchairs or transport chairs (222 mentions per 10,000 reviews, 2.22%). Gait or transfer belts had the (substantially) highest Unintended movement rate (184 mentions per 10,000 reviews, 1.84%), the next highest being walkers or rollators (101 mentions per 10,000 reviews, 1.01%).

Instability rates were highest among canes (180 mentions per 10,000 reviews, 1.8%) and ramps (129 mentions per 10,000 reviews, 1.29%), with users often reporting that they felt unsure about the item’s (ramp’s) ability to securely support them because it felt wobbly or unsteady. Poor, uneven surface handling was (mostly) limited to wheeled devices: wheelchairs or transport chairs (46 mentions per 10,000 reviews, 0.46%) and walkers or rollators (26 mentions per 10,000 reviews, 0.26%). Walkers or rollators and canes shared the highest Trip hazards rates (15 mentions per 10,000 reviews, 0.15%). As for Design failure, canes had the highest review rate (205 mentions per 10,000 reviews, 2.05%), followed by ramps (187 mentions per 10,000 reviews, 1.87%).

From the 6 identified pathways, Critical component breakage (855 mentions per 10,000 reviews, 8.55%), Design failure (699 mentions per 10,000 reviews, 6.99%), and Instability (457 mentions per 10,000 reviews, 4.57%) were found to be the most frequently encountered device failure mechanisms, also accounting for approximately 70% of all major injuries within the data set. Tables S6 and S7 in Multimedia Appendix 1 present key points for each category, rather than review snippets shown in Table 3 and Textbox 1. Moreover, normalized counts regarding device-specific failure mechanisms, discussed in the following section, can be found in Tables S3 and S8 in Multimedia Appendix 1. Table S9 in Multimedia Appendix 1 contains the data set’s nonnormalized (raw) injury-type distributions by mobility-assistive device.

Discussion

Principal Findings

We created a novel data set of safety concerns from reviews of mobility-assistive devices on Amazon’s US website. The data set provided a better understanding of (1) the contexts and conditions leading to user injury and (2) the severity of injuries associated with these mobility-assistive devices. Prior studies of older adult injuries while using mobility-assistive devices reported incidence rates of ED visits and types of injuries associated with mobility-assistive device use [18,19]. By contrast, this study provides novel information about pathways to injuries, including device critical component failures; unintended device movement; poor, uneven surface handling; instability; and trip hazards. Contrary to existing databases such as NEISS and MAUDE, our data set provides a unique consumer-focused perspective on user injury, emphasizing the context and manner in which the devices failed, while also providing basic information regarding injury (none, potential, minor, or major). Demonstrating that timely information can be extracted from online reviews, which are readily available and likely more frequently used than government voluntary reporting systems (owing to ease-of-use reportability) by consumers seeking to report product malfunctions.

Our findings suggest that many mobility-assistive device injuries may be preventable through patient and caregiver education about how to evaluate new and existing equipment for potential future injuries.

We found gait or transfer belts to have an unusually high rate of injuries linked to Unintended movement. Reviews generally described the belt slipping through the buckle or sliding up to the user’s chest, resulting in harm to the user. The contents of the reviews provide evidence that gait belt material should be antislip and that buckling systems may need to be redesigned to ensure that belts remain securely fastened. These findings align with those of nursing assistant gait belt reports in Garg et al [20], in which concerns about the belts’ tendency to slip up on the patient were expressed. Walkers or rollators also had relatively high Unintended movement rates because users regularly reported that these had faulty brakes, which were incapable of completely stopping device movement, or wheels (especially plastic) with poor traction, placing the older adult at increased fall risk.

Beyond weak braking systems, reviews of walkers or rollators also revealed that some were alarmingly susceptible to Critical component breakage or decoupling, having the third highest rate behind that of wheelchairs or transport chairs and canes. Reviews of canes identified bases, handles, and seats (for canes with attached seats) as frequent component breakage locations, with breakage often occurring while the device was being used. Walker and wheelchair user reviews reported similar component breakage issues for both product categories, with mentions of handles or frames snapping and seats failing being fairly common (paralleling canes), in addition to wheels or legs falling off or breaking and screws or nuts coming loose. Given that older adults physically rely upon these devices, repeated mentions of primary user-support–component breakage (handles and seats), as well as critical device-support–location breakage (cane bases, walker legs and wheels, and wheelchair wheels) are extremely worrisome. Furthermore, it should be noted that Critical component breakage or decoupling accounted for nearly half of all major injuries within the data set, indicating that re-examination (in light of component breakage) of these mobility-assistive devices may be necessary because serious injuries seem to be occurring because of defective items (ie, sudden breakage), rather than user misuse. Hence, although several studies [21-23] have explored the benefits of adopting such devices during late adulthood, more research is still needed to continue improving the design of these mobility-assistive aids [19].

Wheelchairs or transport chairs and walkers or rollators also lead Poor, uneven surface handling counts, with reviews from both device categories describing instances in which these devices were incapable of safely crossing over unlevel surfaces such as seams (as small as tile grout), thresholds, and uneven pavement. These reviews provide further evidence of the need for re-examination of these 2 mobility-assistive device categories, especially walkers or rollators, which also accounted for more than three-quarters of all identified major injuries and had the highest Trip hazards rates (alongside canes). Stevens et al [19] also emphasized the need to continue improving the design of mobility-assistive devices, particularly walkers, because they were found to be associated with 7 times as many injuries as canes.

With regard to walker or rollator and cane Trip hazards, two-thirds of the data set’s walker or rollator cases were attributed to rear wheel or leg placement being too far back, whereas >80% of the cane incidents (within the data set) were associated with quad cane use. Albeit easily addressed with simple design modifications, both Trip hazards issues stem from the desire to increase stability, meaning that delicate adjustment is necessary, as evidenced in Mortenson et al [24], to maximize the device’s ease of use while avoiding a permanent decrease to desired stability capabilities.

Regarding this, we found the bases of quad and tripod canes to be highly susceptible to instability, comprising more than two-thirds of identified Instability cane instances, implying that, contrary to popular belief, larger cane bases do not to lead to increased stability; rather, they seem to generate more instability than traditional canes. Reviews of ramps also regularly expressed that the device felt unstable, describing it as bending, being depressed, or wobbling in the middle when loaded with moderate weight, generating insecurity and at times forcing the user to implement makeshift (and potentially unsafe) solutions to remediate the issue.

Aside from instability, users frequently reported these 2 product categories (canes and ramps) as having design-related issues (Design failure). With regard to ramps, reviews often described them as having sharp edges, increasing the risk of laceration when handling the device and (potentially) aggravating the consequences of a fall injury, or being too steep, leading to increased fall risk and abnormal gait mechanics to control descent. Cane design issues were commonly associated with poor ergonomic handle design, causing pain while in use, and poor self-standing capabilities (quad or tripod canes), forcing users to repeatedly bend down to pick up the device, potentially leading to, or exacerbating existing, back issues.

In support of injury prevention efforts for this population classified as susceptible, our findings may inform educational interventions to inform older adults and their caregivers about safety issues associated with mobility-assistive equipment. Educational interventions would include information about how to select safe mobility-assistive products, how to fit the mobility-assistive device to the user, how to safely use the newly adopted mobility-assistive devices, and how to assess the safety of the product over time. Moreover, beyond our available data, which provide actionable consumer safety information, future use of the presented injury pathway categories may aid businesses and legislators seeking to monitor device safety because these may allow for more specific search parameters and more effective monitoring of frequent device failure mechanisms.

Limitations

This study includes several limitations. Although the source data span >600,000 reviews and dozens of categories, we focused only on a subset of 1 category (mobility-assistive devices). We analyzed only injury incidents and concerns actively self-reported by older adults and caregivers within online mobility-assistive device reviews on a single major marketplace (Amazon’s US website); products listed by other retailers may yield different findings. As the sample is subject to self-selection bias, results should be taken as indicative of injury occurrence, not representative of injury occurrence.

We selected Amazon over other retailers because it is one of the largest and most popular online retailers in the United States, containing dedicated assistive device categories offering hundreds of different brands and products. Selecting a smaller retailer would have likely yielded fewer data, with diminished brand and product diversity. However, because we restricted our sample to reviews on Amazon’s US website, the study does not include experiences concerning devices acquired through medical device stores or specialized pharmacies, which may offer device-fitting services and have trained staff to assist in the selection and purchase of the correct device. As Amazon does not provide any of these services, consumers must conduct their own research at the time of purchase, which may result in certain negative experiences owing to poor equipment selection and fit rather than actual equipment malfunction.

Regardless, there may be skew in who is reporting issues via product reviews and thus some underreporting bias; however, our goal was to surface issues that regulators and manufacturers previously were not aware of to focus remediation efforts, rather than the conventional epidemiological concern of quantifying the incidence and distribution of incidents. The manageable volume of substantial issues surfaced by our method means that regulators and manufacturers can manually investigate the veracity of each report, as well as prioritize and plan remediation efforts.

In addition, within the curated data set, less frequently adopted devices, namely, gait or transfer belts and ramps, have considerably smaller sample sizes than categories such as canes and walkers or rollators. Beyond increased bias, this affects study results in that both normalized (Tables 2 and 4) and raw counts (Tables S4 and S5 in Multimedia Appendix 1) may not be indicative of their injury types and pathways. Most importantly, additional product malfunction mechanisms may have remained hidden because of their low popularity, resulting in the study missing critical information concerning both these devices.

Finally, although the large-scale coding efforts resulted in 100,000 coded reviews, because of limited volunteer labor resources, only instances coded initially as “major injury,” “minor injury,” or “potential future injury” (n=6400, 6.4%) by the volunteers were verified by the authors and doctoral student. Future work could explore the possibility of training machine learning classifiers on the data set to use the models to rapidly annotate hundreds of thousands of reviews, which could then be filtered by desired injury and product type and manually analyzed.

Conclusions

We introduced a novel online review data set containing safety concerns for mobility-assistive devices typically used by older adults. Our findings highlight device-specific contexts within which these devices are harming older adults and indicate that older adults and their caregivers tend to attribute serious injuries to defective items, rather than misuse.