Published on in Vol 18, No 10 (2016): October

The Effectiveness of Lower-Limb Wearable Technology for Improving Activity and Participation in Adult Stroke Survivors: A Systematic Review

The Effectiveness of Lower-Limb Wearable Technology for Improving Activity and Participation in Adult Stroke Survivors: A Systematic Review

The Effectiveness of Lower-Limb Wearable Technology for Improving Activity and Participation in Adult Stroke Survivors: A Systematic Review

Original Paper

School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, United Kingdom

Corresponding Author:

Lauren Powell, BSc (Hons), MSc

School of Health and Related Research (ScHARR)

University of Sheffield

Regent Court, 30 Regent Street

Sheffield, S14DA

United Kingdom

Phone: 44 1142228275

Fax:44 1142220749

Email: l.a.powell@sheffield.ac.uk


Background: With advances in technology, the adoption of wearable devices has become a viable adjunct in poststroke rehabilitation. Regaining ambulation is a top priority for an increasing number of stroke survivors. However, despite an increase in research exploring these devices for lower limb rehabilitation, little is known of the effectiveness.

Objective: This review aims to assess the effectiveness of lower limb wearable technology for improving activity and participation in adult stroke survivors.

Methods: Randomized controlled trials (RCTs) of lower limb wearable technology for poststroke rehabilitation were included. Primary outcome measures were validated measures of activity and participation as defined by the International Classification of Functioning, Disability and Health. Databases searched were MEDLINE, Web of Science (Core collection), CINAHL, and the Cochrane Library. The Cochrane Risk of Bias Tool was used to assess the methodological quality of the RCTs.

Results: In the review, we included 11 RCTs with collectively 550 participants at baseline and 474 participants at final follow-up including control groups and participants post stroke. Participants' stroke type and severity varied. Only one study found significant between-group differences for systems functioning and activity. Across the included RCTs, the lowest number of participants was 12 and the highest was 151 with a mean of 49 participants. The lowest number of participants to drop out of an RCT was zero in two of the studies and 19 in one study. Significant between-group differences were found across three of the 11 included trials. Out of the activity and participation measures alone, P values ranged from P=.87 to P ≤.001.

Conclusions: This review has highlighted a number of reasons for insignificant findings in this area including low sample sizes, appropriateness of the RCT methodology for complex interventions, a lack of appropriate analysis of outcome data, and participant stroke severity.

J Med Internet Res 2016;18(10):e259

doi:10.2196/jmir.5891

Keywords



The worldwide incidence of stroke is set to escalate from 15.3 million to 23 million by 2030 [1]. In the United Kingdom, strokes are the largest single cause of disability [2] resulting in a cost to the economy of £8.9 billion a year [3]. It is estimated that following a stroke, only 15% will gain complete functional recovery for both the upper and lower extremities [4] with walking and mobility being key issues for many stroke survivors who report the importance of regaining mobility [5]. However, with the ever-increasing financial challenges facing the National Health Service (NHS), service needs cannot be met. Therefore, utilizing information and communication technology together with the implementation of well-evidenced medical technologies is essential for continued rehabilitation for stroke survivors.

The adoption of technological solutions can facilitate patient and caregiver empowerment and a paradigm shift in control and decision making to that of a shared responsibility and self-management [6]. It also has the potential to reduce the administrative burden for care professionals and support the development of new interventions [7]. Incorporating technology into the daily lives of stroke survivors is a key objective in safeguarding a better quality of life for them.

Evidence exists supporting the need for intensity and repetition of motor skills in order to promote neuroplasticity and motor relearning [8]. A number of technological aids with a potential to enhance poststroke motor recovery has been explored [9]. However, many include the use of expensive, large, complex, cumbersome apparatus that necessitates the therapist to be present during use [10]. Therefore inexpensive, externally wearable, commercially available sensors have become a more viable option for independent home-based poststroke rehabilitation [11].

Recent systematic and non-systematic reviews highlight the growing use of externally wearable devices to augment poststroke rehabilitation in both clinical and non-clinical settings for motion analysis and physical activity monitoring [12-15]. These include microelectromechanical systems containing accelerometers, gyroscopes, and magnetometers; fabric and body-worn sensor networks [16]; and physiological monitoring such as blood pressure and oxygen saturation [17,18]. Other wearable devices specifically designed and used for poststroke rehabilitation also include robotics [19], virtual reality [20], Functional Electrical Stimulation (FES) [21], electromyographic biofeedback (EMG-BFB) [22], and Transcutaneous Electrical Nerve Stimulation (TENS) [23,24].

However, while these devices have the potential to reliably measure duration, frequency, intensity, and quality of activity and movement, all of which are key variables for poststroke recovery [8], no reviews have synthesized the effectiveness of these devices for poststroke lower-limb rehabilitation.

The International Classification of Functioning, Disability and Health (ICF) [25] considers the interaction between pathology (body structure and function), impairment (signs and symptoms), activities (functionality), and participation (social integration) and has now become the main conceptual framework for poststroke rehabilitation [26-28]. For this review, we focused on the activities and participation domain of the ICF as this would provide an indication of how the interventions have or have not led to functional gains in everyday life, which is the rehabilitation goal for both clinicians and stroke survivors [28].

Therefore, the aim of this review was to examine how effective external wearable devices are as interventions for improving function of the lower limb in adult stroke survivors.


The review protocol was registered on PROSPERO (CRD42015020544). The review was undertaken in accordance with the general principles recommended in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [29].

Search Methods

The following databases were searched from inception to March 2016: MEDLINE, Web of Science (Core collection), CINAHL, and the Cochrane Library. Medical Subject Headings (MeSH) keywords used were cerebrovascular disorders, hemorrhage, cerebral hemorrhage, self-help devices, telemedicine, physical therapy modalities, physical and rehabilitation medicine, exercise, exercise therapy, exercise movement techniques, self-evaluation programs, sensory feedback, motor skills, gait disorders, neurologic, gait apraxia, and gait ataxia. Text terms used were stroke, technology, physiotherapy, lower limb, rehabilitation, and gait. These were combined with text term synonyms: cerebrovascular accident (CVA), poststroke, cerebrovascular, brain ischemia, IT (information technology), ICT (information and communications technology), assistive technology, telehealth, telecare, telerehabilitation, physical therapy, physiatric, exercise, lower extremity, lower limb, ambulant, walk, locomotion, mobile, move, motion, biofeedback, sensory feedback, advise, result, evaluation, observe, assess, inform, train, therapy, treat, motor skills, motor re-learn, re-educate, re-learn, recovery enhance, promote, support, function, activity, physical, ambulant, and walking. Terms were combined using Boolean logic (“AND”, “OR”). MeSH are specific recognized terms used for the purpose of indexing journal articles and books in electronic databases. Free text terms and synonyms are specific words that the search strategy looks for in the title and abstract.

A copy of the MEDLINE search strategy is presented in Multimedia Appendix 1. Electronic citations were downloaded to Endnote software. The inclusion criteria are described in Table 1.

Table 1. Inclusion and exclusion criteria for this review.
Inclusion criteriaExclusion criteria
English language articlesStudies including upper limb
Studies recruiting people over the age of 18 yearsStudies where the intervention is not clearly defined
Studies evaluating lower-limb and wearable technologyStudies not using one of the chosen 11 outcome measures (see Outcome measurement/assessment below)
Studies reporting an RCTaStudies not reporting an RCTa
Studies measuring activity and participation as classified by the World Health Organization ICFbStudies not measuring activity and participation as classified by the World Health Organization ICFb

aRCT: randomized controlled trial.

bICF: International Classification of Functioning, Disability and Health.

As this is a review of effectiveness, RCTs were chosen as the appropriate study design to answer the research question. Inclusion of non-RCT evidence is outside the scope of this review.

Comparators could be exercise/physical therapy, sham stimulation, conventional gait therapy, or treatment as usual. The primary outcome for this review was changes in activity and participation assessed by any of the following methods: the Rivermead Mobility Index, the Barthel Index, the Berg Balance Scale, the Six Minute Walk Test, the Functional Ambulatory Category, the Timed Up and Go test, the Motricity Index, the Stroke Self-Efficacy Scale, and the Performance-Oriented Mobility Assessment.

Quality Assessment

Methodological quality of included RCTs was assessed using the Cochrane Collaboration risk of bias assessment criteria [30]. This tool addresses specific domains, namely, sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective outcome reporting. For the selective reporting domain, a proxy judgement was made that if a trial reported that a study protocol had been approved and the trial report described primary and secondary outcomes with results, then the trial could be considered at low risk of selective reporting bias. We classified RCTs as being at overall low risk of bias if they were rated as “low” for each of three key domains: (1) allocation concealment [31], (2) blinding of outcome assessment, and (3) completeness of outcome data. RCTs judged as being at high risk of bias for any of these domains were judged at overall high risk. Similarly, RCTs judged as being at unclear risk of bias for any of these domains were judged at overall unclear risk.

Data Extraction

Retrieved titles, abstracts, and/or papers were screened independently by 2 review authors (LAP, JP) to identify studies that met the inclusion criteria. Disagreements were resolved between reviewers through discussion. A standardized form was used for data extraction using Excel. Details of the RCT characteristics, included participants, the intervention, and comparator. Data extraction was carried out by reviewer LP and checked for accuracy by reviewer JP. Missing data were requested from study authors.

Outcome Measurement Assessment

When undertaking a systematic review, it is essential that the quality of the outcome measures used in each study is assessed in order to ensure that the results of the study are valid and reliable. In order to do this, three clear domains need to be considered for each of the outcome measures used: (1) whether the psychometric properties of the scale have been assessed previously [32], (2) whether the clinimetric properties of the scale have been considered [33-37], specifically the Minimally Clinically Important Difference (MCID) [36], and (3) whether the design and analysis of the measurement scale fulfils the requirements of measurement theory [38-40].

We identified all the outcome measures (N=19) used in the 11 trials and reviewed each individually to assess whether they fulfilled the first two domains outlined above. The outcome measures were:

  • The Rivermead Mobility Index (RMI)
  • 10 Meter Walk Test (10MWT)
  • Nottingham Activities of Daily Living Index (ADL)
  • The Barthel Index (BI)
  • The Berg Balance Scale (BBS)
  • 6 Minute Walk Test (6MWT)
  • Functional Ambulatory Category (FAC)
  • Timed Up and Go Test (TUG)
  • Emory Functional Ambulation Profile (EFAP)
  • Short Physical Performance Battery (SPPB)
  • Performance-Oriented Mobility Assessment (POMA)
  • Motricity Index (MI)
  • Average Daily Walking Time
  • Fastest Safe 15-meter Walking Speed
  • Changes in Walking Duration
  • Step Numbers
  • Daily Walking Activities with an average cadence of walking events (bouts)
  • Stroke Impact Scale (SIS)
  • Stroke Self-Efficacy Questionnaire (SEQ)

This was established by reviewing the literature on each of the measuring scales. We then examined each measurement scale to establish how the data were scored and how data collected were subsequently analyzed within the results section of each trial.

We classified the measures against the three domains within the World Health Organization ICF, as the aim of this review was to assess the effectiveness of lower-limb wearable technology for improving activity and participation. We wanted to exclude any measurements of “body structures” (impairment) such as the Fugl-Meyer assessment or the Ashworth scale. All 19 outcome measures included were measures of “activity” and 2 were measure of “participation” as classified by the ICF [23].

Data Synthesis

We have presented a narrative overview of the included RCTs with supporting evidence tables and text. A meta-analysis was not undertaken.


Search Results

The electronic searches identified 940 citations following de-duplication. No additional citations were identified through reference searches/other sources. We excluded 780 citations at the title and 128 at abstract stage. We then obtained 32 citations as full-text articles. Of these, 21 were excluded at the full-text stage; details of these excluded studies with the reason for exclusion are shown in Multimedia Appendix 2 [41-59]. Eleven RCTs reported across 11 publications were included in the review (see Figure 1).

Quality Assessment

Full details from the Cochrane risk of bias assessment are presented in Multimedia Appendix 3. A summary of the risk of bias assessment is presented in Table 2, and a summary of the outcome measurement quality assessment can be found in Multimedia Appendix 4.

Seven of the 11 included RCTs were considered to be at overall high risk of bias [60-66]. Six of these were judged to be at high risk of an attrition bias [60-63,65,66], and two reported that the outcome assessment was not blinded [64,66]. The remaining three RCTs were considered to be at overall unclear risk of bias. None of the included RCTs were considered to be at high risk for the concealment of allocation domain.

Table 2. Risk of bias summary.

Random
sequence
generation
Allocation
concealment
Blinding of
participants and
personnel
Blinding of
outcome
assessment
Incomplete
outcome data
Selective
reporting
Overall
Bauer, 2015 [60]Low riskLow riskHigh riskLow riskHigh riskLow riskHigh risk
Bradley, 1998 [61]UnclearUnclearUnclearUnclearHigh riskUnclearHigh risk
Dorsch, 2015 [62]Low riskUnclearUnclearLow riskHigh riskLow riskHigh risk
Intiso, 1994 [63]UnclearUnclearUnclearLow riskHigh riskUnclearHigh risk
Mansfield, 2015 [67]Low riskLow riskUnclearLow riskLow riskLow riskLow risk
Mirelman, 2009 [68]UnclearUnclearHigh riskUnclearLow riskUnclearUnclear
Salisbury, 2013 [64]Low riskLow riskUnclearHigh riskLow riskUnclearHigh risk
Shamay, 2009 [65]Low riskUnclearHigh riskLow riskHigh riskUnclearHigh risk
Solopova, 2011 [69]UnclearUnclearUnclearUnclearLow riskUnclearUnclear
Stein, 2014
[14]
UnclearUnclearUnclearUnclearLow riskLow riskUnclear
Watanabe, 2014 [66]High riskUnclearHigh riskHigh riskHigh riskLow riskHigh risk
Figure 1. Selection of articles for review.
View this figure

Quality Assessment of Measurement Scales

Eight of the 11 [14,60-62,64,66-68] included RCTs used a combination of ordinal and ratio scales of measurement all with established psychometric properties; however, it was unclear what the minimally clinically important difference (MCID) was for the ratio data. Therefore, the clinical significance of the findings is difficult to establish. Two papers [63,69] used the Bartell Index alone, which has been proven not to be a unidimensional scale. Therefore, the analysis of the data was inappropriate, putting the findings at risk. One paper [65] used ratio levels of measurement, but again the MCID was unclear. Relative results are not reported in the RCTs.


Principal Findings

This review set out to answer the question “What is the effectiveness of lower-limb wearable technology for improving activity and participation in adult stroke survivors?” The review found that there is little evidence in the literature to support the use of wearable technologies to improve activity and participation. Following exclusions, outcome measure assessment and quality assessment of RCTs, 11 studies were included (see Table 3).

The interventions used in eight of the 11 RCTs identified made no significant between-group differences in functional and participation abilities in adults post stroke. Three of the 11 studies did demonstrate significant between-group differences. One study that found significant between-group differences recruited 109 participants [65] comparing TENs together with a task-related exercise program modified from the training programs [70] with placebo and exercise and a control with no active treatment. The study provided evidence that the subjects receiving this intervention in a home environment had a significantly greater absolute and percentage increase in gait velocity and a reduction in timed get up and go scores from Week 2 onwards.

Another study recruited 60 participants [67] where all participants wore accelerometers around both ankles and were randomly assigned to either receive feedback on the accelerometer data from their physiotherapist or to not receive feedback. The study provided evidence that providing feedback to the participants significantly improved their cadence of daily walking.

The third study recruited 40 participants [60] where all participants underwent 20 minutes of active leg cycling with or without FES application to the muscles of the paretic upper leg. The study provided evidence for the intervention improving participants gait and balance (measured using the POMA); however, these improvements were not sustained when participants were followed up. It could be argued, however, that the high dropout rate (n=19) could have affected the significance of the lasting effects of the study.

As described fully in the quality assessment section of this paper, seven of the 11 included RCTs were considered to be at overall high risk of bias [66]. However, this does not mean that the interventions were not effective for improving gait for people post stroke. A number of conclusions could be drawn from this result. One may be that interventions that rely heavily on direct clinical input may not be suitable for this population where self-managed interventions may be more appropriate.

A number of measurement scales used in the trials were not incorporated in the outcome data for the review, as they were not validated scales: the Bobath scale [61], the 5X Sit-To-Stand-Test [14], and the California Functional Evaluation 40 [14]. Of the 11 RCTs included in the review, eight used a combination of ordinal and ratio data with proven psychometric properties; however, the clinicmetric properties were not described. The lack of evidence, therefore, in eight studies could have been due to the lack of a clinically meaningful, responsive outcome measurement scale combined with a potential lack of statistical power due to small sample sizes. The three studies that did have significant results used a combination of ordinal and ratio data with only one study [60] that provided estimates of MCID together with appropriate anayisis of the FAC data. While Shamay et al did not consider the clinical meaning or significance of the change in scores, they did report research supporting the “practical significance” of the TUG [71], which found that older adults who were able to complete the TUG task in less than 20 seconds were more likely to be independent in the transfer tasks needed for activites of daily living.

The results from this systematic review should be generalized to a wider stroke population cautiously due to the low recruitment figures for the majority of the included RCTs. Observations of lack of efficacy should also be interpreted with caution, given the uncertainty surrounding the methodological quality of the existing evidence base. Only a small number of papers with small sample sizes were able to be included in this review. Three of the selected studies recruited fewer than 20 participants [61,63,64,72], and only two recruited over 100 participants [62,65]. This could be for a number of reasons including difficulty to recruit a poststroke population to such studies. Despite the plethora of research in poststroke gait research, only 11 RCTs were selected for this review. This could be due to the difficulty of including complex interventions within an RCT design.

An RCT aims to control conditions for each arm of the study, frequently aggregating group data to provide mean values. However, no stroke is the same, recovery varies across individuals, and recovery is naturally accelerated soon after the stroke compared to those who suffered a stroke a long time ago. These factors coupled with different causes and different types of stroke, make it very difficult to control each arm of a study. Therefore, it is difficult to infer if certain interventions improve functionality post stroke or if other variables are responsible. Exploring individual change over time particularly when evaluating novel technologies with complex conditions may provide more valuable information. It has been suggested [73] that the integration of a realist evaluation perspective within an RCT design may be more appropriate and a paradigm shift for evidence-based medicine where “statistically significant benefits may be marginal in clinical practice” [74].

Table 3. Study, participant, and intervention characteristics and results.
Authors, year, country, study designNumber recruited (N) & final follow-up (n) overall and between groupsGender, mean age, L/Ra hemisphere stroke, mean time since strokeIntervention length/ frequencyActivity and participation outcome measure(s)Activity and participation outcome results summary and reported P values
Bauer et al, 2015 [60], Austria, monocentric single-blinded RCTb, active leg cycling with (intervention) and without (control) FEScN (n)=40 (21). 21 (12), intervention; 19 (9), control9M/9Fd, 64±11 years, 10R/8L, 42±45 days (control), 12M/7F, 59±14 years, 5R/14L, 62±43 days (intervention)20 mins, 3x/week over 4 weeks. Total of 12 sessionsFACe, 10MWTfThe intervention group increased by a median of 2 categories for the FAC and a median of 1 category for the control group (P=.01). No significant between-group differences found for the 10MWT (P=.65). Significant between-group differences found for the POMAg (P ≤.001); however, these differences were not maintained at follow-up (P=.69)
Bradley et al, 1998 [61], UK, 2-arm RCT, EMGh biofeedback (intervention) or EMG biofeedback with EMG switched off (control)N (n)=23 (21); 12 (12) intervention, 11 (9) control12M/11F, 77/68 yrs (mild/severe control), 66.6/72.4yrs (mild/se- vere intervention), 5L/16R, 35.6 days6 weeks/ 3x/weekRMI, 10MWT, Nottingham ADLNo significant between-group differences (RMI, 10MWT, Notting- ham ADL), although all groups improved in time taken and step count for the 10MWT and all groups improved their Nottingham ADL scores
Dorsch et al, 2015 [62], USA, Phase III randomized single-blind parallel group clinical trial, participants wore accelerometers on each ankle and received speed-only feedback [67] or AFkN (n)=151 (125). 73 (58) SF; 78 (67) AF28%F/72%M , 65.0 ± 13.2yrs, 42%R/29%L, 8.5days [67]; 31%F/69%M, 61.8 ± 15.7yrs, 44%R/34%L, 8days (AF)Feedback provided 3x/week, weekend use of accelerometers was optionalFACNo significant between-group differences found for the FAC (P=.39), SISl-16 (P=.68), 15-M walking speed (P=.96) or average daily walking time (P=.54)
Intiso et al, 1994 [63], Italy, 2-arm RCT, electromyographic feedback and physical therapy (intervention) or physical therapy only (control)N (n)=16 (14), 8 (8) intervention, 8 (6) control)9M,/7F, 53.5yrs (control), 61.3yrs (intervention), 9R/7L, 8.3 months (control), 11.3 months (intervention)2 months/60 mins dailyBImNo significant between-group differences (BI), 4/8 participants found to have significant increased BI scores
Mansfield et al, 2015 [67], Canada, single-blind RCT, accelerometer with (intervention) and without (control) feedback from physiotherapistN (n)= 60 (57). 29 (29) intervention; 31 (28) control20M/9F, 64yrs, 11R/16L/2B, 26 days (intervention) 16M/12F, 61.5yrs, 13R/13L/2B, 23 days (control)3-26 days per participant in each group. Mode=11 days per participantBBSnNo significant between-group differences step numbers (P=.39), changes in walking duration (P=.74), number of walking bouts (P=.21) or the SEQo (P=.48). Significant between-group differences found for daily walking activity with average cadence (P=.01)
Mirelman et al, 2009 [68], USA, 2-arm single-blind RCT, training with robotic device coupled with virtual reality training (intervention) or robotic device alone (control)N (n)=18 (18), 9 (9) intervention, 9 (9) control)15M/3F, 61yrs (control), 61.8yrs (intervention), 8R/10L, 58.2 months (control), 37.7 months (intervention)4 weeks/60 mins 3x/weekBBS, 6MWTpNo significant between-group differences (6MWT), BBS results/ P values not reported
Salisbury et al, 2013 [64], Scotland, 2-arm feasibility RCT, routine gait re-education and orthotic device (intervention and control) with ankle foot orthosis (control) or FES (intervention)N (n)=16 (14). 9 (8) intervention, 7 (6) control6M/10F, 52.6yrs (control), 55.8yrs (intervention), 10R/6L, 69days (control), 51.7 days (intervention)12 weeks/20 mins 5 days/ weekFAC, 10MWT (velocity & cadence), SISNo significant between-group differences observed (FAC 6 weeks P=.53, 12 weeks P=.75; 10MWT velocity/cadence 6 weeks P=.46/ P=.24, 12 weeks P=.87; SIS 6 weeks P=.1, 12 weeks P=.3)
Shamay, 2009 [65], Hong Kong, 4-arm placebo RCT, 1. transcutaneous electrical nerve stimulation [23], 2. TENSq+Exercise, 3. Placebo stimulation+exercise, 4. control group (no active treatment) – home-based programN (n)=109 (101). 29 (27) control, 28 (25) TENS, 25 (23) placebo+Exr, 27 (26) TENS+Ex85M/24F, 56.5yrs, 57.8yrs (TENS+Ex), 56.9yrs (placebo stimulation+Ex), 55.5yrs (control), 10%R/18%L [23], 10%R/17%L (TENS+Ex), 12%L/13%R (placebo stimulation+Ex), 9%L/20%R (control), 4.9yrs [23], 4.7yrs (TENS+Ex), 4.3yrs (placebo stimulation+Ex), 5yrs (control)4 weeks/TENS: 60 mins electrical stimulation, TENS+Ex & placebo stimulation + Ex 60 mins of Ex then 60 mins electrical or placebo stimulation. Subjects attended 8 instruction sessions prior to data collection6MWT, TUGsCompared to all other groups, TENS+Ex group showed significant decreased TUG results (P=.01) when compared to the control and TENS group, they cov- ered more distance during the 6MWT (P ≤.01)
Solopova et al, 2011 [69], Russia, 2-arm RCT, conventional therapy and FES combined with progressive limb loading (intervention) or conventional therapy only (control)N (n)=61 (61). 32 intervention, 29 control33M/28F, 64±18yrs, 19R/42L, 9.3±4.5 days (control), 8.2±4.3 days (intervention)2 weeks/30 mins 5 days per weekBINo significant between-group differences, Significant improvements after the intervention in the experimental group were observed (BI P ≤.05)
Stein et al, 2014 [14], USA, 2-arm RCT, exercise group therapy (control) or experimental robotic therapy (intervention).N (n)=12 (10), 12 (10) intervention, 12 (10) control)58%M (control), 83%M (intervention), 57.6yrs (control), 56.6yrs (intervention), L/R stroke not reported, 88.5 months (control), 49.1 months (intervention)6 weeks/60 mins 3 days per weekBBS, 6MWT, TUG, 10MWT, EFAPtBBS scores favored the intervention group and the EFAP scores favored the control group. No statistically significant between-group differences observed (BBS, 6MWT, TUG, 10MWT, EFAP)
Watanabe et al, 2014 [66], Japan, 2-arm RCT single leg version of HALu (intervention) or conventional gait training (control).N (n)=32 (22). 17 (11) intervention, 15 (11) control11M/11F, 75.6±13.9 (control), 67.0±16.8 (intervention), 11R/11L, 50.6±33.8 days (control), 58.9±46.5 days (intervention)4 weeks/12 20-min sessions6MWT, FAC, TUG, SPPBvNo significant between-group differences were observed (6MWT, TUG, FAC, SPPB). Intervention group improved more than the control group (FAC P=.04)






aL/R/B: left/right/both hemisphere stroke.

bRCT: randomized controlled trial.

cFES: functional electrical stimulation.

dM/F: male/female.

eFAC: functional ambulatory category.

f10MWT: 10 Meter Walk Test.

gPOMA: Performance-Oriented Mobility Assessment.

hEMG: electromyography.

iRMI: Rivermead Mobility Index.

jADL: activities of daily living.

kAF: augmented feedback.

lSIS: Stroke Impact Scale.

mBI: Barthel Index.

nBBS: Berg Balance Scale.

oSEQ: Stroke Self-Efficacy Questionnaire.

p6MWT: 6 Minute Walk Test.

qTENS: transcutaneous electrical nerve stimulation.

rEx: exercise.

sTUG: Timed Up and Go Test.

tEFAP: Emory Functional Ambulation Profile.

uHAL: Hybrid Assistive Limb.

vSPPB: Short Physical Performance Battery.

The results of the RCTs were not combined for a meta-analysis due to the varied types and quality of data collected for the primary outcome measures. It would also be difficult to compare primary outcomes across RCTs accurately as there were a wide variety of functional and participation outcome measures used across the 11 RCTs, some of which lacked validity as a measure of activity and participation.

Evidence exists supporting the need for task specificity, intensity, and repetition of motor skills in order to promote neuroplasticity and motor relearning; however, seven of the interventions in this review of RCTs were reliant on staff presence. This automatically eliminates the ability of stroke survivors to self-manage their rehabilitation, increasing both intensity and repetition within a task-specific environment.

This review included 550 participants at baseline and 474 participants at final follow-up, 260 from two studies alone [62,65]. Stroke severity can affect the rate by which individuals recover from a stroke and how they may or may not respond to interventions. Only two [61] of 11 papers in this review reported the stroke severity of their participants. Perhaps the severity was low and therefore it was difficult to infer a significant improvement of function. One paper [65] reported clinically and statistically significant results for the use of lower-limb wearable technologies with rehabilitation, although the technology was TENs, a technology that may not support a self-management paradigm and is not always tolerated by stroke survivors.

Perhaps future research should consider larger sample sizes, with valid, reliable, and responsive measurement tools ensuring clarity when reporting outcomes. Population descriptors should be used when exploring technology enhanced self-management models of poststroke rehabilitation. Outcome measures should be chosen only if they have psychometric or clinimetric properties reported. Where possible, individuals’ change over time should be captured and analyzed to ensure we begin to understand what works for whom and in what respect [75].

Conclusion

This review found that there is little evidence in the literature to support the use of wearable technologies to improve activity and participation following a stroke. However, this review has highlighted a number of reasons for a lack of significant findings including low sample sizes, the appropriateness of RCT methodology for complex interventions, a lack of appropriate analysis of outcome data, and participant stroke severity.

Acknowledgments

This research was funded/supported by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care Yorkshire and Humber. The views and opinions expressed are those of the authors and are not necessarily those of the National Health Service, the National Institute for Health Research, or the UK’s Department of Health.

Authors' Contributions

LP carried out the search strategy and wrote the methods section with MMSJ. LP updated the review, and all authors updated their sections. JP acted as a second reviewer for accuracy and wrote the introduction. SM wrote the discussion and the outcome measurement assessment. MMSJ acted as an advisor for the searches and carried out the risk of bias assessment. All authors read and approved the final manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

MEDLINE search strategy.

PDF File (Adobe PDF File), 23KB

Multimedia Appendix 2

Papers and reasons for exclusion.

PDF File (Adobe PDF File), 18KB

Multimedia Appendix 3

Details of quality assessment.

PDF File (Adobe PDF File), 47KB

Multimedia Appendix 4

Summary of outcome measure quality assessment.

PDF File (Adobe PDF File), 36KB

  1. Strong K, Mathers C, Bonita R. Preventing stroke: saving lives around the world. Lancet Neurol 2007 Feb;6(2):182-187. [CrossRef] [Medline]
  2. Department of Health: National Stroke Strategy. Department of Health. London: Crown Copyright; 2007.
  3. Saka O, McGuire A, Wolfe C. Cost of stroke in the United Kingdom. Age Ageing 2009 Jan;38(1):27-32 [FREE Full text] [CrossRef] [Medline]
  4. Hendricks HT, van LJ, Geurts AC, Zwarts MJ. Motor recovery after stroke: a systematic review of the literature. Arch Phys Med Rehabil 2002 Nov;83(11):1629-1637. [Medline]
  5. Luker J, Lynch E, Bernhardsson S, Bennett L, Bernhardt J. Stroke Survivors' Experiences of Physical Rehabilitation: A Systematic Review of Qualitative Studies. Arch Phys Med Rehabil 2015 Sep;96(9):1698-1708.e10. [CrossRef] [Medline]
  6. Mawson S, Nasr N, Parker J, Davies R, Zheng H, Mountain G. A Personalized Self-Management Rehabilitation System with an Intelligent Shoe for Stroke Survivors: A Realist Evaluation. JMIR Rehabil Assist Technol 2016 Jan 07;3(1):e1. [CrossRef]
  7. National Information Board: Personalised HealthCare 2020. Using Data and Technology to Transform Outcomes for Patients and Citizens: A Framework for Action. 2014.   URL: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/384650/NIB_Report.pdf [accessed 2016-09-19] [WebCite Cache]
  8. Kleim JA, Jones TA. Principles of experience-dependent neural plasticity: implications for rehabilitation after brain damage. J Speech Lang Hear Res 2008 Feb;51(1):S225-S239. [CrossRef] [Medline]
  9. Stein J, Hughes R, Hogan N. Technological Aids for Motor Recovery. In: Stein J, Macko RF, Winstein CJ, Zorowitz RD, editors. Stroke Recovery and Rehabilitation. New York: Demos Medical; 2009:307.
  10. Parker J, Mountain G, Hammerton J. A review of the evidence underpinning the use of visual and auditory feedback for computer technology in post-stroke upper-limb rehabilitation. Disabil Rehabil Assist Technol 2011;6(6):465-472. [CrossRef] [Medline]
  11. Steins D, Dawes H, Esser P, Collett J. Wearable accelerometry-based technology capable of assessing functional activities in neurological populations in community settings: a systematic review. J Neuroeng Rehabil 2014;11:36 [FREE Full text] [CrossRef] [Medline]
  12. Gebruers N, Vanroy C, Truijen S, Engelborghs S, De Deyn P. Monitoring of physical activity after stroke: a systematic review of accelerometry-based measures. Arch Phys Med Rehabil 2010 Feb;91(2):288-297. [CrossRef] [Medline]
  13. Patel S, Park H, Bonato P, Chan L, Rodgers M. A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehabil 2012;9:21 [FREE Full text] [CrossRef] [Medline]
  14. Stein J, Bishop L, Stein DJ, Wong CK. Gait training with a robotic leg brace after stroke: a randomized controlled pilot study. Am J Phys Med Rehabil 2014 Nov;93(11):987-994. [CrossRef] [Medline]
  15. Zheng H, Black ND, Harris ND. Position-sensing technologies for movement analysis in stroke rehabilitation. Med Biol Eng Comput 2005 Jul;43(4):413-420. [Medline]
  16. DeVaul R, Sung M, Gips J, Pentland A. MIThril 2003: applications and architecture. In: ISWC '03. 2003 Presented at: 7th IEEE International Symposium on Wearable Computers; Oct. 21-23, 2003; White Plains, NY.
  17. Asada HH, Shaltis P, Reisner A, Rhee S, Hutchinson RC. Mobile monitoring with wearable photoplethysmographic biosensors. IEEE Eng Med Biol Mag 2003;22(3):28-40. [Medline]
  18. Shaltis PA, Reisner A, Asada HH. Wearable, cuff-less PPG-based blood pressure monitor with novel height sensor. Conf Proc IEEE Eng Med Biol Soc 2006;1:908-911. [CrossRef] [Medline]
  19. Kwakkel G, Kollen BJ, Krebs HI. Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil Neural Repair 2008;22(2):111-121 [FREE Full text] [CrossRef] [Medline]
  20. Laver K, George S, Thomas S, Deutsch JE, Crotty M. Virtual reality for stroke rehabilitation: an abridged version of a Cochrane review. Eur J Phys Rehabil Med 2015 Aug;51(4):497-506 [FREE Full text] [Medline]
  21. Howlett OA, Lannin NA, Ada L, McKinstry C. Functional electrical stimulation improves activity after stroke: a systematic review with meta-analysis. Arch Phys Med Rehabil 2015 May;96(5):934-943. [CrossRef] [Medline]
  22. Teasell RW, Bhogal SK, Foley NC, Speechley MR. Gait retraining post stroke. Top Stroke Rehabil 2003;10(2):34-65. [CrossRef] [Medline]
  23. Salter K, Jutai JW, Teasell R, Foley NC, Bitensky J. Issues for selection of outcome measures in stroke rehabilitation: ICF Body Functions. Disabil Rehabil 2005 Feb 18;27(4):191-207. [CrossRef] [Medline]
  24. Robbins SM, Houghton PE, Woodbury MG, Brown JL. The therapeutic effect of functional and transcutaneous electric stimulation on improving gait speed in stroke patients: a meta-analysis. Arch Phys Med Rehabil 2006 Jun;87(6):853-859. [CrossRef] [Medline]
  25. World Health Organization. International classification of functioning, disability and health. 2001.   URL: http://www.who.int/classifications/icf/en/ [accessed 2016-09-13] [WebCite Cache]
  26. Geyh S, Cieza A, Schouten J, Dickson H, Frommelt P, Omar Z, et al. ICF Core Sets for stroke. J Rehabil Med 2004 Jul(44 Suppl):135-141 [FREE Full text] [CrossRef] [Medline]
  27. Gottlieb A, Golander H, Bar-Tal Y, Gottlieb D. The influence of social support and perceived control on handicap and quality of life after stroke. Aging (Milano) 2001 Feb;13(1):11-15. [Medline]
  28. Intercollegiate Stroke Working Party. National clinical guideline for stroke. 4th edition edn: Royal College of Physicians; 2012.
  29. Moher D, Liberati A, Tezlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med 2009;9:151-264. [CrossRef]
  30. Higgins J, Altman D, Sterne J. Cochrane Handbook for Systematic Reviews of Interventions, Version 510.: The Cochrane Collaboration; 2011. Assessing risk of bias in included studies   URL: http://handbook.cochrane.org/ [accessed 2016-07-26] [WebCite Cache]
  31. Tanaka N, Saitou H, Takao T, Iizuka N, Okuno J, Yano H, et al. Effects of gait rehabilitation with a footpad-type locomotion interface in patients with chronic post-stroke hemiparesis: a pilot study. Clin Rehabil 2012 Aug;26(8):686-695. [CrossRef] [Medline]
  32. Tyson S, Connell L. The psychometric properties and clinical utility of measures of walking and mobility in neurological conditions: a systematic review. Clin Rehabil 2009 Nov;23(11):1018-1033. [CrossRef] [Medline]
  33. Altaim T. PhD Thesis.: University of Sheffield; 2015. Developing a functional outcome measure for individuals with low back pain within a Jordanian physiotherapy service   URL: http://etheses.whiterose.ac.uk/9602/ [accessed 2016-09-19] [WebCite Cache]
  34. de Vet HCW, Terwee CB, Bouter LM. Current challenges in clinimetrics. J Clin Epidemiol 2003 Dec;56(12):1137-1141. [Medline]
  35. Feinstein AR. An additional basic science for clinical medicine: IV. The development of clinimetrics. Ann Intern Med 1983 Dec;99(6):843-848. [Medline]
  36. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis 1985;38(1):27-36. [Medline]
  37. Okasheh ROA. Clinical measurement of functional outcomes of pulmonary rehabilitation. In: PhD Thesis. United Kingdom: Sheffield Hallam University-British Library; 2011:1.
  38. Mawson SJ. Measuring Physiotherapy Outcome in Stroke Rehabilitation. Physiotherapy 1993 Nov;79(11):762-765. [CrossRef]
  39. Michels E. Measurement in physical therapy. On the rules for assigning numerals to observations. Phys Ther 1983 Feb;63(2):209-215 [FREE Full text] [Medline]
  40. Stevens SS. On the theory of scales of measurement. Science 1946 Jun 7;103(2684):677-680. [Medline]
  41. Boysen G, Krarup L, Zeng X, Oskedra A, Kõrv J, Andersen G, ExStroke Pilot Trial Group. ExStroke Pilot Trial of the effect of repeated instructions to improve physical activity after ischaemic stroke: a multinational randomised controlled clinical trial. BMJ 2009;339:b2810 [FREE Full text] [Medline]
  42. Dragin AS, Konstantinović LM, Veg A, Schwirtlich LB. Gait training of poststroke patients assisted by the Walkaround (body postural support). Int J Rehabil Res 2014 Mar;37(1):22-28. [CrossRef] [Medline]
  43. Alptekin N, Gok H, Geler-Kulcu D, Dincer G. Efficacy of treatment with a kinaesthetic ability training device on balance and mobility after stroke: a randomized controlled study. Clin Rehabil 2008;22(10-11):922-930. [CrossRef] [Medline]
  44. Peurala SH, Tarkka IM, Pitkänen K, Sivenius J. The effectiveness of body weight-supported gait training and floor walking in patients with chronic stroke. Arch Phys Med Rehabil 2005 Aug;86(8):1557-1564. [CrossRef] [Medline]
  45. Peurala SH, Airaksinen O, Huuskonen P, Jäkälä P, Juhakoski M, Sandell K, et al. Effects of intensive therapy using gait trainer or floor walking exercises early after stroke. J Rehabil Med 2009 Feb;41(3):166-173 [FREE Full text] [CrossRef] [Medline]
  46. Ng MFW, Tong RKY, Li LSW. A pilot study of randomized clinical controlled trial of gait training in subacute stroke patients with partial body-weight support electromechanical gait trainer and functional electrical stimulation: six-month follow-up. Stroke 2008 Jan;39(1):154-160 [FREE Full text] [CrossRef] [Medline]
  47. Tong RK, Ng MF, Li LS. Effectiveness of gait training using an electromechanical gait trainer, with and without functional electric stimulation, in subacute stroke: a randomized controlled trial. Arch Phys Med Rehabil 2006 Oct;87(10):1298-1304. [CrossRef] [Medline]
  48. Lau RWK, Yip SP, Pang MYC. Whole-body vibration has no effect on neuromotor function and falls in chronic stroke. Med Sci Sports Exerc 2012 Aug;44(8):1409-1418. [CrossRef] [Medline]
  49. Hidler J, Nichols D, Pelliccio M, Brady K, Campbell DD, Kahn JH, et al. Multicenter randomized clinical trial evaluating the effectiveness of the Lokomat in subacute stroke. Neurorehabil Neural Repair 2009 Jan;23(1):5-13. [CrossRef] [Medline]
  50. Lewek MD, Cruz TH, Moore JL, Roth HR, Dhaher YY, Hornby TG. Allowing intralimb kinematic variability during locomotor training poststroke improves kinematic consistency: a subgroup analysis from a randomized clinical trial. Phys Ther 2009 Aug;89(8):829-839 [FREE Full text] [CrossRef] [Medline]
  51. Schwartz I, Sajin A, Fisher I, Neeb M, Shochina M, Katz-Leurer M, et al. The effectiveness of locomotor therapy using robotic-assisted gait training in subacute stroke patients: a randomized controlled trial. PM R 2009 Jun;1(6):516-523. [CrossRef] [Medline]
  52. Ochi M, Wada F, Saeki S, Hachisuka K. Gait training in subacute non-ambulatory stroke patients using a full weight-bearing gait-assistance robot: A prospective, randomized, open, blinded-endpoint trial. J Neurol Sci 2015;353(1-2):130-136. [CrossRef] [Medline]
  53. van Nunen MPM, Gerrits KHL, Konijnenbelt M, Janssen TWJ, de Haan A. Recovery of walking ability using a robotic device in subacute stroke patients: a randomized controlled study. Disabil Rehabil Assist Technol 2015 Mar;10(2):141-148. [CrossRef] [Medline]
  54. Peurala SH, Titianova EB, Mateev P, Pitkänen K, Sivenius J, Tarkka IM. Gait characteristics after gait-oriented rehabilitation in chronic stroke. Restor Neurol Neurosci 2005;23(2):57-65. [Medline]
  55. Kunkel D, Pickering RM, Burnett M, Littlewood J, Burridge JH, Ashburn A, Stroke Association Rehabilitation Research Centre. Functional electrical stimulation with exercises for standing balance and weight transfer in acute stroke patients: a feasibility randomized controlled trial. Neuromodulation 2013;16(2):168-177. [CrossRef] [Medline]
  56. Kakuda W, Abo M, Nakayama Y, Kiyama A, Yoshida H. High-frequency rTMS using a double cone coil for gait disturbance. Acta Neurol Scand 2013 Aug;128(2):100-106. [CrossRef] [Medline]
  57. Paoloni M, Mangone M, Scettri P, Procaccianti R, Cometa A, Santilli V. Segmental muscle vibration improves walking in chronic stroke patients with foot drop: a randomized controlled trial. Neurorehabil Neural Repair 2010;24(3):254-262. [CrossRef] [Medline]
  58. Ng SM, Hui-Chan CWY. Transcutaneous electrical nerve stimulation combined with task-related training improves lower limb functions in subjects with chronic stroke. Stroke 2007 Nov;38(11):2953-2959 [FREE Full text] [CrossRef] [Medline]
  59. Jung K, Kim Y, Cha Y, In T, Hur Y, Chung Y. Effects of gait training with a cane and an augmented pressure sensor for enhancement of weight bearing over the affected lower limb in patients with stroke: a randomized controlled pilot study. Clin Rehabil 2015 Feb;29(2):135-142. [CrossRef] [Medline]
  60. Bauer P, Krewer C, Golaszewski S, Koenig E, Müller F. Functional electrical stimulation-assisted active cycling--therapeutic effects in patients with hemiparesis from 7 days to 6 months after stroke: a randomized controlled pilot study. Arch Phys Med Rehabil 2015 Feb;96(2):188-196. [CrossRef] [Medline]
  61. Bradley L, Hart BB, Mandana S, Flowers K, Riches M, Sanderson P. Electromyographic biofeedback for gait training after stroke. Clin Rehabil 1998 Feb;12(1):11-22. [Medline]
  62. Dorsch AK, Thomas S, Xu X, Kaiser W, Dobkin BH. SIRRACT: An International Randomized Clinical Trial of Activity Feedback During Inpatient Stroke Rehabilitation Enabled by Wireless Sensing. Neurorehabil Neural Repair 2015 Jun;29(5):407-415. [CrossRef] [Medline]
  63. Intiso D, Santilli V, Grasso MG, Rossi R, Caruso I. Rehabilitation of walking with electromyographic biofeedback in foot-drop after stroke. Stroke 1994 Jun;25(6):1189-1192. [Medline]
  64. Salisbury L, Shiels J, Todd I, Dennis M. A feasibility study to investigate the clinical application of functional electrical stimulation (FES), for dropped foot, during the sub-acute phase of stroke - A randomized controlled trial. Physiother Theory Pract 2013 Jan;29(1):31-40. [CrossRef] [Medline]
  65. Ng SM, Hui-Chan CWY. Does the use of TENS increase the effectiveness of exercise for improving walking after stroke? A randomized controlled clinical trial. Clin Rehabil 2009 Dec;23(12):1093-1103. [CrossRef] [Medline]
  66. Watanabe H, Tanaka N, Inuta T, Saitou H, Yanagi H. Locomotion improvement using a hybrid assistive limb in recovery phase stroke patients: a randomized controlled pilot study. Arch Phys Med Rehabil 2014 Nov;95(11):2006-2012. [CrossRef] [Medline]
  67. Mansfield A, Wong JS, Bryce J, Brunton K, Inness EL, Knorr S, et al. Use of Accelerometer-Based Feedback of Walking Activity for Appraising Progress With Walking-Related Goals in Inpatient Stroke Rehabilitation: A Randomized Controlled Trial. Neurorehabil Neural Repair 2015 Oct;29(9):847-857. [CrossRef] [Medline]
  68. Mirelman A, Bonato P, Deutsch JE. Effects of training with a robot-virtual reality system compared with a robot alone on the gait of individuals after stroke. Stroke 2009 Jan;40(1):169-174 [FREE Full text] [CrossRef] [Medline]
  69. Solopova IA, Tihonova DY, Grishin AA, Ivanenko YP. Assisted leg displacements and progressive loading by a tilt table combined with FES promote gait recovery in acute stroke. NeuroRehabilitation 2011;29(1):67-77. [CrossRef] [Medline]
  70. Carr J, Shepherd R. Neurological Rehabilitation: Optimizing motor performance. Oxford, UK: Butterworth-Heinemann; 1998.
  71. Podsiadlo D, Richardson S. The timed “Up & Go”: a test of basic functional mobility for frail elderly persons. J Am Geriatr Soc 1991 Feb;39(2):142-148. [Medline]
  72. Mirelman A, Patritti BL, Bonato P, Deutsch JE. Effects of virtual reality training on gait biomechanics of individuals post-stroke. Gait Posture 2010 Apr;31(4):433-437. [CrossRef] [Medline]
  73. Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med 2012 Dec;75(12):2299-2306. [CrossRef] [Medline]
  74. Greenhalgh T, Howick J, Maskrey N, Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis? BMJ 2014;348:g3725 [FREE Full text] [Medline]
  75. Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.


10MWT: 10 minute walk test
6MWT: Six Minute Walk Test
BBS: Berg Balance Scale
BI: Barthel Index
EFAP: Emory Functional Ambulation Profile
EMG-BFB: Electromyographic biofeedback
FAC: Functional Ambulatory Category
FES: Functional electrical stimulation
ICF: International Classification of Functioning, Disability and Health
L/R: Left/Right hemisphere stroke
M/F: Male/Female
MCID: Minimally clinically important difference
NHS: National Health Service
MI: Motricity Index
Nottingham ADL: Nottingham Activities of Daily Living Index
POMA: Performance-Oriented Mobility Assessment
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
RCT: Randomised controlled trial
RMI: Rivermead mobility index
SEQ: Stroke Self-Efficacy Questionnaire
SIS: Stroke Impact Scale
SPPB: Short Physical Performance Battery
TENS: Transcutaneous Electrical Nerve Stimulation
TUG: Timed Up and Go Test


Edited by G Eysenbach; submitted 20.04.16; peer-reviewed by B Dobkin, A Tognetti; comments to author 25.07.16; revised version received 02.08.16; accepted 21.08.16; published 07.10.16

Copyright

©Lauren Powell, Jack Parker, Marrissa Martyn St-James, Susan Mawson. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 07.10.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.