Background:

Motor disability following stroke is common and results in difficulty performing everyday tasks such as dressing and meal preparation. Constraint-induced movement therapy (CIMT) is unique among motor rehabilitation paradigms because it employs intensive motor practice to build strength and function and also incorporates behavioral techniques to maximize use of the more-impaired arm during daily activities (self-monitoring of arm use, goal-setting, and guided problem-solving to overcome barriers to not using a hemiparetic arm). Randomized controlled trials (RCTs) consistently show that CIMT is superior at producing improvements in daily arm use, yet it has only minimally penetrated clinical practice because of cost and its demanding treatment schedule. With the advent of new camera sensors, it became possible to program video games for rehabilitation that use a person's own body movements to drive game play. Rehabilitation gaming provides a solution for making the intensive training and behavioral techniques of CIMT more accessible to underserved individuals. A previous pilot study established the feasibility of this treatment approach, but a definitive trial of the comparative effectiveness of this novel model of CIMT delivery is needed to guide clinical decision-making.

Objectives:

This multisite RCT aimed to (1) establish the comparative effectiveness of 2 different implementations of an in-home video game delivery model of CIMT vs traditional clinic-based CIMT and vs standard care; and (2) examine individual factors that may differentially influence response to 1 treatment vs another.

Methods:

In total, 193 individuals with chronic stroke were randomly assigned to 1 of 4 interventions that occurred over a 3-week period: (1) traditional CIMT (35 therapist/client contact hours over 10 sessions); (2) therapist-as-consultant Gaming CIMT (5 therapist/client contact hours over 4 sessions and 15 hours of independent game play at home); (3) therapist-as-consultant Gaming CIMT with additional teleconsultation (5 therapist/client contact hours over 4 sessions, 6 additional video conference “check-ins,” and 15 hours of independent game play in the home); and (4) standard occupational therapy (5 hours over 4 sessions plus a home exercise program). The gaming system logged active play time to monitor adherence to the prescribed unsupervised in-home game play. Motor function/speed (Wolf Motor Function Test [WMFT]) and self-reported daily use of the more-affected hand (Motor Activity Log [MAL]) served as the primary outcome measures. Participants were assessed immediately before treatment, after 3 weeks of treatment, and 6 months later. Between-group differences in treatment response/maintenance were assessed through intent-to-treat analyses using linear mixed models. Initial motor ability, cognition, tactile sense, use of the more-affected arm for daily activities (outside the clinic), treatment adherence, age, sex, and chronicity were examined as potential moderators of motor outcome.

Results:

Of 193 participants who consented to the study, 168 started treatment, 150 completed treatment, and 113 completed 6-month follow-up assessment. All 3 treatments incorporating a form of CIMT (both game-based CIMT groups and traditional in-clinic CIMT) produced clinically meaningful improvements in use of the more-affected arm for daily activities (MAL), whereas standard care did not. Between-group differences (95% CIs) relative to standard care were 0.8 points (0.5-1.0), 1.0 points (0.8-1.3), and 1.2 points (1.0-1.5) for the gaming, gaming with teleconsultation, and in-clinic CIMT groups, respectively. Retention of MAL gains at 6 months posttreatment was 57%. Regarding gains in motor function/speed (WMFT), all groups attained clinically meaningful improvements that were maintained 6 months later (median proportional improvements of about 20%). There were also marginally significant (P = .05-.08) comparative treatment effects immediately posttreatment whereby in-clinic CIMT produced greater improvements in natural log-transformed WMFT scores during the treatment period than Gaming and Standard Care; between-group differences were −0.18 (−0.37 to 0.00) and −0.17 (−0.36 to 0.02), respectively, which corresponds to a performance time decrease of 0.85 seconds per task for in-clinic CIMT relative to both Gaming and Standard Care. These marginally significant comparative treatment effects can likely be attributed to greater dosing of motor practice in the in-clinic CIMT group. Comparative treatment effects on the WMFT were absent in follow-up. Poorer motor ability at baseline was associated with significantly greater improvements in motor function and poorer improvements in daily arm use.

Conclusions:

Superior gains in daily arm use were achieved from all CIMT and gaming CIMT interventions in which the therapist delivered behavioral interventions compared with standard care (in which therapy time focused primarily on motor practice). Clinically meaningful long-term gains in motor function were roughly equivalent between groups, suggesting that survivors of chronic stroke can continue to make meaningful improvements in motor function regardless of how they access motor practice (ie, whether through virtual reality, CIMT, or standard care). Video games can thus provide a vehicle for therapists to monitor and encourage in-home intensive motor rehabilitation while freeing up therapist time to conduct the critical behavioral elements of CIMT. This therapist-supported self-management model maximizes both motor function and daily arm use within the typical constraints of managed care.

Limitations:

Adherence to in-home gaming rehabilitation was imperfect, particularly for the gaming group that received fewer therapist consultations. Thus, a dose-matched comparison of the efficacy of game-based vs in-clinic CIMT for promoting motor restoration could not be achieved from this pragmatic study design. Follow-up data should be cautiously interpreted given substantial and unequal attrition between groups in follow-up.

Stakeholders Involved

Stroke Advisory Board

Our Stroke Advisory Board consists of patients, their families, and community therapists. The board was formed during the initial PCORI pilot project and has continued to provide planning and input throughout the current trial. Some board meetings were held with all groups present, whereas other, smaller meetings were held to accomplish specific tasks that require the input of either survivors/families (eg, recruitment planning) or therapists (eg, planning the standard care treatment protocol). Table 1 shows the breakdown of the board and provides examples of functions served by each group of stakeholders.

Table 1

Functions Served by Stroke Advisory Board Members.

Formulating Research Questions and Study Design

A half-day conference with the patient and family stakeholders on the Stroke Advisory Board was held for the purpose of planning this work before the initial grant submission. First, board members informed relevant comparisons, yielding the 4-group design proposed. As an example of the contribution that the board made to the initial project design, the game was conceptualized to be a stand-alone therapy application that individuals who did not have ready access to a clinic could use independently in their homes. Many board members expressed the view, however, that attending therapy was beneficial and important to stroke survivors. Therapist stakeholders also expressed concern that clients would be missing an opportunity to problem-solve through specific routines of importance to them (eg, how to tie their own shoes or eat a specialty food) in a stand-alone implementation of the game. We thus modified the design of our trial to be in keeping with these patient preferences.

A few members of our Stroke Advisory Board (particularly those residing in rural areas) indicated that travel would be prohibitive for some people and favored a rehabilitation solution that would not require clients to travel. We thus minimized travel by incorporating teleconsultation visits for gaming+ (ie, therapist-as-consultant gaming CIMT with additional teleconsultation). The board also informed the schedule of the treatments, advising that the treatment be delivered over a 2- to 4-week period instead of over a strict 2-week period, as initially proposed. This change allowed participants and families to more easily arrange participation around their other commitments and provided more flexibility when relying on transportation, as could occur in standard care settings.

At this initial conference, the board also informed the covariates and dependent variables for the study. They selected 2 primary outcome measures that were most meaningful to them. They also identified which personal characteristics they felt were relevant for examining heterogeneity of treatment effect.

The board retained final say over the inclusion criteria, which were collaboratively discussed with the academic members of the team. There was 1 exclusion criterion on which board members and academic team members initially disagreed: the ability to include participants who were receiving onabotulinumtoxinA (hereafter referred to as Botox) in their weaker upper extremity. Although the board would have liked to include participants who were taking Botox to manage hypertonicity (difficulty opening the hand because of overactivity of the finger flexors), they ultimately accepted the scientists’ concerns that the fluctuating effects of Botox on hypertonicity (and hence motor function) would introduce additional error into the analyses (ie, that motor function would increase or decrease depending on where a person was in their Botox cycle, making it difficult to measure the true treatment effect). We did, however, agree that study participation could be timed to Botox administrations so that potential participants would have the option of delaying a dose by 3 weeks in order to participate.

Finally, the board suggested the following strategies to increase retention and adherence to the therapy protocol: (1) feedback regarding adherence during therapist encounters; (2) reminder phone calls when necessary and feasible (to attend in-person therapy, complete scheduled at-home game play, and perform home exercises); (3) providing an instructional DVD/video links for the client and family (eg, to educate/inform family members about CIMT); and (4) being provided with a T-shirt at study completion if >90% adherence was achieved. Some members of the board provided edits to the grant proposal following this initial meeting.

A second half-day meeting of the board was convened to finalize the study protocol before beginning the study. During this meeting, the board suggested minor changes to 1 of the Gaming groups. Initially, the therapist consultation sessions for 1 Gaming group were to occur entirely through teleconsultation, while the other Gaming group was to experience the same number/length of in-person therapist consultation (4 therapist contact sessions). Members of the board indicated that the in-clinic CIMT group would have more face-to-face contacts (4 vs 10), however, which they felt could create a difference in the sense of accountability they felt to adhere to the treatment and to the number of opportunities they had to receive feedback and encouragement from the therapist. Accordingly, before the start of the study, we modified the protocol to test this hypothesis by having both Gaming groups receive 4 in-person visits, while 1 Gaming group would receive 6 additional teleconsultations to match the number of therapist encounters with that of the CIMT group. During both conferences with the board, all participants contributed at least 1 idea to the discussion and voted on all ideas.

In determining the protocol for the Standard Care comparison group, several nonacademic therapist stakeholders were consulted 1 on 1 regarding the therapeutic protocol that would most closely resemble the care they would provide in the community within the constraints of limited face-to-face contact time.

Participating in and Monitoring the Conduct of the Project

The project employed co–principal investigator (PI) Ms Strahl as a core member of the research team. In this role, she participated in frequent teleconferences with the PI and worked alongside the academic team to design and implement the detailed study protocol. She established the PMMC study team, recruited and screened participants for the PMMC site, and revised the study treatment forms to be more patient centered. To prepare for this role, she participated in an online Collaborative Institutional Training Initiative (CITI) program and NIH-sponsored clinical trials management course.

The board participated throughout the course of the project, most extensively in advising recruitment strategy. The members affirmed that the recruitment strategy with the greatest likelihood of success was the one currently being employed but, given underrecruitment during the latter half of the project, suggested additional recruitment strategies that the team could try. Unfortunately, these additional strategies (placing bookmarks in stroke-related library books, placing flyers in senior centers or at senior day centers, National Public Radio advertisement) were not widely successful, yielding only a few additional participants.

A final meeting of the board was convened to discuss and interpret the study findings. The academic PIs presented the findings via Microsoft PowerPoint slides but left the interpretation open to board input. Board members offered some likely explanations that meshed with the researchers’ interpretations but also offered additional interpretation of study findings that the researchers had not initially considered. For example, board members noted that if gaming participants retained access to the game, they could possibly continue to improve motor function during follow-up; hence, the follow-up results were perhaps an underestimation of what could occur.

Planning Dissemination of the Study Results

The proposed dissemination plan was further refined during quarterly meetings with the Stroke Advisory Board. Board members participated in generating additional ideas for methods of dissemination. Initial ideas established collaboratively between the board and the academic team included the following: presenting at local support groups, harnessing social media, generating interest from TV and radio media outlets, and drafting a lay publication to be submitted to StrokeSmart, the publication of the National Stroke Association. To ensure that results are communicated most effectively to the community, members of the Stroke Advisory Board and the co-PI Ms Strahl will draft a lay explanation of study findings to share with former participants. They will continue to direct dissemination activities and will participate in drafting the research publication. Select members of the board have also expressed their enthusiasm for working with the media to share their experiences (and some have already done so). For example, Ms Strahl was co-featured with Lynn Gauthier, PhD, on a May 25, 2020, Choiceology podcast (https://www.schwab.com/resource-center/insights/content/choiceology-season-5-episode-6) episode. Other members of the board have volunteered to share on social media (eg, stroke support Facebook groups, Smart Patients) because their posts would be more trusted by the stroke community than those posted by the academic team.

Therapist stakeholders have indicated their willingness to make the therapist community aware of this new intervention at continuing education conferences and through routine clinical interaction. As an example, some therapist stakeholders co-presented with Dr Gauthier a half-day preconference symposium at the American Congress of Rehabilitation Medicine offering practical instruction in the behavioral techniques of CIMT that promote increased arm use for everyday activities. We additionally presented a similar talk on how to implement the therapist-as-consultant treatment model at the American Occupational Therapy Association conference in March 2020 (presented as a virtual webinar because of COVID-19).

Study Overview

Study Design

We created an RCT with 4 parallel groups: (1) in-clinic CIMT, (2) therapist-as-consultant gaming CIMT (Gaming), (3) therapist-as-consultant gaming CIMT with additional teleconsultation (Gaming+), and (4) Standard Care (Table 2). Each treatment was scheduled to occur over a 3-week period.

Table 2

Description of Study Treatments.

Participants

Community-dwelling stroke survivors were recruited from February 2016 through May 2019 (the end of the recruitment phase of the PCORI contract). At the 3 academic medical centers, prospective participants were identified primarily through mining electronic medical records for ICD-9 and ICD-10 codes for stroke and hemiparesis. Potential participants then received a letter informing them of the study and providing them with study-related contact information. Potential participants at the 2 outpatient clinic sites were primarily identified through referrals from therapists and physicians. To lessen the burden on potential participants, study staff offered a brief phone screening. Participants who appeared to meet eligibility criteria based on the phone screening were then invited to participate in an in-person screening session to ensure that they met the full inclusion criteria before being enrolled in the study.

Interventions

The main objective of the research was to examine the comparative effectiveness of a new self-management approach to delivering CIMT using an in-home video game^37,38 between consultation sessions with a therapist. The following are the most informative between-group comparisons:

• Gaming CIMT vs gold-standard in-clinic CIMT. This comparison examines whether a self-management in-home approach to motor skills practice can produce equivalent motor outcomes to an intensive in-clinic treatment. The total time of prescribed active motor practice is the same between the CIMT and Gaming groups, and both employ the same behavioral techniques to promote everyday use of the weaker arm.
• Gaming CIMT vs the standard approach to treatment. This comparison examines whether providing at-home access to a video game for self-managed motor practice and focusing therapists’ interactions on behavioral intervention can enhance motor outcomes compared with standard care. The number and duration of in-clinic visits with a therapist is the same between these 2 interventions, but the use of a game at home for motor practice “frees up” therapist time to focus on behavioral strategies to promote greater arm use, an aspect of treatment that is typically absent from standard care. Thus, the face-to-face time with a therapist is spent mostly on behavioral strategies in the Gaming groups and on motor practice in the Standard Care group.

Treatments are described in more detail below and depicted in Table 2 above. All treatments were scheduled over a 3-week period, which could be extended to 4 weeks in the case of illness or other unanticipated conflicts or events.

Gaming CIMT

Given that many stroke survivors have insufficient access to rehabilitation, the intervention for this group was designed to empower stroke survivors to manage their own care. A common challenge reported by stroke survivors and therapists alike is difficulty adhering to self-managed home practice programs. Computerized gaming interventions are preferred by stakeholders and have established efficacy²⁹; therefore, a custom kayak adventure–themed rehabilitation game, Recovery Rapids,^31,37,38 provided a more palatable vehicle for intensive practice at home. Recovery Rapids uses the Microsoft Kinect sensor to capture 9 proximal and distal movements made by participants that drive navigation of an avatar through various obstacles. Each movement is mapped to an intuitive game action (eg, right shoulder abduction with elbow extension to navigate the boat to the right, grasping and releasing to collect objects). As with CIMT, Recovery Rapids employs a performance-based algorithm to automatically increase the required difficulty as a person improves. For example, the algorithm will increase the required range of motion at the shoulder and elbow during shoulder abduction until the player can exercise sufficient range of motion to trigger game actions on only about 80% of attempts. In keeping with CIMT's constraint of the stronger arm, only movements made with the more-affected upper extremity can trigger game actions. Figure 1 is a screenshot of the game play. Table 3 describes how the game design was in keeping with several principles of motor learning. Additional detail on game design and screenshots of the game can be found in prior publications.^31,37,38

Figure 1

Screenshot of Recovery Rapids.

Table 3

Principles of Motor Learning Underlying the Development of Recovery Rapids.

Participants agreed to play Recovery Rapids for a total of 15 hours (1.5 hours per day) on 10 treatment days over 3 weeks. This treatment schedule was designed to dose-match the duration of active motor practice provided in CIMT (the same duration of active motor practice could be accomplished in half the time because gaming treatment involves continuous practice without breaks for task setup, and automated feedback occurs concurrently with game play). Participants could self-pace their game play (ie, hit “Pause” for short rests or begin another session later if longer breaks were needed). Participants were encouraged to break play into 3 separate sessions per day to avoid fatigue, but ultimately they had the freedom to play for as long as they wished. Participants had complete freedom when scheduling their gameplay but were asked to make up any missed play time on a nontreatment day to encourage adherence to the prescribed motor practice. Game play was driven solely by movements made with the 29 more-affected upper extremity. The gaming system logged compliance with in-home game play (ie, active play time).

A total of 5 hours of in-clinic therapist consultation was delivered over 4 treatment sessions (initial session = 2 hours). This treatment schedule was chosen because the frequency of in-clinic visits (1 to 2 times per week) reflects that of routine clinical practice. Therapist sessions focused on teaching game play (about 30 minutes in session 1) and on delivering the same CIMT behavioral techniques described above under “In-Clinic CIMT (High-Intensity Comparator)” and elsewhere.^15,39 In lieu of the CIMT restraint mitt (a component of CIMT that is not well liked by participants), participants were provided with a smartwatch application that tracked use of the weaker arm and notified the participant (vibration with message to “Please use me”) when it detected prolonged inactivity. In the month following treatment, continued use of the weaker arm during daily activities was encouraged by prompting the participant to complete the MAL via a Research Electronic Capture database (REDCap) survey, an exercise that required participants to periodically reflect on daily arm use.

Primary Outcomes

Qualitative analysis of feedback from our Stroke Advisory Board indicated 2 main therapy objectives: (1) regaining sufficient motor control to accomplish daily tasks/hobbies independently and (2) decreasing the time and effort required to perform tasks. To address the former participation outcome, quality of arm use for daily activities was measured via the MAL, which is a reliable and valid measure of quality of arm use during daily activities.^57,58 Arm use for each of 28 ADLs was self-rated on an 11-point scale, from 0 (no use) to 5 (normal ability) assessed at 0.5-point intervals. The minimally clinically important difference (MCID) for the MAL is about 1.0,⁵⁹ which qualitatively reflects the difference between requiring help from the stronger hand to accomplish a task and being able to accomplish a task with effort using just the weaker hand.

To address stakeholders’ priority for increased speed of movement, we used the Wolf Motor Function Test (WMFT), which measures the time to complete standardized functional movements (eg, turning over cards, lifting a can to one's mouth, folding a towel).^60-62 The WMFT has established reliability and validity^60-62 and has been commonly employed in previous CIMT trials.^8,10,22,63 The MCID on the WMFT is a 16% decrease in performance time.⁶⁴ WMFT performance time scores were natural log (Ln) transformed before analysis, following precedent,⁶⁵ for 3 reasons. First, an Ln transform renders positively skewed WMFT performance time data approximately normally distributed. Second, improvements on an Ln scale better characterize clinical improvement. For example, an improvement in performance time from 12 seconds to 2 seconds is not clinically equivalent to an improvement from 112 to 102 seconds; the former would be viewed by stakeholders as appreciably more meaningful, and the Ln transform reflects that. Finally, Ln-transformed WMFT scores are more easily interpretable because they approximate percentage change.

The study group was masked to those conducting the testing. Assessments were conducted within a week before treatment, a week following treatment, and within 5 to 7 months after posttest (see Table 4). Assessments were video recorded to enable later review (eg, if a data entry error or test administration error was suspected).

Table 4

Timeline of Participation.

Secondary Outcomes

Secondary/exploratory outcomes for this trial were scores on measures whose psychometric properties are still being established (Quality of Life in Neurological Disorders [Neuro-QoL]) or whose sensitivity to clinical improvement may be limited (eg, 9-HPT).⁶⁶ Additional clinical measurements were obtained to determine how factors such as sensation (Semmes-Weinstein monofilament [SWM] touch test), cognition (Montreal Cognitive Assessment [MoCA]), or adherence (eg, duration of game play) could affect treatment response. Each measurement is described below in turn.

Sample Size Calculations and Power

A sample size was selected to provide 80% power to detect an MCID between 1 group and the other 3 groups on the WMFT. Power was estimated in MATLAB using a Monte Carlo approach⁷⁹ in which data were repeatedly randomly sampled from a simulated data set and the statistical significance calculated for each of 5000 permutations. The simulated data set employed within-group variability from a previous CIMT study⁸ and a between-group difference equal to the MCID for the WMFT (16% [Lin et al]⁶⁴ of the baseline WMFT mean performance time from this study Ln transformed⁸). A 2-tailed test with an α level of .05 indicated that 51 participants per group would yield 80% power. Accounting for an estimated 10% attrition (based on the pilot study³¹), our targeted enrollment was thus 224 participants. As the MCID for the MAL is proportionally larger than the MCID for the WMFT and the MAL showed larger comparative treatment effects in prior work,⁸ power for the MAL approached 100% at the target sample size of 224.

Time Frame for the Study

A 3-week intervention period for all treatments was chosen based on feedback from the Stroke Advisory Board that members would prefer a more compressed intervention to a diffuse one. They thus favored the traditional CIMT schedule but petitioned for it to be extended from 2 weeks to 3 weeks to allow more flexibility in scheduling transportation and other commitments. A follow-up of 6 months was the longest that could be feasibly accomplished given the study timeline.

Procedures to Maximize Study Adherence and Retention

To maximize retention, all therapy sessions and pretreatment, posttreatment, and 6-month follow-up testing appointments were scheduled at the time of enrollment. A financial incentive of $50 per occasion was provided to encourage attendance at testing sessions. A thank-you letter was sent to participants a month after study completion, with a reminder of their follow-up testing date. Members of the Stroke Advisory Board suggested the following adherence-enhancing measures that were employed uniformly for this trial: (1) feedback regarding adherence during therapist encounters; (2) reminder phone calls when necessary and feasible (to attend in-person therapy, complete scheduled at-home game play, and perform home exercises); (3) providing an instructional DVD/video links for the client and family (eg, to educate/inform family members about CIMT); and (4) being provided with a T-shirt at study completion if >90% adherence was achieved.

Analytical and Statistical Approaches

Demographic measures and initial scores on outcome variables were compared between groups using analysis of variance (ANOVA) for continuous variables or χ² tests for categorical variables to ensure initial group equivalency. Therapy-induced changes in motor function (WMFT) and daily arm use (MAL) were analyzed separately via mixed-effects linear models in intent-to-treat (ITT) analyses (primary analyses). In these analyses, data from participants who voluntarily withdrew from the study (during or after treatment) and those with poor adherence to the protocol were included. Each model included treatment and time (as well as their interaction) as fixed effects, and study site and participant as random effects. In these models, the interaction of treatment and time was the primary effect of interest because it tests the difference in change over time between the 4 treatment groups.

The following participant factors were entered as covariates in the model in a stepwise manner to determine whether they influenced the treatment response (factor × time): baseline motor ability, baseline arm use, therapy adherence (number of hours of active motor practice), tactile sense (SWM touch test⁷²), cognition (MoCA),⁸⁰ age, sex, chronicity, and whether the dominant hand was more affected. The comparative treatment response (3-way factor × time × group interaction) was examined for any factors that influenced treatment outcome (ie, showed a significant factor × time effect). Baseline motor ability was shown to influence both primary outcomes in prior research,^33-35 so its potential influence was examined first (ie, by adding the interaction between baseline motor ability and time as a fixed effect to the original mixed-effects general linear model). Each of the aforementioned covariates was retained in the model only if it significantly influenced treatment response (ie, interacted with time, P < .05) or influenced comparative treatment response (ie, produced a P < .05 three-way interaction). Variables and 3-way interactions were removed from the model when the addition of other variables reduced the statistical significance of the variable/interaction to P > .10. Dummy coded contrasts were used for categorical variables. Of interest was the extent to which each factor influenced the outcome variable (ie, partial slopes); the interaction of the factor with time; and the 3-way interaction between factor, time, and group. To determine whether the observed relationships were, in fact, linear, we constructed added variable regression plots.⁸¹ These plots display the added variable against the residual space of the other covariates.

Outlier Detection

Extreme outliers can skew overall treatment estimates. The MAL was examined for extreme outliers because these could indicate 1 or more of the following: (1) failure of the tester to follow the testing manual (ie, paraphrased or incomplete instructions), (2) a participant misunderstanding the scale, (3) a participant's limited awareness of his or her deficits, or (4) a participant's poor memory of how the weaker arm was used during activities over the 2 days before MAL administration. Given that participants with significant cognitive impairments were included in the sample and that the MAL relies on accurate self-report, it was necessary to examine the data for outliers with z-scores >3 SDs from the mean. Time-series outliers were replaced following random forest imputation (RFI; described in more detail in the “Missing Data” section below). Outlier analysis was also conducted on the WMFT and SWM touch test data to detect potential data-entry errors. Outliers were to be adjusted only for these objective measures in the case of test administration error.

Missing Data

Missingness was examined via logistic regression to determine whether the pattern of missingness related to study group or measured participant factors.

Two different ITT analyses were then examined, 1 treating voluntary dropouts as nonresponders and the other using imputation of missing data. The most conservative analysis (outlined in the original study protocol³²) treated voluntary dropouts as nonresponders (pretreatment scores were carried over to subsequent time points). It thus determined the relative effectiveness of each treatment approach on the entire population that was treatment eligible. To avoid “penalizing” the treatment effect if participants were forced to withdraw for reasons unrelated to the study (eg, medical issues), it was determined a priori that missing data for these participants should be interpolated (thus, last observation carried forward [LOCF] was used to replace missing data for voluntary dropouts, while multiple imputation was used in the case of medical withdrawals). Imputing the data for medical withdrawals was more appropriate than treating them as nonresponders, because many of these participants would return to resume the treatment protocol at a later date (ie, when medically stable) within a clinical setting that does not place time constraints on treatment completion. Missing data for individuals who withdrew during follow-up were also imputed to avoid overestimating retention of clinical gains (LOCF would produce estimates of 100% retention of the treatment effect).

Multiple imputation was performed by multiple imputation by chained equations (MICE) in the R statistical package.⁸² MICE employed 800 iterations⁸³ of the random forest (RF) prediction algorithm to estimate the missing data from available data.⁸² RF is a machine-learning technique that discovers nonlinear and complex relations between different variables in a data set, thus providing a more-accurate and less-biased estimation of missing data compared with popular imputation techniques such as predictive mean matching or multiple regression,^84-87 which may be prone to overfitting or rely too heavily on particular participant features.^88,89 RF used group assignment, clinical assessment data (summary scores on the SWM touch test, MoCA, WMFT, and MAL, as well as individual item responses on the WMFT and MAL), and select demographic information (affected side, sex, handedness) to estimate the missing data. To verify that the imputed values were not dependent on RF's initial randomly generated parameters (they were not), RF was repeated 5 times, each time with randomly generated initial RF parameters. No appreciable change in the estimates was realized as the number of iterations was further increased above 800, suggesting that 800 repetitions was sufficient.

The first ITT analysis described previously estimates the treatment effects among the entire population of eligible candidates, conservatively assuming no treatment response for individuals who withdrew from the study before posttesting. This analysis is useful for illuminating the pragmatic effectiveness of the treatments because it accounts for the attrition that negatively affects outcomes. The Stroke Advisory Board felt that this analysis was less useful for individual decision-making, however, because stroke survivors who initiate treatment typically do not do so under the impression that they may soon withdraw from treatment. Thus, a priori knowledge of the likely benefit of a treatment approach for those who remain in the treatment program seems more relevant at the point of care. Accordingly, a second ITT analysis was carried out in which all missing data were interpolated using RFI. The latter analysis estimates the typical treatment response of those who remain in treatment. It is the analysis discussed most in-depth in the Results and Discussion sections given its identified relevance to stakeholders.

Finally, stakeholders felt that it was relevant to have information about the relative effectiveness of treatments when they were completed as prescribed (per protocol). This comparison is particularly relevant for this trial; 63% percent of Gaming and Gaming+ participants did not fully adhere to the prescribed in-home game play, whereas all participants adhered almost completely to the in-person treatments. This differential adherence could skew ITT estimates of effectiveness in favor of in-person therapies. For this analysis, missing follow-up data were imputed using RFI.

Changes to the Original Study Protocol

Modifications to the Gaming+ group (Gaming group with teleconsultation) were made before the start of the study based on stakeholder feedback.

For the ITT analysis assessing comparative effectiveness, a slight modification was made to use imputation via RFI rather than list-wise deletion in order to account for missing data after treatment among individuals who were unable to complete the study protocol because of medical or other nonstudy-related causes. This modification was made for 2 reasons: (1) because list-wise deletion methods have been shown to produce biased parameters and estimates, and (2) because imputation was proposed for the analysis of data that were missing at follow-up; employing this modification throughout the study maintained greater consistency. Two related but separate ITT analyses were also performed instead of the single analysis originally proposed. The first analysis (proposed in the published study protocol³²) used LOCF to address missing data for individuals who voluntarily withdrew before posttreatment testing (simulating no treatment response). This ITT analysis would thus slightly underestimate the treatment effects, because some participants completed the majority of treatment before withdrawing and thus likely received some benefit from treatment. Accordingly, stakeholders expressed that it would be more relevant to their treatment decision-making to know the comparative effectiveness of the interventions among those who remained engaged in treatment (because participants began treatment under the assumption that they would remain in treatment). An additional ITT analysis was therefore carried out to more closely approximate the desired comparison; data imputation was used to address all missing posttreatment data instead of the LOCF method. This additional ITT analysis is also responsive to the revised PCORI Methodology Standards; these standards discourage the use of the LOCF method to deal with missing data.

For the analysis of participant factors that may influence treatment response, the Brief Kinesthesia Test was dropped as a potential covariate because it was found to be highly confounded by motor ability (and thus not a true measure of proprioceptive ability, as originally intended). Baseline motor function was also analyzed as a continuous rather than a dichotomous (higher/lower) variable to preserve variability and improve power; thus, the baseline score on the WMFT was used as the covariate rather than the ability to place any pegs on the 9-HPT, as originally proposed.

Regulatory Oversight

Regulatory oversight was conducted by the IRBs of OSU, OhioHealth, UAB, and the University of Missouri. The PMMC IRB ceded review to OSU (central IRB). This trial was prospectively registered on ClinicalTrials.gov (NCT02631850: https://clinicaltrials.gov/ct2/show/NCT02631850?term=NCT02631850&draw=2&rank=1).

Enrollment and Retention

In total, 193 participants were enrolled in the study. There was higher-than-expected attrition of 13% before the start of treatment. An additional 10.7% of participants dropped out during treatment, and 24.7% dropped out during the follow-up phase. One CIMT participant was withdrawn from the study during treatment after review of her medical records revealed that she had never experienced a stroke (she had experienced a transient ischemic attack with no subsequent motor impairment; nonuse of her arm was related to an old orthopedic injury). During-treatment attrition was consistent with the 9% attrition typical of outpatient stroke trials that recruited participants from the community.⁹⁰ Higher-than-expected attrition in follow-up was observed for the groups that had received the least amount of therapist contact, while attrition for the other groups was slightly lower than in previous reports.⁹¹ The flow of participants through the study is outlined in Figure 2. A site-specific recruitment/retention breakdown is shown in Table 5.

Figure 2

CONSORT Flow Diagram.

Table 5

Enrollment and Retention, by Site.

Demographics of the Study Sample

The study sample was disproportionately male (64% male, P = .001), reflective of the 41% higher stroke prevalence among men in the general population.⁹² Disproportionally fewer men were randomly assigned to the Gaming groups than to the Standard Care or CIMT groups (χ² = 10.07, P = .02). Aside from the gender imbalance between groups, there were no other significant demographic differences between groups or study sites at baseline. Baseline characteristics of participants by study group are shown in Table 6. Baseline characteristics of participants by site are shown in Table 7.

Table 6

Baseline Characteristics of the ITT Sample.

Table 7

Baseline Participant Characteristics, by Study Site.

Pattern of Missingness

Primary outcomes data were captured from all participants who remained in the study. One UAB participant with transportation challenges received the MAL only at posttreatment because of an inability to travel for testing (the MAL was administered via teleconference). This same participant arrived late to the follow-up testing session; thus, only the WMFT and SWM touch test were administered (the participant could not be contacted afterward to complete the MAL and Neuro-QoL remotely). There were 6 participants with missing data on the touch test (secondary outcome) at posttreatment and follow-up. Five instances of missing touch test data were resulted from testing protocol deviation at 1 of the sites; the other instance of missing data at follow-up resulted from excessive hypotonicity, which precluded opening the participant's hand to administer the test. Neuro-QoL data (secondary outcome) were missing for 21 participants at pretreatment, 16 participants at posttreatment, and 33 participants at follow-up. The majority of missing Neuro-QoL data resulted from hospital firewall or other issues accessing the CAT, particularly at 2 of the study sites.

For the analysis of the comparative treatment effects during the treatment period (baseline to posttreatment), the pattern of missingness (ie, dropout) was not significantly related to study group or to any measured participant factors (motor scores at baseline on the WMFT and MAL, age, time in years since the stroke, tactile sensation [SWM touch test], cognitive status [MoCA], whether the dominant hand was affected, or rural location). Rather, the pattern of missingness during the treatment period was presumed to be related to unquantifiable participant factors, such as unrelated medical events or reduced ability to travel.

Attrition in follow-up was also unrelated to baseline characteristics, but receiving fewer therapist contacts during treatment was associated with significantly greater attrition during the follow-up period (P = .01). During this period, the dropout rate was more than twice as high for the Gaming CIMT and Standard Care groups that received just 4 therapist consultation sessions than for the CIMT and Gaming+ groups that received 10 therapist consultations (Table 8). Although the reasons for attrition during follow-up were often unknown (eg, participant did not return calls from study personnel), between-group differences in attrition may be the result of participants in the reduced contact groups feeling less connected to study personnel following the active treatment phase of the study, which may have reduced their overall long-term commitment to the study. As follow-up attrition appears to be related to group assignment, follow-up treatment change estimates and comparative treatment effects, particularly among the 2 groups with high attrition (Gaming and Standard Care), should be interpreted with caution. The UAB site had significantly higher overall attrition than the other study sites during the follow-up period (P < .01). Table 8 shows the characteristics of participants who dropped out and who were retained in the study.

Table 8

Characteristics of Participants Who Dropped Out During Treatment, Dropped Out During Follow-up, and Were Retained Throughout the Study (N = 167).

Outlier Adjustment

Three pretreatment data points on the MAL were flagged as outliers and imputed via RFI. Two had excessively high scores (z > 3) that were inconsistent with MAL scores obtained during screening. The third approached the threshold for outlier detection (z > 2), and video review revealed that instructions had been read incorrectly to the participant during pretreatment MAL administration. One outlier in follow-up was adjusted via RFI; this participant had very poor cognition (MoCA = 6), and his report of fair-good quality of arm use during follow-up differed from his clinical presentation. One participant with cognitive impairment was removed altogether from the MAL analysis (list-wise deletion in lieu of interpolating MAL data from all 3 time points) because of inconsistent patterns of responding (eg, different items rated as not attempted at pretreatment vs posttreatment vs follow-up, better quality ratings for some harder items than for easier items).⁵⁷ For the WMFT, 1 CIMT participant showed unusually strong recovery during the treatment period (z < −3), and 1 Standard Care participant showed exceptionally strong recovery during the 6-month follow-up period (z < −3); video review confirmed accurate test administration in both of these cases, so these outliers were not replaced with interpolated data. There were no outliers on the SWM touch test.

ITT Analysis With All Missing Data Interpolated

This analysis is the most relevant for patient decision-making because it measures the effectiveness of the interventions among participants who remain engaged in treatment but who may not adhere perfectly to the prescribed treatment. The LOCF analyses are presented in Appendix A. Per-protocol analyses are presented in Appendix B. Table 9 shows the means and SDs of each treatment group at each time point. Table 10 shows the comparative treatment effects.

Table 9

Means and SDs of Each Treatment Group at Each Time Point.

Table 10

Comparative Treatment Effects.

Arm Use for Daily Activities (MAL)

The mixed-effects general linear model revealed a significant group × time interaction (P < .0001), with the 3 groups that received the behavioral components of CIMT (ie, CIMT and both gaming groups) significantly outperforming the standard care group. The groups that received behavioral intervention (CIMT and both Gaming groups) showed significant (P < .0001) and clinically meaningful improvements in daily arm use (MAL treatment change >1.0 MCID) during the treatment period, while the Standard Care group showed a statistically significant improvement during treatment (P < .0001) that was not clinically meaningful (MAL treatment change < 1.0 MCID). The treatment response of all 3 groups that received the behavioral components was more than twice as large as that of Standard Care participants. The portion of participants who achieved a clinically meaningful treatment response on the MAL was 92%, 80%, 70%, and 24% for the CIMT, Gaming+, Gaming, and Standard Care groups, respectively. Participants exhibited, on average, 57% retention of MAL gains at 6-month follow-up; retention did not significantly differ between groups. Imperfect retention is not surprising in the context of 100% self-management of behavior change for the 5 to 6 months that preceded the follow- up assessment. The portion of participants who maintained a clinically meaningful treatment response on the MAL after 6 months was 56%, 38%, 32%, and 21% for the CIMT, Gaming+, Gaming, and Standard Care groups, respectively. Although some statistically significant differences between the groups remained at 6-month follow-up, clinically meaningful differences were no longer present between groups. Figure 3 illustrates the comparative treatment effects and their 6-month retention.

Figure 3

Treatment Change on the MAL (Change in Raw Score From Baseline) by Group During the Intervention Period (Blue, Left) and 6-Month Follow-up (Purple, Right).

Motor Function Outcomes (WMFT)

All groups attained statistically significant and clinically meaningful improvements in motor function ranging from 20% (Standard Care) to 35% (CIMT) of their maximum possible improvement [(posttreatment ™ pretreatment)/(best possible score ™ pretreatment)]. These improvements were maintained over the follow-up period (Figure 4). The CIMT group showed marginally greater initial improvements in motor function than the Gaming and Standard care groups (P = .05 and .08, respectively). No comparative treatment effects remained by follow-up.

Figure 4

Treatment Change on the WMFT (Ln Transformed) by Group During the Intervention Period (Blue Bars, Left) and at 6-Month Follow-up (Yellow Bars, Right).

Adherence to Treatment

Adherence to rehabilitation gaming practice varied markedly among individuals who completed treatment (ie, remained in the study). The Gaming group completed a median of 7 hours of motor practice (46% of the prescribed amount), while the Gaming+ group completed a median of 12 hours (81%) of the exercise dose (Figure 5). While 24% of Gaming participants and 37% of Gaming+ participants achieved or exceeded the prescribed 15 hours of gaming practice (ie, were fully adherent), nonadherence led an approximately equal portion (36% of Gaming participants and 22% of Gaming+ participants) to ultimately receive a lower dose of motor practice than the Standard Care group (ie, <5 hours). Though the focus of the therapist consultations was centered on behavioral techniques to increase daily arm use, with only a brief adherence check-in at each session, participants adhered better to independent game play when they were more closely followed by a therapist (Gaming vs Gaming+ 1-tailed Wilcoxon rank sum test, z = 1.74, P = .04). Each additional teleconsultation yielded a median increase of 52.1 minutes of independent play time.

Figure 5

Box Plot Demonstrating Significantly Greater Adherence to Game Play Among the Gaming+ (Right) Participants Than Among the Gaming (Left) Participants.

Analysis of Secondary Outcomes

Secondary outcomes assessed at all time points (pretreatment, posttreatment, and 6-month follow-up) included the Neuro-QoL, 9-HPT, and the SWM touch test. The 9-HPT suffered from extensive floor effects in our sample (60% of participants were unable to place any pegs at baseline), so it was dropped from further analysis of comparative treatment effects.

The Neuro-QoL incorporates a relatively new CAT system to measure QOL across multiple domains. There were no significant treatment-related improvements on the Neuro-QoL (main effect of time), and the mean treatment changes on all scales were smaller than the minimally detectable change for the measure.⁹⁴ There were no significant comparative treatment effects (group × time interactions). These results are consistent with some prior work that shows that QOL improvements from CIMT are limited to ADLs and motor domains (ie, did not generalize to the social, family, cognitive, mood domains, etc, that were measured here)⁹⁵ or that general QOL improvements are not generally realized from CIMT.⁹⁶ However, these findings are inconsistent with another previous study that showed QOL improvements following CIMT across multiple nonmotor domains that strongly correlated with improvements in daily arm use.⁹⁷ The inconsistencies in findings may be partially explained by differences in the assessments that were used, suboptimal psychometric properties of existing QOL assessments⁷⁰ (eg, in our sample, the composite T score had an SD of just 2.4, and each subscale had substantially lower-than-expected SDs, producing a restricted range for analysis), or participant characteristics.⁹⁸ In sum, Neuro-QoL results should be interpreted with caution given the inconsistently demonstrated validity of the assessment, general inconsistency within the literature regarding the impact of motor training on QOL (likely in part stemming from limitations of the assessment tools), and the salient impact that mood and educational level have on self-reports of QOL.⁹⁸

An SWM touch test was employed to determine whether motor rehabilitation could restore tactile sensation (measured in the index finger). Tactile sensation significantly improved over time, independent of group. Participants could detect a pressure stimulus of about 0.24 g less at posttreatment (P < .01) and 6-month follow-up (P < .01). Participants with poorer sensation at baseline made significantly greater improvements. Of the 42 people with severe impairment at baseline (loss of protective sensation), 2 (5%) were able to fully regain protective sensation and 10 (24%) were able to regain some protective sensation. Of the 25 participants with moderate impairment at baseline (diminished protective sensation), 1 (4%) was able to fully recover ability to detect light touch and 8 (32%) were able to fully regain protective sensation. Of the 62 participants with mildly impaired sensation at baseline, 15 (24%) were able to fully recover ability to detect light touch.

Characteristics Associated With Treatment Response

Motor Function (WMFT, Primary Outcome)

Participants with poorer initial motor ability made more robust improvements in motor function (WMFT) that were retained through 6-month follow-up (Table 10, Figure 6). There were no significant 3-way interactions, indicating that the comparative treatment effect between groups did not depend on initial motor ability. Table 11 shows the effect sizes of the covariates for the full model.

Figure 6

Participants With Poorer Initial Motor Ability (Higher Scores) Made More Robust Improvements in Motor Function (WMFT) That Were Retained Through 6-Month Follow-up.

Table 11

Effect Sizes for Patient Characteristics Associated With WMFT Response.

Arm Use (MAL, Primary Outcome)

Contrary to its effect on motor function improvements, poorer motor function at baseline was associated with significantly poorer improvement and retention of gains in daily arm use (Figure 7). Higher baseline MAL scores were also associated with smaller treatment-induced improvements on the MAL (Figure 8). Comparative treatment effects were not influenced by baseline motor function or arm use (3-way interactions were not significant). Table 12 shows the effect sizes from the full model.

Figure 7

Poorer Motor Function at Baseline Associated With Significantly Poorer Improvement and Retention of Gains in Daily Arm Use.

Figure 8

Higher Baseline MAL Scores Predicted Smaller Treatment-Induced Improvements on the MAL.

Table 12

Effect Sizes for Patient Characteristics Associated With MAL Response.

SWM Touch Test

Poorer sensation at baseline was associated with significantly greater improvements in sensation (partial slopes = −0.26 for treatment and follow-up, P ≤ .04; Figure 9).

Figure 9

Poorer Sensation at Baseline Associated With Significantly Greater Improvements in Sensation.

None of the following participant-related factors influenced treatment response or the comparative treatment effect on any primary or secondary outcome measure: cognition, touch sensation, age, sex, whether the dominant hand was more affected, or time since stroke. The mixed-effects general linear models that best characterize treatment response are included in the Appendices.

Adverse Events

Twelve participants experienced medical events unrelated to the study, 10 of whom discontinued participation as a result. Two participants reported study-related adverse events (AEs). One Gaming participant experienced bruising from wearing the activity tracker too tightly, and another experienced nausea secondary to muscle soreness. Both AEs resolved quickly or spontaneously and did not affect study participation.

Exploratory Analyses

To ensure that all study groups had the opportunity to receive intensive motor training, the Standard Care group was given the opportunity to receive in-home game play after completing the 6-month follow-up. Before taking the game home, participants in this group received a 2-hour therapist consultation session in which they learned how to operate the game and were introduced to the behavioral components of the intervention, which they subsequently completed independently. The structure of this 2-hour session was identical to the first in-person consultation from the Gaming and Gaming+ groups, as was the prescribed game play and agreed-upon in-home task practice. Following this initial consultation, crossover participants completed the remainder of the program independently (ie, with no additional therapist consultation). Although controlled comparisons cannot be drawn from this crossover period, it provides some preliminary evidence of (1) the impact of therapist consultations on adherence to game play, (2) the impact of therapist consultations on improvements in daily arm use, and (3) whether a second bout of motor practice months later may continue to improve motor function.

Nineteen participants completed both the crossover intervention and post-crossover motor testing. Mixed-effects general linear models applied to these 19 complete cases examined the main effect of time on the primary outcome variables. No interaction terms were included in the models given the small sample size.

Adherence was approximately equal to that of the Gaming group, with a median active game play time of 6.87 hours. Only 1 individual completed the prescribed 15 hours. Motor function continued to improve through the post-crossover period. By the end of the crossover period, participants attained motor gains that were similar to those attained by the CIMT and Gaming+ groups during their initial treatment period (WMFT Ln post-crossover – WMFT Ln pretreatment = −0.34; Figure 10, left panel). Statistically significant improvements in daily arm use were also achieved (pretreatment to post–cross-over gains = 0.84; 95% CI, 0.51-1.17), but these failed to reach the threshold for clinically meaningful improvement by the end of the crossover period (Figure 10, right panel). Given a much smaller improvement in daily arm use during the crossover period (post-crossover MAL – follow-up MAL = 0.1 point) than that achieved by the Gaming and Gaming+ participants during the period of active gaming intervention (1.3 and 1.5 points, respectively), continued therapist interaction appeared critical for promoting behavior change.

Figure 10

WMFT and MAL Scores at Each Time Point.

Subpopulation Considerations

As expected, comparative treatment effects do not appear to be influenced by demographic or clinical characteristics. Therapists should thus base their choice of treatment on participant preferences and a treatment's overall effectiveness for achieving participant goals. For example, participants who reside in rural areas or who lack transportation may be more likely to prefer a treatment approach that can be delivered via teleconsultation, such as the gaming therapies employed here. Provided that these participants can adhere to a sufficiently intense program of self-management and receive adequate therapist support through teleconsultation, they can achieve gains that are equal to intensive in-person therapies, such as CIMT.

For all outcome measures that showed a treatment response (MAL, WMFT, SWM touch test), larger treatment gains were achieved by those who initially scored the poorest. This finding supports the concept of a proportional rehabilitation response (eg, those with the poorest function who make a 25% proportional recovery will achieve greater gains because their range of possible improvement was initially larger). Consistent with previous research, individuals with initially poorer motor ability show more pronounced improvements in motor function in response to motor practice yet also have more difficulty sustaining daily arm use.^33-35 These individuals remain excellent candidates for motor practice using any approach (ie, in-home gaming vs in-clinic rehabilitation), but they may need continued support integrating their weaker arm into daily activities. In contrast, individuals with initially better motor ability may benefit more from devoting more treatment time to the behavioral techniques of CIMT than to structured motor practice.

Implications for Future Research and Clinical Practice

This study provided evidence in support of “technology outsourcing” the time that a therapist typically spends on therapeutic exercises to enable the therapist to focus limited therapy time on behavioral techniques that promote habit change; technology can be effectively used to encourage intensive and effective independent practice of therapeutic exercises. This finding has important, broad implications for rehabilitation of other neurologic conditions in which daily use falls short of what would be expected from the degree of recovery, such as infrequent speech production in the context of mild expressive aphasia or verbal apraxia. Study findings have consistently shown that habit change (eg, increased use) does not occur unless the client receives specific behavioral techniques.^8,14 Because improved function during everyday activities and interactions is a primary goal of rehabilitation, there appears to be a primary disconnect between the techniques that can best accomplish this goal and what is currently practiced in rehabilitation. Dissemination initiatives should therefore have a 2-prong focus: (1) to solve rehabilitation access barriers by promoting adoption and viable payment structures for rehabilitative gaming technologies; and (2) to educate clinicians about the behavioral techniques employed in CIMT and how they can be broadly applied.

Limitations

Appendix A.

ITT Analysis in Which Dropouts Are Considered Complete Nonresponders (PDF, 161K)

Appendix B.

Per-Protocol Analysis of the WMFT (PDF, 261K)

Appendix C.

Statistical Models: ITT Analysis With All Missing Data Interpolated (PDF, 80K)