Although proper prospective epidemiological studies are lacking, several cohort studies suggest that impairments of motor function and activities of the upper limb (UL) affect up to 80% in acute stroke [1,2]. As a consequence, most patients with stroke experience a reduced upper limb capacity characterized by an inferior quality of movement (QoM) [3,4], independency in their activities of daily living (ADL) [5,6] and health-related quality of life (HrQoL) post stroke [7]. In a recent longitudinal study combining four different cohort studies (N=412) with weekly or biweekly measurements in time, we showed that the time course of upper limb function measured with Fugl-Meyer Upper Extremity (FM-UE) scores is heterogeneous and follows a proportional amount of spontaneous neurobiological recovery in most subjects with a first ischemic stroke [8,9]. Forty to 50% of all subjects suffering from a minor stroke show good or excellent motor recovery with recovery coefficients beyond 80% that plateaus within the first 2 to 3 weeks post stroke [8]. Another 20 to 30% of stroke survivors show moderate to mild recovery coefficients ranging from 10 to 80% and plateauing mainly within 5 to 8 weeks post stroke [8], whereas 20 to 30% showed poor recovery in which the coefficients remained below the 10% [8]. Longitudinal kinematic analyses showed that the intra-limb coordination in multi-joint movements like reaching or pointing is always compromised and characterized by a reduced ability to control simultaneously all degrees of freedom (DoF) when performing a multi-joint movement like reaching or pointing, even in those subjects with a minor stroke [10,11] Interestingly, this recovery in intra-limb coordination occurs within the same time window of 5 to 8 weeks [2,12-14], indicating that behavioral restitution (or recovery) in intralimb coordination parallels recovery of muscle synergies, and metrics that characterizes return of QoM such as precision, speed and smoothness [2,4]. Importantly, the window of behavioral restitution of 5 to 8 weeks defining ‘recoverers’ and ‘non-recoverers’, is not unique for the most affected upper limb but also found for recovery of the lower limb [14,15] and other neurological impairments, such as visuospatial neglect [16,17], aphasia [18,19] and somatosensory deficits [20]. These findings suggest that behavioral and cognitive restitution is mainly driven by the same, underlying processes of intrinsic, spontaneous neurobiological recovery [9,21,22]. The next challenge is to investigate the causal factors that drive these intrinsic processes of recovery [8] as well as variables that may interact with these processes [22,23] such as tasks-specific practice [24], neuromodulation therapies [25] and other innovative technologies like upper limb robotics (UL-RT) [26].

Currently, there are more than 38 different types of UL-RTs that have flooded the market in recent 2 decades, allowing that more subjects are able to increase the number of repetitions and thus intensity of practice [26,27]. Importantly, UL-RTs do not get tired, are more precise in their guidance during movement and able to make exercises more attractive by offering simultaneously features such as serious gaming, in contrast to physical (PT) and occupational therapists (OT). Our main question we addressed in our recently published comprehensive systematic review and meta-analysis (i.e., research synthesis) [26] was: What is the current evidence that these expensive devices work? [26] Subsequently, we did investigate how patient, trial, robot and other intervention variables are associated with found effect sizes (ESs). On request of the editorial board of Journal of Experimental Neurology, in this article we will briefly present the main findings of our research synthesis and elaborate on more fundamental questions that need to be addressed for designing the next generation of commercial UL-RTs as well as futures trials after stroke.

Search strategy

Randomized controlled trials (RCTs) comparing UL-RTs with any other intervention on patients with UL limitations post-stroke were identified in electronic searches from PubMed, Wiley/Cochrane Libraries, Embase, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Web of Science, SportDISCUS, Physiotherapy Evidence Database (PEDro), and Google Scholar from inception until August 1^st, 2022. The full search strategy and further details about data selection and extraction are available in the published report of our research synthesis [26]. The cited report follows the PRISMA guidelines [28].

Methodological quality

We assessed the methodological quality of included trials using the PEDro scale [29]. The analysis was performed using published data only.

Quantitative analysis

The extracted data were pooled in a meta-analysis using Review Manager (RevMan) Version 5.4 (Copenhagen: The Nordic Cochrane Centre, Cochrane Collaboration, 2014). RevMan, version 5.4, using Hedges’ g model for continuous outcomes, is a widely used tool for data management, analysis and reporting [30,31] provided by The Cochrane Collaboration, a renowned institution in high quality research syntheses. We applied sensitivity analyses to identify interference variables that may be significantly associated with the individual ESs found, in terms of UL-muscle synergies, muscle strength and/or capacity. We investigated the relationship of each candidate modifier with the individual ESs using meta-regression analyses. The significance of associations between continuous outcomes and individual ESs was investigated by calculating Pearson or Spearman rank correlation coefficients, depending on the distribution of the data. Associations with binary outcomes were investigated using Mann-Whitney U-tests for independent samples. Meta-regression analyses were performed using SPSS (IBM Corp. Released 2021. IBM SPSS Statistics for Windows, Version 28.0. Armonk, NY: IBM Corp). SPSS is a valuable and widely used tool in statistical analysis in various research fields [32]. A two-tailed p-value <0.05 was considered statistically significant.

Search strategy

We included 90 RCTs involving 4,311 subjects out of a total of 1625 unique hits in the electronic search. The time gap from stroke to randomization was less than 3 months in 23 studies, while 43 studies included patients in the late subacute phase post-stroke [33], i.e., after more than 3 months, and in 24 studies the post-stroke phase was not evaluated as a criterion. The duration of the intervention ranged from 2 weeks to 12 weeks (mean 5.81, SD=2.59 weeks), with a mean UL-RT therapy time of 53.55 (SD=33.9) minutes per session and 4.56 (SD 1.73) sessions per week [26]. The number of repetitions performed per session was reported in 23 of the included studies for the intervention groups and in 7 for the control groups. In the intervention groups, the mean number of repetitions per session was 805.89 (SD=656.54), with a range from 50 [34] to 3,600 [35], while in the control groups, the average number of repetitions was 357.84 (SD=385.6), ranging from 50 [34] to 976 [36].

The extracted outcome measures used in these trials were categorized according to the domains described in the ICF [37] to investigate the following constructs: upper limb motor functions, activity, participation and ADLs. Further details are available in the published report [26].

Risk of bias

The PEDro score was calculated for each study, and the median PEDro score was 6 [6-7]. The PEDro score was confirmed on the PEDro platform for 52.22% of the included studies (47/90), the other 43 studies were either not present on the platform (K=33) or had a different scoring (K=10). The methodological quality of the included studies based on the PEDro assessment tool ranged from 3 to 9.

Quantitative analysis

Eighty-six (N=4,240) out of the 90 included RCTs were found suitable for quantitative analysis. Pooling of data for subgroup or sensitivity analyses was not possible when only one study per subgroup was available. Four trials were excluded from pooling due to a lack of point measures and/or estimates of variability [34,38-40].

Figures 1a-c show the forest plots for muscle synergies (measured with Fugl-Meyer Assessment of the Upper Limb – FM-UL), UL-capacity, and muscle power respectively. The relevant results of the overall analyses are available in the published report [26].

A significant homogeneous Effect-Size summary (ES_summary) was found for muscle synergies, reflecting an average between-group difference of 2.23 [1.11–3.35] out of 66 points on the FM-UL Similarly, non-homogeneous statistically significant SMDs were found for UL muscle power, with an SMD of 0.39 [0.16–0.61] and for basic ADLs with an SMD of 0.28 [0.10–0.45]. A homogeneous statistically significant SMD of 0.11 [0.00–0.21] was found for performance measures. A non-significant heterogeneous ES_summary was found for UL-capacity (SMD 0.04 [-0.10-0.18]). We found no evidence for publication bias and obtained symmetric funnel plots between trial ESs and Standard Error.

We found a positive, significant correlation coefficient between the baseline SD value of FM-UL of each trial that reported these data and the ES for FM-UL (Pearson’s r=0.276; p=0.024) and for UL-capacity (Spearman’s r=0.339; p=0.03). Higher ES_summary values for muscle synergies significantly correlated with PEDro scores (Spearman’s r=0·26; p=0·031) and time spent training in each session (Spearman’s r=-0·251; p=0·04). Trials that used exoskeletons showed a significant correlation with higher ES_summary for UL-capacity (Mann-Whitney U-test; z=2·89; p=0·004) and muscle power (Mann-Whitney U-test z=2·24; p=0·025). Higher ES_summary values for muscle power also significantly correlated with restriction of trunk compensations during UL-RT training (Mann-Whitney U-test z=-2·36; p=0·017). However, meta-regression showed no significant association between on the one hand, muscle synergies (i.e., FM-UL scores), muscle strength, and UL-capacity and, on the other hand, trials’ ESs with respect to: publication year; total number of subjects included in the trial; methodological quality of the trial; proximal (shoulder–elbow) and/or distal (wrist–hand) UL-RT focus of intervention; degrees of freedom that are controlled by the UL-RT; gravity support (yes/no); simultaneous virtual gaming during UL-RT (yes/no); dosing of therapy in terms of total duration of the intervention and number of repetitions; and competing interests declared by the authors.

Upper limb rehabilitation encompasses task-specific training in an intensive way after stroke [23]. Commercial UL-RTs, serving both pillars may serve as an effective therapy to achieve clinically meaningful improvements post stroke. However, in the present research synthesis [26], we found small, but significant mean difference of about 3.4% on FM-UE scores without clinically meaningful effects at level of UL-capacity. Our meta-regression confirms that selected subjects with some potential of UL-recovery may benefit most from therapy such as UL-RT, especially when applied in the first 3 months after stroke onset [1,2]. The robustness and consistency of our findings challenges the selling of commercial UL-RTs for the clinical field and rather indicate that a better mechanistic understanding about assumed underlying interaction effects between motor recovery and motor learning is needed. Importantly, our results show almost the same between group difference of 2.27 points for FM-UE and no differences for outcome UL-capacity in the only phase IV, RATULS trial involving 770 subjects poststroke [41]. Also, the same significant, homogeneous mean difference (MD) of about 2.15 points (CI95%: 0.73 -3.57) on outcome of FM-UE scores (~3.2%) without significant effects on UL-capacity after stroke (i.e., 0.04; 95%CI: -0.12, 0.19) [27], were found after pooling 38 trials (N=1206) in 2017 as well as after pooling 10 small-sampled phase II trials (N=218) in 2008 [42]. Obviously, despite the fast-growing technology with respect to software and hardware for manufacturing UL-RTs, the overall effect-size remained the same following FM-UE without effects on UL-capacity [26]. Considering the robustness and consistency of neutral effects, we did recommend in our manuscript to stop with swamping health care services with new commercially available UL-RT devices. Instead, we propose the UL-RT-industry to go back to the drawing table and develop a better theoretical framework for designing the next generation of UL-RT devices post stroke. In contrast to pharmacodynamics, the fundamental step in making assumptions about what and how stroke subjects learn to improve their quality of movement when performing upper limb multi-joint tasks after stroke, seems to be skipped in this research pipeline when designing UL-RTs.

Cost-effectiveness

Despite the lack of evidence and proper theoretical framework that targets the dynamic interaction between motor control and motor learning during stroke recovery, a commonly raised argument is that UL-RTs may serve as a cost-effective alternative replacement for usual care offered by a physical and/or occupational therapist [43,44]. However, the cost-effectiveness is not supported in the literature. Analysis of Health Technology Assessment in the only phase IV RATULS trial showed that using the MIT-manus robot is accompanied with higher costs when compared to usual care given by therapists [45,46]. In a previous Cochrane review, Mehrholz and colleagues suggested that electromechanical and robot-assisted training could create additional costs of rehabilitation after stroke [47]. The complete analysis of the costs would consider the initial investments, operational costs, and disposal costs. The initial investment includes the cost of the device and its installation and may vary a lot depending on the complexity of the device (e.g., 50K - 60K € circa for end-effectors such as the MIT-manus [45], to 150k €+ for complex exoskeletons). Operational costs include maintenance and repair, training and specialized staffing, software updates and licensing. Unfortunately, no data are currently available for operational costs, whereas disposal costs have not been quantified to date. Despite the lack of these data that would increase the cost/effect ratio, the amount of quality evidence produced over the years shows the lack of cost-effectiveness for commercial UL-RTs.

What is needed?

In our opinion, to understand the dynamics of motor recovery early post stroke, it is important to acknowledge the non-linear time course of intrinsic, spontaneous neurobiological recovery and the need to delineate between behavioral restitution at level of body function and compensation [23,48]. As suggested, the time course of behavioral restitution of impairments is non-linear and plateauing within 5 to 8 weeks post stroke, suggesting that selection of subjects at baseline and timing to achieve recovery is paramount in designing trials [1,2].

Second, fine-grained, serial biomechanical measures of movement quality in skill acquisition indicate that therapy-induced improvements are mainly caused by an adaptive control in which the same multi-joint movement, is controlled in a different way with to a less efficient performance when compared to the movement quality seen in healthy age and gender-matched subjects [2,21,49]. As already shown by Bernstein in the sixties, to achieve optimization in performance, subjects need variability in practice in its own preferred context allowing to use different task parameters such as speed distance and difficulty to achieve long-term retention [50]. Importantly, ‘normal movement’ seen before stroke, or when compared to age- and gender-matched healthy subjects may not serve as an adequate reference to understand the adaptive ways how stroke subjects control their multi-joint movement when performing a meaningful task after stroke. In addition, just increasing the intensity of practice by performing more repetitions of the same movement without variation, does not lead to motor learning and, with that, a better quality of movement [26,51].

Third, longitudinal biomechanical studies are needed in which behavioral restitution is distinguished from compensation allowing to understand what patients learn by an intervention when they improve in their upper limb capacity and QoM after stroke [2]. However, after reviewing 32 longitudinal studies reporting 46 different kinematic metrics none of these studies did explicitly addressed the distinction between behavioral restitution and compensation [2]. Recently, we achieved consensus in the International Stroke Recovery and Rehabilitation Alliance (ISRRA) how to measure QoM post stroke in terms of behavioral restitution and compensation based on using high-fidelity systems to capture movement, by defining four ‘performance assays’, in which behavioral compensation is prevented and healthy age- and gender-matched controls are used as a comparator [3].

Fourth, none of the designers of commercial UL-RTs did publish a peer reviewed paper describing how to measure changes in QoM, but rather choose a common motor impairment scale, such as FM-UE scores. FM-UE reflecting muscle synergies and with that, strongly associated with the DoF that patients can control for are mainly defined at the level of body function. However, improvements in FM-UE scores are also flawed by the 8% improvements in muscle strength, as seen in our research synthesis and UL-RT [26] and not reflecting a pure assessment of ability of dissociation post stroke. Therefore, FM-UE scores that plateaus within 8 weeks might be not the ideal primary measurement of outcome for capturing adaptive motor learning in UL-RT trials. It is more straightforward to measure the impact on compensation strategies to improve UL-capacity and search for possible interaction effects of motor learning with QoM [23,52].

Fifth, none of the designers made hypotheses about how the UL-RT targets the interaction between motor control and motor learning during stroke recovery and with that, none of designers of commercial UL-RTs made assumptions about how the device can align with the preferred compensatory movement of the subject when performing a meaningful task. Based on principles of practice makes perfect in neurorehabilitation and increasing intensity of practice by repetition without repetition, there is a fundamental need for insight in the neurobiological assumptions of manufactured UL-RTs. At least these assumptions should address how the UL-RT interface interact between motor control and motor learning and deals with the redundancy (or abundancy) in the number of DoF in multi-joint movements like reaching in a patient-specific, natural way [26,53-55].

Sixth, most UL-RTs using haptic, force, electromyographical, proprioceptive, visual and/or auditory ways of feedback for movement correction. However, the chosen feedback, i.e., the form of perception provided, determines different forms of ‘resistance’ to the preferred movement that the patient would perform in a goal-directed multi-joint movement, i.e., the ‘action’. Whether these different modalities of action-perception mismatch have different effects on adaptive ‘motor learning’ is unclear. The control by the end-user should be continuous, shared when optimizing a preferred adaptive movement and assisted only if needed. Several mathematical models have been created to try to describe how the central nervous system (CNS) deals with the behavioral dynamics, such as the minimum-jerk model [56], the threshold-position control model [50] or the minimum torque-change model [57]. However, all these theoretical models are insufficiently considered for their pros and cons in perspective motor control and motor learning when improving in QoM after stroke.

In sum, the current knowledge gap that links the non-linear relationship between body (UL) and machine (RT) in a real-world environment during stroke recovery, is still a ‘black box’, that raises questions such as “Who is in charge and who is adapting when performing a multi-joint goal-directed movement such as reaching?” “Is it the UL-robot to the subject or the subject with the impaired arm to the robot?” These fundamental dilemmas need to be addressed before making assumptions about interfacing robot and machine. Subjects with compromised control in their degrees of freedom after stroke, need to be able to explore their multi-joint task by themselves within their own symptom-specific constraints, and not just follow the haptic corrected guidance. Therefore, questions about ‘who is in charge in goal-directed multi-joint movements, need to be addressed first in designing new UL-RTs.

In this ‘two-learners problem’ [58,59] in which on both sides, the subject and the robot, need to co-adapt in their performance making the movement in the most effective and efficient way [59,59], greater authority should be given to the user of the device than the programmed control of the device self. Recently, Kang and colleagues suggested that Artificial intelligence (AI) or deep learning techniques may be a better way to optimize the real time interaction between body and device in a patient preferred, task-specific way [60].

What is next? – directions for future research

As emphasized in our research synthesis [26], it is significant to note, that the aim of UL-RTs designed to support adaptive motor learning, should not be confused with robot-devices that are able to ‘replace’ or ‘take over’ parts of the affected limb, such as a bionic limbs or devices designed to bypass the CST using Brain-Computer Interfaces (BCI) or Brain-Machine Interfaces (BMI) [61]. At the same moment, we need to acknowledge that advances in technology constantly stay ahead of the research (which takes years to conduct and refine). Being optimistic about UL-RTs and certainly not closing the door as emphasized in the discussion of our review [26], there are several basic steps needed to follow first in this field.

First, in line with the vision of the International Stroke Recovery and Rehabilitation Alliance (ISRRA), there is an urgent need to give a set of definitions and recommendations about, respectively the used taxonomy and underlying assumptions of the UL-RT. Even a consensus-based definition of UL-RT is lacking. In line with other interventions and outcomes, the International Classification of Functioning and Disability model (ICF) may start as a framework and the taxonomy should be aligned with those of movement, neuro- and rehabilitation sciences.

Second, to improve the efficiency in this translational and transdisciplinary research pipeline in designing UL-RT, we suggest improving the transparency of the theoretical rationale and fundamental assumptions of newly designed UL-RTs [26]. At least, the neurobiological and technical assumptions about the working of UL-RT how motor control and motor learning may interact with dynamics of motor recovery, should be clear, which is not different from knowledge about pharmacodynamics of a medication and testing before prescribing commercial available pills [26].

Third, to improve the trial methodology including stratification of underpowered UL-RT studies acknowledging the problem of heterogeneity of spontaneous motor recovery and need for selecting patients with some potential for recovery. The positive association between baseline FM-UE score and RT-induced improvement in UL-capacity is in line with prognostic research showing that about 20% to 30% of subjects with FM-UE scores of 18 points or lower fail to show spontaneous neurobiological recovery at all [8,52].

Fourth, there is a need for longitudinal studies investigating how UL-RTs may influence adaptive learning by measuring QoM during stroke recovery, rather than the question if we can change FM-UE scores at impairment level [62]. Preferably, these aims should be in line with the recommendations on how to monitor behavioral restitution and compensation post stroke [3,63].

Fifth, longitudinal studies have shown that about 60% of these improvements are defined by recovery of the hand and less by the transport of the hand (i.e., arm and trunk) [2,64]. However, proof-of-concept trials investigating the effects of hand-robots on improving dexterity are still in their infancy.

Finally, fundamental research is needed about the parallelism between spontaneous motor recovery and ‘motor learning’ and address questions how neural restitution and substitution are longitudinally associated with behavioural restitution and compensation post stroke [65,66]. This delineation is not imperative for improving UL-RT trials but also for allowing proper interpretation of neuroimaging studies to investigate how UL-RT-induced brain plasticity, may contribute to stroke recovery [3].

The current research synthesis has some limitations. First, our effort is compromised by the lack of an appropriate comparator in current trials. The heterogeneity of the control interventions across different trials can make pooling of data in one meta-analysis questionable and reduce the reliability of the results [67,68]. Therefore, further classification of ‘usual’ or ‘control care’ as a comparator is required at this point. Second, the present findings focus on UL-RTs designed as a training device in support of rehabilitation services. This restriction does not mean that other robot types are ineffective from the perspective of regaining UL-capacity. Depending on the definition that is used, UL robots could be perfectly able to ‘take over’ pre-programmed activities of the paretic limb. Finally, the conclusions in the present meta-analyses are based at group level of RCTs and not at an individual level in which the numbers and characteristics of responders and non-responders of clinical meaning full effects on UL-muscle synergies and UL-capacity can be identified.

We would like to thank Dr Janne Veerbeek and Hans Ket, co-authors of the research synthesis to which we refer in this manuscript, for their contribution.