A vision-based gamified assessment app that enables patients to track upper-limb recovery at home through motion games and stay motivated.
Product Designers: Xinke (Coco) Wu
Jhih Rou (Iris) Sun
Animation Design: Yameng Li
Xinke (Coco) Wu
James Jun
Neuroscientists at
The Multiple Sclerosis Center of Georgia
Georgia Tech Contextual Computing Lab
Quick Glance.
Context.
“I have to go to the clinic to know how I’m doing at home...”
Every year, millions of people worldwide live with impaired upper-limb function caused by neurological conditions such as stroke, multiple sclerosis, or spinal cord injuries. Rehabilitation is a long, slow, and highly repetitive process.
While most recovery happens at home, patients still rely heavily on occasional clinic visits to understand how well they are progressing.
This creates a painful gap. Clinicians struggle to understand patients’ real day-to-day recovery, while patients are left uncertain about their own progress—making recovery feel invisible, unmeasurable, and emotionally exhausting.
Problem.
3 Key challenges in long-term recovery progress
Through literature review and stakeholder interview with healthcare experts, patients with upper-limb impairments caused by neurological conditions, we identified three key challenges that patients face due to the clinical-isolated assessments.
Research process.
Research-Grounded, Clinically Informed
We conducted mixed-methods research to define, explore, and validate our design opportunities for making assessments more engaging and accessible. These research activities were structured to progressively reduce clinical, technical, and behavioral uncertainties.
key Insight1.
Clinical assessments use standardized metrics — but are still subjectively judged
In the clinics, upper-limb recovery is evaluated through a small set of standardized movement tests, including joint range of motion tests and coordination tests.
Although these assessments follow standardized protocols, their evaluation largely depends on clinician observation and experience.
This makes progress tracking inconsistent and difficult to quantify — especially outside clinical settings.
Standard Upper-Limb Clinical Assessments
Camera-based Technical Approach Experimentation
key Insight2.
Camera-based tracking is promising for objective, hands-free assessment
Expert interviews revealed a critical accessibility barrier: many sensor-based solutions require gripping or force input — actions that stroke and MS patients often struggle to perform.
In contrast, camera-based computer vision enables hands-free assessment, allowing patients to complete self-evaluations using natural movements alone. This opened the opportunity for truly accessible at-home recovery tracking.
Design implications.
Supporting accessible, at-home motion-based self-evaluation
We came up with the following design principles to guide the design toward a assessment system that is measurable, accessible, and behaviorally sustainable.
Evaluation.
Real-World Feedback from Patients and Doctors
We’ve conducted user testing sessions with patients, therapists, and neurologists. All the stakeholders strongly agreed that the system improves accessibility (5.7/7), visibility of recovery progress (5.6/7), and long-term adherence (6.3/7).
