Research for JYU: An AI-Driven, Fully Remote Mobile Application for Functional Exercise Testing

Abstract As people live longer, the incidence and severity of health problems increases, placing strain on healthcare systems. There is an urgent need for resource-wise approaches to healthcare. We present a system built using open-source tools that allows health and functional capacity data to be collected remotely. The app records performance on functional tests using the phone’s built-in camera and provides users with immediate feedback. Pose estimation is used to detect the user in the video. The x, y coordinates of key body landmarks are then used to compute further metrics such as joint angles and repetition durations. In a proof-of-concept study, we collected data from 13 patients who had recently undergone knee ligament or knee replacement surgery. Patients performed the sit-to-stand test twice, with an average difference in test duration of 1.12 s (range: 1.16–3.2 s). Y-coordinate locations allowed us to automatically identify repetition start and end times, while x, y coordinates were used to compute joint angles, a common rehabilitation outcome variable. Mean difference in repetition duration was 0.1 s (range: −0.4–0.4 s) between trials 1 and 2. Bland-Altman plots confirmed general test-retest consistency within participants. We present a mobile app that enables functional tests to be performed remotely and without supervision. We also demonstrate real-world feasibility, including the ability to automate the entire process, from testing to analysis and the provision of real-time feedback. This approach is scalable, and could form part of national health strategies, allowing healthcare providers to minimise the need for in-person appointments whilst yielding cost savings.