The FreeMoCap Project - and - Gaze/Hand coupling during a combined three-ball juggling and balance task

We present a broad scale, long term, open science endeavor that aims to create a free, open source, low cost, research-grade full-body motion capture system alongside a research project that utilizes this tool to investigate visual motor coupling during a combined three-ball juggling and balance task. The FreeMoCap Project - The computer vision community is making tremendous advances in the field of markerless motion capture software. However, these advances often require a high floor of technical knowledge to be used effectively. This limits their utility to the scientific community and creates a near insurmountable barrier for the general population. The FreeMoCap system leverages emerging markerless motion capture software (e.g. OpenPose, MediaPipe, DeepLabCut, etc) to create a streamlined ‘one-click’ pipeline for 3D kinematic reconstruction of full-body human, animal, and robotic movement. This system works with arbitrary camera hardware and provides methods for synchronous recording of wired cameras (e.g. USB webcams) as well as the post-hoc synchronization of independent cameras (e.g. GoPros). The FreeMoCap project emphasizes ease-of-use, with the eventual goal of developing a system that will allow a 14-year-old with no technical training and no outside assistance to recreate a research-grade motion capture system for less than 100 US Dollars. Juggling/Balance Task - The FreeMoCap system was used to record a subject performing a three-ball juggling task while balancing on a ‘wobble-board’ platform. The full body (and hand) kinematic data produced by FreeMoCap was spatiotemporally calibrated with binocular gaze data recorded by a Pupil Labs mobile eye tracker in a manner analogous to methods used in Matthis, Yates, and Hayhoe (2018), Matthis, et al (PLoS Comp Bio, In Press) and Wirth and Matthis (VSS 2022). The resulting data reveals a tight coupling between gaze and the hand/juggling ball system as well as a complex relationship between the juggling and balance task.