• Register date: July 15th, 2024.
• Release of training data: July 25th, 2024.
• Open for submission on validation set: August 1st, 2024.
• Submission deadline on validation set: September 23rd, 2024.
• Open for submission on test set: September 23rd, 2024.
• Submission deadline on test set: October 1st, 2024.
• Submission deadline for the technical report: October 10th, 2024.
• Winner and invitation speakers: No later than October 20th, 2024.

Human motion capture (MoCap) is the process of recording human movement represented by a sequence of 3D positions and rotations of mesh or joints of the human body. Industrial motion capture systems have been widely applied in movie and game production, sports analysis, medical diagnosis, etc. However, these systems usually consist of tens of synchronized cameras or a group of wearable sensors and specific signal receivers. Despite their high accuracy for human motion capture, individual consumers can hardly afford the high cost and learn professional configurations. Therefore, we investigate accurate human motion capture with consumer-affordable devices and easy-to-use operations for daily applications like eXtended Reality (XR), mobile video production, live video streaming, etc.

The ACM MM'24 Multimodal Human Motion Capture Challenge seeks to harness these advancements by encouraging participants to develop novel algorithms and systems that effectively combine and utilize data from diverse sources. The goal is to create solutions that can perform robust motion capture in real-world environments, overcoming the limitations of traditional systems and paving the way for new applications and innovations.

The challenge will be structured in several phases to ensure a comprehensive evaluation of the submitted solutions:

1. Registration and Dataset Release: Participants will register for the challenge and gain access to the provided multimodal dataset. The dataset will include multi-modality data of IMUs signals and RGB videos labeled with joint positions, joint rotations, SMPL parameters, etc..
   • License: This dataset is available under licence. Please download and read the data license agreement. If you accept all license terms, please fill out and sign the license agreement, then upload it during the registration.
   Download License
   • Registration: Please register here. We will send you a successful registration email within three days after your registration. If you have not received it, please contact us.
   Registration
2. Track Selection: Our challenge is divided into three tracks, a team can participate in no more than two (≤ 2) of the following three race tracks.
   1. Monocular Motion Capture Track: participants can only take monocular RGB data for model training and inference.
   2. Sparse IMUs-based Motion Capture Track: participants can only take data from 6 IMU sensors for model training and inference.
   3. Multi-modal Motion Capture Track: participants can use data from a monocular camera and 6 IMU sensors for model training and inference.
3. Development Phase: During this phase, participants will develop their motion capture algorithms using the provided dataset. They are encouraged to explore innovative techniques for data fusion, noise reduction, and performance improvement.
4. Evaluation Phase: Submitted solutions will be evaluated based on a set of predefined criteria, including MPJPE, the mean global rotation error, Jitter, and so on. A separate validation dataset, which is not included in the initial release, will be used for the evaluation to ensure the fairness and reliability of the results.
5. Paper Submission and Oral Presentation: The top three winners of each track are required to submit a technical report, and the first-place winners are required to deliver an oral presentation on-site or online.

The ACM MM'24 Multimodal Human Motion Capture Challenge will offer attractive prizes for the top-performing teams of each track, including certificates of recognition. In addition, the winners will have the opportunity to publish their work in the conference proceedings, gaining visibility and recognition in the academic and industrial communities.

We invite researchers, developers, and innovators to join us in this exciting challenge and contribute to the future of human motion capture technology. Whether you are an experienced professional or a passionate newcomer to the field, your participation will help drive the progress of multimodal motion capture systems and unlock new possibilities for their application. We look forward to your participation and to seeing the groundbreaking solutions that will emerge from this competition. Let’s advance human motion capture together at ACM MM'24!