ExMR: Extremity Motion Refinement

for Monocular Human Motion Capture

Tom Wehrbein Bodo Rosenhahn

Leibniz University Hannover

Journal extension of FootMR (3DV 2026)

arXiv (soon) Video Code (soon) FootMR (3DV 2026)
Input video frames of a ballet dancer (top) and the ExMR reconstruction overlaid as a blue body mesh (bottom), with zoomed insets on hands and feet.
Monocular motion reconstruction with ExMR. Our work focuses on the articulation of the extremities (the hands and feet) within full-body motion capture.

Abstract

Detailed hand and foot articulation is important for animation and motion analysis, yet remains challenging to recover from monocular video. Recent expressive video methods infer hand articulation from shared whole-body representations. Our conference work suggests that this underexploits local evidence: the Foot Motion Refinement (FootMR) module refines ankle rotations from foot keypoints, improving reconstruction even when the initial estimator receives the same keypoints. We extend this principle to hands with the Hand Motion Refinement (HandMR) module. Because 2D finger keypoints are often missing or unreliable, HandMR combines image features of a pretrained hand reconstruction model with initial arm and hand rotations to refine elbows, wrists, and fingers. We integrate FootMR and HandMR into ExMR (Extremity Motion Refinement), a system for reconstructing 3D body, hand, and foot motion from monocular video. An initial temporal estimator predicts whole-body motion, after which both refiners revisit local evidence to predict residual joint rotations. Experiments on ARCTIC, MOYO, RICH, and MOOF show that ExMR outperforms state-of-the-art methods in hand and foot reconstruction, reducing hand per-vertex error and ankle joint angle error on MOYO by 19% and 18%, respectively, relative to the strongest competitor. Code will be made available for research purposes.

Video

Qualitative examples of 3D full-body motion reconstructed from monocular video using ExMR, including hand and foot articulation. All results use automatically estimated person and hand bounding boxes. No ground-truth information is used at inference.

Method

ExMR architecture: an initial temporal estimator followed by the HandMR and FootMR refinement modules.
Overview of ExMR at inference. Given a monocular video, the initial estimator combines per-frame body and hand image features with 2D body-and-foot keypoints to predict a SMPL-X motion sequence. The HandMR and FootMR modules reuse the corresponding visual evidence and predict residual rotations conditioned on part-specific initial rotations: elbows, wrists, and fingers for HandMR, and knees and ankles for FootMR. HandMR refines elbows, wrists, and fingers, while FootMR refines ankles. The initial estimator and both refinement modules are jointly trained temporal transformers. Snowflakes denote frozen pretrained models.

Results

Input frames, initial estimates, and refined ExMR estimates with hand close-ups for examples from MOYO, RICH, and ARCTIC.
Qualitative examples of extremity refinement in ExMR. Rows show one example each from MOYO (top), RICH (middle), and ARCTIC (bottom). Initial and refined estimates are outputs of the same ExMR checkpoint. Amber circles in the MOYO row highlight the correction of the initial foot orientation.
Side-by-side comparison of SAM3D-Body (gray meshes) and ExMR (blue meshes) on six examples with zoomed insets.
Qualitative comparison with SAM3D-Body. Each example shows, from left to right, the input image, the SAM3D-Body prediction (gray), and the ExMR prediction (blue). Panels (a), (c), and (e) show object-manipulation examples; (b) and (d) show dance poses; (f) shows a seated pose. Within each example, corresponding insets use identical image regions and magnification. SAM3D-Body overlays show its native MHR meshes, while ExMR overlays show SMPL-X meshes.

BibTeX

@article{wehrbein2026exmr,
  title   = {{ExMR}: Extremity Motion Refinement for Monocular Human Motion Capture},
  author  = {Wehrbein, Tom and Rosenhahn, Bodo},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}

Please also consider citing the conference paper:

@inproceedings{wehrbein26footmr,
  title     = {Improving {3D} Foot Motion Reconstruction in Markerless Monocular Human Motion Capture},
  author    = {Wehrbein, Tom and Rosenhahn, Bodo},
  booktitle = {Proceedings of the International Conference on 3D Vision (3DV)},
  year      = {2026},
  pages     = {871--881},
  doi       = {10.1109/3DV69130.2026.00088}
}

Acknowledgements

This work was supported by the Federal Ministry of Education and Research (BMBF), Germany, under the AI service center KISSKI (grant no. 01IS22093C), the Deutsche Forschungsgemeinschaft (DFG) under Germany's Excellence Strategy within the Cluster of Excellence PhoenixD (EXC 2122), and the European Union within the Horizon Europe research and innovation programme under grant agreement no. 101136006–XTREME.