MVDoppler-Pose: Multi-Modal Multi-View mmWave Sensing for Long-Distance Self-Occluded Human Walking Pose Estimation

Jaeho Choi1,2, Soheil Hor2, Shubo Yang2, Amin Arbabian2
1DGIST, 2Stanford University
CVPR 2025
Interpolation end reference image.

We demonstrate that the use of mmWave technology can tackle the challenges of camera pose estimation, particularly in scenarios involving occlusions and long distances. Moreover, unlike previous mmWave approaches limited to controlled human movement scenarios in restricted space, our MVDoppler-Pose enables HPE even for random walking subjects across a much larger space.

Abstract

One of the main challenges in reliable camera-based 3D pose estimation for walking subjects is to deal with self-occlusions, especially in the case of using low-resolution cameras or at longer distance scenarios. In recent years, millimeter-wave (mmWave) radar has emerged as a promising alternative, offering inherent resilience to the effect of occlusions and distance variations. However, mmWave-based human walking pose estimation (HWPE) is still in the nascent development stages, primarily due to its unique set of practical challenges including the quality of the observed radar signal dependent on the subject’s motion direction. This paper introduces the first comprehensive study comparing mmWave radar to camera systems for HWPE, highlighting its utility for distance-agnostic and occlusion-resilient pose estimation. Building upon mmWave’s unique advantages, we address its intrinsic directionality issue through a new approach—the synergetic integration of multi-modal, multi-view mmWave signals, achieving robust HWPE against variations both in distance and walking direction. Extensive experiments on a newly curated dataset not only demonstrate the superior potential of mmWave technology over traditional camera-based HWPE systems, but also validate the effectiveness of our approach in overcoming the core limitations of mmWave HWPE.

BibTeX


  @inproceedings{Choi:2025_CVPR,
    author = {Choi, Jaeho and Hor, Soheil and Yang, Shubo and Arbabian, Amin},
    booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    title = {MVDoppler: Unleashing the Power of Multi-View Doppler for MicroMotion-based Gait Classification},
    year = {2025}
  }