PCHMR: Empowering and Benchmarking Human Mesh Recovery in Privacy-Constrained Real-World Settings

Key points

• However, doing so for existing HMR methods often assumes aggregating user-side data from real-world HMR-deployed devices to a central training server, posing a significant privacy risk as sensitive human images are transmitted.1
• How can human mesh recovery be evaluated and improved in privacy-constrained real-world settings?1
• We study this question through a benchmark in which state-of-the-art HMR models are trained with federated learning and a secure-aggregation variant, neither of which centralizes raw images, and evaluated across a wide range of client-scale and data-heterogeneity settings.1
• Furthermore, to study the data bottleneck of privacy-constrained HMR training, we propose DePoser, a local annotation and fine-tuning pipeline enhanced with depth foundation models, with which collaboratively trained HMR models can be locally tailored to the end user's distribution.1

Sources (1)

• [1]PCHMR: Empowering and Benchmarking Human Mesh Recovery in Privacy-Constrained Real-World Settings
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 17, 07:46 PM

  Fine-tuning directly on user-side data is an effective way to improve the performance of widely adopted data-driven human mesh recovery (HMR) models.

  However, doing so for existing HMR methods often assumes aggregating user-side data from real-world HMR-deployed devices to a central training server, posing a significant privacy risk as sensitive human images are transmitted.

Extractive summary: sentences quoted from the sources.