Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs
Here we present a model that directly co-generates the video and 3D motion for any supplied rig, along with a camera that registers this motion to the scene.
ProofPaper ↗
Key points
- Video generation models can now depict remarkably realistic motion.
- Yet it is unclear how to use this motion beyond pixels: to animate a specific 3D asset and replay the animation in its 3D scene.
- Prior video-motion models either handle only human skeletons, often adding billions of parameters, or triangulate 3D motion from generated 2D pose maps.
- To train across many skeletons, we use a generative video editor to place animated 3D assets in scenes suited to their actions, curating the edits so their known motion remains valid.
Sources (2)
- [1]Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary RigsHugging Face Daily Papers · Oct 20, 12:00 AM
Here we present a model that directly co-generates the video and 3D motion for any supplied rig, along with a camera that registers this motion to the scene.
Video generation models can now depict remarkably realistic motion.
- [2]Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary RigsarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:59 PM · same content
Extractive summary: sentences quoted from the sources.
Before this
- Oct 19, 2026Amazon SageMaker JumpStart: 2026 year-to-date in review
- Oct 19, 2026Push-based KV transfer to improve TTFT and TPOT for disaggregated inference in vLLM
- Oct 19, 2026MiniMax M3.1 Flash shows China's AI price race has no floor in sight
- Oct 19, 2026NVIDIA Cosmos3-H-Dreams: Bringing Clinical Surgery to Life Through Generative Simulation
- Oct 19, 2026From Short Clips to Real-Time Streams: Identity-Aware Supervision and Context-Efficient Generation for Video Face Swapping
- Oct 19, 2026Frechet Forcing: Training Long-Horizon Streaming World Models from Short Clips