SolarRenderer: Few-Step Generative Reference Rendering

Key points

• Game engines and coding agents can specify the layout, motion, and camera of a scene, yet turning such coarse structure into realistic video requires a renderer that follows the structure and completes the appearance it leaves open.1
• Structural videos enter the native reference pathway of the backbone at the positions of the frames they describe, without a type identifier and described only in plain text, so that a single control LoRA follows arbitrary structural representations.1
• A block-causal variant extends the same design to streaming and renders 768p video in real time.1
• Experiments on an in-distribution test set and a generalization benchmark of 16 structural representations show that SolarRenderer achieves state-of-the-art results across both, and generalizes to 11 unseen representations, including pose skeletons, drawings, and point clouds.1

Sources (1)

• [1]SolarRenderer: Few-Step Generative Reference Rendering
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 20, 05:59 PM

  We present SolarRenderer, a structure-conditioned video renderer that adapts a frozen MiniMax-H3 33B backbone with two lightweight LoRAs.

  Game engines and coding agents can specify the layout, motion, and camera of a scene, yet turning such coarse structure into realistic video requires a renderer that follows the structure and completes the appearance it leaves open.

Extractive summary: sentences quoted from the sources.