WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention

Key points

• Rotary Positional Embedding (RoPE) is widely used to encode relative positions in Transformers.1
• In multi-view settings, however, patch coordinates are defined independently in each image: the same 3D point can therefore have different coordinates across views, making them insufficient for establishing cross-view correspondences.1
• WorldRoPE integrates the rotary operator over this distribution in closed form, combining rotations at the projected primitive centers according to their mixture weights with covariance-dependent attenuation.1
• Experiments on novel view synthesis, depth estimation, and 3D geometry reconstruction demonstrate the effectiveness of WorldRoPE for multi-view Transformers.1

Sources (1)

• [1]WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 12, 07:56 AM

  We introduce WorldRoPE, a probabilistic world-centric RoPE for multi-view attention.

  Rotary Positional Embedding (RoPE) is widely used to encode relative positions in Transformers.

Extractive summary: sentences quoted from the sources.