RIG-RoPE fixes multimodal position encoding with zero added parameters

Rotary positional encoding (RoPE) is a core component of modern language models, and its multimodal variant M-RoPE splits position channels into temporal, height, and width subspaces. But static multidimensional position assignment breaks down in interleaved multimodal contexts. Spatial rotations get applied to token pairs whose displacement isn't a well-defined geometric object, causing cross-modal and inter-instance interference. Meanwhile, temporal coordinates treat text, images, and video as equal-step counters, ignoring drastically different information densities.

Donggen Li's RIG-RoPE tackles both issues. Each token gets a modality indicator, a visual instance identifier, and a scalar information-duration coordinate. Height/width rotations only fire for query-key pairs within the same visual instance; unknown displacements are marginalized instead of zeroed. Temporal rotations use interpolated cumulative block durations: text consumes unit duration, images use a dimension-aware logarithmic spatial scale, and videos add logarithmic temporal extension over effective frames. The paper provides a gauge-invariance argument for avoiding cross-instance rotation, an impossibility result for static IDs under shared H/W subspaces, and a duration-consistency argument against equal-step multimodal time. Crucially, RIG-RoPE adds no learned parameters and works inside tiled attention kernels with constant metadata overhead. The 15-page report (arXiv:2608.05154) is preliminary, establishing formulation and validation paths without claiming empirical superiority—so expect benchmarks and open-source implementations to follow.