Possible to do background replacement with parallax and a camera move?
User achieves convincing parallax-ready background replacement using AI, bypassing complex manual 3D work.
A filmmaker has demonstrated a significant leap in AI-assisted visual effects, using the Nano Banana 2 model to generate a convincing 3D-style background from a simple green screen shot. After manually keying out the green screen, the user prompted the AI to 'inpaint the black background to provide the correct perspective on a continuing cardboard city similar to that of the foreground.' The result was a background plate that automatically matched the style, lighting, texture, and—crucially—the perspective of the live-action foreground, creating a cohesive composite with minimal manual intervention.
The success highlights AI's growing capability to understand and replicate complex 3D scene properties from 2D information, a task traditionally requiring extensive manual modeling or matte painting. However, the user notes a key limitation: applying this technique to a shot with a camera move and true parallax would be far more challenging, as maintaining consistency across frames is difficult for current generative AI. This points to the next frontier: tools that can integrate 3D tracking data to generate consistent 3D plates or layers, a pipeline the user is exploring with ComfyUI and local hardware like an RTX 5080.
- Nano Banana 2 AI generated a stylistically matched 3D background from a text prompt and a keyed green screen shot.
- The AI successfully replicated lighting, texture, and perspective without manual inpainting, surprising the user with its effectiveness.
- The main unsolved challenge is maintaining consistency for shots with camera movement and parallax, pushing development towards 3D-aware AI pipelines.
Why It Matters
This shows AI is moving beyond simple image generation into complex VFX tasks, potentially democratizing high-end post-production.