PyTorch's Dynamo fix avoids graph breaks for native bmm router operations

PyTorch's latest pull request ( #186388 ) addresses a critical performance issue in its Dynamo JIT compiler. The problem occurred when Dynamo attempted to trace the native eager router for aten overrides—specifically for batched matrix multiply (bmm)—under certain conditions. When a torch.device context was active, Dynamo would inline DeviceContext.__torch_function__ and trace into the native bmm eager router. At trace time, the router evaluated eager-only code such as torch._C._is_cow_tensor and relied on the boxed fallback path, causing graph breaks before Inductor could use the normal aten path. These graph breaks hurt compilation performance and prevented PyTorch from fully optimizing the computation graph.

To solve this, the PyTorch team introduced a narrow check using torch.compiler.is_dynamo_compiling() to detect when Dynamo is actively tracing the Python router. In that specific case, the fix emits the resolved aten overload instead of triggering the eager-only predicates and boxed fallback. This approach is deliberately narrower than using torch.compiler.is_compiling(), which could recurse back into the same backend router during real eager execution. The solution ensures that ordinary eager execution still uses the existing predicate and captured boxed fallback path, while Dynamo tracing gets a clean aten path. The fix was validated with multiple tests including test_compile_session_flag_falls_through_without_recursion and test_device_context_matmul_avoids_native_bmm_router_graph_break. Benchmark results on CUDA with shape (64,8,1)x(64,1,128) show a total time of 0.025610 s for 1000 iterations, or 25.610 us per call—a significant improvement for bmm-heavy workloads.