trunk/8103181932aa7fce98ec3edac0135594d806c89c: [nativert] Add nested tensor deserialization support (#174843)
The fix enables models with nested tensor lists to run properly, fixing a critical 'None' input bug.
Meta's PyTorch team has resolved a significant technical limitation in its NativeRT (nativert) runtime with commit 8103181. The update, titled "Add nested tensor deserialization support," fixes a critical bug where the serialization layer could not process nested tensor structures, causing model inputs to default to None and fail. This enhancement is essential for the PT2 (PyTorch 2) export pipeline and sigmoid interpreter, allowing more complex AI model architectures that rely on multi-level data representations to be serialized and executed correctly. The change was specifically tested with ranking and recommendation models at Meta, indicating its importance for production-scale AI systems.
The technical implementation involved adding a new `NestedTensorList` type to the Graph type system across both fbcode (Meta's internal codebase) and xplat (cross-platform) components. Engineers updated the deserialization logic for the `AS_NESTED_TENSORS` argument type and modified the `PrimKernelRegistry` to map the new type correctly. Key fixes also included updating the `datafm_merge_and_dedup_by_reference` operator with proper validation checks and refining benchmark comparison logic for better debugging. This low-level update, while technical, unblocks the development and deployment of sophisticated AI models that require handling hierarchical data, a common need in domains like recommendation systems and natural language processing where inputs are often nested collections of tensors.
- Adds support for deserializing List[List[Tensor]] structures in PyTorch's NativeRT, fixing a 'None' input bug.
- Enables proper export and execution for models using nested tensor operators via the PT2/sigmoid pipeline.
- Involved adding a new NestedTensorList type to the Graph system and updating core serialization logic.
Why It Matters
Unblocks development of complex AI models with nested data structures, crucial for recommendation systems and advanced ML research.