ReFine3D boosts 3D vision-language models with regularized fine-tuning

Domain adaptation remains a critical challenge in 3D vision, especially for multimodal foundation models aligning point clouds with visual and textual data. Fine-tuning on limited downstream data often causes overfitting and catastrophic forgetting. To address this, researchers Sneha Paul, Zachary Patterson, and Nizar Bouguila propose ReFine3D, a regularized fine-tuning framework designed for domain-generalizable adaptation of 3D large multimodal models (LMMs). The framework selectively tunes only key layers while applying two targeted regularization strategies: multi-view consistency across augmented point clouds, and text diversity through synonym-based prompts generated by large language models. Additionally, ReFine3D incorporates point-rendered vision supervision and a test-time augmentation mechanism with confidence-based aggregation to further boost robustness.

Extensive experiments across 3D domain generalization benchmarks show consistent gains. ReFine3D improves base-to-novel class generalization by 1.36%, cross-dataset transfer by 2.43%, robustness to corruption by 1.80%, and few-shot accuracy by up to 3.11%, outperforming prior state-of-the-art methods. All these gains come with minimal added computational overhead, making the approach practical for real-world deployment. The work has been accepted at Transactions on Machine Learning Research (TMLR) and represents a significant step toward making 3D vision-language models more adaptable and reliable across diverse environments.