3D General Representations: Discussions on how to learn from multi-dataset to obtain the general representations
Date:
In this talk, I will give a brief infroduction about how to enable the 3D model to learn from different domains with distinct domain differences.
Current 3D perception model follows a single dataset-specific training and testing paradigm, which often faces a serious detection accuracy drop when they are directly deployed in another dataset and mainly faces two main challenges:
- 1) Significant gaps in different 3D perceptual datasets: The existing 3D perceptual datasets are collected by different manufacturers using inconsistent types of sensors, having strong dataset- and taxonomy-level distribution differences;
- 2) Weak generalization ability of 3D perceptual models: When perceptual models are trained on one dataset (domain) and directly deployed to another dataset, they typically face more severe perceptual performance degradation.
To this end, this talk is mainly focused on the above-mentioned challenges faced by the current single dataset-specific training and testing paradigm for 3D perceptual models. Based on our preliminary exploration and investigation experience, we will briefly introduce two papers published by CVPR-2023: Uni3D and Bi3D, and deeply analyze how to achieve open-scene applications of 3D perceptual models at a lower cost.