A camera-LiDAR Fusion Fully Convolutional Neural Network (CLFCN) for Semantic Segmentation on iseAuto Dataset

The project aims to provide a demonstration of the iseAuto training dataset. A fully convolutional neural network was developed to conduct the late fusion of the camera and LiDAR data. The project focuses on the domain adaptation analysis of the network. The popular Waymo open dataset was used in the experiments for baseline model training. The transfer learning and semi-supervised learning techniques were used in the experiments to combine the knowledge from Waymo open dataset and iseAuto training dataset.

The video above visualize the semantic segmentation results of the iseAuto training dataset. The iseAuto training dataset was divided into four subcategories based on the weather and illumination conditions, namely, light-dry, light-wet, dark-dry, dark-wet. Two classes, vehicle and human, were labeled and predicted in the dataset. One might can notice that the dark and wet scenarios in the iseAuto training dataset is very challenging compared with other popular open training dataset. Please refer to the paper for more details.