TY - JOUR
T1 - 3D Vehicle Detection Using Multi-Level Fusion From Point Clouds and Images
AU - Zhao, Kun
AU - Ma, Lingfei
AU - Meng, Yu
AU - Liu, Li
AU - Wang, Junbo
AU - Junior, Jose Marcato
AU - Goncalves, Wesley Nunes
AU - Li, Jonathan
N1 - Publisher Copyright:
© 2000-2011 IEEE.
PY - 2022/9/1
Y1 - 2022/9/1
N2 - 3D vehicle detectors based on point clouds generally have higher detection performance than detectors based on multi-sensors. However, with the lack of texture information, point-based methods get many missing detection of occluded and distant vehicles, and false detection with high-confidence of similarly shaped objects, which is a potential threat to traffic safety. Therefore, in the long run, fusion-based methods have more potential. This paper presents a multi-level fusion network for 3D vehicle detection from point clouds and images. The fusion network includes three stages: data-level fusion of point clouds and images, feature-level fusion of voxel and Bird's Eye View (BEV) in the point cloud branch, and feature-level fusion of point clouds and images. Besides, a novel coarse-fine detection header is proposed, which simulates the two-stage detectors, generating coarse proposals on the encoder, and refining them on the decoder. Extensive experiments show that the proposed network has better detection performance on occluded and distant vehicles, and reduces the false detection of similarly shaped objects, proving its superiority over some state-of-the-art detectors on the challenging KITTI benchmark. Ablation studies have also demonstrated the effectiveness of each designed module.
AB - 3D vehicle detectors based on point clouds generally have higher detection performance than detectors based on multi-sensors. However, with the lack of texture information, point-based methods get many missing detection of occluded and distant vehicles, and false detection with high-confidence of similarly shaped objects, which is a potential threat to traffic safety. Therefore, in the long run, fusion-based methods have more potential. This paper presents a multi-level fusion network for 3D vehicle detection from point clouds and images. The fusion network includes three stages: data-level fusion of point clouds and images, feature-level fusion of voxel and Bird's Eye View (BEV) in the point cloud branch, and feature-level fusion of point clouds and images. Besides, a novel coarse-fine detection header is proposed, which simulates the two-stage detectors, generating coarse proposals on the encoder, and refining them on the decoder. Extensive experiments show that the proposed network has better detection performance on occluded and distant vehicles, and reduces the false detection of similarly shaped objects, proving its superiority over some state-of-the-art detectors on the challenging KITTI benchmark. Ablation studies have also demonstrated the effectiveness of each designed module.
KW - 3D vehicle detection
KW - autonomous driving
KW - data fusion
KW - deep learning
KW - false detection
KW - point cloud processing
UR - https://www.scopus.com/pages/publications/85123356538
U2 - 10.1109/TITS.2021.3137392
DO - 10.1109/TITS.2021.3137392
M3 - 文章
AN - SCOPUS:85123356538
SN - 1524-9050
VL - 23
SP - 15146
EP - 15154
JO - IEEE Transactions on Intelligent Transportation Systems
JF - IEEE Transactions on Intelligent Transportation Systems
IS - 9
ER -