Multi-path optimization for cost aggregation in semi-global matching and its hardware implementation with 53% memory reduction

Cost aggregation is a crucial step in the accurate stereo depth estimation process known as semi-global matching. However, this step is challenged by storing large amounts of aggregated data, which is necessary to achieve high matching accuracy under large resolution and large disparity conditions. In this paper, we propose a multi-path optimization aggregation strategy and re-select the complementary combinations of key paths in the forward and backward scanning directions to improve the matching accuracy as much as possible. An error rate of only 5.21% is achieved on the KITTI 2015 dataset. Next, we propose DCT-based truncated compression and selective storage to alleviate the problem of memory increase due to the introduction of reverse critical aggregation paths. Experiments show that the matching error rate increases by only 0.6% on the KITTI 2015 dataset with 53% memory savings. Finally, 1920 × 1080 @62 fps @128 MHz is achieved on ZCU102 FPGA.