Sequential Viewpoint Selection and Grasping with Partial Observability Reinforcement Learning

Despite the success of vision-based object grasping due to deep learning development, fixed-view object grasping methods still face information loss with limited performance. Recently some rule-based or heuristic-based methods have begun to sequentially consider multiple views to improve the perceptibility of the environment, which shows better performance. However, their sequence lengths are too short, or their viewpoint selection is myopic and ignores the long-term effect. This paper models sequential viewpoints selection as a Markov Decision Process. The Sequential Decided Multi-View Grasping (SDMVG) method is proposed based on reinforcement learning, and an RNN-based policy is introduced. Considering long-term return, SDMVG can generate viewpoints sequence which achieves most information gain. Numerical experiments show SDMVG can achieve 10% accuracy improvement compared with rule-or heuristic-based baselines on Multi-View GraspNet Benchmark. Moreover, SDMVG approaches the global optimum with only 1/40 wall time compared with the brute-force method.