Robotic Grasping Detection Based on Continual Learning Using Perceptual Loss and Multibranch Deep Fusion

Vision-based grasping detection is extensively utilized in the field of production and manufacturing, leveraging multisource visual data to generate feature maps and achieve robust autonomous grasps. However, significant challenges remain in effectively integrating multisource visual inputs and overcoming catastrophic forgetting in scenarios that vary with time. To address these issues, this article proposes: 1) a three-branch RGB-D fusion module for cross-modal feature synthesis, integrated into the GR-ConvNet framework to optimize antipodal grasping detection; 2) a composite distillation strategy combining perceptual loss with smooth L1 loss to stabilize knowledge retention across sequential tasks; and 3) a robotic grasping detection system driven by RGB-D sensor integration to facilitate autonomous grasping of objects with diverse shapes. Comprehensive evaluations demonstrate state-of-the-art performance of our methods: 98.9% grasping detection accuracy on the Cornell dataset, 89.12% mean grasp accuracy on the final continual learning task, and 82% grasp success rate in real-world robotic trials. Moreover, ablation experiments conducted on our proposed model and the corresponding continual learning approach demonstrate the effectiveness of the three-branch deep fusion (3-BDF) module and the combined distillation loss. To our knowledge, this is the first application of a perceptual loss approach in RGB-D sensor-driven grasping detection tasks designed for continuously changing scenarios. Code and Video are available at: https://github.com/lyxhnu/Cornell-CL