Fine-Grained Open-Set Object Detection with Discrepancy Enhancement and Detector-Driven Clustering

Object detection is a foundational task in computer vision and robot perception. Recently, open-set detection has been widely investigated for unseen object discovery. However, many of the existing methods suffer from a deficient recall rate of unknown objects and confusion of known-unknown representation. In this paper, we propose an open-set detector for these issues. First, we integrate the cues of semantics and localization into the region proposal phrase, which significantly improves the awareness of unknown objects. Secondly, we propose a discrepancy enhancement module during training from the perspective of similarity, energy, and classification. This method effectively suppresses the confusion of known-and unknown-class representation, leading to better open-set detection performance. Finally, to subdivide a single unknown class into an adaptive number of novel classes, we develop a detector-driven clustering method, where the well-learned detector features can be leveraged for fine-grained novel-class clustering. Extensive experiments are conducted on VOC and COCO benchmarks, validating the effectiveness of our proposed approaches.