2D Region-of-interest-based 9D object pose detection using RGB-D data for robotic preview tracking control

Abstract
Due to visual delay, the selection of an appropriate look-ahead point plays a crucial role in the tracking performance of nursing robots. However, accurate computation of the look-ahead point remains challenging, particularly when small or distant objects are difficult to detect reliably. To address these challenges, this paper proposes a 2D region-of-interest (ROI)-based 9D object pose detection framework using RGB-D data for robotic preview tracking control. First, an Adaptively Spatial Channel-Coordinate Driven Feature Fusion (ASCCFF) network is introduced to replace the conventional FPN in the neck of the NR-CNN detector. By resizing multi-level features to a unified resolution and adaptively fusing spatial, channel, and coordinate-dimensional information, the proposed ASCCFF network overcomes the limitations of traditional single-dimension feature fusion strategies, significantly enhancing 2D ROI detection accuracy for small and distant objects. Second, a 2D ROI-based texture enhancement mechanism is proposed to amplify keypoint features within detected 2D regions of interest. This mechanism improves keypoint extraction and direct 2D-3D feature matching, thereby enhancing 9D pose estimation performance for distant, small-scale, and low-textured objects in complex scenes. Third, an ID assignment algorithm is employed to track object identities across sequential RGB-D frames captured by an Intel RealSense D435i sensor. This enables robust estimation of objects’ 9D movement posture information, including 3D position, orientation, velocity, acceleration, and trajectory prediction, which is subsequently used to drive a 9D movement posture-driven preview tracking control strategy that explicitly considers social distance constraints. Extensive experimental results demonstrate that the proposed ASCCFF network substantially improves NRCNN performance in detecting small and distant objects. Moreover, the enhanced perception capability leads to superior downstream performance in 9D object pose detection and preview tracking control for single-target robotic tracking scenarios, validating the effectiveness and robustness of the proposed framework.