广州中国科学院沈阳自动化研究所分所知识库

热像仪、RGB相机、惯性测量单元(inertial measurement unit， IMU)在图像模态、感知精度和响应速度方面具有异质和互补的特性，在机器人和计算机视觉领域受到越来越多的关注。在热像仪、RGB相机和IMU传感器之间进行准确的标定是实现三者有效融合必不可少的过程。为此，本文提出了一种空间联合标定方法，将热像仪、RGB相机和IMU坐标系之间的空间关系关联在一起。为了保证联合标定的一致性，在棋盘格的角点处分别固定尺寸相同、直径约5mm的LED阵列，作为热像仪的标定板。在提取热像仪标定图像光源中心环节，本文采用混合高斯模型来提取标定位置信息，以此来保证LED光源部分缺失情况下鲁棒的位置估计。再建立目标优化函数，优化热像仪图像中提取的热源中心重投影误差、RGB图像中提取的棋盘角角点重投影误差、加速度和角速度误差来联合估计外参。实验环节，在搭建的热像仪-RGB相机-IMU多模态感知平台，采集了大量的实验数据，对所收集的热像仪-RGB相机-IMU测量结果进行了深入分析，可靠且准确的空间标定结果验证了所提方法的有效性。

Thermal imager, RGB camera, and inertial measurement unit (IMU) are heterogeneous and complementary in terms of image modality, sensing accuracy and response speed. They have attracted increasing attention in the fields of robotics and computer vision. To effectively integrate the thermal imager, RGB camera and IMU, the accurate spatial calibration among sensors is an indispensable process. In this study, a joint spatial calibration method is proposed to correlate the spatial relationship among thermal imager, RGB camera and IMU. To ensure the consistency of the joint calibration, LED bulb arrays with the same size and about 5mm diameter are fixed at the corner points of the checkerboard, which are used as the calibration board of thermal imager. In the process of extracting the center of the image light source for the calibration of the thermal imager, this paper uses the Gaussian mixture model to achieve calibration position information, which can ensure robust position estimation in the case of partial missing LED light sources. The extrinsic parameters are estimated by jointly optimizing both the chessboard corner reprojection errors and acceleration & angular velocity error terms. In the experiment, a large amount of experimental data are collected through the built-up thermal imager-RGB camera-IMU sensing platform. The collected thermal imager-RGB camera-IMU measurement results are analyzed in depth. Reliable and accurate spatial calibration results verify the effectiveness of the proposed method.