SIA OpenIR  > 空间自动化技术研究室
面向移动机器人的室内环境物体语义主动获取与增量学习方法研究
Alternative TitleResearch on Active Acquisition and Incremental Learning Methods of Object-level Semantic Information for Mobile robots in Indoor Scene
韩小宁
Department空间自动化技术研究室
Keyword移动机器人 语义信息获取 物体发现 主动物体识别 类别增量学习
Pages106页
Degree Discipline机械电子工程
Degree Name博士
2021-05-20
Degree Grantor中国科学院沈阳自动化研究所
Place of Conferral沈阳
Abstract自世界上第一台机器人诞生以来,让机器人替代人类从事繁琐的各种劳动,便一直存在于人类的幻想之中,激励着人们不断对机器人技术进行探索。人与机器人在室内环境共存是机器人技术未来发展的趋势。感知环境并建立地图是实现人机共存的基础之一。相较于空间地图,语义地图能够赋予室内移动机器人更高的智能,受到了研究人员的广泛关注。获取环境语义信息是构建语义地图的关键步骤,其中,物体的类别信息是环境语义描述的重要内容。目前,因其对人类参与的依赖度低,在不同场景中具有较强的通用性,基于计算机视觉技术的方法,如物体识别、物体检测或语义分割等方法,成为语义获取的主要方法。目前,利用物体识别获取物体语义的研究有两方面的局限:其一,当前研究多关注于利用被动采集的图像进行识别,缺少对于机器人能动性的考虑;其二,计算机视觉中的识别方法多存在闭集约束,无法应对机器人长期作业所面临的开放世界挑战。针对上述两点,本论文对主动物体识别与类别增量学习问题进行研究。同时,在环境中进行物体识别前,需要定位环境中的潜在物体,因此,我们首先对环境中物体发现任务展开研究。对于物体发现任务,本文根据物体的显著性以及物体多放于支撑平面上的特点,提出了基于显著性启发的室内环境物体发现方法。为提升在室内环境中进行平面检测的效率,我们根据逆深度图像中的平面表达形式,提出一种基于区域生长的逆深度图像中的平面检测方法,借助网络局部撒种方法、贪婪生长策略及表面法向信息,实现了从图像中直接进行平面检测的方法。基于上述平面检测方法,进一步利用环境中物体的显著性信息生成种子,以支撑平面作为几何约束对种子进行筛选,利用区域生长从图像中分割出潜在物体。在实验中,我们利用公开数据集对平面检测算法的检测率、精度及效率进行了评估,并验证了本方法在室内环境中进行物体发现的可行性。受一阶段物体检测方法启发,本文探索了利用深度学习方法进行物体发现的可能性。物体检测任务与物体发现任务具有一定的相似性,都需要对物体进行定位。我们设计了一个用于物体发现任务的神经网络结构。该网络输入为RGBD图像,以充分利用深度图像中的空间信息。网络同时输出物体包络框信息及物体性度量。并针对物体发现任务设计了相应的误差函数,利用权重来平衡物体性度量及定位两个子任务间的误差。通过不同数据集上的实验,证明了利用深度图像作为补充输入可提升该物体发现方法的表现,同时也验证了该方法在简单场景进行物体发现的可行性。针对主动物体识别问题,本文将其建模为马尔科夫决策过程。通过将深度Q学习网络应用于主动识别任务,以机器人拍摄的图像及目标物体的包络框作为当前机器人的状态表达,并根据识别结果设计了相应的奖励函数,实现了基于深度Q学习的单步动作决策方法。针对移动机器人在主动物体识别中的动作控制需求,进一步提出了基于深度Q学习网络的多步动作决策方法。将动作分解为动作种类及动作幅度,相应地,在深度Q学习网络中引入决斗结构,以同时预测动作种类及动作幅度,从而实现了多步动作决策。最后,在主动视觉数据集中实验,验证了利用深度Q学习网络进行主动物体方法的可行性。通过与单步动作决策对比,证明了使用多步动作决策方法能进一步提升识别效率。面对机器人长期作业面临的开放世界挑战,类别增量学习方法是潜在的解决方案。在类别增量学习中,可能面临的问题包括:新旧样本间的不平衡,以及增量学习可能导致的灾难性遗忘。基于类别表达的类别增量学习方法使用预训练的神经网络作为特征提取器,避免了重新训练网络,从而一定程度上抑制了遗忘旧类别。本文进一步将特征分布考虑到类别表达中,通过正态化检验,发现每个类别的绝大多数特征可以接受其服从正态分布的假设。本文提出利用正态分布拟合特征的类条件概率分布,利用样本均值及方差表征新类别,并使用高斯朴素贝叶斯分类器实现查询样本分类,从而实现类别增量学习。本方法对于每个新类别仅存储均值及方差,所需存储空间随类别数据线性增长,显著地减少了对于存储空间的需求。在实验中,与典型的几种类别增量方法作对比,本文提出的方法取得了最好的分类结果,证明了本方法进行类别增量学习的有效性。通过消融实验,进一步分析了本方法对每类样本数量的需求,发现本方法需要一定数量的样本才能获取较好的分类结果。
Other AbstractSince the emergence of the first robot in the world, there is an ideal to replace human laborers with robots in tedious tasks, which has been inspiring human beings to explore robotics. In the near future, it will be possible for robots to co-exist with humans in indoor scenes. Sensing and mapping is a fundamental requirement for a robot to co-exist with human beings. Building semantic maps endow a mobile robot with more intelligence, and it has been attracting the general interests of researchers for many years. Acquisition of semantic information of environment is key processing of building semantic maps, and the category of objects is the main content of the semantic description of scenes. Nowadays, object recognition is the main source of semantic information of object categories. However, there are two limitations lay in current works, the first is that most of the current research focuses on extraction category information of objects from images captured passively, which do not take mobility of robots into consideration. The second is there is a close-set assumption in object recognition in computer science, it hinders a robot to handle the open-world challenges in long-term automation. To this end, the topic of this dissertation focuses on the active acquisition and incremental learning of semantic information of object categories. Before going further into those two topics, potential objects in scenes should be localized, this problem is known as object discovery. For object discovery, based on structured planes and saliency of objects in indoor scenes, an object discovery method has been proposed based on saliency. To improve the efficiency of plane extraction in structured scenes, based on plane formation in inverse depth images, planes are extracted from inverse depth images directly. With the employment of grid-local seeding policy, greedy-growing strategy, and local surface normal, planes can be extracted from images with precise boundaries. To discover objects, seeds are selected based on saliency information and filtered with geometric constraints of support planes, and the regions of objects are segmented based on region growth. The experiment results on public accessible datasets have shown the efficiency and precision of the plane extraction method and the effectiveness of the proposed object discovery method. Inspired by impressive one-stage object detection methods, we explore the potential to apply deep learning in object discovery. There is some similarity between object detection and object discovery. We design a network for object discovery. The network takes RGBD images as input, which employs spatial information in depth images. In the output of the network, elements of bounding boxes and the objectness of each object are predicted. Through experiments on a different dataset, it shows that depth images as complementary input can improve the performance of object discovery, and the generality of the method has been evaluated. For active object recognition, we formulated it asa Markov action decision process. Deep Q-learning network has been applied in active object recognition in this dissertation. The network takes images captured by robot and bounding box of the target object, which is state of the environment. The reward function is designed carefully for the recognition task. To further improve the exploration efficiency in action space, an action is decoupled into action type and action range. Dueling architecture is introduced into DQN accordingly, which enables it predicts action type and action range simultaneously, thus multi-step action decision has been realized. At last, compared with several baseline methods, it is proven that the advantages of the proposed method. Besides, with different splits, the feasibility of the proposed method in unknown scenes has been evaluated. Faced with open-world challenge, class incremental learning is a potential solution. The problem in class incremental learning is that catastrophic forgetting in continuous learning. A class incremental learning method is proposed based on Gaussian naïve Bayes. This method is based class representation, which employs pretrained network as feature extractor. It is found that extracted features obeys normal distribution. Thus the features of new unknown classes are counted and represented by normal distribution. In the process of incremental learning, Gaussian naïve Beyes classification is employed. The experiment results on public dataset show that, compared with several state-of-the-art methods, superior performance has been achieved by the proposed method. Through further ablation study, it is shown that pretrain plays an important role in the proposed method. The proposed method should be applied with dozens of sample to ensure performance.
Language中文
Contribution Rank1
Document Type学位论文
Identifierhttp://ir.sia.cn/handle/173321/29007
Collection空间自动化技术研究室
Affiliation中国科学院沈阳自动化研究所
Recommended Citation
GB/T 7714
韩小宁. 面向移动机器人的室内环境物体语义主动获取与增量学习方法研究[D]. 沈阳. 中国科学院沈阳自动化研究所,2021.
Files in This Item:
File Name/Size DocType Version Access License
面向移动机器人的室内环境物体语义主动获取(4845KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[韩小宁]'s Articles
Baidu academic
Similar articles in Baidu academic
[韩小宁]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[韩小宁]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.