SIA OpenIR  > 水下机器人研究室
基于自主学习的水空跨域海洋机器人运动控制方法研究
Alternative TitleSelf-learning Motion Control for Water-air Cross-domain Marine Robot
霍雨佳1,2
Department水下机器人研究室
Thesis Advisor封锡盛 ; 李一平
Keyword水下机器人 倾转旋翼机器人 自主学习 高斯过程回归
Pages114页
Degree Discipline机械电子工程
Degree Name博士
2020-08-28
Degree Grantor中国科学院沈阳自动化研究所
Place of Conferral沈阳
Abstract水空跨域海洋机器人作为一种新型机器人,结合了无人机的高机动性和水下机器人的隐蔽性,极大地扩展单一的一种机器人的工作领域。这种融合了多种无人系统特点的,由单一平台组成并实现工作的无人系统,避免了空中、水面和水下协同工作带来的任务复杂性和低可靠性,代表了海上无人系统的发展趋势。本文以一种新型水空跨域海洋机器人为平台,该平台借鉴了倾转旋翼机器人的结构特点,具备水下航行、水面垂直起飞和空中固定翼飞行等工作模式。跨域海洋机器人具有多种介质下的工作环境,其结构根据不同的任务需求改变。同时,跨域机器人也受到未知环境下扰动的影响,因此预先建立一套完备精确的机器人控制模型是困难的,这些机器人系统的模型不确定性也给跨域海洋机器人的运动控制精度提出了挑战。因此需要研究智能化的控制方法,解决跨域机器人在未知环境下的复杂系统的控制问题。为此,本文建立了跨域海洋机器人系统模型,搭建一套跨域海洋机器人的训练仿真平台,提出了基于强化学习的机器人运动控制的训练学习方法,提出了先进的模型学习方法,并且基于模型学习方法研究了跨域海洋机器人的运动控制问题。本文研究内容包括如下4个方面。(1)跨域海洋机器人运动建模与仿真环境。基于跨域机器人的倾转旋翼特性,分析了跨域倾转机器人在各工作介质下的受力,特别的,分析了跨域机器人在水空跨域过程中机体的受力问题,建立了结合倾转旋翼和升力体机翼等结合的动力学模型。并以此为基础,搭建了基于ROS和Gazebo的机器人动力仿真训练平台,为下一步跨域海洋机器人的运动控制仿真训练提供了基础。(2)基于强化学习的跨域海洋机器人运动控制方法研究。为解决机器人控制系统因先验知识的不完备影响运动控制性能,采用利用深度强化学习方法,通过机器与环境间的交互训练,对跨域机器人在空气中以四旋翼模式、固定翼模式和水中以水下机器人模式工作的位置控制和连续路径跟踪问题进行研究。同时,对机器人以倾转旋翼过渡模式下的姿态稳定和高度稳定进行了仿真研究。在此基础上,利用循环神经网络对原有的深度强化学习方法进行改进,解决了实际问题中因机器人传感器受限等问题造成状态部分可观问题。仿真表明,该方法可以在对机器人控制模型缺乏准确认知的情况下,利用机器人与环境的不断交互训练,实现跨域机器人的无模型运动控制问题。(3)基于高斯过程的模型学习方法研究。为降低机器人在未知工作环境下的模型不确定性,提高机器人对环境的扰动的适应性,利用跨域机器人在未知环境下运动状态的观测数据,研究了具有非参数化的高斯过程回归模型。该方法能够降低噪声对观测数据的影响,进而实现系统模型的非参数化高精度辨识。为提高高斯过程在数据量较大情况下的计算速度,建立了基于局部高斯过程回归的模型学习方法。根据局部学习理论,针对复杂系统的模型学习,该局部高斯过程的模型学习方法具有学习速速快、辨识精度高和鲁棒性强的特点。(4)基于模型学习的跨域海洋机器人运动控制方法研究。基于无模型强化学习的运动控制训练是非常耗时费力的,此外,在机器人与环境的交互训练过程中,跨域机器人的安全性也无法得到保障,这使得基于无模型的强化学习算法在运用到实际的机器训练变得十分困难。采用基于Dyna框架的基于模型学习的运动控制训练方法,使得机器人通过与环境交互获取实际经验的同时,这些数据会被机器人利用模型学习方法得到环境的估计模型,在机器人作业的同时,该模型会运行产生虚拟样本用来更新控制策略。这样,机器人不是通过直接的与真实环境进行交互获得控制策略的知识,而是在后台,与在线构建的虚拟环境模型进行交互,训练得到当前环境下的控制策略。该方法能够加快收敛速度,迅速找到最优解的同时,保证了机器人在实际工作环境下的安全性。并通过仿真获得验证,实现跨域倾转机器人在环境扰动下由水和空气间的跨域运动、水面垂直起降、倾转过渡和固定翼飞行的全过程工作仿真,证明了该方法的有效性。
Other AbstractAs a new type of robot, the water-air cross-domain marine robot combines the high mobility of the UAV with the stealthiness of the underwater robot, which greatly expands the working field of a single robot. This unmanned marine system, combining varieties characteristics of unmanned systems, composed of a single platform, represents the development trend of unmanned marine systems. In this paper, a new type of water-air cross-domain marine robot with similar structural characteristics of the tilt-rotor robot is studied, which has the working modes of underwater navigation, vertical taking-off and landing at the water surface, fixed-wing flight in the air. The cross-domain marine robot has working environments under varieties of media, and its structure changes according to different mission needs. At the same time, the cross-domain robot is also affected by disturbances in unknown environments, so it is difficult to establish a complete and accurate robot control model in advance, and the model uncertainty of these robot systems also challenges the motion control accuracy of the cross-domain marine robot. Therefore, it is necessary to study intelligent control methods to solve the control problems of complex systems of cross-domain robots in unknown environments. To this end, this paper establishes a cross-domain marine robot system model, builds a training simulation platform for cross-domain marine robots, puts forward a training and learning method based on reinforcement learning robot motion control, puts forward an advanced model learning method, and studies the motion control problem of the cross-domain marine robot with the model learning method. The research content of this paper includes the following four sections. (1) Cross-domain marine robot motion modeling and simulation environment Based on the characteristics of tilt-rotor cross-domain robots, the force of cross-domain tilt robots under various working media is analyzed, in particular, the force of the bodies in the process of water-air cross-domain. A dynamic model of combining tilt-rotors and the aerodynamics of wings is established. Based on this, a robot dynamic simulation training platform based on ROS and Gazebo is built, which provides a basis for the next step of motion control simulation training for cross-domain marine robots. (2) Study on the motion control method of the cross-domain marine robot based on reinforcement learning To solve the problem that the robot control system affects the motion control performance due to the incomplete knowledge of prior knowledge, the position control and continuous path tracking of the cross-domain robot in the air with quad-rotor mode, fixed-wing mode, and underwater robot mode are studied by using the deep-reinforcement learning method with the interactive training between the robot and the environment. At the same time, the attitude stability and height stability of the robot in the transition mode of the tilt-rotor are simulated. On this basis, the traditional deep-reinforcement learning method is improved by using the current neural network, which solves some Partially Observable problems caused by the restriction of robot sensors in physical problems. The simulation shows that the method can solve the problem of model-free motion control of the cross-domain robot with continuous interactive training between robot and environment without an accurate robot control model. (3) Study of model learning methods based on Gaussian processes To reduce the model uncertainty of the robot in the unknown working environment and improve the adaptability of the robot to the disturbance of the environment, the non-parametric Gaussian process regression model is studied with the observation data of the motion state of the cross-domain robot in the unknown environment. This method can reduce the influence of noise on the observation data, and then realize the non-parameter high precise identification of the system model. To improve the computational speed of the Gaussian process in the case of a large amount of data, a model learning method based on local Gaussian process regression is established. According to the theory of local learning, the model learning method of the local Gaussian process has the characteristics of fast learning speed, high recognition accuracy, and robustness. (4) Study on the motion control method of the cross-domain marine robot based on model learning method The physical control training based on model-free reinforcement learning is very time-consuming and labor-intensive, and the safety of the cross-domain robot can not be guaranteed during the interactive training between robot and environment, which makes it very difficult to apply model-free reinforcement learning algorithm to physical training. Using the model-based learning motion control training method based on Dyna framework, the robot can gain practical experience by interacting with the environment, while the data will be used by the robot to obtain the estimated model of the environment using the model learning method, and the model will run a virtual sample to update the control policy while the robot is working. In this way, instead of directly interacting with the real environment to gain knowledge of the control policy, the robot interacts with the virtual environment model built online in the background and trains to obtain the control policy in the current environment. This method can speed up the convergence speed and find the optimal solution quickly, at the same time, ensure the safety of the robot in the physical working environment. This method guarantees the work of the whole process of cross-domain tilt-robot working mode, underwater mode cross-domain mode, vertical taking-off and landing mode of water surface, tilt transition mode, and fixed-wing flight mode under environmental disturbance.
Language中文
Contribution Rank1
Document Type学位论文
Identifierhttp://ir.sia.cn/handle/173321/27985
Collection水下机器人研究室
Affiliation1.中国科学院沈阳自动化研究所
2.中国科学院大学
Recommended Citation
GB/T 7714
霍雨佳. 基于自主学习的水空跨域海洋机器人运动控制方法研究[D]. 沈阳. 中国科学院沈阳自动化研究所,2020.
Files in This Item:
File Name/Size DocType Version Access License
基于自主学习的水空跨域海洋机器人运动控制(5651KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[霍雨佳]'s Articles
Baidu academic
Similar articles in Baidu academic
[霍雨佳]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[霍雨佳]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.