SIA OpenIR  > 机器人学研究室
基于深度学习的图像恢复算法研究
Alternative TitleResearch on Image Restoration Algorithms Based on Deep Learning
王强1,2
Department机器人学研究室
Thesis Advisor唐延东
Keyword卷积神经网络 图像恢复 卷积循环网络 生成对抗网络
Pages108页
Degree Discipline模式识别与智能系统
Degree Name博士
2019-11-20
Degree Grantor中国科学院沈阳自动化研究所
Place of Conferral沈阳
Abstract本文从深度学习的机制出发,对退化图像进行建模,提出了有效的图像恢复 模型与算法完成对图像损失区域的检测和修复。大量试验验证了所提出模型算 法在图像结构和细节恢复方面的有效性。通过与目前主流算法的试验比较,本文 提出的模型和算法在性能上具有较为明显的优势。本文的工作主要包括以下几个方面:(1) 基于卷积对抗网络的图像恢复模型 提出了基于卷积对抗网络的图像恢复模型,使用金字塔结构的生成网络实现了图像恢复。该模型将图像恢复过程分为两个阶段,分别为结构恢复和细节恢复。通过使用金字塔式的逐步学习过程,生成出损失区域的结构信息;通过使用 改进的残差学习网络完成对图像高频细节的丰富。在训练迭代过程中,使用多重 损失函数约束生成过程来优化模型。基于卷积对抗网络的图像恢复模型使用联 合损失函数,兼顾图像结构和细节的损失,生成出与原始图像最为接近的内容信 息,使生成的图像真实、逼真。(2) 基于上下文的对抗生成网络的图像修复模型 提出了基于上下文的对抗生成网络模型,主动学习不同尺度下的图像特征,从不同的视角得到图像的内部特征;通过整合网络将不同视角的图像特征联合成一个图像表达,完成图像的重建。该模型充分考虑了图像的高层语义的恢复和图像高频细节的恢复,利用多层语义特征分别完成图像结构和细节的生成。(3) 多任务下的图像修复模型 提出了一种多任务下的图像恢复模型,可以同时完成图像中损失区域的检测和修复任务。模型首先学习图像的多层语义特征;通过对多层语义特征的分析,分别设计不同的任务支路,一条支路称为检测网络,主要负责图像中损失区域的 检测;另一条支路为修复网络,结合检测网络的输出,完成对图像损失区域的修 复;最后使用两个判定网络,分别从全局和局部对图像的真实性进行判定,保证 生成图像和周围信息的连续性和一致性。(4) 基于循环卷积网络的图像修复模型 提出了一种基于循环卷积网络修复模型和相应的图像修复算法。该模型使用循环卷积网络学习图像的内部规则,将特征学习的任务从卷积网络中分离出来。
Other AbstractBased on the deep learning, in this dissertation, we modeled the corrupted image with convolutional neural networks。Several effective image completion models are proposed to predict and restore the missing region in an image. The experimental results of these models and algorithms show that the proposed models can achieve more realistic and natural restoration results, and the proposed models are superior to the state-of-the-art algorithms. The main work of this paper includes the following aspects: (1) Laplacian Pyramid Adversarial Network for Image Inpainting. Recently, generative adversarial networks (GANs) have demonstrated high- quality reconstruction in image completion. There is still much room for improvement over the conventional GAN models that do not explicitly address the texture details problem. A Laplacian-pyramid-based generative framework for image com- pletion is proposed. This framework can produce more realistic results by deriving precise content information of missing face regions in a coarse-to-fine fashion and propagating the high-frequency details from the surrounding area via a modified residual learning model. Furthermore, a multiloss function is proposed to super- vise the generative process. To optimize the model, we train the entire generative model with deep supervision using a joint reconstruction loss, which ensures that the generated image is as realistic as the original. (2) Deeply Supervised Image Completion with Multi-context Generative Adversarial Network A unified model by introducing multi-context structures within generative adversarial networks is proposed. This model, named as Multi-Context Generative Adversarial Networks (MCGAN), can automatically learn the hierarchical appearances of a corrupted image and predicted the missing regions from different perspectives. In this model, semantic understanding and high-frequency details are both taken into account and modeled with two parallel networks respectively. While one learns the semantic understanding of the input face image at a high level, the other extracts low-level features for high-frequency details prediction. (3) A Deep Multi-task Generative Adversarial Network for Face Completion We present a Deep Multi-task Generative Adversarial Network (DMGAN) for simultaneous missing region detection and completion in face imagery tasks. The model first learns rich hierarchical representations, which are critical for missing region detection and completion, automatically. With these hierarchical representations, we then design two complementary sub-networks: DetectionNet and CompletionNet. DetectionNet is built upon a fully convolutional neural net and detects the location and geometry information of the missing region in a coarse- to-fine manner. CompletionNet is designed with a skip connection architecture and predicts the missing region with multi-scale and multi-level features. Additionally, we train two context discriminators to ensure the consistency of the generated image. (4) Recurrent Generative Adversarial Network for Image Completion. Most recently-proposed image completion algorithms use high-level features extracted from convolutional neural networks (CNNs) to recover semantic texture content. Although the completed face is natural-looking, the synthesized content still lacks lots of high-frequency details, since the high-level features cannot supply sufficient spatial information for detail recovery. To tackle this limitation, A Recurrent Generative Adversarial Network (RGAN) for image completion is pro- posed. Unlike the state-of-the-art algorithms, RGAN can take full advantage of multi-level features, and further provide advanced representations from multiple perspectives, which can well restore spatial information and details in face completion. Specifically, our RGAN model is composed of a CompletionNet and a DisctiminationNet, where the CompletionNet consists of two deep CNNs and a recurrent neural network (RNN). The first deep CNN is presented to learn the internal regulations of a masked image and represent it with multi-level features. The RNN model then exploits the relationships among the multi-level features and transfers these features in another domain, which can be used to complete the face image. Benefiting from bidirectional short links, another CNN is used to fuse multi-level features transferred from RNN and reconstruct the images in different scales.
Language中文
Contribution Rank1
Document Type学位论文
Identifierhttp://ir.sia.cn/handle/173321/25931
Collection机器人学研究室
Affiliation1.中国科学院沈阳自动化研究所;
2.中国科学院大学
Recommended Citation
GB/T 7714
王强. 基于深度学习的图像恢复算法研究[D]. 沈阳. 中国科学院沈阳自动化研究所,2019.
Files in This Item:
File Name/Size DocType Version Access License
基于深度学习的图像恢复算法研究.pdf(11126KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[王强]'s Articles
Baidu academic
Similar articles in Baidu academic
[王强]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[王强]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.