SIA OpenIR  > 光电信息技术研究室
光学与超声图像复原方法研究
Alternative TitleResearch on optical and ultrasonic image restoration method
刘鹏飞1,2
Department光电信息技术研究室
Thesis Advisor赵怀慈
Keyword图像去模糊 高光谱重建 卷积神经网络 特征金字塔 特征迁移
Pages111页
Degree Discipline模式识别与智能系统
Degree Name博士
2020-11-26
Degree Grantor中国科学院沈阳自动化研究所
Place of Conferral沈阳
Abstract在图像处理领域,图像复原技术起着举足轻重的作用,其在底层视觉研究中十分重要,与多个学科有着紧密联系,在交通监控、刑侦探案、医疗诊断和目标检测识别等领域有着广泛的应用需求,一直是学者重点关注的热门方向。图像复原技术可以提供比原始图像更多的信息,有利于拓展成更多应用,比如目标检测,识别与跟踪。图像复原的具体内涵包括图像去模糊、超分辨率重建、多光谱信息重建和结构信息恢复等。图像质量是各学科应用的前提,受到硬件设备的限制、拍摄条件以及噪声影响,图像会出现模糊、分辨率不足的情况,严重制约着图像的应用范围,因此通过图像复原技术,在已有单幅图像基础上获得高质量的图像是决定图像后续应用的关键。现实世界是一个由众多目标和特定环境构成的复杂场景,在计算机视觉任务中,高层视觉处理技术需要以高质量图像为前提,才能完成目标检测、识别与跟踪等任务。在本文中,以普通可见光相机、红外相机、可见光成像光谱仪和超声成像仪等多种设备拍摄的多模态图像为研究背景,开展图像运动模糊去除、图像超分辨率重建、高光谱信息重建与超声图像斑点噪声去除以及缺失边缘恢复等方法研究,突破图像基于深度学习的复原技术,本文主要研究内容和研究成果如下:1. 针对单幅噪声模糊图像盲复原问题,提出了一种基于高斯金字塔的卷积神经网络模型,输入为单幅模糊图像,无需估计模糊核,网络直接输出清晰图像。通过下采样获得比原始图像分辨率小的图像,对应的清晰图像也做下采样操作,采集不同感受野大小的特征信息,输出低分辨率结果后,再通过上采样操作与上一层的输入图像融合作为上一层网络的输入,保证了高分辨率输出图像包含了低分辨率图像信息。设计了稀疏编码器限制输入范围,使判别器权重参数更快达到最优。实验证明该网络能够完成运动模糊图像去除任务,在自动滤除噪声的同时能够消除运动模糊核的影响,输出清晰图像,其在三个标准模糊测试集上展示了算法的优越性。2. 基于超分辨率重建技术展开研究,针对其在深度学习中客观存在的问题,提出基于特征迁移的八层网络模型,模型为全卷积结构,可以适应任意大小的输入图像。打破了传统深度学习方法只能有三层卷积神经网络的限制,实现了深层全卷积网络。首先,通过预先训练得到前四层网络参数,然后以预训练好的参数作为基础实行特征迁移,在更深层的网络中,重新对新网络进行训练。定量评价采用客观指标为峰值信噪比PSNR(Peak Signal to Noise Ratio)和结构相似度SSIM(Structural Similarity),从结果可看出,相比于对比算法,本文模型能够得到更加优质的图像,当放大倍数观看时,依然可以看到较为清晰的纹理。3. 针对单幅RGB图像重建高光谱图像的严重不适定问题,提出了一种基于注意力机制的生成模型,无需硬件辅助或相机光谱响应,由单幅RGB图像重建高光谱信息。在网络内部建立了特征金字塔结构,并结合尺度注意力机制,根据感受野大小融合局部和全局特征,减小尺度噪声。本文方法与稀疏字典方法相比,均方误差RMSE(Root Mean Square Error)降低了42%,相对均方误差RMSERel(Root Mean Square Error Relative)降低了46.6%,在此基础上,提出了新的W-Net结构替换U-Net,改进后的模型引入了边缘图像监督模块,可以丰富模型重建图像高频信号的能力,与稀疏字典方法对比,RMSE降低了45%,RMSERel降低了50%。使用RGB相机拍摄真实场景,输入输出光谱数据均未知,在进行双盲实验时,W-Net重建图像质量优于对比算法。4. 针对胎儿超声图像面临的斑点噪声和边缘缺失问题,提出一种基于特征融合感知的去噪及复原方法。为了能够充分地补全胎儿头围和腹围缺失信息,首先利用深度卷积网络中不同层输出特征图对信息描述不同的特点,提出基于软尺度注意力机制的特征融合方法。随后,采用将每一层级输出的预测结果融合到特征图中,提高了分割精度。实验结果表明,该方法能够有效应对拍摄时可能遭受的斑点噪声、形变等挑战,并展示了出色的鲁棒能力。
Other AbstractImage restoration technology is one of the most basic research topics in image processing and low-level vision subject, which is a hot and difficult research topic, and is closely related to many disciplines. The richer information provided by image restoration has been beneficial to numerous applications, such as traffic monitoring, criminal investigation, medical diagnosis, target detection and recognition. The image restoration is always a hot issue. The key technologies of image restoration include image deblurring, image super-resolution reconstruction, hyperspectral image reconstruction, and image structural information restoration. The image quality is the premise of the various applications. Due to the limitation of hardware equipment, shooting conditions and noise effect, images would be blurred and the image resolution is insufficient, which seriously restrict the application of the image. Therefore, obtaining high-quality images is the key to subsequent application based on the single image through image restoration technology. The real world is a complex scene composed of many targets and specific environments. In computer vision tasks, high-level visual processing technology needs to take high-quality images as the premise to complete the tasks of target detection, recognition and tracking. This paper takes images captured by ordinary visible camera, infrared camera, ultrasonic imaging equipment and imaging spectrometer as the background, and researches on image deblurring, image super-resolution, hyperspectral reconstruction, speckle noise removal and boundary restoration of fetal ultrasound image, they are all restored by one single image under complex backgrounds. The main contributions of this paper are as follows: 1. To solve the problem of blind restoration of a single noise blurred image, we propose a multi-scale convolutional neural network based on Gaussian pyramid that restores sharp images in an end-to-end manner. We can obtain low-resolution images from original blurred and clear images through down sampling. In the training process, different pyramid levels can increase the receptive field and explore different features. We put domain constraint layer into network to restrict domain of input latent variables, which it also accelerates convergence. The output of every low-resolution image after up sampling concatenates with corresponding high-resolution blurred image as the input of next layer. The proposed network can remove the motion-blurred influence, and filter out the noise. Experiments show the superiority of the algorithm on three standard blurred datasets. 2. In order to meet the requirements of higher super-resolution reconstruction precision, an eight-layer end-to-end structure is proposed based on feature transfer. The input is a single low-resolution image, and the output is a high-resolution image. We can bulid a deep full-convolution network that the first four layers of the network obtain the shallow information by feature transfer and pre-training, and the last four layers realize feature enhancement. PSNR(Peak Signal to Noise Ratio)and SSIM(Structural Similarity)are used to quantitatively evaluate performance. Experimental results demonstrate the effectiveness of the proposed method on image super-resolution reconstruction, and network convergence speed is faster, which can recover image texture details better in comparison with traditional methods. 3. Aiming at the serious ill posed problem of hyperspectral image reconstruction from a single RGB image, a generation model based on attention mechanism is proposed. The hyperspectral reconstruction from a single RGB image is done with no need of additional hardware or camera spectral response known in advance. The feature pyramid is used inside the network and a scale attention module is designed to fuse local and global information according to the size of receptive field to reduce scale noise. Experimental results show that the proposed method outperforms the representative alternatives both qualitatively and quantitatively on two public standard test sets. Compared with sparse coding, the proposed method SAPUNet improves on the ICVL dataset natural scene by 42% and 46.6% in terms of RMSE(root mean square error)and relative RMSE, respectively. To provide a more accurate solution, we propose another distinct architecture, named W-Net, which builds one more branch compared to U-Net to conduct boundary supervision. The W-Net drops 45% and 50% in terms of RMSE and relative RMSE on the ICVL dataset than sparse coding. When using real-world image, which the input and output spectral data is unknown, the reconstructed image by W-Net has better quality than contrast algorithms. 4. Aiming at the speckle noise and structural incomplete boundary of fetal ultrasound images, a feature fusion perception-based denoising and restoration method is proposed. In order to infer the missing boundary, firstly, the output feature map of different layers in the deep convolution network is used to describe different characteristics of information, and a feature fusion method based on soft-scale attention mechanism is proposed. Then, the intermediate prediction result of each level is fused into the feature map, which improves the segmentation accuracy. The experimental results show that this method can deal with the challenges such as speckle noise and deformation effectively, and show the excellent robustness.
Language中文
Contribution Rank1
Document Type学位论文
Identifierhttp://ir.sia.cn/handle/173321/27980
Collection光电信息技术研究室
Affiliation1.中国科学院沈阳自动化研究所
2.中国科学院大学
Recommended Citation
GB/T 7714
刘鹏飞. 光学与超声图像复原方法研究[D]. 沈阳. 中国科学院沈阳自动化研究所,2020.
Files in This Item:
File Name/Size DocType Version Access License
光学与超声图像复原方法研究.pdf(5122KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[刘鹏飞]'s Articles
Baidu academic
Similar articles in Baidu academic
[刘鹏飞]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[刘鹏飞]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.