SIA OpenIR  > 工业控制网络与系统研究室
基于梯度的深度网络剪枝算法
Alternative TitleGradient-based deep network pruning algorithm
王忠锋1,2,3; 徐志远1,2,3,4; 宋纯贺1,2,3; 张宏宇5; 蔡颖凯5
Department工业控制网络与系统研究室 ; 工业控制网络与系统研究室
Source Publication计算机应用
ISSN1001-9081
2020
Volume40Issue:5Pages:1253-1259
Indexed ByCSCD
CSCD IDCSCD:6718235
Contribution Rank1
Keyword深度网络 深度网络 压缩与加速 压缩与加速 剪枝 剪枝 自适应阈值 神经网络 自适应阈值 神经网络
Abstract

深度神经网络模型通常存在大量冗余的权重参数,当计算深度网络模型时需要占用大量的计算资源和存储空间,导致其难以部署在一些边缘设备和嵌入式设备上。针对这一问题,提出了一种基于梯度的深度网络剪枝算法——GDP算法(Gradient-based Deep network Pruning)。GDP算法核心思想是以梯度作为评判权值重要性的依据。然后通过自适应的方法找出阈值进行权值参数的筛选,目的是剔除那些小于阈值的梯度所对应的权值。最后,重新训练剪枝后的深度网络模型恢复精度。实验结果表明:在CIFAR-10数据集上,GDP算法在精度仅下降0.14个百分点的情况下,计算量减少了35.3个百分点;与当前流行的PFEC算法相比,GDP算法使网络模型精度提高0.13个百分点,计算量下降1.1个百分点,具有更优越的深度网络压缩与加速性能。

;

深度神经网络模型通常存在大量冗余的权重参数,当计算深度网络模型时需要占用大量的计算资源和存储空间,导致其难以部署在一些边缘设备和嵌入式设备上。针对这一问题,提出了一种基于梯度的深度网络剪枝算法——GDP算法(Gradient-based Deep network Pruning)。GDP算法核心思想是以梯度作为评判权值重要性的依据。然后通过自适应的方法找出阈值进行权值参数的筛选,目的是剔除那些小于阈值的梯度所对应的权值。最后,重新训练剪枝后的深度网络模型恢复精度。实验结果表明:在CIFAR-10数据集上,GDP算法在精度仅下降0.14个百分点的情况下,计算量减少了35.3个百分点;与当前流行的PFEC算法相比,GDP算法使网络模型精度提高0.13个百分点,计算量下降1.1个百分点,具有更优越的深度网络压缩与加速性能。

Other Abstract

Deep neural network models usually have a large number of redundant weight parameters. When calculating the deep network model, it requires a large amount of computing resources and storage pace, which makes it difficult to deploy on some edge devices and embedded devices. To resolve this issue, a gradient-based deep network pruning algorithm is proposed, namely GDP (Gradient-based Deep network Pruning). The core idea of GDP pruning algorithm is to use gradient as the basis for choosing the network weights. To eliminate the weights corresponding to gradients that are less than the threshold, an adaptive method was used to find the gradient threshold to identify the weights. Finally, the deep network model was retrained after pruning to restore the network performance. Experimental results show that the GDP algorithm reduces the computational complexity by 35.3percent with a precision loss of only 0.14 percent in the CIFAR-10 dataset. Compared with the state-of-the-art PFEC algorithm, the GDP increases the network classification accuracy by 0.13 percent, and the computational costs is reduced by 1.1 percent, which exerts superior performance of deep network in terms of both compression and acceleration.

Language中文
Citation statistics
Document Type期刊论文
Identifierhttp://ir.sia.cn/handle/173321/25788
Collection工业控制网络与系统研究室
Corresponding Author徐志远
Affiliation1.中国科学院沈阳自动化研究所机器人学国家重点实验室
2.中国科学院网络化控制系统重点实验室
3.中国科学院机器人与智能制造创新研究院
4.中国科学院大学
5.国网辽宁省电力有限公司
Recommended Citation
GB/T 7714
王忠锋,徐志远,宋纯贺,等. 基于梯度的深度网络剪枝算法[J]. 计算机应用,2020,40(5):1253-1259.
APA 王忠锋,徐志远,宋纯贺,张宏宇,&蔡颖凯.(2020).基于梯度的深度网络剪枝算法.计算机应用,40(5),1253-1259.
MLA 王忠锋,et al."基于梯度的深度网络剪枝算法".计算机应用 40.5(2020):1253-1259.
Files in This Item: Download All
File Name/Size DocType Version Access License
基于梯度的深度网络剪枝算法.pdf(1742KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[王忠锋]'s Articles
[徐志远]'s Articles
[宋纯贺]'s Articles
Baidu academic
Similar articles in Baidu academic
[王忠锋]'s Articles
[徐志远]'s Articles
[宋纯贺]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[王忠锋]'s Articles
[徐志远]'s Articles
[宋纯贺]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 基于梯度的深度网络剪枝算法.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.