SIA OpenIR  > 工业控制网络与系统研究室
Alternative TitleGradient-based deep network pruning algorithm
王忠锋1,2,3; 徐志远1,2,3,4; 宋纯贺1,2,3; 张宏宇5; 蔡颖凯5
Source Publication计算机应用
Contribution Rank1
Keyword深度网络 压缩与加速 剪枝 自适应阈值 神经网络
Abstract深度神经网络模型通常存在大量冗余的权重参数,当计算深度网络模型时需要占用大量的计算资源和存储空间,导致其难以部署在一些边缘设备和嵌入式设备上。针对这一问题,提出了一种基于梯度的深度网络剪枝算法——GDP算法(Gradient-based Deep network Pruning)。GDP算法核心思想是以梯度作为评判权值重要性的依据。然后通过自适应的方法找出阈值进行权值参数的筛选,目的是剔除那些小于阈值的梯度所对应的权值。最后,重新训练剪枝后的深度网络模型恢复精度。实验结果表明:在CIFAR-10数据集上,GDP算法在精度仅下降0.14个百分点的情况下,计算量减少了35.3个百分点;与当前流行的PFEC算法相比,GDP算法使网络模型精度提高0.13个百分点,计算量下降1.1个百分点,具有更优越的深度网络压缩与加速性能。
Other AbstractDeep neural network models usually have a large number of redundant weight parameters. When calculating the deep network model, it requires a large amount of computing resources and storage pace, which makes it difficult to deploy on some edge devices and embedded devices. To resolve this issue, a gradient-based deep network pruning algorithm is proposed, namely GDP (Gradient-based Deep network Pruning). The core idea of GDP pruning algorithm is to use gradient as the basis for choosing the network weights. To eliminate the weights corresponding to gradients that are less than the threshold, an adaptive method was used to find the gradient threshold to identify the weights. Finally, the deep network model was retrained after pruning to restore the network performance. Experimental results show that the GDP algorithm reduces the computational complexity by 35.3percent with a precision loss of only 0.14 percent in the CIFAR-10 dataset. Compared with the state-of-the-art PFEC algorithm, the GDP increases the network classification accuracy by 0.13 percent, and the computational costs is reduced by 1.1 percent, which exerts superior performance of deep network in terms of both compression and acceleration.
Document Type期刊论文
Recommended Citation
GB/T 7714
王忠锋,徐志远,宋纯贺,等. 基于梯度的深度网络剪枝算法[J]. 计算机应用,2020:1-10.
APA 王忠锋,徐志远,宋纯贺,张宏宇,&蔡颖凯.(2020).基于梯度的深度网络剪枝算法.计算机应用,1-10.
MLA 王忠锋,et al."基于梯度的深度网络剪枝算法".计算机应用 (2020):1-10.
Files in This Item: Download All
File Name/Size DocType Version Access License
基于梯度的深度网络剪枝算法.pdf(722KB)期刊论文出版稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[王忠锋]'s Articles
[徐志远]'s Articles
[宋纯贺]'s Articles
Baidu academic
Similar articles in Baidu academic
[王忠锋]'s Articles
[徐志远]'s Articles
[宋纯贺]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[王忠锋]'s Articles
[徐志远]'s Articles
[宋纯贺]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 基于梯度的深度网络剪枝算法.pdf
Format: Adobe PDF
This file does not support browsing at this time
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.