SIA OpenIR  > 空间自动化技术研究室
CNN-Based Facial Expression Recognition from Annotated RGB-D Images for Human-Robot Interaction
Li, Jing1,2; Mi, Yang1; Li, Gongfa2; Ju ZJ(琚兆杰)3,4
Department空间自动化技术研究室
Corresponding AuthorJu, Zhaojie(zhaojie.ju@port.ac.uk)
Source PublicationInternational Journal of Humanoid Robotics
ISSN0219-8436
2019
Volume16Issue:4Pages:17
Indexed BySCI ; EI
EI Accession number20192907210348
WOS IDWOS:000488067600003
Contribution Rank3
Funding OrganizationNational Natural Science Foundation of China ; Natural Science Foundation for Distinguished Young Scholars of Jiangxi Province ; Open Fund of Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology ; National Key R&D Program of China ; CAS Interdisciplinary Innovation Team
KeywordFacial expression recognition convolutional neural network Kinect RGB-D images depth information
AbstractFacial expression recognition has been widely used in human computer interaction (HCI) systems. Over the years, researchers have proposed different feature descriptors, implemented different classification methods, and carried out a number of experiments on various datasets for automatic facial expression recognition. However, most of them used 2D static images or 2D video sequences for the recognition task. The main limitations of 2D-based analysis are problems associated with variations in pose and illumination, which reduce the recognition accuracy. Therefore, an alternative way is to incorporate depth information acquired by 3D sensor, because it is invariant in both pose and illumination. In this paper, we present a two-stream convolutional neural network (CNN)-based facial expression recognition system and test it on our own RGB-D facial expression dataset collected by Microsoft Kinect for XBOX in unspontaneous scenarios since Kinect is an inexpensive and portable device to capture both RGB and depth information. Our fully annotated dataset includes seven expressions (i.e., neutral, sadness, disgust, fear, happiness, anger, and surprise) for 15 subjects (9 males and 6 females) aged from 20 to 25. The two individual CNNs are identical in architecture but do not share parameters. To combine the detection results produced by these two CNNs, we propose the late fusion approach. The experimental results demonstrate that the proposed two-stream network using RGB-D images is superior to that of using only RGB images or depth images.
Language英语
WOS SubjectRobotics
WOS KeywordFACE
WOS Research AreaRobotics
Funding ProjectNational Natural Science Foundation of China[61703198] ; National Natural Science Foundation of China[51575412] ; Natural Science Foundation for Distinguished Young Scholars of Jiangxi Province[2018ACB21014] ; Open Fund of Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology[2017B02] ; National Key R&D Program of China[2018YFB1304600] ; CAS Interdisciplinary Innovation Team[JCTD-2018-11]
Citation statistics
Document Type期刊论文
Identifierhttp://ir.sia.cn/handle/173321/25306
Collection空间自动化技术研究室
Corresponding AuthorJu ZJ(琚兆杰)
Affiliation1.School of Information Engineering, Nanchang University, Nanchang 330031, China
2.Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081, China
3.State Key Laboratory of Robotics, Shenyang Institute of Automation Chinese, Academy of Sciences, China
4.School of Computing, University of Portsmouth, Portsmouth, PO1 3HE, United Kingdom
Recommended Citation
GB/T 7714
Li, Jing,Mi, Yang,Li, Gongfa,et al. CNN-Based Facial Expression Recognition from Annotated RGB-D Images for Human-Robot Interaction[J]. International Journal of Humanoid Robotics,2019,16(4):17.
APA Li, Jing,Mi, Yang,Li, Gongfa,&Ju ZJ.(2019).CNN-Based Facial Expression Recognition from Annotated RGB-D Images for Human-Robot Interaction.International Journal of Humanoid Robotics,16(4),17.
MLA Li, Jing,et al."CNN-Based Facial Expression Recognition from Annotated RGB-D Images for Human-Robot Interaction".International Journal of Humanoid Robotics 16.4(2019):17.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Li, Jing]'s Articles
[Mi, Yang]'s Articles
[Li, Gongfa]'s Articles
Baidu academic
Similar articles in Baidu academic
[Li, Jing]'s Articles
[Mi, Yang]'s Articles
[Li, Gongfa]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Li, Jing]'s Articles
[Mi, Yang]'s Articles
[Li, Gongfa]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.