SIA OpenIR  > 其他
面向陪伴机器人应用的人脸表情识别
Alternative TitleFacial Expression Recognition for Companion Robot Application
张文萍
Department其他
Thesis Advisor贾凯
Keyword人脸表情识别 陪伴机器人 类内类间差异 分布不平衡 实时人脸表情识别
Pages71页
Degree Discipline控制工程
Degree Name专业学位硕士
2021-05-21
Degree Grantor中国科学院沈阳自动化研究所
Place of Conferral沈阳
Abstract人脸表情识别在人们日常交流中占有非常重要的地位,在人机交互、行为分析和情感计算等领域都有着广泛的应用前景。随着中国老龄化社会的加剧,很多老人缺少子女的陪伴,常处于孤独的状态,所以面向陪伴机器人应用的人脸表情识别成为一个重要的研究课题。随着深度学习的发展,人脸表情识别这一研究领域也有了很大的进展,但是由于表情识别容易受主体、姿态和光照等因素的影响,因此人脸表情识别中仍然存在着很多问题。本文主要针对主体因素和表情数目因素对表情识别的影响分别给出解决方案,并针对实时人脸表情识别建立一个更完善的识别流程。本课题的主要工作总结如下:(1)针对主体因素对表情识别造成的相同表情表现差异大不同表情表现差异小的影响,本文首先使用cGAN生成表情脸对应的中性脸,然后利用中间提取的残留的表情信息用作CNN的特征输入;其次在CNN中,本文使用了Softmax损失函数与ECE损失函数共同优化网络。相比于Island损失函数,ECE损失函数考虑了困难样本对类内距离损失的影响,希望降低那些离样本中心点太远的特征对网络的优化。通过不同的损失函数在Oulu_CASIA数据集上的测试性能结果与CK+,JAFFE,SFEW,RAF和Older数据集上的泛化性能结果对比,结果表明ECE损失函数相比于Island损失函数同时提高了测试性能与泛化性能。(2)针对表情数目因素对表情识别造成的多样本表情识别准确率高而少样本表情识别准确率低的影响,本文使用RAF-DB数据集和FERPlus数据集进行测试性能实验与交叉泛化性能实验。利用ResNet18网络进行表情特征提取与分类,并使用Softmax损失函数,weighted_Softmax损失函数,以及weighted_Softmax损失函数与weighted_Island损失函数,weighted_Cluster损失函数,weighted_ECE损失函数和weighted_LCE损失函数配合使用的几种损失函数分别优化网络,对比其对数据集中每一类表情的识别性能、平均识别准确率和全局识别准确率。观察各个表情的识别准确率发现,改进的损失函数相比于Softmax损失函数在少样本的表情类别的识别准确率上均有不同程度的提升,平均识别准确率也得到了提升。(3)针对实时人脸表情识别的整个流程,本文给出了详细的介绍,并针对其中耗时最长的人脸检测部分进行了检测时间与检测性能的对比。通过对检测时间与性能的对比以及不同的检测方法对表情识别的影响发现,MTCNN人脸检测器相比于dlib(n=1)人脸检测器检测时间更快,检测效果相差不大,但是由于本文采用的MTCNN的预训练模型人脸检测之后得到的检测框太大,导致表情识别不稳定。dlib(n=2)的检测效果要比dlib(n=1)和MTCNN稍微差一些,但是检测速度更快,而且其检测得到的人脸大小更适合送入表情识别网络进行表情识别,因此不考虑识别距离这一因素,本文在实时人脸表情识别中采用dlib(n=2)进行人脸检测。
Other AbstractFacial expression recognition plays a very important role in people's daily communication. It has a wide range of applications in human-computer interaction, behavior analysis and emotional computing. With the aggravation of China's aging society, many elderly people are often lonely due to the lack of children's company. Therefore, facial expression recognition for accompanying robot applications has become an important research topic. With the development of deep learning, the research field of facial expression recognition has made great progress. However, there are still many problems in facial expression recognition because facial expression recognition is easily affected by factors such as subject, pose and illumination. This paper focuses on the subject factors and the number of facial expression factors on the impact of expression recognition, respectively, to give solutions, and for real-time facial expression recognition to establish a more perfect recognition process. The main work of this project is summarized as follows: (1) In view of the influence of subject factors on expression recognition, the same expression has great difference, while different expressions have little difference, this paper first uses cGAN to generate neutral picture corresponding to expression picture, then the residual expression information extracted from the middle to be used as the feature input of CNN; secondly, in CNN, Softmax loss function and ECE loss function are used to optimize the network together. Compared with Island loss function, ECE loss function takes the influence of difficult samples on the intra-class distance into account, and hopes to reduce the optimization of the network caused by the features too far from the center of the sample. Observe different loss functions acquired the test performance results on Oulu_CASIA dataset and the generalization performance results on CK+, JAFFE, SFEW, RAF and Older datasets. The results show that ECE loss function improves the test performance and generalization performance compared with Island loss function. (2) In view of the influence of the number of expressions on the high accuracy of multi sample expression recognition and the low accuracy of small sample expression recognition, RAF-DB and FERPlus datasets are used for test and cross generalization performance experiments in this paper. The expression feature extraction and classification are carried out by ResNet18 network, and Softmax loss function, weighted_Softmax loss function, and weighted_Softmax loss function assist weighted_Island loss function, weighted_Cluster loss function, weighted_ECE loss function and weighted_LCE loss function are used to optimize the network, respectively. Compare the recognition performance of each expression in the dataset, the average recognition accuracy and the global recognition accuracy. The accuracy of each expression was observed, compared with Softmax loss function, the improved loss function improved the recognition accuracy of expression categories with less samples, and the average recognition accuracy was also improved. (3) In view of the whole process of real-time facial expression recognition, this paper gives a detailed introduction, and compares the detection time and performance of the longest time-consuming face detection method. Through the comparison of face detection time and performance and the influence of different detection methods on facial recognition, it is found that MTCNN face detector has faster detection time and similar detection effect than dlib (n=1) face detector. However, because the MTCNN pre-training model used in this paper detected larger bounding box than true facial region for face detection, it leads to unstable expression recognition. The detection effect of dlib (n=2) is slightly worse than that of dlib (n=1) and MTCNN, but the detection speed is faster, and the detected face size is more suitable for expression recognition network. Therefore, the detection distance is not considered, in this paper, dlib (n=2) is used in real-time recognition to detect face.
Language中文
Contribution Rank1
Document Type学位论文
Identifierhttp://ir.sia.cn/handle/173321/28972
Collection其他
Affiliation中国科学院沈阳自动化研究所
Recommended Citation
GB/T 7714
张文萍. 面向陪伴机器人应用的人脸表情识别[D]. 沈阳. 中国科学院沈阳自动化研究所,2021.
Files in This Item:
File Name/Size DocType Version Access License
面向陪伴机器人应用的人脸表情识别.pdf(3016KB)学位论文 开放获取CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[张文萍]'s Articles
Baidu academic
Similar articles in Baidu academic
[张文萍]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[张文萍]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.