SIA OpenIR  > 空间自动化技术研究室
A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion
Gao Q(高庆)1,2,3; Ogenyi, Uchenna Emeoha4; Liu JG(刘金国)1,2; Ju ZJ(琚兆杰)1,2,4; Liu, Honghai4
Department空间自动化技术研究室
Conference Name19th Annual UK Workshop on Computational Intelligence, UKCI 2019
Conference DateSeptember 4, 2019 - September 6, 2019
Conference PlacePortsmouth, United kingdom
Source PublicationAdvances in Computational Intelligence Systems - Contributions Presented at the 19th UK Workshop on Computational Intelligence, 2019
PublisherSpringer Verlag
2019
Pages107-118
Indexed ByEI
EI Accession number20194107502842
Contribution Rank1
ISSN2194-5357
ISBN978-3-030-29932-3
KeywordHand gesture recognition CNN Multimodal data fusion
AbstractAt present, vision-based hand gesture recognition is very important in human-robot interaction (HRI). This non-contact method enables natural and friendly interaction between people and robots. Aiming at this technology, a two-stream CNN framework (2S-CNN) is proposed to recognize the American sign language (ASL) hand gestures based on multimodal (RGB and depth) data fusion. Firstly, the hand gesture data is enhanced to remove the influence of background and noise. Secondly, hand gesture RGB and depth features are extracted for hand gesture recognition using CNNs on two streams, respectively. Finally, a fusion layer is designed for fusing the recognition results of the two streams. This method utilizes multimodal data to increase the recognition accuracy of the ASL hand gestures. The experiments prove that the recognition accuracy of 2S-CNN can reach 92.08 $$\%$$ on ASL fingerspelling database and is higher than that of baseline methods.
Language英语
Document Type会议论文
Identifierhttp://ir.sia.cn/handle/173321/25741
Collection空间自动化技术研究室
Corresponding AuthorLiu JG(刘金国)
Affiliation1.State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China
2.Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China
3.University of Chinese Academy of Sciences, Beijing 100049, China
4.School of Computing, University of Portsmouth, Portsmouth
5.PO1 3HE, United Kingdom
Recommended Citation
GB/T 7714
Gao Q,Ogenyi, Uchenna Emeoha,Liu JG,et al. A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion[C]:Springer Verlag,2019:107-118.
Files in This Item:
File Name/Size DocType Version Access License
A Two-Stream CNN Fra(1217KB)会议论文 开放获取CC BY-NC-SAView Application Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Gao Q(高庆)]'s Articles
[Ogenyi, Uchenna Emeoha]'s Articles
[Liu JG(刘金国)]'s Articles
Baidu academic
Similar articles in Baidu academic
[Gao Q(高庆)]'s Articles
[Ogenyi, Uchenna Emeoha]'s Articles
[Liu JG(刘金国)]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Gao Q(高庆)]'s Articles
[Ogenyi, Uchenna Emeoha]'s Articles
[Liu JG(刘金国)]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.