基于特性分流的多模態(tài)對(duì)話情緒感知算法

打開(kāi)文本圖片集

doi：10.19734/j. issn.1001-3695.2024.12.0466

Multimodal dialogue emotion perception algorithm based on feature divergence

Ren Qinze a，b ，Yuan Yea，b，F(xiàn)u Ketinga，?，F(xiàn)u Junxiua，?，Xu Kanga，b，Liu Na，bt （a.Institutefcneelie，ooflhe&，Ueitffone&oi ，China）

Abstract：Multimodalemotion perceptioniscrucialfor monitoring personal healthand providing medicalcareinthe fieldof proactive health.Currnt multimodal dialogue emotionperceptiontechnologiesfacechallenges ifusing informationacross differentmodalities，particularlyincapturinglocalrelationshipsbetweenmodalies.Theproposedmultimodalfusionalgorithm basedonfeaturediversion，MEPAD（multimodalemotionperceptionalgorithmwith featurediversion），addressedthesechallenges bycapturing global information indialogues using graphneural networks and integrating homogeneous and specific features across modalities through thehypercomplex number system and pairwise feature fusion mechanisms.Experiments on he IEMOCAP and MOSEI datasets demonstratethat MEPAD significantlyoutperforms existing methodsin multimodal dialogue emotionperceptiontasks，highlightingitsefectiveessandpotentialinhandlingomplexemotionaldata.Thisresearchoffs newinsights for theapplication of multimodal emotion perception technology in proactive health.

Key words：multimodal emotionrecognition；graph neural networks；hypercomplex number system；pairwisefeature fusion; dialogue emotion perception

0 引言

情緒感知是人類交流中的一個(gè)關(guān)鍵組成部分，在主動(dòng)健康領(lǐng)域，對(duì)話者情緒變化也可以作為評(píng)估人類健康信息監(jiān)控的重要指標(biāo)[1]。（剩余18982字）