When the Body Speaks and the Tone Echoes: Exploring Gai Multimodal Emotion Recognition

Posters

Keyword: Other

Abstract: We assessed emotion recognition in three Gemini GAI models versus humans using dynamic vocal/bodily stimuli, via zero-shot prompting with multiple-choice questions. GAIs matched human vocal accuracy but lagged on gestures, showing higher accuracy for positive over negative emotions. GAIs excelled with bodily versus vocal cues, highlighting their current social-cognitive profile.

• David Piterman (Presenting Author)
• Hadar Dery (Author)
• Kfir Bar (Author)
• Gunther Meinlschmidt (Author)
• Alon Geller (Author)
• Dorit Hadar Shoval (Author)
• Zohar Elyoseph (Author)
• Elad Refoua (Author)