When the Body Speaks and the Tone Echoes: Exploring Gai Multimodal Emotion Recognition

Virtual · October 2025

Poster · Personality/Emotion

David Piterman
University of Haifa, Israel
Hadar Dery
Kfir Bar
Reichman University
Gunther Meinlschmidt
: University of Basel and University Hospital Basel
Gunther Meinlschmidt
Trier University, Germany
Alon Geller
Ruppin Academic Center
Dorit Hadar Shoval
Max Stern Yezreel Valley College
Zohar Elyoseph
University of Haifa
Elad Refoua
Bar-Ilan University, Israel

Abstract

We assessed emotion recognition in three Gemini GAI models versus humans using dynamic vocal/bodily stimuli, via zero-shot prompting with multiple-choice questions. GAIs matched human vocal accuracy but lagged on gestures, showing higher accuracy for positive over negative emotions. GAIs excelled with bodily versus vocal cues, highlighting their current social-cognitive profile.

Other

← Posters

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.

Necessary

Always Enabled

Functional

Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.

Cookie	Duration	Description
__cf_bm	30 minutes	This cookie, set by Cloudflare, is used to support Cloudflare Bot Management.

Performance

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Cookie	Duration	Description
AWSELBCORS	5 minutes	This cookie is used by Elastic Load Balancing from Amazon Web Services to effectively balance load on the servers.

Analytics

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.

Cookie	Duration	Description
at-rand	never	AddThis sets this cookie to track page visits, sources of traffic and share counts.
CONSENT	2 years	YouTube sets this cookie via embedded youtube-videos and registers anonymous statistical data.
uvc	1 year 27 days	Set by addthis.com to determine the usage of addthis.com service.
_ga	2 years	The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.
_gat_gtag_UA_3507334_1	1 minute	Set by Google to distinguish users.
_gid	1 day	Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously.

Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.

Cookie	Duration	Description
loc	1 year 27 days	AddThis sets this geolocation cookie to help understand the location of users who share the information.
VISITOR_INFO1_LIVE	5 months 27 days	A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.
YSC	session	YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.
yt-remote-connected-devices	never	YouTube sets this cookie to store the video preferences of the user using embedded YouTube video.
yt-remote-device-id	never	YouTube sets this cookie to store the video preferences of the user using embedded YouTube video.
yt.innertube::nextId	never	This cookie, set by YouTube, registers a unique ID to store data on what videos from YouTube the user has seen.
yt.innertube::requests	never	This cookie, set by YouTube, registers a unique ID to store data on what videos from YouTube the user has seen.

Others

When the Body Speaks and the Tone Echoes: Exploring Gai Multimodal Emotion Recognition

Abstract

Cookies