My work focuses on the nonverbal expression of emotions. I am particularly interested in how the emotion process drives the temporal unfolding of expression in different modalities, I adopt an appraisal perspective to study emotion expressions, and I consider expressive cues as potential efferent effects of emotion appraisals.
I am the head Applied Affective Sciences research unit and I am responsible of several basic and applied research projects done in collaboration with external partners.
I completed a PhD course in Psychology of Communication and Linguistic Processes at the Catholic University of Milan in 2007. My work focused on emotion expression and ANS correlates of emotion episodes. My thesis was entitled "Multicomponential analysis of emotional experience. Study of expressive and physiological components and their implications for affective computing". The goal was to study how facial and vocal expressions are related to modifications in the ANS activity in consequence of the experience of an emotion.
The same year, I joined the Swiss Center for Affective Sciences as a Post-doctoral researcher with the goal of studying emotion expression from a multimodal point of view.
In 2012 I was nominated Senior Scientist and I became the Head of the Applied Affective Sciences research unit
Multimodal expression of emotion:
I am interested in how different expressive modalities are jointly used during an emotional episode and how perception is influenced by unimodal and multimodal information.A particular feature of multimodal emotion expression is the way in which the different expressive modalities recruited during an emotion episode are timely used. With this respect, I am interested in how the degreee of temporal synchronization influences perception of emotional authenticity.
Nonverbal correlates of emotional appraisals
Planned studies include the investigation of the links between specific expressive features (vocal, facial, and gestures) and emotion appraisals.
We are developing instruments that can be used to assess emotional responses in several application settings. The measures include nonverbal behavior as well as self-report questionnaires.
"Measuring emotional intelligence in the workplace: Development of a new test" (Principal Investigator). FNS Grant
The aim of this project is to develop and validate a new test to measure emotional intelligence in the context of job related tasks. Emotional intelligence tests that are currently available have been criticized for their methodological approaches and for being generic and based on self-report measures. Conversely, this test will be a performance-based test that will include scenarios taken from real work situations.
"Automatic detection of subtle positive emotions from facial expressions"
This is a collaborative research project between GfK, Fraunhofer IIS and the University of Geneva (UNIGE) aimed at developing a semi-automatic tool for detecting subtle emotions in response to audio-visual stimuli.
"I can cope with that!: The effect of stress on emotion components: appraisal, expression, and physiology." (Principal Investigator) 2012-13
In this project we investigated the appraisals that are typically involved in the experience of a series of emotions. Furthermore the influence of stress on appraisals and other emotional components was investigated. The hypothesis is that stress can bias individuals’ evaluations by reducing his or her estimation of the sense of control on the situation (i.e., the coping potential).
"Emotional factors in ethical choices". 2010-11
This project addressed the relationship between different appraisals in the experience of different positive affective states and the role of emotion appraisals in ethical decisions. In particular we investigated how moral emotions are experienced in the context of socially responsible behavior. More generally, the project investigated how emotions can influence decisions about ethical issues.
Schlegel, K., & Mortillaro, M. (in press). The Geneva Emotional Competence Test (GECo): an ability measure of workplace emotional intelligence. Journal of Applied Psychology
Schuller, B., Weninger,, F., Zhang, Y., Ringeval, F., Batliner, A., Steidl, S., Eyben, F., Marchi, E., Vinciarelli, A., Scherer, K., Chetouani, M., & Mortillaro, M. (2019). Affective and Behavioural Computing: Lessons Learnt from the First Computational Paralinguistics Challenge. Computer Speech and Language, 53, 156-180.
Mortillaro, M., & Dukes, D. (2018). Jumping for joy: the importance of the body and of dynamics in the expression and recognition of positive emotions. Frontiers in Psychology , 9, 763.
Scherer, K.R., Mortillaro, M., Rotondi, I., Sergi, I., Trznadel, S. (2018). Appraisal-driven facial actions as building blocks for emotion inference. Journal of Personality and Social Psychology, 114(3), 358-379.
Soleymani, M., & Mortillaro, M. (2018). Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition. Frontiers in ICT - Human Media Interaction, 5, 17.
Dukes, D., Clément, F., Audrin, C., & Mortillaro, M. (2017). Looking beyond the static face in emotion recognition: The informative case of interest. Visual Cognition, 25 (4-6), 575-588..
Scherer, K., Mortillaro, M., & Mehu, M. (2017). Facial expression is driven by appraisal and generates appraisal inference. In J. M. Fernandez-Dols and J. A. Russell (Eds.), The science of Facial Expression (353-373). New York: Oxford University Press
Mortillaro, M., & Mehu, M. (2015). Emotions: Methods of Assessment. In J. D. Wright (Ed.), International Encyclopedia of the Social & Behavioral Sciences, 2nd edition, Vol 7 (pp. 519-525). Oxford: Elsevier.
Mortillaro, M., Meuleman, B., & Scherer, K. R. (2015). Automated recognition of emotion appraisals. In J. Vallverdú (ed.), Handbook of Research on Synthesizing Human Emotion in Intelligent Systems and Robotics (pp.338-351). Hershey, PA: Information Science Reference, IGI Global.
Visch, V., Goudbeek, M.B., & Mortillaro, M. (2014). Robust anger: Recognition of deteriorated dynamic bodily emotion expressions. Cognition & Emotion, 28(5), 936-946.
Mortillaro, M., & Scherer, K. R. (2014). Stressed out: How stress develops and how to cope with it. GfK Marketing Intelligence Review, 6(1), 17-21.
Mortillaro, M., Ricci-Bitti, P.E., Bellelli, G., Galati, D., & Scherer, K.R. (2013). Pride is not created equal: variations between Northern and Southern Italiy. In K.R. Scherer and J.R. Fontaine (Eds.), Components of Emotional Meaning: A sourcebook (pp. 366-376). New York: Oxford University Press.
Weninger, F., Eyben F., Schuller B.. W., Mortillaro M., & Scherer K. R. (2013). On the Acoustics of Emotion in Audio: What Speech, Music and Sound have in Common. Frontiers in Psychology. 4(292)
Scherer, K. R., Mortillaro, M., & Mehu, M. (2013).Understanding the Mechanisms Underlying the Production of Facial Expression of Emotion: A Componential Perspective. Emotion Review, 5(1), 47-53
Mortillaro, M., Mehu, M., & Scherer, K.R. (2013). The evolutionary origin of multimodal synchronization in emotional expression. In E. Altenmüller, S. Schmidt, and E. Zimmerman (Eds.), Evolution of Emotional Communication: from Sounds in Nonhuman Mammals to Speech and Music in Man (pp.3-25). New York: Oxford University Press.
Mehu, M., Mortillaro, M., Bänziger, T, & Scherer, K. R. (2012). Reliable facial muscles activation enhances recognisability and credibility of emotional expression. Emotion,12(4), 701-715.
Mortillaro, M., Meuleman, B., & Scherer, K. R. (2012). Advocating a componential appraisal model to guide emotion recognition. International Journal of Synthetic Emotions, 3(1), 18-32.
Dael, N., Mortillaro, M., & Scherer, K.R. (2012). The Body Action and Posture coding system (BAP): Development and reliability. Journal of Nonverbal Behavior. 36, 97-121.
Dael, N., Mortillaro, M., & Scherer, K.R. (2012). Emotion expression in body action and posture. Emotion. 12(5), 1085-1101.
Bänziger, T., Mortillaro, M., & Scherer, K.R. (2012). Introducing the Geneva Multimodal Expression corpus for experimental research on emotion perception. Emotion, 12(5), 1161-1179.
Mortillaro, M., Mehu, M., & Scherer, K. R. (2011). Subtly different positive emotions can be distinguished by their facial expressions. Social Psychological and Personality Science, 2, 262-271.
Scherer, K.R., Clark- Polner, E., & Mortillaro, M. (2011). In the eye of the beholder? Universality and cultural specificity in the expression and perception of emotion. International Journal of Psychology, 46, 401-435.
Glowinski, D., Dael, N., Camurri, A., Volpe, G., Mortillaro, M., & Scherer, K. R. (2011). Towards a minimal representation of affective gestures. IEEE Transactions on Affective Computing, 2, 106-118.
Mohammadi, G.,. Vinciarelli, & Mortillaro, M. (2010). The voice of personality: Mapping vocal nonverbal behavior into trait attributions. Proceedings of the International Workshop on Social Signal Processing, pp. 17-20, Florence, 2010.
Mantovani, F., Confalonieri, L., Mortillaro, M., Realdon, O., Zurloni, V., & Anolli, L. (2010). The potential of affective computing in e-learning: the journey from theory to practice in the “Myself” project. In A. Tzanavari and N. Tapatsoulis (Eds.), Affective, Interactive and Cognitive Methods for E- Learning Design: Creating an Optimal Education Experience (pp. 260-274). Hershey, PA: IGI Global.
Mortillaro, M., & Scherer, K. R. (2009). Multimodal expression of emotion. In D. Sander & K. R.Scherer (Eds.), The Oxford Companion to Emotion and the Affective Sciences (pp.268-269). New York: Oxford University Press.
Mortillaro, M., & Scherer, K. R. (2009). Bodily expression of emotion. In D. Sander & K. R. Scherer (Eds.), The Oxford Companion to Emotion and the Affective Sciences (pp.78-79). New York: Oxford University Press.
Castellano, G., Mortillaro, M., Camurri, A., Volpe, G., & Scherer, K. R. (2008). Automated analysis of body movement in emotionally expressive piano performances. Music Perception, 26, 103-120.
Zurloni, V., Mortillaro, M., Vescovo, A., & Anolli, L., Mantovani, F. (2008). Addressing emotions within e-learning systems. In T. T. Kidd & H. Song (Eds.), Handbook of Research on Instructional Systems and Technology (pp. 803-816). Hershey, PA: IGI Global.
Anolli, L., Mantovani, F., Balestra M., Kommers, P., Robotti, O., Salamin, A. D., Agliati, A., Realdon, O., Zurloni, V., Confalonieri, L., Mortillaro, M., Vescovo, A., Venturini, P., Wetterling, & Tarocchi, A. (2008). Experiential learning and affective computing for training soft-skills: Myself project. In T. T. Kidd & H. Song (Eds.), Handbook of Research on Instructional Systems and Technology (pp. 360- 372). Hershey, PA: Information Science Reference.
Anolli, L., Mortillaro, M., Zurloni, V., Confalonieri, L., Mantovani, F., (2008)."Affective computing" e apprendimento: analisi di dispositivi per il riconoscimento automatico delle emozioni all’interno di piattaforme di e-learning. In E. Magno Caldognetto and F. Cavicchio (Eds.), Aspetti Emotivi e Relazionali nell’E-Learning (pp.171-199). Firenze, IT: Firenze University Press .
Mortillaro, M. (2007).Multicomponential analysis of emotional experience. Study of expressive and physiological components and their implications for affective computing. Unpublished doctoral dissertation. Catholic University of Milan, Milan, Italy.
Mantovani, F., Agliati, A., Mortillaro, M., Vescovo, A., & Zurloni V. (2006). Communication – Presence roundtrip: Travelling along theoretical, methodological and applicative connections. In G. Riva, M. T. Anguera, B. K. Wiederhold, & F. Mantovani, (Eds.), From Communication to Presence: Cognition, Emotion and Culture towards the Ultimate Communicative Experiences (pp.3-22). Amsterdam, The Netherlands; IOS Press.
Realdon, O., Zurloni V., Confalonieri, L., Mortillaro, M., & Mantovani, F., (2006). Learning Communication Skills through computer-based Interactive Simulations. In G. Riva, M. T. Anguera, B. K. Wiederhold, & F. Mantovani, (Eds.), From Communication to Presence: Cognition, Emotion and Culture towards the Ultimate Communicative Experiences (pp.281-303). Amsterdam, The Netherlands; IOS Press.
Anolli, L., Mantovani, F., Mortillaro, M., Vescovo, A., Agliati, A., Confalonieri, L., Realdon, O., Zurloni, V., & Sacchi, A. (2005). A Multimodal Database as a Background for Emotional Synthesis, Recognition and Training in E-Learning Systems. In J. Tao, T. Tan, & R. W. Picard (Eds.), Affective Computing and Intelligent Interaction: First International Conference, ACII 2005, Beijing, China, October 22-24, 2005. Proceedings. Lecture Notes in Computer Science, 3784, (pp. 566 – 573).