Digital technology and emotions :a framework for the dynamic understanding of human-computer interaction -

Abstract

Humans are nowadays surrounded by digital devices that weaved themselves into the fabric of their everyday life in an intelligent way. This is the result of recent advancements in the “Human Computing” domain that witnessed the emergence of humancentered applications building models based on human-centered data. Since most of these applications rely on understanding the user's context and desires through sensory interfaces, a lot of work is yet to be done on designing implicit interactions with systems that operate in a seamless and unobtrusive way. Particularly in the human-computer interaction (HCI), very little emphasis has been placed on enabling computer functions that would understand and emulate the human emotional state. Hence, the objective of this work is to develop a framework for characterizing human's use of personal computer, and evaluating the relationship between digital activity and personal emotions. The framework involves: 1) A proposal for a robust ground-truth system for the natural capture of a person's emotion, and that is unobtrusive and seamless in the context of computer usage. 2) The development of personalized models for emotion recognition which can serve as feed for personalization applications such as those trying to alter the interface depending on the emotional state of the user or moderate the user's emotions for improved mental health. To realize the thesis objectives, a groundtruth model is designed for emotion recognition by combining facial expressions analysis, self-assessment and rater-based assessment. We designed and conducted real life experiments for people working in their workspace. Human-centered data was collected about their comprehensive computer activity and their emotional experience. A large number of features were then extracted reflecting the user's context and behavior. Features were annotated using a novel class labels extraction strategy. Finally, a Bayesian Network is proposed as a classifier for emotion recognition. Results show evidence tha