Detección automática de acciones sociales mediante sensores de movimiento vestibles y Edge AI

The field of Social Signal Processing (SSP) deals with the automatic interpretation, analysis, and synthesis
of social signals or stimuli, such as facial expressions, tone of voice, gestures, and body language. Part of
the SSP area focuses on detecting and analyzing these social signals using Artificial Intelligence (AI) in
conjunction with sensors such as cameras, microphones, physiological sensors (e.g., heart rate), or
wearable inertial sensors. However, there are serious concerns related to privacy and the processing of
data obtained by these sensors, especially for vulnerable populations such as minors or people with
illnesses.
The use of wearable inertial sensors, such as accelerometers or gyroscopes, for the automatic analysis of
social signals and interactions has proven to be a feasible alternative to audio and video recording in
interactions in real-life environments, offering greater privacy preservation for users.
However, the vast majority of SSP work focuses on the use of sensing devices as data collection devices.
Thus, they only record and store raw signals (video, audio, motion, etc.) without any inference processing,
leaving the execution of AI algorithms for prediction or inference to be performed offline later or on external
servers.
This project, carried out in collaboration with the Socially Perceptive Computing Lab at Delft University of
Technology (Netherlands), aims to develop and implement a prototype in real-life scenarios of EdgeAIbased
algorithms for the automatic detection of social signals such as speaker detection and conversational
gestures using information from motion sensors, capable of running on low-cost wearable devices. The aim
is to bring inference processes using AI closer to the user and, consequently, prevent the flow of their
private information to external servers.
The ultimate goal of the entire line of research (see roadmap) is the use of non-intrusive sensors for the
non-invasive monitoring of people in their real work and study environments, focusing on their mental wellbeing.
This project constitutes the second part of that research line on human-computer interaction.

Desarrollar algoritmos para la detección automática de señales
sociales de habla y gestos en dispositivos vestibles mediante
sensores de movimiento

Telecomunicaciones y monitorización remota