Umeå University's logo

As AI systems increasingly engage in social interactions, comprehending human social dynamics is crucial. Affect recognition enables systems to respond appropriately to emotional nuances in social situations. However, existing multimodal approaches lack accounting for the social appropriateness of detected emotions within their contexts. This paper presents a novel methodology leveraging sentence embeddings to distinguish socially appropriate and inappropriate interactions for more context-aware AI systems. Our approach measures the semantic distance between facial expression descriptions and predefined reference points. We evaluate our method using a benchmark dataset and a real-world robot deployment in a library, combining GPT-4(V) for expression descriptions and ada-2 for sentence embeddings to detect socially inappropriate interactions. Our results underscore the importance of considering contextual factors for effective social interaction understanding through context-aware affect recognition, contributing to the development of socially intelligent AI capable of interpreting and responding to human affect appropriately.

Mental disorders including neurodevelopmental diffculties are frequent, creating a substantial disparity between the demand for mental health care and the available resources. The potential of therapeutic technologies to address this treatment gap is immense, offering scalable solutions to enhance access. Yet, the intricate nature of mental disorders, woven with diverse risk factors, poses challenges to a comprehensive understanding of their mechanisms and assessment of this potential. Current assessments of mental disorders heavily rely on the expertise of trained clinicians, making it imperative to explore innovative avenues such as "digital phenotyping" to capture nuanced behaviors. However, integrating technology into healthcare encounters obstacles exacerbated by the divergent cultures of medical professionals and engineers. While technical feasibility is a priority for engineers, it often needs to match the acceptability standards set by healthcare professionals. Navigating the complexity of the healthcare ecosystem compounds the challenge of identifying precise needs. Furthermore, the time-intensive nature of clinical research methods hinders the swift evaluation of effcacy. To surmount these hurdles, we advocate for the incorporation of user-centered design methodologies and participatory research in the development of therapeutic technologies. This chapter delves into the multifaceted challenges of designing technologies, such as robots, for therapeutic programs focused on individuals with neurodevelopmental disorders. By proposing solutions that prioritize participatory co-design environments, we aim to empower individuals from diverse backgrounds to collaboratively support those undergoing therapy with technology, ensuring its effcacy and benefts.

Neurodivergent children with cognitive and communicative difficulties often experience a lower level of social integration in comparison to neurotypical children. Therefore it is crucial to understand social inclusion challenges and address exclusion. Since previous work shows that gamified robotic activities have a high potential to enable inclusive and collaborative environments we propose using robot-mediated games for enhancing social inclusion. In this work, we present the design of a multiplayer tangible Pacman game with three different inter-player interaction modalities: semi-dependent collaborative, dependent collaborative, and competitive. The initial usability evaluation and the observations of the experiments show the benefits of the game for creating collaborative and cooperative practices for the players and thus also potential for social interaction and social inclusion. Importantly, we observe that inter-player interaction design affects the communication between the players and their physical interaction with the game.