Umeå University's logo

Introduction: Human-centric artificial intelligence (HCAI) focuses on systems that support and collaborate with humans to achieve their goals. To better understand how collaboration develops in human-AI teaming, further exploration grounded in a theoretical model is needed. Tuckman's model describes how team development among humans evolves by transitioning through the stages of forming, storming, norming, performing, and adjourning. The purpose of this pilot study was to explore transitions between the first three stages in a collaborative task involving a human and a human-centric agent.

Method: The collaborative task was selected based on commonly performed tasks in a therapeutic healthcare context. It involved planning activities for the upcoming week to achieve health-related goals. A calendar application served as a tool for this task. This application embedded a collaborative agent designed to interact with humans following Tuckman's stages of team development. Eight participants completed the collaborative calendar planning task, followed by a semi-structured interview. Interviews were transcribed and analyzed using inductive content analysis.

Results: The results revealed that the participants initiated the storming stage in most cases (n = 7/8) and that the agent initiated the norming stage in most cases (n = 5/8). Additionally, three main categories emerged from the content analyses of the interviews related to participants' transition through team development stages: (i) participants' experiences of Tuckman's first three stages of team development; (ii) their reactions to the agent's behavior in the three stages; and (iii) factors important to the participants to team up with a collaborative agent.

Conclusion: Results suggest ways to further personalize the agent to contribute to human-agent teamwork. In addition, this study revealed the need to further examine the integration of explicit conflict management into human-agent collaboration for human-agent teamwork.

Health-promoting digital agents, taking on the role of an assistant, coach or companion, are expected to have knowledge about a person's medical and health aspects, yet they typically lack knowledge about the person's activities. These activities may vary daily or weekly and are contextually situated, posing challenges for the human-Agent interaction. This pilot study aimed to explore the experiences and behaviors of older adults when interacting with an initially unknowledgeable digital agent that queries them about an activity that they are simultaneously engaged in. Five older adults participated in a scenario involving preparing coffee followed by having coffee with a guest. While performing these activities, participants educated the smartwatch-embedded agent, named Virtual Occupational Therapist (VOT), about their activity performance by answering a set of activity-ontology based questions posed by the VOT. Participants' interactions with the VOT were observed, followed by a semi-structured interview focusing on their experience with the VOT. Collected data were analyzed using an activity-Theoretical framework. Results revealed participants exhibited agency and autonomy, deciding whether to adapt to the VOT's actions in three phases: Adjustment to the VOT, partial adjustment, and the exercise of agency by putting the VOT to sleep after the social conditions and activity changed. Results imply that the VOT should incorporate the ability to distinguish when humans collaborate as expected by the VOT and when they choose not to comply and instead act according to their own agenda. Future research focuses on how collaboration evolves and how the VOT needs to adapt in the process.

Dialogues between humans are complex due to the challenges in predicting how they will unfold as people may want to achieve different purposes. For instance, to act together, they co-create a common goal; to learn, they co-create knowledge; to build relationships, they share emotions and beliefs. Apart from different purposes, people may want to achieve multiple purposes in a dialogue, introducing a movement between goals. Such actions cause problems in understanding and conflicts among the participants. Activity Theory denotes such situations as breakdown situations, which also occur when people have dialogues with software agents driven by Artificial Intelligence (AI). This thesis falls within the domain of human-centred AI, focusing on software agents able to collaborate and support people to achieve their goals. We call these software agents socially intelligent agents.

This thesis has two aims: (1) to develop an increased understanding of breakdown situations in dialogues between humans and socially intelligent agents and (2) to develop computational frameworks based on the developed understanding to manage breakdown situations, which could be embedded in an agent's cognitive architecture. The theoretical frameworks from social sciences, particularly Activity Theory, were applied to address the aims. They provided an alternate perspective that considers breakdown situations as opportunities to learn something new rather than the traditional view of them being errors or failures.

The main contributions addressing the first aim were theory-driven analysis and empirical findings that provided increased knowledge of breakdown situations, resulting in design implications and future agendas guiding the subsequent research. The results informed the three strategies to manage breakdown situations by aligning, partially aligning or not aligning with human's intentions. We found that participants considered partial alignment as a sufficient level of agreement for potential collaboration, which would be interesting to verify in future studies. To address the second aim, two novel computational frameworks were provided. These frameworks were based on linguistics and social sciences theories, allowing an agent to interpret the dialogue's syntax, semantics, and social aspects, facilitating a deeper understanding of dialogues. Finally, a novel computational framework was developed to reason about conflicts and be able to plan by adopting the strategy of aligning with the human's intentions. 

We conceptualised a cognitive architecture based on our research findings. The cognitive architecture embeds mechanisms for socially intelligent agents to manage breakdown situations in dialogues with humans. 

Background and Objective: When agents (e.g. a person and a social robot) perform a joint activity to achieve a joint goal, they require sharing a relevant group intention, which has been defined as a We-intention. In forming We-intentions, breakdown situations due to conflicts between internal and “external” intentions are unavoidable, particularly in healthcare scenarios. To study such We-intention formation and “reparation” of conflicts, this paper has a two-fold objective: introduce a general computational mechanism allowing We-intention formation and reparation in interactions between a social robot and a person; and exemplify how the formal framework can be applied to facilitate interaction between a person and a social robot for healthcare scenarios.

Method: The formal computational framework for managing We-intentions was defined in terms of Answer set programming and a Belief-Desire-Intention control loop. We exemplify the formal framework based on earlier theory-based user studies consisting of human-robot dialogue scenarios conducted in a Wizard of Oz setup, video-recorded and evaluated with 20 participants. Data was collected through semi-structured interviews, which were analyzed qualitatively using thematic analysis. N=20 participants (women n=12, men=8, age range 23-72) were part of the study. Two age groups were established for the analysis: younger participants (ages 23-40) and older participants (ages 41-72).

Results: We proved four theoretical propositions, which are well-desired characteristics of any rational social robot. In our study, most participants suggested that people were the cause of breakdown situations. Over half of the young participants perceived the social robot's avoidant behavior in the scenarios.

Conclusions: This work covered in depth the challenge of aligning the intentions of two agents (for example, in a person-robot interaction) when they try to achieve a joint goal. Our framework provides a novel formalization of the We-intentions theory from social science. The framework is supported by formal properties proving that our computational mechanism generates consistent potential plans. At the same time, the agent can handle incomplete and inconsistent intentions shared by another agent (for example, a person). Finally, our qualitative results suggested that this approach could provide an acceptable level of action/intention agreement generation and reparation from a person-centric perspective.

Purpose: The purpose of this study is to explore how older, middle aged and younger adults perceive breakdown situations caused by lack of or inconsistent knowledge, sudden focus shifts, and conflicting intentions in dialogues between a human and a socially intelligent robot in a home environment, and how they perceive strategies to manage breakdown situations.

Methods: Scenarios embedding dialogues on health-related topics were constructed based on activity-theoretical and argumentation frameworks. Different reasons for breakdown situations and strategies to handle these were embedded. The scenarios were recorded in a Wizard-of-Oz setup, with a human actor and a Nao robot. Twenty participants between 23 and 72 years of age viewed the recordings and participated in semi-structured interviews conducted remotely. Data were analyzed qualitatively using thematic analysis.

Results: Four themes relating to breakdown situations emerged: expecting, understanding, relating, and interacting. The themes span complex human activity at different complementary levels and provide further developed understanding of breakdown situations in human–robot interaction (HRI). Older and middle-aged adults emphasized emphatic behavior and adherence to social norms, while younger adults focused on functional aspects such as gaze, response time, and length of utterances. A hierarchical taxonomy of aspects relating to breakdown situations was formed, and design implications are provided, guiding future research.

Conclusion: We conclude that a socially intelligent robot agent needs strategies to 1) construct and manage its understanding related to emotions of the human, social norms, knowledge, and motive on a higher level of meaningful human activity, 2) act accordingly, for instance, adhering to transparent social roles, and 3) resolve conflicting motives, and identify reasons to prevent and manage breakdown situations at different levels of collaborative activity. Furthermore, the novel methodology to frame the dynamics of human–robot dialogues in complex activities using Activity Theory and argumentation theory was instrumental in this work and will guide the future work on tailoring the robot's behavior.}

Dialogues are an integral and intuitive communication mode for humans to reach a consensus and for their effective application to human-robot interaction a formal model of their structure is required. Formalization of dialogues enable robots to have constructive interaction and management of dialogue breaches such as ambiguity, sudden topic change and misunderstanding. In line with viewing dialogues as collaborative, a formal model of dialogues is proposed through Cooperative Distributed Grammar Systems (CDGS), which are abstract models based on ‘Blackboard problem’ in Artificial Intelligence, for building multi-agent systems, through formal grammars. For this research formalisation is done in two stages: first, a corpus of movie scripts is selected, and annotated with the help of an annotation scheme (combination of speech acts, response and social obligation tag). Then the annotated labels are structured through dialogue policies (progressive, binding and co-occurring) proposed in this paper. Finally, formalization of derived structure through CDGS is done.

Joint activity between human and robot agent requires them to not only form joint intention and share a mutual understanding about it but also to determine their type of commitment. Such commitment types allows robot agent to select appropriate strategies based on what can be expected from others involved in performing the given joint activity. This work proposes an architecture embedding commitments as we-intentional modes in a belief-desire-intention (BDI) based Theory of Mind (ToM) model. Dialogue mediation gathers observations facilitating ToM to infer the joint activity and hierarchical task network (HTN) plans the execution.The work is ongoing and currently the proposed architecture is being implemented to be evaluated during human-robot interaction studies.

Mutual understanding of the dialogue purpose and role of participants is vital to co-create meaning and satisfactory outcomes during human and social robot dialogues. This study explored how younger and older adults perceive roles, goals and behaviors of dialogue participants when breakdown situations occur during co-creation of meaning in a dialogue. The aim was to identify intuitive mechanisms that a social robot can use to manage breakdown situations. A set of dialogues were authored, based on activity-theoretical models of human activity and recorded in a Wizard-of-Oz setup with a Nao robot. Results of 8 (four younger and four old adults) participant interviews after observing the recorded dialogues are presented. Thematic analysis of interviews resulted into three themes of perceptions and expectations on roles and relationship, the robot’s understanding of the situation and relevant behavioral norms. Themes were translated into design implications for developing activity-centric dialogue systems to seamlessly manage breakdown situations. Future work includes implementation and further study of the proposed design implications for human robot dialogues.

In a natural dialogue, humans can handle misunderstanding, non-understanding, and sudden topic change integrally. An essential aspect of human-machine interaction is natural language understanding (NLU). This work proposes a hybrid model for NLU combining feature extraction with indicator classes (syntactic tokens and sequences) and semantic similarity for automatic labelling and a deep CNN learning model to integrally detect a sudden topic change, misunderstanding and non-understanding. The results report a significant improvement for the convolution model compared to the baseline multi-layer perceptron model for the classification task.

This work proposes a framework to predict sequences in dialogues, using turn based syntactic features and dialogue control functions. Syntactic features were extracted using dependency parsing, while dialogue control functions were manually labelled. These features were transformed using tf-idf and word embedding; feature selection was done using Principal Component Analysis (PCA). We ran experiments on six combinations of features to predict sequences with Hierarchical Agglomerative Clustering. An analysis of the clustering results indicate that using word embeddings and syntactic features, significantly improved the results.