Social interaction in virtual environments: the relationship between mutual gaze, task performance and social presence

Abstract

Everyday face-to-face social interaction is increasingly being supplemented by computer- and video-mediated communication. With mediation, however, comes the potential loss of important non-verbal cues. It is therefore important to attempt to maintain the quality of the mediated interaction, such that it retains as many of the aspects of a real-world interaction as possible. Social presence is a measure of how similar a mediated interaction is to face-to-face, the most socially present situation, in terms of perceptions of and behaviour towards an interlocutor. Social presence can be mediated by many factors, one of which is mutual gaze, and social perceptions of an interlocutor are also thought to be related to task performance. For a successful interaction, therefore, an optimum amount of mutual gaze for maximising social presence and task performance is desirable. This research aims to investigate the relationship between mutual gaze, task performance and social presence, in order to discover the ideal conditions under which a successful mediated interaction can occur. Previous gaze research paradigms have involved one conversational partner staring continuously at the other, and the resulting mutual gaze being measured. It is hypothesised that this method may actually suppress mutual gaze, primarily due to social reasons. It is potentially, therefore, not the optimum experimental design for mutual gaze research. The first study in this thesis used eye-tracking to explore this hypothesis and investigate the relationship between mutual gaze and task performance. A suitable paradigm was developed, based on that used in previous research into eye movements and non-verbal communication. Two participants – Instruction Giver (IG) and Instruction Follower (IF) – communicated via avatars in Second Life to solve simple arithmetic tasks. There were two between-participant looking conditions: staring (the IG’s avatar stared continuously at the IF); and notstaring, (IG’s avatar looked at IF and task-relevant objects). Constant staring did, indeed, show evidence of decreasing mutual gaze within the dyad. Mutual gaze was positively correlated with task performance scores, but only in the not-staring condition. When not engaged in mutual gaze, the IF looked more at task-related objects in the not-staring condition than in the staring condition; this suggests that social factors are likely to be driving the gaze aversion in the staring condition. Furthermore, there are no task-related benefits to staring. The second study explored further how much looking by one person at another will maximise both mutual gaze and task performance between the dyad. It also investigated the relationship between mutual gaze, task performance and both manipulated and perceived social presence. Individual participants interacted with a virtual agent within the Second Life paradigm previously used in the human-human study. Participants were either told they were interacting with a computer (i.e. an agent) or another human (an avatar). This provided the between-participants manipulated social presence variable, or agency. The virtual agent was programmed to look at the participant during either 0%, 25%, 50% or 75% of the interaction, providing the within-participants variable looking condition. The majority of effects were found in the 75% looking condition, including the highest mutual gaze uptake and the highest social presence ratings (measured via a questionnaire). Although the questionnaire did not detect any differences in social presence between the agent and avatar condition, participants were significantly faster to complete the tasks in the avatar condition than in the agent condition. This suggests that behavioural measures may be more effective at detecting differences in social presence than questionnaires alone. The results are discussed in relation to different theories of social interaction. Implications and limitations of the findings are considered and suggestions for future work are made.