|dc.description.abstract||Beyond the words a speaker produces, meaning can be recovered from the many
ways in which the words are delivered. During everyday discourse, the way in
which a speaker produces an utterance - both in their spoken delivery and
accompanying movements | is an important part of the communication process.
This thesis investigates the ways in which meaning is carried within these nonlinguistic
behaviours, focussing on how they may provide signals about upcoming
message content and about speakers intentions.
Previous research has shown that the manner of spoken delivery (for instance, rates
of speech, intonation, and fluency) influences listeners content-based expectations
as well as pragmatic comprehension. These effects have been evidenced both
post-hoc and during the moment-to-moment processing of speech. Considerably
less attention has been paid on whether speakers movements and non-verbal
behaviours have similar effects: Research on gesture has tended to focus on the
content represented by gesture (rather than its potential to signal information
about the message and/or speaker).
We focus on two ways in which non-linguistic behaviours influence comprehension:
Firstly, as signals of speech planning difficulty (and so of upcoming content), and
secondly as indicators of a speakers intention to deceive. The former has been
studied in relation to speech disfluency, but not to gestures. The latter has been studied extensively in relation to many linguistic and non-linguistic behaviours,
but has only recently begun to be addressed with respect to the time course of
Focussing on these two areas, we address the broader question about the perceptual
relevance of non-linguistic signals in comprehension through a dialogue study and a
series of comprehension experiments combining eye and mouse tracking techniques.
Extending the VisualWorld Paradigm|commonly used in comprehension studies - to include a video component showing the speaker, we measure listeners' eye
movements and mouse coordinates as they select objects in the on-screen display.
By directly manipulating the presence of different non-linguistic behaviours, we
investigate whether and when these behaviours are interpreted as signals of 1)
upcoming difficulty in speech, and 2) the speaker's intention to deceive.
Our results demonstrate that, like for speech disfluency, listeners interpret
the presence of representational gesturing to inform explicit predictions about
upcoming referents. This follows from the findings from our dialogue study which
suggest that speakers produce more of this type of gesturing relative to speech when
describing shapes which are more conceptually difficult. We also show that listeners
reliably perceive certain motoric behaviours of a speaker to indicate deception,
and that the influence these behavioural cues have on listeners' interpretation can
be detected alongside the unfolding linguistic input.
Non-linguistic cues to perceived deception are found to be robust to contexts where
speakers produce a variety of cues in different modalities. However, findings point
to differences in how listeners link visual and spoken cues with deception. Results
indicate that when cues are present in both modalities, visual cues tend to drive
listeners' biases towards interpreting an utterance as dishonest. The time course
supports a view in which non-linguistic cues influence pragmatic comprehension
at an early stage, and suggests that linking visual cues with deception may be
more resource demanding than it is for spoken cues.
Listeners' associations between the presence of certain non-linguistic behaviours
and judgements of deception also hold in situations where there are other
alternative explanations for a given cue. However, the availability of an alternative
explanations for a given non-linguistic behaviour are found to influence early stages
of comprehension, suggesting that listeners may engage in dynamic reasoning
about the possible causes of a speaker's manner of delivery. Taken together, the
results from studies presented in this thesis highlight the role of both the spoken
and visual delivery of an utterance in shaping comprehension, highlighting the
fact that communication is fundamentally multi-modal.||en
|dc.relation.hasversion||King, J. P. J., Loy, J. E., & Corley, M. (2018). Contextual Effects on Online Pragmatic Inferences of Deception. Discourse Processes, 55 (2), 123-135. Retrieved from https://www.tandfonline.com/doi/full/10.1080/ 0163853X.2017.1330041 doi: 10.1080/0163853X.2017.1330041||en