Jesus Maldonado, Mason Oakes, & Dr. Peter Krause
Talkers’ facial cues are known to aid speech perception and signal social turn-taking conventions. Less is understood about how they affect specific temporal coordination between partners. We tasked participants with timing keypresses to coincide with the beginnings of recorded utterances. In our experiments, audiovisual recordings of talkers initiating utterances were drawn from a corpus of natural conversations. Final audiovisual stimuli were animations of isolated facial features, created using Python scripts operating over face-tracking data. Stimuli were presented in three counterbalanced conditions (within participants). In one experiment we had: talker’s lips animated over a black background with sound, talker’s head tilt animated over a black background with sound, and with the face completely invisible with sound. In another experiment we had: talker’s lips animated over a black background with sound, talker’s lips animated over a black background without sound, and with the face completely invisible with sound. In our former experiment key press RTs were the shortest with lips only. The no visibility and head tilt conditions had the same reaction times. In our subsequent experiment keypress RTs were shortest in the lips only condition, followed by lips and sound and then sound only. Other implications for models of turn-taking, and possible future directions, will be discussed.