The how and why of gesture in the brain

Abstract

Two fMRI experiments explain the brain mechanisms underlying the extraction of meaningful information from observed co-speech gestures (i.e., the how) and the result of the operation of this mechanism on listeners (i.e., the why). Specifically, the brain represents observed gestures as motor programs to produce those gestures. Through feedback, the motor system predicts the sounds and words associated with those motor programs. The engagement of this mechanism results in a subsequent decrease in the participation of brain areas involved in semantic selection or retrieval and early sensory processing. This implies that the brain needs to sample the external world less after confirming predictions about what sounds and words are associated with observed gestures. Thus, accurate predictions free up neural resources that may then be allocated to additional sensory processing or elaboration upon an interpretation of what is being heard. This might explain why some gestures aid in language comprehension.