We ran a number of different studies...
Watch my lips!
The study that we have analysed most extensively was the one where we explored how the brain of the native BSL signer takes account of mouth actions. Many signs use speechlike mouth actions. We were interested to see if the brain detected the difference between signs that may use such actions (like the sign BLUE) and signs that use mouth actions that are not speechlike (like the sign TRUE). Upper parts of the temporal lobe liked signs with speechlike mouth actions, and lower parts preferred signs where there was no speechlike action. This might be important in helping people with brain damage recover their BSL skills – and their speechreading. These findings have been published in Neuropsychologia and the Journal of Cognitive Neuroscience (see Publications page).
We also completed the first detailed study of how the brain perceives fingerspelled words. It turns out that fingerspelling activated the same regions as signed language – but more so. One region which was especially active for fingerspelling was a small region in the lower part of the temporal lobe which had already been identified as active when hearing people read written words (and which is less active in people with dyslexia).
Because this region is involved in word reading, some researchers have called it
the ‘visual word form area'. Until our work, it was thought that this region was specialised to process the visual characteristics of letter groups that make up words – letter groups like ‘BA' or ‘CH'. Since fingerspelled words activated this region too, and fingerspelled letters do not look very much like printed letters, this cannot be the whole story of the function of this region. Dafydd Waters and Mairéad MacSweeney are following up on this study. This study was published in NeuroImage (see Publications page).
How low can you go?
Another study with deaf native signers explored how the brain managed signs where you could not see the signer easily, but only small points of light on their body. You cannot identify who someone is or what they are signing when you see just a photograph of this: all you see is a jumble of points of light on a dark background. But when the person moves, you see the patterns of movement they make as they move their hands and body. In the first frame, below, the signer is standing still. In the second and third frames, you see the beginning and end points for the sign DRESS.