How and why we see what we do

Researchers at the UCL Institute of Ophthalmology have shown that virtual robots trained to correctly identify surfaces, also - as a consequence - see the same illusions that humans see.

The new study, published in the latest edition of PLoS Computational Biology, shows that illusions are an inevitable consequence of learning useful behaviour in a complex environment.

The study's senior author, Dr Beau Lotto, explained: "Sometimes the best way to understand how the visual brain works is to understand why sometimes it does not, which is one reason why lightness illusions have been the focus of scientists, philosophers and artists interested in how the mind works for centuries. And yet why we see them is still unclear."

To understand why humans see illusions, the team used artificial neural networks, effectively virtual toy robots with miniature virtual brains, to model, not human vision as such, but a natural visual ecology. Dr David Corney in Dr Lotto's lab trained the virtual robots to predict the reflectance - shade of grey - of surfaces in different 3D scenes not unlike those found in nature.

Dr Lotto said: "they not only learned to get it right like we do, but they also - as a consequence - got it wrong like we do too. This provides causal evidence for the view that perceptions represent not the world as it really is, but what proved useful to see in our past interactions with the world.

The virtual robots in this study were driven solely by the statistics of their training history and used these statistics as the basis of both their correct and incorrect decisions. We believe the human brain generates perceptions of the world in the same way, by encoding the statistical relationships between images and scenes in our past visual experience, and uses this as the basis for behaving usefully and consistently towards the sources of visual images."

Although the artificial neural networks used in the research are much less complex than the human visual system, this simplicity helped the researchers to identify and further understand what they believe is a fundamental principle behind why we see illusions: the statistics of our past visual experiences. Since the brain does not have direct access to the reflectance of objects in the world, but only images of those objects, it has to call on the statistics of how it behaved in the past to towards these objects understand how to behave in the future.

Dr Corney said: "Every image is ambiguous to us, to animals and to robots. Our eyes and brains have evolved to let us behave effectively and so survive. So when presented with any image of the world, what we see is what would have been useful to see in the past. Illusions are uncommon and so misinterpreting such images rarely matters."

Dr Lotto added: "The study also suggests a biologically-based definition of what an illusion is: a condition of the world in which the actual source of a stimulus differs from its most likely source. In other words, an illusion is more a function of the world than of perception. And since resolving ambiguous sensory information is a challenge faced by all visual systems, including the virtual robots in this study, it is likely that illusions are experienced by all visual animals regardless of their particular neural machinery. Visual illusions have been central to the science and philosophy of human consciousness for centuries and this research demonstrates that how we respond to them can give vital information about the processes behind vision."

Image 1:  This is a synthetic 'dead-leaves' image, which preserves many of the statistical structures common to natural images. Artificial neural networks were trained to recognize surfaces within these images under illumination that vary in space and time, with the result that the trained networks exhibited responses commensurate with human lightness illusions