Close
Mapping Acoustic and Semantic Dimensions of Auditory Perception
Auditory categorisation is a function of sensory perception which allows humans to generalise across many different sounds present in the environment and classify them into behaviourally relevant categories. These categories cover not only the variance of acoustic properties of the signal but also a wide variety of sound sources. However, it is unclear to what extent the acoustic structure of sound is associated with, and conveys, different facets of semantic category information. Whether people use such data and what drives their decisions when both acoustic and semantic information about the sound is available, also remains unknown.
To answer these questions, we propose to use the existing methods broadly practised in linguistics, acoustics and cognitive science, and bridge these domains by delineating their shared space. Firstly, we took a model-free exploratory approach to examine the underlying structure and inherent patterns in our dataset. To this end, we ran principal components, clustering and multidimensional scaling analyses. At the same time, we drew sound labels’ semantic space topography based on corpus-based word embeddings vectors. We then built an LDA model predicting class membership and compared the model-free approach and model predictions with the actual taxonomy. Finally, by conducting a series of web-based behavioural experiments, we aim to investigate whether acoustic and semantic topographies relate to perceptual judgements. This analysis pipeline aims to show whether there is any overlap between acoustic, semantic and perceptual representations of sound categories at different levels of abstraction. Results from our studies will help to recognise the role of physical sound characteristics and their meaning in the process of sound perception. Understanding the intrinsic dynamics between semantic and acoustic spaces will provide an essential contribution to improving recognition and evaluation systems, as well as an invaluable insight into the mechanisms governing the machine-based and human classifications.