The perceptual and cerebral representation of natural sounds

Every day, we encounter diverse natural sounds (chirps, impulses) and effortlessly recognise the objects and events producing them (a bird call, a nail being hammered). Studying how we perceive these sounds provides a unique window into how the auditory system operates under real-world conditions, moving beyond the traditional focus on speech and music. Investigating these signals promises a broader understanding of auditory processing, encompassing both acoustic complexity and semantic structure. 
I will start this talk by describing my early behavioural and fMRI work attempting to tease apart acoustic and semantic components of natural sound representation. I will then focus on more recent collaborative efforts that combined behavioural measures, ultra-high-field (7T) fMRI, and convolutional neural networks (CNNs) to reveal that CNN-based models closely mirror perceptual and fMRI representations. Here, mid-layer CNN representations best predicted brain and behavioural data, likely emphasising the acoustic structure critical for identifying objects and actions (hyperacoustics). I will then describe ongoing work that aims to track the temporal unfolding of natural sound representations through magnetoencephalography. I will conclude the talk with an overview of a new collaborative effort that aims to extend our AI + computationally modelling framework from isolated sounds to the complexity of real-life auditory scenes. By embracing the full richness of everyday listening, we seek a unified perspective on how the auditory system decodes our ever-changing acoustic environment, ultimately integrating neural, behavioural, and computational insights to illuminate the complexity of real-world hearing.

Join at imt.lu/sagrestia