The auditory system: complex and largely unexplained

When people think of auditory research, they usually tend to think of the ears. However, just as how vision is about more than the eyes, “hearing” involves more than the ears alone. Sound entering the ear causes the eardrum to vibrate, and this vibration travels through the cochlea, a snail-shaped organ in the inner ear, where it is converted into neural signals by the auditory nerve. These signals then travel through the brainstem and thalamus before reaching the auditory cortex in the cerebral cortex. This pathway is exceedingly complex. Therefore, as Professor Matsui explains, many aspects of auditory function remain unexplained.

“The auditory pathway from the ear to the auditory cortex travels from the peripheral system, including the ear and cochlea, to deep parts of the brain, such as the brainstem, making it difficult to observe. Moreover, there are many neural nuclei that relay neural signals, resulting in a very complex pathway. So, how can we investigate it? One method is to use computational models and simulations. Various researchers have already developed auditory models, and by utilizing these, I am attempting to elucidate parts of the complex auditory mechanism through psychophysical experiments that verify their accuracy.”

Using a simulated hearing impairment system to replicate the actual hearing experience of the elderly

One of the methods used by Professor Matsui is the Wadai Hearing Impairment Simulator, which was developed by Professor Toshio Irino and his team at Wakayama University. This system can replicate the auditory experience of individuals with hearing impairments, making it a valuable tool for understanding the difficulties associated with hearing loss and the experience of those with it.

“It is often said that, as people get older, their hearing declines. Using this simulated hearing impairment system, we can verify exactly what parts of the hearing process deteriorate and how it affects their hearing. Additionally, we intend to determine whether this system truly corresponds to the real hearing experiences of the elderly. If it does not, we want to identify the causes and use that information to make improvements,” says Professor Matsui.

The simulated hearing impairment system is an excellent tool that allows us, simply by inputting the results of a subject's audiogram, to simulate the processing in their peripheral auditory system and reproduce the sounds heard by the individual. However, because the simulation is inherently an approximation, its reproducibility can vary depending on the environment and conditions. Professor Matsui is investigating under what circumstances the system may not accurately reproduce hearing.

“For example, we conducted an experiment involving both young individuals with normal hearing and elderly participants. The young participants were not only involved in regular auditory perception experiments, but also listened to sounds processed by the simulated hearing impairment system, effectively participating in this case as ‘simulated elderly individuals.’ As a result, it was found that the simulated hearing impairment system could relatively accurately reproduce the hearing of elderly individuals with good hearing, but its reproducibility was lower for those with poor hearing. Additionally, reproducing hearing in environments with irregular noise, such as background chatter, proved challenging. In other words, simulations limited to the peripheral system alone may not sufficiently explain the state of hearing.”

It has long been known that elderly individuals have difficulty hearing consonants. They also struggle to catch notification sounds from household appliances such as chimes or spoken messages. Using the simulated hearing impairment system, Professor Matsui and her team confirmed that elderly individuals (or those with simulated elderly hearing) also have difficulty detecting the engine sound of approaching cars. As the number of people with age-related hearing loss is expected to continue increasing, designing sound environments that facilitate easy hearing will become extremely important. In this respect, being able to clarify the specific “hearing” characteristics of elderly individuals will be very valuable.

“Hearing loss is considered a risk factor for many diseases, including dementia. Additionally, as hearing loss progresses, it becomes difficult to communicate with others, which can hinder social life. Moving forward, we aim to use this simulated hearing impairment system to clarify what kind of clear speech makes it easy for everyone to hear. We also hope to contribute to the education of speech-language-hearing therapists and the improvement of hearing aids.”

Understanding the auditory mechanism through reactions to and perceptions of sound

Another area of focus for Professor Matsui is the field of “auditory feedback in vocalization.” This refers to the function by which humans hear their own voice and adjust their speech accordingly. Although this function itself has long been understood, Professor Matsui and her team are now investigating auditory feedback using sounds produced from artificial stimuli.

“For example, we are currently conducting experiments as a part of a joint research project with Prof. Hideki Kawahara, Professor Emeritus of Wakayama University. In these experiments, we introduce slight tremors or irregular changes in the pitch and duration of the stimulus sound, and observe how these variations affect human vocalization when participants adjust their voices to match the pitch.”

Professor Matsui and her team focus on a rapid response known as the “involuntary response.” Previous research has shown that there are two types of vocal responses to sounds which include fluctuations and last for 500 ms or longer, which are differentiated by the latency period from stimulus onset to response. The slower response varies depending on experimental conditions, whereas the faster response shifts the vocal pitch in the opposite direction of the stimulus pitch variation, compensating for it. Due to its short latency, this fast response is considered involuntary. Based on this knowledge, Professor Matsui and her team are conducting experiments to investigate the characteristics of auditory feedback in greater detail by varying the types and magnitudes of stimulus sounds.

“The sounds we typically hear consist of a fundamental frequency, which is the base frequency, with harmonic overtones that are integer multiples of that frequency, creating complex sounds. In contrast, a sound that contains only a single frequency component is called a pure tone, and a sound with only higher harmonics, excluding the fundamental frequency, is known as a missing fundamental. We investigated involuntary responses by having participants listen to these various sounds and vocalize in response to them. The results showed that pure tones failed to elicit any responses, while harmonic complex sounds containing higher harmonics but lacking fundamental frequency drew only limited responses. This indicates that the responses to instantaneous fundamental frequency variations are likely influenced by the lower harmonics.”

What are Professor Matsui and her team trying to uncover through this research? She explains, “If we can identify the indicators necessary to control the relationship between hearing and vocalization, we may be able to gain clues about the pathways and processes occurring in the brain.” In other words, they are seeking to gain insights that could help elucidate the fundamental mechanisms of the still largely unexplained auditory system.
Additionally, they are conducting experiments to look at how humans distinguish between male and female voices, differentiate between various speakers, and perceive non-verbal information, such as distinguishing angry voices from neutral ones.

“Furthermore, in our laboratory, we also focus on music. For example, one of our students set up a program that can generate melodies and found that songs with more syncopation tended to sound better, even if the melodic contour is unchanged. Personally, having played the piano since childhood, I have always wondered why listening to music can be so fun, and spark various emotions within us. I also want to explore the characteristics of trained musicians who have accumulated years of practice. My ultimate dream” concluded Professor Matsui, “is to be able to explain music in terms of logic,”.

Reporter's Note

Starting piano lessons in kindergarten, Professor Matsui went on to attend Nishinomiya High School in Hyogo Prefecture, which is renowned for its music program, , before enrolling at Kyoto City University of Arts. During her graduate studies, she spent a year studying piano in the United States before returning to Japan to complete a master’s degree in instrumental performance.

“To earn a master’s degree in performance, I had to give an 80-minute solo recital as well as writing my thesis. The thesis I wrote on music psychology back then has led me to where I am today,” Professor Matsui explains. Although, Professor Matsui chose to concentrate on research rather than performance after graduating. It is her unique combination of research with her experience and perspective as a performer that makes us particularly anticipate her future work .