Font Size

HOME > No.23, Nov. 2020 > The 'eyes' say more than the 'mouth', and can distinguish English sounds

The 'eyes' say more than the 'mouth', and can distinguish English sounds

A new method to estimate English /l/ /r/ discrimination ability from human pupillary responseBy Yuya Kinzuka
Yuya Kinzuka

A joint research team comprised of members of Toyohashi University of Technology’s Department of Computer Science and Engineering, and EIIRIS (Electronics-Inspired Interdisciplinary Research Institute), has discovered that the difference in the ability to hear and distinguish English words including L and R, which are considered difficult for Japanese people, appears in pupillary (the so-called "black part of the eye") responses. While pupil dilation performs a primary role of adjusting the amount of light that enters the eye, it is known that it can also reflect the cognitive state of humans. In this study, the research team conducted experiments to simultaneously measure the size of the pupil while playing English words in combinations such as "Light" and "Right", and clarified that it is possible to objectively estimate the ability to distinguish English words by observing pupil dilation.

In an increasingly globalized world, we often hear about the importance of improving English proficiency. However, we Japanese are said to be very weak in the pronunciation of and the hearing and distinguishing of L and R which are sounds that do not originally exist in Japanese, such as in "glass" and "grass". Given our understanding that words that cannot be recognized cannot be pronounced, the ability of each person to hear and distinguish English is a very important indicator in effective English learning.

As in the proverb: "The eyes say more than the mouth", it is known that pupil dilation can reveal various cognitive states. So, the research team tried to estimate the ability to hear and distinguish L and R by focusing on pupillary dilation response in which the pupil dilates with respect to the difference in sound. For this study, the research team played a repeating loop of words with an English L sound (e.g., "light") into which were randomly dispersed examples of the same word but with an R sound (e.g., "right"). The team then investigated how the pupils of test participants responded to those sounds. Participants were classified into two groups according to their scores in a test of their ability to distinguish English sounds, which was performed in advance, to compare the pupillary response of both groups.

Correlation between pupil diameter and English auditory distinguishing ability
Scene of the pupillometry experiment

As a result, the group with a strong ability to hear and distinguish the L and R sounds showed a larger pupillary response than the group with a weak ability to hear and distinguish them. It was also found that this pupillary response alone could estimate the ability of the participants who had been tested in advance to hear and distinguish English, with extremely high accuracy. Participants of the experiment were not required to pay attention to the English words they were listening to, they just needed to let them play, but their ability to hear and distinguish could be estimated from their pupillary responses alone at that time. The researchers believe that in future, this finding could provide a new indicator for a simple estimate of the ability of a person to hear and distinguish English.

"Up to now, the evaluation of an individual’s English listening ability has been carried out by actually performing a test in which they are made to listen to English words, and scoring whether the answers are correct or incorrect. However, we focused on the pupil, which is a biological signal, with the goal of extracting objective abilities that did not depend on the participants’ responses. Although all research participants could identify that there were two different sounds being played, their pupillary responses differed according to their English ability. So, I believe that this indicates that there is a possibility that our pupillary responses are reflecting differences in unconscious language processing", the lead author Yuya Kinzuka, a PhD candidate, explained.

Professor Shigeki Nakauchi, who is the leader of the research team, explained, "It was difficult for even the person themselves to recognize their listening ability, which sometimes led to a decrease in training motivation. However, this research has made it possible for not only the person themselves, but also a third party to visualize the listening ability of the learner objectively from the outside. I expect that in the future, objective measurement of the ability to hear and distinguish things will progress in various fields such as language and music."

In addition, explains research member Professor Tetsuto Minami, "This discovery shows that not only simple sounds such as pure tones, but also higher-order factors such as differences in utterances are reflected in pupillary response. I expect that it will be useful as an English learning method if it is possible to improve the ability to hear and distinguish things by controlling pupil dilation from the outside."

The research team has suggested that a new method to estimate the ability to distinguish English sounds from pupillary response, based on these research results, could form the basis of a system for efficiently studying the ability of Japanese people to distinguish the challenging L and R sounds. Furthermore, it is known that learning difficulties caused by the distinguishing of sounds that do not exist in the native language also occur when for example an English speaker learns Chinese. Ultimately then, we hope that this will become a new indicator of estimating language ability that is not limited to Japanese. In addition, these research results are expected to be useful for language learning in patients with movement and speech disorders as there is no need for the participants to pay attention to or physically respond to English words.

This study was conducted with the assistance of Grants-in-Aid for Scientific Research A(26240043) and Basic Research B(17H01807) from the Japan Society for the Promotion of Science, Ministry of education.

Reference

Kinzuka, Y., Minami, T., & Nakauchi, S. (2020). Pupil dilation reflects English /l//r/ discrimination ability for Japanese learners of English: a pilot study. Scientific Reports, 10(1), 1-9,
DOI: 10.1038/s41598-020-65020-1

“眼”は口ほどに物を言い、英語を聞き分ける

ヒトの瞳孔反応から英語の“L”と“R”の聞き分け能力を推定する新たな手法
By 金塚 裕也

豊橋技術科学大学 情報・知能工学系とエレクトロニクス先端融合研究所の研究チームは、日本人にとって難しいとされているLとRを含む英単語の聞き分け能力の違いが瞳孔(いわゆる黒目と呼ばれる部分)の反応に現れることを発見しました。瞳孔は眼に入る光の量を調整する役割を持っている一方で、ヒトの認知状態を反映して大きさが変化することが知られています。本研究は、”Light”と”Right”のような組み合わせの英単語を再生しながら、同時に瞳孔の大きさを計測する実験を実施し、眼からヒトの英語聞き分け能力を客観的に推定できることを明らかにしました。

社会のグローバル化に伴い、英語能力の向上は様々な分野において注目されています。しかしながら私たち日本人は、”Glass”と”Grass”のように日本語にもともと存在しない音であるLとRの発音や聞き分けが非常に苦手と言われています。聞き分けることができない単語は発音できないように、個人の英語聞き分け能力は効率的な英語学習において非常に重要な指標であるとされています。

「眼は口ほどに物を言う」ということわざのように、私たちの瞳孔は様々な認知の状態を反映することが知られています。そこで、研究チームは、音の違いに対して瞳孔が散大する瞳孔散大反応に着目して、LとRの聞き分け能力の推定を試みました。具体的には、連続して再生されるLを含む英単語(例えば”Light”)にときどきRを含む英単語(例えば”Right”)を混ぜ、その音に実験参加者の瞳孔がどのように反応するか調べました。

事前に行った英語聞き分け課題の点数に応じて実験参加者を二つの群に分け、瞳孔反応を比較したところ、LとRの聞き分け能力が高い群は低い群と比較して大きな瞳孔反応を示していることがわかりました。また、この瞳孔反応のみから事前に行った実験参加者の英語聞き分け能力を非常に高い精度で推定できることもわかりました。実験参加者は聞いている英単語に注意を向ける必要はなく、単に聞き流せば良いのですが、そのときの瞳孔反応のみから聞き取り能力を推定できるわけです。この発見は、将来的にヒトの英語聞き分け能力を簡易的に推定する新たな指標になりうると研究者らは考えています。

「これまで個人の英語リスニング能力を評価するには実際に英単語を聞かせるテストを行い、それが正解か不正解か採点することによって行っていましたが、参加者の応答によらない客観的な能力を抽出することを目標に、生体信号の一つである瞳孔に着目しました。聞き流しているLとRを含む英単語は全ての参加者にとって容易に聞き分け可能であるにもかかわらず、能力に応じて瞳孔反応が異なったことから、私たちの瞳孔反応には無意識な言語処理の違いも反映されている可能性を示していると考えています。」と筆頭著者である博士後期課程1年の金塚裕也は説明します。

研究チームのリーダーである中内茂樹教授は「リスニング能力は本人でさえ自覚することが難しく、それがトレーニングのモティベーション低下につながることさえありましたが、この研究によって本人はもちろん、第三者も学習者のリスニング能力を外から客観的に可視化することができるようになりました。今後は、語学や音楽などのさまざまな分野で聞き分け能力の客観計測が進むものと期待しています。」と述べました。また、研究メンバーである南哲人教授は「今回の発見は、瞳孔反応に純音のような単純な音だけでなく、発話音声の違いといった高次な要因が反映されることを示しています。瞳孔散大を外部からコントロールすることによって、聞き分け能力の向上につながるのであれば、英語学習手法としても有用であるのではないかと期待しています。」と説明します。

研究チームは、今回明らかにした研究成果から瞳孔反応から英語聞き分け能力を推定する新たな手法は、日本人が苦手とするLとRの聞き分けを効率よく学習できるシステムの構築につながる可能性を示しています。また、母国語に存在しない音の聞き分けによって生じる学習の障害は、英語話者が中国語を学習する際などにも起こることが知られており、最終的には日本語に限定しない言語能力を推定する新たな指標になればと考えています。加えて、本研究成果は英単語に注意を向けたり、参加者が応答したりする必要がないため、運動障害や発話障害を持つ患者の言語学習に役立っていくことが期待されます。

本研究は文部科学省・日本学術振興会科学研究費基盤A(26240043)及び基盤研究B(17H01807)の助成を受けて実施されました。

Share this story

Researcher Profile

Yuya Kinzuka
Name Yuya Kinzuka
Affiliation Department of Computer Science and Engineering
Title PhD student
Fields of Research Neuroscience, Psychophysiology, Pupillometry

ページトップへ