TUT Research

Font Size


HOME > No.34, Dec. 2023 > Diagnosis of voice condition from call audio

Diagnosis of voice condition from call audio

Toward the development of a dysarthria diagnostic system in telemedicine Yuya Hosoda
Yuya Hosoda

Assistant Professor Yuya Hosoda of the Center for IT-Based Education (CITE), Toyohashi University of Technology developed a method for estimating the pitch of vocal cord vibrations of humans from call audio. In this method, the pitch is estimated by integrating the feature quantities extracted from the amplitude and phase spectra of speech on the complex plane. Through experiments, we have demonstrated that the proposed method is not only efficient for calls where the frequency band is restricted by communication standards, but also works robustly in an environment with background noise.

Early diagnosis of dysarthria, which is an early symptom of neurodegenerative illnesses such as Parkinson's disease, can help to reduce the severity of such diseases. Dysarthria is characterized by tremors in the voice and disturbed breathing. Although clinical tests diagnose symptoms from the patient's voice, they are time consuming and labor intensive. Additionally, conducting face-to-face interviews in remote locations such as mountainous areas is difficult. Therefore, in this research, we aim to develop a system that automatically diagnoses dysarthria through telemedicine by performing ward rounds via communication devices.

In patients with dysarthria, abnormalities occur during vocalization wherein the voice is produced by vocal cord vibrations generated by air released from the lungs in the throat and oral cavity. In this study, our purpose is to estimate the vibration period (pitch) to diagnose the condition of these vocal cord vibrations. Prior to this new technique, pitch measurement methods that were robust against background noise were devised based on features related to the amplitude spectrum obtained by frequency analysis of speech. However, due to communication standards, call audio via telemedicine may lack some of the desired amplitude spectrum. Extracting feature quantities from an amplitude spectrum in such cases can lead to errors in pitch estimation.

In this research, we propose a method to extract additional feature quantities from the phase spectrum, a by-product of frequency analysis, in addition to the amplitude spectrum. Deriving a relational equation between the phase shift and pitch in the time and frequency directions, we have verified that pitch can be estimated by applying the observed phase shift to the relational equation. Based on this finding, we extracted new feature quantities from the phase spectrum to quantitatively evaluate the degree of fit to the relational equation. Finally, by integrating the feature quantities extracted from the amplitude spectrum on the complex plane, we compensated for the lack of feature quantities occurring in the pitch estimation of call audio while maintaining robustness against background noise.

Pitch estimation results for call audio. The red and white lines indicate the measured and estimated values, respectively. (a) Prior studies (b) Proposed method
Pitch estimation results for call audio.
The red and white lines indicate the measured and estimated values, respectively.
(a) Prior studies (b) Proposed method

The figure shows the pitch estimation results for a call audio. In previous studies that used only the amplitude spectrum, since the amount of information was reduced by band limitation, the pitch was estimated to be higher than the original value. However, in the proposed method, the pitch is accurately estimated from call audio using the feature quantities related to the amplitude and phase spectra. Further, the gross pitch error (GPE), an evaluation index that indicates the percentage of segments where errors occurred, improved to 9.5% in the proposed method, compared to 42.2% in the previous study. In addition, even for call audio with background noise, this method achieved a GPE of 15.2%, demonstrating robustness.

Although this study focused on pitch estimation to detect abnormalities in vocal cord vibrations, respiratory and oral abnormalities also cause dysarthria. To detect these symptoms, methods that extract feature quantities from the amplitude spectrum have been devised. However, the use of the phase spectrum has not been sufficiently validated. In the future, we will work on extracting relevant feature quantities from the phase spectra for the other cases as well. Further, by comprehensively analyzing these feature quantities, we aim to develop a dysarthria diagnostic system that can function effectively with telemedicine.


Yuya Hosoda, Arata Kawamura and Youji Iiguni, "Complex-Domain Pitch Estimation Algorithm for Narrowband Speech Signals," IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2067-2078, 2023.


遠隔医療における構音障害診断システムの開発に向けて 細田 侑也





図は、通話音声に対するピッチ推定結果を表します。振幅スペクトルのみを使用する先行研究では、帯域制限で情報量が削減されているため、本来よりも高くピッチを推定しました。一方で提案法では、振幅及び位相スペクトルに関する特徴量を用いて通話音声から正確にピッチを推定しました。また、誤差が生じた区間の割合を表す評価指標 Gross Pitch Error (GPE)では、先行研究の42.2%に対して、提案法は9.5%まで改善しました。加えて、背景雑音が付加された通話音声に対しても15.2%を達成して頑健性を示しました。


Share this story

Researcher Profile

Yuya Hosoda
Name Yuya Hosoda
Affiliation Center for IT-Based Education (CITE)
Title Assistant Professor
Fields of Research Speech Signal Processing

Page Top