Pitch, depth, and codecs analysis
To guage the standard, similarities, and variations between artificial and pure voices of ALS patients, we carried out a complete evaluation utilizing a number of key vocal biomarkers: pitch, depth, and formants (F1 and F2). Determine 2 illustrates the extraction of vocal options from each pure and artificial voices utilizing HiFi-GAN.
Extraction of audio waveforms, spectrograms, pitch, and depth options from each pure and artificial voices utilizing HiFi-GAN throughout numerous check dataset scripts. The analyzed sentences embrace: U1 – Select what to do as soon as in Naples. U2 – Please, activate the TV for me. U3 – Would you thoughts if I got here too? U4 – I used to be considering we might lease a automotive as soon as in Rome. U5 – The cheeks had been flat and never protruding. U6 – Let’s return to the earlier web page for a second. U7 – Select what to do as soon as in Naples, come on. U8 – Have you ever already packed your luggage? U9 – The person acknowledged the significance of the discovering. U10 – I would like an ochre-colored wool sweater. These sentences, chosen for their similarities in common sentence size, character rely, content material, and linguistic options, display the generalizability and robustness of the extracted options throughout each pure and artificial speech.
The pitch values of artificial voices carefully matched these of the pure voices of ALS patients, indicating that the expressive pitch high quality of the pure human voice was well-preserved even beneath pathological circumstances (Fig. 3). Particularly, pure voices exhibited a median pitch of 155 ± 33 Hz, whereas artificial voices generated by HiFi-GAN displayed a median pitch of 155.20 ± 35 Hz (Fig. 3A). The proportion distinction in pitch between pure and artificial voices, calculated utilizing the next components:(Diff:left(%proper)=:frac{pitch:SV-pitch:RV}{pitch:RV}:instances::100)
confirmed a variation vary of ± 5%, with a median distinction of 1.94% (Fig. 3AI), demonstrating a greater pitch preservation in comparison with the research reported by Yamagishi et al.27.
For depth, pure voices had a median depth of 60.93 ± 5.39 dB, whereas artificial voices demonstrated the next common depth of 68.29 ± 4.04 dB, leading to a proportion distinction of 12.84% (Fig. 3B-BI). This discovering is especially vital because it means that regardless of the presence of dysarthria in ALS patients, the HiFi-GAN mannequin can generate artificial voices with enhanced depth in comparison with pure voices.
Such information reveals that the HiFi-GAN mannequin successfully replicates the pitch of pure voices whereas enhancing the depth, probably enhancing voice high quality for ALS patients. This might have vital implications for preserving sure points of expressiveness in ALS patients’ voices, because the artificial voices replicate the pitch high quality of their pure voices. Nonetheless, whereas pitch is vital for sustaining components of emotional expression and vocal id, we acknowledge that emotional expressiveness additionally is determined by context and different components, reminiscent of tone, rhythm, and interplay dynamics. As well as, the elevated depth in artificial voices might enhance the audibility and readability of speech, which is usually compromised in ALS patients as a consequence of dysarthria and aphasia.
(A) Comparability of pure and artificial voices in ALS patients exhibits that HiFi-GAN can carefully replicate pure voice pitch (AI) with a minimal common distinction of 1.94%. (B) Artificial voices generated by HiFi-GAN exhibit greater common depth in comparison with pure voices, (BI) leading to a 12.84% improve.
Subsequent, we centered on analyzing the F1 and F2 formants, as illustrated in Fig. 4. For this evaluation, we particularly focused the vowel ‘a’ extracted from phrases the place this vowel was positioned between two consonants. The chosen vowel parts had a median length of 100 to 150 milliseconds, making certain constant and dependable comparisons between pure and artificial voice samples.
The common F1 values confirmed outstanding similarity between pure and artificial voices, with averages of 774.71 Hz and 776.02 Hz, respectively (Fig. 4A). The proportion distinction between the 2 datasets fluctuated inside ± 10%, indicating a excessive diploma of correspondence within the decrease formant frequencies (Fig. 4AI).
For the F2 formant, the typical values had been 1485.68 Hz for pure voices and 1496.67 Hz for artificial voices (Fig. 4B). The proportion distinction between the 2 was minimal, at ± 0.62%, demonstrating that the HiFi-GAN mannequin precisely replicates the upper formant frequencies as nicely (Fig. 4BI).
Comparability of decrease (F1) and better (F2) formant frequencies between pure and artificial voices. (A) The common F1 values for pure voices had been 774.71 Hz, whereas artificial voices averaged 776.02 Hz, (A1) exhibiting a minimal proportion distinction inside ± 10%. (B) For the F2 formant, pure voices averaged 1485.68 Hz and artificial voices averaged 1496.67 Hz, (B1) with a minimal distinction of ± 0.62%.
The shut match in each F1 and F2 formant frequencies between pure and artificial voices underscores the effectiveness of the HiFi-GAN mannequin in capturing the acoustic traits of human speech. That is significantly vital as a result of formants play a vital position in vowel articulation and total speech intelligibility. The flexibility to duplicate these formants precisely ensures that the artificial voices are usually not solely comparable in pitch and depth but additionally of their phonetic particulars, contributing to a extra pure and understandable artificial speech output.
Moreover, such outcomes confirmed an enchancment in formant matching accuracy in comparison with the findings of Creer et al.28 who reported a deviation of ± 15%, highlighting an enhancement in efficiency.
Total, the detailed evaluation of F1 and F2 formants reaffirms the aptitude of the HiFi-GAN mannequin to supply high-quality artificial voices that carefully resemble the pure voices of ALS patients, thereby enhancing communication effectiveness and preserving vocal id.
Lastly, we carried out a statistical evaluation of the vocal biomarkers—pitch, depth, and formants (F1 and F2)—obtained from the pure and artificial voice samples, as proven in Fig. 5. The evaluation was carried out utilizing a two-way ANOVA. Values had been expressed as means ± SD, with *p < 0.1 thought of vital.
As anticipated, there was no statistically vital distinction between the pitch values of pure and artificial voices (Fig. 5A). This confirms that the HiFi-GAN mannequin successfully preserves the pitch high quality of the unique voices, even beneath pathological circumstances. In distinction, the depth biomarker (Fig. 5B) revealed a big distinction between the pure and artificial voices (**p < 0.01). This means that whereas the artificial voices are in a position to preserve comparable pitch, they have an inclination to have greater depth ranges in comparison with pure voices.
Concerning the formants F1 (Fig. 5C) and F2 (Fig. 5D), the statistical evaluation confirmed that there have been no vital variations between the pure and artificial voice samples. This reinforces the discovering that the HiFi-GAN mannequin precisely replicates the acoustic traits of human speech, sustaining the pure formant construction.
Such outcomes underscore the proficiency of the HiFi-GAN mannequin in producing artificial voices that carefully match pure voices by way of pitch and formant frequencies, whereas exhibiting an elevated depth. These findings are vital as they display the mannequin’s potential to supply artificial speech that’s each natural-sounding and extra intense, which may benefit ALS patients by enhancing speech intelligibility and audibility, thus highlighting the developments in voice synthesis know-how for people with speech impairments.
Statistical evaluation of voice options between pure and artificial voices. (A) No statistically vital distinction in pitch values was noticed, confirming the mannequin’s potential to protect the unique pitch high quality. (B) Depth ranges confirmed a big distinction (**p < 0.01), with artificial voices exhibiting greater depth in comparison with pure voices. (C–D) Evaluation of formant frequencies F1 and F2 indicated no vital variations, demonstrating the mannequin’s accuracy in replicating pure formant buildings, highlighting HiFi-GAN’s proficiency in producing natural-sounding, intense artificial speech, which may enhance speech intelligibility for ALS patients.
Mel cepstral distance (MCD) analysis
The following step was to combine the extraction and research of the Mel Frequency Cepstral Coefficients (MFCC) and the Mel Cepstral Distance (MCD) into the evaluation of actual and artificial voices for ALS patients.
MFCCs are extensively utilized in audio sign evaluation and speech recognition29. These coefficients had been extracted from the audio sign spectrograms of the dataset samples via the next steps: (1) pre-processing—the audio sign is split into quick time segments, sometimes 20 to 40 milliseconds every; (2) fourier Remodel—every phase undergoes a Fourier remodel to acquire its frequency illustration; (3) Mel Filtering—the spectral energy values are remodeled right into a Mel frequency scale, which approximates human auditory notion; (4) Logarithmization calculated for every Mel frequency band; (5) Discrete Cosine Remodel (DCT)—The DCT is utilized to the logarithms of the facility values, producing the Mel Cepstral coefficients.
The MCD is derived from the spectral traits of the MFCC coefficients and serves as a metric to measure the gap between two units of Mel Cepstral coefficients. This distance will be calculated utilizing numerous metrics, reminiscent of Euclidean or Mahalanobis distance. MCD is especially helpful for evaluating the variety between two vocal tracks, reminiscent of a pure voice and an artificial voice. The MCD is a vital parameter for distinguishing pure voices from artificial ones as a result of it captures variations in intonation, prosody, harmonics, and different vocal attributes. By measuring the gap between the MFCC vectors of an artificial voice and a pure voice, we are able to consider the similarity or dissimilarity of the artificial voice by way of vocal traits.
Therefore, from the spectral traits of the MFCC coefficients (Fig. 6A), we extrapolated the MCD values (Fig. 6B). These values present a quantitative measure of how carefully the artificial voices generated by the HiFi-GAN mannequin resemble the pure voices of ALS patients.
The combination of MFCC and MCD into our evaluation permits for a extra nuanced evaluation of artificial voice high quality. It affords a complete analysis of how nicely the artificial voices preserve the pure traits of human speech, which is essential for growing efficient communication aids for ALS patients. The artificial voice samples generated utilizing the HiFi-GAN mannequin exhibit MCD values starting from 12.80 to 23.83, with a median of 16.23 ± 3.6 (Fig. 6B). These outcomes align with findings from different AI fashions within the literature, reminiscent of SV2TTS30, FastSpeech24, and V2C31, which report common MCD values of 17.41, 12.08, and 11.79, respectively. This comparability signifies that the HiFi-GAN mannequin performs competitively with present state-of-the-art fashions in producing high-quality artificial voices.
To additional consider the standard of generated speech, we carried out an evaluation through the use of a subjective analysis metric, following the settings reported by Chen et al.31, utilizing the Imply Opinion Rating (MOS) analysis method primarily based on subjective listening checks. The enrolled patients primarily evaluated the AI-generated speech for naturalness and similarity, specializing in whether or not the synthesized speech successfully conveyed the speaker’s id and emotional expression. As reported by Chen et al.31, so as for the standard of the generated speech to be perceived as pure, the MOS-naturalness values have to be excessive (i.e., MOS ≥ 4). Experimental outcomes present that our HiFi-GAN mannequin produces artificial voices that listeners fee as extremely just like pure speech, with a median MOS rating of 6.013 ± 0.77. Whereas MOS scores primarily assess the perceived naturalness and similarity of artificial speech, they don’t straight measure psychological influence. However, the popularity of acquainted voice qualities by ALS patients can considerably improve their well-being. Preserving a affected person’s pure voice fosters a way of private id in social interactions, reduces nervousness about future communication limitations, and helps preserve a way of autonomy as their situation progresses. This psychological assist underscores the broader worth of AI and VB applied sciences past technical assessments alone.
Spectral evaluation utilizing MFCC coefficients and MCD values for pure and artificial voices. (A) The MFCC coefficients spotlight the spectral traits of the voices. (B) MCD values, starting from 12.80 to 23.83 with a median of 16.23 ± 3.6, quantify the resemblance of artificial voices to pure voices.