【audio evaluation】objective evaluation

MCD

mel cepstral distortion (MCD) ：
$$MC{D}_K=\frac{1}{T}\sum _{t=0}^{T-1}\sqrt{\sum _{k=1}^K{\left(c_{t,k}-c_{t,k}^{\prime }\right)}^2}$$
where $c_{t,k}$ and $c_{t,k}^{\prime }$ are the k-th mel frequency cepstral coefficient (MFCC) of the t-th frame from the ground-truth and converted utterance, respectively. In addition, $T$ and $K$ are the frame length and number of coefficients, respectively.

F0RMSE

F0 root mean squared error (RMSE)
$$F_{0}RMSE=\sqrt{\frac{1}{T}\sum _{i=1}^T{\left(f_i-f_i^{\prime }\right)}^2}$$
where $f_i$ and $f_i^{\prime }$ are the fundamental frequencies of the i-th frame from the ground-truth and converted utterance, respectively.
A lower MCD and $F_0$ RMSE refer to the smaller distortion between the two utterances.

Inference Speed

Testing the inference that a certain number of samples takes time.

speaker classification accuracy

speaker classification accuracy, a network trained to predict the speakers.

Word Error Rate (WER)

automatic speech recognition (ASR) test to calculate the word error rate (WER).can use the Google Speech-to-Text API for the ASR model

character error rate (CER)

evaluate linguistic consistency
automatic speaker verification (ASV)
automatic speaker verifification (ASV) to compute the equal error rate from (EER) the sample pairs from the target and converted speech.

PCC (Pearson correlation coeffificient)

PCC (Pearson correlation coeffificient) was used to calculate prosody consistency

Speaker Similarity score

use a speaker verification system based on I-vectors, which formulate the speaker verification problem in the total variability space
After length normalization, cosine distance scoring is used for i-vector modeling.

Cosine distance

speaker verification (SV) system to evaluate the similarity between the converted speech and the target speakers’ speech.

Reference：
Duration Controllable Voice Conversion via Phoneme-Based Information Bottleneck
FREEVC: TOWARDS HIGH-QUALITY TEXT-FREE ONE-SHOT VOICE CONVERSION
Overview of Voice Conversion Methods Based on Deep Learning