说话人识别matlab实现

说话人识别

这里，博主对说话人两个baseline模型应该matlab的MSR工具箱进行处理。

1、GMM-UBM说话人识别

这里主要分为4个步骤：

1、训练UBM通用背景模型

2、最大后验准则MAP从UBM通用背景模型里面训练每一个说话人的声学模型

3、交叉得分

4、计算最终的测试效果，这里用AUC和EER表示，可以方便与最近的深度学习方法做比较。

具体程序

设置环境参数：说话人有20个。每一帧的维度为13，这里可以根据MFCC的维度进行修改。一般语音数据都是单信道，这里可以对信道进行设置，本实验的信道为10。

nSpeakers = 20;
nDims = 13;             % dimensionality of feature vectors
nMixtures = 32;         % How many mixtures used to generate data
nChannels = 10;         % Number of channels (sessions) per speaker
nFrames = 100;         % Frames per speaker (10 seconds assuming 100 Hz)
nWorkers = 1;           % Number of parfor workers, if available

这里为了方便不用一般的语音数据库如TIMIT，直接生产随机多信道的音频数据（10信道）。这里trainSpeakerData和testSpeakerData为20*10的cell，20为说话人的个数，10为说话人的信道数。每个说话人在训练和测试集里面是一一对应的。在每一个cell里面维度为13*100,13为分帧之后的维度，100位帧数，在实际中分帧后的语音都会经过MFCC特征提取。

% Pick random centers for all the mixtures.
mixtureVariance = .10;
channelVariance = .05;
mixtureCenters = randn(nDims, nMixtures, nSpeakers);
channelCenters = randn(nDims, nMixtures, nSpeakers, nChannels)*.1;
trainSpeakerData = cell(nSpeakers, nChannels);
testSpeakerData = cell(nSpeakers, nChannels);
speakerID = zeros(nSpeakers, nChannels);

% Create the random data. Both training and testing data have the same
% layout.
disp('Create the random data');
for s=1:nSpeakers
    trainSpeechData = zeros(nDims, nMixtures);
    testSpeechData = zeros(nDims, nMixtures);
    for c=1:nChannels
        for m=1:nMixtures
            % Create data from mixture m for speaker s
            frameIndices = m:nMixtures:nFrames;
            nMixFrames = length(frameIndices);
            trainSpeechData(:,frameIndices) = ...
                randn(nDims, nMixFrames)*sqrt(mixtureVariance) + ...
                repmat(mixtureCenters(:,m,s),1,nMixFrames) + ...
                repmat(channelCenters(:,m,s,c),1,nMixFrames);
            testSpeechData(:,frameIndices) = ...
                randn(nDims, nMixFrames)*sqrt(mixtureVariance) + ...
                repmat(mixtureCenters(:,m,s),1,nMixFrames) + ...
                repmat(channelCenters(:,m,s,c),1,nMixFrames);
        end
        trainSpeakerData{s, c} = trainSpeechData;
        testSpeakerData{s, c} = testSpeechData;
        speakerID(s,c) = s;                 % Keep track of who this is
    end
end

现在正式进入GMM-UBM阶段：

训练UBM通用背景模型，UBM也可以理解成混合高斯模型，说白了就是多个告诉模型的加权和。它的作用可以在说话人语料不足的情况下，依据UBM模型自适应得到集内说话人的模型。我们对高斯模型进行参数估计，会得到一个ubm的结构体，里面包含了每个说话人的权值、mu、sigma。

% Step1: Create the universal background model from all the training speaker data
disp('Create the universal background model');
nmix = nMixtures;           % In this case, we know the # of mixtures needed
final_niter = 10;
ds_factor = 1;
ubm = gmm_em(trainSpeakerData(:), nmix, final_niter, ds_factor, nWorkers);

最大后验准则MAP从UBM通用背景模型里面训练每一个说话人的声学模型，自适应的策略是根据目标说话人的训练集trainSpeakerData特征矢量与第一步求得的UBM的相似程度，将UBM的各个高斯分量按训练集特征矢量进行调整，从而形成目标说话人的声学模型。再根据EM重估公式，计算每一个说话人修正模型的最优参数。

% Step2: Now adapt the UBM to each speaker to create GMM speaker model.
disp('Adapt the UBM to each speaker');
map_tau = 10.0;
config = 'mwv';
gmm = cell(nSpeakers, 1);
for s=1:nSpeakers
    disp(['for the ',num2str(s),' speaker...']);
    gmm{s} = mapAdapt(trainSpeakerData(s, :), ubm, map_tau, config);
end

计算每个说话人模型的得分。因为在说话人确认系统中，与说话人辨认不同，测试目标testSpeakerData变为确认某段测试语音是否来源于某个目标说话人，本实验为20个说话人。如果测试语音与目标语音来源于相同的说话人，则此次测试为目标测试(target test)；反之，如果测试语音与目标语音来源与不同的说话人，则此次测试为非目标测试(non-target test)。将目标测试与非目标测试的后验概率比作为得分。

% Step3: Now calculate the score for each model versus each speaker's data.
% Generate a list that tests each model (first column) against all the
% testSpeakerData.
trials = zeros(nSpeakers*nChannels*nSpeakers, 2);
answers = zeros(nSpeakers*nChannels*nSpeakers, 1);
for ix = 1 : nSpeakers,
    b = (ix-1)*nSpeakers*nChannels + 1;
    e = b + nSpeakers*nChannels - 1;
    trials(b:e, :)  = [ix * ones(nSpeakers*nChannels, 1), (1:nSpeakers*nChannels)'];
    answers((ix-1)*nChannels+b : (ix-1)*nChannels+b+nChannels-1) = 1;
end
disp('Calculate the score for each model vs test speaker');
gmmScores = score_gmm_trials(gmm, reshape(testSpeakerData', nSpeakers*nChannels,1), trials, ubm);

计算指标AUC和EER。对于开集的说话人辨认系统，需要将测试语音的输出得分与特定的阈值进行比较，以做出是否是集外说话人的判决。对于说话人确认系统，需要对测试语音的输出得分进行判决，一般是将其与一特定的阈值进行比较，若大于此阈值则接受其为目标说话人，否则判定其为冒认说话人。因而，阈值的选取对说话人识别系统的性能有着直接的影响，尤其是在实用的说话人识别系统研究中，阈值选取问题更是得到了研究者们的广泛关注，提出了许多有效的阈值选取方法，其中比较常用的有等错误率(equal error rate，EER)阈值。这里，博主加入了AUC，可以方便与深度学习方法做对比。

% Step4: Now compute the EER and plot the DET curve and confusion matrix
imagesc(reshape(gmmScores,nSpeakers*nChannels, nSpeakers))
title('Speaker Verification Likelihood (GMM Model)');
ylabel('Test # (Channel x Speaker)'); xlabel('Model #');
colorbar; drawnow; axis xy
figure
disp('Compute the EER');
[eer,auc] = compute_eer(gmmScores, answers, true);

2、基于ivector的GMM-UBM说话人识别

基于ivector的GMM-UBM模型是最近比较流行的baseline方法，这篇 博客说得比较详细。这里就不再啰嗦地说明了。具体实现步骤为

1、训练UBM通用背景模型

2、计算通用背景模型的总变化空间

3、训练Gaussian 概率线性判别PLDA模型，这样可以极大程度地提高ivector对说话人识别的影响

4、交叉得分

5、计算最终的测试效果，这里用AUC和EER表示，可以方便与最近的深度学习方法做比较。

其具体部分程序如下：

% Step1: Create the universal background model from all the training speaker data
nmix = nMixtures;           % In this case, we know the # of mixtures needed
final_niter = 10;
ds_factor = 1;
ubm = gmm_em(trainSpeakerData(:), nmix, final_niter, ds_factor, nWorkers);


%%
% Step2.1: Calculate the statistics needed for the iVector model.
stats = cell(nSpeakers, nChannels);
for s=1:nSpeakers
    for c=1:nChannels
        [N,F] = compute_bw_stats(trainSpeakerData{s,c}, ubm);
        stats{s,c} = [N; F];
    end
end

% Step2.2: Learn the total variability subspace from all the speaker data.
tvDim = 100;
niter = 5;
T = train_tv_space(stats(:), ubm, tvDim, niter, nWorkers);
%
% Now compute the ivectors for each speaker and channel.  The result is size
%   tvDim x nSpeakers x nChannels
devIVs = zeros(tvDim, nSpeakers, nChannels);
for s=1:nSpeakers
    for c=1:nChannels
        devIVs(:, s, c) = extract_ivector(stats{s, c}, ubm, T);
    end
end

%%
% Step3.1: Now do LDA on the iVectors to find the dimensions that matter.
ldaDim = min(100, nSpeakers-1);
devIVbySpeaker = reshape(devIVs, tvDim, nSpeakers*nChannels);
[V,D] = lda(devIVbySpeaker, speakerID(:));
finalDevIVs = V(:, 1:ldaDim)' * devIVbySpeaker;

% Step3.2: Now train a Gaussian PLDA model with development i-vectors
nphi = ldaDim;                  % should be <= ldaDim
niter = 10;
pLDA = gplda_em(finalDevIVs, speakerID(:), nphi, niter);

%%
% Step4.1: OK now we have the channel and LDA models. Let's build actual speaker
% models. Normally we do that with new enrollment data, but now we'll just
% reuse the development set.
averageIVs = mean(devIVs, 3);           % Average IVs across channels.
modelIVs = V(:, 1:ldaDim)' * averageIVs;


% Step4.2: Now compute the ivectors for the test set 
% and score the utterances against the models
testIVs = zeros(tvDim, nSpeakers, nChannels); 
for s=1:nSpeakers
    for c=1:nChannels
        [N, F] = compute_bw_stats(testSpeakerData{s, c}, ubm);
        testIVs(:, s, c) = extract_ivector([N; F], ubm, T);
    end
end
testIVbySpeaker = reshape(permute(testIVs, [1 3 2]), ...
                            tvDim, nSpeakers*nChannels);
finalTestIVs = V(:, 1:ldaDim)' * testIVbySpeaker;

3、参考文献

[1] N. Dehak, P. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, "Front-end factor analysis for speaker verification," IEEE TASLP, vol. 19, pp. 788-798, May 2011. 
[2] P. Kenny, "A small footprint i-vector extractor," in Proc. Odyssey,  The Speaker and Language Recognition Workshop, Jun. 2012.

[3] D. Matrouf, N. Scheffer, B. Fauve, J.-F. Bonastre, "A straightforward and efficient implementation of the factor analysis model for speaker verification," in Proc.         INTERSPEECH, Antwerp, Belgium, Aug. 2007, pp. 1242-1245.