信息论Matlab仿真——信源熵

最新推荐文章于 2024-09-15 22:35:33 发布

Kevin&1123

最新推荐文章于 2024-09-15 22:35:33 发布

阅读量4.8k

点赞数 13

文章标签： matlab

本文链接：https://blog.csdn.net/zsl710/article/details/117953425

版权

本文深入探讨了信息论中的信源熵、条件熵、联合熵、平均互信息量等概念，并通过多个计算实例进行详细解析，如离散信源熵的最大值、马尔可夫信源熵等。同时，介绍了加权熵和交互熵的计算方法，为理解信息传输的不确定性提供了理论基础。

摘要由CSDN通过智能技术生成

《信息论与编码》第二章——信源熵的计算

%@author Zhang
%@copyright Zhang
%@release 2021.06.16

%信源熵：信源的平均不确定度
%性质：①非负性 ②对称性 ③最大离散熵定理：信源X中包括n个不同的离散信息，则HX <= log2(n) 
% PX = [1/2,1/4,1/8,1/8];
% HX = -PX * log2(PX');
%条件熵
% Pxy = [1/8,3/8;3/8,1/8];%联合概率
% PY = sum(Pxy,2);%按行求和
% PX = sum(Pxy,1);%按列求和
% PY = repmat(PY,[1,2]);%进行行扩展
% PX = repmat(PX,[2,1]);%进行列扩展
% PYx = Pxy ./ PX;
% PXy = Pxy ./ PY;
% HXy = -sum(sum(Pxy .* log2(PXy)));
%联合熵
% Hxy = -sum(sum(Pxy .* log2(Pxy)));
%加权熵
% w = [1,2,3,4];%权值
% HwX = -(w .* PX) * log2(PX');
%平均互信息量/交互熵
% PXY = PXy ./ PX; 
% PYX = PYx ./ PY; 
% IXY = sum(sum(Pxy .* log2(PXY)));
% IYX = sum(sum(Pxy .* log2(PYX)));

%例2.1.5：
% PX = [0.5,0.5];
% PX = repmat(PX,[2,1]);
% PYx = [0.98,0.2;0.02,0.8];
% Pxy = PYx .* PX;
% PY = sum(Pxy,2);%按行求和
% PY = repmat(PY,[1,2]);
% PXy = Pxy ./ PY;
% %平均互信息量I(X;Y)
% P = PXy ./ PX;
% IXy = sum(sum(Pxy .* log2(P)));
% %疑义度H(X/Y)
% HXy = -sum(sum(Pxy .* log2(PXy)));
% %噪声熵H(Y/X)
% HYx = -sum(sum(Pxy .* log2(PYx)));
% %联合熵H(XY)
% HXY = -sum(sum(Pxy .* log2(Pxy)));

%例2.1.7
% PX = [0.5,0.5];
% PXy = [0.8,0.2;0.2,0.8];
% PYz = [0.9,0.1;0.1,0.9];
% PY = inv(PXy) * PX';
% Pxy = PXy .* repmat(PY,[1,2]);
% PZ = inv(PYz) * PY;
% Pyz = PYz .* repmat(PZ,[1,2]);
% PXY = PXy ./ repmat(PX,[2,1]);
% PYZ = PYz ./ repmat(PY,[1,2]);
% IXY = sum(sum(Pxy .* log2(PXY)));
% IYZ = sum(sum(Pyz .* log2(PYZ)));
% HX = -sum(sum(PX .* log2(PX)));
% HZ = -sum(sum(PZ .* log2(PZ)));
% Pxz = PXy * Pyz;
% Hxz = -sum(sum(Pxz .* log2(Pxz)));
% IXZ = HX+HZ-Hxz;

%例2.2.4 马尔可夫信源熵
% P = [0.8,0.2,eps,eps;eps,eps,0.5,0.5;0.5,0.5,eps,eps;eps,eps,0.2,0.8];
% A = eye(size(P)) + [zeros(size(P,2)-1,size(P,1));ones(1,size(P,2))]-P';
% B = [zeros(size(P,1)-1,1);1];
% PS = inv(A) * B;
% HS = -sum(sum(repmat(PS,[1,4]) .* P .* log2(P)));

%二次扩展信源熵
% syms a1 a2 a3;
% a1 = 1/2;
% a2 = 1/4;
% a3 = 1/4;
% PX2 = [a1,a2,a3];
% PX2 = kron(PX2,PX2);
% HX=-PX2*log2(PX2)';