自信息量部分
I ( x i ) = l o g 2 1 p ( x i ) = − l o g 2 p ( x i ) I ( x i , y i ) = − l o n 2 p ( x i y i ) , X , Y 独 立 时 = I ( x i ) + I ( y i ) = − l o g 2 p ( x i ) − l o g 2 p ( y i ) I ( x i ∣ y i ) = − l o g 2 p ( x i ∣ y i ) I ( x i y i ) = − l o g 2 p ( x i y i ) = − l o g 2 ( p ( y i ∣ x i ) × p ( x i ) p ( x i ) ) = − l o g 2 p ( y i ∣ x i ) p ( x i ) = I ( y i ∣ x i ) + I ( x i ) = I ( x i ∣ y i ) + I ( y i ) I ( x i ; y i ) = I ( x i ) − I ( x i ∣ y i ) = l o g 2 p ( x i ∣ y i ) p ( x i ) = I ( y i ; x i ) I(x_i)=log_2{\frac{1}{p(x_i)}=-log_2p(x_i)}\\ I(x_i,y_i)=-lon_2p(x_iy_i),X,Y独立时\ =I(x_i)+I(y_i)=-log_2p(x_i)-log_2p(y_i)\\ \\ \\ I(x_i|y_i)=-log_2p(x_i|y_i)\\ I(x_iy_i)\\=-log_2p(x_iy_i)\\=-log_2(\frac{p(y_i|x_i)\times p(x_i)}{p(x_i)})\\=-log_2p(y_i|x_i)p(x_i)\\=I(y_i|x_i)+I(x_i)\\ =I(x_i|y_i)+I(y_i)\\ \\ I(x_i;y_i)=I(x_i)-I(x_i|y_i)\\=log_2\frac{p(x_i|y_i)}{p(x_i)}=I(y_i;x_i)\\ I(xi)=log2p(xi)1=−log2p(xi)I(xi,yi)=−lon2p(xiyi),X,Y独立时 =I(xi)+I(yi)=−log2p(xi)−log2p(yi)I(xi∣yi)=−log2p(xi∣yi)I(xiyi)=−log2p(xiyi)=−log2(p(xi)p(yi∣xi)×p(xi))=−log2p(yi∣xi)p(xi)=I(yi∣xi)+I(xi)=I(xi∣yi)+I(yi)I(xi;yi)=I(xi)−I(xi∣yi)=log2p(xi)p(xi∣yi)=I(yi;xi)
熵部分
H ( X ) = E [ I ( x ) ] = − ∑ A l l p ( x ) ⋅ l o g 2 p ( x ) H ( X ) ≤ l o g 2 n H ( X Y ) = − ∑ ∑ E a c h p a i r p ( x y ) l o g 2 p ( x y ) ≤ H ( X ) + H ( Y ) , X , Y 统 计 独 立 时 等 号 成 立 H ( X ∣ Y ) = − ∑ ∑ E a c h p a i r p ( x y ) ⋅ l o g 2 p ( x ∣ y ) H ( X Y ) = − ∑ p ( x y ) ⋅ l o g 2 p ( x y ) = H ( Y ) + H ( X ∣ Y ) H ( X ∣ Y ) ≤ H ( X ) , 统 计 独 立 时 等 号 成 立 . H(X)=E[I(x)]=-\sum_{All}p(x)\cdot log_2p(x)\\ H(X)\leq log_2 n H(XY)=-\sum\sum_{Each\ pair} p(xy)log_2p(xy)\leq H(X)+H(Y),X,Y统计独立时等号成立\\ H(X|Y)=-\sum\sum_{Each\ pair} p(xy)\cdot log_2p(x|y)\\ H(XY)=-\sum p(xy)\cdot log_2 p(xy)\\= H(Y)+H(X|Y)\\ H(X|Y)\leq H(X),统计独立时等号成立.\\ H(X)=E[I(x)]=−All∑p(x)⋅log2p(x)H(X)≤log2nH(XY)=−∑Each pair∑p(xy)log2p(xy)≤H(X)+H(Y),X,Y统计独立时等号成立H(X∣Y)=−∑Each pair∑p(xy)⋅log2p(x∣y)H(XY)=−∑p(xy)⋅log2p(xy)=H(Y)+H(X∣Y)H(X∣Y)≤H(X),统计独立时等号成立.
平均互信息量
I ( X ; Y ) = E ( I ( x ; y ) ) = ∑ ∑ p ( x y ) I ( x ; y ) = H ( X ) + H ( Y ) − H ( X Y ) = − ( ∑ p ( x ) ⋅ l o g 2 p ( x ) ) − ( ∑ p ( y ) ⋅ l o g 2 p ( y ) ) + ∑ p ( x y ) ⋅ l o g 2 p ( x y ) I(X;Y)=E(I(x;y))=\sum\sum p(xy)I(x;y)\\=H(X)+H(Y)-H(XY)\\=-(\sum p(x)\cdot log_2p(x))-(\sum p(y)\cdot log_2p(y))+\sum p(xy)\cdot log_2p(xy)\\ I(X;Y)=E(I(x;y))=∑∑p(xy)I(x;y)=H(X)+H(Y)−H(XY)=−(∑p(x)⋅log2p(x))−(∑p(y)⋅log2p(y))+∑p(xy)⋅log2p(xy)
密码体制中的熵
- ∣ P ∣ ≤ ∣ C ∣ |P|\leq|C| ∣P∣≤∣C∣
- H ( P ∣ K C ) = H ( C ∣ K P ) = 0 H(P|KC)=H(C|KP)=0 H(P∣KC)=H(C∣KP)=0
- H ( P K ) = H ( P ) + H ( K ) H(PK)=H(P)+H(K) H(PK)=H(P)+H(K) (明文和密钥无统计关系)
- H ( K ∣ C ) = H ( P ) + H ( K ) − H ( C ) H(K|C)=H(P)+H(K)-H(C) H(K∣C)=H(P)+H(K)−H(C)
- H ( P ∣ C ) ≤ H ( K ∣ C ) H(P|C)\leq H(K|C) H(P∣C)≤H(K∣C)
- i f s i z e o f ( P ) = s i z e o f ( C ) , 且 P 随 机 等 概 率 分 布 , C 一 定 随 机 等 概 率 分 布 , H ( K ∣ C ) = H ( K ) if sizeof(P)=sizeof(C), 且P随机等概率分布,C一定随机等概率分布,H(K|C)=H(K) ifsizeof(P)=sizeof(C),且P随机等概率分布,C一定随机等概率分布,H(K∣C)=H(K)
完善密码体制下:
- H ( P ∣ C ) = H ( P ) H(P|C)=H(P) H(P∣C)=H(P)
- ∣ P ∣ ≤ ∣ C ∣ ≤ ∣ K ∣ ( P ( y ∣ x ) = p ( y ) ) |P|\leq|C|\leq|K|\ (P(y|x)=p(y)) ∣P∣≤∣C∣≤∣K∣ (P(y∣x)=p(y))
- H ( P ) ≤ H ( C ) ≤ H ( K ) H(P)\leq H(C)\leq H(K) H(P)≤H(C)≤H(K)
扫一扫
3万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
