特征选择——互信息量

衡量单变量的相关性指标有很多，比如Pearson相关系数、Pearson卡方检验、Fisher得分、互信息等。

目录

信息量
信息熵
条件熵
互信息量
用于特征选择(Python)OC

信息量

信息熵

条件熵

互信息量

用于特征选择(Python)

import pandas as pd
import numpy as np

def mutual_infor(X, y):
	'''
	Mutual Information
	X and y are both categorical variables.

	Input : {
	X : one-dimensional array、list or series from Pandas
	y : one-dimensional array、list or series from Pandas
	}
	
	'''
	X = np.array(X).reshape(-1)
	y = np.array(y).reshape(-1)

	if len(X) != len(y):
		print('Length of X and y are inconsistent !')

	X_level = list(set(X))
	y_level = list(set(y))
	N = X.shape[0]
	I = 0

	for i in X_level:
		for j in y_level:
			p_xy = np.sum(X == i) & (y == j) / N
			p_x = np.sum(X == i) /N
			p_y = np.sum(y == j) /N
			I += p_xy * np.log(p_xy / (p_y * p_x))

	return I
	```
————————————————
版权声明：本文为CSDN博主「朱聪明着呢～」的原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/weixin_43854177/article/details/102813039