ig信息增益 java_Python信息增益实现

我目前正在使用scikit learn对20ng数据集进行文本分类。我想计算矢量化数据集的信息增益。对我来说，{a1}可以使用sklearn的mutual_info_classif来完成。但是，这个方法非常慢，所以我试图基于这个post实现自己的信息增益。在

我想出了以下解决方案：from scipy.stats import entropy

import numpy as np

def information_gain(X, y):

def _entropy(labels):

counts = np.bincount(labels)

return entropy(counts, base=None)

def _ig(x, y):

# indices where x is set/not set

x_set = np.nonzero(x)[1]

x_not_set = np.delete(np.arange(x.shape[1]), x_set)

h_x_set = _entropy(y[x_set])

h_x_not_set = _entropy(y[x_not_set])

return entropy_full - (((len(x_set) / f_size) * h_x_set)

+ ((len(x_not_set) / f_size) * h_x_not_set))

entropy_full = _entropy(y)

f_size = float(X.shape[0])

scores = np.array([_ig(x, y) for x in X.T])

return scores

使用一个非常小的数据集，sklearn和我的实现中的大多数分数是相等的。然而，sklearn似乎考虑了频率，而我的算法显然没有考虑

^{pr2}$

样本输出：center: mi=0.011824, ig=0.003548

christian: mi=0.128629, ig=0.127122

color: mi=0.028413, ig=0.026397

com: mi=0.041184, ig=0.030458

computer: mi=0.020590, ig=0.012327

cs: mi=0.007291, ig=0.001574

data: mi=0.020734, ig=0.008986

did: mi=0.035613, ig=0.024604

different: mi=0.011432, ig=0.005492

distribution: mi=0.007175, ig=0.004675

does: mi=0.019564, ig=0.006162

don: mi=0.024000, ig=0.017605

earth: mi=0.039409, ig=0.032981

edu: mi=0.023659, ig=0.008442

file: mi=0.048056, ig=0.045746

files: mi=0.041367, ig=0.037860

ftp: mi=0.031302, ig=0.026949

gif: mi=0.028128, ig=0.023744

god: mi=0.122525, ig=0.113637

good: mi=0.016181, ig=0.008511

gov: mi=0.053547, ig=0.048207

所以我想知道我的实现是错误的，还是正确的，但是scikit learn使用的互信息算法的不同变体。在