文本分类学习笔记（6）- 贝叶斯

贝叶斯分类器：
先验概率P(c)= 类c下单词总数/整个训练样本的单词总数
类条件概率P(tk|c)=(类c下单词tk在各个文档中出现过的次数之和+1)/(类c下单词总数+|V|)
V是训练样本的单词表（即抽取单词，单词出现多次，只算一个），|V|则表示训练样本包含多少“个”单词。P(tk|c)可以看作是单词tk在证明d属于类c上提供了多大的证据，而P(c)则可以认为是类别c在整体上占多大比例(有多大可能性)。

注：在实际计算过程，特征维度较，单个概率较小，连乘的结果会造成精度丢失，因此采用对数函数对概率进行放大，而且不需要计算P(Doc)，即：

预测算法是根据上述公式计算argmax{P(Ci|Doc)}

#coding=utf-8
from scipy import sparse,io
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import MultinomialNB
from sklearn import metrics
from numpy import *
import warnings
warnings.filterwarnings("ignore")

class Naivebayes:
    N = []   #k * n 
    k = 0    #category
    n = 0    #samples dimensions
    m = 0   #training samples
    category = []          # 类别
    log_probability_c = []   # 属于每个类的概率
    log_probability_t_c=[]  # 拉普拉斯平滑后概率

    def __init__(self, X, Y):
        if len(X) != len(Y):
            print 'samples\' length not equal labels\' length'
        elif len(Y) == 0:
            print 'samples\' size is zero'
        else:
            self.m = len(Y)
            self.n = len(X[0])
            D = []
            for i in range(self.m):
                # 对于新的类别添加一个新集合
                if Y[i] not in self.category:
                    self.category.append(Y[i])
                    D.append([])
                    D[-1].append(X[i])
                else:
                    D[self.category.index(Y[i])].append(X[i])
            # 计算每个类的概率
            for i in range(len(self.category)):
                self.log_probability_c.append(len(D[i]))
            self.log_probability_c = log(self.log_probability_c)
            # 赋值k、n
            self.k = len(self.category)
            self.N = zeros((self.k, self.n))
            for i in range(self.k):
                self.N[i] = array(D[i]).sum(0)
            # +1平滑
            self.log_probability_t_c = self.N + 1
            s = self.log_probability_t_c.sum(1)
            self.log_probability_t_c = log(self.log_probability_t_c / s.reshape(len(s), 1))
    #分类
    def predict(self, x):
        p = self.log_probability_c + x.dot(self.log_probability_t_c.transpose())
        i = p.argmax(1)
        label = []
        for j in range(len(i)):
            label.append(self.category[i[j]])
        return label

if __name__ == '__main__':
    #读取中间数据
    data = io.loadmat('SetMat1.mat')
    vectormat = data['trainSet']
    labeled_names = data['train_labeled'][0]
    labeled_names1 = data['test_labeled'][0]
    vectormat1 = data['testSet']
    nb = Naivebayes(vectormat, labeled_names)
    labels = nb.predict(vectormat1)
    calculate_result(labeled_names1,labels)
    #print labels
    c = zeros((10,10), dtype=int)
    for i in range(len(labels)):
        c[labeled_names1[i]-1][labels[i]-1] = c[labeled_names1[i]-1][labels[i]-1] + 1
    print c

运行结果
中间数据采用的是TF\IDF值，依据词频做了简单特征筛选

predict info:
accuracy:0.779
precision:0.759
recall:0.779
f1-score:0.735

使用Bool型特征（one-hot）则有明显提高
count_vec = CountVectorizer(binary = True,decode_error=’replace’)

predict info:
accuracy:0.849
precision:0.835
recall:0.849
f1-score:0.824
[[ 705    0    3    5    0    0    2    0    4    0]
 [   0    1    0    3   51    0    0    0    1    0]
 [  13    0  165    0    0    0    0    3    8    0]
 [  30    0    3 1051    0    0    1    0    2    0]
 [   0    1    0    5  126    0    1    4   12    0]
 [   0    0    0    1    1   40   86    0    3    0]
 [   0    0    1    1    0    8  166    0    3    0]
 [   3    0   49    1   11    0    0   16    9    0]
 [   0    0    1    7    3    0   11    0   95    0]
 [   0    1    0    3   64    0    0    1    2    0]]