达观杯（Day4）

逻辑回归（LR）

1. LR模型
   逻辑回归（Logistic Regression）是一个非线性模型（sigmod函数），但是他本质又是一个线性回归模型，因为除去sigmod映射函数关系，其他的步骤，算法都算是线性回归
2. LR的代价函数
   逻辑回归定义：$$\begin{array}{l}P(Y=1|x)=\frac{1}{1+\exp (-wx)}=\sigma (x)\\ P(Y=0|x)=1-\sigma (x)\end{array}$$
   负对数函数作为损失函数：$$\begin{array}{rl}L(w)& =-\log \left(\prod _{i=1}^N{\left[\sigma \left(x_i\right)\right]}^{y_{[}}{\left[1-\sigma \left(x_i\right)\right]}^{1-y_i}\right)\\ & =-\sum _{i=1}^N[y_i\log \sigma \left(x_i\right)+\left(1-y_i\right)\log (1-\sigma )]\\ & =-\sum _{i=1}^N\left[y_i\log \frac{\sigma \left(x_i\right)}{1-\sigma \left(x_i\right)}+\log \left(1-\sigma \left(x_i\right)\right)\right]\\ L(w)& =-\sum ^N\left[y_i\left(w{x}_i\right)-\log \left(1+\exp \left(w{x}_i\right)\right)\right]\end{array}$$
   求梯度：$$\begin{array}{rl}\frac{\partial L(w)}{\partial w}=& -\sum _{i=1}^N\left[y_i{x}_i-\frac{\exp \left(w{x}_i\right)}{1+\exp \left(w{x}_i\right)}{x}_i\right]\\ =& \sum _{i=1}^N\left[\sigma \left(x_i\right)-y_i\right]x_i\end{array}$$
   最后在使用梯度下降求解参数：$$\begin{array}{l}P(Y=k|x)=\frac{\exp \left(w_{k}x\right)}{1+{\sum }_{k=1}^{K-1}\exp \left(w_{k}x\right)}k=1,2,\dots ,K-1\\ P(Y=K|x)=\frac{1}{1+{\sum }_{k=1}^{K-1}\exp \left(w_{k}x\right)}\end{array}$$

支持向量机（svm）模型

• 支持向量机（Support Vector Machines, SVM）是一种二分类模型。它的基本模型是定义在特征空间上的间隔最大的线性分类器，间隔最大使它有别于感知机；支持向量机还包括核技巧，这使其成为实质上的非线性分类器。
• SVM 的学习策略就是间隔最大化，可形式化为一个求解凸二次规划的问题，也等价于正则化的合页损失函数的最小化问题。
• SVM 的最优化算法是求解凸二次规划的最优化算法

代码实现

import numpy as np
import pandas as pd

training = pd.read_csv("D:/ML/competition/daguan/new_data/train_set.csv")
# print(training.head())

# print(training.shape)
# print(training.columns)
print(training.info())

import time
t_start = time.time()

from sklearn.feature_extraction.text import TfidfVectorizer
vec = TfidfVectorizer(ngram_range=(1,2),min_df=3, max_df=0.9,use_idf=1,smooth_idf=1, sublinear_tf=1)
tfidfX_train = vec.fit_transform(training["word_seg"])
# print(tfidfX_train)

from sklearn.model_selection import train_test_split #to create validation data set
# X_train = training.drop("class", axis=1)
y_train = training["class"]
X_training, X_valid, y_training, y_valid = train_test_split(tfidfX_train, y_train, test_size=0.2, random_state=0)

from sklearn import svm
from sklearn.linear_model import  LogisticRegression
from sklearn.metrics import f1_score
clf = svm.LinearSVC(C=5, dual=False)
# clf = LogisticRegression(C=120, dual=True)

clf.fit(X_training, y_training)
y_prediction = clf.predict(X_valid)
f1 = f1_score(y_valid, y_prediction, average='micro')

print(f1)

t_end = time.time()
print("耗时：{}min".format((t_end-t_start)/60))