彻底搞懂softmax、逻辑回归

1 使用Tensorflow对minist进行分类，使用最简单的softmax，下面是代码：

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
from datetime import datetime
import math
import time
#载入数据集
mnist = input_data.read_data_sets("MNIST_data",one_hot=True)

# 配置每个 GPU 上占用的内存的比例 有GPU时候使用
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.95)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))

x = tf.placeholder(tf.float32,[None, 784]) 
W = tf.Variable(tf.zeros([784, 10])) 
b = tf.Variable(tf.zeros([10])) 
y = tf.nn.softmax(tf.matmul(x,W)+b) 
y_ = tf.placeholder(tf.float32,[None, 10]) 
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1])) 
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy) 
init = tf.initialize_all_variables() 
sess = tf.Session() #无GPU时候打开
sess.run(init) 

for i in range(1000): 
    batch_xs, batch_ys = mnist.train.next_batch(100) 
    sess.run(train_step, feed_dict = {x: batch_xs, y_: batch_ys}) 

correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(y_,1)) 
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 
print(sess.run(accuracy, feed_dict={x:mnist.test.images, y_: mnist.test.labels})) 

最终输出：0.92左右
2 模型中训练出的W和b到底是多少呢，可以通过打印看看：

seeb = sess.run(b) 
seew = sess.run(W) 
print(seeb)
print(seew[50])
print(seeb.shape)
print(seew.shape)

可以看到，训练出的W和b已经被取出来了，我们想用想保存都可以
3 验证模型输出：

print(mnist.test.images.shape)
print(mnist.test.labels.shape)

输出：这样可以明白了test的结构
(10000, 784)
(10000, 10)
我们取出第一个，并预测：

testx = [mnist.test.images[0]]
#print(testx)
testy = [mnist.test.labels[0]]
print(testy)
sess.run(y, feed_dict={x:testx})

输出：
[array([ 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.])]
Out[46]:
array([[ 3.53539799e-05, 4.24364188e-09, 1.03236089e-04,
1.41154591e-03, 9.89227829e-07, 2.13419171e-05,
1.82815807e-08, 9.98052120e-01, 2.00856193e-05,
3.55230470e-04]], dtype=float32)
明显，预测为7，预测正确
4 自己写代码验证整个模型：

from numpy import *;
import numpy as np; #这个方式使用numpy的函数时，需要以np.开头。
import math

testx = [mnist.test.images[0]]
a1=mat(testx);
a2=mat(seew);
a3=a1*a2;
a4 = a3+seeb
etab=[0,0,0,0,0,0,0,0,0,0]
for i in range(0,10):
    t = a4[0:,i]
    t = t.tolist()[0]
    t = t[0]
    etab[i] = math.exp(t)
a = sum(etab)
for i in range(0,10):
    t = etab[i]
    t = t/a
    print(t)

最终输出结果：
3.53539897083e-05
4.24364244027e-09
0.000103236158687
0.00141154597564
9.89226982999e-07
2.13419156007e-05
1.82816162943e-08
0.998052194092
2.00856209105e-05
0.000355230495374
和模型预测结果一致！
5 sklearn库的逻辑回归模型

from sklearn.linear_model import LinearRegression  
from sklearn.linear_model import LogisticRegression
from sklearn.linear_model import LogisticRegressionCV

Flag = mnist.train.labels.argmax(axis=1)

LR = LogisticRegressionCV()
LR.fit(mnist.train.images,Flag)
#预测
t = LR.predict(mnist.test.images)
print(t)
err = 0
Flag = mnist.test.labels.argmax(axis=1)
for i in range(0,t.shape[0]):
    if t[i] != Flag[i]:
        err = err+1
err = err/t.shape[0]
print(1-err)

最后输出：0.9197
取出训练的参数，套用上面的验证模型验证：

w2 = LR.coef_.T
b2 = LR.intercept_
#print(w2.shape)
#print(b2.shape)

from numpy import *;
import numpy as np; 
import math

testx = [mnist.test.images[0]]
a1=mat(testx);
a2=mat(w2);
a3=a1*a2;
a4 = a3+b2
etab=[0,0,0,0,0,0,0,0,0,0]
for i in range(0,10):
    t = a4[0:,i]
    t = t.tolist()[0]
    t = t[0]
    etab[i] = math.exp(t)
a = sum(etab)
for i in range(0,10):
    t = etab[i]
    t = t/a
    print(t)

最终输出：
1.1880514704e-07
1.14077025507e-14
3.43722382983e-07
6.02540011641e-05
6.69096332382e-09
1.60123847877e-06
1.49785118331e-11
0.999934460415
1.7622763634e-07
3.03888425803e-06
预测值为7，说明的确建立的模型和tensflow一样一样的
6 sklearn库的线性回归模型（这部分和softmax没有关系，仅做参考）

from sklearn.linear_model import LinearRegression  
from sklearn.linear_model import LogisticRegression
from sklearn.linear_model import LogisticRegressionCV

LR = LinearRegression()
LR.fit(mnist.train.images, mnist.train.labels)
print(LR.coef_.shape)
print(LR.intercept_)

#预测
temp = LR.predict(mnist.test.images)
#t = temp.max(axis=1)
t = temp.argmax(axis=1)
print(t)
err = 0
Flag = mnist.test.labels.argmax(axis=1)
for i in range(0,t.shape[0]):
    if t[i] != Flag[i]:
        err = err+1
err = err/t.shape[0]
print(1-err)

最终输出准确率：0.861
7 sklearn库的随机森林模型（这部分和softmax没有关系，仅做参考）

from sklearn.linear_model import LinearRegression  
from sklearn.linear_model import LogisticRegression
from sklearn.linear_model import LogisticRegressionCV
from sklearn.ensemble import RandomForestRegressor  

Flag = mnist.train.labels.argmax(axis=1)

LR = RandomForestRegressor()
LR.fit(mnist.train.images,Flag)
#预测
t = LR.predict(mnist.test.images)
t = np.around(t)
print(t)
err = 0
Flag = mnist.test.labels.argmax(axis=1)
for i in range(0,t.shape[0]):
    if t[i] != Flag[i]:
        err = err+1
err = err/t.shape[0]
print(1-err)

准确率：0.7163999999999999