作业是：【中文】【吴恩达课后编程作业】Course 1 - 神经网络和深度学习 - 第二周作业 https://blog.csdn.net/u013733326/article/details/79639509
本文对作业程序进行全面测试。
#为测试内容，前面的#后为测试程序，后面的#后为测试结果。
总结和感悟：通过此次作业，我发现w刚开始是一个空壳，将它定义成：w.T与X可以相乘（符合矩阵乘法），w.T的shape为(1,像素总数)，X的shape为（像素总数，图片数），两者乘积的shape为（1,图片数），加上b，是一个矩阵加法的广播。再用sigmoid函数作用在其结果上，得到一个0-1的取值，根据其函数特性，0和1都取不到，只能取一个中间值，即（0,1）。然后将这个（1,图片数）的行向量（array）中的每个值与0.5比较，大于0.5，就取值为1，小于0.5，取值为0。这个是预测值（prediction）,图片数据集本身就人为根据是不是猫，已经标注了1或者0。两者进行比对，就可以得到正确率。训练集和测试集是一个东西，不过是为了平行推演，将在A品种上测试的产品用在非A品种看看效果而已。在这个基础上，只计算了一次前向传播，为了修正正确率，需要自己设计迭代次数，通过反向传播，一次一次的降低w和b的梯度，最终实现提高正确率的目的。在降低梯度的过程重，需要设置一个梯度下降的速率（学习率）来人为控制和调整。通过不断地反向传播将让w和b取得更优值，该值是否优劣取决于是否让成本函数取得最小值。下图就是利用反向传播，降低w和b梯度的过程。附上视频和程序中涉及到梯度下降的核心截图。
图 利用反向传播，降低w和b梯度
图 吴恩达视频中降低梯度的公式
图 程序中计算dw的语句
注释：
1 simoid函数：Sigmoid函数是一个在生物学中常见的S型函数，也称为S型生长曲线。 在信息科学中，由于其单增以及反函数单增等性质，Sigmoid函数常被用作神经网络的激活函数，将变量映射到0,1之间。
2 前向传播：所谓的前向传播算法就是：将上一层的输出作为下一层的输入，并计算下一层的输出，一直到运算到输出层为止。
3 反向传播："误差反向传播" 的简称，也称为backprop，允许来自代价函数的信息通过网络向后流动，以便计算梯度。
参考资料：
吴恩达_神经网络与深度学习_神经网络基础 https://blog.csdn.net/nyist_yangguang/article/details/109312166
矩阵和向量之间的 “ * “运算和 dot 运算的区别 https://blog.csdn.net/nyist_yangguang/article/details/109535486
吴恩达视频中——logistic回归中的梯度下降法前向传播时公式求导问题 https://blog.csdn.net/nyist_yangguang/article/details/110233726
numpy(np)模块的使用方法笔记 https://blog.csdn.net/nyist_yangguang/article/details/110391749
识别猫程序中的一些问题 https://blog.csdn.net/nyist_yangguang/article/details/110427808
python中矩阵(rank为1就是向量)和array(数组)的区别 https://blog.csdn.net/nyist_yangguang/article/details/110441899
1 load_dataset 函数详解
import numpy as np
import h5py
def load_dataset():
train_dataset=h5py.File('datasets/train_catvnoncat.h5',"r")#双引单引无所谓
# print(type(train_dataset)) #<class 'h5py._hl.files.File'>
train_set_x_orig=np.array(train_dataset["train_set_x"][:])
train_set_y_orig=np.array(train_dataset["train_set_y"][:])
# print(type(train_set_x_orig)) #<class 'numpy.ndarray'>
# print(train_set_x_orig[0][0][0][:]) #[17 31 56]
#print(np.shape(train_set_x_orig)) #(209, 64, 64, 3)
#print(train_set_y_orig[0:3]) #[0 0 1]
#print(np.shape(train_set_y_orig)) #(209,) rank为1的数组 (numpy)
test_dataset=h5py.File("datasets/test_catvnoncat.h5",'r') #双引单引无所谓
test_set_x_orig=np.array(test_dataset["test_set_x"][:])
test_set_y_orig=np.array(test_dataset["test_set_y"][:])
# print(test_set_x_orig[0][0][0][:]) #[158 104 83]
# print(np.shape(test_set_x_orig)) #(50, 64, 64, 3)
# print(test_set_y_orig[0:3]) #[1 1 1]
# print(np.shape(test_set_y_orig)) #(50,)
classes=np.array(test_dataset["list_classes"][:])
# print(classes[:]) #[b'non-cat' b'cat']
# print(type(classes)) #<class 'numpy.ndarray'>
# print(classes[0][7]) # 4:(c对应的ascII码值)99 其它下标：[0][0]... 110 111 110 45 99 97 116 对应着：non-cat
# #从[0][7]开始 IndexError: index out of range
# print(classes[0][:]) #b'non-cat'
# print(classes[1][:]) # b'cat'
# print(classes[2][:]) #IndexError: index 2 is out of bounds for axis 0 with size 2
# print(classes.dtype) #|S7
# print(classes.shape) #(2,)
train_set_y_orig=train_set_y_orig.reshape((1,train_set_y_orig.shape[0]))
test_set_y_orig=test_set_y_orig.reshape((1,test_set_y_orig.shape[0]))
# print(type(train_set_x_orig)) #<class 'numpy.ndarray'>
# print(train_set_y_orig.shape) #(1, 209) 之前有问题，不是一个正确的矩阵(向量),现在是一个列向量
# print(test_set_y_orig.shape) #(1, 50)
return train_set_x_orig,train_set_y_orig,test_set_x_orig,test_set_y_orig,classes
load_dataset()2 sigmoid函数详解
import numpy as np
def sigmoid(z):
s=1/(1+np.exp(-z)) #函数讲解：https://baike.baidu.com/item/Sigmoid%E5%87%BD%E6%95%B0/7981407?fr=aladdin
return s
#print(sigmoid(1)) #0.7310585786300049
#print(sigmoid(0)) #0.5
#print(sigmoid(-1)) #0.26894142136999513 主函数详解
主函数在c/c++中均有main字样，但是python中不写main也可以，此程序也无main函数，根据缩进程度，选出最先执行的主函数部分。
costs=np.squeeze(d["costs"])
plt.plot(costs)
plt.show()
costs=np.squeeze(d["costs"])
plt.plot(costs)
plt.ylabel("cost")
plt.xlabel("iterations(per hundreds)")
plt.title("learning rate= "+ str(d["learning_rate"]))
plt.show()
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from matplotlib.pyplot import plot,ylabel
import h5py
from lr_utils import load_dataset
# 训练集的图像数据，训练集中图像对应的分类值，测试集...
def initialize_with_zeros(dim):
w=np.zeros(shape=(dim,1))
# print(w) #[[0.]] [[0.],[0.]] [[0.], [0.], [0.]]
b=0
# 以下仅测试print(initialize_with_zeros(1))
# 改变w的值，使维度不相等 eg: w=[2,3]
# w=np.array([2,3]) #AssertionError
#print(w.shape) #(2,)
# w=w.reshape(2,1) #AssertionError
# print(w.shape) #(2, 1)
# print(w) # [[2]
# # [3]]
assert(w.shape==(dim,1))
# b='c'#AssertionError
# print(type(b)) #<class 'str'>
assert(isinstance(b,float) or isinstance(b,int))
return (w,b)
# print(initialize_with_zeros(1)) #(array([[0.]]), 0)
# print(initialize_with_zeros(2)) #(array([[0.],[0.]]), 0)
# print(initialize_with_zeros(3)) #(array([[0.], [0.], [0.]]), 0)
def sigmoid(z):
s=1/(1+np.exp(-z)) #函数讲解：https://baike.baidu.com/item/Sigmoid%E5%87%BD%E6%95%B0/7981407?fr=aladdin
return s
def propagate(w,b,X,Y):
m=X.shape[1]
A=sigmoid(np.dot(w.T,X)+b)
# print(np.dot(w.T,X),type(np.dot(w.T,X)),np.dot(w.T,X).shape) #[[-2. -2.]] <class 'numpy.ndarray'> (1, 2)
# print(m,A) #2 [[0.26894142 0.26894142]]
cost=(-1/m)*np.sum(Y*np.log(A)+(1-Y)*(np.log(1-A))) #底数，默认为 e
dw=(1/m)*np.dot(X,(A-Y).T)
# print(np.dot(X,(A-Y).T),np.dot(X,(A-Y).T).dtype,type(np.dot(X,(A-Y).T))) # [[-2.19317574]
# # [-5.11741005]] float64 <class 'numpy.ndarray'>
# print(dw,dw.dtype,type(dw)) # [[-1.09658787]
# # [-2.55870503]] float64 <class 'numpy.ndarray'>
db=(1/m)*np.sum(A-Y)
# print(np.sum(A-Y))
# print(A-Y,sum(A-Y),sum(A-Y).shape,((1/m)*np.sum(A-Y)).shape) #[[-0.73105858 -0.73105858]] [-0.73105858 -0.73105858] (2,) ()
# # sum(矩阵)是计算矩阵所有元素的和
# print(db,type(db),db.shape)#-0.7310585786300049 <class 'numpy.float64'> ()
assert(dw.shape==w.shape)
assert(db.dtype == float) # AttributeError: 'numpy.float64' object has no attribute 'dtpye' 不好意思，这应该是dtype
# print(cost,cost.dtype,type(cost),cost.shape) #1.3132616875182228 float64 <class 'numpy.float64'> ()
# cost1=[1.0] #AttributeError: 'list' object has no attribute 'shape'
# cost1=1.0 #AttributeError: 'float' object has no attribute 'shape'
# cost1=np.array([1.0]) #(1,) [1.] float64 <class 'numpy.ndarray'>
# print(sum(cost1),sum(cost1).shape,cost1.dtype,type(cost1)) #1.0 () float64 <class 'numpy.ndarray'>
# print(cost1.shape,cost1,cost1.dtype,type(cost1))
cost=np.squeeze(cost)
cost3=np.array([[1,0,2],[0,0,0]]) # [[1 0 2]
# [0 0 0]]
cost3=np.squeeze(cost3) #https://blog.csdn.net/nyist_yangguang/article/details/110427808
# print(cost3)
assert(cost.shape==())
grads={
"dw":dw,
"db":db
}
return (grads,cost)
# w=np.array([[1.0],[-1.0]])
# # w1=[1,2,3]
# # print(w.dtype,type(w1))# float64 <class 'list'>
# b=1
# # print(type(b))
# X=np.array([[1.0,2.0],[3.0,4.0]])
# Y=np.array([[1.0,1.0]])
# # print(w,w.shape,type(w)) #[[ 1.]
# # # [-1.]] (2, 1) <class 'numpy.ndarray'>
# # print(X,X.shape,type(X)) #[[1. 2.]
# # # [3. 4.]] (2, 2) <class 'numpy.ndarray'>
# # print(Y,Y.shape,type(Y)) #[[1. 1.]] (1, 2) <class 'numpy.ndarray'>
# # print(propagate(w,b,X,Y))
# #({'dw': array([[-1.09658787],
# # [-2.55870503]]), 'db': -0.7310585786300049}, 1.3132616875182228)
# grads_new,cost_new=propagate(w,b,X,Y)
# # print(grads_new["dw"]) #[[-1.09658787]
# # # [-2.55870503]]
# # print(grads_new[:]) #TypeError: unhashable type: 'slice'
# # print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
# # print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
def optimize(w,b,X,Y,num_iterations,learning_rate,print_cost=False):
costs=[]
for i in range(num_iterations):
grads,cost=propagate(w,b,X,Y)
dw=grads["dw"]
db=grads["db"]
w=w-learning_rate*dw
b=b-learning_rate*db
if i%100==0:
costs.append(cost)
if(print_cost)and(i%100==0):
print("迭代的次数：%i,误差值：%f"%(i,cost))
# 迭代的次数：0, 误差值：1.313262
# 迭代的次数：100, 误差值：0.008212
# 迭代的次数：200, 误差值：0.004165
# 迭代的次数：300, 误差值：0.002800
# 迭代的次数：400, 误差值：0.002111
# 迭代的次数：500, 误差值：0.001696
# 迭代的次数：600, 误差值：0.001418
# 迭代的次数：700, 误差值：0.001219
# 迭代的次数：800, 误差值：0.001069
# 迭代的次数：900, 误差值：0.000952
# 迭代的次数：1000, 误差值：0.000858
params={
"w":w,
"b":b
}
grads={
"dw":dw,
"db":db
}
return params,grads,costs
# params,grads,costs=optimize(w,b,X,Y,1001,0.1,True)
# print(params)
# # {'w': array([[-0.0009077 ],
# # [-0.00262293]]), 'db': -0.0008576164239226247}
# print(grads)
# # {'dw': array([[-0.0009077 ],
# # [-0.00262293]]), 'db': -0.0008576164239226247}
# print(costs)
# # [1.3132616875182228, 0.008211563080222594, 0.0041654309736682636, 0.0027995659824453178, 0.002111333903432137, 0.0016960932319482194, 0.001418069094628399, 0.001218782485685477, 0.001068876940625712, 0.0009519888068423773, 0.0008582717476016353]
def predict(w,b,X):
m=X.shape[1]
#print(X.shape) #(2, 2)
Y_prediction=np.zeros((1,m))
w=w.reshape(X.shape[0],1)
A=sigmoid(np.dot(w.T,X)+b)
for i in range(A.shape[1]):
Y_prediction[0,i]=1 if A[0,i]>0.5 else 0
# # equal： if A[0,i]>0.5
# Y_prediciton[0,i]=1
# else
# Y_prediction[0,i]=0
#print(Y_prediction.shape) #(1, 2)
assert(Y_prediction.shape==(1,m))
return Y_prediction
# w=np.array([[1.0],[-1.0]])
# # w1=[1,2,3]
# # print(w.dtype,type(w1))# float64 <class 'list'>
# b=1
# # print(type(b))
# X=np.array([[1.0,2.0],[3.0,4.0]])
# #print(predict(w,b,X))#[[0. 0.]]
train_set_x_orig,train_set_y,test_set_x_orig,test_set_y,classes=load_dataset()
train_set_x_flatten=train_set_x_orig.reshape(train_set_x_orig.shape[0],-1).T
test_set_x_flatten=test_set_x_orig.reshape(test_set_x_orig.shape[0],-1).T
# print(np.shape(train_set_x_orig)) #(209, 64, 64, 3)
# print(np.shape(train_set_x_orig.reshape(train_set_x_orig.shape[0],-1))) #(209, 12288)
# print(np.shape(train_set_x_flatten)) #(12288, 209) 从这就可以看出来-1的作用
# print(np.shape(test_set_x_orig)) #(50, 64, 64, 3)
# print(np.shape(test_set_x_orig.reshape(test_set_x_orig.shape[0],-1))) #(50, 12288)
#print(np.shape(test_set_x_flatten)) #(12288, 50) 从这就可以看出来-1的作用
train_set_x=train_set_x_flatten/255 #像素归一化，减小误差（我的见解）
test_set_x=test_set_x_flatten/255 #像素归一化，减小误差（我的见解）
def model(X_train,Y_train,X_test,Y_test,num_iterations=2000,learning_rate=0.5,print_cost=False):
w,b=initialize_with_zeros(X_train.shape[0])
# print(X_train.shape) #(12288, 209)
# print(w.shape) #(12288, 1) one picture
# print(b.shape) #AttributeError: 'int' object has no attribute 'shape'
#print(X_train.shape,Y_test.shape) #(12288, 209) (1, 50)
parameters,grads,costs=optimize(w,b,X_train,Y_train,num_iterations,learning_rate,print_cost)
w,b=parameters["w"],parameters["b"]
Y_prediction_test=predict(w,b,X_test)
Y_prediction_train=predict(w,b,X_train)
# print(np.mean(Y_prediction_train), np.mean(Y_prediction_test))
# print(np.mean(np.abs((Y_prediction_train-Y_train))))
# print(np.mean(np.abs((Y_prediction_test-Y_test))))
print("训练集准确性：",format(100-np.mean(np.abs(Y_prediction_train-Y_train))*100),"%")
print("测试集准确性：",format(100-np.mean(np.abs(Y_prediction_test-Y_test))*100),"%")
d={
"costs":costs,
"Y_prediction_test":Y_prediction_test,
"Y_prediction_train":Y_prediction_train,
"w":w,
"b":b,
"learning_rate":learning_rate,
"num_iterations":num_iterations
}
# print(type(d)) #<class 'dict'>
# print(d.type) #AttributeError: 'dict' object has no attribute 'type'
return d
d=model(train_set_x,train_set_y,test_set_x,test_set_y,num_iterations=2001,learning_rate=0.005,print_cost=True)
costs=np.squeeze(d["costs"])
plt.plot(costs)
plt.ylabel("cost")
plt.xlabel("iterations(per hundreds)")
plt.title("learning rate= "+ str(d["learning_rate"]))
plt.show()
4 initialize_with_zeros函数详解
import numpy as np
def initialize_with_zeros(dim):
w=np.zeros(shape=(dim,1))
# print(w) #[[0.]] [[0.],[0.]] [[0.], [0.], [0.]]
b=0
# 以下仅测试print(initialize_with_zeros(1))
# 改变w的值，使维度不相等 eg: w=[2,3]
# w=np.array([2,3]) #AssertionError
#print(w.shape) #(2,)
# w=w.reshape(2,1) #AssertionError
# print(w.shape) #(2, 1)
# print(w) # [[2]
# # [3]]
assert(w.shape==(dim,1))
# b='c'#AssertionError
# print(type(b)) #<class 'str'>
assert(isinstance(b,float) or isinstance(b,int))
return (w,b)
#print(initialize_with_zeros(1)) #(array([[0.]]), 0)
#print(initialize_with_zeros(2)) #(array([[0.],[0.]]), 0)
#print(initialize_with_zeros(3)) #(array([[0.], [0.], [0.]]), 0)5 propagate函数详解
import numpy as np
def sigmoid(z):
s=1/(1+np.exp(-z)) #函数讲解：https://baike.baidu.com/item/Sigmoid%E5%87%BD%E6%95%B0/7981407?fr=aladdin
return s
def propagate(w,b,X,Y):
m=X.shape[1]
A=sigmoid(np.dot(w.T,X)+b)
# print(np.dot(w.T,X),type(np.dot(w.T,X)),np.dot(w.T,X).shape) #[[-2. -2.]] <class 'numpy.ndarray'> (1, 2)
# print(m,A) #2 [[0.26894142 0.26894142]]
cost=(-1/m)*np.sum(Y*np.log(A)+(1-Y)*(np.log(1-A))) #底数，默认为 e
dw=(1/m)*np.dot(X,(A-Y).T)
# print(np.dot(X,(A-Y).T),np.dot(X,(A-Y).T).dtype,type(np.dot(X,(A-Y).T))) # [[-2.19317574]
# # [-5.11741005]] float64 <class 'numpy.ndarray'>
# print(dw,dw.dtype,type(dw)) # [[-1.09658787]
# # [-2.55870503]] float64 <class 'numpy.ndarray'>
db=(1/m)*np.sum(A-Y)
# print(np.sum(A-Y))
# print(A-Y,sum(A-Y),sum(A-Y).shape,((1/m)*np.sum(A-Y)).shape) #[[-0.73105858 -0.73105858]] [-0.73105858 -0.73105858] (2,) ()
# # sum(矩阵)是计算矩阵所有元素的和
# print(db,type(db),db.shape)#-0.7310585786300049 <class 'numpy.float64'> ()
assert(dw.shape==w.shape)
assert(db.dtype == float) # AttributeError: 'numpy.float64' object has no attribute 'dtpye' 不好意思，这应该是dtype
# print(cost,cost.dtype,type(cost),cost.shape) #1.3132616875182228 float64 <class 'numpy.float64'> ()
# cost1=[1.0] #AttributeError: 'list' object has no attribute 'shape'
# cost1=1.0 #AttributeError: 'float' object has no attribute 'shape'
# cost1=np.array([1.0]) #(1,) [1.] float64 <class 'numpy.ndarray'>
# print(sum(cost1),sum(cost1).shape,cost1.dtype,type(cost1)) #1.0 () float64 <class 'numpy.ndarray'>
# print(cost1.shape,cost1,cost1.dtype,type(cost1))
cost=np.squeeze(cost)
cost3=np.array([[1,0,2],[0,0,0]]) # [[1 0 2]
# [0 0 0]]
cost3=np.squeeze(cost3) #https://blog.csdn.net/nyist_yangguang/article/details/110427808
# print(cost3)
assert(cost.shape==())
# print(cost.shape==(,)) #SyntaxError: invalid syntax
grads={
"dw":dw,
"db":db
}
return (grads,cost)
w=np.array([[1.0],[-1.0]])
# w1=[1,2,3]
# print(w.dtype,type(w1))# float64 <class 'list'>
b=1
# print(type(b))
X=np.array([[1.0,2.0],[3.0,4.0]])
Y=np.array([[1.0,1.0]])
# print(w,w.shape,type(w)) #[[ 1.]
# # [-1.]] (2, 1) <class 'numpy.ndarray'>
# print(X,X.shape,type(X)) #[[1. 2.]
# # [3. 4.]] (2, 2) <class 'numpy.ndarray'>
# print(Y,Y.shape,type(Y)) #[[1. 1.]] (1, 2) <class 'numpy.ndarray'>
# print(propagate(w,b,X,Y))
#({'dw': array([[-1.09658787],
# [-2.55870503]]), 'db': -0.7310585786300049}, 1.3132616875182228)
grads_new,cost_new=propagate(w,b,X,Y)
# print(grads_new["dw"]) #[[-1.09658787]
# # [-2.55870503]]
# print(grads_new[:]) #TypeError: unhashable type: 'slice'
# print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
# print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
#print(type(grads_new)) #<class 'dict'>6 optimize函数详解
import numpy as np
def sigmoid(z):
s=1/(1+np.exp(-z)) #函数讲解：https://baike.baidu.com/item/Sigmoid%E5%87%BD%E6%95%B0/7981407?fr=aladdin
return s
def propagate(w,b,X,Y):
m=X.shape[1]
A=sigmoid(np.dot(w.T,X)+b)
# print(np.dot(w.T,X),type(np.dot(w.T,X)),np.dot(w.T,X).shape) #[[-2. -2.]] <class 'numpy.ndarray'> (1, 2)
# print(m,A) #2 [[0.26894142 0.26894142]]
cost=(-1/m)*np.sum(Y*np.log(A)+(1-Y)*(np.log(1-A))) #底数，默认为 e
dw=(1/m)*np.dot(X,(A-Y).T)
# print(np.dot(X,(A-Y).T),np.dot(X,(A-Y).T).dtype,type(np.dot(X,(A-Y).T))) # [[-2.19317574]
# # [-5.11741005]] float64 <class 'numpy.ndarray'>
# print(dw,dw.dtype,type(dw)) # [[-1.09658787]
# # [-2.55870503]] float64 <class 'numpy.ndarray'>
db=(1/m)*np.sum(A-Y)
# print(np.sum(A-Y))
# print(A-Y,sum(A-Y),sum(A-Y).shape,((1/m)*np.sum(A-Y)).shape) #[[-0.73105858 -0.73105858]] [-0.73105858 -0.73105858] (2,) ()
# # sum(矩阵)是计算矩阵所有元素的和
# print(db,type(db),db.shape)#-0.7310585786300049 <class 'numpy.float64'> ()
assert(dw.shape==w.shape)
assert(db.dtype == float) # AttributeError: 'numpy.float64' object has no attribute 'dtpye' 不好意思，这应该是dtype
# print(cost,cost.dtype,type(cost),cost.shape) #1.3132616875182228 float64 <class 'numpy.float64'> ()
# cost1=[1.0] #AttributeError: 'list' object has no attribute 'shape'
# cost1=1.0 #AttributeError: 'float' object has no attribute 'shape'
# cost1=np.array([1.0]) #(1,) [1.] float64 <class 'numpy.ndarray'>
# print(sum(cost1),sum(cost1).shape,cost1.dtype,type(cost1)) #1.0 () float64 <class 'numpy.ndarray'>
# print(cost1.shape,cost1,cost1.dtype,type(cost1))
cost=np.squeeze(cost)
cost3=np.array([[1,0,2],[0,0,0]]) # [[1 0 2]
# [0 0 0]]
cost3=np.squeeze(cost3) #https://blog.csdn.net/nyist_yangguang/article/details/110427808
# print(cost3)
assert(cost.shape==())
grads={
"dw":dw,
"db":db
}
return (grads,cost)
w=np.array([[1.0],[-1.0]])
# w1=[1,2,3]
# print(w.dtype,type(w1))# float64 <class 'list'>
b=1
# print(type(b))
X=np.array([[1.0,2.0],[3.0,4.0]])
Y=np.array([[1.0,1.0]])
# print(w,w.shape,type(w)) #[[ 1.]
# # [-1.]] (2, 1) <class 'numpy.ndarray'>
# print(X,X.shape,type(X)) #[[1. 2.]
# # [3. 4.]] (2, 2) <class 'numpy.ndarray'>
# print(Y,Y.shape,type(Y)) #[[1. 1.]] (1, 2) <class 'numpy.ndarray'>
# print(propagate(w,b,X,Y))
#({'dw': array([[-1.09658787],
# [-2.55870503]]), 'db': -0.7310585786300049}, 1.3132616875182228)
grads_new,cost_new=propagate(w,b,X,Y)
# print(grads_new["dw"]) #[[-1.09658787]
# # [-2.55870503]]
# print(grads_new[:]) #TypeError: unhashable type: 'slice'
# print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
# print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
def optimize(w,b,X,Y,num_iterations,learning_rate,print_cost=False):
costs=[]
for i in range(num_iterations):
grads,cost=propagate(w,b,X,Y)
dw=grads["dw"]
db=grads["db"]
w=w-learning_rate*dw
b=b-learning_rate*db
if i%100==0:
costs.append(cost)
if(print_cost)and(i%100==0):
print("迭代的次数：%i,误差值：%f"%(i,cost))
# 迭代的次数：0, 误差值：1.313262
# 迭代的次数：100, 误差值：0.008212
# 迭代的次数：200, 误差值：0.004165
# 迭代的次数：300, 误差值：0.002800
# 迭代的次数：400, 误差值：0.002111
# 迭代的次数：500, 误差值：0.001696
# 迭代的次数：600, 误差值：0.001418
# 迭代的次数：700, 误差值：0.001219
# 迭代的次数：800, 误差值：0.001069
# 迭代的次数：900, 误差值：0.000952
# 迭代的次数：1000, 误差值：0.000858
params={
"w":w,
"b":b
}
grads={
"dw":dw,
"db":db
}
return params,grads,costs
params,grads,costs=optimize(w,b,X,Y,1001,0.1,True)
#print(params)
#{'w': array([[1.76999173],
# [1.01220814]]), 'b': 1.6211082052124597}
# print(grads)
# # {'dw': array([[-0.0009077 ],
# # [-0.00262293]]), 'db': -0.0008576164239226247}
# print(costs)
# # [1.3132616875182228, 0.008211563080222594, 0.0041654309736682636, 0.0027995659824453178, 0.002111333903432137, 0.0016960932319482194, 0.001418069094628399, 0.001218782485685477, 0.001068876940625712, 0.0009519888068423773, 0.0008582717476016353]
7 predict函数详解
import numpy as np
def sigmoid(z):
s=1/(1+np.exp(-z)) #函数讲解：https://baike.baidu.com/item/Sigmoid%E5%87%BD%E6%95%B0/7981407?fr=aladdin
return s
#print(sigmoid(1)) #0.7310585786300049
#print(sigmoid(0)) #0.5
#print(sigmoid(-1)) #0.2689414213699951
def predict(w,b,X):
m=X.shape[1]
#print(X.shape) #(2, 2)
Y_prediction=np.zeros((1,m))
w=w.reshape(X.shape[0],1)
A=sigmoid(np.dot(w.T,X)+b)
for i in range(A.shape[1]):
Y_prediction[0,i]=1 if A[0,i]>0.5 else 0
# # equal： if A[0,i]>0.5
# Y_prediciton[0,i]=1
# else
# Y_prediction[0,i]=0
#print(Y_prediction.shape) #(1, 2)
assert(Y_prediction.shape==(1,m))
return Y_prediction
w=np.array([[1.0],[-1.0]])
# w1=[1,2,3]
# print(w.dtype,type(w1))# float64 <class 'list'>
b=1
# print(type(b))
X=np.array([[1.0,2.0],[3.0,4.0]])
#print(predict(w,b,X))#[[0. 0.]]8 显示学习过程详解
并没有感觉
这个起到了什么作用
models内容：
{'0.01': {'costs': [0.6931471805599453, 0.823920868162269, 0.4189442213534804, 0.6173497048572021, 0.522115770306867, 0.3877087478946796, 0.23625445668031034, 0.1542221330624998, 0.13532782831877796, 0.12497148000593818, 0.11647833125883059, 0.10919251128302268, 0.10280446418272836, 0.09712981007970017, 0.09204326923596717, 0.08745251991704941, 0.08328603053562557, 0.07948657037734264, 0.07600734572003641, 0.07280949458435325, 0.06986035604936246], 'Y_prediction_test': array([[1., 1., 1., 1., 1., 1., 0., 1., 1., 1., 0., 0., 1., 1., 0., 1.,
0., 1., 0., 0., 1., 0., 0., 1., 1., 1., 1., 0., 0., 1., 0., 1.,
1., 1., 1., 0., 0., 1., 0., 0., 1., 0., 1., 0., 1., 1., 0., 1.,
1., 0.]]), 'Y_prediction_train': array([[0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 1., 1., 0.,
0., 0., 0., 1., 0., 0., 0., 0., 1., 1., 0., 1., 0., 1., 0., 0.,
0., 0., 0., 0., 0., 0., 1., 0., 0., 1., 1., 0., 0., 0., 0., 1.,
0., 0., 1., 0., 0., 0., 1., 0., 1., 1., 0., 1., 1., 1., 0., 0.,
0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0.,
0., 0., 0., 1., 1., 0., 0., 0., 1., 0., 0., 0., 1., 1., 1., 0.,
0., 1., 0., 0., 0., 0., 1., 0., 1., 0., 1., 1., 1., 1., 1., 1.,
0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 1., 0., 1., 0.,
1., 1., 0., 0., 0., 1., 1., 1., 1., 1., 0., 0., 0., 0., 1., 0.,
1., 1., 1., 0., 1., 1., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0.,
0., 0., 1., 0., 1., 0., 1., 0., 0., 1., 1., 1., 0., 0., 1., 1.,
0., 1., 0., 1., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0.,
1., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.,
0.]]), 'w': array([[ 0.01405214],
[-0.03767732],
[-0.01678701],
...,
[-0.01583083],
[-0.03966076],
[ 0.03512588]]), 'b': -0.003830556275492493, 'learning_rate': 0.01, 'num_iterations': 2001}, '0.001': {'costs': [0.6931471805599453, 0.5912894260003537, 0.5557961107127088, 0.5289765131562365, 0.5068812917435517, 0.4878798632171657, 0.47110827803124367, 0.456045809698285, 0.4423502279336529, 0.42978171535077836, 0.4181638209364326, 0.4073617499582191, 0.39726946872697994, 0.3878016072295409, 0.3788881303593957, 0.3704706898360003, 0.3625000422834636, 0.35493416988240534, 0.34773687946679727, 0.34087673680743863, 0.33432624155022783], 'Y_prediction_test': array([[1., 1., 1., 1., 1., 0., 0., 1., 1., 1., 0., 0., 1., 1., 0., 1.,
0., 1., 0., 0., 1., 0., 0., 0., 1., 1., 0., 0., 0., 1., 0., 1.,
1., 0., 1., 0., 0., 1., 0., 0., 1., 1., 1., 0., 1., 0., 0., 1.,
0., 0.]]), 'Y_prediction_train': array([[0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 1., 0., 0.,
0., 0., 0., 1., 0., 0., 0., 0., 1., 1., 0., 1., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 1.,
0., 0., 1., 0., 0., 1., 1., 0., 1., 1., 0., 1., 1., 1., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0.,
0., 0., 0., 1., 1., 0., 0., 0., 1., 0., 0., 0., 0., 1., 1., 0.,
0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0., 1., 1., 1., 1.,
0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0.,
1., 1., 0., 0., 0., 1., 1., 0., 1., 1., 0., 0., 0., 0., 1., 0.,
0., 1., 1., 0., 1., 1., 1., 0., 0., 1., 0., 1., 1., 0., 0., 0.,
0., 0., 1., 0., 1., 0., 1., 0., 0., 0., 1., 1., 0., 0., 1., 1.,
0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0.,
0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.,
0.]]), 'w': array([[ 0.00453208],
[-0.00994549],
[-0.00468509],
...,
[-0.00520158],
[-0.01335417],
[ 0.00825271]]), 'b': -0.012819253443411708, 'learning_rate': 0.001, 'num_iterations': 2001}, '0.0001': {'costs': [0.6931471805599453, 0.6436767556935213, 0.6357371814059836, 0.6285720456492652, 0.6220395010251998, 0.6160293786905876, 0.6104550830063585, 0.6052481726085716, 0.6003541917337943, 0.5957294843812225, 0.5913387663913113, 0.5871532754554301, 0.5831493591357294, 0.5793073939793076, 0.5756109548449317, 0.5720461735818175, 0.5686012413673299, 0.5652660204017266, 0.5620317391748741, 0.5588907518764314, 0.5558363472658949], 'Y_prediction_test': array([[0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 1.,
0., 0.]]), 'Y_prediction_train': array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 1., 1., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0., 0., 0., 0., 1.,
0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 1.,
0., 1., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0.]]), 'w': array([[ 0.00112764],
[-0.00131899],
[-0.00027951],
...,
[-0.0007171 ],
[-0.00233239],
[ 0.00115069]]), 'b': -0.0028359192036077267, 'learning_rate': 0.0001, 'num_iterations': 2001}}
完整示例：
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from matplotlib.pyplot import plot,ylabel
import h5py
from lr_utils import load_dataset
# 训练集的图像数据，训练集中图像对应的分类值，测试集...
def initialize_with_zeros(dim):
w=np.zeros(shape=(dim,1))
# print(w) #[[0.]] [[0.],[0.]] [[0.], [0.], [0.]]
b=0
# 以下仅测试print(initialize_with_zeros(1))
# 改变w的值，使维度不相等 eg: w=[2,3]
# w=np.array([2,3]) #AssertionError
#print(w.shape) #(2,)
# w=w.reshape(2,1) #AssertionError
# print(w.shape) #(2, 1)
# print(w) # [[2]
# # [3]]
assert(w.shape==(dim,1))
# b='c'#AssertionError
# print(type(b)) #<class 'str'>
assert(isinstance(b,float) or isinstance(b,int))
return (w,b)
# print(initialize_with_zeros(1)) #(array([[0.]]), 0)
# print(initialize_with_zeros(2)) #(array([[0.],[0.]]), 0)
# print(initialize_with_zeros(3)) #(array([[0.], [0.], [0.]]), 0)
def sigmoid(z):
s=1/(1+np.exp(-z)) #函数讲解：https://baike.baidu.com/item/Sigmoid%E5%87%BD%E6%95%B0/7981407?fr=aladdin
return s
def propagate(w,b,X,Y):
m=X.shape[1]
A=sigmoid(np.dot(w.T,X)+b)
# print(np.dot(w.T,X),type(np.dot(w.T,X)),np.dot(w.T,X).shape) #[[-2. -2.]] <class 'numpy.ndarray'> (1, 2)
# print(m,A) #2 [[0.26894142 0.26894142]]
cost=(-1/m)*np.sum(Y*np.log(A)+(1-Y)*(np.log(1-A))) #底数，默认为 e
dw=(1/m)*np.dot(X,(A-Y).T)
# print(np.dot(X,(A-Y).T),np.dot(X,(A-Y).T).dtype,type(np.dot(X,(A-Y).T))) # [[-2.19317574]
# # [-5.11741005]] float64 <class 'numpy.ndarray'>
# print(dw,dw.dtype,type(dw)) # [[-1.09658787]
# # [-2.55870503]] float64 <class 'numpy.ndarray'>
db=(1/m)*np.sum(A-Y)
# print(np.sum(A-Y))
# print(A-Y,sum(A-Y),sum(A-Y).shape,((1/m)*np.sum(A-Y)).shape) #[[-0.73105858 -0.73105858]] [-0.73105858 -0.73105858] (2,) ()
# # sum(矩阵)是计算矩阵所有元素的和
# print(db,type(db),db.shape)#-0.7310585786300049 <class 'numpy.float64'> ()
assert(dw.shape==w.shape)
assert(db.dtype == float) # AttributeError: 'numpy.float64' object has no attribute 'dtpye' 不好意思，这应该是dtype
# print(cost,cost.dtype,type(cost),cost.shape) #1.3132616875182228 float64 <class 'numpy.float64'> ()
# cost1=[1.0] #AttributeError: 'list' object has no attribute 'shape'
# cost1=1.0 #AttributeError: 'float' object has no attribute 'shape'
# cost1=np.array([1.0]) #(1,) [1.] float64 <class 'numpy.ndarray'>
# print(sum(cost1),sum(cost1).shape,cost1.dtype,type(cost1)) #1.0 () float64 <class 'numpy.ndarray'>
# print(cost1.shape,cost1,cost1.dtype,type(cost1))
cost=np.squeeze(cost)
cost3=np.array([[1,0,2],[0,0,0]]) # [[1 0 2]
# [0 0 0]]
cost3=np.squeeze(cost3) #https://blog.csdn.net/nyist_yangguang/article/details/110427808
# print(cost3)
assert(cost.shape==())
grads={
"dw":dw,
"db":db
}
return (grads,cost)
# w=np.array([[1.0],[-1.0]])
# # w1=[1,2,3]
# # print(w.dtype,type(w1))# float64 <class 'list'>
# b=1
# # print(type(b))
# X=np.array([[1.0,2.0],[3.0,4.0]])
# Y=np.array([[1.0,1.0]])
# # print(w,w.shape,type(w)) #[[ 1.]
# # # [-1.]] (2, 1) <class 'numpy.ndarray'>
# # print(X,X.shape,type(X)) #[[1. 2.]
# # # [3. 4.]] (2, 2) <class 'numpy.ndarray'>
# # print(Y,Y.shape,type(Y)) #[[1. 1.]] (1, 2) <class 'numpy.ndarray'>
# # print(propagate(w,b,X,Y))
# #({'dw': array([[-1.09658787],
# # [-2.55870503]]), 'db': -0.7310585786300049}, 1.3132616875182228)
# grads_new,cost_new=propagate(w,b,X,Y)
# # print(grads_new["dw"]) #[[-1.09658787]
# # # [-2.55870503]]
# # print(grads_new[:]) #TypeError: unhashable type: 'slice'
# # print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
# # print(grads_new[0][1]) #TypeError: unhashable type: 'slice'
def optimize(w,b,X,Y,num_iterations,learning_rate,print_cost=False):
costs=[]
for i in range(num_iterations):
grads,cost=propagate(w,b,X,Y)
dw=grads["dw"]
db=grads["db"]
w=w-learning_rate*dw
b=b-learning_rate*db
if i%100==0:
costs.append(cost)
if(print_cost)and(i%100==0):
print("迭代的次数：%i,误差值：%f"%(i,cost))
# 迭代的次数：0, 误差值：1.313262
# 迭代的次数：100, 误差值：0.008212
# 迭代的次数：200, 误差值：0.004165
# 迭代的次数：300, 误差值：0.002800
# 迭代的次数：400, 误差值：0.002111
# 迭代的次数：500, 误差值：0.001696
# 迭代的次数：600, 误差值：0.001418
# 迭代的次数：700, 误差值：0.001219
# 迭代的次数：800, 误差值：0.001069
# 迭代的次数：900, 误差值：0.000952
# 迭代的次数：1000, 误差值：0.000858
params={
"w":w,
"b":b
}
grads={
"dw":dw,
"db":db
}
return params,grads,costs
# params,grads,costs=optimize(w,b,X,Y,1001,0.1,True)
# print(params)
# # {'w': array([[-0.0009077 ],
# # [-0.00262293]]), 'db': -0.0008576164239226247}
# print(grads)
# # {'dw': array([[-0.0009077 ],
# # [-0.00262293]]), 'db': -0.0008576164239226247}
# print(costs)
# # [1.3132616875182228, 0.008211563080222594, 0.0041654309736682636, 0.0027995659824453178, 0.002111333903432137, 0.0016960932319482194, 0.001418069094628399, 0.001218782485685477, 0.001068876940625712, 0.0009519888068423773, 0.0008582717476016353]
def predict(w,b,X):
m=X.shape[1]
#print(X.shape) #(2, 2)
Y_prediction=np.zeros((1,m))
w=w.reshape(X.shape[0],1)
A=sigmoid(np.dot(w.T,X)+b)
for i in range(A.shape[1]):
Y_prediction[0,i]=1 if A[0,i]>0.5 else 0
# # equal： if A[0,i]>0.5
# Y_prediciton[0,i]=1
# else
# Y_prediction[0,i]=0
#print(Y_prediction.shape) #(1, 2)
assert(Y_prediction.shape==(1,m))
return Y_prediction
# w=np.array([[1.0],[-1.0]])
# # w1=[1,2,3]
# # print(w.dtype,type(w1))# float64 <class 'list'>
# b=1
# # print(type(b))
# X=np.array([[1.0,2.0],[3.0,4.0]])
# #print(predict(w,b,X))#[[0. 0.]]
train_set_x_orig,train_set_y,test_set_x_orig,test_set_y,classes=load_dataset()
train_set_x_flatten=train_set_x_orig.reshape(train_set_x_orig.shape[0],-1).T
test_set_x_flatten=test_set_x_orig.reshape(test_set_x_orig.shape[0],-1).T
# print(np.shape(train_set_x_orig)) #(209, 64, 64, 3)
# print(np.shape(train_set_x_orig.reshape(train_set_x_orig.shape[0],-1))) #(209, 12288)
# print(np.shape(train_set_x_flatten)) #(12288, 209) 从这就可以看出来-1的作用
# print(np.shape(test_set_x_orig)) #(50, 64, 64, 3)
# print(np.shape(test_set_x_orig.reshape(test_set_x_orig.shape[0],-1))) #(50, 12288)
#print(np.shape(test_set_x_flatten)) #(12288, 50) 从这就可以看出来-1的作用
train_set_x=train_set_x_flatten/255 #像素归一化，减小误差（我的见解）
test_set_x=test_set_x_flatten/255 #像素归一化，减小误差（我的见解）
def model(X_train,Y_train,X_test,Y_test,num_iterations=2000,learning_rate=0.5,print_cost=False):
w,b=initialize_with_zeros(X_train.shape[0])
# print(X_train.shape) #(12288, 209)
# print(w.shape) #(12288, 1) one picture
# print(b.shape) #AttributeError: 'int' object has no attribute 'shape'
#print(X_train.shape,Y_test.shape) #(12288, 209) (1, 50)
parameters,grads,costs=optimize(w,b,X_train,Y_train,num_iterations,learning_rate,print_cost)
w,b=parameters["w"],parameters["b"]
Y_prediction_test=predict(w,b,X_test)
Y_prediction_train=predict(w,b,X_train)
# print(np.mean(Y_prediction_train), np.mean(Y_prediction_test))
# print(np.mean(np.abs((Y_prediction_train-Y_train))))
# print(np.mean(np.abs((Y_prediction_test-Y_test))))
print("训练集准确性：",format(100-np.mean(np.abs(Y_prediction_train-Y_train))*100),"%")
print("测试集准确性：",format(100-np.mean(np.abs(Y_prediction_test-Y_test))*100),"%")
d={
"costs":costs,
"Y_prediction_test":Y_prediction_test,
"Y_prediction_train":Y_prediction_train,
"w":w,
"b":b,
"learning_rate":learning_rate,
"num_iterations":num_iterations
}
# print(type(d)) #<class 'dict'>
# print(d.type) #AttributeError: 'dict' object has no attribute 'type'
return d
# d=model(train_set_x,train_set_y,test_set_x,test_set_y,num_iterations=2001,learning_rate=0.005,print_cost=True)
# costs=np.squeeze(d["costs"])
# plt.plot(costs)
# plt.ylabel("cost")
# plt.xlabel("iterations(per hundreds)")
# plt.title("learning rate= "+ str(d["learning_rate"]))
# plt.show()
models={}
learning_rates=[0.01,0.001,0.0001]
for i in learning_rates:
# print(i) #0.01 0.001 0.0001
# print(type(i)) #<class 'float'>
# print(i.dtype) #'float' object has no attribute 'dtype'
# cost[str[i]]=i #TypeError: 'type' object is not subscriptable
print("learning rate is :"+str(i)) #print("learning rates is :",str(i)) can also
models[str(i)]=model(train_set_x,train_set_y,test_set_x,test_set_y,num_iterations=2001,learning_rate=i,print_cost=True)
#print(models.dtype) #AttributeError: 'dict' object has no attribute 'dtype'
#print(models.shape) #AttributeError: 'dict' object has no attribute 'shape'
# models.append((model(train_set_x,train_set_y,test_set_x,test_set_y,num_iterations=2001,learning_rate[i],print_cost=True)))
print("\n"+"-----------------------------------------------"+"\n")
for i in learning_rates:
plt.plot(np.squeeze(models[str(i)]["costs"]),label=str(models[str(i)]["learning_rate"]))
plt.ylabel("cost")
plt.xlabel('iterations')
legend=plt.legend(loc="upper center",shadow=True) #legend : 图例
frame=legend.get_frame()
frame.set_facecolor("0.90")
plt.show()
2323
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
