张量定义
张量(tensor):多维数组或列表,其中阶表示张量的维数,即张量可以表示0阶到n阶的数组或列表
维数 阶 名称 例子 0D 0 标量 scalar s=1,s=2,s=3 1D 1 向量 vector v=[1,2,3] 2D 2 矩阵 matrix m=[[1,2],[3,4]] nD n 张量 tensor t=[[[[…,连续n个“[”
tensorflow数据类型
tf.int , tf.float … e.g. : tf.int 32, tf.float32, tf.float64tf.bool e.g. : tf.constant([True, False])tf.string e.g. : tf.constant(“Hello, world!”)
创建张量tensor方法
创建一个张量 tf.constant(张量内容,dtype=数据类型(可选))
import tensorflow as tf
a= tf. constant( [ 1 , 5 ] , dtype= tf. int64)
print ( a)
print ( a. dtype)
print ( a. shape)
< tf. Tensor( [ 1 , 5 ] , shape= ( 2 , ) , dtype= int64)
< dtype: 'int64' >
( 2 , )
将numpy的数据类型转换为Tensor数据类型 tf. convert_to_tensor(数据名,dtype=数据类型(可选))
import tensorflow as tf
import numpy as np
a = np. arange( 0 , 5 )
b = tf. convert_to_tensor( a, dtype= tf. int64)
print ( a)
print ( b)
[ 0 1 2 3 4 ]
tf. Tensor( [ 0 1 2 3 4 ] , shape= ( 5 , ) , dtype= int64)
创建特殊张量
全为0的张量:tf. zeros(维度) 全为1的张量:tf. ones(维度) 全为指定值的张量:tf. fill(维度,指定值) 维度:一维直接写个数;二维用[行,列];多维用[n,m,j,k……]
a = tf. zeros( [ 2 , 3 ] )
b = tf. ones( 4 )
c = tf. fill( [ 2 , 2 ] , 9 )
print ( a)
print ( b)
print ( c)
tf. Tensor( [ [ 0 . 0 . 0 . ] [ 0 . 0 . 0 . ] ] , shape= ( 2 , 3 ) , dtype= float32)
tf. Tensor( [ 1 . 1 . 1 . 1 . ] , shape= ( 4 , ) , dtype= float32)
tf. Tensor( [ [ 9 9 ] [ 9 9 ] ] , shape= ( 2 , 2 ) , dtype= int32)
生成正态分布的随机数,默认均值为0,标准差为1: tf. random.normal(维度,mean=均值,stddev=标准差) 生成截断式正态分布的随机数 tf. random.truncated_normal (维度,mean=均值,stddev=标准差) 在tf.truncated_normal中如果随机生成数据的取值在(μ-2σ,μ+2σ)之外则重新进行生成,保证了生成值在均值附近
d = tf. random. normal( [ 2 , 2 ] , mean= 0.5 , stddev= 1 )
print ( d)
e = tf. random. truncated_normal( [ 2 , 2 ] , mean= 0.5 , stddev= 1 )
print ( e)
tf. Tensor(
[ [ 0.7925745 0.643315 ]
[ 1.4752257 0.2533372 ] ] , shape= ( 2 , 2 ) , dtype= float32)
tf. Tensor(
[ [ 1.3688478 1.0125661 ]
[ 0.17475659 - 0.02224463 ] ] , shape= ( 2 , 2 ) , dtype= float32)
生成均匀分布随机数[minval, maxval) tf. random. uniform(维度,minval=最小值,maxval=最大值)
f = tf. random. uniform( [ 2 , 2 ] , minval= 0 , maxval= 1 )
print ( f)
tf. Tensor(
[ [ 0.28219545 0.15581512 ]
[ 0.77972126 0.47817433 ] ] , shape= ( 2 , 2 ) , dtype= float32)
tensorflow常用函数
强制tensor转换为该数据类型 tf.cast(张量名,dtype=数据类型)
x1 = tf. constant( [ 1 . , 2 . , 3 . ] , dtype= tf. float64)
print ( x1)
x2 = tf. cast( x1, tf. int32)
print ( x2)
tf. Tensor( [ 1 . 2 . 3 . ] , shape= ( 3 , ) , dtype= float64)
tf. Tensor( [ 1 2 3 ] , shape= ( 3 , ) , dtype= int32)
计算张量维度上元素的最值
计算张量维度上元素的最小值:tf.reduce_min(张量名) 计算张量维度上元素的最大值:tf.reduce_max(张量名)
print ( tf. reduce_min( x2) , tf. reduce_max( x2) )
tf. Tensor( 1 , shape= ( ) , dtype= int32)
tf. Tensor( 3 , shape= ( ) , dtype= int32)
维度操作函数
计算张量沿着指定维度的平均值:tf.reduce_mean(张量名,axis=操作轴) 计算张量沿着指定维度的和:tf.reduce_sum(张量名,axis=操作轴) 返回张量沿指定维度最大值的索引:tf.argmax(张量名,axis=操作轴) 在一个二维张量或数组中,可以通过调整axis等于0或1控制执行维度。axis=0代表跨行(经度,down),而axis=1代表跨列(纬度,across)。如果不指定axis,则所有元素参与计算
x= tf. constant( [ [ 1 , 2 , 3 ] , [ 2 , 2 , 3 ] ] )
print ( x)
print ( tf. reduce_mean( x) )
print ( tf. reduce_sum( x, axis= 1 ) )
import numpy as np
test = np. array( [ [ 1 , 2 , 3 ] , [ 2 , 3 , 4 ] , [ 5 , 4 , 3 ] , [ 8 , 7 , 2 ] ] )
print ( test)
print ( tf. argmax( test, axis= 0 ) )
print ( tf. argmax( test, axis= 1 ) )
tf. Tensor( [ [ 1 2 3 ] [ 2 2 3 ] ] , shape= ( 2 , 3 ) , dtype= int32)
tf. Tensor( 2 , shape= ( ) , dtype= int32)
tf. Tensor( [ 6 7 ] , shape= ( 2 , ) , dtype= int32)
[ [ 1 2 3 ]
[ 2 3 4 ]
[ 5 4 3 ]
[ 8 7 2 ] ]
tf. Tensor( [ 3 3 1 ] , shape= ( 3 , ) , dtype= int64)
tf. Tensor( [ 2 2 0 0 ] , shape= ( 4 , ) , dtype= int64)
参数更新函数
将变量标记为“可训练” tf.Variable(初始值) 被标记的变量会在反向传播中记录梯度信息。神经网络训练中,常用该函数标记待训练参数
w = tf. Variable( tf. random. normal( [ 2 , 2 ] , mean= 0 , stddev= 1 ) )
求出张量的梯度 grad=tape.gradient(函数,对谁求导) with结构记录计算过程,gradient求出张量的梯度 e.g. : 令loss=w2 ,则loss’=2w
with tf. GradientTape( ) as tape:
w = tf. Variable( tf. constant( 3.0 ) )
loss = tf. pow ( w, 2 )
grad = tape. gradient( loss, w)
print ( grad)
tf. Tensor( 6.0 , shape= ( ) , dtype= float32)
自减函数 w.assign_sub(w要自减的内容) 赋值操作,更新参数的值并返回。调用assign_sub前,先用tf.Variable定义变量w为可训练(可自更新)。
w = tf. Variable( 4 )
w. assign_sub( 1 )
print ( w)
< tf. Variable 'Variable:0' shape= ( ) dtype= int32, numpy= 3 >
四则运算
实现两个张量的对应元素相加 tf.add(张量1,张量2) 实现两个张量的对应元素相减 tf.subtract(张量1,张量2) 实现两个张量的对应元素相乘 tf.multiply(张量1,张量2) 实现两个张量的对应元素相除 tf.divide(张量1,张量2) 只有维度相同的张量才可以做四则运算
a = tf. ones( [ 1 , 3 ] )
b = tf. fill( [ 1 , 3 ] , 3 . )
print ( a)
print ( b)
print ( tf. add( a, b) )
print ( tf. subtract( a, b) )
print ( tf. multiply( a, b) )
print ( tf. divide( b, a) )
tf. Tensor( [ [ 1 . 1 . 1 . ] ] , shape= ( 1 , 3 ) , dtype= float32)
tf. Tensor( [ [ 3 . 3 . 3 . ] ] , shape= ( 1 , 3 ) , dtype= float32
tf. Tensor( [ [ 4 . 4 . 4 . ] ] , shape= ( 1 , 3 ) , dtype= float32)
tf. Tensor( [ [ - 2 . - 2 . - 2 . ] ] , shape= ( 1 , 3 ) , dtype= float32)
tf. Tensor( [ [ 3 . 3 . 3 . ] ] , shape= ( 1 , 3 ) , dtype= float32)
tf. Tensor( [ [ 3 . 3 . 3 . ] ] , shape= ( 1 , 3 ) , dtype= float32)
平方、次方与开方
计算某个张量的平方 tf.square(张量名) 计算某个张量的n次方 tf.pow(张量名,n次方数) 计算某个张量的开方 tf.sqrt(张量名)
a = tf. fill( [ 1 , 2 ] , 3 . )
print ( a)
print ( tf. pow ( a, 3 ) )
print ( tf. square( a) )
print ( tf. sqrt( a) )
tf. Tensor( [ [ 3 . 3 . ] ] , shape= ( 1 , 2 ) , dtype= float32)
tf. Tensor( [ [ 27 . 27 . ] ] , shape= ( 1 , 2 ) , dtype= float32)
tf. Tensor( [ [ 9 . 9 . ] ] , shape= ( 1 , 2 ) , dtype= float32)
tf. Tensor( [ [ 1.7320508 1.7320508 ] ] , shape= ( 1 , 2 ) , dtype= float32)
矩阵乘法 tf.matmul(矩阵1,矩阵2)
a = tf. ones( [ 3 , 2 ] )
b = tf. fill( [ 2 , 3 ] , 3 . )
print ( tf. matmul( a, b) )
tf. Tensor(
[ [ 6 . 6 . 6 . ]
[ 6 . 6 . 6 . ]
[ 6 . 6 . 6 . ] ] , shape= ( 3 , 3 ) , dtype= float32)
切分传入张量的第一维度,生成输入特征/标签对 data = tf.data.Dataset.from_tensor_slices((输入特征, 标签)) Numpy和Tensor格式都可用该语句读入数据
features = tf. constant( [ 12 , 23 , 10 , 17 ] )
labels = tf. constant( [ 0 , 1 , 1 , 0 ] )
dataset = tf. data. Dataset. from_tensor_slices( ( features, labels) )
print ( dataset)
for element in dataset:
print ( element)
< TensorSliceDataset shapes: ( ( ) , ( ) ) , types: ( tf. int32, tf. int32) ) >
( < tf. Tensor: id = 9 , shape= ( ) , dtype= int32, numpy= 12 > , < tf. Tensor: id = 10 , shape= ( ) , dtype= int32, numpy= 0 > )
( < tf. Tensor: id = 11 , shape= ( ) , dtype= int32, numpy= 23 > , < tf. Tensor: id = 12 , shape= ( ) , dtype= int32, numpy= 1 > )
( < tf. Tensor: id = 13 , shape= ( ) , dtype= int32, numpy= 10 > , < tf. Tensor: id = 14 , shape= ( ) , dtype= int32, numpy= 1 > )
( < tf. Tensor: id = 15 , shape= ( ) , dtype= int32, numpy= 17 > , < tf. Tensor: id = 16 , shape= ( ) , dtype= int32, numpy= 0 > )
枚举 enumerate(列表名) enumerate是python的内建函数,它可遍历每个元素(如列表、元组或字符串),组合为:索引元素,常在for循环中使用。
seq= [ 'one' , 'two' , 'three' ]
for i, element in enumerate ( seq) :
print ( i, element)
0 one
1 two
2 three
独热编码 tf.one_hot(待转换数据, depth=几分类) tf.one_hot()函数将待转换数据,转换为one-hot形式的数据输出 独热编码(one-hot encoding):在分类问题中,常用独热码做标签,标记类别:1表示是,0表示非。
0狗尾草鸢尾 1杂色鸢尾 2弗吉尼亚鸢尾 标签 0 独热码 1. 0. 0. 标签 1 独热码 0. 1. 0. 标签 2 独热码 0. 0. 1.
classes = 3
labels = tf. constant( [ 1 , 0 , 2 ] )
output = tf. one_hot( labels, depth= classes)
print ( output)
[ [ 0 . 1 . 0 . ]
[ 1 . 0 . 0 . ]
[ 0 . 0 . 1 . ] ] , shape= ( 3 , 3 ) , dtype= float32)
激活函数
sigmoid函数 tf.nn.sigmoid(待激活张量) tanh函数 tf.nn.tanh(待激活张量) relu函数 tf.nn.relu(待激活张量)
sess= tf. Session( )
a= tf. constant( [ 10 , 2 , 1 , 0.5 , 0 , - 0.5 , - 1 , - 2 , - 10 ] )
print ( sess. run( a) )
S= tf. nn. sigmoid( a)
print ( sess. run( S) )
T= tf. nn. tanh( a)
print ( sess. run( T) )
R= tf. nn. relu( a)
print ( sess. run( R) )
[ 10 . 2 . 1 . 0.5 0 . - 0.5 - 1 . - 2 . - 10 . ]
[ 9.9995458e-01 8.8079709e-01 7.3105860e-01 6.2245929e-01 5.0000000e-01 3.7754071e-01 2.6894143e-01 1.1920291e-01 4.5397872e-05 ]
[ 1 . 0.9640276 0.7615942 0.46211717 0 . - 0.46211717 - 0.7615942 - 0.9640276 - 1 . ]
[ 10 . 2 . 1 . 0.5 0 . 0 . 0 . 0 . 0 . ]
损失函数
均方误差mse tf.reduce_mean(tf.square(标准值 - 预测值)) 交叉熵损失函数CE tf.losses.categorical_crossentropy(标准值,预测值)
loss1= tf. losses. categorical_crossentropy( [ 1 , 0 ] , [ 0.6 , 0.4 ] )
loss2= tf. losses. categorical_crossentropy( [ 1 , 0 ] , [ 0.8 , 0.2 ] )
print ( loss1)
print ( loss2)
tf. Tensor( 0.5108256 , shape= ( ) , dtype= float32)
tf. Tensor( 0.22314353 , shape= ( ) , dtype= float32)
softmax函数 tf.nn.softmax(x)
y = tf. constant( [ 1.01 , 2.01 , - 0.66 ] )
y_pro= tf. nn. softmax( y)
print ( y_pro)
tf. Tensor( [ 0.255981740 .695830460 .0481878 ] , shape= ( 3 , ) , dtype= float32)
softmax与交叉熵结合 tf.nn.softmax_cross_entropy_with_logits(标准值 - 预测值) 输出先过softmax函数,再计算y与y_的交叉熵损失函数
y_ = np. array( [ [ 1 , 0 , 0 ] , [ 0 , 1 , 0 ] , [ 0 , 0 , 1 ] , [ 1 , 0 , 0 ] , [ 0 , 1 , 0 ] ] )
y = np. array( [ [ 12 , 3 , 2 ] , [ 3 , 10 , 1 ] , [ 1 , 2 , 5 ] , [ 4 , 6.5 , 1.2 ] , [ 3 , 6 , 1 ] ] )
y_pro= tf. nn. softmax( y)
loss1 = tf. losses. categorical_crossentropy( y_, y_pro)
loss2 = tf. nn. softmax_cross_entropy_with_logits( y_, y)
print ( '分步计算的结果:\n' , loss1)
print ( '结合计算的结果:\n' , loss2)
分步计算的结果:
tf. Tensor( [ 1.68795487e-041 .03475622e-036 .58839038e-022 .58349207e+005 .49852354e-02 ] , shape= ( 5 , ) , dtype= float64)
结合计算的结果:
tf. Tensor( [ 1.68795487e-041 .03475622e-036 .58839038e-022 .58349207e+005 .49852354e-02 ] , shape= ( 5 , ) , dtype= float64)
实例:训练Iris数据集
整体思路
准备数据 • 数据集读入 • 数据集乱序 • 生成训练集和测试集 • 配成(输入特征,标签)对,每次读入一小撮(batch) 搭建网络 • 定义神经网路中所有可训练参数 参数优化 • 嵌套循环迭代,with结构更新参数,显示当前loss 测试效果 • 计算当前参数前向传播后的准确率,显示当前acc acc / loss可视化
import tensorflow as tf
from sklearn import datasets
from matplotlib import pyplot as plt
import numpy as np
x_data = datasets. load_iris( ) . data
y_data = datasets. load_iris( ) . target
np. random. seed( 0 )
np. random. shuffle( x_data)
np. random. seed( 0 )
np. random. shuffle( y_data)
tf. random. set_seed( 0 )
x_train = x_data[ : - 30 ]
y_train = y_data[ : - 30 ]
x_test = x_data[ - 30 : ]
y_test = y_data[ - 30 : ]
x_train = tf. cast( x_train, tf. float32)
x_test = tf. cast( x_test, tf. float32)
train_db = tf. data. Dataset. from_tensor_slices( ( x_train, y_train) ) . batch( 32 )
test_db = tf. data. Dataset. from_tensor_slices( ( x_test, y_test) ) . batch( 32 )
w1 = tf. Variable( tf. random. truncated_normal( [ 4 , 3 ] , stddev= 0.1 , seed= 1 ) )
b1 = tf. Variable( tf. random. truncated_normal( [ 3 ] , stddev= 0.1 , seed= 1 ) )
lr = 0.1
train_loss_results = [ ]
test_acc = [ ]
epoch = 500
loss_all = 0
for epoch in range ( epoch) :
for step, ( x_train, y_train) in enumerate ( train_db) :
with tf. GradientTape( ) as tape:
y = tf. matmul( x_train, w1) + b1
y = tf. nn. softmax( y)
y_ = tf. one_hot( y_train, depth= 3 )
loss = tf. reduce_mean( tf. square( y_ - y) )
loss_all += loss. numpy( )
grads = tape. gradient( loss, [ w1, b1] )
w1. assign_sub( lr * grads[ 0 ] )
b1. assign_sub( lr * grads[ 1 ] )
print ( "Epoch {}, loss: {}" . format ( epoch, loss_all/ 4 ) )
train_loss_results. append( loss_all / 4 )
loss_all = 0
total_correct, total_number = 0 , 0
for x_test, y_test in test_db:
y = tf. matmul( x_test, w1) + b1
y = tf. nn. softmax( y)
pred = tf. argmax( y, axis= 1 )
pred = tf. cast( pred, dtype= y_test. dtype)
correct = tf. cast( tf. equal( pred, y_test) , dtype= tf. int32)
correct = tf. reduce_sum( correct)
total_correct += int ( correct)
total_number += x_test. shape[ 0 ]
acc = total_correct / total_number
test_acc. append( acc)
print ( "Test_acc:" , acc)
print ( "--------------------------" )
plt. title( 'Loss Function Curve' )
plt. xlabel( 'Epoch' )
plt. ylabel( 'Loss' )
plt. plot( train_loss_results, label= "$Loss$" )
plt. legend( )
plt. show( )
plt. title( 'Acc Curve' )
plt. xlabel( 'Epoch' )
plt. ylabel( 'Acc' )
plt. plot( test_acc, label= "$Accuracy$" )
plt. legend( )
plt. show( )