tensorflow构建CNN模型时的常用接口函数

(1)tf.nn.max_pool()函数

解释:

tf.nn.max_pool(value, ksize, strides, padding, data_format='NHWC', name=None)

需要设置的参数主要有四个:

第一个参数value:需要池化的输入,一般池化层接在卷积层后面,所以输入通常是feature map,依然是[batch, height, width, channels]这样的shape

第二个参数ksize:池化窗口的大小,取一个四维向量,一般是[1, height, width, 1],因为我们不想在batch和channels上做池化,所以这两个维度设为了1

第三个参数strides:和卷积类似,窗口在每一个维度上滑动的步长,一般也是[1, stride,stride, 1]

第四个参数padding:和卷积类似,可以取'VALID' 或者'SAME'

返回一个Tensor,类型不变,shape仍然是[batch, height, width, channels]这种形式

示例

程序:

import tensorflow as tf  

a=tf.constant([  
        [[1.0,2.0,3.0,4.0],  
        [5.0,6.0,7.0,8.0],  
        [8.0,7.0,6.0,5.0],  
        [4.0,3.0,2.0,1.0]],  
        [[4.0,3.0,2.0,1.0],  
         [8.0,7.0,6.0,5.0],  
         [1.0,2.0,3.0,4.0],  
         [5.0,6.0,7.0,8.0]]  
    ])  

a=tf.reshape(a,[1,4,4,2])  

pooling=tf.nn.max_pool(a,[1,2,2,1],[1,1,1,1],padding='VALID')  
with tf.Session() as sess:  
    print("image:")  
    image=sess.run(a)  
    print (image)  
    print("reslut:")  
    result=sess.run(pooling)  
    print (result)  

运行结果:

image:  
[[[[ 1.  2.]  
   [ 3.  4.]  
   [ 5.  6.]  
   [ 7.  8.]]  

  [[ 8.  7.]  
   [ 6.  5.]  
   [ 4.  3.]  
   [ 2.  1.]]  

  [[ 4.  3.]  
   [ 2.  1.]  
   [ 8.  7.]  
   [ 6.  5.]]  

  [[ 1.  2.]  
   [ 3.  4.]  
   [ 5.  6.]  
   [ 7.  8.]]]]  
reslut:  
[[[[ 8.  7.]  
   [ 6.  6.]  
   [ 7.  8.]]  

  [[ 8.  7.]  
   [ 8.  7.]  
   [ 8.  7.]]  

  [[ 4.  4.]  
   [ 8.  7.]  
   [ 8.  8.]]]]  

(2)tf.nn.dropout函数

解释

tf.nn.dropout(x, keep_prob, noise_shape=None, seed=None, name=None)

此函数是为了防止在训练中过拟合的操作,将训练输出按一定规则进行变换

参数:
x:输入
keep_prob:保留比例。        取值 (0,1] 。每一个参数都将按这个比例随机变更
noise_shape:干扰形状。     此字段默认是None,表示第一个元素的操作都是独立,但是也不一定。比例:数据的形状是shape(x)=[k, l, m, n],而noise_shape=[k, 1, 1, n],则第14列是独立保留或删除,第23列是要么全部保留,要么全部删除。
seed:整形变量,随机数种子。
name:名字,没啥用
返回:Tnesor

(3)tf.nn.local_response_normalization函数

公式说明

local response normalization最早是由Krizhevsky和Hinton在关于ImageNet的论文里面使用的一种数据标准化方法,即使现在,也依然会有不少CNN网络会使用到这种正则手段,现在记录一下lrn方法的计算流程以及tensorflow的实现,方便后面查阅

无

以上是这种归一手段的公式,其中a的上标指该层的第几个feature map,a的下标x,y表示feature map的像素位置,N指feature map的总数量,公式里的其它参数都是超参,需要自己指定的。

这种方法是受到神经科学的启发,激活的神经元会抑制其邻近神经元的活动(侧抑制现象),至于为什么使用这种正则手段,以及它为什么有效,查阅了很多文献似乎也没有详细的解释,可能是由于后来提出的batch normalization手段太过火热,渐渐的就把local response normalization掩盖了吧

解释

tf.nn.local_response_normalization(input, depth_radius=5, bias=1, alpha=1, beta=0.5, name=None)

除去name参数用以指定该操作的name,与方法有关的一共五个参数: 

第一个参数input:这个输入就是feature map了,既然是feature map,那么它就具有[batch, height, width, channels]这样的shape
第二个参数depth_radius:这个值需要自己指定,就是上述公式中的n/2
第三个参数bias:上述公式中的k
第四个参数alpha:上述公式中的α
第五个参数beta:上述公式中的β

程序

import tensorflow as tf  

a = tf.constant([  
    [[1.0, 2.0, 3.0, 4.0],  
     [5.0, 6.0, 7.0, 8.0],  
     [8.0, 7.0, 6.0, 5.0],  
     [4.0, 3.0, 2.0, 1.0]],  
    [[4.0, 3.0, 2.0, 1.0],  
     [8.0, 7.0, 6.0, 5.0],  
     [1.0, 2.0, 3.0, 4.0],  
     [5.0, 6.0, 7.0, 8.0]]  
])  
#reshape a,get the feature map [batch:1 height:2 width:2 channels:8]  
a = tf.reshape(a, [1, 2, 2, 8])  

normal_a=tf.nn.local_response_normalization(a,2,0,1,1)  
with tf.Session() as sess:  
    print("feature map:")  
    image = sess.run(a)  
    print (image)  
    print("normalized feature map:")  
    normal = sess.run(normal_a)  
    print (normal)  

输出结果

feature map:  
[[[[ 1.  2.  3.  4.  5.  6.  7.  8.]  
   [ 8.  7.  6.  5.  4.  3.  2.  1.]]  

  [[ 4.  3.  2.  1.  8.  7.  6.  5.]  
   [ 1.  2.  3.  4.  5.  6.  7.  8.]]]]  
normalized feature map:  
[[[[ 0.07142857  0.06666667  0.05454545  0.04444445  0.03703704  0.03157895  
     0.04022989  0.05369128]  
   [ 0.05369128  0.04022989  0.03157895  0.03703704  0.04444445  0.05454545  
     0.06666667  0.07142857]]  

  [[ 0.13793103  0.10000001  0.0212766   0.00787402  0.05194805  0.04  
     0.03448276  0.04545454]  
   [ 0.07142857  0.06666667  0.05454545  0.04444445  0.03703704  0.03157895  
     0.04022989  0.05369128]]]]  

(4)tf.get_variable函数

函数定义

get_variable(
    name,
    shape=None,
    dtype=None,
    initializer=None,
    regularizer=None,
    trainable=True,
    collections=None,
    caching_device=None,
    partitioner=None,
    validate_shape=True,
    use_resource=None,
    custom_getter=None
)

其中参数分别为

参数:
name:新变量或现有变量的名称。

shape:新变量或现有变量的形状。

dtype:新变量或现有变量的类型(默认为 DT_FLOAT)。

initializer:创建变量的初始化器。

regularizer:一个函数(张量 - >张量或无);将其应用于新创建的变量的结果将被添加到集合 tf.GraphKeys.REGULARIZATION_LOSSES 中,并可用于正则化。

trainable:如果为 True,还将变量添加到图形集合:GraphKeys.TRAINABLE_VARIABLES。
collections:要将变量添加到其中的图形集合键的列表。默认为 [GraphKeys.LOCAL_VARIABLES]。

caching_device:可选的设备字符串或函数,描述变量应该被缓存以读取的位置。默认为变量的设备,如果不是 None,则在其他设备上进行缓存。典型的用法的在使用该变量的操作所在的设备上进行缓存,通过 Switch 和其他条件语句来复制重复数据删除。

partitioner:(可选)可调用性,它接受要创建的变量的完全定义的 TensorShape 和 dtype,并且返回每个坐标轴的分区列表(当前只能对一个坐标轴进行分区)。

validate_shape:如果为假,则允许使用未知形状的值初始化变量。如果为真,则默认情况下,initial_value 的形状必须是已知的。
use_resource:如果为假,则创建一个常规变量。如果为真,则创建一个实验性的 ResourceVariable,而不是具有明确定义的语义。默认为假(稍后将更改为真)。

custom_getter:可调用的,将第一个参数作为真正的 getter,并允许覆盖内部的 get_variable 方法。custom_getter 的签名应该符合这种方法,但最经得起未来考验的版本将允许更改:def custom_getter(getter, *args, **kwargs)。还允许直接访问所有 get_variable 参数:def custom_getter(getter, name, *args, **kwargs)。创建具有修改的名称的变量的简单标识自定义 getter 是:python def custom_getter(getter, name, *args, **kwargs): return getter(name + '_suffix', *args, **kwargs) 

使用例子

w = tf.get_variable("w", shape = [inputD, outputD], dtype = "float")
b = tf.get_variable("b", [outputD], dtype = "float")

(5)tf.variable_scope函数

函数原理

用于定义创建变量(层)的操作的上下文管理器。
此上下文管理器验证(可选)values是否来自同一图形,确保图形是默认的图形,并推送名称范围和变量范围。
如果name_or_scope不是None,则使用as is。如果scope是None,则使用default_name。在这种情况下,如果以前在同一范围内使用过相同的名称,则通过添加_N来使其具有唯一性。
变量范围允许您创建新变量并共享已创建的变量,同时提供检查以防止意外创建或共享。在本文中我们提供了几个基本示例。

如何创建一个新变量

with tf.variable_scope("foo"):
    with tf.variable_scope("bar"):
        v = tf.get_variable("v", [1])
        assert v.name == "foo/bar/v:0"

(6)tf.nn.relu函数

解释

这个函数的作用是计算激活函数relu,即max(features, 0)。即将矩阵中每行的非最大值置0。

类似的还有tf.sigmoid , tf.tanh

函数定义

>>> help(tf.nn.relu)
Help on function relu in module tensorflow.python.ops.gen_nn_ops:

relu(features, name=None)
    Computes rectified linear: `max(features, 0)`.

    Args:
      features: A `Tensor`. Must be one of the following types: `float32`, `float64`, `int32`, `uint8`, `int16`, `int8`, `int64`, `bfloat16`, `uint16`, `half`, `uint32`, `uint64`.
      name: A name for the operation (optional).

    Returns:
      A `Tensor`. Has the same type as `features`.

程序示例

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import tensorflow as tf

a = tf.constant([-1.0, 2.0])
with tf.Session() as sess:
    b = tf.nn.relu(a)
    print sess.run(b)

运行结果

[0. 2.]

(7)tf.nn.bias_add函数

函数定义

tf.nn.bias_add(value, bias, data_format=None, name=None)


对value加一偏置量
此函数为tf.add的特殊情况,bias仅为一维,
函数通过广播机制进行与value求和,
数据格式可以与value不同,返回为与value相同格式

官方释义

>>> help(tf.nn.bias_add)
Help on function bias_add in module tensorflow.python.ops.nn_ops:

bias_add(value, bias, data_format=None, name=None)
    Adds `bias` to `value`.

    This is (mostly) a special case of `tf.add` where `bias` is restricted to 1-D.
    Broadcasting is supported, so `value` may have any number of dimensions.
    Unlike `tf.add`, the type of `bias` is allowed to differ from `value` in the
    case where both types are quantized.

    Args:
      value: A `Tensor` with type `float`, `double`, `int64`, `int32`, `uint8`,
        `int16`, `int8`, `complex64`, or `complex128`.
      bias: A 1-D `Tensor` with size matching the last dimension of `value`.
        Must be the same type as `value` unless `value` is a quantized type,
        in which case a different quantized type may be used.
      data_format: A string. 'NHWC' and 'NCHW' are supported.
      name: A name for the operation (optional).

    Returns:
      A `Tensor` with the same type as `value`.

使用示例

out = tf.nn.bias_add(mergeFeatureMap, b)

(8)tf.nn.xw_plus_b函数

官方解释

>>> help(tf.nn.xw_plus_b)
Help on function xw_plus_b in module tensorflow.python.ops.nn_ops:

xw_plus_b(x, weights, biases, name=None)
    Computes matmul(x, weights) + biases.

    Args:
      x: a 2D tensor.  Dimensions typically: batch, in_units
      weights: a 2D tensor.  Dimensions typically: in_units, out_units
      biases: a 1D tensor.  Dimensions: out_units
      name: A name for the operation (optional).  If not specified
        "xw_plus_b" is used.

    Returns:
      A 2-D Tensor computing matmul(x, weights) + biases.
      Dimensions typically: batch, out_units.

使用示例

out = tf.nn.xw_plus_b(x, w, b, name = scope.name)

解释

xw_plus_b(x, weights, biases, name=None)相当于matmul(x, weights) + biases.

(9)tf.nn.conv2d函数

官方解释

>>> help(tf.nn.conv2d)
Help on function conv2d in module tensorflow.python.ops.gen_nn_ops:

conv2d(input, filter, strides, padding, use_cudnn_on_gpu=True, data_format='NHWC', dilations=[1, 1, 1, 1], name=None)
    Computes a 2-D convolution given 4-D `input` and `filter` tensors.

    Given an input tensor of shape `[batch, in_height, in_width, in_channels]`

    and a filter / kernel tensor of shape

    `[filter_height, filter_width, in_channels, out_channels]`, this op

    performs the following:



    1. Flattens the filter to a 2-D matrix with shape

       `[filter_height * filter_width * in_channels, output_channels]`.

    2. Extracts image patches from the input tensor to form a *virtual*

       tensor of shape `[batch, out_height, out_width,

       filter_height * filter_width * in_channels]`.

    3. For each patch, right-multiplies the filter matrix and the image patch

       vector.



    In detail, with the default NHWC format,



        output[b, i, j, k] =

            sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] *

                            filter[di, dj, q, k]



    Must have `strides[0] = strides[3] = 1`.  For the most common case of the same

    horizontal and vertices strides, `strides = [1, stride, stride, 1]`.

    Args:
      input: A `Tensor`. Must be one of the following types: `half`, `bfloat16`, `float32`.
        A 4-D tensor. The dimension order is interpreted according to the value

        of `data_format`, see below for details.
      filter: A `Tensor`. Must have the same type as `input`.
        A 4-D tensor of shape

        `[filter_height, filter_width, in_channels, out_channels]`
      strides: A list of `ints`.
        1-D tensor of length 4.  The stride of the sliding window for each

        dimension of `input`. The dimension order is determined by the value of

        `data_format`, see below for details.
      padding: A `string` from: `"SAME", "VALID"`.
        The type of padding algorithm to use.
      use_cudnn_on_gpu: An optional `bool`. Defaults to `True`.
      data_format: An optional `string` from: `"NHWC", "NCHW"`. Defaults to `"NHWC"`.
        Specify the data format of the input and output data. With the

        default format "NHWC", the data is stored in the order of:

            [batch, height, width, channels].

        Alternatively, the format could be "NCHW", the data storage order of:

            [batch, channels, height, width].
      dilations: An optional list of `ints`. Defaults to `[1, 1, 1, 1]`.
        1-D tensor of length 4.  The dilation factor for each dimension of

        `input`. If set to k > 1, there will be k-1 skipped cells between each

        filter element on that dimension. The dimension order is determined by the

        value of `data_format`, see above for details. Dilations in the batch and

        depth dimensions must be 1.
      name: A name for the operation (optional).

    Returns:
      A `Tensor`. Has the same type as `input`.

函数释义

此函数的作用是在给定四维输入(input)和权重W(filter)的情况下计算二维卷积。

参数解释:
input:
一个Tensor,每个元素的格式必须为float32或float64.
input的形状:[batch,in_height,in_width,in_channels],
batch为训练过程中每迭代一次迭代的照片数。
in_height,in_width分别为图片的高和宽
in_channels为图片的道。
filter:
一个Tensor,每个元素的类型和input类型一致。
filter的形状:[filter_height,filter_width,in_channels,out_channels]
分别为权重的height,width,输入的channels和输出的channels
stride:
长度为4的list,元素类型为int。表示每一维度滑动的步长。
需要注意的是,strides[0]=strides[3]=1.
padding:
可选参数为"Same","VALID"
边距,一般设为0,即padding='SAME'
use_cudnn_on_gpu:
bool类型,有True和False两种选择。
name:
此操作的名字
函数执行以下操作:
1.将参数filter变为一个二维矩阵,形状为:[filter_height*filter_width*in_channels,output_channels]
2.将输入(input)转化为一个具有如下形状的Tensor,形状为:[batch,out_height,out_width,filter_height * filter_width * in_channels]
3.将filter矩阵和步骤2得到的矩阵相乘。

函数的返回值:
元素类型和input相同。

output[b, i, j, k] =sum_{di, dj, q} input[b, strides[1] * i + di, strides[2] * j + dj, q] * filter[di, dj, q, k]

编程示例

kernel = tf.Variable(tf.truncated_normal([3,3,384,256], dtype=tf.float32, stddev=1e-1), name='weights')
conv = tf.nn.conv2d(conv3, kernel, [1,1,1,1],padding='SAME')

(10)tf.constant函数

官方解释

>>> help(tf.constant)
Help on function constant in module tensorflow.python.framework.constant_op:

constant(value, dtype=None, shape=None, name='Const', verify_shape=False)
    Creates a constant tensor.

    The resulting tensor is populated with values of type `dtype`, as
    specified by arguments `value` and (optionally) `shape` (see examples
    below).

    The argument `value` can be a constant value, or a list of values of type
    `dtype`. If `value` is a list, then the length of the list must be less
    than or equal to the number of elements implied by the `shape` argument (if
    specified). In the case where the list length is less than the number of
    elements specified by `shape`, the last element in the list will be used
    to fill the remaining entries.

    The argument `shape` is optional. If present, it specifies the dimensions of
    the resulting tensor. If not present, the shape of `value` is used.

    If the argument `dtype` is not specified, then the type is inferred from
    the type of `value`.

    For example:

    ```python
    # Constant 1-D Tensor populated with value list.
    tensor = tf.constant([1, 2, 3, 4, 5, 6, 7]) => [1 2 3 4 5 6 7]

    # Constant 2-D tensor populated with scalar value -1.
    tensor = tf.constant(-1.0, shape=[2, 3]) => [[-1. -1. -1.]
                                                 [-1. -1. -1.]]
    ```

    Args:
      value:          A constant value (or list) of output type `dtype`.

      dtype:          The type of the elements of the resulting tensor.

      shape:          Optional dimensions of resulting tensor.

      name:           Optional name for the tensor.

      verify_shape:   Boolean that enables verification of a shape of values.

    Returns:
      A Constant Tensor.

    Raises:
      TypeError: if shape is incorrectly specified or unsupported.

程序示例

无

最近学习tensorflow始终有点不得其法,不知道该怎么入门,欢迎各位来交流。

已标记关键词 清除标记
©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页