tf API：网络构建模块tf.nn，tf.layers， tf.contrib

        在使用tensorflow构建神经网络时，经常看到tf.nn，tf.layers，tf.contrib这三个模块，它们有很多函数的功能是相同的，部分可以混用，这就导致使用者很容易被迷惑，本文在此做一下笔记以理清思路。

tf.nn：用于原始神经网络（NN）操作的包装程序，属于基础操作，常见包括卷积（conv、conv_transpose、dilation2D、separable_conv）、池化（avg_pool，max_pool）、归一化（batch_normalization）、部分loss（ctc_loss）、激活函数（relu，crelu、swish）、分类操作（softmax）、Dropout、RNN（rnn_cell）等。

tf.layers：提供公共API构建网络，可看作是比nn更高层的封装，每个函数有更多可选的输入参数设置，主要是卷积（Conv）、全连接（Dense）、展平（Flatten）、Dropout、池化（Pool）、空间约束（Layer、InputSpec）、归一化（BatchNormalization）等。

tf.contrib: 各类模块的实验代码，包括各类高级操作的API，在构建网络的高级操作API中，包含tf.contrib.layers、tf.contrib.nn和tf.contrib.slim（好用的模块），属于实验代码，不稳定。但在TF2.0移除了contrib模块。

个人：由于tensorflow已经发展到2.0以上，1.x移植到2.x还是存在不兼容，建议采用tf.keras, 采用tensorflow作为后端。

 

以下为各模块对应的子模块、成员与函数：

tf.nn：

Modules

rnn_cell module: 构建RNN.

Functions

all_candidate_sampler(...): 生成所有类的集合.

atrous_conv2d(...): 卷积（又称带孔卷积或膨胀卷积）.

atrous_conv2d_transpose(...): atrous_conv2d的转置.

avg_pool(...): 平均池化.

avg_pool1d(...): 平均池化.

avg_pool2d(...): 平均池化.

avg_pool3d(...): 平均池化.

avg_pool_v2(...): 平均池化.

batch_norm_with_global_normalization(...): 批量归一化.

batch_normalization(...): 批量归一化.

bias_add(...): 添加偏置.

bidirectional_dynamic_rnn(...): 创建双向递归神经网络的动态版本. (deprecated)

collapse_repeated(...): 将重复的标签合并为单个标签.

compute_accidental_hits(...): 计算与true_classes匹配的sampled_candidates中的位置ID.

compute_average_loss(...): 使用sample_weights缩放每个示例的损失并计算其平均值.

conv1d(...): 给定3D输入和滤波器张量，计算1D卷积。. (deprecated argument values) (deprecated argument values)

conv1d_transpose(...): 转置 conv1d.

conv2d(...): 给定4-D输入和滤波器张量，计算2-D卷积。.

conv2d_backprop_filter(...): 计算相对于滤波器的卷积梯度.

conv2d_backprop_input(...): 计算相对于输入的卷积梯度.

conv2d_transpose(...): 转置 of conv2d.

conv3d(...): 给定5-D输入和滤波器张量，计算3-D卷积.

conv3d_backprop_filter(...): 计算3D卷积相对于滤波器的梯度.

conv3d_backprop_filter_v2(...): 计算3D卷积相对于滤波器的梯度.

conv3d_transpose(...): 转置 of conv3d.

conv_transpose(...): 转置 of convolution.

convolution(...): 计算N-D卷积的和 (actually cross-correlation).

crelu(...): 计算级联的ReLU.

ctc_beam_search_decoder(...): 对输入中给出的对数执行波束搜索解码.

ctc_beam_search_decoder_v2(...): 对输入中给出的对数执行波束搜索解码.

ctc_greedy_decoder(...): 对输入中给出的对数执行贪婪解码（最佳路径）.

ctc_loss(...): 计算CTC（连接器时间分类）损失.

ctc_loss_v2(...): 计算CTC（连接器时间分类）损失.

ctc_unique_labels(...): 获取tf.nn.ctc_loss的批处理标签的唯一标签和索引 .

depth_to_space(...): T型张量的DepthToSpace.

depthwise_conv2d(...): Depthwise 2-D convolution.

depthwise_conv2d_backprop_filter(...): 计算相对于滤波器的深度卷积的梯度.

depthwise_conv2d_backprop_input(...): 计算相对于输入的深度卷积的梯度.

depthwise_conv2d_native(...): 给定4-D输入和滤波器张量，计算二维深度卷积.

depthwise_conv2d_native_backprop_filter(...): 计算相对于滤波器的深度卷积的梯度.

depthwise_conv2d_native_backprop_input(...): 计算相对于输入的深度卷积的梯度.

dilation2d(...): 计算4D输入和3D滤波器张量的灰度膨胀.

dropout(...): Computes dropout. (deprecated arguments)

dynamic_rnn(...): 创建由RNNCell指定的递归神经网络 cell. (deprecated)

elu(...):计算指数线性: exp(features) - 1 if < 0, features otherwise.

embedding_lookup(...): 在嵌入张量列表中查找id.

embedding_lookup_sparse(...): 计算给定id和权重的嵌入.

erosion2d(...): 计算4-D值和3-D内核张量的灰度腐蚀.

fixed_unigram_candidate_sampler(...): 使用提供的（固定的）基本分发对一组类进行采样.

fractional_avg_pool(...): 对输入执行分数平均池. (deprecated)

fractional_max_pool(...): 对输入执行分数最大池化. (deprecated)

fused_batch_norm(...): Batch normalization.

in_top_k(...): 判断目标是否在前K个预测中.

l2_loss(...): L2 Loss.

l2_normalize(...): 使用L2范数沿尺寸轴标准化. (deprecated arguments)

leaky_relu(...): 计算 Leaky ReLU 激活函数.

learned_unigram_candidate_sampler(...): 从训练期间学习到的分布中抽样一组类别.

local_response_normalization(...): Local Response Normalization.

log_poisson_loss(...): 计算给定的对数泊松损失 log_input.

log_softmax(...): 计算log softmax activations. (deprecated arguments)

log_uniform_candidate_sampler(...): 使用对数均匀（Zipfian）基本分布对一组类进行采样.

lrn(...): Local Response Normalization.

max_pool(...): 最大池化.

max_pool1d(...): 最大池化.

max_pool2d(...): 最大池化.

max_pool3d(...): 最大池化.

max_pool_v2(...): 最大池化.

max_pool_with_argmax(...): 对输入执行最大池化，并输出最大值和索引.

moments(...): 计算x的均值和方差.

nce_loss(...): 计算并返回噪声对比估计训练损失.

normalize_moments(...): 根据足够的统计量计算的均值和方差.

pool(...): 执行N-D池化操作.

quantized_avg_pool(...): 产生量化类型的输入张量的平均池.

quantized_conv2d(...): 给定量化的4D输入和滤波器张量，计算2D卷积.

quantized_max_pool(...): 为量化类型产生输入张量的最大池.

quantized_relu_x(...): 计算量化整流线性X: min(max(features, 0), max_value)

raw_rnn(...): 创建由RNNCell单元指定的RNN和循环函数loop_fn.

relu(...): 计算整流线性: max(features, 0).

relu6(...): 计算整流线性6: min(max(features, 0), 6).

relu_layer(...): 计算Relu(x * weight + biases).

safe_embedding_lookup_sparse(...): 查找嵌入结果，说明无效的ID和空白功能.

sampled_softmax_loss(...): 计算并返回采样的softmax训练损失。

scale_regularization_loss(...): 按副本数缩放给定正则化损失的总和.

selu(...): 计算比例指数线性: scale * alpha * (exp(features) - 1)

separable_conv2d(...): 带可分离滤波器的二维卷积.

sigmoid(...): 计算x元素的S形.

sigmoid_cross_entropy_with_logits(...): 计算给定logits的S形交叉熵 .

softmax(...): 计算softmax 激活. (deprecated arguments)

softmax_cross_entropy_with_logits(...): 计算S形交叉熵  between logits and labels. (deprecated)

softmax_cross_entropy_with_logits_v2(...): 计算S形交叉熵 between logits and labels. (deprecated arguments)

softplus(...): 计算 softplus: log(exp(features) + 1).

softsign(...): 计算 softsign: features / (abs(features) + 1).

space_to_batch(...): T型4维张量的SpaceToBatch.

space_to_depth(...): T型张量的SpaceToDepth.

sparse_softmax_cross_entropy_with_logits(...): 计算稀疏softmax交叉熵 between logitsand labels.

static_bidirectional_rnn(...): 创建双向递归神经网络. (deprecated)

static_rnn(...): 创建一个指定的递归神经网络 by RNNCell cell. (deprecated)

static_state_saving_rnn(...): RNN接受状态保存程序以进行时间截断的RNN计算. (deprecated)

sufficient_statistics(...): 计算x的均值和方差的足够统计量.

swish(...): 计算 Swish activation function: x * sigmoid(x).

tanh(...): 计算 hyperbolic tangent of x element-wise.

top_k(...): 查找最后一个维度的k个最大条目的值和索引.

uniform_candidate_sampler(...): 使用统一的基本分布对一组类进行采样.

weighted_cross_entropy_with_logits(...): 计算加权交叉熵. (deprecated arguments)

weighted_moments(...): 返回x的频率加权均值和方差.

with_space_to_batch(...): 对输入的空分批表示执行运算.

xw_plus_b(...): Computes matmul(x, weights) + biases.

zero_fraction(...): 返回值中零的分数.

ers

 

tf.layers：

Modules

experimental module: Public API for tf.layers.experimental namespace.

Classes

class AveragePooling1D: Average Pooling layer for 1D inputs.

class AveragePooling2D: Average pooling layer for 2D inputs (e.g. images).

class AveragePooling3D: Average pooling layer for 3D inputs (e.g. volumes).

class BatchNormalization: Batch Normalization layer from http://arxiv.org/abs/1502.03167.

class Conv1D: 1D convolution layer (e.g. temporal convolution).

class Conv2D: 2D convolution layer (e.g. spatial convolution over images).

class Conv2DTranspose: Transposed 2D convolution layer (sometimes called 2D Deconvolution).

class Conv3D: 3D convolution layer (e.g. spatial convolution over volumes).

class Conv3DTranspose: Transposed 3D convolution layer (sometimes called 3D Deconvolution).

class Dense: Densely-connected layer class.

class Dropout: Applies Dropout to the input.

class Flatten: Flattens an input tensor while preserving the batch axis (axis 0).

class InputSpec: Specifies the ndim, dtype and shape of every input to a layer.

class Layer: Base layer class.

class MaxPooling1D: Max Pooling layer for 1D inputs.

class MaxPooling2D: Max pooling layer for 2D inputs (e.g. images).

class MaxPooling3D: Max pooling layer for 3D inputs (e.g. volumes).

class SeparableConv1D: Depthwise separable 1D convolution.

class SeparableConv2D: Depthwise separable 2D convolution.

Functions

average_pooling1d(...): Average Pooling layer for 1D inputs. (deprecated)

average_pooling2d(...): Average pooling layer for 2D inputs (e.g. images). (deprecated)

average_pooling3d(...): Average pooling layer for 3D inputs (e.g. volumes). (deprecated)

batch_normalization(...): Functional interface for the batch normalization layer. (deprecated)

conv1d(...): Functional interface for 1D convolution layer (e.g. temporal convolution). (deprecated)

conv2d(...): Functional interface for the 2D convolution layer. (deprecated)

conv2d_transpose(...): Functional interface for transposed 2D convolution layer. (deprecated)

conv3d(...): Functional interface for the 3D convolution layer. (deprecated)

conv3d_transpose(...): Functional interface for transposed 3D convolution layer. (deprecated)

dense(...): Functional interface for the densely-connected layer. (deprecated)

dropout(...): Applies Dropout to the input. (deprecated)

flatten(...): Flattens an input tensor while preserving the batch axis (axis 0). (deprecated)

max_pooling1d(...): Max Pooling layer for 1D inputs. (deprecated)

max_pooling2d(...): Max pooling layer for 2D inputs (e.g. images). (deprecated)

max_pooling3d(...): Max pooling layer for 3D inputs (e.g. (deprecated)

separable_conv1d(...): Functional interface for the depthwise separable 1D convolution layer. (deprecated)

separable_conv2d(...): Functional interface for the depthwise separable 2D convolution layer. (deprecated)

 

tf.contrib：

Modules

autograph module: This is the legacy module for AutoGraph, kept for backward compatibility.

batching module: Ops and modules related to batch.

bayesflow module: Ops for representing Bayesian computation.

checkpoint module: Tools for working with object-based checkpoints.

cloud module: Module for cloud ops.

cluster_resolver module: Standard imports for Cluster Resolvers.

compiler module: A module for controlling the Tensorflow/XLA JIT compiler.

constrained_optimization module: A library for performing constrained optimization in TensorFlow.

copy_graph module: Functions to copy elements between graphs.

crf module: Linear-chain CRF layer.

cudnn_rnn module: Ops for fused Cudnn RNN models.

data module: Experimental API for building input pipelines.

deprecated module: Non-core alias for the deprecated tf.X_summary ops.

distribute module: A distributed computation library for TF.

distributions module: Classes representing statistical distributions and ops for working with them.

eager module: TensorFlow Eager execution prototype.

estimator module: estimator python module.

factorization module: Ops and modules related to factorization.

feature_column module: Experimental utilities for tf.feature_column.

ffmpeg module: Working with audio using FFmpeg.

framework module: Framework utilities.

graph_editor module: TensorFlow Graph Editor.

grid_rnn module: GridRNN cells

image module: Ops for image manipulation.

input_pipeline module: Ops and modules related to input_pipeline.

integrate module: Integration and ODE solvers.

keras module: Implementation of the Keras API meant to be a high-level API for TensorFlow.

kernel_methods module: Ops and estimators that enable explicit kernel methods in TensorFlow.

labeled_tensor module: Labels for TensorFlow.

layers module: Ops for building neural network layers, regularizers, summaries, etc.

learn module: High level API for learning (DEPRECATED).

legacy_seq2seq module: Deprecated library for creating sequence-to-sequence models in TensorFlow.

linear_optimizer module: Ops for training linear models.

lookup module: Ops for lookup operations.

losses module: Ops for building neural network losses.

memory_stats module: Ops for memory statistics.

metrics module: Ops for evaluation metrics and summary statistics.

mixed_precision module: Library for mixed precision training.

model_pruning module: Model pruning implementation in tensorflow.

nn module: Module for variants of ops in tf.nn.

opt module: A module containing optimization routines.

optimizer_v2 module: Distribution-aware version of Optimizer.

periodic_resample module: Custom op used by periodic_resample.

predictor module: Modules for Predictors.

proto module: Ops and modules related to proto.

quantization module: Ops for building quantized models.

quantize module: Functions for rewriting graphs for quantized training.

receptive_field module: Module that declares the functions in tf.contrib.receptive_field's API.

recurrent module: Recurrent computations library.

reduce_slice_ops module: reduce by slice

remote_fused_graph module: Remote fused graph ops python library.

resampler module: Ops and modules related to resampler.

rnn module: RNN Cells and additional RNN operations.

rpc module: Ops and modules related to RPC.

saved_model module: SavedModel contrib support.

seq2seq module: Ops for building neural network seq2seq decoders and losses.

signal module: Signal processing operations.

slim module: Slim is an interface to contrib functions, examples and models.

solvers module: Ops for representing Bayesian computation.

sparsemax module: Module that implements sparsemax and sparsemax loss, see [1].

specs module: Init file, giving convenient access to all specs ops.

staging module: contrib module containing StagingArea.

stat_summarizer module: Exposes the Python wrapper for StatSummarizer utility class.

stateless module: Stateless random ops which take seed as a tensor input.

summary module: TensorFlow Summary API v2.

tensor_forest module: Random forest implementation in tensorflow.

tensorboard module: tensorboard module containing volatile or experimental code.

testing module: Testing utilities.

tfprof module: tfprof is a tool that profile various aspect of TensorFlow model.

timeseries module: A time series library in TensorFlow (TFTS).

tpu module: Ops related to Tensor Processing Units.

training module: Training and input utilities.

util module: Utilities for dealing with Tensors.