深度学习序列——稀疏自编码器模型(SAP)

本文链接：https://blog.csdn.net/hit1524468/article/details/80233203

本文介绍了稀疏自编码器模型（SAP），它旨在学习恒等变换，并通过稀疏性约束使隐藏层神经元活动和权重矩阵保持稀疏，以实现正则化。模型的实现和测试过程也进行了详细阐述。

摘要由CSDN通过智能技术生成

1. 模型简介

(1) 模型要能够学习恒等变换

(2) 隐藏层的神经元的活动性满足稀疏性(模型大脑)

(3) 隐藏层的权重矩阵同样满足稀疏性(一种正则化方法，实际上，在稀疏编码取得好的效果有启发作用)

2. 模型实现

# Sparse Auto-Encoder 
#
# Author: HSW
# Date: 2018-05-07
#

import tensorflow as tf
import numpy      as np

def axvier_init(fan_in, fan_out, constant = 1):
    ''' Initial weights '''
    low   = -constant + np.sqrt(6.0 / (fan_in + fan_out))
    high =  constant + np.sqrt(6.0 / (fan_in + fan_out))

    return tf.random_uniform((fan_in, fan_out), minval = low, maxval = high, dtype = tf.float32)


class SparseAutoEncoder(object):

    def __init__(self, AutoEncoder_Shape = (128, 256, 128), transfer_function = tf.nn.softplus, optimizer = tf.train.AdamOptimizer(), p = 0.05, sparse_beta = 0.1, weights_lambda = 0.00001):
        ''' Constructor Funcion '''
        self.LayerCnt          = len(AutoEncoder_Shape)
        self.AutoEncoder_Shape = AutoEncoder_Shape
        self.transfer          = transfer_function
        self.p                 = p
        self.sparse_beta       = sparse_beta 
        self.weights_lambda    = weights_lambda
        self.weights           = self.init_weights()
        self.layers            = self.init_layers()
        self.total_loss        = self.init_loss()
        self.optimizer         = optimizer.minimize(self.total_loss)
        init                   = tf.global_variables_init