tensorflow实现自编码器

学习tensflow自编码的相关知识

tensorflow学习相关代码仓库：
https://github.com/joliph/tensorflow
本节代码：
https://github.com/joliph/tensorflow/blob/master/autoencoder.py

自编码是一种无监督式机器学习，相比于含有label的训练，自编码技术所需的数据要求更低
例如： 输入的图片有784个像素 要输出784个像素的恒等图片
但是这个恒等是宏观意义上的恒等，比如输入的图片是3，输出的图片也是3即可，但不要前后784个像素要一模一样完全对应

使用1层200个节点的隐藏层，将784个低阶特征进行高阶处理，生成更加浓缩的高阶特征（比如用70个像素总结一个横线这样的高阶特征…..）然后再通过高阶特征重建784个输出，达到自编码宏观上的恒等效果

这个例子中使用mnist训练集并加上噪声，损失函数使用重建的数据与原始数据的差的平方和，ada方法优化这个值，循环训练20次。
得到的结果：

Epoch: 0001 cost= 18639.842830682
Epoch: 0002 cost= 11933.035290909
Epoch: 0003 cost= 11832.580152273
Epoch: 0004 cost= 11103.513238636
Epoch: 0005 cost= 9793.832962500
Epoch: 0006 cost= 9504.805407386
Epoch: 0007 cost= 9220.456495455
Epoch: 0008 cost= 9057.789976136
Epoch: 0009 cost= 8685.858406250
Epoch: 0010 cost= 8860.839155114
Epoch: 0011 cost= 8579.519001136
Epoch: 0012 cost= 8345.770128977
Epoch: 0013 cost= 8605.232664205
Epoch: 0014 cost= 7581.979616477
Epoch: 0015 cost= 7992.928135795
Epoch: 0016 cost= 8291.554967045
Epoch: 0017 cost= 9087.374957955
Epoch: 0018 cost= 8071.232046023
Epoch: 0019 cost= 7996.882356818
Epoch: 0020 cost= 8248.178536932
Total cost: 649521.1

循环训练训练100次的结果：

Epoch: 0001 cost= 20897.994755682
Epoch: 0002 cost= 13323.546232955
Epoch: 0003 cost= 10139.543405682
Epoch: 0004 cost= 9698.337911932
Epoch: 0005 cost= 9322.908848295
Epoch: 0006 cost= 10191.493428977
Epoch: 0007 cost= 9018.830190909
Epoch: 0008 cost= 8327.412848295
Epoch: 0009 cost= 8439.414705114
Epoch: 0010 cost= 8015.416200568
Epoch: 0011 cost= 7541.549031818
Epoch: 0012 cost= 8005.284478977
Epoch: 0013 cost= 7748.398861364
Epoch: 0014 cost= 8295.715686364
Epoch: 0015 cost= 7902.735401705
Epoch: 0016 cost= 7932.701900000
Epoch: 0017 cost= 7687.836106818
Epoch: 0018 cost= 7057.748807955
Epoch: 0019 cost= 7860.597039205
Epoch: 0020 cost= 9060.556185795
Epoch: 0021 cost= 7403.233606818
Epoch: 0022 cost= 7664.444532955
Epoch: 0023 cost= 7617.226611364
Epoch: 0024 cost= 7822.862035227
Epoch: 0025 cost= 8279.358309659
Epoch: 0026 cost= 7939.353970455
Epoch: 0027 cost= 7836.960899432
Epoch: 0028 cost= 7315.375623295
Epoch: 0029 cost= 7891.151431818
Epoch: 0030 cost= 8147.461523864
Epoch: 0031 cost= 7723.446823864
Epoch: 0032 cost= 7369.095822727
Epoch: 0033 cost= 7480.961957386
Epoch: 0034 cost= 7433.558855114
Epoch: 0035 cost= 7406.572330114
Epoch: 0036 cost= 7463.833806818
Epoch: 0037 cost= 7017.529206250
Epoch: 0038 cost= 7461.646855682
Epoch: 0039 cost= 7338.416114773
Epoch: 0040 cost= 7965.016548295
Epoch: 0041 cost= 7045.250907955
Epoch: 0042 cost= 7798.697734659
Epoch: 0043 cost= 7536.523734659
Epoch: 0044 cost= 6978.538052841
Epoch: 0045 cost= 7583.568821591
Epoch: 0046 cost= 7805.579671023
Epoch: 0047 cost= 7291.973233523
Epoch: 0048 cost= 8005.160710795
Epoch: 0049 cost= 7822.103503977
Epoch: 0050 cost= 7580.498638636
Epoch: 0051 cost= 6909.949681818
Epoch: 0052 cost= 7408.430946023
Epoch: 0053 cost= 7599.592603977
Epoch: 0054 cost= 7662.740237500
Epoch: 0055 cost= 7704.320689205
Epoch: 0056 cost= 7390.774231818
Epoch: 0057 cost= 7100.965060227
Epoch: 0058 cost= 7433.153627841
Epoch: 0059 cost= 6949.785436364
Epoch: 0060 cost= 8132.225667045
Epoch: 0061 cost= 6985.143222159
Epoch: 0062 cost= 7107.063428409
Epoch: 0063 cost= 8041.408292045
Epoch: 0064 cost= 7359.334472159
Epoch: 0065 cost= 7144.590045455
Epoch: 0066 cost= 7385.787763636
Epoch: 0067 cost= 7114.617158523
Epoch: 0068 cost= 6955.011002273
Epoch: 0069 cost= 7003.742123864
Epoch: 0070 cost= 7159.479443182
Epoch: 0071 cost= 7289.120740909
Epoch: 0072 cost= 7831.053788636
Epoch: 0073 cost= 6571.717068182
Epoch: 0074 cost= 7303.479096023
Epoch: 0075 cost= 7629.674424432
Epoch: 0076 cost= 7576.975913636
Epoch: 0077 cost= 7178.717941477
Epoch: 0078 cost= 7237.719469318
Epoch: 0079 cost= 7424.526951705
Epoch: 0080 cost= 7882.358154545
Epoch: 0081 cost= 7795.144244318
Epoch: 0082 cost= 7272.062825000
Epoch: 0083 cost= 7280.093562500
Epoch: 0084 cost= 7785.554835795
Epoch: 0085 cost= 7205.475767045
Epoch: 0086 cost= 7063.570214773
Epoch: 0087 cost= 7112.735668182
Epoch: 0088 cost= 7180.632331818
Epoch: 0089 cost= 7612.753513636
Epoch: 0090 cost= 7612.275922159
Epoch: 0091 cost= 6982.684964773
Epoch: 0092 cost= 7020.079210227
Epoch: 0093 cost= 7555.833397727
Epoch: 0094 cost= 7056.109856818
Epoch: 0095 cost= 7229.730031250
Epoch: 0096 cost= 7602.359388068
Epoch: 0097 cost= 7159.126964773
Epoch: 0098 cost= 7390.675930682
Epoch: 0099 cost= 7489.342825000
Epoch: 0100 cost= 6866.383679545
Total cost: 610532.2

可见，20次训练和100次训练并没有对cost起到比较好的影响，下一篇会尝试使用2层隐藏层来提取更高阶的特征来还原数据。