【计算机科学】【2016.01】自动编码器能从大脑中学习到什么？

本文为Gerben van den Broeke的硕士论文，共87页。

本文探讨了无监督深度学习算法的基本改进。以无监督学习的目的为理论视角，选择共同学习的有向生成模型中的学习近似推理作为方法，主要问题是如何通过同时重新思考学习方式和推理方式来改进这种方法的现有实现，特别是自动编码的实现。在这种网络体系结构中，两个可用的相对路径，一个用于推理，一个用于生成，允许利用它们之间的对称性，并允许其中一个向另一个提供反馈信号。这些信号可用于仅从本地可用信息来确定连接权重的有效更新，从而消除了对传统反向传播路径的需要，并缓解了与之相关的问题。此外，反馈回路可以添加到通常的前馈网络中，以改进推理本身。大脑新皮质区域之间的相互联系为迭代修正和验证所提出的解释如何能够公平地近似于最佳贝叶斯推理提供了灵感。

本文从深度学习和皮层功能的研究中提取并结合了潜在的思想，对生成模型、近似推理、局部学习规则、目标传播、循环、横向和偏态竞争、预测编码、迭代和摊销推理等相关概念进行了探讨。通过以上主题，我们试图建立一个复杂的见解，可以为未来无监督的深度学习方法研究提供指导。

This thesis explores fundamental improvements in unsupervised deep learning algorithms. Taking a theoretical perspective on the purpose of unsupervised learning, and choosing learnt approximate inference in a jointly learnt directed generative model as the approach, the main question is how existing implementations of this approach, in particular auto-encoders, could be improved by simultaneously rethinking the way they learn and the way they perform inference. In such network architectures, the availability of two opposing pathways, one for inference and one for generation, allows to exploit the symmetry between them and to let either provide feedback signals to the other. The signals can be used to determine helpful updates for the connection weights from only locally available information, removing the need for the conventional back-propagation path and mitigating the issues associated with it. Moreover, feedback loops can be added to the usual usual feed-forward network to improve inference itself. The reciprocal connectivity between regions in the brain’s neocortex provides inspiration for how the iterative revision and verification of proposed interpretations could result in a fair approximation to optimal Bayesian inference. While extracting and combining underlying ideas from research in deep learning and cortical functioning, this thesis walks through the concepts of generative models, approximate inference, local learning rules, target propagation, recirculation, lateral and biased competition, predictive coding, iterative and amortised inference, and other related topics, in an attempt to build up a complex of insights that could provide direction to future research in unsupervised deep learning methods.

1 无监督学习：概念

2 无监督学习：算法

3 无监督学习：大脑

4 产生反馈信息用于学习

5 产生反馈信息用于推理

6 综合分析

下载英文原文地址：

http://page3.dfpan.com/fs/2lc3j2221e298161a07/

更多精彩文章请关注微信号：