慢慢读《Deep Learning In Natural Language Processing》（一）

第一次浪潮：Rationalism

     “The approaches, based on the belief that knowledge of language in the human mind is fixed in advance by generic       inheritance” 

    例证：特定领域的专家系统(exper system)

    缺点：过分依赖于特定领域的知识工程(paradigm of expert knowledge engineering)，泛化性极差

第二次浪潮：Empiricism

    数据驱动下的shallow machine learning 和 statistical learning 大行其道

    “The key algorithms and methods for machine learning include EM(expectation-maximization), Bayesian networks, support vector machines, decision trees, and, for neural networks, backpropagation algorithm.”（欠缺得补）

    缺点：不够deep，“shallow layers”导致的模型解空间大小有限。

第三次浪潮：Deep learning

    现实NLPer的写照：“given an NLP task, apply standard sequence models based on (bidirectional) LSTMs, add attention mechanisms if information required in the task needs to flow from another source,and then train the full models in an end-to-end manner.”

    局限：“have no reasoning and explaining capabilities”、句子间的关系推理

NLP未来的方向

    1、Neural-Symbolic Integration（集成网络和符号主义），“人话” 就是追求神经网络可解释性

    2、Structure, Memory, and Knowledge，认可lstm的基础上，追求更好的Memory model

    3、Unsupervised and Generative Deep Learning

    4、Multimodal and Multitask Deep Learning，跨领域的学习

    5、Meta-learning，“The goal of meta-learning is to learn how to learn new tasks faster by reusing previous experience”