Resources
Deep Learning
- Ian Goodfellow. Deep Learning Book
- CS230 Deep Learning
Natural Language Processing
- Dan Jurafsky. Speech and Language Processing (3rd)
- Yoav Goldberg. Neural Network Methods for Natural Language Processing
- Jacob Eisenstein. Natural Language Processing
- CS224n: Natural Language Processing with Deep Learning
- CS11-747, Neural Networks for NLP
Reinforcement Learning
- David Silver. Course on Reiforcement Learning
- CS234: Reinforcement Learning Winter 2019
- Deep Reinforcement Learning: An Overview
- Spinning Up in Deep RL produced by OpenAI
Machine Learning
- 李航. 《统计学习方法》
- Christopher Bishop. Pattern Recognition and Machine Learning
Natural Language Processing 学习
接下来成为专家的领域:Transformer + TL -> 知识图谱 -> DRL
Topic | cs224n | slp3 | CS11-747 | others |
---|---|---|---|---|
nlp basics: math and optimizers | lecture 0 | |||
Word Vectors | lecture 1: Introduction and Word Vectors lecture 2: Word Vectors 2 and Word Senses lecture 12: Information from parts of words: Subword Models | chapter 6: Vector Semantics | Distributional Semantics and Word Vectors (1/22/2019) | ruder.io/word-embeddings |
Neural Networks | lecture 3: Word Window Classification, Neural Networks, and Matrix Calculus lecture 4: Backpropagation and Computation Graphs | chapter 7: Neural Networks and Neural LM | Neural Networks and Deep Learning | |
RNN and Language Models | lecture 6: Recurrent Neural Networks and Language Models lecture 7: Vanishing Gradients, Fancy RNNs | chapter 9: Sequence Processing with Recurrent Networks | A Simple (?) Exercise: Predicting the Next Word in a Sentence (1/17/2019) Recurrent Networks for Sentence or Language Modeling (1/29/2019) | Recurrent Neural Networks Tutorial, Part 1–Introduction to RNNs Understanding LSTM Networks The Unreasonable Effectiveness of Recurrent Neural Networks |
seq2seq+Attention | lecture 8: Machine Translation, Seq2Seq and Attention | chapter 22: Machine Translation | Conditioned Generation (2/5/2019) Attention (2/7/2019) | https://github.com/tensorflow/nmt https://arxiv.org/abs/1703.01619 |
CNN for Text | lecture 11: ConvNets for NLP | Convolutional Neural Nets for Text (1/24/2019) | ||
Contextual | lecture 13: contexts of use: Contextual Representations and Pretraining | Sentence and Contextual Word Representations (2/12/2019) | http://jalammar.github.io/illustrated-bert/ | |
✨***Transformer***✨ | Transformers and Self-Attention For Generative Models | https://jalammar.github.io/illustrated-transformer/ http://nlp.seas.harvard.edu/2018/04/03/attention.html | ||
Dependency Parsing | lecture 5: Linguistic Structure: Dependency Parsing | chapter 8: Part-of-Speech Tagging | ||
Structured Prediction Models | Search-based Structured Prediction (2/19/2019) Reinforcement Learning (2/21/2019) Structured Prediction with Local Independence Assumptions (2/26/2019) | |||
Advanced Learning Techniques | Latent Random Variables (3/5/2019) Adversarial Methods for Text (3/7/2019) Unsupervised and Semi-supervised Learning of Structure (3/28/2019) | |||
✨Models of Knowledge and Context✨ | Reference in Language and Coreference Resolution | Models of Dialog (4/2/2019) Document-level Models (4/4/2019) Learning from/for Knowledge Graphs (4/9/2019) Machine Reading w/ Neural Nets (4/16/2019) | ||
Multi-task and Multilingual Learning | Multitask Learning: A general model for NLP? | Multi-task Multi-lingual Learning Models (4/18/2019) Multimodal Models (4/23/2019) |