<<Machine learning: Trends, perspectives, and prospects>> 总结

利用差不多大半天时间，看了一下2015年science上的这篇关于机器学习的综述。在这里对文章作一个小结。

• 什么是机器学习？

  1. Machine leaning addresses the question of how to build computers that improve automatically through experience.
  2. Conceptually, machine-learning algorithms can be viewed as searching through a large space of candidate programs, guided by training experience, to find a program that optimizes the performance metric.

• 它和计算机/统计/人工智能/数据科学的关系

  1. It is one of rapidly growing technical fields, lying at the intersection at the intersection of computer science and statistics, and at the core of artificial intelligence and data science.
  2. Related discipline: psychological study of human learning, the study of evolution, the study of educational practice, neuroscience, organizational behaviour, and economics.

• 解决的问题

  1. How can one construct computer systems that automatically improve through experience?
  2. What are the fundamental statistical-computational-information-theoretic laws that govern all learning systems, including computers, humans and organizations?

• 解决问题的标准—sample complexity and computational complexity

  1. How accurately can the algorithm learn from a particular type and volume of training data?
  2. How robust is the algorithm to errors in its modeling assumptions or to errors in the training data?
  3. Is it possible to design a successful algorithm or is the learning problem fundamentally intractable?

• 主要的方法

  1. Supervised learning methods
     Decision trees, decision forests, logistic regression, support vector machines, neural networks, kernel machines, and Bayesian classifiers. Generic procedures such as boosting and multiple kernel learning that combine the outputs of multiple learning algorithms.High-impact of progress in supervised learning: Deep network(GPU, etc.).
  2. Reinforcement learning
     The information available in the training data is intermediated between supervised and unsupervised learning. Instead of training examples that indicate the correct output for a given input, the training data in reinforcement learning are assumed to provide only an indication as to whether an action is correct or not. It generally make use of ideas that are familiar from control theory, such as policy iteration, value iteration, rollouts, and variance reduction.
  3. Unsupervised learning methods
     Dimension reduction methods: PCA, manifold learning, factor analysis, random projections, autoencoders—make different specific assumptions regarding the underlying manifold.
     A criterion function id defined that embodies these assumptions—often making use of general statistical principles such as maximum likelihood, the method of moments, or Bayesian integration as well as optimization or sampling algorithms.
     Clustering.

• 趋势和机会

  1. big data(parallel and distributed computing)/personality
     A major goal of prior line of research is to bring kinds of statistical resources studied in machine learning into contact with classical computational resources of times and space.
     Recent research focuses on methods such as subsampling, random projection, and algorithm weakening to achieve scalability while retaining statistical control.
  2. Opportunities can be seen by contrasting current machine-learning methods to the types of learning we observe in naturally occurring systems.