Machine Learning之概述

学习机器学习（Machine Learning）需要的背景知识：

• 线性代数：矢量/矩阵操作，属性。
• 微积分：偏导数
•  概率论：常见的分布；贝叶斯法则
• 统计理论：平均值/中值/众数；最大似然
• Sheldon Ross：

课本：

• ChristopherBishop: 

“Pattern Recognition and Machine Learning”,  2006.

• Ethem Alpaydin:

“Introduction to Machine Learning”,  2^nd  edition, 2010.

 

什么是机器学习？

1.It is very hard to write programs that solve problems likerecognizing a face.

– We don’t know whatprogram to write because we don’t know how our brain does it.

– Even if we had agood idea about how to do it, the program might be horrendously complicated.

2.Instead of writing a program by hand, we collect lots of examplesthat specify the correct output for a given input.

3. A machine learning algorithm then takes these examples and producesa program that does the job.

– The programproduced by the learning algorithm may look very different from a typicalhand-written program. It may contain millions of numbers.

– If we do it right,the program works for new cases as well as the ones we trained it on. 

4.In truth, we write the structure of the program and the computertunes many internal parameters.

 

为什么学习机器学习？

  1.Develop enhanced computer systems

– Automatically adaptto user, customize

– Often difficult toacquire necessary knowledge

2.Improve understanding of human, biological learning

– Computationalanalysis provides concrete theory, predictions

– Explosion ofmethods to analyze brain activity during learning

3.Timing is good

– Ever growingamounts of data available

– Cheap and powerfulcomputers

– Suite ofalgorithms, theory already developed

 

以下是需要机器学习的经典例子

哪些是数字2？

 

其他利用机器学习算法解决的例子：

1.Recognizing patterns:

– Facial identitiesor facial expressions

– Handwritten orspoken words

– Medical images

2.Digital images and videos:

– Locating, tracking,and identifying objects

– Driving a car

3. Recognizing anomalies:

– Unusual sequencesof credit card transactions 

– Unusual patterns ofsensor readings in a nuclear power plant or unusual sound in your car engine.

4. Prediction:

– Future stock pricesor currency exchange rates 

 

一些基于web的使用机器学习的例子：

The web contains a lot of data. Tasks withvery big datasets often use machine learning

1.Spam filtering, fraud detection: 

– The enemy adapts sowe must adapt too.

2.Recommendation systems:

– Lots of noisy data,commercial pay-off (e.g., Amazon, Netflix).

3.Information retrieval:

– Find documents orimages with similar content.

4. Financial markets

– Huge data volume:1.5 million messages/sec

 

数据爆炸（Data Explosion）：文本

1.“大”本文数据集

– 1,000,000  words in 1967

– 1,000,000,000,000words in 2006

2.成功的应用

– Speech recognition

– Machine translation

 

– Lots of labeled data

– Memorization is useful

 

机器学习的类型:

1.Supervised learning

– Given examples ofinputs and corresponding desired outputs, predict outputs on future inputs,e.g., classification, regression, time series prediction

– Canonical paradigmof ML

2.Unsupervised learning

– Create a newrepresentation of the input, e.g., form clusters; extract features; compression;detect outliers

– This is the newfrontier of machine learning because most big datasets do not come with labels

3. Reinforcement learning

– Learn action to maximize payoff

%Not much information in a payoff signal

  %Payoff is often delayed

– An important areanot covered in this course – interesting examples include SmartHouse, games

 

Supervised Learning

1.Classification

– Outputs arecategorical (1-of-N)

– Inputs are anything

– Goal: selectcorrect class for new inputs

– Ex: speech, objectrecognition, medical diagnosis

2.Regression

– Outputs arecontinuous

– Inputs are anything(typically continuous) 

– Goal: predictoutputs accurately for new inputs 

– Ex: predictingmarket prices, customer rating of movie

3.Temporal Prediction

– Goal: performclassification/regression on new input sequences values at future time points

– Given input valuesand corresponding class labels/outputs at some previous time points

 

 

Unsupervised Learning

1.Clustering: 

– Inputs are vectoror categorical

– Goal: group datacases into a finite number of clusters so that within each cluster all caseshave very similar inputs

2. Compression

– Inputs aretypically vector

– Goal: deliver anencoder and decoder such that size of encoder output is much smaller thanoriginal input, but composition of encoder followed by decode verysimilar tooriginal input

3.Outlier detection

– Inputs are anything

– Goal: select highlyunusual cases from new and given data

 

Machine Learning & DataMining

1.Data-mining: Typically using very simple machine learningtechniques on very large databases because computers are too slow to doanything more interesting withten billion examples

2.Previously used in anegative sense – misguided statistical procedure of looking for all kinds of relationshipsin the data until finally find one

3.Now lines are blurred: manyML problems involve tons of data

4.But problems with AI flavor(e.g., recognition, robot navigation) still domain of ML

 

Machine Learning &Statistics

1.ML uses statistical theory tobuild models – core task is inference from a sample

2.A lot of ML is rediscoveryof things statisticians already knew; often disguised by differences interminology:

3.But the emphasis is verydifferent:

– Good piece of statistics:Clever proof that relatively simple estimation procedure is asymptoticallyunbiased.

– Good piece of ML: Demothat a complicated algorithm produces impressive results on a specific task.

4.Can view ML as applyingcomputational techniques to statistical problems. But go beyond typicalstatistics problems, with different aims (speed vs. accuracy).

 

内容学习自：http://www.cs.toronto.edu/~zemel/documents/2515/01intro-2515.pdf