极限学习机(Extreme Learning Machine, ELM)原理详解和MATLAB实现

目录

引言

极限学习机原理

MATLAB中重点函数解读

极限学习机的MATLAB实践

---

引言

极限学习机不是一个新的东西，只是在算法（方法）上有新的内容。在神经网络结构上，就是一个前向传播的神经网络，和之前几篇博文讲的意义。

为什么我们需要ELM？

The learning speed of feedforward neural networks is in general far slower than required and it has been a major bottleneck in their applications for past decades. Two key reasons behind may be:

1) the slow gradient-based learning algorithms are extensively used to train neural networks.

2) all the parameters of the networks are tuned iteratively by using such learning algorithms.

最大的创新点：

1）输入层和隐含层的连接权值、隐含层的阈值可以随机设定，且设定完后不用再调整。这和BP神经网络不一样，BP需要不断反向去调整权值和阈值。因此这里就能减少一半的运算量了。

2）隐含层和输出层之间的连接权值β不需要迭代调整，而是通过解方程组方式一次性确定。

研究表明，通过这样的规则，模型的泛化性能很好，速度提高了不少。

一言概之，ELM最大的特点就是对于传统的神经网络，尤其是单隐层前馈神经网络(SLFNs)，在保证学习精度的前提下比传统的学习算法速度更快。

Compared BP Algorithm and SVM，ELM has several salient features:

•Ease of use. No parameters need to be manually tuned except predefined network architecture.只有隐含层神经元个数需要我们调整。

•Faster learning speed. Most training can be completed in milliseconds, seconds, and minutes.

•Higher generalization performance. It could obtain better generalization performance than BP in most cases, and reach generalization performance similar to or better than SVM.（泛化能力提升）

•Suitable for almost all nonlinear activation functions.Almost all piecewise continuous (including discontinuous, differential, non-differential functions) can be used as activation functions.

•Suitable for fully complex activation functions. Fully complex functions can also be used as activation functions in ELM.

极限学习机原理

ELM是一种新型的快速学习算法，对于单隐层神经网络，ELM可以随机初始化输入权重和偏置并得到相应的输出权重。

对于一个单隐层神经网络(见上面的图)，假设有个任意的样本，其中，。对于一个有个隐层节点的单隐层神经网络可以表示为

其中，为激活函数，