算法组合 优化算法_探索不同的优化算法

算法组合 优化算法

Machine learning is a field of study in the broad spectrum of artificial intelligence (AI) that can make predictions using data without being explicitly programmed to do so. Machine learning algorithms are used in a wide variety of applications, such as recommendation engines, computer vision, spam filtering and so much more. They perform extraordinary well where it is difficult or infeasible to develop conventional algorithms to perform the needed tasks.

机器学习是广泛的人工智能(AI)研究领域，可以使用数据进行预测，而无需进行明确的编程。 机器学习算法被广泛用于各种应用程序中，例如推荐引擎，计算机视觉，垃圾邮件过滤等等。 在难以或无法开发常规算法来执行所需任务的情况下，它们的性能非常出色。

While many machine learning algorithms have been around for a long time, the ability to automatically apply complex mathematical calculations to big data— over and over, faster and faster — is a recent development.

尽管许多机器学习算法已经存在很长时间了，但是自动将复杂的数学计算自动应用于大数据的能力(一遍又一遍，越来越快)是最近的发展。

神经网络和优化器的工作 (Neural networks and the job of an optimizer)

One of the most overwhelmingly represented machine learning techniques is a neural network. Since intelligence is foremost human, insights from the human mind were derived in order to create intelligent machines. It is based on the idea of a neuron. I assume you’re already familiar with the basics of how neural networks work and won’t go into much detail on the architecture.

神经网络是压倒性代表最多的机器学习技术之一。 由于智能是人类的头等大事，因此从人类思想中得出的见解就是为了创造智能机器。 它基于神经元的思想。 我假设您已经熟悉了神经网络的工作原理，并且不会在体系结构上做太多详细介绍。

Once the model trains, a metrics has to be defined to measure the model performance i.e the loss or cost function. There is not a single uniform loss function that we could apply in each case. Each serves its own purpose. For example, we could be using a cats versus dogs data and would therefore need a binary loss that outputs the possibility of it being a dog or a cat. And so long for tasks, such as multiclass classification, linear regression, clustering algorithms etc. In some cases, a custom loss function is required.

一旦训练了模型，就必须定义一个度量来测量模型性能，即损失或成本函数。 在每种情况下都没有一个统一的损失函数可以应用。 每个都有自己的目的。 例如，我们可能使用的是猫与狗的数据，因此需要二进制损失来输出它可能是狗还是猫的可能性。 并且对于诸如多类分类，线性回归，聚类算法等任务的时间如此长。在某些情况下，需要自定义损失函数。

The optimizer’s job is to optimize the weights and biases so that the loss is minimized. How could we achieve this?

优化器的工作是优化权重和偏差，以使损失最小化。 我们如何实现这一目标？

Let me introduce you to gradient descent!

让我向您介绍梯度下降！

梯度下降 (Gradient Descent)

Given a loss function [J] we can calculate the partial derivatives of J by the weight matrix. Note: gradient vector of the function shows in the direction of steepest ascent. Check https://www.khanacademy.org/math/multivariable-calculus or any other source.

给定损失函数[J]，我们可以通过权重矩阵计算J的偏导数。 注意：函数的梯度向量以最陡峭的上升方向显示。 检查https://www.khanacademy.org/math/multivariable-calculus或任何其他来源。

Since our goal is to find the point where loss is minimised (this point is also reffered to as the global minimum) all we have to do is to go in the exact opposite direction.

因为我们的目标是找到使损失最小化的点(该点也称为全局最小值)，所以我们要做的就是朝着完全相反的方向前进。

Here, we introduce a hyperparameter called the learning rate, which determines the step size at each iteration and should be set between 0 and 1. It ultimately represents the speed at which the model learns.

在这里，我们引入了一个称为学习率的超参数，该参数确定每次迭代的步长，并且应将其设置为0到1。它最终代表了模型学习的速度。