激活函数之logistic sigmoid函数介绍及C++实现

logistic sigmoid函数：

logistic sigmoid函数通常用来产生Bernoulli分布中的参数ø，因为它的范围是(0,1)，处在ø的有效取值范围内。logisitic sigmoid函数在变量取绝对值非常大的正值或负值时会出现饱和(saturate)现象，意味着函数会变得很平，并且对输入的微小改变会变得不敏感。

logistic sigmoid函数在深度学习中经常用作激活函数。有时为了加快计算速度，也使用称为fast sigmoid 函数，即：σ(x)=x/(1+|x|)

Logistic function：A logistic function or logistic curve is a common “S” shape(sigmoid curve), with equation:

where, e = the natural logarithm base(also known as Euler’s number); x₀ = the x-value of the sigmoid’s midpoint; L = the curve’s maximum value; k = the steepness of the curve. For values of x in the range of real numbers from -∞ to +∞.

The standard logistic function is the logistic function with parameters (k = 1, x₀= 0, L = 1) which yields:

The logistic function has the symmetry(对称) property that: 1-f(x)=f(-x).

以上内容摘自： 《深度学习中文版》和 维基百科

以下是C++测试code:

#include "funset.hpp"
#include <math.h>
#include <iostream>
#include <string>
#include <vector>
#include <opencv2/opencv.hpp>
#include "common.hpp"

// =============================== 计算 sigmoid函数 ==========================
template<typename _Tp>
int sigmoid_function(const _Tp* src, _Tp* dst, int length)
{
	for (int i = 0; i < length; ++i) {
		dst[i] = (_Tp)(1. / (1. + exp(-src[i])));
	}

	return 0;
}

template<typename _Tp>
int sigmoid_function_fast(const _Tp* src, _Tp* dst, int length)
{
	for (int i = 0; i < length; ++i) {
		dst[i] = (_Tp)(src[i] / (1. + fabs(src[i])));
	}

	return 0;
}

int test_sigmoid_function()
{
	std::vector<double> src{ 1.23f, 4.14f, -3.23f, -1.23f, 5.21f, 0.234f, -0.78f, 6.23f };
	int length = src.size();
	std::vector<double> dst1(length), dst2(length);

	fprintf(stderr, "source vector: \n");
	fbc::print_matrix(src);
	fprintf(stderr, "calculate sigmoid function:\n");
	fprintf(stderr, "type: sigmoid functioin, result: \n");
	fbc::sigmoid_function(src.data(), dst1.data(), length);
	fbc::print_matrix(dst1);
	fprintf(stderr, "type: fast sigmoid function, result: \n");
	fbc::sigmoid_function_fast(src.data(), dst2.data(), length);
	fbc::print_matrix(dst2);

	return 0;
}

GitHub： https://github.com/fengbingchun/NN_Test