Maxout

Reading Paper-Network In Network(1)-Maxout

The maxout is designed to solve the problem that the representations that achieve good abstraction are generally highly nonlinear functions of the input data. Generally, the convolutional layers generate feature maps by liner convolutional filters followed by nonlinear activation functions. the feature map can be calculated as follows:

here it is a generalized linear model(GLM). However, some papers points that replacing the GLM with nonlinear function approximator can enhance the performance.
1 Maxout Network




For the maxout network, it adds another network between adjacent layers witch performs nonliner function to input data.