02/16/2020 Deep learning - Day 1

Deep learning - Day 1

- DL+ lecture+2 - PPT REVIEW

• ML can be used to assign labels to images from a fixed number of classes (segmentation).
• What is an objective function?
  An objective function attempts to maximize profits or minimize losses based on a set of constraints and the relationship between one or more decision variables.
  (Categorical crossentropy is a loss function that is used for single label categorization. This is when only one category is applicable for each data point. In other words, an example can belong to one class only.)
  
• neuron
  
• what dose the layers learn?
  
• what is fully connected layer?
  Do high-level reasoning after convolutions and max pooling.
  Neurons in FC layer have connections to all connections in the previous layer.
  (Fully connected layers connect every neuron in one layer to every neuron in another layer.)
  
• CBR block: convolutional layer (Conv-L), batch normalization layer (BN-L), RELU layer

-what is batch normalization layer?
Batch normalization reduces the amount by what the hidden unit values shift around (covariance shift) to speed up learning.
To increase the stability of a neural network, batch normalization normalizes the output of a previous activation layer by subtracting the batch mean and dividing by the batch standard deviation.

-为什么引入非线性激励函数

如果不用激励函数，在这种情况下你每一层输出都是上层输入的线性函数，很容易验证，无论你有多少层神经网络，输出的都是输入的线性组合。
激活函数是用来加入非线性因素的，因为线性模型的表达能力不够
为什么引入Relu呢
第一，采用sigmoid函数，算激活函数时（指数运算），计算量大。而使用Relu，整个计算节省了很多。
第二，对于深层网络，sigmoid函数反向传播时，很容易出现梯度消失的情况，（sigmoid接近饱和区的时候，变化太缓慢，导数趋于0）从而无法完成深层网络的训练。
第三，Relu会使一部分神经元的输出为0，这样就造成了网络的稀疏性，并且减少了参数之间互相依存的关系，缓解了过拟合的发生。

-What is Image Segmentation?
Image segmentation is a critical process in computer vision. It involves dividing a visual input into segments to simplify image analysis. Segments represent objects or parts of objects, and comprise sets of pixels, or “super-pixels”. Image segmentation sorts pixels into larger components, eliminating the need to consider individual pixels as units of observation. There are three levels of image analysis:

Classification – categorizing the entire image into a class such as “people”, “animals”, “outdoors”
Object detection – detecting objects within an image and drawing a rectangle around them, for example, a person or a sheep.
Segmentation – identifying parts of the image and understanding what object they belong to. Segmentation lays the basis for performing object detection and classification.

-What is transfer learning?