tf 2.x keras: Losses,Optimizers,metrics 损失函数优化器度量学习率排程

tf 2.x keras

Losses,Optimizers,metrics

https://www.tensorflow.org/api_docs/python/tf/keras/

https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/keras/

https://keras.io/api/

 

1.损失函数Losses：

https://www.tensorflow.org/api_docs/python/tf/keras/losses/

https://keras.io/api/losses/

CategoricalCrossentropy，Use this crossentropy loss function when there are two or more label classes. We expect labels to be provided in a one_hot representation.

SparseCategoricalCrossentropy，Use this crossentropy loss function when there are two or more label classes. We expect labels to be provided as integers.

BinaryCrossentropy，Use this cross-entropy loss when there are only two label classes (assumed to be 0 and 1).

KLDivergence  Computes Kullback-Leibler divergence loss between y_true and y_pred.

MeanAbsoluteError: Computes the mean of absolute difference between labels and predictions.

MeanAbsolutePercentageError: Computes the mean absolute percentage error between y_true and y_pred.

MeanSquaredError: Computes the mean of squares of errors between labels and predictions.

MeanSquaredLogarithmicError: Computes the mean squared logarithmic error between y_true and y_pred.

 

2.优化器Optimizers：

https://www.tensorflow.org/api_docs/python/tf/keras/optimizers

https://keras.io/api/optimizers/

class Adadelta: Optimizer that implements the Adadelta algorithm.
class Adagrad: Optimizer that implements the Adagrad algorithm.
class Adam: Optimizer that implements the Adam algorithm.
class Adamax: Optimizer that implements the Adamax algorithm.
class Ftrl: Optimizer that implements the FTRL algorithm.
class Nadam: Optimizer that implements the NAdam algorithm.
class RMSprop: Optimizer that implements the RMSprop algorithm.
class SGD: Gradient descent (with momentum) optimizer.
class Optimizer: Base class for Keras optimizers.

3.学习率排程 Learning Rate Scheduling

Power scheduling
Exponential scheduling
Piecewise constant scheduling
Performance scheduling
1cycle scheduling

 

4.度量Metrics

https://www.tensorflow.org/api_docs/python/tf/keras/metrics/

https://keras.io/api/metrics/

Accuracy metrics

Probabilistic metrics

Regression metrics

Classification metrics based on True/False positives & negatives

==================================

PyTorch

1.Losses：

 

https://pytorch.org/docs/stable/_modules/torch/nn/modules/loss.html

PyTorch Loss Functions: The Ultimate Guide

https://neptune.ai/blog/pytorch-loss-functions

Ultimate Guide To Loss functions In PyTorch With Python Implementation

https://analyticsindiamag.com/all-pytorch-loss-function/

 

2.Optimizers：

 

torch.optim

https://pytorch.org/docs/stable/optim.html?highlight=optimizer#torch.optim.Optimizer

Ultimate guide to PyTorch Optimizers

https://analyticsindiamag.com/ultimate-guide-to-pytorch-optimizers/

 

3.Metrics

 

==================================

优化器（Optimizer）介绍

https://blog.csdn.net/weixin_41417982/article/details/81561210

AdamW, LAMB: 大型预训练模型常用优化器

https://blog.csdn.net/weixin_43269174/article/details/106255084