tf 2.x keras
Losses,Optimizers,metrics
https://www.tensorflow.org/api_docs/python/tf/keras/
https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/keras/
1.损失函数Losses:
https://www.tensorflow.org/api_docs/python/tf/keras/losses/
CategoricalCrossentropy,Use this crossentropy loss function when there are two or more label classes. We expect labels to be provided in a one_hot representation.
SparseCategoricalCrossentropy,Use this crossentropy loss function when there are two or more label classes. We expect labels to be provided as integers.
BinaryCrossentropy,Use this cross-entropy loss when there are only two label classes (assumed to be 0 and 1).
KLDivergence Computes Kullback-Leibler divergence loss between y_true and y_pred.
MeanAbsoluteError: Computes the mean of absolute difference between labels and predictions.
MeanAbsolutePercentageError: Computes the mean absolute percentage error between y_true and y_pred.
MeanSquaredError: Computes the mean of squares of errors between labels and predictions.
MeanSquaredLogarithmicError: Computes the mean squared logarithmic error between y_true and y_pred.
2.优化器Optimizers:
https://www.tensorflow.org/api_docs/python/tf/keras/optimizers
https://keras.io/api/optimizers/
class Adadelta: Optimizer that implements the Adadelta algorithm.
class Adagrad: Optimizer that implements the Adagrad algorithm.
class Adam: Optimizer that implements the Adam algorithm.
class Adamax: Optimizer that implements the Adamax algorithm.
class Ftrl: Optimizer that implements the FTRL algorithm.
class Nadam: Optimizer that implements the NAdam algorithm.
class RMSprop: Optimizer that implements the RMSprop algorithm.
class SGD: Gradient descent (with momentum) optimizer.
class Optimizer: Base class for Keras optimizers.
3.学习率排程 Learning Rate Scheduling
Power scheduling
Exponential scheduling
Piecewise constant scheduling
Performance scheduling
1cycle scheduling
4.度量Metrics
https://www.tensorflow.org/api_docs/python/tf/keras/metrics/
Accuracy metrics
Probabilistic metrics
Regression metrics
Classification metrics based on True/False positives & negatives
==================================
PyTorch
1.Losses:
https://pytorch.org/docs/stable/_modules/torch/nn/modules/loss.html
PyTorch Loss Functions: The Ultimate Guide
https://neptune.ai/blog/pytorch-loss-functions
Ultimate Guide To Loss functions In PyTorch With Python Implementation
https://analyticsindiamag.com/all-pytorch-loss-function/
2.Optimizers:
torch.optim
https://pytorch.org/docs/stable/optim.html?highlight=optimizer#torch.optim.Optimizer
Ultimate guide to PyTorch Optimizers
https://analyticsindiamag.com/ultimate-guide-to-pytorch-optimizers/
3.Metrics
==================================
优化器(Optimizer)介绍
https://blog.csdn.net/weixin_41417982/article/details/81561210
AdamW, LAMB: 大型预训练模型常用优化器
https://blog.csdn.net/weixin_43269174/article/details/106255084