Click on the "Little White Learning Vision" above, select to add "Starred" or "Top"
Important content delivered promptly
Author: mingo_敏Editor: Deep Learning Natural Language ProcessingLink:https://blog.csdn.net/shanglianlm/article/details/85019768TensorFlow and PyTorch are quite similar, here we take PyTorch as an example.
19 Types of Loss Functions
1. L1 Loss L1Loss
Calculates the absolute difference between output and target.
torch.nn.L1Loss(reduction='mean')Parameters:
reduction – Three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
2. Mean Squared Error Loss MSELossCalculates the mean squared difference between output and target.
torch.nn.MSELoss(reduction='mean')Parameters:
reduction – Three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
3. Cross Entropy Loss CrossEntropyLossEffective for training classification problems with C categories. The optional parameter weight must be a 1D Tensor, and weights will be assigned to each category. Very effective for unbalanced training sets.In multi-class tasks, softmax activation function + cross-entropy loss function is often used because cross-entropy describes the difference between two probability distributions, while the neural network outputs a vector instead of a probability distribution. Therefore, the softmax activation function is needed to “normalize” a vector into a probability distribution, and then the cross-entropy loss function is used to calculate the loss.
torch.nn.CrossEntropyLoss(weight=None, ignore_index=-100, reduction='mean')Parameters:
weight (Tensor, optional) – Custom weights for each category. Must be a Tensor of length C.ignore_index (int, optional) – Sets a target value that will be ignored, thus not affecting the input gradient.reduction – Three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
4. KL Divergence Loss KLDivLossCalculates the KL divergence between input and target. KL divergence can be used to measure the distance between different continuous distributions, and is effective when performing direct regression on the space of continuous output distributions (discrete sampling).
torch.nn.KLDivLoss(reduction='mean')Parameters:
reduction – Three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
5. Binary Cross Entropy Loss BCELossCross-entropy calculation function for binary classification tasks. Used to measure reconstruction error, such as in autoencoders. Note that the target values t[i] should be in the range of 0 to 1.
torch.nn.BCELoss(weight=None, reduction='mean')Parameters:
weight (Tensor, optional) – Custom weights for each batch element’s loss. Must be a Tensor of length “nbatch”.
6. BCEWithLogitsLossThe BCEWithLogitsLoss function integrates the Sigmoid layer into the BCELoss class. This version is numerically more stable compared to using a simple Sigmoid layer and BCELoss separately, as merging these two operations into one layer allows for the use of log-sum-exp techniques for numerical stability.
torch.nn.BCEWithLogitsLoss(weight=None, reduction='mean', pos_weight=None)Parameters:
weight (Tensor, optional) – Custom weights for each batch element’s loss. Must be a Tensor of length “nbatch”.
7. Margin Ranking Loss
torch.nn.MarginRankingLoss(margin=0.0, reduction='mean')The loss function for each instance in a mini-batch is as follows:
Parameters:
margin: default value 0
8. Hinge Embedding Loss
torch.nn.HingeEmbeddingLoss(margin=1.0, reduction='mean')The loss function for each instance in a mini-batch is as follows:
Parameters:
margin: default value 1
9. Multi-Label Classification Loss MultiLabelMarginLoss
torch.nn.MultiLabelMarginLoss(reduction='mean')For each sample in a mini-batch, the loss is calculated as follows:
10. Smooth L1 Loss SmoothL1Loss
Also known as the Huber loss function.
torch.nn.SmoothL1Loss(reduction='mean')
Where
11. Logistic Loss for 2-Class SoftMarginLoss
torch.nn.SoftMarginLoss(reduction='mean')
12. Multi-Label One-Versus-All Loss MultiLabelSoftMarginLoss
torch.nn.MultiLabelSoftMarginLoss(weight=None, reduction='mean')
13. Cosine Loss CosineEmbeddingLoss
torch.nn.CosineEmbeddingLoss(margin=0.0, reduction='mean')
Parameters:
margin: default value 0
14. Multi-Class Hinge Loss MultiMarginLoss
torch.nn.MultiMarginLoss(p=1, margin=1.0, weight=None, reduction='mean')
Parameters:
p=1 or 2 Default: 1margin: default value 1
15. Triplet Loss TripletMarginLoss
Similar to Siamese networks, a specific example: given A, then give B and C, to see which one is more similar to A.
torch.nn.TripletMarginLoss(margin=1.0, p=2.0, eps=1e-06, swap=False, reduction='mean')
Where:
16. Connectionist Temporal Classification Loss CTCLossCTC connectionist temporal classification loss can automatically align data that is not aligned, mainly used for training serialized data without prior alignment. For example, speech recognition, OCR recognition, etc.
torch.nn.CTCLoss(blank=0, reduction='mean')Parameters:
reduction – Three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
17. Negative Log Likelihood Loss NLLLossNegative log likelihood loss. Used for training classification problems with C categories.
torch.nn.NLLLoss(weight=None, ignore_index=-100, reduction='mean')Parameters:
weight (Tensor, optional) – Custom weights for each category. Must be a Tensor of length C.ignore_index (int, optional) – Sets a target value that will be ignored, thus not affecting the input gradient.
18. NLLLoss2dNegative log likelihood loss for image inputs. It calculates the negative log likelihood loss for each pixel.
torch.nn.NLLLoss2d(weight=None, ignore_index=-100, reduction='mean')Parameters:
weight (Tensor, optional) – Custom weights for each category. Must be a Tensor of length C.reduction – Three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
19. Poisson NLL LossNegative log likelihood loss for targets that follow a Poisson distribution.
torch.nn.PoissonNLLLoss(log_input=True, full=False, eps=1e-08, reduction='mean')Parameters:log_input (bool, optional) – If set to True, the loss will be calculated according to the formula exp(input) – target * input. If set to False, the loss will be calculated as input – target * log(input + eps).full (bool, optional) – Whether to compute the full loss, i.e., including the Stirling approximation term target * log(target) – target + 0.5 * log(2 * pi * target).eps (float, optional) – Default: 1e-8
Download 1: OpenCV-Contrib Extension Module Chinese Version Tutorial
Reply "Extension Module Chinese Tutorial" in the background of "Little White Learning Vision" public account to download the first Chinese version of OpenCV extension module tutorial on the internet, covering installation of extension modules, SFM algorithms, stereo vision, target tracking, biological vision, super-resolution processing, and more than twenty chapters of content.
Download 2: Python Visual Practical Projects 52 Lectures
Reply "Python Visual Practical Projects" in the background of "Little White Learning Vision" public account to download 31 visual practical projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, facial recognition, etc., to help quickly learn computer vision.
Download 3: OpenCV Practical Projects 20 Lectures
Reply "OpenCV Practical Projects 20 Lectures" in the background of "Little White Learning Vision" public account to download 20 practical projects based on OpenCV, achieving advancement in OpenCV learning.
Discussion Group
Welcome to join the reader group of the public account to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (these will gradually be subdivided in the future). Please scan the WeChat ID below to join the group, and note: "Nickname + School/Company + Research Direction", for example: "Zhang San + Shanghai Jiao Tong University + Visual SLAM". Please follow the format for remarks, otherwise, it will not be approved. After successful addition, you will be invited to related WeChat groups based on your research direction. Please do not send advertisements in the group, otherwise, you will be removed from the group. Thank you for your understanding~