Click on the above “Beginner Learning Vision” to select and add to favorites or “pin” to receive high-quality content promptly.
Added by zenRRanhttps://blog.csdn.net/shanglianlm/article/details/85019768Source: Deep Learning Natural Language Processing
TensorFlow and PyTorch are quite similar, and here we take PyTorch as an example.
19 Loss Functions
1. L1 Loss L1Loss
Calculates the absolute difference between output and target.
torch.nn.L1Loss(reduction='mean')Parameters:
reduction – three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
2. Mean Squared Error Loss MSELossCalculates the mean squared difference between output and target.
torch.nn.MSELoss(reduction='mean')Parameters:
reduction – three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
3. Cross Entropy Loss CrossEntropyLossEffective for training classification problems with C classes. The optional parameter weight must be a 1D Tensor, and weights will be assigned to each class. Very effective for imbalanced training sets.In multi-class tasks, softmax activation function + cross-entropy loss function is often used because cross-entropy describes the difference between two probability distributions, while the neural network outputs a vector, not in the form of a probability distribution. Therefore, the softmax activation function is needed to “normalize” a vector into the form of a probability distribution, and then use the cross-entropy loss function to calculate the loss.
torch.nn.CrossEntropyLoss(weight=None, ignore_index=-100, reduction='mean')Parameters:
weight (Tensor, optional) – custom weights for each class. Must be a Tensor of length C.ignore_index (int, optional) – sets a target value that will be ignored, thus not affecting the input gradient.reduction – three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
4. KL Divergence Loss KLDivLossCalculates the KL divergence between input and target. KL divergence can be used to measure the distance between different continuous distributions, and is very effective when performing direct regression on continuous output distribution space (discrete sampling).
torch.nn.KLDivLoss(reduction='mean')Parameters:
reduction – three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
5. Binary Cross Entropy Loss BCELossCross-entropy calculation function for binary classification tasks. Used to measure reconstruction error, such as in autoencoders. Note that the target value t[i] should be in the range of 0 to 1.
torch.nn.BCELoss(weight=None, reduction='mean')Parameters:
weight (Tensor, optional) – custom weights for each batch element’s loss. Must be a Tensor of length “nbatch”.
6. BCEWithLogitsLossThe BCEWithLogitsLoss function integrates the Sigmoid layer into the BCELoss class. This version is numerically more stable than using a simple Sigmoid layer and BCELoss, as combining these two operations into one layer can leverage log-sum-exp techniques for numerical stability.
torch.nn.BCEWithLogitsLoss(weight=None, reduction='mean', pos_weight=None)Parameters:
weight (Tensor, optional) – custom weights for each batch element’s loss. Must be a Tensor of length “nbatch”.
7. Margin Ranking Loss
torch.nn.MarginRankingLoss(margin=0.0, reduction='mean')For each instance in the mini-batch, the loss function is as follows:
Parameters:
margin: default value 0
8. Hinge Embedding Loss
torch.nn.HingeEmbeddingLoss(margin=1.0, reduction='mean')For each instance in the mini-batch, the loss function is as follows:
Parameters:
margin: default value 1
9. Multi-label Classification Loss MultiLabelMarginLoss
torch.nn.MultiLabelMarginLoss(reduction='mean')For each sample in the mini-batch, the loss is calculated as follows:
10. Smooth L1 Loss SmoothL1Loss
Also known as the Huber loss function.
torch.nn.SmoothL1Loss(reduction='mean')
Where
11. Logistic Loss for 2-class SoftMarginLoss
torch.nn.SoftMarginLoss(reduction='mean')
12. Multi-label One-versus-all Loss MultiLabelSoftMarginLoss
torch.nn.MultiLabelSoftMarginLoss(weight=None, reduction='mean')
13. Cosine Loss CosineEmbeddingLoss
torch.nn.CosineEmbeddingLoss(margin=0.0, reduction='mean')
Parameters:
margin: default value 0
14. Multi-class Hinge Loss MultiMarginLoss
torch.nn.MultiMarginLoss(p=1, margin=1.0, weight=None, reduction='mean')
Parameters:
p=1 or 2, default value: 1margin: default value 1
15. Triplet Loss TripletMarginLoss
Similar to Siamese networks, for example: give A, then give B, C, and see which is more similar to A.
torch.nn.TripletMarginLoss(margin=1.0, p=2.0, eps=1e-06, swap=False, reduction='mean')
Where:
16. Connectionist Temporal Classification Loss CTCLossCTC loss can automatically align data that is not aligned, mainly used for training serialized data without prior alignment, such as speech recognition, OCR recognition, etc.
torch.nn.CTCLoss(blank=0, reduction='mean')Parameters:
reduction – three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
17. Negative Log Likelihood Loss NLLLossNegative log likelihood loss. Used for training classification problems with C classes.
torch.nn.NLLLoss(weight=None, ignore_index=-100, reduction='mean')Parameters:
weight (Tensor, optional) – custom weights for each class. Must be a Tensor of length C.ignore_index (int, optional) – sets a target value that will be ignored, thus not affecting the input gradient.
18. NLLLoss2dNegative log likelihood loss for image inputs. It calculates negative log likelihood loss for each pixel.
torch.nn.NLLLoss2d(weight=None, ignore_index=-100, reduction='mean')Parameters:
weight (Tensor, optional) – custom weights for each class. Must be a Tensor of length C.reduction – three values: none: no reduction; mean: returns the average of the losses; sum: returns the sum of the losses. Default: mean.
19. Poisson NLL LossNegative log likelihood loss for target values following a Poisson distribution.
torch.nn.PoissonNLLLoss(log_input=True, full=False, eps=1e-08, reduction='mean')Parameters:
log_input (bool, optional) – if set to True, loss will be calculated using the formula exp(input) – target * input. If set to False, loss will be calculated as input – target * log(input + eps).full (bool, optional) – whether to calculate the full loss, i.e., including Stirling’s approximation term target * log(target) – target + 0.5 * log(2 * pi * target).eps (float, optional) – default value: 1e-8.
ReferencesPytorch Loss Function Summaryhttp://www.voidcn.com/article/p-rtzqgqkz-bpg.html
Download 1: OpenCV-Contrib Extension Module Chinese Tutorial
Reply in the “Beginner Learning Vision” WeChat public account: Extension Module Chinese Tutorial to download the first OpenCV extension module tutorial in Chinese available online, covering installation of extension modules, SFM algorithms, stereo vision, object tracking, biological vision, super-resolution processing, and over twenty chapters of content.
Download 2: Python Vision Practical Project 52 Lectures
Reply in the “Beginner Learning Vision” WeChat public account: Python Vision Practical Project to download 31 practical vision projects including image segmentation, mask detection, lane line detection, vehicle counting, eyeliner addition, license plate recognition, character recognition, emotion detection, text content extraction, and face recognition to help quickly learn computer vision.
Download 3: OpenCV Practical Project 20 Lectures
Reply in the “Beginner Learning Vision” WeChat public account: OpenCV Practical Project 20 Lectures to download 20 practical projects based on OpenCV implementation, achieving advanced learning of OpenCV.
Discussion Group
Welcome to join the readers' group of the public account to communicate with peers. Currently, there are WeChat groups for SLAM, 3D vision, sensors, autonomous driving, computational photography, detection, segmentation, recognition, medical imaging, GAN, algorithm competitions, etc. (will gradually be subdivided in the future). Please scan the WeChat ID below to join the group, and note: “Nickname + School/Company + Research Direction”, for example: “Zhang San + Shanghai Jiao Tong University + Vision SLAM”. Please follow the format; otherwise, you will not be approved. After successfully adding, you will be invited to relevant WeChat groups based on research direction. Please do not send advertisements in the group; otherwise, you will be removed from the group. Thank you for your understanding~