classification

AsymmetricLoss(gamma_neg=4, gamma_pos=0, m=0.05, eps=1e-08, disable_torch_grad_focal_loss=False, apply_sigmoid=True)

Bases: torch.nn.Module

Notice - optimized version, minimizes memory allocation and gpu uploading, favors inplace operations.

Parameters:

• gamma_neg (float) –

  gamma for negative samples

• gamma_pos (float) –

  gamma for positive samples

• m (float) –

  bias value added to negative samples

• eps (float) –

  epsilon to avoid division by zero

• disable_torch_grad_focal_loss (bool) –

  if True, disables torch grad for focal loss

• apply_sigmoid (bool) –

  if True, applies sigmoid to input before computing loss

Source code in quadra/losses/classification/asl.py

def __init__(
    self,
    gamma_neg: float = 4,
    gamma_pos: float = 0,
    m: float = 0.05,
    eps: float = 1e-8,
    disable_torch_grad_focal_loss: bool = False,
    apply_sigmoid: bool = True,
):
    super().__init__()

    self.gamma_neg = gamma_neg
    self.gamma_pos = gamma_pos
    self.m = m
    self.eps = eps
    self.disable_torch_grad_focal_loss = disable_torch_grad_focal_loss
    self.apply_sigmoid = apply_sigmoid

    # prevent memory allocation and gpu uploading every iteration, and encourages inplace operations
    self.targets: torch.Tensor
    self.anti_targets: torch.Tensor
    self.xs_pos: torch.Tensor
    self.xs_neg: torch.Tensor
    self.asymmetric_w: torch.Tensor
    self.loss: torch.Tensor

forward(x, y)

Compute the asymmetric loss.

Parameters:

• x (torch.Tensor) –

  input logits (after sigmoid)

• y (torch.Tensor) –

  targets (multi-label binarized vector)

Returns:

• torch.Tensor –

  asymettric loss

Source code in quadra/losses/classification/asl.py

def forward(self, x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
    """Compute the asymmetric loss.

    Args:
        x: input logits (after sigmoid)
        y: targets (multi-label binarized vector)

    Returns:
        asymettric loss
    """
    self.targets = y
    self.anti_targets = 1 - y

    # Calculating Probabilities
    self.xs_pos = x
    if self.apply_sigmoid:
        self.xs_pos = torch.sigmoid(self.xs_pos)
    self.xs_neg = 1.0 - self.xs_pos

    # Asymmetric clipping
    if self.m is not None and self.m > 0:
        self.xs_neg.add_(self.m).clamp_(max=1)

    # Basic CE calculation
    self.loss = self.targets * torch.log(self.xs_pos.clamp(min=self.eps))
    self.loss.add_(self.anti_targets * torch.log(self.xs_neg.clamp(min=self.eps)))

    # Asymmetric Focusing
    if self.gamma_neg > 0 or self.gamma_pos > 0:
        if self.disable_torch_grad_focal_loss:
            torch.set_grad_enabled(False)
        self.xs_pos = self.xs_pos * self.targets
        self.xs_neg = self.xs_neg * self.anti_targets
        self.asymmetric_w = torch.pow(
            1 - self.xs_pos - self.xs_neg, self.gamma_pos * self.targets + self.gamma_neg * self.anti_targets
        )
        if self.disable_torch_grad_focal_loss:
            torch.set_grad_enabled(True)
        self.loss *= self.asymmetric_w

    return -self.loss.sum()

FocalLoss(alpha, gamma=2.0, reduction='none', eps=None)

Bases: nn.Module

Criterion that computes Focal loss.

According to :cite:lin2018focal, the Focal loss is computed as follows:

.. math::

\text{FL}(p_t) = -\alpha_t (1 - p_t)^{\gamma} \, \text{log}(p_t)

Where

• :math:p_t is the model's estimated probability for each class.

Parameters:

• alpha (float) –

  Weighting factor :math:\alpha \in [0, 1].

• gamma (float) –

  Focusing parameter :math:\gamma >= 0.

• reduction (str) –

  Specifies the reduction to apply to the output: 'none' | 'mean' | 'sum'. 'none': no reduction will be applied, 'mean': the sum of the output will be divided by the number of elements in the output, 'sum': the output will be summed.

• eps (Optional[float]) –

  used.

Shape

• Input: :math:(N, C, *) where C = number of classes.
• Target: :math:(N, *) where each value is :math:0 ≤ targets[i] ≤ C−1.

Example

N = 5 # num_classes kwargs = {"alpha": 0.5, "gamma": 2.0, "reduction": 'mean'} criterion = FocalLoss(**kwargs) input = torch.randn(1, N, 3, 5, requires_grad=True) target = torch.empty(1, 3, 5, dtype=torch.long).random_(N) output = criterion(input, target) output.backward()

Source code in quadra/losses/classification/focal.py

def __init__(self, alpha: float, gamma: float = 2.0, reduction: str = "none", eps: Optional[float] = None) -> None:
    super().__init__()
    self.alpha: float = alpha
    self.gamma: float = gamma
    self.reduction: str = reduction
    self.eps: Optional[float] = eps

forward(input_tensor, target)

Forward call computation.

Source code in quadra/losses/classification/focal.py

def forward(self, input_tensor: torch.Tensor, target: torch.Tensor) -> torch.Tensor:
    """Forward call computation."""
    return focal_loss(input_tensor, target, self.alpha, self.gamma, self.reduction, self.eps)