a h+@sddlZddlmZddlmZddlmZddlmZm Z m Z m Z erVddl m Z dejejeeed d d ZGd d d eZGddde ZdddZdS)N)center_to_corners_format)is_scipy_available)HungarianMatcher ImageLoss _set_aux_lossgeneralized_box_ioulinear_sum_assignment?inputstargets num_boxesalphagammac Csv|}tjj||dd}||d|d|}|d||}|dkrj||d|d|} | |}||S)aR Loss used in RetinaNet for dense detection: https://huggingface.co/papers/1708.02002. Args: inputs (`torch.FloatTensor` of arbitrary shape): The predictions for each example. targets (`torch.FloatTensor` with the same shape as `inputs`) A tensor storing the binary classification label for each element in the `inputs` (0 for the negative class and 1 for the positive class). num_boxes (`int`): The total number of boxes in the batch. alpha (`float`, *optional*, defaults to 0.25): Optional weighting factor in the range (0,1) to balance positive vs. negative examples. gamma (`int`, *optional*, defaults to 2): Exponent of the modulating factor (1 - p_t) to balance easy vs hard examples. Returns: Loss tensor none)Z reductionrr)sigmoidnnZ functionalZ binary_cross_entropy_with_logitssum) rrrrrZprobZce_lossZp_tlossZalpha_tra/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/loss/loss_grounding_dino.pysigmoid_focal_losssrc@seZdZeddZdS)GroundingDinoHungarianMatchercCsd|djdd\}}|ddd}|ddd}|d}tdd t||D}||jd d d }td d |D}d} d} d| || d|d } | d|| |d } | | |} tj ||dd}t t |t | }|j ||j | |j|}|||d }dd |D}dd t||d D}dd |DS)a Args: outputs (`dict`): A dictionary that contains at least these entries: * "logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits * "pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates. * "label_maps": Tuple of tensors of dim [num_classes, hidden_dim]. targets (`list[dict]`): A list of targets (len(targets) = batch_size), where each target is a dict containing: * "class_labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth objects in the target) containing the class labels * "boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinates. Returns: `list[Tuple]`: A list of size `batch_size`, containing tuples of (index_i, index_j) where: - index_i is the indices of the selected predictions (in order) - index_j is the indices of the corresponding selected targets (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes) logitsNrrr pred_boxes label_mapscSsg|]\}}||dqS) class_labelsr).0Z label_maptargetrrr bz9GroundingDinoHungarianMatcher.forward..T)dimZkeepdimcSsg|] }|dqSboxesrr vrrrr"gr#r @g:0yE>)pcSsg|]}t|dqSr&lenr(rrrr"{r#cSsg|]\}}t||qSrr )r icrrrr"|r#cSs0g|](\}}tj|tjdtj|tjdfqS))dtype)torchZ as_tensorZint64)r r.jrrrr"}r#)shapeflattenrr1catziprlogtZcdistr r bbox_cost class_cost giou_costviewcpu enumeratesplit)selfoutputsrZ batch_sizeZ num_queriesZout_probZout_bboxrZ target_bboxrrZneg_cost_classZpos_cost_classr:r9r;Z cost_matrixsizesindicesrrrforwardDs&"z%GroundingDinoHungarianMatcher.forwardN)__name__ __module__ __qualname__r1Zno_gradrDrrrrrCsrc@s(eZdZdZddZddZddZdS) GroundingDinoImageLossa This class computes the losses for `GroundingDinoForObjectDetection`. The process happens in two steps: 1) we compute hungarian assignment between ground truth boxes and the outputs of the model 2) we supervise each pair of matched ground-truth / prediction (supervise class and box). Args: matcher (`GroundingDinoHungarianMatcher`): Module able to compute a matching between targets and proposals. focal_alpha (`float`): Alpha parameter in focal loss. losses (`list[str]`): List of all the losses to be applied. See `get_loss` for a list of all available losses. cCs"tj|||_||_||_dSN)rModule__init__matcher focal_alphalosses)r@rLrMrNrrrrKs zGroundingDinoImageLoss.__init__c srd}tfddtt||D}tjddd}||}tj||jtjd}||tj||<|S)z> Create one_hot based on the matching indices rcsHg|]@\}\}\}}|dkr8|d|td|n |d|qS)rrrr,)r r.r!_JrArrr"szFGroundingDinoImageLoss._get_target_classes_one_hot..rr)r%)devicer0) r1r5r>r6Z_get_source_permutation_idxZ zeros_likerRlongto) r@rArrCrrridxtarget_classes_onehotrrQr_get_target_classes_one_hots   z2GroundingDinoImageLoss._get_target_classes_one_hotc Cs~d|vrtdd|vr td||||}|d}|d}t||}t||}|}t||||jdd}d|i} | S)z Classification loss (Binary focal loss) targets dicts must contain the key "class_labels" containing a tensor of dim [nb_target_boxes] rz#No logits were found in the outputs text_maskz&No text_mask were found in the outputsrr loss_ce)KeyErrorrWr1Z masked_selectfloatrrM) r@rArrCrrVZ source_logitsrXrYrNrrr loss_labelss&  z"GroundingDinoImageLoss.loss_labelsN)rErFrG__doc__rKrWr\rrrrrHsrHc srt|j|j|jd} gd} t| |j| d} | |i}||d<||d<||d<||d<d}|jrt||}|D]}||d<||d<qr||d<| |||j r| | ||d }| ||}d d | D} |d |j |j d |j rdd  D} ||jrPi}t|jdD]"| fdd  Dq" |tfddD}||fS)N)r:r9r;)labelsr'Z cardinality)rLrMrNrrrrXauxiliary_outputs)rrrrXcSsi|]\}}|d|qSZ_encrr kr)rrr r#z7GroundingDinoForObjectDetectionLoss..r*)rYZ loss_bboxZ loss_gioucSsi|]\}}|d|qSr`rrarrrrcr#rcs i|]\}}|d|qS)rOrra)r.rrrc r#c3s&|]}|vr||VqdSrIr)r rb) loss_dict weight_dictrr r#z6GroundingDinoForObjectDetectionLoss..)rr:r9r;rHrMrTZauxiliary_lossrZ two_stageitemsupdateZbbox_loss_coefficientZgiou_loss_coefficientrangeZdecoder_layersr)rr^rRrconfigrrXZ outputs_classZ outputs_coordZencoder_logitsZencoder_pred_boxesrLrN criterionZ outputs_lossr_Z aux_outputZencoder_outputs_lossZencoder_loss_dictZenc_weight_dictZaux_weight_dictrr)r.rdrer#GroundingDinoForObjectDetectionLosss\          rl)r r)NNNN)r1Ztorch.nnrZimage_transformsrutilsrZloss_for_object_detectionrrrr Zscipy.optimizer ZTensorintr[rrrHrlrrrrs,     '=Q