a h@sddlmZddlmZmZddlZddlZddlmZddl m Z ddl m Z ddl mZmZd d lmZd d lmZeeZeGd dde ZeGddde ZdgZdS)) dataclass)OptionalUnionN)Cache)$ImageClassifierOutputWithNoAttention)PreTrainedModel)auto_docstringlogging)AutoModelForImageTextToText)ShieldGemma2Configc@s$eZdZUdZdZeejed<dS)0ShieldGemma2ImageClassifierOutputWithNoAttentionz^ShieldGemma2 classifies imags as violative or not relative to a specific policy Args: N probabilities) __name__ __module__ __qualname____doc__rrtorchTensor__annotations__rrr/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/models/shieldgemma2/modeling_shieldgemma2.pyr$s rcseZdZUeed<dddddZedfdd Zd d Zd d ZddZ ddZ ddZ ddZ ddZ edeejeejeejeejeeeejefeejeejeejeejeeeeeeeeeeejfedddZZS)"ShieldGemma2ForImageClassificationconfigzmodel.model.language_modelzmodel.model.vision_towerz!model.model.multi_modal_projectorz model.lm_head)zmodel.language_model.modelzmodel.vision_towerzmodel.multi_modal_projectorzmodel.language_model.lm_headrcs<tj|dt|dd|_t|dd|_tj|d|_dS)Nryes_token_indexi *no_token_indexi)super__init__getattrrrr from_configmodel)selfr __class__rrr 7sz+ShieldGemma2ForImageClassification.__init__cCs |jjSN)r#language_modelget_input_embeddingsr$rrrr)=sz7ShieldGemma2ForImageClassification.get_input_embeddingscCs|jj|dSr')r#r(set_input_embeddings)r$valuerrrr+@sz7ShieldGemma2ForImageClassification.set_input_embeddingscCs |jjSr')r#r(get_output_embeddingsr*rrrr-Csz8ShieldGemma2ForImageClassification.get_output_embeddingscCs|jj|dSr')r#r(set_output_embeddings)r$Znew_embeddingsrrrr.Fsz8ShieldGemma2ForImageClassification.set_output_embeddingscCs|jj|dSr')r#r( set_decoder)r$decoderrrrr/Isz.ShieldGemma2ForImageClassification.set_decodercCs |jjSr')r#r( get_decoderr*rrrr1Lsz.ShieldGemma2ForImageClassification.get_decodercCs |jjSr')r#r( tie_weightsr*rrrr2Osz.ShieldGemma2ForImageClassification.tie_weightsNr) input_ids pixel_valuesattention_mask position_idspast_key_valuestoken_type_idscache_position inputs_embedslabels use_cacheoutput_attentionsoutput_hidden_states return_dictlogits_to_keepreturncKsh|jf||||||||| | | | | |d|}|j}|ddd|j|jgf}tj|dd}t||dS)aY Returns: A `ShieldGemma2ImageClassifierOutputWithNoAttention` instance containing the logits and probabilities associated with the model predicting the `Yes` or `No` token as the response to that prompt, captured in the following properties. * `logits` (`torch.Tensor` of shape `(batch_size, 2)`): The first position along dim=1 is the logits for the `Yes` token and the second position along dim=1 is the logits for the `No` token. * `probabilities` (`torch.Tensor` of shape `(batch_size, 2)`): The first position along dim=1 is the probability of predicting the `Yes` token and the second position along dim=1 is the probability of predicting the `No` token. ShieldGemma prompts are constructed such that predicting the `Yes` token means the content *does violate* the policy as described. If you are only interested in the violative condition, use `violated = outputs.probabilities[:, 1]` to extract that slice from the output tensors. When used with the `ShieldGemma2Processor`, the `batch_size` will be equal to `len(images) * len(policies)`, and the order within the batch will be img1_policy1, ... img1_policyN, ... imgM_policyN. )r3r4r5r6r7r8r9r:r;r<r=r>r?r@N)dim)logitsr)r#rDrrrZsoftmaxr)r$r3r4r5r6r7r8r9r:r;r<r=r>r?r@Z lm_kwargsoutputsrDZselected_logitsrrrrforwardRs2'z*ShieldGemma2ForImageClassification.forward)NNNNNNNNNNNNNr)rrrrrZ_checkpoint_conversion_mappingr r)r+r-r.r/r1r2r rrZ LongTensorZ FloatTensorrrlistrboolintrrF __classcell__rrr%rr-s\  r) dataclassesrtypingrrrZtorch.utils.checkpointZ cache_utilsrZmodeling_outputsrZmodeling_utilsrutilsr r autor Zconfiguration_shieldgemma2rZ get_loggerrloggerrr__all__rrrrs       f