a hK$@sddlmZmZddlZddlmZddlmZddl m Z m Z m Z m Z ddlmZmZddlmZd d lmZGd d d e d dZGddde ZdgZdS))OptionalUnionN) BatchFeature) ImageInput)MultiModalDataProcessingKwargsProcessorMixinUnpack)PreTokenizedInput TextInput) TensorType) AutoTokenizerc@s&eZdZddddddejdZdS)AriaProcessorKwargsF)paddingreturn_mm_token_type_ids)max_image_sizeZ split_image) text_kwargs images_kwargsreturn_tensorsN)__name__ __module__ __qualname__r ZPYTORCH _defaultsrrd/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/models/aria/processing_aria.pyr!srF)totalcseZdZdZddgZdZdZdeee fe e e e ee e fe fdfdd Zdeeeeeeefe eeeed d d Zdd dZeddZZS) AriaProcessora AriaProcessor is a processor for the Aria model which wraps the Aria image preprocessor and the LLama slow tokenizer. Args: image_processor (`AriaImageProcessor`, *optional*): The AriaImageProcessor to use for image preprocessing. tokenizer (`PreTrainedTokenizerBase`, *optional*): An instance of [`PreTrainedTokenizerBase`]. This should correspond with the model's text model. The tokenizer is a required input. chat_template (`str`, *optional*): A Jinja template which will be used to convert lists of messages in a chat into a tokenizable string. size_conversion (`Dict`, *optional*): A dictionary indicating size conversions for images. image_processor tokenizerZAriaImageProcessorrN)r! chat_templatesize_conversioncsf|durddd}dd|D|_|j|_|j|_|durP|jdurP|j|_tj|||ddS)N)ircSsi|]\}}t||qSr)int).0kvrrr Kz*AriaProcessor.__init__..)r")itemsr# image_tokenimage_token_idZ pad_tokenZ unk_tokensuper__init__)selfr r!r"r# __class__rrr0Bs zAriaProcessor.__init__)textimageskwargsreturncKsd|jtfd|jji|}t|tr,|g}n t|tsLt|dtsLtd|dur|j|fi|d}|j |j j d}g} | d|} |D]&} | |jj|jj| } | | qni}|} |d d d} |d d d } |j| fi|dd di}|j| |d gd | rPt|d}t|d}d|||jk<||d<ti||| dS)a Main method to prepare for the model one or several sequences(s) and image(s). Args: text (`TextInput`, `PreTokenizedInput`, `list[TextInput]`, `list[PreTokenizedInput]`): The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). images (`ImageInput`): The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch tensor. Both channels-first and channels-last formats are supported. Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. - **pixel_mask** -- Pixel mask to be fed to a model. Returned when `images` is not `None`. Ztokenizer_init_kwargsrzAInvalid input text. Please provide a string, or a list of stringsNrr num_cropsrrrFimage)Z modalitiesZ input_idsmm_token_type_ids)dataZ tensor_type)Z _merge_kwargsrr!Z init_kwargs isinstancestrlist TypeErrorr r#Z pixel_valuesshapepopreplacer-appendZ_check_special_mm_tokensnparrayZ zeros_liker.tolistr)r1r4r5ZaudioZvideosr6Z output_kwargsZ image_inputsZtokens_per_imageZprompt_stringsr8samplerrZ text_inputsZ array_idsr;rrr__call__Ts>  zAriaProcessor.__call__c s~i}|durptjdi|ddp6jjfdd|D}fdd|D}|||dtfi|S)a Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (`list[list[int]]`, *optional*): The input sizes formatted as (height, width) per each image. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. Nrrcs"g|]}jjg|RqSr)r Zget_number_of_image_patches)r'Z image_size)rr1rr sz.csg|]}j|qSr)r#)r'Z num_patches)max_sizer1rrrJr+)num_image_tokensnum_image_patches)rrgetupdater rr)r1Z image_sizesr6Z vision_datarMrLr)rrKr1r_get_num_multimodal_tokenss   z(AriaProcessor._get_num_multimodal_tokenscCs0|jj}|jj}dd|D}tt||S)NcSsg|]}|dkr|qS)r8r)r'namerrrrJr+z3AriaProcessor.model_input_names..)r!model_input_namesr r?dictfromkeys)r1Ztokenizer_input_namesZimage_processor_input_namesrrrrRszAriaProcessor.model_input_names)NNNN)NNN)N)rrr__doc__ attributesZimage_processor_classZtokenizer_classrrr>rrSfloatr&r0r r r?rr rrrIrPpropertyrR __classcell__rrr2rr/s2  D r)typingrrnumpyrEZimage_processing_utilsrZ image_utilsrZprocessing_utilsrrr r Ztokenization_utilsr r utilsr autorrr__all__rrrrs