a h@sddlmZddlmZddlmZddlmZmZm Z ddl m Z m Z ddl mZeeZGdd d ed d ZGd d d eZd gZdS))Union) BatchFeature) ImageInput)ProcessingKwargsProcessorMixinUnpack)PreTokenizedInput TextInput)loggingc@seZdZddiidZdS)Ovis2ProcessorKwargspaddingF) text_kwargsZ image_kwargsN)__name__ __module__ __qualname__ _defaultsrrf/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/models/ovis2/processing_ovis2.pyr sr F)totalcseZdZdZddgZdZdZdfd d Zdee e e e e e e fe eed d d Ze e e e edddZddZddZeddZZS)Ovis2Processora Constructs a Ovis2 processor which wraps Ovis2 image processor and a Qwen2 tokenizer into a single processor. [`Ovis2Processor`] offers all the functionalities of [`Ovis2VideoProcessor`], [`Ovis2ImageProcessor`] and [`Qwen2TokenizerFast`]. See the [`~Ovis2Processor.__call__`] and [`~Ovis2Processor.decode`] for more information. Args: image_processor ([`Ovis2ImageProcessor`], *optional*): The image processor is a required input. tokenizer ([`Qwen2TokenizerFast`], *optional*): The tokenizer is a required input. chat_template (`str`, *optional*): A Jinja template which will be used to convert lists of messages in a chat into a tokenizable string. image_token (`str`, *optional*, defaults to `""`): Special token used to denote image location. image_seq_length (`int`, *optional*, defaults to 256): The number of image tokens to be used for each image in the input. image_processor tokenizerZAutoImageProcessorZ AutoTokenizerNc sZ||_t|dr|jn||_t|ddr.|jn ||j|_tj||fd|i|dS)N image_tokenimage_token_id chat_template)image_seq_lengthhasattrrgetattrrZconvert_tokens_to_idssuper__init__)selfrrrrrkwargs __class__rrr"=s   zOvis2Processor.__init__)imagestextr$returncKs|jtfd|jji|}t|tr,|g}n t|tsLt|dtsLtdi}|dur|j|fi|d}| d }| ||}|j|fi|d}t i||dS) a Main method to prepare for the model one or several sequences(s) and image(s). This method forwards the `text` and `kwargs` arguments to Qwen2TokenizerFast's [`~Qwen2TokenizerFast.__call__`] if `text` is not `None` to encode the text. To prepare the image(s), this method forwards the `images` and `kwrags` arguments to Ovis2ImageProcessor's [`~Ovis2ImageProcessor.__call__`] if `images` is not `None`. Please refer to the docstring of the above two methods for more information. Args: images (`PIL.Image.Image`, `np.ndarray`, `torch.Tensor`, `List[PIL.Image.Image]`, `List[np.ndarray]`, `List[torch.Tensor]`): The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch tensor. Both channels-first and channels-last formats are supported. text (`str`, `List[str]`, `List[List[str]]`): The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. - **image_sizes** -- Size of each image that will be used to unpad an image. Returned when `images` is not `None`. Ztokenizer_init_kwargsrzAInvalid input text. Please provide a string, or a list of stringsNZ images_kwargsgridsr)data) Z _merge_kwargsr rZ init_kwargs isinstancestrlist ValueErrorrpoptolist_expand_image_tokensr)r#r'r(r$Z output_kwargsZ image_inputsZ image_gridsZ text_inputsrrr__call__Os$!  zOvis2Processor.__call__)r(r*c Csg}d}|D]}d|vr||}|d|d}}dd|jd} ||dkrt|D]J} t|D](} | d|j7} | |dkrd| d7} qd| |dkrX| d7} qX| d 7} |d| d}|d7}q||q |S) Nrrz z z z z z )rrangereplaceappend) r#r(r*Zprocessed_textZ grid_indexsamplegridrowcol placeholderrcrrrr2s(         z#Ovis2Processor._expand_image_tokenscOs|jj|i|S)z This method forwards all its arguments to Qwen2TokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of this method for more information. )r batch_decoder#argsr$rrrr?szOvis2Processor.batch_decodecOs|jj|i|S)z This method forwards all its arguments to Qwen2TokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this method for more information. )rdecoder@rrrrBszOvis2Processor.decodecCs |jj}|jj}t|t|S)N)rmodel_input_namesrr.)r#Ztokenizer_input_namesZimage_processor_input_namesrrrrCsz Ovis2Processor.model_input_names)NNNrr)NN)rrr__doc__ attributesZimage_processor_classZtokenizer_classr"rrr r r.rr rr3intr2r?rBpropertyrC __classcell__rrr%rr%s2 8 rN)typingrZfeature_extraction_utilsrZ image_utilsrZprocessing_utilsrrrZtokenization_utils_baser r utilsr Z get_loggerrloggerr r__all__rrrrs