a hM(@sddlmZmZmZddlmZddlmZmZm Z m Z m Z m Z ddl mZmZe rnddlmZddlmZerdd lmZe rdd lZdd lmZe eZeed d d GdddeZd S))AnyUnionoverload)GenerationConfig)add_end_docstringsis_tf_availableis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args)Image) load_image)'TF_MODEL_FOR_VISION_2_SEQ_MAPPING_NAMESN)$MODEL_FOR_VISION_2_SEQ_MAPPING_NAMEST)Z has_tokenizerZhas_image_processorcseZdZdZdZdZdZdZdZe ddZ fddZ dd d Z e eed feeeeefd d dZe eeeed feeeeeefd ddZeeeed ed fdfdd ZdddZddZddZZS)ImageToTextPipelinea Image To Text pipeline using a `AutoModelForVision2Seq`. This pipeline predicts a caption for a given image. Unless the model you're using explicitly sets these generation parameters in its configuration files (`generation_config.json`), the following default values will be used: - max_new_tokens: 256 Example: ```python >>> from transformers import pipeline >>> captioner = pipeline(model="ydshieh/vit-gpt2-coco-en") >>> captioner("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png") [{'generated_text': 'two birds are standing next to each other '}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This image to text pipeline can currently be loaded from pipeline() using the following task identifier: "image-to-text". See the list of available models on [huggingface.co/models](https://huggingface.co/models?pipeline_tag=image-to-text). TF)max_new_tokenscs8tj|i|t|d||jdkr.tntdS)NZvisiontf)super__init__r Zcheck_model_type frameworkrr)selfargskwargs __class__`/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/pipelines/image_to_text.pyrTs  zImageToTextPipeline.__init__NcCsi}i}|dur||d<|dur(||d<|dur8||d<|durb|durXd|vrXtd|||jdurv|j|d<|jdur|j|d<|j|d<||ifS)Nprompttimeoutrzp`max_new_tokens` is defined both as an argument and inside `generate_kwargs` argument, please use only 1 versionassistant_model tokenizerassistant_tokenizer) ValueErrorupdater$r&r%)rrgenerate_kwargsr"r#Zforward_paramsZpreprocess_paramsr r r!_sanitize_parameters[s(      z(ImageToTextPipeline._sanitize_parametersz Image.Image)inputsrreturncKsdSNr rr+rr r r!__call__vszImageToTextPipeline.__call__cKsdSr-r r.r r r!r/ys)r+c s6d|vr|d}|dur"tdtj|fi|S)a Assign labels to the image(s) passed as inputs. Args: inputs (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`): The pipeline handles three types of images: - A string containing a HTTP(s) link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. max_new_tokens (`int`, *optional*): The amount of maximum tokens to generate. By default it will use `generate` default. generate_kwargs (`Dict`, *optional*): Pass it to send all of these arguments directly to `generate` allowing full control of this function. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A list or a list of list of `dict`: Each result comes as a dictionary with the following key: - **generated_text** (`str`) -- The generated text. imagesNzBCannot call the image-to-text pipeline without an inputs argument!)popr'rr/r.rr r!r/|s  cCst||d}|durFtdt|ts>tdt|d|jjj }|dkr|j ||j d}|j dkrv| |j }|j|dd j}|jjg|}t|d }|d |in|d kr|j |||j d }|j dkr| |j }n^|dkr4|j ||j d}|j dkr| |j }|j||j d}||ntd|dn(|j ||j d}|j dkrn| |j }|jjj dkr|durd|d <|S)N)r#uPassing `prompt` to the `image-to-text` pipeline is deprecated and will be removed in version 4.48 of 🤗 Transformers. Use the `image-text-to-text` pipeline insteadz&Received an invalid text input, got - zy - but expected a single string. Note also that one single text can be provided for conditional image to text generation.git)r0return_tensorsptF)textZadd_special_tokensr input_idsZ pix2struct)r0Z header_textr3zvision-encoder-decoder)r3z Model type z- does not support conditional text generation)rloggerZ warning_once isinstancestrr'typemodelconfig model_typeZimage_processorrtoZdtyper%r6Z cls_token_idtorchZtensorZ unsqueezer()rimager"r#r= model_inputsr6Z text_inputsr r r! preprocesssF             zImageToTextPipeline.preprocesscKspd|vr4t|dtr4tdd|dDr4d|d<d|vrF|j|d<||jj}|jj|fi||}|S)Nr6css|]}|duVqdSr-r ).0xr r r! z/ImageToTextPipeline._forward..generation_config)r8listallrGr1r;Zmain_input_namegenerate)rrAr)r+ model_outputsr r r!_forwards  zImageToTextPipeline._forwardcCs0g}|D]"}d|jj|ddi}||q|S)NZgenerated_textT)Zskip_special_tokens)r%decodeappend)rrKrecordsZ output_idsrecordr r r! postprocesss zImageToTextPipeline.postprocess)NNNN)NN)__name__ __module__ __qualname____doc__Z_pipeline_calls_generateZ_load_processorZ_load_image_processorZ_load_feature_extractorZ_load_tokenizerrZ_default_generation_configrr*rrr9rrHdictr/rBrLrQ __classcell__r r rr!r.s&  (4&$ 3r)typingrrrZ generationrutilsrrr r r r baserrZPILrZ image_utilsrZmodels.auto.modeling_tf_autorr?Zmodels.auto.modeling_autorZ get_loggerrRr7rr r r r!s