a hYl@sddlmZddlmZddlmZmZddlmZm Z m Z m Z ddl Z ddlmZmZmZddlmZmZmZmZmZdd lmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#dd l$m%Z%dd l&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-dd l.m/Z/e,rdd lm0Z0e)r ddl1Z1e*rBddlm2Z2e+r4ddl3m4Z5n ddl6m4Z5ndZ2e-7e8Z9edddddddddddddddej:fe e;e e<e e;e e ee e;e ee e;e ee de e e?e'fe edddZ@d)de e>ddddZAeee=edddZBe=deCe>dddZDe e jEdfe>e=e e jEdfd d!d"ZFGd#d$d$e d%d&ZGe(Gd'd(d(eZHdS)*)Iterable)deepcopy) lru_cachepartial)AnyOptional TypedDictUnionN)BaseImageProcessor BatchFeature get_size_dict)convert_to_rgbget_resize_output_image_sizeget_size_with_aspect_ratiogroup_images_by_shapereorder_images) ChannelDimension ImageInput ImageTypeSizeDictget_image_size#get_image_size_for_max_height_widthget_image_typeinfer_channel_dimension_formatmake_flat_list_of_imagesvalidate_kwargsvalidate_preprocess_arguments)Unpack) TensorTypeauto_docstringis_torch_availableis_torchvision_availableis_torchvision_v2_availableis_vision_availablelogging)is_rocm_platform)PILImageResampling)pil_torch_interpolation_mapping) functional maxsizeF.InterpolationMode do_rescalerescale_factor do_normalize image_mean image_stddo_padsize_divisibilitydo_center_crop crop_size do_resizesize interpolationreturn_tensors data_formatcCsNt|||||||||| | | d | dur8| dkr8td| tjkrJtddS)z Checks validity of typically used arguments in an `ImageProcessorFast` `preprocess` method. Raises `ValueError` if arguments incompatibility is caught. ) r/r0r1r2r3r4r5r6r7r8r9r:Nptz6Only returning PyTorch tensors is currently supported.z6Only channel first data format is currently supported.)r ValueErrorrFIRSTr.r@d/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/image_processing_utils_fast.py"validate_fast_preprocess_argumentsKs$ rB torch.Tensor)tensoraxisreturncCs:|dur|Sz|j|dWSty4|YS0dS)zF Squeezes a tensor, but only if the axis specified has dim 1. N)rE)Zsqueezer>)rDrEr@r@rA safe_squeezevs  rG)valuesrFcCsddt|DS)zO Return the maximum value across all indices of an iterable of values. cSsg|] }t|qSr@)max).0Zvalues_ir@r@rA z&max_across_indices..)zip)rHr@r@rAmax_across_indicessrN)imagesrFcCs tdd|D\}}}||fS)zH Get the maximum height and width across all images in a batch. cSsg|] }|jqSr@)shaperJZimgr@r@rArKrLz(get_max_height_width..)rN)rO_ max_height max_widthr@r@rAget_max_height_widthsrU)image patch_sizerFcCsjg}t|tjd\}}td||D]B}td||D]0}|dd||||||f}||q2q"|S)a6 Divides an image into patches of a specified size. Args: image (`Union[np.array, "torch.Tensor"]`): The input image. patch_size (`int`): The size of each patch. Returns: list: A list of Union[np.array, "torch.Tensor"] representing the patches. )Z channel_dimrN)rrr?rangeappend)rVrWZpatchesheightwidthijpatchr@r@rAdivide_to_patchess"r_c@seZdZUeeed<eeeefed<eeed<ee ded<eeed<eeeefed<eeed<ee ee fed <eeed <ee e e e fed <ee e e e fed <eeed <ee ee fed<ee ed<ee ee fed<eded<eeed<dS)DefaultFastImageProcessorKwargsr8r9default_to_square)r'r-resampler6r7r/r0r1r2r3do_convert_rgbr;r<input_data_format torch.devicedevicedisable_groupingN)__name__ __module__ __qualname__rbool__annotations__dictstrintr floatlistrrr@r@r@rAr`s"         r`F)totalcseZdZdZdZdZdZdZdZdZ dZ dZ dZ dZ dZdZejZdZdZdgZeZdZeedfdd Zeedd d ZdHd ed ed d ddZe dId e!e"e"fe#d ed dddZ$d e%d dddZ&d e'e%e(e%fe'e%e(e%fd dddZ)e*dddJe#ee#e'e%e+e%fe#e'e%e+e%fe#ee#e%e#de!dddZ,d ee%ee'e%e+e%fe'e%e+e%fd dd d!Z-d e.e/e"fd d"d#d$Z0e1e1d%d&d'Z2e.dd(d)Z3dKe1e"e1d+d,d-Z4dLe1e#ee#e'e/efe#dd d.d/d0Z5dMe1e#ee#e'e/efe#de"e+d d1d2d3Z6dNe#ee#ee#ee#e'e%e+e%fe#e'e%e+e%fe#ee.d4d5d6Z7dOe#ee#e%e#ee#e'e%e!e%fe#e'e%e!e%fe#ee#ee#ee#ee#d e#e'e/e8fe#ed7 d8d9Z9e1eee:d:d;d<Z;eZ=dd?e1eee#e'e/dfeee:d@dAdBZ>e+d eee#d eeee%ee#e'e%e+e%fe#e'e%e+e%fe#ee#e'e/e8fe:dCdDdEZ?fdFdGZ@ZAS)PBaseImageProcessorFastNTgp? pixel_values)kwargsc stjfi|||}|d|j}|durHt||d|jdnd|_|d|j}|durpt|ddnd|_|jj D]>}||d}|durt |||q~t ||t t ||dq~t |jj |_dS)Nr9rar9rar7 param_name)super__init__filter_out_unused_kwargspopr9r rar7 valid_kwargsrlsetattrrgetattrrqkeys_valid_kwargs_names)selfrur9r7keykwarg __class__r@rArzs   zBaseImageProcessorFast.__init__)rFcCsdS)zv `bool`: Whether or not this image processor is a fast processor (backed by PyTorch and TorchVision). Tr@)rr@r@rAis_fastszBaseImageProcessorFast.is_fastrCr-)rVr9r: antialiasrFcKs|dur |ntjj}|jr>|jr>t|dd|j|j}np|jrZt||jdtj d}nT|j r|j rt |dd|j |j }n*|j r|jr|j |jf}ntd|dtjrtr|||||Stj||||dS)a@ Resize an image to `(size["height"], size["width"])`. Args: image (`torch.Tensor`): Image to resize. size (`SizeDict`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. interpolation (`InterpolationMode`, *optional*, defaults to `InterpolationMode.BILINEAR`): `InterpolationMode` filter to use when resizing the image e.g. `InterpolationMode.BICUBIC`. Returns: `torch.Tensor`: The resized image. NF)r9rardzjSize must contain 'height' and 'width' keys, or 'max_height' and 'max_width', or 'shortest_edge' key. Got .r:r)FZInterpolationModeZBILINEARZ shortest_edgeZ longest_edgerr9rrr?rSrTrrZr[r>torchcompilerZ is_compilingr&compile_friendly_resizeresize)rrVr9r:rrunew_sizer@r@rArs4   zBaseImageProcessorFast.resize)rVrr:rrFcCs~|jtjkrh|d}tj||||d}|d}t|dkd|}t|dkd|}|tj}ntj||||d}|S)z{ A wrapper around `F.resize` so that it is compatible with torch.compile when the image is a uint8 tensor. rr) dtyperZuint8rprrwhereroundto)rVrr:rr@r@rAr(s  z.BaseImageProcessorFast.compile_friendly_resize)rVscalerFcKs||S)a? Rescale an image by a scale factor. image = image * scale. Args: image (`torch.Tensor`): Image to rescale. scale (`float`): The scaling factor to rescale pixel values by. Returns: `torch.Tensor`: The rescaled image. r@)rrVrrur@r@rArescale=szBaseImageProcessorFast.rescale)rVmeanstdrFcKst|||S)a Normalize an image. image = (image - image_mean) / image_std. Args: image (`torch.Tensor`): Image to normalize. mean (`torch.Tensor`, `float` or `Iterable[float]`): Image mean to use for normalization. std (`torch.Tensor`, `float` or `Iterable[float]`): Image standard deviation to use for normalization. Returns: `torch.Tensor`: The normalized image. )r normalize)rrVrrrur@r@rArQsz BaseImageProcessorFast.normalizer*r+re)r1r2r3r/r0rfrFcCsB|r8|r8tj||dd|}tj||dd|}d}|||fS)Nrfg?F)rrD)rr1r2r3r/r0rfr@r@rA!_fuse_mean_std_and_rescale_factorhs z8BaseImageProcessorFast._fuse_mean_std_and_rescale_factor)rOr/r0r1r2r3rFcCsP|j||||||jd\}}}|r<||jtjd||}n|rL|||}|S)z/ Rescale and normalize images. )r1r2r3r/r0rf)r)rrfrrrZfloat32r)rrOr/r0r1r2r3r@r@rArescale_and_normalizeys   z,BaseImageProcessorFast.rescale_and_normalize)rVr9rFcKs>|jdus|jdur&td|t||d|dfS)a Center crop an image to `(size["height"], size["width"])`. If the input size is smaller than `crop_size` along any edge, the image is padded with 0's and then center cropped. Args: image (`"torch.Tensor"`): Image to center crop. size (`dict[str, int]`): Size of the output image. Returns: `torch.Tensor`: The center cropped image. Nz=The size dictionary must have keys 'height' and 'width'. Got rZr[)rZr[r>rr center_crop)rrVr9rur@r@rArsz"BaseImageProcessorFast.center_crop)rVrFcCst|S)a' Converts an image to RGB format. Only converts if the image is of type PIL.Image.Image, otherwise returns the image as is. Args: image (ImageInput): The image to convert. Returns: ImageInput: The converted image. )r)rrVr@r@rArsz%BaseImageProcessorFast.convert_to_rgbcCsB|jdur|S|jD](}||vrtd|d||q|S)zJ Filter out the unused kwargs from the kwargs dictionary. Nz!This processor does not use the `z ` parameter. It will be ignored.) unused_kwargsloggerZ warning_oncer|)rru kwarg_namer@r@rAr{s   z/BaseImageProcessorFast.filter_out_unused_kwargs)rOexpected_ndimsrFcCs||}t||dS)z Prepare the images structure for processing. Args: images (`ImageInput`): The input images to process. Returns: `ImageInput`: The images with a valid nesting. r)Z fetch_imagesr)rrOrr@r@rA_prepare_images_structures z0BaseImageProcessorFast._prepare_images_structure)rVrcrdrfrFcCst|}|tjtjtjfvr*td||r8||}|tjkrNt|}n|tjkrft | }|j dkrz| d}|durt|}|tjkr|ddd }|dur||}|S)NzUnsupported input image type rr )rrZPILZTORCHZNUMPYr>rrZ pil_to_tensorrZ from_numpy contiguousndimZ unsqueezerrZLASTZpermuter)rrVrcrdrfZ image_typer@r@rA_process_images$        z%BaseImageProcessorFast._process_image)rOrcrdrfrrFcsl|j||d}t|j|||dt|dko..r@)rJZ nested_listrr@rArK(rLzEBaseImageProcessorFast._prepare_image_like_inputs..csg|] }|qSr@r@rQrr@rArK*rL)rrrlen isinstancerqtuple)rrOrcrdrfrZhas_nested_structureprocessed_imagesr@rrA_prepare_image_like_inputss z1BaseImageProcessorFast._prepare_image_like_inputs)r9r7rar2r3r<rFc Ks|dur i}|dur*tfit||d}|durHtfit|dd}t|trZt|}t|trlt|}|durztj}||d<||d<||d<||d<||d<|d }t|tt frt |n||d <|S) z Update kwargs that need further processing before being validated Can be overridden by subclasses to customize the processing of kwargs. Nrvr7rwr9r2r3r<rbr:) rr rrqrrr?r|r'ror() rr9r7rar2r3r<rurbr@r@rA_further_process_kwargs.s*   z.BaseImageProcessorFast._further_process_kwargs r/r0r1r2r3r8r9r6r7r:r;r<c Ks$t||||||||| | | | d dS)z@ validate the kwargs for the preprocess method. rN)rB)rr/r0r1r2r3r8r9r6r7r:r;r<rur@r@rA_validate_preprocess_kwargsZsz2BaseImageProcessorFast._validate_preprocess_kwargs)rOrurFcOs|j|g|Ri|S)N) preprocess)rrOargsrur@r@rA__call__|szBaseImageProcessorFast.__call__cOst||jd|jD]}||t||dq|d}|d}|d}|jfi|}|jfi||d|j|g|R|||d|S)N)Zcaptured_kwargsZvalid_processor_keysrcrdrfr<r) rrr setdefaultrr|rr_preprocess_image_like_inputs)rrOrrurrcrdrfr@r@rArs$     z!BaseImageProcessorFast.preprocessr)rOrcrdrfrurFcOs*|j||||d}|j|g|Ri|S)z Preprocess image-like inputs. To be overridden by subclasses when image-like inputs other than images should be processed. It can be used for segmentation maps, depth maps, etc. )rOrcrdrf)r _preprocess)rrOrcrdrfrrur@r@rArsz4BaseImageProcessorFast._preprocess_image_like_inputs)rOr8r9r:r6r7r/r0r1r2r3rgr;rFc Kst|| d\}}i}|D]$\}}|r8|j|||d}|||<qt||}t|| d\}}i}|D]4\}}|r|||}||||| | | }|||<qht||}| rtj|ddn|}td|i| dS)N)rg)rVr9r:r)dimrt)dataZ tensor_type) ritemsrrrrrstackr )rrOr8r9r:r6r7r/r0r1r2r3rgr;ruZgrouped_imagesZgrouped_images_indexZresized_images_groupedrPZstacked_imagesZresized_imagesZprocessed_images_groupedrr@r@rArs&      z"BaseImageProcessorFast._preprocesscs&t}|dd|dd|S)NZ_valid_processor_keysr)ryto_dictr|)rZ encoder_dictrr@rArs   zBaseImageProcessorFast.to_dict)NT)NT)NNNNNN)r)NNN)NNNr)NNNNNN) NNNNNNNNNNNN)Brhrirjrbr2r3r9rar7r8r6r/r0r1rcr;rr?r<rdrfZmodel_input_namesr`r}rrrzpropertyrkrrr staticmethodrrorrrprr rrrrqrrrmrnrrrr{rrrrrrr rr rrrr __classcell__r@r@rrArssf  6         ) . . "   ,rs)N)Icollections.abcrcopyr functoolsrrtypingrrrr numpynpZimage_processing_utilsr r r Zimage_transformsrrrrrZ image_utilsrrrrrrrrrrrZprocessing_utilsrutilsrr r!r"r#r$r%Zutils.import_utilsr&r'rr(Ztorchvision.transforms.v2r)rZtorchvision.transformsZ get_loggerrhrr?rkrprqrornrBrGrNrrUarrayr_r`rsr@r@r@rAsz  4 $    *