a h@sddlZddlZddlmZddlmZddlmZddlm Z m Z ddl Z ddl Z ddlmZddlmZmZmZmZmZmZmZmZmZmZmZmZdd lmZmZm Z m!Z!m"Z"m#Z#erZddl$Z%ddl&Z%e'e'e%j(j)e'd kre%j*j+Z,ne%j*Z,erVdd l-m.Z.e,j/er$e.j0ne.j/e,j1e.j1e,j2e.j2e,j3e.j3e,j4e.j4e,j5e.j5iZ6niZ6erjddl7Z7e8e9Z:e d e j;d eGdddeZ?GdddeZ@eAeBe eCeBed5d6d7ZTdfe j;e e e>eBfeCd8d9d:ZUdge j;e>eSeCeCfd;dd?d@ZWeAeBe ez*is_valid_list_of_images..allr=r"r"r#is_valid_list_of_imagessrEcCs\t|dtrdd|DSt|dtjr:tj|ddSt|dtjrXtj|ddSdS)NrcSsg|]}|D]}|q qSr"r")r@Zsublistitemr"r"r# rBz$concatenate_list..Zaxis)dim)r*listnpndarrayZ concatenater1Tensorcat)Z input_listr"r"r#concatenate_lists rOcCs8t|ttfr(|D]}t|sdSqn t|s4dSdS)NFT)r*rJtuple valid_imagesr<)Zimgsr.r"r"r#rQs rQcCst|ttfrt|dSdS)NrF)r*rJrPr<r-r"r"r# is_batcheds rR)r:returncCs,|jtjkrdSt|dko*t|dkS)zV Checks to see whether the pixel values have already been rescaled to [0, 1]. Frr)dtyperKuint8minmaxr9r"r"r#is_scaled_images rX)expected_ndimsrScCst|r |St|r|gSt|rr|j|dkr:t|}n4|j|krL|g}n"td|dd|d|jd|Stdt|ddS) a Ensure that the output is a list of images. If the input is a single image, it is converted to a list of length 1. If the input is a batch of images, it is converted to a list of images. Args: images (`ImageInput`): Image of images to turn into a list of images. expected_ndims (`int`, *optional*, defaults to 3): Expected number of dimensions for a single input image. If the input image has a different number of dimensions, an error is raised. rz%Invalid image shape. Expected either z or z dimensions, but got z dimensions.ztInvalid image type. Expected either PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray, but got .N)rRr/r<ndimrJr7r8r>rZr"r"r#make_list_of_imagess(   r^)r>rZrScCst|ttfr@tdd|Dr@tdd|Dr@dd|DSt|ttfrt|rt|dsp|dj|krt|S|dj|dkrdd|DSt|rt|s|j|kr|gS|j|dkrt|Std |d S) a Ensure that the output is a flat list of images. If the input is a single image, it is converted to a list of length 1. If the input is a nested list of images, it is converted to a flat list of images. Args: images (`Union[list[ImageInput], ImageInput]`): The input image. expected_ndims (`int`, *optional*, defaults to 3): The expected number of dimensions for a single input image. Returns: list: A list of images or a 4d array of images. css|]}t|ttfVqdSr)r*rJrPr@Zimages_ir"r"r#rArBz+make_flat_list_of_images..css|]}t|VqdSr)rEr`r"r"r#rArBcSsg|]}|D]}|q qSr"r"r@Zimg_listr.r"r"r#rGrBz,make_flat_list_of_images..rrcSsg|]}|D]}|q qSr"r"rbr"r"r#rGrBz*Could not make a flat list of images from N r*rJrPrDrEr/r\r<r7r]r"r"r#make_flat_list_of_imagess$ rdcCst|ttfr6tdd|Dr6tdd|Dr6|St|ttfrt|rt|dsf|dj|krl|gS|dj|dkrdd|DSt|rt|s|j|kr|ggS|j|dkrt|gStdd S) as Ensure that the output is a nested list of images. Args: images (`Union[list[ImageInput], ImageInput]`): The input image. expected_ndims (`int`, *optional*, defaults to 3): The expected number of dimensions for a single input image. Returns: list: A list of list of images or a list of 4d array of images. css|]}t|ttfVqdSr)r_r`r"r"r#rArBz-make_nested_list_of_images..css|]}t|VqdSr)rar`r"r"r#rArBrrcSsg|] }t|qSr")rJr?r"r"r#rGrBz.make_nested_list_of_images..z]Invalid input type. Must be a single image, a list of images, or a list of batches of images.Nrcr]r"r"r#make_nested_list_of_imagess$  re)rScCs@t|stdt|tr8t|tjjr8t|St |S)NzInvalid image type: ) r<r7r8rr*r+r,rKarrayrr-r"r"r#to_numpy_array&s  rg.)r: num_channelsrScCs|dur |nd}t|tr |fn|}|jdkr8d\}}n8|jdkrLd\}}n$|jdkr`d\}}ntd|j|j||vr|j||vrtd |jd tjS|j||vrtjS|j||vrtj Std dS) a[ Infers the channel dimension format of `image`. Args: image (`np.ndarray`): The image to infer the channel dimension of. num_channels (`int` or `tuple[int, ...]`, *optional*, defaults to `(1, 3)`): The number of channels of the image. Returns: The channel dimension of the image. NrrYrY)r)rjrkz(Unsupported number of image dimensions: z4The channel dimension is ambiguous. Got image shape z. Assuming channels are the first dimension. Use the [input_data_format](https://huggingface.co/docs/transformers/main/internal/image_processing_utils#transformers.image_transforms.rescale.input_data_format) parameter to assign the channel dimension.z(Unable to infer channel dimension format) r*intr\r7shapeloggerwarningrr r!)r:rhZ first_dimZlast_dimr"r"r#infer_channel_dimension_format/s&       rq)r:input_data_formatrScCsJ|durt|}|tjkr$|jdS|tjkr8|jdStd|dS)a Returns the channel dimension axis of the image. Args: image (`np.ndarray`): The image to get the channel dimension axis of. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the image. If `None`, will infer the channel dimension from the image. Returns: The channel dimension axis of the image. NrYrUnsupported data format: )rqrr r\r!r7)r:rrr"r"r#get_channel_dimension_axisVs    rt)r: channel_dimrScCs^|durt|}|tjkr.|jd|jdfS|tjkrL|jd|jdfStd|dS)a Returns the (height, width) dimensions of the image. Args: image (`np.ndarray`): The image to get the dimensions of. channel_dim (`ChannelDimension`, *optional*): Which dimension the channel dimension is in. If `None`, will infer the channel dimension from the image. Returns: A tuple of the image's height and width. Nrs)rqrr rnr!r7)r:rur"r"r#get_image_sizens   ry) image_size max_height max_widthrSc CsB|\}}||}||}t||}t||}t||} || fS)a Computes the output image size given the input image and the maximum allowed height and width. Keep aspect ratio. Important, even if image_height < max_height and image_width < max_width, the image will be resized to at least one of the edges be equal to max_height or max_width. For example: - input_size: (100, 200), max_height: 50, max_width: 50 -> output_size: (25, 50) - input_size: (100, 200), max_height: 200, max_width: 500 -> output_size: (200, 400) Args: image_size (`tuple[int, int]`): The image to resize. max_height (`int`): The maximum allowed height. max_width (`int`): The maximum allowed width. )rVrm) rzr{r|heightwidthZ height_scaleZ width_scaleZ min_scaleZ new_heightZ new_widthr"r"r##get_image_size_for_max_height_widths   r) annotationrScCsVt|trRd|vrRd|vrRt|dttfrRt|ddksNt|ddtrRdSdS)Nimage_id annotationsrTFr*dictrJrPlenrr"r"r#"is_valid_annotation_coco_detections rcCs^t|trZd|vrZd|vrZd|vrZt|dttfrZt|ddksVt|ddtrZdSdS)NrZ segments_info file_namerTFrrr"r"r#!is_valid_annotation_coco_panoptics  r)rrScCstdd|DS)Ncss|]}t|VqdSr))rr@annr"r"r#rArBz3valid_coco_detection_annotations..rCrr"r"r# valid_coco_detection_annotationssrcCstdd|DS)Ncss|]}t|VqdSr))rrr"r"r#rArBz2valid_coco_panoptic_annotations..rCrr"r"r#valid_coco_panoptic_annotationssr)r:timeoutrSc Csttdgt|tr|ds*|drHtjtt j ||dj }qt j |rbtj|}q|drz|dd}z"t|}tjt|}Wqty}z td|d |WYd }~qd }~00nt|tjjr|}ntd tj|}|d }|S) a3 Loads `image` to a PIL Image. Args: image (`str` or `PIL.Image.Image`): The image to convert to the PIL Image format. timeout (`float`, *optional*): The timeout value in seconds for the URL request. Returns: `PIL.Image.Image`: A PIL Image. Zvisionzhttp://zhttps://rz data:image/,rzIncorrect image source. Must be a valid URL starting with `http://` or `https://`, a valid path to an image file, or a base64 encoded string. Got z. Failed with NzuIncorrect format used for image. Should be an url linking to an image, a base64 string, a local path, or a PIL image.RGB)r load_imager*str startswithr+r,openrrequestsgetcontentospathisfilesplitbase64 decodebytesencode Exceptionr7 TypeErrorZImageOpsZexif_transposeconvert)r:rZb64er"r"r#rs.      r)r>rrScs^t|ttfrNt|r:t|dttfr:fdd|DSfdd|DSn t|dSdS)aLoads images, handling different levels of nesting. Args: images: A single image, a list of images, or a list of lists of images to load. timeout: Timeout for loading images. Returns: A single image, a list of images, a list of lists of images. rcsg|]}fdd|DqS)csg|]}t|dqSrrr?rr"r#rGrBz*load_images...r")r@Z image_grouprr"r#rGrBzload_images..csg|]}t|dqSrrr?rr"r#rGrBrN)r*rJrPrr)r>rr"rr# load_imagess rPILImageResamplingr Z do_rescaleZrescale_factorZ do_normalizeZ image_meanZ image_stdZdo_padZsize_divisibilityZdo_center_cropZ crop_sizeZ do_resizesizeresample interpolationc Cs|r|durtd|r(|dur(td|rD|dus<|durDtd|rX|durXtd| durp| durptd| r| dur| dus| dustddS)a Checks validity of typically used arguments in an `ImageProcessor` `preprocess` method. Raises `ValueError` if arguments incompatibility is caught. Many incompatibilities are model-specific. `do_pad` sometimes needs `size_divisor`, sometimes `size_divisibility`, and sometimes `size`. New models and processors added should follow existing arguments when possible. Nz=`rescale_factor` must be specified if `do_rescale` is `True`.zzDepending on the model, `size_divisibility`, `size_divisor`, `pad_size` or `size` must be specified if `do_pad` is `True`.zP`image_mean` and `image_std` must both be specified if `do_normalize` is `True`.z<`crop_size` must be specified if `do_center_crop` is `True`.zbOnly one of `interpolation` and `resample` should be specified, depending on image processor type.zO`size` and `resample/interpolation` must be specified if `do_resize` is `True`.)r7rr"r"r#validate_preprocess_arguments s    rc@seZdZdZddZdddZddZeje e e fejd d d Z dd dZ ddZdddZd ddZddZddZd!ddZdS)"ImageFeatureExtractionMixinzD Mixin that contain utilities for preparing image features. cCs4t|tjjtjfs0t|s0tdt|ddS)Nz Got type zS which is not supported, only `PIL.Image.Image`, `np.array` and `torch.Tensor` are.)r*r+r,rKrLrr7r8selfr:r"r"r#_ensure_format_supportedBsz4ImageFeatureExtractionMixin._ensure_format_supportedNcCs||t|r|}t|tjr|dur@t|jdtj}|jdkrf|j ddvrf| ddd}|rr|d}| tj }t j|S|S)a" Converts `image` to a PIL Image. Optionally rescales it and puts the channel dimension back as the last axis if needed. Args: image (`PIL.Image.Image` or `numpy.ndarray` or `torch.Tensor`): The image to convert to the PIL Image format. rescale (`bool`, *optional*): Whether or not to apply the scaling factor (to make pixel values integers between 0 and 255). Will default to `True` if the image type is a floating type, `False` otherwise. NrrYrirrj)rrr2r*rKrLflatZfloatingr\rn transposeastyperUr+r,Z fromarray)rr:rescaler"r"r# to_pil_imageIs    z(ImageFeatureExtractionMixin.to_pil_imagecCs&||t|tjjs|S|dS)z Converts `PIL.Image.Image` to RGB format. Args: image (`PIL.Image.Image`): The image to convert. r)rr*r+r,rrr"r"r# convert_rgbgs z'ImageFeatureExtractionMixin.convert_rgb)r:scalerScCs||||S)z7 Rescale a numpy image by scale amount )r)rr:rr"r"r#rus z#ImageFeatureExtractionMixin.rescaleTcCs||t|tjjr"t|}t|r2|}|durLt|jdtj n|}|rh| | tj d}|r|j dkr|ddd}|S)a Converts `image` to a numpy array. Optionally rescales it and puts the channel dimension as the first dimension. Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Tensor`): The image to convert to a NumPy array. rescale (`bool`, *optional*): Whether or not to apply the scaling factor (to make pixel values floats between 0. and 1.). Will default to `True` if the image is a PIL Image or an array/tensor of integers, `False` otherwise. channel_first (`bool`, *optional*, defaults to `True`): Whether or not to permute the dimensions of the image to put the channel dimension first. Nrp?rYrjr)rr*r+r,rKrfrr2rintegerrrfloat32r\r)rr:r channel_firstr"r"r#rg|s  z*ImageFeatureExtractionMixin.to_numpy_arraycCsB||t|tjjr|St|r0|d}ntj|dd}|S)z Expands 2-dimensional `image` to 3 dimensions. Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Tensor`): The image to expand. rrH)rr*r+r,rZ unsqueezerK expand_dimsrr"r"r#rs  z'ImageFeatureExtractionMixin.expand_dimsFcCsv||t|tjjr(|j|dd}n>|rft|tjrN||tj d}nt |rf|| d}t|tjrt|tjst ||j }t|tjst ||j }nrt |r"ddl}t||jst|tjr||}n ||}t||js"t|tjr||}n ||}|jdkrf|jddvrf||ddddf|ddddfS|||SdS)a  Normalizes `image` with `mean` and `std`. Note that this will trigger a conversion of `image` to a NumPy array if it's a PIL Image. Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Tensor`): The image to normalize. mean (`list[float]` or `np.ndarray` or `torch.Tensor`): The mean (per channel) to use for normalization. std (`list[float]` or `np.ndarray` or `torch.Tensor`): The standard deviation (per channel) to use for normalization. rescale (`bool`, *optional*, defaults to `False`): Whether or not to rescale the image to be between 0 and 1. If a PIL image is provided, scaling will happen automatically. T)rrrNrYri)rr*r+r,rgrKrLrrrrfloatrfrTr1rMZ from_numpyZtensorr\rn)rr:meanZstdrr1r"r"r# normalizes4            (z%ImageFeatureExtractionMixin.normalizec CsT|dur |ntj}||t|tjjs4||}t|trFt|}t|t s^t |dkrF|rt|t rt||fn|d|df}n|j \}}||kr||fn||f\}} t|t r|n|d} || kr|S| t | | |} } |dur,|| kr t d|d|| |kr,t || | |} } ||kr>| | fn| | f}|j ||dS)a Resizes `image`. Enforces conversion of input to PIL.Image. Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Tensor`): The image to resize. size (`int` or `tuple[int, int]`): The size to use for resizing the image. If `size` is a sequence like (h, w), output size will be matched to this. If `size` is an int and `default_to_square` is `True`, then image will be resized to (size, size). If `size` is an int and `default_to_square` is `False`, then smaller edge of the image will be matched to this number. i.e, if height > width, then image will be rescaled to (size * height / width, size). resample (`int`, *optional*, defaults to `PILImageResampling.BILINEAR`): The filter to user for resampling. default_to_square (`bool`, *optional*, defaults to `True`): How to convert `size` when it is a single int. If set to `True`, the `size` will be converted to a square (`size`,`size`). If set to `False`, will replicate [`torchvision.transforms.Resize`](https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.Resize) with support for resizing only the smallest edge and providing an optional `max_size`. max_size (`int`, *optional*, defaults to `None`): The maximum allowed for the longer edge of the resized image: if the longer edge of the image is greater than `max_size` after being resized according to `size`, then the image is resized again so that the longer edge is equal to `max_size`. As a result, `size` might be overruled, i.e the smaller edge may be shorter than `size`. Only used if `default_to_square` is `False`. Returns: image: A resized `PIL.Image.Image`. Nrrz max_size = zN must be strictly greater than the requested size for the smaller edge size = )r)rBILINEARrr*r+r,rrJrPrmrrr7resize) rr:rrZdefault_to_squaremax_sizer~r}shortlongZrequested_new_shortZ new_shortZnew_longr"r"r#rs4   $    z"ImageFeatureExtractionMixin.resizecCs||t|ts||f}t|s0t|tjrp|jdkrD||}|jddvr`|jddn |jdd}n|j d|j df}|d|dd}||d}|d|dd}||d}t|t j j r| ||||fS|jddv}|s,t|tjr| ddd}t|r,|ddd}|dkrr||dkrr|dkrr||dkrr|d||||fS|jddt|d|dt|d|df} t|tjrtj|| d} nt|r|| } | d|dd} | |d} | d |dd} | |d}|| d| | | |f<|| 7}|| 7}|| 7}|| 7}| dtd|t| jd|td|t| jd |f} | S) a Crops `image` to the given size using a center crop. Note that if the image is too small to be cropped to the size given, it will be padded (so the returned result has the size asked). Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Tensor` of shape (n_channels, height, width) or (height, width, n_channels)): The image to resize. size (`int` or `tuple[int, int]`): The size to which crop the image. Returns: new_image: A center cropped `PIL.Image.Image` or `np.ndarray` or `torch.Tensor` of shape: (n_channels, height, width). rjrrirN.rv)rnrw)rr*rPrrKrLr\rrnrr+r,croprZpermuterWZ zeros_likeZ new_zerosrV)rr:rZ image_shapetopbottomleftrightrZ new_shapeZ new_imageZtop_padZ bottom_padZleft_padZ right_padr"r"r# center_crop'sP    ,   02    4z'ImageFeatureExtractionMixin.center_cropcCs>||t|tjjr"||}|dddddddfS)a Flips the channel order of `image` from RGB to BGR, or vice versa. Note that this will trigger a conversion of `image` to a NumPy array if it's a PIL Image. Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Tensor`): The image whose color channels to flip. If `np.ndarray` or `torch.Tensor`, the channel dimension should be first. Nrw)rr*r+r,rgrr"r"r#flip_channel_orderrs  z.ImageFeatureExtractionMixin.flip_channel_orderrcCsL|dur |ntjj}||t|tjjs6||}|j||||||dS)a Returns a rotated copy of `image`. This method returns a copy of `image`, rotated the given number of degrees counter clockwise around its centre. Args: image (`PIL.Image.Image` or `np.ndarray` or `torch.Tensor`): The image to rotate. If `np.ndarray` or `torch.Tensor`, will be converted to `PIL.Image.Image` before rotating. Returns: image: A rotated `PIL.Image.Image`. N)rexpandcenter translate fillcolor)r+r,NEARESTrr*rrotate)rr:Zanglerrrrrr"r"r#rs    z"ImageFeatureExtractionMixin.rotate)N)NT)F)NTN)NrNNN)rrr__doc__rrrrKrLrrrmrrgrrrrrrr"r"r"r#r=s   4 CKr)annotation_formatsupported_annotation_formatsrrScCsT||vrtdtd||tjur6t|s6td|tjurPt|sPtddS)NzUnsupported annotation format: z must be one of zInvalid COCO detection annotations. Annotations must a dict (single image) or list of dicts (batch of images) with the following keys: `image_id` and `annotations`, with the latter being a list of annotations in the COCO format.zInvalid COCO panoptic annotations. Annotations must a dict (single image) or list of dicts (batch of images) with the following keys: `image_id`, `file_name` and `segments_info`, with the latter being a list of annotations in the COCO format.)r7formatr$r%rr&r)rrrr"r"r#validate_annotationss  r)valid_processor_keyscaptured_kwargscCs6t|t|}|r2d|}td|ddS)Nz, zUnused or unrecognized kwargs: r[)set differencejoinrorp)rrZ unused_keysZunused_key_strr"r"r#validate_kwargss rT)frozenc@szeZdZUdZdZeeed<dZeeed<dZ eeed<dZ eeed<dZ eeed<dZ eeed<d d Z dS) SizeDictz> Hashable dictionary to store image size information. Nr}r~ longest_edge shortest_edger{r|cCs(t||rt||Std|ddS)NzKey z not found in SizeDict.)hasattrgetattrKeyError)rkeyr"r"r# __getitem__s  zSizeDict.__getitem__)rrrrr}rrm__annotations__r~rrr{r|rr"r"r"r#rs r)rY)rY)rY)N)N)N)N)N) NNNNNNNNNNNNN)drrcollections.abcr dataclassesriortypingrrr2rKr packagingrutilsr r r r r rrrrrrrZutils.constantsrrrrrrZ PIL.Imager+Z PIL.ImageOpsparse __version__ base_versionr,Z ResamplingrZtorchvision.transformsrrZ NEAREST_EXACTZBOXrZHAMMINGZBICUBICZLANCZOSZpil_torch_interpolation_mappingr1Z get_loggerrrorLrJZ ImageInputrr$r'rrrmZAnnotationTyper/r0r;r<rErOrQrRboolrXr^rdrergrPrqrtryrrrrrrrrrrrrrr"r"r"r#s    8         ) ( '  (     $$ -  0b