a h>@sUdZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZmZm Z m!Z!m"Z"m#Z#d d l$m%Z%m&Z&m'Z'm(Z(d d l)m*Z*m+Z+e rdd l,m-Z-m.Z.ddl/m0Z0e1Z2ee1e fZ3ee4efZ5ee5e3e2dfZ6ddhZ7e8e9Z:e GdddZ;e GdddZddZ?ee6ee5dddZ@eedeeddddZ@eee6eee5ddfdd dZ@e6e1dd!d"ZAe6e1e1d#d$d%ZBe1dd&d'd(ZCe4edd)d*ZDe4edd+d,ZEe4ddd-d.ZFee4efed/d0d1ZGee4eHeee1ee+fd2d3d4ZIee4eHeee1ee+fd2d5d6ZJe4eHeee1e+fd7d8d9ZKee4ee*d:d;d<ZLee4ee*d:d=d>ZMe4ee*d?d@dAZNdBdCee4dDdEdFZOiZPeee1ee1feQdG<ee1ee1ddHdIdJZRee1ee1dKdLdMZSeedNdOdPZTee1ee1e!dQdRdSZUdS)TzEContains utilities used by both the sync and async inference clients.N)contextmanager) dataclass)Path) TYPE_CHECKINGAny AsyncIterableBinaryIOContextManagerDict GeneratorIterableListLiteralNoReturnOptionalUnionoverload) HTTPError)GenerationErrorIncompleteGenerationErrorOverloadedErrorTextGenerationError UnknownErrorValidationError) get_sessionis_aiohttp_availableis_numpy_availableis_pillow_available)ChatCompletionStreamOutputTextGenerationStreamOutput)ClientResponse ClientSessionImager%z text-to-imagezimage-to-imagec@s\eZdZUeed<eed<eeed<eeeeefed<ee ed<eee fed<dS)RequestParametersurltaskmodeljsondataheadersN) __name__ __module__ __qualname__str__annotations__rrr r ContentTrr3r3]/var/www/html/assistant/venv/lib/python3.9/site-packages/huggingface_hub/inference/_common.pyr&Is   r&c@s2eZdZUdZeed<eed<eed<eed<dS) ModelStatusas This Dataclass represents the model status in the HF Inference API. Args: loaded (`bool`): If the model is currently loaded into HF's Inference API. Models are loaded on-demand, leading to the user's first request taking longer. If a model is loaded, you can be assured that it is in a healthy state. state (`str`): The current state of the model. This can be 'Loaded', 'Loadable', 'TooBig'. If a model's state is 'Loadable', it's not too big and has a supported backend. Loadable models are automatically loaded when the user first requests inference on the endpoint. This means it is transparent for the user to load a model, except that the first call takes longer to complete. compute_type (`Dict`): Information about the compute resource the model is using or will use, such as 'gpu' type and number of replicas. framework (`str`): The name of the framework that the model was built with, such as 'transformers' or 'text-generation-inference'. ZloadedstateZ compute_typeZ frameworkN)r-r.r/__doc__boolr1r0r r3r3r3r4r5Ts r5cCststdddl}|S)NzMPlease install aiohttp to use `AsyncInferenceClient` (`pip install aiohttp`).r)r ImportErroraiohttp)r:r3r3r4_import_aiohttpusr;cCststdddl}|S)z.Make sure `numpy` is installed on the machine.zGPlease install numpy to use deal with embeddings (`pip install numpy`).rN)rr9numpy)r<r3r3r4 _import_numpy~sr=cCststdddlm}|S)z,Make sure `PIL` is installed on the machine.zPlease install Pillow to use deal with images (`pip install Pillow`). If you don't want the image to be post-processed, use `client.post(...)` and get the raw response from the server.rr$)rr9PILr%r$r3r3r4_import_pil_images  r?)contentreturncCsdSNr3r@r3r3r4_open_as_binarysrDcCsdSrBr3rCr3r3r4rDsccst|trb|ds|drBtd|t|jVdSt|}| sbt d|dt|trtd|| d}|VWdn1s0YdSt rd d l m}t||jrtd t}|j||jpd d |VdS|VdS)zOpen `content` as a binary file, either from a URL, a local path, raw bytes, or a PIL Image. Do nothing if `content` is None. TODO: handle base64 as input https://http://zDownloading content from NzFile not found at z. If `data` is a string, it must either be a URL or a path to a local file. To pass raw content, please encode it as bytes first.zOpening content from rbrr$zConverting PIL Image to bytesPNG)format) isinstancer0 startswithloggerdebugrgetr@rexistsFileNotFoundErroropenrr>r%ioBytesIOsaverIgetvalue)r@fr%bufferr3r3r4rDs0    $   cCsPt|4}t|tr|n|}t|WdS1sB0YdS)z\Encode a raw file (image, audio) into base64. Can be bytes, an opened file, a path or a URL.N)rDrJbytesreadbase64 b64encodedecode)r@r+Z data_as_bytesr3r3r4 _b64_encodes r])r@default_mime_typerAcCst|tr"|ds|dr"|Sd}t|ttfrHtj|ddd}n2trzddlm}t||jrzd|j ppd }|p|}t |}d |d |S) NrErFF)strictrr$zimage/rHdata:z;base64,) rJr0rKr mimetypes guess_typerr>r%rIlowerr])r@r^Z mime_typer%Z encoded_datar3r3r4_as_urls  rd) encoded_imagerAcCst}|tt|S)z/Parse a base64-encoded string into a PIL Image.)r?rQrRrSrZ b64decode)rer%r3r3r4 _b64_to_imagesrgcCst|S)acParse bytes from a Response object into a Python list. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_dict` and will not complain if the returned data is a dictionary. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. r*loadsr\rCr3r3r4_bytes_to_listsrjcCst|S)acParse bytes from a Response object into a Python dictionary. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_list` and will not complain if the returned data is a list. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. rhrCr3r3r4_bytes_to_dictsrkcCst}|t|S)zParse bytes from a Response object into a PIL Image. Expects the response body to be raw bytes. To deal with b64 encoded images, use `_b64_to_image` instead. )r?rQrRrS)r@r%r3r3r4_bytes_to_imagesrl)responserAcCst|trt|S|SrB)rJrXr*ri)rmr3r3r4_as_dict srn)bytes_output_as_linesdetailsrAc csD|D]:}zt||}Wnty.Yq@Yn0|dur|VqdS)z*Used in `InferenceClient.text_generation`.N%_format_text_generation_stream_output StopIterationrorp byte_payloadoutputr3r3r4 _stream_text_generation_responses  rwc CsN|2zB3dHW}zt||}Wnty6YqJYn0|dur|Vq6dS)z/Used in `AsyncInferenceClient.text_generation`.Nrqrtr3r3r4&_async_stream_text_generation_response%s  rx)rurprAcCs~|dsdS|dkr"td|d}t|dd}|ddurdt |d|dt |}|sz|j j S|S Nsdata:s data: [DONE]z[DONE] signal received.zutf-8r`z/nerror error_type)rKstriprsr\r*rilstriprstriprN_parse_text_generation_errorr!parse_obj_as_instancetokentext)rurppayload json_payloadrvr3r3r4rr3s    rr) bytes_linesrAc csB|D]8}z t|}Wnty,Yq>Yn0|dur|VqdS)zFUsed in `InferenceClient.chat_completion` if model is served with TGI.N%_format_chat_completion_stream_outputrsritemrvr3r3r4 _stream_chat_completion_responseIs   rc CsL|2z@3dHW}z t|}Wnty4YqHYn0|dur|Vq6dS)z/Used in `AsyncInferenceClient.chat_completion`.Nrrr3r3r4&_async_stream_chat_completion_responseVs   r)rurAcCsn|dsdS|dkr"td|d}t|dd}|ddurdt |d|dt |Sry) rKr|rsr\r*rir}r~rNrr r)rurrr3r3r4rcs   rr#r")clientrmrAcCsHz2|j2z3dHW}|Vq6W|IdHn|IdH0dSrB)r@r|close)rrmrur3r3r4_async_yield_fromxsr#_UNSUPPORTED_TEXT_GENERATION_KWARGS)r)unsupported_kwargsrAcCst|g|dSrB)r setdefaultextend)r)rr3r3r4'_set_unsupported_text_generation_kwargssr)r)rAcCs t|gSrB)rrN)r)r3r3r4'_get_unsupported_text_generation_kwargssr) http_errorrAcCsfz.t|ddp|j}|d}|d}WntyD|Yn0|dur^t||}|||dS)z Try to parse text-generation-inference error message and raise HTTPError in any case. Args: error (`HTTPError`): The HTTPError that have been raised. Zresponse_error_payloadNrzr{)getattrrmr*rN Exceptionr)rrrzr{ exceptionr3r3r4raise_text_generation_errors     r)rzr{rAcCsH|dkrt|S|dkr t|S|dkr0t|S|dkr@t|St|S)NZ generationZincomplete_generationZ overloadedZ validation)rrrrr)rzr{r3r3r4rsr)Vr7rZrRr*loggingra contextlibr dataclassesrpathlibrtypingrrrrr r r r r rrrrrrequestsrZhuggingface_hub.errorsrrrrrrutilsrrrrZ_generated.typesr r!r:r"r#Z PIL.Imager%r0ZUrlTZPathTrXZBinaryTr2ZTASKS_EXPECTING_IMAGES getLoggerr-rLr&r5r;r=r?rDr]rdrgrjrkrlrnr8rwrxrrrrrrrr1rrrrr3r3r3r4s   @           $*