a hi@sBddlZddlZddlZddlmZmZddlmZddlm Z m Z ddl m Z m Z mZmZmZddlZddlmZddlmZmZmZmZd d lmZmZmZe rdd lmZdd lm Z dd l!m"Z"ddl#m$Z$erddl%m&Z&e'e(Z)dZ*dZ+ej,GdddZ-GdddeZ.Gddde.eZ/Gddde/Z0dS)N)ABCabstractmethod) OrderedDict)IterableMapping) TYPE_CHECKINGAnyCallableOptionalUnion)version) TensorTypeis_torch_availableis_vision_availablelogging)ParameterFormat compute_effective_axis_dimension"compute_serialized_parameters_size)PretrainedConfigFeatureExtractionMixinImageProcessingMixinPreTrainedTokenizerBase)Image lc@sJeZdZUdZeed<eed<eed<dZe eed<dZ e eed<dS) PatchingSpeca Data class that holds patching specifications. Args: o: Module / object where the op to patch is located name: Name of the op to monkey patch custom_op: Custom op that patches the original op orig_op: Original op that is being patched op_wrapper: Wrapper (optional) that wraps both the original and custom ops. It is useful for ops that are class or static methods for instance. oname custom_opNorig_op op_wrapper) __name__ __module__ __qualname____doc__r__annotations__strr r#r r$r+r+T/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/onnx/config.pyr0s  rc@seZdZdZdZdZdZedZ e dddd ie d ddd ie dddd ie ddd ddd ddd d e dddd ie dddd ie dd diie ddd ddd d e ddd ddd de ddddddie dddd ie dd diie dddd ie dddd ie dddd idZ dPde e eedddZedQde ddddZeeee eee ffddd Zeee eee ffdd!d"Zee ee efdd#d$Zeedd%d&Zeedd'd(Zeedd)d*Zeedd+d,Zeedd-d.Z ee!dd/d0Z"e#ee!d1d2d3Z$dReeeed6d7d8Z%dSeeeed<d=d>Z&dTe'dAeeee!e e(eeeeeee dBee efdCdDdEZ)ee efee efdFdGdHZ*dIdJZ+dKdLZ,ee e-ee.e efdMdNdOZ/dS)U OnnxConfigzv Base class for ONNX exportable model describing metadata on how to export the model through the ONNX format. r z1.8logitsbatchsequencerrZlast_hidden_state)r0 pred_boxesZ pred_masksr)r0r4)Z start_logitsZ end_logitsZ num_labelsheightwidth)rrr decoder_sequence)z causal-lmdefaultzimage-classificationzimage-segmentationz masked-imz masked-lmmultiple-choicezobject-detectionzquestion-answeringzsemantic-segmentationz seq2seq-lmzsequence-classificationztoken-classificationz vision2seq-lmz speech2seq-lmr9Nr)configtaskpatching_specscCs||_||jvr(t|d|j||_g|_|dur@|ngD]6}|}|jdurntj|t |j |j d}|j |qDdS)Nz+ is not a supported task, supported tasks: )r#) _config_tasks_to_common_outputs ValueErrorkeysr<_patching_specsr# dataclassesreplacegetattrr r!append)selfr;r<r=specZ final_specr+r+r,__init__ps  zOnnxConfig.__init__r;r<returncCs |||dS)z Instantiate a OnnxConfig for a specific model Args: config: The model's configuration to use when exporting to ONNX Returns: OnnxConfig for this model )r<r+clsr;r<r+r+r,from_model_configs zOnnxConfig.from_model_configrKcCs tdS)z Mapping containing the axis definition of the input tensors to provide to the model Returns: For each input: its name associated to the axes symbolic name and the axis position within the tensor N)NotImplementedErrorrGr+r+r,inputss zOnnxConfig.inputscCs|j|j}t|S)z Mapping containing the axis definition of the output tensors to provide to the model Returns: For each output: its name associated to the axes symbolic name and the axis position within the tensor )r?r<copydeepcopyrGcommon_outputsr+r+r,outputss zOnnxConfig.outputscCst|jdrddiSdS)z Dictionary of keys to override in the model's config before exporting Returns: Dictionary with the keys (and their corresponding values) to override use_cacheFN)hasattrr>rQr+r+r,values_overrides zOnnxConfig.values_overridecCstjS)zp The default batch size to use if no other indication Returns: Integer > 0 )r-default_fixed_batchrQr+r+r,default_batch_sizes zOnnxConfig.default_batch_sizecCstjS)zu The default sequence length to use if no other indication Returns: Integer > 0 )r-default_fixed_sequencerQr+r+r,default_sequence_lengthsz"OnnxConfig.default_sequence_lengthcCstjS)zw The default number of choices to use if no other indication Returns: Integer > 0 )r-default_fixed_num_choicesrQr+r+r,default_num_choicesszOnnxConfig.default_num_choicescCstS)z{ Which onnx opset to use when exporting the model Returns: Integer ONNX Opset version )DEFAULT_ONNX_OPSETrQr+r+r,default_onnx_opsetszOnnxConfig.default_onnx_opsetcCsdS)z What absolute tolerance value to use during model conversion validation. Returns: Float absolute tolerance value. gh㈵>r+rQr+r+r,atol_for_validationszOnnxConfig.atol_for_validationcCs,tr$ddlm}t||jkSdSdS)z The minimum PyTorch version required to export the model. Returns: `bool`: Whether the installed version of PyTorch is compatible with the model. r)get_torch_versionFN)rZtransformers.utilsrdr parsetorch_onnx_minimum_version)rGrdr+r+r,is_torch_support_availables z%OnnxConfig.is_torch_support_available)num_parametersrKcCst|tjtkS)a Flag indicating if the model requires using external data format Args: num_parameters: Number of parameter on the model Returns: True if model.num_parameters() * size_of(float32) >= 2Gb False otherwise )rrFloatEXTERNAL_DATA_FORMAT_SIZE_LIMIT)rhr+r+r,use_external_data_formats z#OnnxConfig.use_external_data_formatr7() batch_size num_channels image_height image_widthcCsFg}t|D]4}tj|||d}|t|ddq |S)NZuint8RGB) rangenprandomZrandrFrZ fromarrayZastypeconvert)rGrmrnrorpimages_datar+r+r,_generate_dummy_imagess  z!OnnxConfig._generate_dummy_images"V@)rm sampling_rate time_duration frequencycCsRg}t|D]@}tjd|t||dd}|dtdtj||q |S)NrF)Zendpointg?r )rsrtZlinspaceintrFsinpi)rGrmr~rrZ audio_datarxtr+r+r,_generate_dummy_audios  $z OnnxConfig._generate_dummy_audioF)rrrr) preprocessorrm seq_length num_choicesis_pair frameworkrnrpror~rr tokenizerrKcs^ddlm}ddlm}ddlm}t||r>| dur>td| dur`t dt t d| }t||rVt |tjd d }||}t |tj|d }|jdurt|jd kr|jnd }d |g|g|}|jd krFt tjd d |}|||d}|D].\}fddtd tD||<qt|j|dSt|||dSt||r|jd dkrtd|jjd|jd t |tjd}|||| |}t|||dSt||r|jd dkrt |tjd}|||| |}t|||dSt||rR|jd dkrRt |tjd}||| | | }t|||dStddS)am Generate inputs to provide to the ONNX exporter for the specific framework Args: preprocessor: ([`PreTrainedTokenizerBase`], [`FeatureExtractionMixin`], or [`ImageProcessingMixin`]): The preprocessor associated with this model configuration. batch_size (`int`, *optional*, defaults to -1): The batch size to export the model for (-1 means dynamic axis). num_choices (`int`, *optional*, defaults to -1): The number of candidate answers provided for multiple choice task (-1 means dynamic axis). seq_length (`int`, *optional*, defaults to -1): The sequence length to export the model for (-1 means dynamic axis). is_pair (`bool`, *optional*, defaults to `False`): Indicate if the input is a pair (sentence 1, sentence 2) framework (`TensorType`, *optional*, defaults to `None`): The framework (PyTorch or TensorFlow) that the tokenizer will generate tensors for. num_channels (`int`, *optional*, defaults to 3): The number of channels of the generated images. image_width (`int`, *optional*, defaults to 40): The width of the generated images. image_height (`int`, *optional*, defaults to 40): The height of the generated images. sampling_rate (`int`, *optional* defaults to 22050) The sampling rate for audio data generation. time_duration (`float`, *optional* defaults to 5.0) Total seconds of sampling for audio data generation. frequency (`int`, *optional* defaults to 220) The desired natural frequency of generated audio. Returns: Mapping[str, Tensor] holding the kwargs to provide to the model's forward function r rrrNzPYou cannot provide both a tokenizer and a preprocessor to generate dummy inputs.ztThe `tokenizer` argument is deprecated and will be removed in version 5 of Transformers. Use `preprocessor` instead.zROverwriting the `preprocessor` argument with `tokenizer` to generate dummy inputs.r)fixed_dimensionZnum_token_to_add0 r:)Z text_paircsg|]}||qSr+r+).0irvr+r, uz4OnnxConfig.generate_dummy_inputs..)Z tensor_type)return_tensorsZ pixel_valuesz*The `preprocessor` is an image processor (zC) and expects `model_input_names[0]` to be "pixel_values", but got )r)rwrZinput_featuresz\Unable to generate dummy inputs for the model. Please provide a tokenizer or a preprocessor.) feature_extraction_utilsrimage_processing_utilsrtokenization_utils_baser isinstancer@warningswarn FutureWarningloggerwarningrr-r[Znum_special_tokens_to_addr]Z unk_tokenlenjoinr<r_itemsrsdictZconvert_to_tensorsZmodel_input_names __class__r%rzr)rGrrmrrrrrnrpror~rrrrrrZ token_to_addZ input_tokenZ dummy_inputZtokenized_inputkr+rr,generate_dummy_inputssz0         (   z OnnxConfig.generate_dummy_inputs)reference_model_inputsrKcCs|S)a Generate inputs for ONNX Runtime using the reference model inputs. Override this to run inference with seq2seq models which have the encoder and decoder exported as separate ONNX files. Args: reference_model_inputs ([`Mapping[str, Tensor]`): Reference inputs for the model. Returns: `Mapping[str, Tensor]`: The mapping holding the kwargs to provide to the model's forward function r+)rGrr+r+r,!generate_dummy_inputs_onnxruntimes z,OnnxConfig.generate_dummy_inputs_onnxruntimecCs<|jD]0}|jdur|jn ||j}t|j|j|qdSN)rBr$r"setattrr r!)rGrHr"r+r+r, patch_opss zOnnxConfig.patch_opscCs<|jD]0}|jdur|jn ||j}t|j|j|qdSr)rBr$r#rr r!)rGrHr#r+r+r, restore_opss zOnnxConfig.restore_opsr!fieldrKcs(ddlm}fddt||DS)a Flatten any potential nested structure expanding the name of the field with the index of the element within the structure. Args: name: The name of the nested structure field: The structure to, potentially, be flattened Returns: (dict[str, Any]): Outputs with flattened structure and key mapping this new structure. r)chaincs i|]\}}d||qS).r+)ridxitemr!r+r, rzAOnnxConfig.flatten_output_collection_property..) itertoolsr enumerate from_iterable)rMr!rrr+rr,"flatten_output_collection_propertys z-OnnxConfig.flatten_output_collection_property)r9N)r9)r r7rlrl)r r{r|r}) rrrFNr7rlrlr{r|r}N)0r%r&r'r(r[r]r_r rerfrr?r*r listrrI classmethodrNpropertyrrrrRrWrrZr\r^r`rbfloatrcboolrg staticmethodrkrzrr rrrrrrrrr+r+r+r,r-Es #  "           x r-c s*eZdZd!deeeeedfdd Ze d"deddd d Z e e ee e effd fd d Ze ee eefd ddZe e d ddZe e d ddZd#de e eeee eefdfdd Zd$e ee e effeedddZddZeeeeeefdfdd ZZS)%OnnxConfigWithPastr9NFr)r;r<r=use_pastcstj|||d||_dS)N)r<r=)superrIr)rGr;r<r=rrr+r,rIszOnnxConfigWithPast.__init__rJcCs|||ddS)z Instantiate a OnnxConfig with `use_past` attribute set to True Args: config: The underlying model's config to use when exporting to ONNX Returns: OnnxConfig with `.use_past = True` T)r<rr+rLr+r+r, with_pasts zOnnxConfigWithPast.with_pastrOcs tj}|jr|j|dd|S)NrW direction)rrWrfill_with_past_key_values_rUrr+r,rWszOnnxConfigWithPast.outputscCst|jdrd|jiSdS)NrX)rYr>rrQr+r+r,rZs  z"OnnxConfigWithPast.values_overridecCst|jdstd|jjS)z The number of layers attribute retrieved from the model config. Override this for model configs where the number of layers attribute is not called `num_layers`. num_layerszcould not find the number of layers attribute in the model configuration, override the num_layers property of the model OnnxConfig to solve this)rYr>AttributeErrorrrQr+r+r,rs  zOnnxConfigWithPast.num_layerscCst|jdstd|jjS)z The number of attention heads attribute retrieved from the model config. Override this for model configs where the number of attention heads attribute is not called `num_attention_heads`. num_attention_headszcould not find the number of attention heads attribute in the model configuration, override the num_attention_heads property of the model OnnxConfig to solve this)rYr>rrrQr+r+r,rs  z&OnnxConfigWithPast.num_attention_headsrrrrmrrrrKcstj|||||d}|jrts,tdnddl}|dj\}} | d} ||j| |jj |jf} d|vr|dj } |j |d|j || | dgdd |d<g|d <t |jD]"} |d || || fq|S) NrmrrrACannot generate dummy past_keys inputs without PyTorch installed.r input_idsr Zattention_mask)dtyper)dimpast_key_values)rrrrr@torchshaperr> hidden_sizercatZonesrsrrFzeros)rGrrmrrr common_inputsrr1ZseqlenZpast_key_values_lengthrZ mask_dtyperxrr+r,rs0       z(OnnxConfigWithPast.generate_dummy_inputs)inputs_or_outputsrinverted_values_shapecCs|dvrtd|d|dkr$dnd}t|jD]X}ddd ||d |d <|rpddd ||d |d <q2ddd ||d |d <q2dS)a Fill the input_or_outputs mapping with past_key_values dynamic axes considering. Args: inputs_or_outputs: The mapping to fill. direction: either "inputs" or "outputs", it specifies whether input_or_outputs is the input mapping or the output mapping, this is important for axes naming. inverted_values_shape: If `True`, store values on dynamic axis 1, else on axis 2. rRrW4direction must either be "inputs" or "outputs", but was givenrRrpresentr1zpast_sequence + sequencerr r.keyr3.valueN)r@rsr)rGrrrr!rr+r+r,r)sz-OnnxConfigWithPast.fill_with_past_key_values_cCs4|d||d|d<|d||d|d<dS)Nrrrrrr+rGflattened_outputr!rrr+r+r,_flatten_past_key_values_Bsz,OnnxConfigWithPast._flatten_past_key_values_rcsBi}|dvr0t|D]\}}|||||qnt||}|S)N)rr)rrrr)rGr!rrrrrr+r,rFs z5OnnxConfigWithPast.flatten_output_collection_property)r9NF)r9)rrFN)F)r%r&r'r*r rrrrIrrrrrrWrrZrrrrrrrrr __classcell__r+r+rr,rsJ   $  + rc seZdZeeeeeeffdfdd Zeeedfdd Z eeedfdd Z de d eee e e eeefd fd d ZeeeeeffedddZddZZS)OnnxSeq2SeqConfigWithPastrOcsrtt|j}|D]D\}}d|vr(dnd}|D]"\}}d|vrN|||<q4|||<q4q|jrn|j|dd|S)Nencoderencoder_sequencer8r2rWr)rrrWrrr)rGrVr!Z axes_namesZ sequence_nameZaxis_idxrr+r,rWRs   z!OnnxSeq2SeqConfigWithPast.outputscs^ztj}||f}WnDtyXt|jdrLt|jdrL|jj|jjf}ntdYn0|S)Nencoder_layersdecoder_layerszcould not find the number of encoder and decoder layers attributes in the model configuration, override the num_layers property of the model OnnxConfig to solve this)rrrrYr>rr)rGrrr+r,rcs   z$OnnxSeq2SeqConfigWithPast.num_layerscs^ztj}||f}WnDtyXt|jdrLt|jdrL|jj|jjf}ntdYn0|S)Nencoder_attention_headsdecoder_attention_headszcould not find the number of attention heads for the encoder and the decoder attributes in the model configuration, override the num_attention_heads property of the model OnnxConfig to solve this)rrrrYr>rr)rGrrr+r,rss   z-OnnxSeq2SeqConfigWithPast.num_attention_headsrFNrrc stt|j|||||d}|js$|nd}tt|j|||||d}dd|D}tfi||} |jrts~tdnddl} | dj d} | dj d} | dj d}|j \} }| | | |j j | f}| ||d |j j |f}g| d <|j \}}t||}t|||}||krd nd }t|D]4}| d | || || || |fq*|d krn|n|}t||D]$}| d | || |fq|| S) NrrcSsi|]\}}d||qS)Zdecoder_r+)rr!Ztensorr+r+r,rrzCOnnxSeq2SeqConfigWithPast.generate_dummy_inputs..rrrZdecoder_input_idsr7rrdecoder)rrrrrrrr@rrrr>rrminmaxrsrFr)rGrrmrrrZencoder_inputsZdecoder_seq_lengthZdecoder_inputsrrr1Zencoder_seq_lengthZnum_encoder_attention_headsZnum_decoder_attention_headsZ encoder_shapeZ decoder_shapenum_encoder_layersnum_decoder_layersmin_num_layersmax_num_layersremaining_side_namerxrrr+r,rsX            "z/OnnxSeq2SeqConfigWithPast.generate_dummy_inputs)rrc Cs2|dvrtd|d|dkr$dnd}|j\}}t||}t|||}||krVdnd}d } |dkrjd nd } t|D]l} d | d ||d| d<d | d ||d| d<d | d ||d| d<d | d ||d| d<qvt||D]>} |dkrd | d } n d | d } | ||d| d|d<qdS)NrrrrRrrrrZpast_encoder_sequenceZpast_decoder_sequencez past_decoder_sequence + sequencer1rr .decoder.key.decoder.value .encoder.key.encoder.valuer)r@rrrrs) rGrrr!rrrrrrr8rZ axes_infor+r+r,rs&      z4OnnxSeq2SeqConfigWithPast.fill_with_past_key_values_cCsd|d||d|d<|d||d|d<|d||d|d<|d||d|d <dS) Nrrrrrr rr7rr+rr+r+r,rsz3OnnxSeq2SeqConfigWithPast._flatten_past_key_values_)rrFN)r%r&r'rrr*rrWtuplerrr rrrrrrrr+r+rr,rQs($ E r)1rSrCrabcrr collectionsrcollections.abcrrtypingrrr r r numpyrt packagingr utilsrrrrrrrZconfiguration_utilsrrrrrrrZPILrZ get_loggerr%rrarj dataclassrr-rrr+r+r+r,s8        {