a h@s:ddlmZmZmZer"ddlZeeZdddZdS))is_optimum_quanto_availableis_torch_availableloggingNFc sddlm}tr2ddlm}m}m}m} m} m } || | | d} d|| d} |durXg}| D]6\}}durvg |t fdd|Dsb|t |tjjr||j|j|jdu|jj| |j| |jd |j|<|j|d d }nJt |tjjrB|jdurB||j|j|j|jdu| |jd |j|<d }Wdn1sX0Ytt| dkrt!||||d \}}"dq`||fS)aZ Public method that recursively replaces the Linear layers of the given model with Quanto quantized layers. Returns the converted model and a boolean that indicates if the conversion has been successful or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`AqlmConfig`, defaults to `None`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list`, *optional*, defaults to `None`): A list of modules to not convert. If a module name is in the list (e.g. `lm_head`), it will not be converted. current_key_name (`list`, *optional*, defaults to `None`): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*, defaults to `None`): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. r)init_empty_weights) QLayerNormQLinearqfloat8qint2qint4qint8)float8int8Zint4Zint2N)Nr rc3s|]}|dvVqdS).N)join).0keycurrent_key_name\/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/integrations/quanto.py Az-replace_with_quanto_layers..) in_features out_featuresbiasdtypeweights activationsFT)r)quantization_configmodules_to_not_convertrhas_been_replaced)#Z acceleraterrZoptimum.quantorrr r r r Znamed_childrenappendany isinstancetorchnnZLinearrrrweightrrrZ_modulesZrequires_grad_Z LayerNormZnormalized_shapeepsZelementwise_affinelenlistchildrenreplace_with_quanto_layerspop)modelrr rr!rrrr r r r Z w_mappingZ a_mappingnamemodule_rrrr-sX       $  r-)NNNF) utilsrrrr&Z get_logger__name__loggerr-rrrrs