a h@s6dZddlmZddlmZddlmZdddZdS) z9VPTQ (Vector Post-Training Quantization) integration fileN)init_empty_weights) VQuantLinearFcCsr|r|ndg}|D]P\}}|dur,g}||d|}|j}|j} t|tjr2||vr2|| vsz|d|vr2| |dp||dd} t |j } |j } t | | | d| d| d| d| d | d | d | d | d dd|j dud|j|<d}|j|dWdn1s(0Ytt|dkr^t|||||d\} }|dq||fS)aw Public method that recursively replaces the Linear layers of the given model with VPTQ quantized layers. `accelerate` is needed to use this method. Returns the converted model and a boolean that indicates if the conversion has been successful or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`VptqConfig`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list[`str`]`, *optional*, defaults to `["lm_head"]`): Names of the modules to not convert in `VQuantLinear`. In practice we keep the `lm_head` in full precision for numerical stability reasons. current_key_name (`list`, *optional*): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. Zlm_headN. vector_lens num_centroidsnum_res_centroids group_num group_size outlier_sizeindices_as_float enable_norm enable_permTF) rrrr r r r r rZis_indice_packedZenable_proxy_errorbiasr)quantization_configmodules_to_not_convertcurrent_key_namehas_been_replaced)Znamed_childrenappendjoinshared_layer_configconfig_for_layers isinstancennZLineargetr in_features out_featuresrrZ_modulesZrequires_grad_lenlistchildrenreplace_with_vptq_linearpop)modelrrrrnamemoduleZ layer_namerrZ layer_paramsrr_r&Z/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/integrations/vptq.pyr sd     0  r )NNNF)__doc__Ztorch.nnrZ acceleraterZvptqrr r&r&r&r's