a h@s`dZddlmZmZmZer&ddlZeeZddZ ddZ d d Z dd d Z dddZ dS)z2HQQ (Half-Quadratic Quantization) integration file)is_hqq_availableis_torch_availableloggingNcCs|D]\}}||_qdSN) named_modulesname)modelrmoduler Y/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/integrations/hqq.pyautoname_modulessr cCsddd|dDS)N.cSs g|]}|dvr|s|qS))r Zlayers) isnumeric).0nr r r !z&name_to_linear_tag..)joinsplit)rr r r name_to_linear_tag srcCsRtrddlm}t}|D](\}}t|tjj|fr | t |q t |S)Nr) HQQLinear) rZhqq.core.quantizersetr isinstancetorchnnLinearaddrlist)r r linear_tagsrr r r r get_linear_tags%s r c Cs|D]\}}|durg}||t|tjjrt|j}||vr||dur|||j|_ t ||j|_ |j| dd}dD]}t ||dqtt|dkrt|||d\}}|dq||fS)NFT)ZW_qmetar patch_paramshas_been_replaced)Znamed_childrenappendrrrrrrZ_modules quant_configtypeZ source_clsZrequires_grad_setattrlenrchildren_prepare_for_hqq_linearpop) r r#r$Zcurrent_key_namerr Z linear_tagZatt_r r r r,0s,     r,Fcs|dur gn|}t|t||j}|j}ttt|t|tfdd|Drtt}| |n t|}t |||d\}}||j |d|j _ |std|S)a Prepares nn.Linear layers for HQQ quantization. Since each layer type can have separate quantization parameters, we need to do the following: 1- tag each module with its name via autoname_modules() 2- Extract linear_tags (e.g. ['self_attn.q_proj', ...]) 3- Map quantization parameters as a dictionary linear_tag -> quant_params as HQQLinear expects it, this is referred to as patch_params Nc3s|]}|vVqdSrr )rkeyrr r krz)prepare_for_hqq_linear..r")r' quant_method skip_moduleszs  %