a ­ Àh\0ã@s”ddlZddlZddlZddlZe e¡Zejddfdd„Z ej Z ddd„Z ejddfd d „Z dd d „Z dd d„Zddd„Zddd„Zddd„ZdS)éNFéÿÿÿÿcCs*t d|j|f¡t ¡}|j\}}t |¡} tj||| d} t ||¡} |dkr\t ¡}|rŒt d|¡t  ¡} || _ tj | | |d} d} |D]f}|jd}|   |¡|   ||¡\}}|| 7}|  ||¡|  ¡| |7} t d| t ¡|f¡q”|  ¡t dt ¡|| f¡| j| jfS) z¤Computes the exact KNN search results for a dataset that possibly does not fit in RAM but for which we have an iterator that returns it block by block. z%knn_ground_truth queries size %s k=%d©Úkeep_maxrúrunning on %d GPUs©ÚcoÚngpurú%d db elements, %.3f szGT time: %.3f s (%d vectors))ÚLOGÚinfoÚshapeÚtimeÚfaissÚis_similarity_metricZ ResultHeapÚ IndexFlatÚ get_num_gpusÚGpuMultipleClonerOptionsÚshardÚindex_cpu_to_all_gpusÚaddÚsearchZ add_resultÚresetÚfinalizeÚDÚI)ÚxqÚ db_iteratorÚkÚ metric_typerrÚt0ÚnqÚdrZrhÚindexrÚi0ÚxbiÚnirr©r&ú[/var/www/html/assistant/venv/lib/python3.9/site-packages/faiss/contrib/exhaustive_search.pyÚknn_ground_truths4      r(éc Cs¦|j\}}t|tjƒ}t |j¡}|r.t|ƒnt|ƒ}t|j |ƒ} t   d|›d| ›d|›d|›¡t   ¡} |  || ¡\} } t   ¡| } |r¬|ddks¢J‚|  d¡} d}d }|d urŽ|sÜ| d d …| d f|k}n| d d …| d f|k}| ¡dkrŽt   d | ¡¡t   ¡} t|tjƒr\|}|rDt |d¡}nt ||j¡}| |¡| |||¡\}}}|r‚| d¡}t   ¡| }t   d ¡t   ¡} |r¬tjntj}||| ||ƒ}tj}|| ƒ|_|| ƒ|_|d ur||ƒ|_||ƒ|_|| d ¡ƒ|_||ƒ|_tj |d d d}| !||ƒ¡|d}tj || j"d}tj |d d}| #||ƒ||ƒ¡t   ¡| }t   d| d›d|d›d|d›d¡|||fS)aAGPU does not support range search, so we emulate it with knn search + fallback to CPU index. The index_cpu can either be: - a CPU index that supports range search - a numpy table, that will be used to construct a Flat index if needed. - None. In that case, at most gpu_k results will be returned z GPU search z queries with k=z is_binary_index=z keep_max=éi€Úint16rNézCPU search remain %dÚcombineÚint64©Údtyperztimes z.3fzs Ús)%r Ú isinstancerÚ IndexBinaryrrÚintÚfloatÚminZntotalr Údebugr rÚastypeÚsumÚnpZndarrayZIndexBinaryFlatrrÚ range_searchZCombinerRangeKNNint16ZCombinerRangeKNNfloatZswig_ptrrrÚmaskÚD_remainÚviewÚ lim_remainÚI_remainÚemptyZ compute_sizesr0Z write_resultÚrange)rÚr2Ú index_gpuZ index_cpuZgpu_kr r!Úis_binary_indexrrrrrÚt1Út2r?r<Úxbr=r@ZCombinerRangeKNNZcombinerÚspZL_resÚnresZD_resZI_resZt3r&r&r'Úrange_search_gpu:sp     ÿ        þ       $rKcs|j\}}t ¡}tj|dd}t ||¡} |dkr­óz&range_ground_truth..cSsg|]}g‘qSr&r&rMr&r&r'rO®rPr,r r.cs"g|]}|gkrt |¡nˆ‘qSr&©r:Úhstack©rNÚi)Úempty_Dr&r'rOÅrPcs"g|]}|gkrt |¡nˆ‘qSr&rQrS)Úempty_Ir&r'rOÆrPcSsg|] }t|ƒ‘qSr&©ÚlenrSr&r&r'rOÇrPÚuint64N)r r r:Zascontiguousarrayrrrr r rrrrBrrKrr;ÚappendÚzerosrXÚcumsumrR)rrÚ thresholdrrrr r!rr"rrDr#rrr$r%Úlims_iÚDiÚIiÚjÚl0Úl1ÚsizesÚlimsr&)rUrVr'Úrange_ground_truth˜sP        rfc Csp|r||k}n||k}t |¡}d}t|ƒD]0\}} t| ƒ} |||| … ¡||<|| 7}q,|||||fS)z select a set of results r)r:Ú zeros_likeÚ enumerater4r9) rJÚdisÚidsÚthreshrr<Znew_nresÚorTÚnrr&r&r'Úthreshold_radius_nresÎs   rnc Cs‚|r||k}n||k}t |¡}t|ƒd}t|ƒD]:}||||d} } |||| | … ¡||d<q4|||||fS)z= restrict range-search results to those below a given radius r,)r:rgrXrBr9) rerirjrkrr<Znew_limsÚnrTrbrcr&r&r'Úthreshold_radiusÝs    "rpc Csìt dd„|Dƒ¡}t|ƒ|ks$J‚|rL| t|ƒ|d¡|d|}n| |¡||}|jdkrrt|ƒ}nt|ƒ}t d|¡d}t |ƒD]@\}\}}} t ||| ||d\}}} |t|ƒ7}||| f||<q”t d |¡||fS) z‘find radius that reduces number of results to target_nres, and applies it in-place to the result batches used in range_search_max_resultscSsg|]\}}}|‘qSr&r&)rNÚ_rir&r&r'rOïrPz apply_maxres..r,rrLz setting radius to %srrz. updated previous results, new nb results %d) r:rRrXÚ partitionr0r5r4r r7rhrn) Ú res_batchesZ target_nresrZalldisÚradiusÚtotresrTrJrirjr&r&r'Ú apply_maxresës(    ÿ  rvcCs˜t|tjƒ}|dur,|dus J‚td|ƒ}|durL|dus@J‚t|dƒ}|dkr\t ¡}|rŽt d|¡t ¡} || _tj || |d} nd} t   ¡} d} } d}}}g}|D].}t   ¡}t  dt |ƒ›d ¡| rôt ||| |ƒ\}}}n| ||¡\}}}|d d…|dd…}|t |ƒ7}|t |ƒ7}t   ¡}|rN| d ¡}|t |ƒ7}| |||f¡|durª||krªt d ||f¡t|||jtjkd \}}t   ¡}| ||7} | ||7} t  dt   ¡| ||f¡q¶t d| | ||f¡|r(||kr(t|||jtjkd \}}t dd„|Dƒ¡}t dd„|Dƒ¡}t dd„|Dƒ¡}tjt |ƒd dd}t |¡|d d…<||||fS)aPerforms a range search with many queries (given by an iterator) and adjusts the threshold on-the-fly so that the total results table does not grow larger than max_results. If ngpu != 0, the function moves the index to this many GPUs to speed up search. Ngš™™™™™é?gø?rrrrz searching z vectorsr,r+z-too many results %d > %d, scaling back radiusrz' [%.3f s] %d queries done, %d resultszBsearch done in %.3f s + %.3f s, total %d results, end threshold %gcSsg|]\}}}|‘qSr&r&©rNÚnres_iZdis_iZids_ir&r&r'rOYrPz,range_search_max_results..cSsg|]\}}}|‘qSr&r&rwr&r&r'rOZrPcSsg|]\}}}|‘qSr&r&rwr&r&r'rO[rPrYr/)r2rr3r4rr r rrrr r7rXrKr;r8rZrvrZMETRIC_INNER_PRODUCTr:rRr[r\)r"Zquery_iteratorrtZ max_resultsZ min_resultsrrZ clip_to_minrErrDZt_startZt_searchZt_post_processZqtotruZ raw_totresrsÚxqirr^r_r`rxrFrGrJrirjrer&r&r'Úrange_search_max_resultss~           ÿ þ   ÿ ÿÿ þ rzé é NccsPt|ƒ}|}d}||krL||||…}|V||kr>|d9}|t|ƒ7}qdS)z¤ produces batches of progressively increasing sizes. This is useful to adjust the search radius progressively without overflowing with intermediate results réNrW)rZstart_bsZmax_bsr ÚbsrTryr&r&r'Úexponential_query_iteratorcsr)r))F)F)F)NNFrF)r{r|)rr Únumpyr:ÚloggingÚ getLoggerÚ__name__r Z METRIC_L2r(ZknnrKrfrnrprvrzrr&r&r&r'Ús" ' ^ÿ 6   þ \