a h<.@sddlmZddlmZddlmZddlZddlm Z m Z m Z m Z m Z ddlmZddd d d Zd"d d ZGdddeZddZGdddeZddZddZddddZd#ddZGdddeZd d!ZdS)$)Counter)suppress) NamedTupleN)_isin _searchsorted _setdiff1ddevice get_namespace is_scalar_nanFreturn_inverse return_countscCs&|jtkrt|||dSt|||dS)aHelper function to find unique values with support for python objects. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : ndarray Values to check for unknowns. return_inverse : bool, default=False If True, also return the indices of the unique values. return_counts : bool, default=False If True, also return the number of times each unique item appears in values. Returns ------- unique : ndarray The sorted unique values. unique_inverse : ndarray The indices to reconstruct the original array from the unique array. Only provided if `return_inverse` is True. unique_counts : ndarray The number of times each of the unique values comes up in the original array. Only provided if `return_counts` is True. r )dtypeobject_unique_python _unique_np)valuesrrrQ/var/www/html/assistant/venv/lib/python3.9/site-packages/sklearn/utils/_encode.py_uniques rc Cs t|\}}d\}}|r0|r0||\}}}}n2|rD||\}}n|rX||\}}n ||}|jrt|drt||j|d}|d|d}|r||||k<|r| ||d||<|d|d}|f} |r| |f7} |r| |f7} t | dkr| dS| S)zHelper function to find unique values for numpy arrays that correctly accounts for nans. See `_unique` documentation for details.)NNxpNrr) r Z unique_allZunique_inverseZ unique_counts unique_valuessizer rnansumlen) rrrr_ZinversecountsuniquesZnan_idxretrrrr=s.     rc@s*eZdZUdZeed<eed<ddZdS) MissingValuesz'Data class for missing data informationrnonecCs*g}|jr|d|jr&|tj|S)z3Convert tuple to a list where None is always first.N)r%appendrnp)selfoutputrrrto_listjs   zMissingValues.to_listN)__name__ __module__ __qualname____doc__bool__annotations__r*rrrrr$ds r$cCsndd|D}|s"|tdddfSd|vrRt|dkrDtddd}q^tddd}n tddd}||}||fS)a.Extract missing values from `values`. Parameters ---------- values: set Set of values to extract missing from. Returns ------- output: set Set with missing values extracted. missing_values: MissingValues Object with missing value information. cSs h|]}|dust|r|qSNr .0valuerrr sz#_extract_missing..F)rr%NrT)r$r)rZmissing_values_setZoutput_missing_valuesr)rrr_extract_missingts  r6cs(eZdZdZfddZddZZS)_nandictz!Dictionary with support for nans.cs4t||D]\}}t|r||_q0qdSr1)super__init__itemsr nan_value)r(mappingkeyr4 __class__rrr9s  z_nandict.__init__cCs$t|drt|r|jSt|dS)Nr;)hasattrr r;KeyErrorr(r=rrr __missing__sz_nandict.__missing__)r+r,r-r.r9rC __classcell__rrr>rr7s r7csDt||\}}tddt|D|jfdd|Dt|dS)z,Map values based on its position in uniques.cSsi|]\}}||qSrr)r3ivalrrr z#_map_to_integer..csg|] }|qSrrr3vtablerr rHz#_map_to_integer..)r )r r7 enumerateZasarrayr )rr"rr rrKr_map_to_integersrOcCsz>t|}t|\}}t|}||tj||jd}Wn@ty~tddtdd|DD}td|Yn0|f}|r|t ||f7}|r|t ||f7}t |dkr|dS|S)Nrcss|] }|jVqdSr1)r-)r3trrr rHz!_unique_python..css|]}t|VqdSr1)typerIrrrrRrHzPEncoders require their input argument must be uniformly strings or numbers. Got rr) setr6sortedextendr*r'arrayr TypeErrorrO _get_countsr)rrr uniques_setZmissing_valuesr"typesr#rrrrs&    rT) check_unknownc Cst||\}}||jdsbz t||WSty^}ztdt|WYd}~qd}~00n2|rt||}|rtdt|t|||dSdS)aHelper function to encode values into [0, n_uniques - 1]. Uses pure python method for object dtype, and numpy method for all other dtypes. The numpy method has the limitation that the `uniques` need to be sorted. Importantly, this is not checked but assumed to already be the case. The calling method needs to ensure this for all non-object values. Parameters ---------- values : ndarray Values to encode. uniques : ndarray The unique values in `values`. If the dtype is not object, then `uniques` needs to be sorted. check_unknown : bool, default=True If True, check for values in `values` that are not in `unique` and raise an error. This is ignored for object dtype, and treated as True in this case. This parameter is useful for _BaseEncoder._transform() to avoid calling _check_unknown() twice. Returns ------- encoded : ndarray Encoded values numericz%y contains previously unseen labels: Nr) r isdtyperrOrA ValueErrorstr_check_unknownr)rr"r\rr ediffrrr_encodes * rdcst||\}}d}||jdst|}t|\}}t|t\|}|jo\j } |jojj } fdd|r|s| s| r|fdd|D}n|jt ||j d}t |}| r| d| r| t jn||} t| ||dd }|r*|jrt|||}n|jt ||j d}|||r|||} || r||jrr|rr||} d || <|| }t |}|r||fS|S) a Helper function to check for unknowns in values to be encoded. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : array Values to check for unknowns. known_values : array Known values. Must be unique. return_mask : bool, default=False If True, return a mask of the same shape as `values` indicating the valid values. Returns ------- diff : list The unique values present in `values` and not in `know_values`. valid_mask : boolean array Additionally returned if ``return_mask=True``. Nr]cs$|vp"jr|dup"jo"t|Sr1)r%rr )r4)missing_in_uniquesrZrris_validsz _check_unknown..is_validcsg|] }|qSrrr2)rfrrrM(rHz"_check_unknown..rPTZ assume_uniquer)r r^rrTr6rr%rWZonesrr/listr&r'rrrranyisnan)rZ known_valuesZ return_maskrr Z valid_maskZ values_setZmissing_in_valuesrcZ nan_in_diffZ none_in_diffrZ diff_is_nanis_nanr)rfrerZrrasJ          racs0eZdZdZfddZddZddZZS) _NaNCounterz$Counter with support for nan values.cst||dSr1)r8r9_generate_items)r(r:r>rrr9Osz_NaNCounter.__init__ccs<|D]2}t|s|Vqt|ds(d|_|jd7_qdS)z>Generate items without nans. Stores the nan counts separately. nan_countrrN)r r@rn)r(r:itemrrrrmRs z_NaNCounter._generate_itemscCs$t|drt|r|jSt|dS)Nrn)r@r rnrArBrrrrC\sz_NaNCounter.__missing__)r+r,r-r.r9rmrCrDrrr>rrlLs  rlc Cs|jjdvrrt|}tjt|tjd}t|D]<\}}tt ||||<Wdq01sb0Yq0|St |dd\}}tj ||dd}t |drt |drd|d<t |||} tj|tjd}|| ||<|S)zGet the count of each of the `uniques` in `values`. The counts will use the order passed in by `uniques`. For non-object dtypes, `uniques` is assumed to be sorted and `np.nan` is at the end. ZOUrPNT)rrgr)rkindrlr'ZzerosrZint64rNrrArisinrjZ searchsortedZ zeros_like) rr"counterr)rErorr!Zuniques_in_valuesZunique_valid_indicesrrrrYbs  , rY)FF)F) collectionsr contextlibrtypingrnumpyr'Z _array_apirrrr r Z_missingr rrr$r6dictr7rOrrdrarlrYrrrrs    ) '&+ V