a h0@s\ddlmZddlmZmZddlZddlmZm Z m Z m Z m Z ddl mZddlmZmZddlmZmZdd lmZmZdd lmZmZmZmZdd lmZdd lm Z m!Z!m"Z"m#Z#dd l$m%Z%ddl&m'Z'ddl(m)Z)m*Z*ddl+m,Z,m-Z-m.Z.m/Z/m0Z0ddl1m2Z2m3Z3ddZ4Gddde e eZ5Gddde5Z6ddZ7ddZ8Gddde5Z9dS))MutableMapping)IntegralRealN) BaseEstimatorClassifierMixinMetaEstimatorMixin _fit_contextclone)NotFittedError) check_scoringget_scorer_names) _CurveScorer!_threshold_scores_to_class_labels)_safe_indexingget_tags) HasMethodsInterval RealNotInt StrOptions)_get_response_values_binary)MetadataRouter MethodMapping_raise_for_paramsprocess_routing) available_if)type_of_target)Paralleldelayed)_check_method_params_estimator_has _num_samplescheck_is_fitted indexable)StratifiedShuffleSplitcheck_cvcCs0zt|jWnty*t|dYn0dS)N estimator_)r" estimatorr )r(r)m/var/www/html/assistant/venv/lib/python3.9/site-packages/sklearn/model_selection/_classification_threshold.py_check_is_fitted/s r+cseZdZUdZeddgeddggehdgdZeed<dd d d Z d d Z e ddddZ e ddZeedddZeedddZeedddZfddZZS)BaseThresholdClassifieraBase class for binary classifiers that set a non-default decision threshold. In this base class, we define the following interface: - the validation of common parameters in `fit`; - the different prediction methods that can be used with the classifier. .. versionadded:: 1.5 Parameters ---------- estimator : estimator instance The binary classifier, fitted or not, for which we want to optimize the decision threshold used during `predict`. response_method : {"auto", "decision_function", "predict_proba"}, default="auto" Methods by the classifier `estimator` corresponding to the decision function for which we want to find a threshold. It can be: * if `"auto"`, it will try to invoke, for each classifier, `"predict_proba"` or `"decision_function"` in that order. * otherwise, one of `"predict_proba"` or `"decision_function"`. If the method is not implemented by the classifier, it will raise an error. fit predict_probadecision_function>autor/r.r(response_method_parameter_constraintsr0r2cCs||_||_dSNr1)selfr(r2r)r)r*__init__Ysz BaseThresholdClassifier.__init__cCs|jdkrddg}n|j}|S)zDefine the response method.r0r.r/r4)r6r2r)r)r*_get_response_method]s  z,BaseThresholdClassifier._get_response_methodF)Zprefer_skip_nested_validationcKst||dt||\}}t|dd}|dkr`. .. versionadded:: 1.5 Parameters ---------- estimator : estimator instance The binary classifier, fitted or not, for which we want to optimize the decision threshold used during `predict`. threshold : {"auto"} or float, default="auto" The decision threshold to use when converting posterior probability estimates (i.e. output of `predict_proba`) or decision scores (i.e. output of `decision_function`) into a class label. When `"auto"`, the threshold is set to 0.5 if `predict_proba` is used as `response_method`, otherwise it is set to 0 (i.e. the default threshold for `decision_function`). pos_label : int, float, bool or str, default=None The label of the positive class. Used to process the output of the `response_method` method. When `pos_label=None`, if `y_true` is in `{-1, 1}` or `{0, 1}`, `pos_label` is set to 1, otherwise an error will be raised. response_method : {"auto", "decision_function", "predict_proba"}, default="auto" Methods by the classifier `estimator` corresponding to the decision function for which we want to find a threshold. It can be: * if `"auto"`, it will try to invoke `"predict_proba"` or `"decision_function"` in that order. * otherwise, one of `"predict_proba"` or `"decision_function"`. If the method is not implemented by the classifier, it will raise an error. Attributes ---------- estimator_ : estimator instance The fitted classifier used when predicting. classes_ : ndarray of shape (n_classes,) The class labels. n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. See Also -------- sklearn.model_selection.TunedThresholdClassifierCV : Classifier that post-tunes the decision threshold based on some metrics and using cross-validation. sklearn.calibration.CalibratedClassifierCV : Estimator that calibrates probabilities. Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.linear_model import LogisticRegression >>> from sklearn.metrics import confusion_matrix >>> from sklearn.model_selection import FixedThresholdClassifier, train_test_split >>> X, y = make_classification( ... n_samples=1_000, weights=[0.9, 0.1], class_sep=0.8, random_state=42 ... ) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, stratify=y, random_state=42 ... ) >>> classifier = LogisticRegression(random_state=0).fit(X_train, y_train) >>> print(confusion_matrix(y_test, classifier.predict(X_test))) [[217 7] [ 19 7]] >>> classifier_other_threshold = FixedThresholdClassifier( ... classifier, threshold=0.1, response_method="predict_proba" ... ).fit(X_train, y_train) >>> print(confusion_matrix(y_test, classifier_other_threshold.predict(X_test))) [[184 40] [ 6 20]] r0booleanN) threshold pos_labelr3)rWrXr2cs tj||d||_||_dSNr1)rGr7rXrW)r6r(rWrXr2rKr)r*r73sz!FixedThresholdClassifier.__init__cCsLt|dd}r|jSzt|j|jjWStyFtdtYn0dS)Nr'z+The underlying estimator is not fitted yet.)rDrCr"r(r AttributeError)r6r(r)r)r*rC?s   z!FixedThresholdClassifier.classes_cKs6t|dfi|}t|jj||fi|jj|_|S)r9r-)rr r(r-r')r6rAr:rB routed_paramsr)r)r*r?Ks zFixedThresholdClassifier._fitcCsht|t|d|j}t||||jdd\}}}|jdkrP|dkrJdnd}n|j}t|||j|jS)OPredict the target of new samples. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) The samples, as accepted by `estimator.predict`. Returns ------- class_labels : ndarray of shape (n_samples,) The predicted class. r'T)rXZreturn_response_method_usedr0r.g?) r+rDr(rr8rXrWrrC)r6rAr(y_score_Zresponse_method_usedZdecision_thresholdr)r)r*predictcs    z FixedThresholdClassifier.predictcCs*t|jjdj|jtjdddd}|S)KGet metadata routing of this object. Please check :ref:`User Guide ` on how the routing mechanism works. Returns ------- routing : MetadataRouter A :class:`~sklearn.utils.metadata_routing.MetadataRouter` encapsulating routing information. ownerr-ZcalleeZcallerr(method_mapping)rrLrMaddr(rr6Zrouterr)r)r*get_metadata_routings z-FixedThresholdClassifier.get_metadata_routing)rMrNrOrPr,r3rrstrrQrRr7rSrCr?r`rirTr)r)rKr*rUs W     "rUcCs|durft||t||}} t||t||} } t|||d} t|||d} |j|| fi| n|||} } } ||| | fi| S)aFit a classifier and compute the scores for different decision thresholds. Parameters ---------- classifier : estimator instance The classifier to fit and use for scoring. If `classifier` is already fitted, it will be used as is. X : {array-like, sparse matrix} of shape (n_samples, n_features) The entire dataset. y : array-like of shape (n_samples,) The entire target vector. fit_params : dict Parameters to pass to the `fit` method of the underlying classifier. train_idx : ndarray of shape (n_train_samples,) or None The indices of the training set. If `None`, `classifier` is expected to be already fitted. val_idx : ndarray of shape (n_val_samples,) The indices of the validation set used to score `classifier`. If `train_idx`, the entire set will be used. curve_scorer : scorer instance The scorer taking `classifier` and the validation set as input and outputting decision thresholds and scores as a curve. Note that this is different from the usual scorer that output a single score value: * when `score_method` is one of the four constraint metrics, the curve scorer will output a curve of two scores parametrized by the decision threshold, e.g. TPR/TNR or precision/recall curves for each threshold; * otherwise, the curve scorer will output a single score value for each threshold. score_params : dict Parameters to pass to the `score` method of the underlying scorer. Returns ------- scores : ndarray of shape (thresholds,) or tuple of such arrays The scores computed for each decision threshold. When TPR/TNR or precision/ recall are computed, `scores` is a tuple of two arrays. potential_thresholds : ndarray of shape (thresholds,) The decision thresholds used to compute the scores. They are returned in ascending order. Nindices)rrr-) classifierrAr: fit_params train_idxval_idx curve_scorer score_paramsX_trainZX_valy_trainZy_valfit_params_trainZscore_params_valr)r)r*_fit_and_score_over_thresholdss=rvcs"tjfddt||DddS)alCompute the mean interpolated score across folds by defining common thresholds. Parameters ---------- target_thresholds : ndarray of shape (thresholds,) The thresholds to use to compute the mean score. cv_thresholds : ndarray of shape (n_folds, thresholds_fold) The thresholds used to compute the scores for each fold. cv_scores : ndarray of shape (n_folds, thresholds_fold) The scores computed for each threshold for each fold. Returns ------- mean_score : ndarray of shape (thresholds,) The mean score across all folds for each target threshold. csg|]\}}t||qSr))npinterp).0split_thresholdsZ split_scoretarget_thresholdsr)r* sz,_mean_interpolated_score..r)Zaxis)rwmeanzip)r| cv_thresholds cv_scoresr)r{r*_mean_interpolated_scores  rc seZdZUdZiejeeee e ge e dddddgdedhe e d d d dgd ge dgd gd gdZeed<dddddddddfdd ZddZddZddZddZZS) TunedThresholdClassifierCVa8Classifier that post-tunes the decision threshold using cross-validation. This estimator post-tunes the decision threshold (cut-off point) that is used for converting posterior probability estimates (i.e. output of `predict_proba`) or decision scores (i.e. output of `decision_function`) into a class label. The tuning is done by optimizing a binary metric, potentially constrained by a another metric. Read more in the :ref:`User Guide `. .. versionadded:: 1.5 Parameters ---------- estimator : estimator instance The classifier, fitted or not, for which we want to optimize the decision threshold used during `predict`. scoring : str or callable, default="balanced_accuracy" The objective metric to be optimized. Can be one of: * a string associated to a scoring function for binary classification (see :ref:`scoring_parameter`); * a scorer callable object created with :func:`~sklearn.metrics.make_scorer`; response_method : {"auto", "decision_function", "predict_proba"}, default="auto" Methods by the classifier `estimator` corresponding to the decision function for which we want to find a threshold. It can be: * if `"auto"`, it will try to invoke, for each classifier, `"predict_proba"` or `"decision_function"` in that order. * otherwise, one of `"predict_proba"` or `"decision_function"`. If the method is not implemented by the classifier, it will raise an error. thresholds : int or array-like, default=100 The number of decision threshold to use when discretizing the output of the classifier `method`. Pass an array-like to manually specify the thresholds to use. cv : int, float, cross-validation generator, iterable or "prefit", default=None Determines the cross-validation splitting strategy to train classifier. Possible inputs for cv are: * `None`, to use the default 5-fold stratified K-fold cross validation; * An integer number, to specify the number of folds in a stratified k-fold; * A float number, to specify a single shuffle split. The floating number should be in (0, 1) and represent the size of the validation set; * An object to be used as a cross-validation generator; * An iterable yielding train, test splits; * `"prefit"`, to bypass the cross-validation. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. warning:: Using `cv="prefit"` and passing the same dataset for fitting `estimator` and tuning the cut-off point is subject to undesired overfitting. You can refer to :ref:`TunedThresholdClassifierCV_no_cv` for an example. This option should only be used when the set used to fit `estimator` is different from the one used to tune the cut-off point (by calling :meth:`TunedThresholdClassifierCV.fit`). refit : bool, default=True Whether or not to refit the classifier on the entire training set once the decision threshold has been found. Note that forcing `refit=False` on cross-validation having more than a single split will raise an error. Similarly, `refit=True` in conjunction with `cv="prefit"` will raise an error. n_jobs : int, default=None The number of jobs to run in parallel. When `cv` represents a cross-validation strategy, the fitting and scoring on each data split is done in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls the randomness of cross-validation when `cv` is a float. See :term:`Glossary `. store_cv_results : bool, default=False Whether to store all scores and thresholds computed during the cross-validation process. Attributes ---------- estimator_ : estimator instance The fitted classifier used when predicting. best_threshold_ : float The new decision threshold. best_score_ : float or None The optimal score of the objective metric, evaluated at `best_threshold_`. cv_results_ : dict or None A dictionary containing the scores and thresholds computed during the cross-validation process. Only exist if `store_cv_results=True`. The keys are `"thresholds"` and `"scores"`. classes_ : ndarray of shape (n_classes,) The class labels. n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. See Also -------- sklearn.model_selection.FixedThresholdClassifier : Classifier that uses a constant threshold. sklearn.calibration.CalibratedClassifierCV : Estimator that calibrates probabilities. Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.ensemble import RandomForestClassifier >>> from sklearn.metrics import classification_report >>> from sklearn.model_selection import TunedThresholdClassifierCV, train_test_split >>> X, y = make_classification( ... n_samples=1_000, weights=[0.9, 0.1], class_sep=0.8, random_state=42 ... ) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, stratify=y, random_state=42 ... ) >>> classifier = RandomForestClassifier(random_state=0).fit(X_train, y_train) >>> print(classification_report(y_test, classifier.predict(X_test))) precision recall f1-score support 0 0.94 0.99 0.96 224 1 0.80 0.46 0.59 26 accuracy 0.93 250 macro avg 0.87 0.72 0.77 250 weighted avg 0.93 0.93 0.92 250 >>> classifier_tuned = TunedThresholdClassifierCV( ... classifier, scoring="balanced_accuracy" ... ).fit(X_train, y_train) >>> print( ... f"Cut-off point found at {classifier_tuned.best_threshold_:.3f}" ... ) Cut-off point found at 0.342 >>> print(classification_report(y_test, classifier_tuned.predict(X_test))) precision recall f1-score support 0 0.96 0.95 0.96 224 1 0.61 0.65 0.63 26 accuracy 0.92 250 macro avg 0.78 0.80 0.79 250 weighted avg 0.92 0.92 0.92 250 r$Nleft)closedz array-likeZ cv_objectprefitr]g?ZneitherrV random_state)scoring thresholdscvrefitn_jobsrstore_cv_resultsr3Zbalanced_accuracyr0dTF)rr2rrrrrrc s>tj||d||_||_||_||_||_||_| |_dSrY) rGr7rrrrrrr) r6r(rr2rrrrrrrKr)r*r7s z#TunedThresholdClassifierCV.__init__c stjtr:djkr"dkr:nntdjjdnjdkrjdurVtdztjdWn.t y}zt d|WYd }~n d }~00jn.t jdd jd ur̈ dkrtd t d fi| _dkrj_jd ttfg}ntj_tjjfijj}jr`jj}}}nFtjfijj\} } t| }t| }tjj| d}jj||fi|ttjdfdd|D\} } tdd| Drtdtdd| D} tdd| D}tjt rNt!j"| |jd}n t!#j}t$|| | }|%}||_&||_'j(r||d_)S)aFit the classifier and post-tune the decision threshold. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) Target values. **params : dict Parameters to pass to the `fit` method of the underlying classifier and to the `scoring` scorer. Returns ------- self : object Returns an instance of self. rr$)Zn_splitsZ test_sizerrTz'When cv='prefit', refit cannot be True.rCz-When cv='prefit', `estimator` must be fitted.N)r:rmFz1When cv has several folds, refit cannot be False.r-rk)rc 3sH|]@\}}ttdkr tnjj||jjjdVqdS)r)rnrorprqrrN)rrvr r(r- _curve_scorerscorerscore)ryrorprArmrr[r6r:r)r* s z2TunedThresholdClassifierCV._fit..css"|]}t|d|dVqdS)rN)rwisclose)rythr)r)r*rzrThe provided estimator makes constant predictions. Therefore, it is impossible to optimize the decision threshold.css|]}|VqdSr5)minryrzr)r)r*r%scss|]}|VqdSr5)maxrr)r)r*r(s)num)rZscores)* isinstancerrr%rrr>r"r(r r&Z get_n_splitsr_get_curve_scorerrr'ranger!r splitsplitterr-nextrrrrranyrrrrrwZlinspaceZasarrayrZargmaxZ best_score_best_threshold_rZ cv_results_)r6rAr:rBexcZsplitsrsrtruror_rrZ min_thresholdZ max_thresholdZdecision_thresholdsZobjective_scoresZbest_idxr)rr*r?s&               zTunedThresholdClassifierCV._fitcCs@t|d|j}t|j|||d\}}t||j|j|S)r\r')rX) r"rZ_get_pos_labelrr'r8rrrC)r6rArXr^r_r)r)r*r`@s    z"TunedThresholdClassifierCV.predictcCs\t|jjdj|jtjddddj|jtjddddj|tjdddd}|S) rarbr-rdrer)rrfr)rrf)rrLrMrgr(rrrrhr)r)r*riZs  z/TunedThresholdClassifierCV.get_metadata_routingcCs(t|j|jd}t|||j}|S)z8Get the curve scorer based on the objective metric used.)r)r r(rrZ from_scorerr8r)r6rrqr)r)r*rws  z,TunedThresholdClassifierCV._get_curve_scorer)rMrNrOrPr,r3rsetr callablerrrrrQrRr7r?r`rirrTr)r)rKr*rs@ # wr):collections.abcrnumbersrrnumpyrwbaserrrr r exceptionsr Zmetricsr r Zmetrics._scorerrrutilsrrZutils._param_validationrrrrZutils._responserZutils.metadata_routingrrrrZutils.metaestimatorsrZutils.multiclassrZutils.parallelrrZutils.validationrr r!r"r#_splitr%r&r+r,rUrvrrr)r)r)r*s.      DI