a h @sdZddlZddlmZmZddlmZmZddlZ ddl m Z ddl m Z mZmZmZmZmZddlmZdd lmZmZdd lmZmZdd lmZdd lmZmZdd l m!Z!m"Z"m#Z#gdZ$eedkrddl m%Z&n ddl m&Z&ddZ'd0ddZ(ddZ)d1ddZ*dd Z+d!d"Z,d#d$Z-Gd%d&d&eeeee ed'Z.Gd(d)d)e.Z/Gd*d+d+e.Z0Gd,d-d-e.Z1Gd.d/d/eee Z2dS)2zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). N)ABCMetaabstractmethod)IntegralReal)svd) BaseEstimatorClassNamePrefixFeaturesOutMixinMultiOutputMixinRegressorMixinTransformerMixin _fit_context)ConvergenceWarning) check_arraycheck_consistent_length)Interval StrOptions)svd_flip) parse_version sp_version) FLOAT_DTYPEScheck_is_fitted validate_data) PLSCanonical PLSRegressionPLSSVDz1.7)pinv)pinv2c Cst|ddd\}}}|jj}ddd}t|||t|j}t||k}|ddd|f}||d|}t t t ||d|S)NF) full_matrices check_finiteg@@g.A)fd) rdtypecharlowernpmaxfinfoepssumZ transpose conjugatedot)ausZvhtfactorZcondZrankr1\/var/www/html/assistant/venv/lib/python3.9/site-packages/sklearn/cross_decomposition/_pls.py _pinv2_old)s  r3Aư>Fc st|jjztfdd|jD}Wn.tyX}ztd|WYd}~n d}~00d}|dkrxt|t|} } t|D]} |dkrt | |} nt |j|t ||} | t t | | } t || } |dkrt | | }nt |j| t | j| }|r,|t t ||}t ||t ||}| |}t |||ksr|j ddkrxq~| }q| d}||krt dt| ||fS) a?Return the first left and right singular vectors of X'Y. Provides an alternative to the svd(X'Y) and uses the power method instead. With norm_y_weights to True and in mode A, this corresponds to the algorithm section 11.3 of the Wegelin's review, except this starts at the "update saliences" part. c3s&|]}tt|kr|VqdSN)r%anyabs).0colr(r1r2 Hz;_get_first_singular_vectors_power_method..y residual is constantNdBz$Maximum number of iterations reached)r%r'r"r(nextT StopIterationr3ranger+sqrtshapewarningswarnr)XYmodemax_itertolnorm_y_weightsZy_scoreeZ x_weights_oldZX_pinvZY_pinvi x_weightsZx_score y_weightsZx_weights_diffZn_iterr1r<r2(_get_first_singular_vectors_power_method;s8    "  rUcCs@t|j|}t|dd\}}}|dddf|dddffS)zbReturn the first left and right singular vectors of X'Y. Here the whole SVD is computed. FrNr)r%r+rDr)rKrLCU_Vtr1r1r2_get_first_singular_vectors_svdvsr[TcCs|jdd}||8}|jdd}||8}|rr|jddd}d||dk<||}|jddd}d||dk<||}n t|jd}t|jd}||||||fS)z{Center X, Y and scale if the scale parameter==True Returns ------- X, Y, x_mean, y_mean, x_std, y_std raxisrB)r]Zddofg?)meanZstdr%ZonesrH)rKrLscaleZx_meanZy_meanZx_stdZy_stdr1r1r2_center_scale_xys     racCs2tt|}t||}||9}||9}dS)z7Same as svd_flip but works on 1d arrays, and is inplaceN)r%Zargmaxr9sign)r-vZbiggest_abs_val_idxrbr1r1r2 _svd_flip_1dsrdcCs,|dur(tdt|dur$td|S|S)NzE`Y` is deprecated in 1.5 and will be removed in 1.7. Use `y` instead.z?Cannot use both `y` and `Y`. Use only `y` as `Y` is deprecated.)rIrJ FutureWarning ValueErroryrLr1r1r2_deprecate_Y_when_optionalsricCs"|dur|durtdt||S)Nzy is required.)rfrirgr1r1r2_deprecate_Y_when_requiredsrjc seZdZUdZeeddddgdgeddhged d hged d hgeeddddgeed dddgdgdZe e d<e d$ddd d ddddddZ e ddd%ddZd&ddZd'ddZd(ddZd)d d!Zfd"d#ZZS)*_PLSaPartial Least Squares (PLS) This class implements the generic PLS algorithm. Main ref: Wegelin, a survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf rBNleftclosedboolean regression canonicalr4rArnipalsr n_componentsr`deflation_moderM algorithmrNrOcopy_parameter_constraintsrTr5r6)r`rurMrvrNrOrwc Cs4||_||_||_||_||_||_||_||_dSr7)rtrurMr`rvrNrOrw) selfrtr`rurMrvrNrOrwr1r1r2__init__s z _PLS.__init__Zprefer_skip_nested_validationc Cs*t||}t||t||tjd|jdd}t|dtjd|jdd}|jdkrbd|_| dd}nd|_|j d }|j d}|j d}|j }|j d krt ||n t |||}||krtd |d |d |j dk|_|j} t|||j\} } |_|_|_|_t||f|_t||f|_t||f|_t||f|_t||f|_t||f|_g|_t| jj } t!|D]} |j"dkrBtj#t$| d| kd d}d| dd|f<z$t%| | |j&|j'|j(| d\}}}WnVt)y2}z`. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, n_features]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in :term:`fit` before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_scores_ : ndarray of shape (n_samples, n_components) The transformed training samples. y_scores_ : ndarray of shape (n_samples, n_components) The transformed training targets. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_target, n_features) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, y) PLSRegression() >>> Y_pred = pls2.predict(X) For a comparison between PLS Regression and :class:`~sklearn.decomposition.PCA`, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_pcr_vs_pls.py`. rxrurMrvrTr5r6r`rNrOrwc s tj||ddd|||ddS)Nrpr4rrrsrrzryrtr`rNrOrwrr1r2rzszPLSRegression.__init__Ncs,t||}t|||j|_|j|_|S)r|)rjrrrZ x_scores_rZ y_scores_)ryrKrhrLrr1r2rs  zPLSRegression.fit)r)NN) rrrrrkrxrrparampoprzrrr1r1rr2r4s g rcsXeZdZUdZiejZeed<dD]Ze eq$d dddddd fd d Z Z S) ra^ Partial Least Squares transformer and regressor. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. algorithm : {'nipals', 'svd'}, default='nipals' The algorithm used to estimate the first singular vectors of the cross-covariance matrix. 'nipals' uses the power method while 'svd' will compute the whole SVD. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. Empty if `algorithm='svd'`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- CCA : Canonical Correlation Analysis. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, y) PLSCanonical() >>> X_c, y_c = plsca.transform(X, y) rx)rurMrTrrr5r6)r`rvrNrOrwc s tj||dd||||ddS)Nrqr4rsr)ryrtr`rvrNrOrwrr1r2rzEs zPLSCanonical.__init__)r rrrrrkrxrrrrrzrr1r1rr2rs b rcsVeZdZUdZiejZeed<dD]Ze eq$d dddddfd d Z Z S) CCAa Canonical Correlation Analysis, also known as "Mode B" PLS. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import CCA >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [3.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> cca = CCA(n_components=1) >>> cca.fit(X, y) CCA(n_components=1) >>> X_c, Y_c = cca.transform(X, y) rxrrTr5r6rc s tj||ddd|||ddS)NrqrArrrsrrrr1r2rzsz CCA.__init__)rrr1r1rr2r[s Z rc@sreZdZUdZeeddddgdgdgdZeed<dd d d d d Z e d ddddZ dddZ dddZ dS)raPartial Least Square SVD. This transformer simply performs a SVD on the cross-covariance matrix `X'Y`. It is able to project both the training data `X` and the targets `Y`. The training data `X` is projected on the left singular vectors, while the targets are projected on the right singular vectors. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 The number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. y_weights_ : ndarray of (n_targets, n_components) The right singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. CCA : Canonical Correlation Analysis. Examples -------- >>> import numpy as np >>> from sklearn.cross_decomposition import PLSSVD >>> X = np.array([[0., 0., 1.], ... [1., 0., 0.], ... [2., 2., 2.], ... [2., 5., 4.]]) >>> y = np.array([[0.1, -0.2], ... [0.9, 1.1], ... [6.2, 5.9], ... [11.9, 12.3]]) >>> pls = PLSSVD(n_components=2).fit(X, y) >>> X_c, y_c = pls.transform(X, y) >>> X_c.shape, y_c.shape ((4, 2), (4, 2)) rBNrlrmrortr`rwrxrT)r`rwcCs||_||_||_dSr7r)ryrtr`rwr1r1r2rzszPLSSVD.__init__r{c Cs0t||}t||t||tjd|jdd}t|dtjd|jdd}|jdkrZ|dd}|j }t |j d |j d|j d}||krt d |d |d t |||j\}}|_|_|_|_t|j|}t|dd \}}} |ddd|f}| d|} t|| \}} | j} ||_| |_|jj d|_|S)aFit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. Y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. .. deprecated:: 1.5 `Y` is deprecated in 1.5 and will be removed in 1.7. Use `y` instead. Returns ------- self : object Fitted estimator. Trr}rhFrrBrrrrrrVN)rjrrr%rrwrrrrtrrHrfrar`rrrrr+rDrrrrr) ryrKrhrLrtrrWrXr.rZVr1r1r2rsR     z PLSSVD.fitcCst||}t|t||tjdd}||j|j}t||j}|durt |ddtjd}|j dkrr| dd}||j |j }t||j}||fS|S)a Apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to be transformed. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. .. deprecated:: 1.5 `Y` is deprecated in 1.5 and will be removed in 1.7. Use `y` instead. Returns ------- x_scores : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. F)r"rNrh)rrr"rBr)rirrr%rrrr+rrrrrrr)ryrKrhrLZXrryrrr1r1r2r`s   zPLSSVD.transformcCs|||||S)aLearn and apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- out : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. rrr1r1r2rszPLSSVD.fit_transform)r)NN)NN)N)rrrrrrrxrrrzr rrrr1r1r1r2rs D G (r)r4r5r6F)T)3rrIabcrrnumbersrrnumpyr%Z scipy.linalgrbaserr r r r r exceptionsrutilsrrZutils._param_validationrrZ utils.extmathrZ utils.fixesrrZutils.validationrrr__all__rrr3rUr[rardrirjrkrrrrr1r1r1r2sP       ;   |#n