a ã ÀhÁã@sZddlmZddlZddlmZmZerHddlZddlm Z ddl m Z Gdd„dƒZ dS)é)Ú annotationsN)Ú TYPE_CHECKINGÚAny)ÚTensor)ÚSentenceTransformerc@sŒeZdZdZdd„Zd"ddddd d œd d „Zd dd dœdd„Zd#ddddddœdd„Zeddœdd„ƒZ ddœdd„Z ddddœd d!„Z dS)$ÚSentenceEvaluatoraÀ Base class for all evaluators. Notably, this class introduces the ``greater_is_better`` and ``primary_metric`` attributes. The former is a boolean indicating whether a higher evaluation score is better, which is used for choosing the best checkpoint if ``load_best_model_at_end`` is set to ``True`` in the training arguments. The latter is a string indicating the primary metric for the evaluator. This has to be defined whenever the evaluator returns a dictionary of metrics, and the primary metric is the key pointing to the primary metric, i.e. the one that is used for model selection and/or logging. Extend this class and implement __call__ for custom evaluators. cCsd|_d|_dS)NT)Zgreater_is_betterÚprimary_metric©Úself©r ún/var/www/html/assistant/venv/lib/python3.9/site-packages/sentence_transformers/evaluation/SentenceEvaluator.pyÚ__init__szSentenceEvaluator.__init__Néÿÿÿÿrz str | NoneÚintzfloat | dict[str, float])ÚmodelÚ output_pathÚepochÚstepsÚreturncCsdS)aÏ This is called during training to evaluate the model. It returns a score for the evaluation with a higher score indicating a better result. Args: model: the model to evaluate output_path: path where predictions and metrics are written to epoch: the epoch where the evaluation takes place. This is used for the file prefixes. If this is -1, then we assume evaluation on test data. steps: the steps in the current epoch at time of the evaluation. This is used for the file prefixes. If this is -1, then we assume evaluation at the end of the epoch. Returns: Either a score for the evaluation with a higher score indicating a better result, or a dictionary with scores. If the latter is chosen, then `evaluator.primary_metric` must be defined Nr )r rrrrr r r Ú__call__szSentenceEvaluator.__call__zdict[str, float]Ústr)ÚmetricsÚnamercspdddœdd„‰ˆs*‡fdd„| ¡DƒS‡‡fdd„| ¡Dƒ}t|dƒrl|j ˆd ¡slˆd |j|_|S) Nr)ÚvaluercSs&z t|ƒWSty |YS0dS©N)ÚfloatÚ ValueError)rr r r Úmaybe_to_float:s  z@SentenceEvaluator.prefix_name_to_metrics..maybe_to_floatcsi|]\}}|ˆ|ƒ“qSr r ©Ú.0Úkeyr)rr r Ú Aóz.cs"i|]\}}ˆd|ˆ|ƒ“qS)Ú_r r©rrr r r!Br"rr#)ÚitemsÚhasattrrÚ startswith)r rrr r$r Úprefix_name_to_metrics9sz(SentenceEvaluator.prefix_name_to_metricsrzdict[str, Any]ÚNone)rrrÚsteprcCs|j ||||¡dSr)Zmodel_card_dataZset_evaluation_metrics)r rrrr*r r r Ú store_metrics_in_model_card_dataGsz2SentenceEvaluator.store_metrics_in_model_card_data)rcCs^|jj}| d¡r"d|dd…}z| d¡}|d|…}WntyNYn0t dd|¡S)a Returns a human-readable description of the evaluator: BinaryClassificationEvaluator -> Binary Classification 1. Replace "CE" prefix with "CrossEncoder" 2. Remove "Evaluator" from the class name 3. Add a space before every capital letter ZCEZ CrossEncoderéNZ Evaluatorz([a-z])([A-Z])z \g<1> \g<2>)Ú __class__Ú__name__r'ÚindexÚ IndexErrorÚreÚsub)r Ú class_namer/r r r Ú descriptionLs    zSentenceEvaluator.descriptioncCsiS)z{ Return a dictionary with all meaningful configuration values of the evaluator to store in the model card. r r r r r Úget_config_dictbsz!SentenceEvaluator.get_config_dictzstr | list[str] | np.ndarrayzPlist[Tensor] | np.ndarray | Tensor | dict[str, Tensor] | list[dict[str, Tensor]])rÚ sentencesrcKs|j|fi|¤ŽS)ae Call the encoder method of the model pass Args: model (SentenceTransformer): Model we are evaluating sentences (str | list[str] | np.ndarray): Text that we are embedding Returns: list[Tensor] | np.ndarray | Tensor | dict[str, Tensor] | list[dict[str, Tensor]]: The associated embedding )Úencode)r rr6Úkwargsr r r Ú embed_inputshszSentenceEvaluator.embed_inputs)Nrr)rr) r.Ú __module__Ú __qualname__Ú__doc__r rr(r+Úpropertyr4r5r9r r r r r s ÿÿr) Ú __future__rr1ÚtypingrrÚnumpyÚnpZtorchrZ)sentence_transformers.SentenceTransformerrrr r r r Ús