a ã Àhã@sjddlmZddlmZddlmZddlZddlmZmZddl m Z ddl m Z Gdd „d ej ƒZdS) é)Ú annotations)ÚIterable)ÚAnyN)ÚTensorÚnn)ÚSentenceTransformer)ÚfullnamecsheZdZe ¡e ¡fdddddœ‡fdd„ Zdddd œd d „Zd ddd œdd„Zddœdd„Z ‡Z S)ÚCosineSimilarityLossrz nn.ModuleÚNone)ÚmodelÚloss_fctÚcos_score_transformationÚreturncs tƒ ¡||_||_||_dS)a– CosineSimilarityLoss expects that the InputExamples consists of two texts and a float label. It computes the vectors ``u = model(sentence_A)`` and ``v = model(sentence_B)`` and measures the cosine-similarity between the two. By default, it minimizes the following loss: ``||input_label - cos_score_transformation(cosine_sim(u,v))||_2``. Args: model: SentenceTransformer model loss_fct: Which pytorch loss function should be used to compare the ``cosine_similarity(u, v)`` with the input_label? By default, MSE is used: ``||input_label - cosine_sim(u, v)||_2`` cos_score_transformation: The cos_score_transformation function is applied on top of cosine_similarity. By default, the identify function is used (i.e. no change). References: - `Training Examples > Semantic Textual Similarity <../../../examples/sentence_transformer/training/sts/README.html>`_ Requirements: 1. Sentence pairs with corresponding similarity scores in range `[0, 1]` Inputs: +--------------------------------+------------------------+ | Texts | Labels | +================================+========================+ | (sentence_A, sentence_B) pairs | float similarity score | +--------------------------------+------------------------+ Relations: - :class:`CoSENTLoss` seems to produce a stronger training signal than CosineSimilarityLoss. In our experiments, CoSENTLoss is recommended. - :class:`AnglELoss` is :class:`CoSENTLoss` with ``pairwise_angle_sim`` as the metric, rather than ``pairwise_cos_sim``. It also produces a stronger training signal than CosineSimilarityLoss. Example: :: from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer, losses from datasets import Dataset model = SentenceTransformer("microsoft/mpnet-base") train_dataset = Dataset.from_dict({ "sentence1": ["It's nice weather outside today.", "He drove to work."], "sentence2": ["It's so sunny.", "She walked to the store."], "score": [1.0, 0.3], }) loss = losses.CosineSimilarityLoss(model) trainer = SentenceTransformerTrainer( model=model, train_dataset=train_dataset, loss=loss, ) trainer.train() N)ÚsuperÚ__init__r r r )Úselfr r r ©Ú __class__©úm/var/www/html/assistant/venv/lib/python3.9/site-packages/sentence_transformers/losses/CosineSimilarityLoss.pyrs; zCosineSimilarityLoss.__init__zIterable[dict[str, Tensor]]r)Úsentence_featuresÚlabelsrcs‡fdd„|Dƒ}ˆ ||¡S)Ncsg|]}ˆ |¡d‘qS)Zsentence_embedding)r )Ú.0Zsentence_feature©rrrÚ Oóz0CosineSimilarityLoss.forward..)Úcompute_loss_from_embeddings)rrrÚ embeddingsrrrÚforwardNszCosineSimilarityLoss.forwardz list[Tensor])rrrcCs0| t |d|d¡¡}| || ¡ d¡¡S)zð Compute the CosineSimilarity loss from embeddings. Args: embeddings: List of embeddings labels: Labels indicating the similarity scores of the pairs Returns: Loss value rééÿÿÿÿ)r ÚtorchZcosine_similarityr ÚfloatÚview)rrrÚoutputrrrrSs z1CosineSimilarityLoss.compute_loss_from_embeddingszdict[str, Any])rcCsdt|jƒiS)Nr )rr rrrrÚget_config_dictasz$CosineSimilarityLoss.get_config_dict) Ú__name__Ú __module__Ú __qualname__rZMSELossZIdentityrrrr%Ú __classcell__rrrrr s ü@r )Ú __future__rÚcollections.abcrÚtypingrr!rrZ)sentence_transformers.SentenceTransformerrZsentence_transformers.utilrÚModuler rrrrÚs