a h=@sVdZddlmZddlZddlZddlZddlmZGdddZGdd d eZ dS) a@ This file contains deprecated code that can only be used with the old `model.fit`-style Sentence Transformers v2.X training. It exists for backwards compatibility with the `model.old_fit` method, but will be removed in a future version. Nowadays, with Sentence Transformers v3+, it is recommended to use the `SentenceTransformerTrainer` class to train models. See https://www.sbert.net/docs/sentence_transformer/training_overview.html for more information. Instead, you should create a `datasets` `Dataset` for training: https://huggingface.co/docs/datasets/create_dataset ) annotationsN) InputExamplec@s6eZdZdZddddejdddfdd Zd d d Zd S) STSDataReadera1Reads in the STS dataset. Each line contains two sentences (s1_col_idx, s2_col_idx) and one label (score_col_idx) Default values expects a tab separated file with the first & second column the sentence pair and third column the score (0...1). Default config normalizes scores from 0...5 to 0...1 rr Tc Cs:||_||_||_||_||_||_||_||_| |_dS)N) dataset_folder score_col_idx s1_col_idx s2_col_idx delimiterquotingnormalize_scores min_score max_score selfr r r r r rrrrrg/var/www/html/assistant/venv/lib/python3.9/site-packages/sentence_transformers/readers/STSDataReader.py__init__s zSTSDataReader.__init__c Cstj|j|}|dr*tj|dddn t|dd}tj||j |j d}g}t |D]~\}}t ||j } |jr| |j|j|j} ||j} ||j} |t|t|| | g| d|dkrXt||krXqqXWd n1s0Y|S) zJfilename specified which data split to use (train.csv, dev.csv, test.csv).z.gzrtutf8)encodingzutf-8)r r)ZguidZtextslabelrN)ospathjoinr endswithgzipopencsvreaderr r enumeratefloatr rrrr r appendrstrlen) rfilenameZ max_examplesfilepathZfIndataZexamplesidrowZscores1s2rrr get_examples0s$    $zSTSDataReader.get_examplesN)r)__name__ __module__ __qualname____doc__r! QUOTE_NONErr/rrrrrs rcs4eZdZdZddddejdddffdd ZZS) STSBenchmarkDataReaderzReader especially for the STS benchmark dataset. There, the sentences are in column 5 and 6, the score is in column 4. Scores are normalized from 0...5 to 0...1 rrTrc s"tj||||||||| d dS)N) r r r r r rrrr)superrr __class__rrrNs zSTSBenchmarkDataReader.__init__)r0r1r2r3r!r4r __classcell__rrr9rr5Isr5) r3 __future__rr!rrrrr5rrrrs  5