a ½ Àhl ã@sbdZddlmZmZddlmZddlmZmZm Z m Z m Z ddl m Z Gdd„deƒZdgZd S) z Processor class for Bros. é)ÚOptionalÚUnioné)ÚProcessorMixin)Ú BatchEncodingÚPaddingStrategyÚPreTokenizedInputÚ TextInputÚTruncationStrategy)Ú TensorTypecsžeZdZdZdgZdZd ‡fdd„ Zdeee e ee e fe ee e e fee e efeeeeeee ee e e e e e eee efed œd d „Z‡ZS)Ú BrosProcessora‘ Constructs a Bros processor which wraps a BERT tokenizer. [`BrosProcessor`] offers all the functionalities of [`BertTokenizerFast`]. See the docstring of [`~BrosProcessor.__call__`] and [`~BrosProcessor.decode`] for more information. Args: tokenizer (`BertTokenizerFast`, *optional*): An instance of ['BertTokenizerFast`]. The tokenizer is a required input. Ú tokenizer)Z BertTokenizerZBertTokenizerFastNc s |durtdƒ‚tƒ |¡dS)Nz"You need to specify a `tokenizer`.)Ú ValueErrorÚsuperÚ__init__)Úselfr Úkwargs©Ú __class__©úd/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/models/bros/processing_bros.pyr)szBrosProcessor.__init__TFr)ÚtextÚadd_special_tokensÚpaddingÚ truncationÚ max_lengthÚstrideÚpad_to_multiple_ofÚreturn_token_type_idsÚreturn_attention_maskÚreturn_overflowing_tokensÚreturn_special_tokens_maskÚreturn_offsets_mappingÚ return_lengthÚverboseÚreturn_tensorsÚreturncKs4|jf||||||||| | | | | ||dœ|¤Ž}|S)z¶ This method uses [`BertTokenizerFast.__call__`] to prepare text for the model. Please refer to the docstring of the above two methods for more information. )rrrrrrrrrr r!r"r#r$r%)r )rrrrrrrrrrr r!r"r#r$r%rÚencodingrrrÚ__call__/s(ñðzBrosProcessor.__call__)N)NTFNNrNNNFFFFTN)Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú attributesZtokenizer_classrrr rÚlistÚboolÚstrrr rÚintr rr(Ú __classcell__rrrrr sJ ð  îr N)r,ÚtypingrrZprocessing_utilsrZtokenization_utils_baserrrr r Úutilsr r Ú__all__rrrrÚs   C