a ½ ÀhÍ ã@s"ddlmZddlmZddlmZddlmZmZm Z m Z m Z ddl m Z mZmZmZmZmZddlmZe e¡ZGd d „d eƒZGd d „d eƒZGd d„de ƒZGdd„deƒZGdd„de ƒZGdd„de ƒZGdd„deƒZGdd„de ƒZGdd„deƒZ Gdd„deƒZ!gd¢Z"dS)é)ÚCausalLMOutputWithPast)ÚUnpack)Úloggingé)ÚDeepseekV3DecoderLayerÚ DeepseekV3MLPÚ DeepseekV3MoEÚDeepseekV3PreTrainedModelÚDeepseekV3TopkRouter)ÚQwen3AttentionÚQwen3ForCausalLMÚ Qwen3ModelÚ Qwen3RMSNormÚQwen3RotaryEmbeddingÚTransformersKwargsé)Ú Dots1Configc@s eZdZdS)Ú Dots1RMSNormN©Ú__name__Ú __module__Ú __qualname__©rrúc/var/www/html/assistant/venv/lib/python3.9/site-packages/transformers/models/dots1/modular_dots1.pyr'src@s eZdZdS)ÚDots1RotaryEmbeddingNrrrrrr+src@s eZdZdS)ÚDots1AttentionNrrrrrr/src@s eZdZdS)ÚDots1MLPNrrrrrr3src@s eZdZdS)ÚDots1MoENrrrrrr7src@s eZdZdS)ÚDots1TopkRouterNrrrrrr;srcs$eZdZeedœ‡fdd„ Z‡ZS)ÚDots1DecoderLayer)ÚconfigÚ layer_idxcstƒ ||¡|j||_dS)N)ÚsuperÚ__init__Z layer_typesZattention_type)Úselfr r!©Ú __class__rrr#@szDots1DecoderLayer.__init__)rrrrÚintr#Ú __classcell__rrr%rr?src@s eZdZdS)ÚDots1PreTrainedModelNrrrrrr)Esr)c@s eZdZdS)Ú Dots1ModelNrrrrrr*Isr*cs(eZdZeeedœ‡fdd„ Z‡ZS)ÚDots1ForCausalLM)Ú super_kwargsÚreturnc stƒjfi|¤ŽS)a~ labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, Dots1ForCausalLM >>> model = Dots1ForCausalLM.from_pretrained("rednote-hilab/dots1.llm1.inst") >>> tokenizer = AutoTokenizer.from_pretrained("rednote-hilab/dots1.llm1.inst") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```)r"Úforward)r$r,r%rrr.NszDots1ForCausalLM.forward)rrrrrrr.r(rrr%rr+Msýr+)r)r*r+N)#Zmodeling_outputsrZprocessing_utilsrÚutilsrZ deepseek_v3.modeling_deepseek_v3rrrr r Zqwen3.modeling_qwen3r r r rrrZconfiguration_dots1rZ get_loggerrÚloggerrrrrrrrr)r*r+Ú__all__rrrrÚs"