a h-@s,dZddlZGdddZGdddZdS)aN Helper classes for working with low precision floating point types that align with the opencompute (OCP) microscaling (MX) specification. * MXFP4Tensor: 4-bit E2M1 floating point data * MXScaleTensor: 8-bit E8M0 floating point data Reference: https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf Nc@s>eZdZdddZddZddZdd Zd d Zd d ZdS) MXFP4TensorNcCsd||_|dur8t|tjs"Jd|j|_|||_n(|durXt|trN|n|f|_ntddS)at Tensor class for working with four bit E2M1 floating point data as defined by the opencompute microscaling specification. Parameters: - data: A torch tensor of float32 numbers to convert to fp4e2m1 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. N%Parameter data must be a torch tensor.Either parameter data or size must be provided device isinstancetorchZTensor _from_floatdatatuplesize ValueErrorselfr r rrM/var/www/html/assistant/venv/lib/python3.9/site-packages/triton/tools/mxfp.py__init__s zMXFP4Tensor.__init__cCsptjdd|jtj|jd}tjdd|jtj|jd}tjdd|jtj|jd}|d>|d>B|Btj|_|S)Nrr dtyper)rrandintr uint8rtyper )rSEMrrrrandom#s zMXFP4Tensor.randomcCs |tjksJd|j}|d?d@|}|d?d@|}|d@|}t|}|dk|dk@}|}|r||} ||} ||} td| } t| dk| | d} t| dk| dd| d}| td| |}|||<|||dk@d9<|tjS) z Convert fp4e2m1 data to float32. Returns: - A torch tensor of type dtype representing the fp4e2m1 data. zCCurrently only float32 is supported for fp4e2m1 to float conversionrrr??r)rfloat32r rZ zeros_likeanypowwhere)rrr rrrvalueis_zeroZ non_zero_maskZS_nzZE_nzZM_nzsignexponentZmantissaZvalue_nzrrrto+s&  zMXFP4Tensor.tocCsVt|tj}t|}|dk}t|t|B}tjgdtj|jd}tjddgtj|jd}g}g} g} |D]} | dkrd} |D]6} | d}|d| }| || | | | qqx| d} |D]:} d| d}|d| }| || | | | qqxtj|tj |jd}tj| tj|jd} tj| tj|jd} | d}|j d}|d}| }||| d<t||d}tj|dd d \}}||k}|dkr| d|d}|dktj}||d }tj|dd }| |}| |}| |j } | |j } d| |<d| |<|d >| d>B| BtjS)a5 Convert float32 numbers to mxf4 e2m1 format. * No encodings are reserved for Inf or NaN in mxf4. * Conversion from float supports roundTiesToEven rounding mode. * If a value exceeds the mxf4 representable range after rounding, clamps to the maximum mxf4 magnitude, preserving the sign. * If a value has magnitude less than the minimum subnormal magnitude in mxf4 after rounding, converts to zero. Parameters: - values: A torch tensor of float32 numbers to convert to fp4 format. r)rrrrrrrr!rr"r T)dimZkeepdimgư>r-r)rZsignbitrrabsisnanisinftensorrappenditemr#viewshapeZ unsqueezemaxminsumexpandint32Zargmin)rvaluesrZ abs_valuesr( is_invalidZE_bitsZM_bitsZcandidate_valuesZ candidate_EZ candidate_Mrr*rZ significandr' candidatesZabs_values_flatNZabs_values_expandedZmax_candidate_valueerrorsZ min_errors_Zis_tieZM_bits_expandedZ tie_breakerZ best_indicesZ E_selectedZ M_selectedrrrr Ns`                 zMXFP4Tensor._from_floatc Cs|j}d|kr|jks&nJd||}|dd}|ddkrdgd|j}|j|ddd}d||<tjjj||ddd}t|j}|||<| |dd|j |}| |dd}| |dd} | d>|B} | S)a Packs two e2m1 elements into a single uint8 along the specified dimension. Parameters: - dim: The dimension along which to pack the elements. Returns: - A torch tensor of dtype uint8 with two e2m1 elements packed into one uint8. rzHThe dimension to pack along is not within the range of tensor dimensionsrrZconstant)moder'r) r ndimr rnnZ functionalpadlistr6insertreshapeselect) rr-r Zsize_along_dimZnew_size_along_dimZ pad_sizesZ pad_index new_shapelowhighpackedrrrto_packed_tensors&       zMXFP4Tensor.to_packed_tensorc Cs|d?d@}|d@}tj||f|dd}t|j}|d|||dg||dd}|j|} ||ddkrtdg| j} td||| |<| t| } | tj S)a Unpacks a tensor where two fp4 elements are packed into a single uint8. Parameters: - packed_tensor: The packed tensor - dim: The dimension along which the tensor was packed. - original_shape: The shape of the original tensor before packing. Returns: - A tensor with the original data unpacked into uint8 elements containing one fp4e2m1 element in the least significant bits. rrr.Nrr) rstackrFr6rHslicerCr rr) rZ packed_tensorr-Zoriginal_shaperLrKZstackedr6rJr indicesrrrunpack_packed_tensors  *  z MXFP4Tensor.unpack_packed_tensor)NNN) __name__ __module__ __qualname__rrr+r rNrSrrrrr s  #X#rc@s0eZdZd ddZd ddZddZdd ZdS) MXScaleTensorNcCsd||_|dur8t|tjs"Jd|j|_|||_n(|durXt|trN|n|f|_ntddS)a6 Tensor class for working with microscaling E8M0 block scale factors. Parameters: - data: A torch tensor of float32 numbers to convert to fp8e8m0 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nrrrrrrrrs zMXScaleTensor.__init__c Csd}|durdntdttt||}|dur:dn"tdtdttt||}||ksnJdtj||d|jtj|j d}||_ |S)zp Generate random E8M0 data within a specified range. * Excludes the NaN encoding (255). Nrz&Low must be less than or equal to highrr) r7intrlog2r2r8rr rrr )rrKrLZbiasZ min_exponentZ max_exponentrrrrrs*0zMXScaleTensor.randomcCs^|tjksJd|j|}|dk}|}d||<|d}td|}tj||<||S)NzBCurrently only float32 is supported for f8e8m0 to float conversionrrXg@)rr#r rcloner%nan)rrr is_nane_biaseder'rrrr+ s   zMXScaleTensor.toc Cstj|tj|jd}t|t|B|dkB}d||<||}tt|}|d}|tj }t |dd}|tj||<|S)aO Convert float32 numbers to E8M0 format. * Values <= 0, NaNs, and Infs are converted to the NaN encoding (255). * Positive values are converted by computing the floor of log2(value) to get the exponent. Parameters: - values: A torch tensor of float32 numbers to convert to E8M0 format. r,rr\rXrY) rZ empty_likerrr0r1floorr[rr;clamp) rr<resultr=Z valid_valuesrar`Z e_biased_intZe_biased_clampedrrrr s   zMXScaleTensor._from_float)NNN)NN)rTrUrVrrr+r rrrrrWs   rW)__doc__rrrWrrrrs^