fp8

#计算机科学#A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization i...

#计算机科学#Microsoft Automatic Mixed Precision Library

An innovative library for efficient LLM inference via low-bit quantization

Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.

#大语言模型#JAX Scalify: end-to-end scaled arithmetics

Cog Single GPU Quantized Implementation of Step-Video-T2V

FP8 dtypes enumeration in python