quantization · GitHub Topics | GitHub 中文社区

LLaMA-Factory

@hiyouga

#大语言模型# Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

fine-tuning language-model llama llm peft transformers rlhf qlora quantization chatglm

Python34.73 k

1 天前

Chinese-LLaMA-Alpaca

Yiming Cui@ymcui

#自然语言处理# 中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

llm plm pre-trained-language-models alpaca llama nlp quantization large-language-models lora alpaca-2

Python18.41 k

7 个月前

faster-whisper

@SYSTRAN

#计算机科学# Faster Whisper transcription with CTranslate2

deep-learning inference quantization speech-recognition speech-to-text transformer whisper openai

Python12.59 k

1 天前

Qbot

@UFund-Me

#计算机科学# [🔥updating ...] AI 自动量化交易机器人(完全本地部署) AI-powered Quantitative Investment Research Platform. 📃 online docs: https://ufund-me.github.io/Qbot ✨ :news: qbot-mini: https://github.com/Charmve/iQuant

funds machine-learning pytrade quantitative-finance quantitative-trading quantization strategies trademarks

Jupyter Notebook7.92 k

13 天前

AutoGPTQ

@AutoGPTQ

#自然语言处理# An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

transformers deep-learning inference large-language-models llms nlp pytorch quantization transformer

Python4.5 k

2 个月前

CTranslate2

OpenNMT@OpenNMT

Fast inference engine for Transformer models

neural-machine-translation cpp mkl quantization cuda thrust opennmt deep-neural-networks openmp onednn

C++3.41 k

3 天前

deepsparse

@neuralmagic

#自然语言处理# Sparsity-aware deep learning inference runtime for CPUs

machinelearning onnx inference computer-vision object-detection pruning quantization pretrained-models nlp cpus

Python3.03 k

4 个月前

optimum

Hugging Face@huggingface

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

onnx pytorch inference training intel graphcore onnxruntime transformers quantization habana

Python2.58 k

5 小时前

mixtral-offloading

@dvmazur

#大语言模型# Run Mixtral-8x7B models in Colab or consumer desktops

colab-notebook deep-learning google-colab language-model llm mixture-of-experts offloading pytorch quantization

Python2.29 k

7 个月前

neural-compressor

Intel Corporation@intel

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

low-precision pruning sparsity auto-tuning knowledge-distillation quantization quantization-aware-training post-training-quantization smoothquant large-language-models

Python2.23 k

14 小时前

micronet

@666DZY666

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Ari...

翻译 - 基于pytorch的模型压缩（1，量化：8/4 / 2bits（dorefa），三进制/二进制值（twn / bnn / xnornet）； 2，修剪：常规，常规和组卷积通道修剪； 3，组卷积结构； 4，特征（A）的二进制值的分批归一化折叠）

quantization pruning dorefa twn bnn xnor-net pytorch model-compression group-convolution batch-normalization-folding

Python2.22 k

3 年前