fastertransformer · GitHub Topics

InternLM / lmdeploy

#大语言模型#LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

cuda-kernels deepspeed fastertransformer llm-inference turbomind internlm llama 大语言模型 codellama llama2 llama3

Python 5.82 k

21 小时前

Curt-Park / serving-codegen-gptj-triton

Serving Example of CodeGen-350M-Mono-GPTJ on Triton Inference Server with Docker and Kubernetes

codegen Docker fastertransformer Kubernetes triton-inference-server PyTorch huggingface-transformers

Python 20

2 年前

detail-novelist / novelist-triton-server

Deploy KoGPT with Triton Inference Server

fastertransformer huggingface kogpt large-language-models transformers triton triton-inference-server

Shell 14

2 年前

clam004 / triton-ft-api

tutorial on how to deploy a scalable autoregressive causal language model transformer using nvidia triton server

FastAPI fastertransformer gpt huggingface Nvidia nvidia-docker

Python 5

2 年前

RajeshThallam / fastertransformer-converter

#大语言模型#This repository is a code sample to serve Large Language Models (LLM) on a Google Kubernetes Engine (GKE) cluster with GPUs running NVIDIA Triton Inference Server with FasterTransformer backend.

fastertransformer gke Google 云 inference 大语言模型 triton-inference-server

Python 0

2 年前