model-serving · GitHub Topics | GitHub 中文社区

vllm

@vllm-project

#大语言模型# A high-throughput and memory-efficient inference and serving engine for LLMs

gpt llm pytorch llmops mlops model-serving transformer llm-serving inference llama

Python30.66 k

6 小时前

BentoML

@bentoml

#计算机科学# The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

翻译 - 轻松进行模型服务

model-serving model-deployment model-management ml-platform ai machine-learning mlops bentoml kubernetes deep-learning

Python7.17 k

3 天前

kserve

@kserve

#计算机科学# Standardized Serverless ML Inference Platform on Kubernetes

knative machine-learning model-interpretability model-serving istio kubeflow kubeflow-pipelines artificial-intelligence tensorflow pytorch

Python3.65 k

2 天前

lightllm

@ModelTC

#自然语言处理# LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

deep-learning gpt llama llm model-serving nlp openai-triton

Python1.93 k

6 个月前

stable-diffusion-deploy

@Lightning-Universe

Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services worki...

model-serving stable-diffusion

Python392

1 年前