llm-serving

#大语言模型#  A high-throughput and memory-efficient inference and serving engine for LLMs

#大语言模型#  本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

#大语言模型#  Run any open-source LLMs, such as Llama, Gemma, as OpenAI compatible API endpoint in the cloud.

#大语言模型#  SGLang is a fast serving framework for large language models and vision language models.

#大语言模型#  AICI: Prompts as (Wasm) Programs

#大语言模型#  RayLLM - LLMs on Ray

#计算机科学#  Efficient AI Inference & Serving