”llm-serving“ 的搜索结果

#大语言模型#A high-throughput and memory-efficient inference and serving engine for LLMs

FlashInfer: Kernel Library for LLM Serving

#大语言模型#LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

FlexFlow Serve: Low-Latency, High-Performance LLM Serving

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

Serving multiple LoRA finetuned LLM as one

Serverless LLM Serving for Everyone.

#自然语言处理#Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

Disaggregated serving system for Large Language Models (LLMs).

⛓️ Serving LangChain LLM apps and agents automagically with FastApi. LLMops

#大语言模型#Dynamic RAG for enterprise. Ready to run with Docker,⚡in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

#自然语言处理#LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Yahoo! Cloud Serving Benchmark

翻译 - 雅虎！云服务基准

#计算机科学#A flexible, high-performance serving system for machine learning models

翻译 - 用于机器学习模型的灵活，高性能的服务系统

Kubernetes-based, scale-to-zero, request-driven compute

翻译 - 基于Kubernetes，从零扩展到请求驱动的计算

favicon serving middleware

Static file serving and directory listing

Serving TensorFlow models with TensorFlow Serving 📙

Facebook 的 LLaMA 模型在 C/C++ 中的移植

faiss serving :)

#大语言模型#The Memory layer for your AI apps

Serving websites over bittorrent

#计算机科学#Efficient AI Inference & Serving

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

A framework for serving GraphQL from Laravel

翻译 - Laravel提供的GraphQL服务框架

A low-latency prediction-serving system

翻译 - 低延迟预测服务系统