vllm · GitHub Topics

#大语言模型#Llama 2 微调/推理方法和示例

人工智能 finetuning langchain llama llama2 大语言模型机器学习 Python PyTorch vllm

Jupyter Notebook 16.43 k

10 小时前

#大语言模型#Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any...

ggml PyTorch chatglm 部署 flan-t5 大语言模型 wizardlm 人工智能机器学习 Whisper inference openai-api mistral gemma llama llamacpp vllm qwen llama3 glm4

Python 6.93 k

1 天前

OpenRLHF / OpenRLHF

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT)

transformers vllm large-language-models raylib reinforcement-learning-from-human-feedback reinforcement-learning openai-o1 proximal-policy-optimization

Python 5.49 k

2 天前

katanaml / sparrow

#大语言模型#Data processing with ML, LLM and Vision LLM

机器学习 huggingface-transformers 自然语言处理机器视觉 gpt 大语言模型 rag vllm

Python 4.4 k

11 小时前

DefTruth / Awesome-LLM-Inference

📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, Flash-Attention, Paged-Attention, Parallelism, etc. 🎉🎉

flash-attention tensorrt-llm vllm llm-inference deepseek deepseek-v3 deepseek-r1

3.62 k

8 天前

containers / ramalama

#大语言模型#The goal of RamaLama is to make working with AI boring.

人工智能 containers inference-server llamacpp podman vllm 大语言模型

Python 1.32 k

5 天前

bricks-cloud / BricksLLM

#大语言模型#🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI...

Go 大语言模型 openai 人工智能 anthropic Azure gpt PostgreSQL REST API ycombinator API Docker 隐私安全 generative-ai Open Source 自托管 vllm

Go 1.01 k

2 个月前

prometheus-eval / prometheus-eval

#大语言模型#Evaluate your LLM's response with Prometheus and GPT4 💯

evaluation 大语言模型 llmops Python vllm gpt4 llm-as-a-judge

Python 877

2 个月前

substratusai / kubeai

#大语言模型#AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

Kubernetes 大语言模型 openai-api autoscaler ollama vllm ollama-operator vllm-operator 人工智能 Whisper faster-whisper

Go 816

4 天前

harleyszhang / llm_note

#大语言模型#LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

大语言模型 llm-inference vllm cuda-programming kv-cache transformer-models

Python 597

2 天前

mustafaaljadery / llama3v

A SOTA vision model built on top of llama3 8B.

llama llama3 vllm

Python 586

9 个月前

jakobdylanc / llmcord

#前端开发#Make Discord your LLM frontend ● Supports any OpenAI compatible API (Ollama, LM Studio, vLLM, OpenRouter, xAI, Mistral, Groq and more)

gpt openai gpt-4 Discord 聊天机器人大语言模型 Bot ollama gpt-4o llama3 llava llama mistral groq xai grok vllm 前端 chat

Python 491

11 天前

ModelTC / llmc

#大语言模型#[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".