llm-inference · GitHub Topics | GitHub 中文社区

gpt4all

@nomic-ai

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

llm-inference ai-chat

C++71.29 k

2 天前

autogen

Microsoft@microsoft

#大语言模型#A programming framework for agentic AI 🤖 PyPi: autogen-agentchat Discord: https://aka.ms/autogen-discord Office Hour: https://aka.ms/autogen-officehour

agent-based-framework agent-oriented-programming chat chat-application Chat Bot ChatGPT gpt gpt-35-turbo gpt-4 llm-agent llm-framework llm-inference llmops agentic agentic-agi

Jupyter Notebook36.16 k

1 天前

llm-action

@liguodongiot

#大语言模型#本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

llm llm-inference llm-serving llm-training llmops

HTML12.07 k

6 天前

litgpt

⚡️ Lightning AI @Lightning-AI

#大语言模型#20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

人工智能深度学习 large-language-models llm llm-inference llms

Python10.92 k

14 天前

mistral-inference

@mistralai

#大语言模型#Official inference library for Mistral models

llm llm-inference mistralai

Jupyter Notebook9.81 k

1 个月前

PowerInfer

@SJTU-IPADS

#大语言模型#PowerInfer 是一个快速的、可运行在消费级GPU、个人电脑上的大模型服务

falcon large-language-models llama llm llm-inference local-inference bamboo-7b

C++8.01 k

4 个月前

openvino

OpenVINO Toolkit@openvinotoolkit

#自然语言处理#OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

翻译 - OpenVINO™工具包存储库

inference 深度学习 openvino 人工智能机器视觉 diffusion-models generative-ai llm-inference 自然语言处理 performance-boost speech-recognition stable-diffusion deploy-ai optimize-ai transformers yolo recommendation-system good-first-issue

C++7.49 k

2 天前

lmdeploy

@InternLM

#大语言模型#LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

cuda-kernels deepspeed fastertransformer llm-inference turbomind internlm llama llm codellama llama2

Python4.87 k

5 天前

GenerativeAIExamples

NVIDIA Corporation@NVIDIA

#大语言模型#Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

gpu-acceleration large-language-models llm llm-inference Microservice nemo rag retrieval-augmented-generation tensorrt triton-inference-server

Python2.55 k

6 天前

dbrx

Databricks@databricks

#大语言模型#Code examples and resources for DBRX, a large language model developed by Databricks

databricks gen-ai generative-ai llm llm-inference llm-training mosaic-ai

Python2.46 k

8 个月前

intel-extension-for-transformers存档

Intel Corporation@intel

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

large-language-model Chat Bot 4-bits llm-inference llm-cpu chatpdf streamingllm attention-sink intel-optimized-llamacpp speculative-decoding

Python2.14 k

2 个月前

llama2-webui

@liltom-eth

#大语言模型#Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.

llama-2 llama2 llm llm-inference

Jupyter Notebook1.97 k

9 个月前

Python1.81 k

24 天前

aici

Microsoft@microsoft

#大语言模型#AICI: Prompts as (Wasm) Programs

人工智能 Rust WebAssembly wasmtime inference language-model llm llm-framework llm-inference llm-serving

Rust1.66 k

8 个月前

distributed-llama

@b4rtaz

#大语言模型#Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.

distributed-computing llama2 llm llm-inference 神经网络 llms open-llm distributed-llm llama3

C++1.57 k

8 天前

ray-llm存档

ray-project@ray-project

#大语言模型#RayLLM - LLMs on Ray

distributed-systems large-language-models ray serving transformers llm llm-inference llm-serving llmops

Python1.24 k

7 个月前

archgw

@katanemo

Arch is an intelligent gateway for agents. Engineered with (fast) LLMs for the secure handling, rich observability, and seamless integration of prompts with your APIs - all outside business logic. Bui...

gateway generative-ai llm-inference llms prompt proxy proxy-server llmops openai Routing (disambiguation)ai-gateway llm-gateway llm-routing envoy envoyproxy

Rust1.03 k

1 天前