llm-inference · GitHub Topics | GitHub 中文社区

gpt4all

@nomic-ai

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

llm-inference ai-chat

C++70.77 k

27 分钟前

llm-action

@liguodongiot

#大语言模型# 本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

llm llm-inference llm-serving llm-training llmops

HTML10.89 k

19 天前

litgpt

⚡️ Lightning AI @Lightning-AI

#大语言模型# 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

ai artificial-intelligence deep-learning large-language-models llm llm-inference llms

Python10.72 k

9 天前

mistral-inference

@mistralai

#大语言模型# Official inference library for Mistral models

llm llm-inference mistralai

Jupyter Notebook9.75 k

11 天前

PowerInfer

@SJTU-IPADS

#大语言模型# PowerInfer 是一个快速的、可运行在消费级GPU、个人电脑上的大模型服务

falcon large-language-models llama llm llm-inference local-inference bamboo-7b

C++7.97 k

3 个月前

openvino

OpenVINO Toolkit@openvinotoolkit

#自然语言处理# OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

翻译 - OpenVINO™工具包存储库

inference deep-learning openvino ai computer-vision diffusion-models generative-ai llm-inference natural-language-processing nlp

C++7.32 k

4 小时前

lmdeploy

@InternLM

#大语言模型# LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

cuda-kernels deepspeed fastertransformer llm-inference turbomind internlm llama llm codellama llama2

Python4.7 k

10 小时前

GenerativeAIExamples

NVIDIA Corporation@NVIDIA

#大语言模型# Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

gpu-acceleration large-language-models llm llm-inference microservice nemo rag retrieval-augmented-generation tensorrt triton-inference-server

Python2.46 k

6 天前

dbrx

Databricks@databricks

#大语言模型# Code examples and resources for DBRX, a large language model developed by Databricks

databricks gen-ai generative-ai llm llm-inference llm-training mosaic-ai

Python2.46 k

7 个月前

intel-extension-for-transformers

Intel Corporation@intel

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

large-language-model chatbot 4-bits llm-inference llm-cpu chatpdf streamingllm attention-sink intel-optimized-llamacpp speculative-decoding

Python2.14 k

1 个月前

llama2-webui

@liltom-eth

#大语言模型# Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.

llama-2 llama2 llm llm-inference

Jupyter Notebook1.97 k

8 个月前

aici

Microsoft@microsoft

#大语言模型# AICI: Prompts as (Wasm) Programs

ai rust wasm wasmtime inference language-model llm llm-framework llm-inference llm-serving

Rust1.66 k

7 个月前

distributed-llama

@b4rtaz

#大语言模型# Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.

distributed-computing llama2 llm llm-inference neural-network llms open-llm distributed-llm llama3

C++1.51 k

1 天前

LeanCopilot

@lean-dojo

#计算机科学# LLMs as Copilots for Theorem Proving in Lean

lean llm-inference machine-learning theorem-proving lean4 formal-mathematics

C++997

19 天前

archgw

@katanemo

Arch is an intelligent prompt gateway. Engineered with (fast) LLMs for the secure handling, robust observability, and seamless integration of prompts with your APIs - outside business logic. Built by ...

gateway generative-ai llm-inference llms prompt proxy proxy-server llmops openai routing

Rust783

43 分钟前