”llm-inference“ 的搜索结果

Facebook 的 LLaMA 模型在 C/C++ 中的移植

📖A curated list of Awesome LLM/VLM Inference Papers with codes, such as FlashAttention, PagedAttention, Parallelism, etc. 🎉🎉

#大语言模型#浏览器中运行大模型，无需 server 支持

#大语言模型#Blazingly fast LLM inference.

Official inference framework for 1-bit LLMs

#大语言模型#A high-throughput and memory-efficient inference and serving engine for LLMs

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

#自然语言处理#Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

Optimizing inference proxy for LLMs

#计算机科学#Large-scale LLM inference engine

a lightweight LLM model inference framework

TinyChatEngine: On-Device LLM Inference Library

Large Language Model (LLM) Inference API and Chatbot

#大语言模型#33B Chinese LLM, DPO QLORA, 100K context, AirLLM 70B inference with single 4GB GPU

💬 Chatbot web app + HTTP and Websocket endpoints for LLM inference with the Petals client

#大语言模型#The Memory layer for your AI apps

#大语言模型#FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

#自然语言处理#LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Open source, local, and self-hosted highly optimized language inference server supporting ASR/STT, TTS, and LLM across WebRTC, REST, and WS

LLaMA模型的推理代码

Universal and Transferable Attacks on Aligned Language Models

#大语言模型#LLM Frontend for Power Users.

the LLM vulnerability scanner

#大语言模型#LLM Finetuning with peft

#大语言模型#[ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

The LLM Evaluation Framework

LLM as a Chatbot Service

#大语言模型#Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such ...