local-inference

#大语言模型#PowerInfer 是一个快速的、可运行在消费级GPU、个人电脑上的大模型服务

#大语言模型#[ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration

#大语言模型#Tool for test diferents large language models without code.

Local LLM Inference Library

#自然语言处理#LLM chatbot example using OpenVINO with RAG (Retrieval Augmented Generation).

#大语言模型#script which performs RAG and use a local LLM for Q&A

Script which takes a .wav audio file, performs speech-to-text using OpenAI/Whisper, and then, using Llama3, summarization and action point from the transcript generated