mllm · GitHub Topics

#自然语言处理#Unilm是一个跨任务、语言和模式的大规模自监督预训练模型

自然语言处理 pre-trained-model unilm minilm layoutlm layoutxlm beit document-ai trocr beit-3 foundation-models 大语言模型 multimodal mllm

Python 20.89 k

9 天前

X-PLUG / MobileAgent

#安卓#Mobile-Agent: The Powerful Mobile Device Operation Assistant Family

agent gpt4v mllm mobile-agents multimodal multimodal-large-language-models multimodal-agent Android App GUI 移动自动化 copilot harmony iOS

Python 3.63 k

15 小时前

ant-research / MagicQuill

[CVPR'25] Official Implementations for Paper - MagicQuill: An Intelligent Interactive Image Editing System

aigc image-editing mllm gradio

Python 3.03 k

13 天前

InternLM / InternLM-XComposer

#大语言模型#InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

ChatGPT visual-language-learning multi-modality foundation gpt-4 instruction-tuning mllm multimodal vision-language-model language-model large-language-model large-vision-language-model 大语言模型 vision-transformer gpt

Python 2.78 k

2 个月前

atfortes / Awesome-LLM-Reasoning

#大语言模型#Reasoning in LLMs: Papers and Resources, including Chain-of-Thought, OpenAI o1, and DeepSeek-R1 🍓

language-models reasoning prompt in-context-learning ChatGPT chain-of-thought prompt-engineering cot Awesome Lists gpt mllm multimodal papers gpt-4o openai-o1 strawberry deepseek deepseek-r1

2.76 k

20 天前

X-PLUG / mPLUG-DocOwl

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

chart-understanding document-understanding mllm multimodal multimodal-large-language-models table-understanding

Python 2.13 k

3 个月前

cambrian-mllm / cambrian

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

聊天机器人 clip 机器视觉 dino instruction-tuning large-language-models llms mllm multimodal-large-language-models representation-learning

Python 1.87 k

4 个月前

BAAI-DCAI / Bunny

#大语言模型#A family of lightweight multimodal models.

mllm ChatGPT gpt-4 multimodal-large-language-models vlm 中文 english

Python 1 k

4 个月前

magic-research / Sa2VA

🔥 Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos

机器视觉 mllm

Python 957

15 天前

simular-ai / Agent-S

[ICLR 2025] Agent S: an open agentic framework that uses computers like a human

ai-agents memory mllm planning retrieval-augmented-generation

Python 840

2 天前

CircleRadon / Osprey

[CVPR2024] The code for "Osprey: Pixel Understanding with Visual Instruction Tuning"

mllm sam visual-instruction-tuning pixel-understanding

Python 800

14 天前

BradyFU / Woodpecker

#大语言模型#✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models

hallucination hallucinations large-language-models 大语言模型 mllm multimodal-large-language-models multimodality

Python 632

3 个月前

NVlabs / EAGLE

#大语言模型#Eagle Family: Exploring Model Designs, Data Recipes and Training Strategies for Frontier-Class Multimodal LLMs

Demo gpt4 huggingface llama llama3 llava lmm mllm 大语言模型 large-language-models

Python 617

1 个月前

FoundationVision / Groma

#大语言模型#[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

grounding 大语言模型 mllm large-language-models foundation-models llama llama2 multimodal vision-language-model

Python 550

9 个月前

taco-group / OpenEMMA

#算法刷题#OpenEMMA, a permissively licensed open source "reproduction" of Waymo’s EMMA model.

算法人工智能 autonomous-driving autonomous-vehicles autonomy generative-ai 机器学习 mllm Network perception

Python 541

21 天前

SkyworkAI / Vitron

NeurIPS 2024 Paper: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

mllm multimodal-large-language-models segmentation

Python 498

5 个月前

gokayfem / ComfyUI_VLM_nodes

#大语言模型#Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation

nodes comfyui custom-nodes llava 大语言模型 image-captioning mllm vlm

Python 473

1 个月前

YingqingHe / Awesome-LLMs-meet-Multimodal-Generation

#大语言模型#🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

aigc large-language-models large-vision-language-models multimodal-generation multimodal-large-language-models multimodal-models multimodality text-to-3d text-to-audio text-to-image text-to-speech text-to-video 大语言模型 mllm

HTML 437

5 天前

Coobiw / MPP-LLaVA

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train...

multimodal-large-language-models deepspeed pipeline-parallelism mllm qwen fine-tuning pretraining

Jupyter Notebook 421

2 天前

dvlab-research / LLMGA

#大语言模型#This project is the official implementation of 'LLMGA: Multimodal Large Language Model based Generation Assistant', ECCV2024 Oral

image-editing image-generation 大语言模型 large-language-model mllm multi-modal aigc

Python 390

7 个月前