audio-language · GitHub Topics

A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities

foundation-models multimodal representation-learning vision-language audio-language vision-and-language vision-transformer contrastive-loss

Python 1.02 k

5 个月前

AudioLLMs / Awesome-Audio-LLM

Audio Large Language Models

audio-language audio-processing

Python 429

4 天前

TXH-mercury / VAST

[NIPS2023] Code and Model for VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset

audio-language dataset vision-language

Jupyter Notebook 267

1 年前

Sreyan88 / GAMA

Code for the paper: GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities

audio dataset question-answering reasoning audio-language large-language-model multimodal-large-language-models

Python 111

3 个月前

Sreyan88 / CompA

#自然语言处理#Code for ICLR 2024 Paper: CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models

人工智能 audio benchmark compositionality 机器学习自然语言处理 audio-language

Python 15

8 个月前