attention-mechanisms

#计算机科学#  Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

#计算机科学#  Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

#计算机科学#  Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

#计算机科学#  Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

#计算机科学#  Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

#计算机科学#  Implementation of Alphafold 3 from Google Deepmind in Pytorch

#计算机科学#  Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

#计算机科学#  Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

#大语言模型#  Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

#计算机科学#  Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch