attention-mechanisms

#计算机科学#Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

#计算机科学#Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

#计算机科学#Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

#计算机科学#Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

#计算机科学#Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

#计算机科学#Implementation of Alphafold 3 from Google Deepmind in Pytorch

#计算机科学#Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

#计算机科学#Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

#大语言模型#Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

#计算机科学#Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch