multihead-attention · GitHub Topics

list of efficient attention modules

翻译 - 有效关注模块列表

transformer attention Awesome Lists reformer longformer linformer multihead-attention self-attention attention-is-all-you-need transformer-network

Python 995

4 年前

tlatkowski / multihead-siamese-nets

#自然语言处理#Implementation of Siamese Neural Networks built upon multihead attention mechanism for text semantic similarity task.

multihead-attention semantic-similarity 深度神经网络 attention 深度学习 text-similarity 自然语言处理 sentence-similarity Tensorflow Python

Jupyter Notebook 182

2 年前

datnnt1997 / multi-head_self-attention

A Faster Pytorch Implementation of Multi-Head Self-Attention

self-attention attention-mechanism attention multihead-attention

Jupyter Notebook 71

3 年前

tensorops / TransformerX

#自然语言处理#Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)

attention attention-mechanism 深度学习 vit multihead-attention 自然语言处理 self-attention transformers

Python 53

1 年前

jk96491 / Advanced_Models

여러가지 유명한 신경망 모델들을 제공합니다. (DCGAN, VAE, Resnet 등등)

Generative Adversarial Network resnet-50 vae PyTorch dcgan cgan multihead-attention gpt-2

Python 50

4 年前

akurniawan / pytorch-transformer

Implementation of "Attention is All You Need" paper

PyTorch attention attention-is-all-you-need multihead-attention

Python 33

8 个月前

changwookjun / Transformer

Chatbot using Tensorflow (Model is transformer) ko

transformer bert 聊天机器人 Tensorflow self-attention multihead-attention

Python 29

6 年前

Syeda-Farhat / awesome-Transformers-For-Segmentation

Semantic segmentation is an important job in computer vision, and its applications have grown in popularity over the last decade.We grouped the publications that used various forms of segmentation in ...

机器视觉 encoder-decoder instance-segmentation multihead-attention segmentation self-attention semantic-segmentation transformer

6 个月前

MirunaPislar / multi-head-attention-labeller

Joint text classification on multiple levels with multiple labels, using a multi-head attention mechanism to wire two prediction tasks together.

sentence-classification multi-task-learning multihead-attention transformer attention-mechanism zero-shot-learning semi-supervised-learning conll-2003 error-detection sentiment-analysis

Python 16

4 年前

iafarhan / causal-synthesizer-multihead-attention

Synthesizer Self-Attention is a very recent alternative to causal self-attention that has potential benefits by removing this dot product.

Python PyTorch attention multihead-attention

Python 12

2 个月前

shawnhan108 / AutoTruckX

An experimental project for autonomous vehicle driving perception with steering angle prediction and semantic segmentation using a combination of UNet, attention and transformers.

autonomous-vehicles autonomous-driving udacity-self-driving-car resnet-50 transfer-learning transformer attention semantic-segmentation unet multihead-attention

Python 8

4 年前

hrithickcodes / transformer-tf

This repository contains the code for the paper "Attention Is All You Need" i.e The Transformer.

attention-is-all-you-need multihead-attention neural-machine-translation self-attention transformer-architecture transformers

Jupyter Notebook 8

2 年前

bkhanal-11 / transformers

The implementation of transformer as presented in the paper "Attention is all you need" from scratch.

attention-is-all-you-need attention-mechanism multihead-attention self-attention transformers

Python 7

2 年前

jaydeepthik / Nano-GPT

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

gpt pytorch-implementation multihead-attention PyTorch transformers

Jupyter Notebook 5

2 年前

antonio-f / GPT_from_scratch

Very simple implementation of GPT architecture using PyTorch and Jupyter.

easy-to-use gpt Jupyter Notebook Python PyTorch 教程 from-scratch noob-friendly simple newbie transformer multihead-attention self-attention

Jupyter Notebook 4

2 年前

qdLMF / LightGlue-with-FlashAttentionV2-TensorRT

A cutlass cute implementation of headdim-64 flashattentionv2 TensorRT plugin for LightGlue. Run on Jetson Orin NX 8GB with TensorRT 8.5.2.

cute cutlass tensorrt feature-matching CUDA flash-attention multihead-attention transformer superpoint

Cuda 4

10 天前

abhilash1910 / GraphAttentionNetworks

This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.

tf2 graph-attention-networks multihead-attention self-attention keras-tensorflow

Python 3

3 年前

dcarpintero / transformer101

Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'.

attention-is-all-you-need multihead-attention self-attention feedforward-neural-network linear-layers PyTorch

Jupyter Notebook 1

1 年前

Resh-97 / MixSeq-Connecting-Macroscopic-Time-Series-Forecasting-with-Microscopic-Time-Series-Data

#时序数据库#Testing the Reproducibility of the paper: MixSeq. Under the assumption that macroscopic time series follow a mixture distribution, they hypothesise that lower variance of constituting latent mixture c...

arma deepar multihead-attention time-series vae-implementation

Jupyter Notebook 1

2 年前

yflyzhang / AnnotatedTransformer

encoder-decoder multihead-attention Parsing transformer

Jupyter Notebook 1

1 个月前