linformer

list of efficient attention modules

翻译 - 有效关注模块列表

#计算机科学#My take on a practical implementation of Linformer for Pytorch.

Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)

#计算机科学#Vision Xformers

#大语言模型#Lumenspark is a lightweight Linformer-based Language Model Trained from Scratch

#自然语言处理#linformer implementation and comparison with vanilla transformers