[ICLR 2024] Efficient Streaming Language Models with Attention Sinks
#自然语言处理#Extend existing LLMs way beyond the original training length with constant memory usage, without retraining
[ATTRIB @ NeurIPS 2024 Oral] When Attention Sink Emerges in Language Models: An Empirical View
[ICML 2024] Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
9.S912 Project
Integration of Attention Sinks into FlashAttention for StarCoder
Dynamic Attention Sinks in Streaming Large Language Models
Efficient Streaming Language Models with Attention Sinks
flink stream, sink to influxdb, sink to redis
xrdp sink / source pulseaudio modules
Snowflake Kafka Connector (Sink Connector)
ESP32 A2DP sink with extra codecs
A Serilog sink that writes events to Elasticsearch
All about attention in neural networks. Soft attention, attention maps, local and global attention and multi-head attention.
Uses Facebook profile pictures to provide high resolution photos for your Google contacts
Bluetooth audio playback (A2DP Sink) connector for Windows 10 2004+
Classification with backbone Resnet and attentions: SE-Channel Attention, BAM - (Spatial Attention, Channel Attention, Joint Attention), CBAM - (Spatial Attention, Channel Attention, Joint Attention)
C# serilog sink for spectre console
Ring attention implementation with flash attention
some attention implements
翻译 - 一些注意工具
#计算机科学#ResNeSt: Split-Attention Networks
翻译 - ResNeSt：注意力分散网络
NativeScript kitchen sink demo. All of NativeScript’s functionality in one app.
翻译 - NativeScript厨房水槽演示。一个应用程序中所有NativeScript的功能。
Docker example with kafka connect and sink
Fast and memory-efficient exact attention
Pytorch implementation of U-Net, R2U-Net, Attention U-Net, and Attention R2U-Net.
A Serilog sink that writes events to Microsoft Azure Application Insights
[CVPR 2023] Neighborhood Attention Transformer and [arXiv] Dilated Neighborhood Attention Transformer repository.
This repository contain various types of attention mechanism like Bahdanau , Soft attention , Additive Attention , Hierarchical Attention etc in Pytorch, Tensorflow, Keras