Related:

FlashAttention-2, 2x faster than FlashAttention - https://news.ycombinator.com/item?id=36761988 - July 2023 (18 comments)

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness - https://news.ycombinator.com/item?id=31568090 - May 2022 (3 comments)