This is a Triton implementation of the Flash Attention v2 algorithm from Tri Dao (https://tridao.me/publications/flash2/flash2.pdf) ...

Chinese AI company Moonshot AI has launched Kimi K3, its most capable model so far and the world’s first open 3T-class AI ...