Big Bird: Transformers for Longer Sequences (Paper Explained)

#ai #nlp #attention The quadratic resource requirements of the attention mechanism are the main roadblock in scaling up transformers to long seque...