Anna Rogers | BERTology nuggets: what we have learned about how BERT works

Sponsored by Evolution AI: Wednesday 15 December 2021 Abstract: BERT and its Transformer-based cousins are at the core of most state-of-the-art NLP systems, yet it is still unclear why they work so well, and what could be our next step. This talk presents a collection of findings on how BERT learns and generalizes, where it fails, and whether the lottery ticket hypothesis holds for this architecture. Bio: Anna Rogers is a post-doctoral researcher at the University of Copenhagen, focusing on analysis and evaluation of deep learning models for NLP, as well as their impact on society. She is also known for her work on NLP methodology, peer review, and as an organizer of the Workshop on Insights from Negative Results. Moderated by: Alessandro Raganato