Transformers are a powerful tool in natural language processing, and they operate by looking at all the text and attending over it to identify key words and concepts. By creating two hypotheses and considering all aspects of the text, they can outperform more recency-bias-based models like recurrent networks.