Large language models like GPT can predict the next word in a sequence by downloading massive amounts of text data from the internet and training the model to find emergent properties. Language models predict text and with a powerful enough neural net, they have the ability to scale up and predict text with surprising accuracy.