By creating higher order models or more sophisticated structures in machine learning, such as transformer networks, it is possible to form compressed representations of large amounts of text. In this way, it is thought that an approximation of a world model could be created that may be useful for reasoning.