Simply scaling up GPT-3 to have more trained data and transformative layers won't address its flaws. A possible way forward is to write programs over the latent space that these models operate on in collaboration with self-supervised models.