The technical challenges of scaling up large networks such as sparsely gated mixture of experts, with hundreds of trillions of parameters that have to run on thousands of years. The challenge is to explore how technologies can be applied without causing great harm.