Clip
The challenge of scaling transformer models
Transformer models require the right questions to be asked to get accurate answers, which can make performance appear random until the correct questions are asked. The threshold or phase shift scale is affected by the benchmark scale and the engineering benchmarks to scale models may have a low threshold.