Clip
The Downside of Model Parameter Count Race
The race to have the highest parameter count in models, such as with GPT-4, may be similar to the gigahertz race of processors in the 90s and 2000s. However, this focus on complexity ignores the fact that models are a compression of all human knowledge and experiences, which may not necessarily be dependent on the number of parameters.