Scaling Laws, Carefully: How to Fit and Interpret AI Models

Scaling laws predict how AI performance improves with more compute. However, fitting them is tricky. This article explores the pitfalls and nuances behind the curves.

MiHiR SEN
MiHiR SEN
·2 min read
Scaling laws show predictable loss decreases with compute, but fitting them is brittle. The famous disagreement between Kaplan and Chinchilla highlights sensitivity to methodology and parameter counting. As data becomes scarce, data repetition models must be considered.

Scaling laws are the bedrock of modern deep learning. They describe how training loss decreases predictably with model size, dataset size, and compute. The observation is simple: it follows a power-law curve, appearing as a straight line on a log-log plot.

This predictability makes them invaluable. Labs use them to extrapolate from small runs to estimate the requirements for billion-dollar training runs. But as the famous disagreement between Kaplan et al. and the Chinchilla paper shows, a small difference in the fit can change the recommendation dramatically.

The Great Scaling Debate

In 2020, Kaplan et al. suggested that for a 10x increase in compute, you should scale the model size by ~5.5x and the data by ~1.8x. The Chinchilla paper (2022) overturned this. They argued that for every doubling of model size, you should also double the data. This disagreement was not due to physics, but due to methodology.

Kaplan et al. counted non-embedding parameters in their fit. Chinchilla counted total parameters. When you account for embedding parameters, the gap narrows. More importantly, Kaplan extrapolated from a regime where embeddings dominated, leading to a steeper exponent.

The Sensitivity of Fits

Fitting a scaling law is fragile. The number of decimal places you round a loss value to matters. The optimizer used for the fit matters (Chinchilla Method 3 used L-BFGS-B and had a bug that led to premature termination). Even the range of data you choose to fit matters, as the exponent can shift.

Data Repetition

The classic scaling laws assume unique tokens. We are now hitting a "data wall". Researchers like Muennighoff et al. have studied data-constrained scaling. They found that repeated tokens lose value, which requires an adjustment to the effective data size.

Conclusion

Scaling laws are a guide, not a physical law. They are powerful but brittle. A careful practitioner must be wary of the assumptions in their fit and the limitations of their data.