In machine learning, a neural scaling law is an empirical scaling law describing how neural network performance changes as key factors are scaled up or down.
The factors typically include the number of parameters, training dataset size, and training cost.
Research extends from Kaplan's original laws through Chinchilla scaling to broken neural scaling laws (BNSL) and inference-time scaling.