In Why Feature Scaling Works: Condition Number as the Key to Gradient Descent Convergence, I study how the condition number of the Hessian helps explain why scaling choices affect convergence. The paper compares standardisation, min-max scaling, robust scaling, and PCA whitening through experiments with different dataset sizes and feature imbalances.
About the paper
- Published September 28, 2026 · Volume 11, Issue 6
- Connects feature scaling, numerical conditioning, and gradient descent convergence
- Examines the limits of whitening when data contains outliers
- DOI: 10.65161/recxYS7FqdJd63gJy