Occam's razor: Simplicity prevents overfitting past data to predict future

Stuart_Armstrong reframes Occam's razor as a practical tool for prediction: to forecast future data, you must generalize from past evidence without overfitting. He illustrates with a toy example: using latitudes and solar angles at the equinox. A 'list hypothesis' that simply records all past measurements (e.g., Ankara: 50°, Brisbane: 63°) and arbitrarily predicts Anchorage's angle as 90° fits the past data perfectly — even better than the true underlying law (angle + latitude = 90°) because it ignores measurement error. Yet it's clearly useless for prediction. This highlights that perfect fit to past data is not enough; you need a theory that compresses that data into a general rule.

Armstrong explains that Occam's razor enforces a penalty for 'leaked information' — extra bits of complexity that allow a theory to match past data without actually capturing the underlying pattern. Formal frameworks like Kolmogorov complexity and Minimum Description Length operationalize this: you can always increase fit by adding more detail, but only at the cost of lower prior probability. This trade-off compensates for the apparent advantage of overfitted theories. He also distinguishes 'simplicity' (absolute length of description) from 'simplifiability' — the inability to make a theory shorter without losing predictive power. For example, General Relativity is more complex than Newtonian gravity in absolute terms, but its structure is not simplifiable; the apparent complexity is necessary to explain phenomena like Mercury's orbit. Thus Occam's razor isn't just 'choose the simpler theory' but 'choose the theory that best balances fit and compressibility for prediction.'