K-fold validation has an advantage over LOOCV in variance reduction.

K-fold validation has an advantage over LOOCV in variance reduction.

• A. It depends
• B. Not defined
• C. True
• D. False

Answer: (C)

Variance of the cross-validated error estimate tends to be lower with K-fold cross-validation than with leave-one-out cross-validation. In LOOCV, each training set is almost the full dataset, leaving out just one observation, so the fitted model and the resulting error can be highly sensitive to that single point, causing the error estimate to vary a lot across splits. With K-fold, you average errors over many folds, each using different training subsets, which smooths out those fluctuations and yields a more stable, lower-variance estimate. There’s a trade-off, but the practical effect is that K-fold reduces variance relative to LOOCV, so the statement is true.

Variance of the cross-validated error estimate tends to be lower with K-fold cross-validation than with leave-one-out cross-validation. In LOOCV, each training set is almost the full dataset, leaving out just one observation, so the fitted model and the resulting error can be highly sensitive to that single point, causing the error estimate to vary a lot across splits. With K-fold, you average errors over many folds, each using different training subsets, which smooths out those fluctuations and yields a more stable, lower-variance estimate. There’s a trade-off, but the practical effect is that K-fold reduces variance relative to LOOCV, so the statement is true.