1. Bootstrap
Draw a “bootstrap sample" by sampling n times with replacement from the sample.
The bootstrap estimates the variability of the sampling process and works well for estimating confidence intervals.
A confidence interval provides a range of values which is likely to contain the population parameter of interest.
ex. I have 95% confidence to believe that the mean of this parameter is in range(x1, x2)
2. Permutation
Concatenate two datasets A & B, randomly reset the indexes, then output new A and new B with no replacement.
Permutation tests test a specific null hypothesis of exchangeability.
3.Cross validation
Cross-validation removes one point at a time, then fits to the remaining points, then sees how well the removed point is fit.
Cross-validation is primarily a way of measuring the predictive performance of a statistical model.