Resampling is a method in statistics, where certain samples are selected to estimate their precision (like median, mean, variances) by drawing them in subsets from available data (jackknifing) or by drawing them randomly (bootstrapping). Resampling also involves other types of tests like cross validation, permutation tests etc.
Resampling had revolutionized statistics and is the method of choice for calculating confidence limits, hypothesis tests, and everyday inferential problems. It eases the way of solving complex problems by simulations which can’t even be done by formulas.
There are 4 major types of Resampling
Jackknife: Developed by John Turkey, this method explores, how a model is influenced by subsets of observations when there are outliers in it. It is all purpose statistical tool generally used in statistical inference to estimate the standard error (variance) and bias of a statistic, where random samples of are used to calculate it.
Bootstrap: Developed by Efron, Bootstrapping is a statistical method for estimating the sampling distribution by sampling with replacement from original samples intended to derive standard errors, confidence intervals etc. It is also used for hypothesis tests.
Cross Validation: Developed by Kurtz, it is a statistical method for validating a predictive model. It is used for partitioning a sample of data into certain subsets such that the analysis is initially performed on a single subset while other subsets are retained for confirming and validating the initial analysis. Simple cross validation, Double cross validation, Multicross validation etc. are some important topics under Cross Validation.
Permutation Test: Developed by R.A Fisher, it is a type of statistical significance test in which the distribution of the test statistic is obtained by calculating its all possible values with rearrangements of labels on the observed data points.
Following example shows that how a typical workbook problem can be solved easily using resampling method (without using any formulas and tables)
The probability of simultaneously flipping 3 coins and having them all land heads
COPY (0 1) coin
COPY 10000 rptCount
SAMPLE 3 coin flip
COUNT flip =1 heads
SCORE heads result
COUNT result =3 successes
DIVIDE successes rptCount probability
The above solution can be copied and pasted in Statistics 101’s editor window to get exact results.