Why bootstrapping

Therefore, what remains unchosen or OOB is the total, or 1 — 0. What remains out of bag, unsampled, is about one-third of your dataset. Each bootstrapped sample has an equal number of data points as the size of your dataset.

For the question of how many times to bootstrap, 1, times is often appropriate, and in some cases more can help to find a high level of certainty about the reliability of your statistics. Bootstrapping can also be accomplished with as few as 50 samples.

There are many ways to implement bootstrapping in Python. Resample can be used from the Scikit Learn library. Bootstrapped is a Python library designed specifically for this purpose, and bootstrapping can also be done in Python using pandas.

Here is an example of how you can bootstrap a population sample and measure your confidence interval using pandas in Python. The formatted code can be viewed on gist. How would you find out? You would probably just count them—finding out the answer to this question does not require bootstrapping.

But what if you wanted to know the average number of shoes that everyone in your office owns? You work for a big technology company, so your office is large. This may still be impractical and time-consuming, though. Instead, you decide to bootstrap it. On the first day you survey 50 people without replacement and record how many shoes each person has.

This is your dataset. Instead of repeating this procedure every day, you take those 50 data points and create a whole lot of bootstrapped samples from them. This "new" sample is not identical to the original sample - indeed we might generate several "new" samples as above. When we look at the variations in the means and estimate, we are able to get a reading on how accurate the original estimates were. The "newer" samples are not identical to the first one and the new estimates based on these will vary.

This simulates repeated samples of the population. The variations in the estimates of the "newer" samples generated by the bootstrap will shed a light on how the sample estimates would vary given different samples from the population. This is in fact how we can get try to measure the accuracy of the original estimates. Of course, instead of bootstrapping you might instead take several new samples from the population but this might be infeasible.

I realize this is an old question with an accepted answer, but I'd like to provide my view of the bootstrap method. I'm in no ways an expert more of a statistics user, as the OP and welcome any corrections or comments. I like to view bootstrap as a generalization of the jackknife method. So, let's say you have a sample S of size and estimate some parameter by using a statistic T S. Now, you would like to know a confidence interval for this point estimate.

This is the jackknife method JK Now, bootstrap is just a randomized version of this. By doing resampling via selection with replacements you would "delete" a random number of elements possibly none and "replace" them by one or more replicates.

By replacing with replicates the resampled dataset always have the same size. For jackknife you may ask what is the effect of jackknifing on samples of size 99 instead of , but if sample size is "sufficiently large" this is likely a non-issue.

In jackknife you never mix delete-1 and delete-2 etc, to make sure the jacked estimates are from samples of same size. You may also consider splitting the sample of size into e. This would in some theoretical aspects be cleaner independent subsets but reduces the sample size from to 10 so much as to be impractical in most cases. You could also consider partially overlapping subsets of certain size. All this is handled in an automatic and uniform and random way by the bootstrap method.

Further, the bootstrap method gives you an estimate of the sampling distribution of your statistic from the empirical distribution of the original sample, so you can analyze further properties of the statistic besides standard error.

Paraphrasing Fox , I would start by saying that the process of repeatedly resampling from your observed sample has been shown to mimic the process of the original sampling from the whole population.

A finite sampling of the population approximates the distribution the same way a histogram approximates it. By re-sampling, each bin count is changed and you get a new approximation.

Large count values fluctuate less that small count values both in the original population and in the sampled set. Since you are explaining this to a layperson, you can argue that for large bin counts this is roughly the square root of the bin count in both cases. So if we approximate the true probability as the sampled one, we can get an estimate of sampling error "around" this value.

I think it is important to stress that the bootstrap does not uncover "new" data, it is just a convenient, non parametric way to approximately determine the sample to sample fluctuations if the true probability is given by the sampled one. Note that in classic inferential statistics the theoretical entity that connects a sample to the population as a good estimator of the population is the sampling distribution all the possible samples that could be drawn from the population.

The bootstrap method is creating a kind of sampling distribution a distribution based on multiple samples. Sure, it is a maximum likelihood method, but the basic logic is not that different from that of the traditional probability theory behind classic normal distribution-based statistics. Imagine you've got a random sample of 9 measurements from some population. The mean of the sample is Can we be sure that the average of the whole population is also 60?

Obviously not because small samples will vary, so the estimate of 60 is likely to be inaccurate. To find out how much samples like this will vary, we can run some experiments - using a method called bootstrapping. The first number in the sample is 74 and the second is 65, so let's imagine a big "pretend" population comprising one ninth 74's, one ninth 65's, and so on.

The easiest way to take a random sample from this population is to take a number at random from the sample of nine, then replace it so you have the original sample of nine again and choose another one at random, and so on until you have a "resample" of 9. When I did this, 74 did not appear at all but some of the other numbers appeared twice, and the mean was Which suggests that there is an error of up to units 44 is 16 below the pretend population mean of 60, 80 is 20 units above in using samples of size 9 to estimate the population mean.

There are a number of assumptions glossed over here, the obvious one being the assumption that the sample gives a useful picture of the population - experience shows this generally works well provided the sample is reasonably large 9 is a bit small but makes it easier to see what's going on. Bootstrap works because it computationally intensively exploits the main premise of our research agenda. To be more specific, in statistics or biology, or most non-theoretical sciences, we study individuals, thus collecting samples.

Yet, from such samples, we want to make inferences on other individuals, presenting to us in the future or in different samples. With bootstrap, by explicitly founding our modeling on the individual components of our sample, we may better with fewer assumptions, usually infer and predict for other individuals.

Sign up to join this community. The best answers are voted up and rise to the top. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams? Learn more. Explaining to laypeople why bootstrapping works Ask Question. Asked 9 years, 7 months ago. Active 9 days ago. Viewed k times. Improve this question. Alan H. There are lots of questions on the bootstrap here. Hopefully it is more clear.

Very confusing to know "which bootstrap" you should using. So, when there is problems with maximum likelihood, you can expect problems with the bootstrap. See the slides starting at slide 71 and the video recording. Show 2 more comments. You estimate the parameters from the data that you have and then you use the estimated distributions to simulate the samples. The nonparametric bootstrap does not work well because sampling with replacement produces exact replicates.

The samples that are identical are going to get clustered together. So, you don't get very much new information. The semi-parametric bootstrap perturbs the data with a bit a noise. For clustering, instead of taking a bootstrap sample and perturbing it, we might take the entire original sample and perturb it. This allows us to identify the original data points on the cluster diagram and see whether they remain in the same clusters or move to new clusters.

Obtaining a confidence interval for a Normal mean a parametric example. Suppose we have a sample of size n and we believe the population is Normally distributed. This is because it does not take into account that we have estimated the variance. There are ways to improve the estimate, but we will not discuss them here.

In this case we will assume that the data are Poisson. Here is what we would do:. We will hold the library sizes fixed. To estimate this interval, it is simplest to use the sorted bootstrap values instead of the histogram. For example, if you drop the 2. This is a parametric bootstrap confidence interval because the bootstrap samples were generated by estimating the Poisson means and then generating samples from the Poisson distribution. The jackknife, the bootstrap, and other resampling plans.

ISBN