Question & Answer
The p value (significance level) for the Bollen-Stine bootstrap that is reported by AMOS is slightly different from the value which I calculated from the detailed results of the bootstrap. I ran Example 19 from the AMOS User's Guide and added a request for the Bollen-Stine bootstrap. I requested 500 bootstrap samples. The default model chi-square for the observed data is 7.853 with 8 Degrees of freedom and a Probability level of .448. The Bollen-Stine output is : "Bollen-Stine Bootstrap (Default model) The model fit better in 263 bootstrap samples. It fit about equally well in 0 bootstrap samples. It fit worse or failed to fit in 237 bootstrap samples. Testing the null hypothesis that the model is correct, Bollen-Stine bootstrap p = .475" So, the default model chi-square for the fit to the bootstrap sample 'fit worse', i.e. was larger than the observed data chi-square, for 237/500 = .474 of the bootstrap samples. However, the Bollen-Stine bootstrap p-value is reported as .475.. There appears to be a correction factor added to the ratio in the calculation of the p-value. I have experimented with the number of bootstrap samples and found that the impact of this correction decreases as the number of bootstrap samples increases. IF there is such a correction factor, how is it defined and what is the rationale for it?"
The original sample chi-square is included in the numerator and denominator of p-value
"The Bollen-Stine bootstrap process involves the transformation of the data to a data set for which the null hypothesis (that the default model fits the data) is true. This requires the use of the covariance matrix to suitably transform the data from which the bootstrap samples are drawn. Each bootstrap sample is sampled from this transformed data and a chi-square is computed for the fit of that bootstrapped data to the model. These chi-squares are not printed, but they are compared internally to the chi-square that was computed for the observed data fit to the model (which is printed in the ""Notes for Model"" section of the output). The proportion of times that the model 'fit worse or failed to fit', i.e. the number of times that the model chi-square for the bootstrapped sample exceeded the chi-square for the observed data, is the Bollen-Stine bootstrap p value.
In your example, the Bollen-Stine p is computed as p = 238/501. The reasoning is that you have 501 chi square values. You know that 500 of them were generated randomly. The null hypothesis is that all 501 were generated randomly. If they were, then the question is, what is the probability that the original chi square value (i.e., the one from the original sample) would have rank number of 238 or less after ranking all 501 chi square values.
16 June 2018