# Exact Tests

Exact Tests provides two additional methods for calculating significance levels for the statistics available through the Crosstabs and Nonparametric Tests procedures. These methods, the exact and Monte Carlo methods, provide a means for obtaining accurate results when your data fail to meet any of the underlying assumptions necessary for reliable results using the standard asymptotic method. Available only if you have purchased the Exact Tests Options.

**Example.** Asymptotic results obtained from small datasets or sparse or unbalanced
tables can be misleading. Exact tests enable you to obtain an accurate
significance level without relying on assumptions that might not be
met by your data. For example, results of an entrance exam for 20
fire fighters in a small township show that all five white applicants
received a pass result, whereas the results for Black, Asian and Hispanic
applicants are mixed. A Pearson chi-square testing the null hypothesis
that results are independent of race produces an asymptotic significance
level of 0.07. This result leads to the conclusion that exam results
are independent of the race of the examinee. However, because the
data contain only 20 cases and the cells have expected frequencies
of less than 5, this result is not trustworthy. The exact significance
of the Pearson chi-square is 0.04, which leads to the opposite conclusion.
Based on the exact significance, you would conclude that exam results
and race of the examinee are related. This demonstrates the importance
of obtaining exact results when the assumptions of the asymptotic
method cannot be met. The exact significance is always reliable, regardless
of the size, distribution, sparseness, or balance of the data.

**Statistics. **Asymptotic significance. Monte Carlo approximation with confidence
level, or exact significance.

- Asymptotic. The significance level based on the asymptotic distribution of a test statistic. Typically, a value of less than 0.05 is considered significant. The asymptotic significance is based on the assumption that the data set is large. If the data set is small or poorly distributed, this may not be a good indication of significance.
- Monte Carlo Estimate. An unbiased estimate of the exact significance level, calculated by repeatedly sampling from a reference set of tables with the same dimensions and row and column margins as the observed table. The Monte Carlo method allows you to estimate exact significance without relying on the assumptions required for the asymptotic method. This method is most useful when the data set is too large to compute exact significance, but the data do not meet the assumptions of the asymptotic method.
- Exact. The probability of the observed outcome or an outcome more extreme is calculated exactly. Typically, a significance level less than 0.05 is considered significant, indicating that there is some relationship between the row and column variables.