Bayesian Inference about Linear Regression Models

This feature requires Custom Tables and Advanced Statistics.

Regression is a statistical method that is broadly used in quantitative modeling. Linear regression is a basic and standard approach in which researchers use the values of several variables to explain or predict values of a scale outcome. Bayesian univariate linear regression is an approach to Linear Regression where the statistical analysis is undertaken within the context of Bayesian inference.

You can invoke the regression procedure and define a full model.

  1. From the menus choose:

    Analyze > Bayesian Statistics > Linear Regression

  2. Select a single, non-string, dependent variable from the Available Variables list. You must select one non-string variable.
  3. Select one or more categorical factor variables for the model from the Available Variables list.
  4. Select one or more, non-string, covariate scale variables from the Available Variables list.
    Note: Both of the Factor(s) and Covariate(s) lists cannot be empty. You must select at least one Factor(s) or Covariate(s) variable.
  5. Optionally, select a single, non-string, variable to serve as the regression weight from the Available Variables list.
  6. Select the desired Bayesian Analysis:
    • Characterize Posterior Distribution: When selected, the Bayesian inference is made from a perspective that is approached by characterizing posterior distributions. You can investigate the marginal posterior distribution of the parameter(s) of interest by integrating out the other nuisance parameters, and further construct credible intervals to draw direct inference. This is the default setting.
    • Estimate Bayes Factor: When selected, estimating Bayes factors (one of the notable methodologies in Bayesian inference) constitutes a natural ratio to compare the marginal likelihoods between a null and an alternative hypothesis.
      Table 1. Commonly used thresholds to define significance of evidence
      Bayes Factor Evidence Category Bayes Factor Evidence Category Bayes Factor Evidence Category
      >100 Extreme Evidence for H0 1-3 Anecdotal Evidence for H0 1/30-1/10 Strong Evidence for H1
      30-100 Very Strong Evidence for H0 1 No Evidence 1/100-1/30 Very Strong Evidence for H1
      10-30 Strong Evidence for H0 1/3-1 Anecdotal Evidence for H1 1/100 Extreme Evidence for H1
      3-10 Moderate Evidence for H0 1/10-1/3 Moderate Evidence for H1    

      H0: Null Hypothesis

      H1: Alternative Hypothesis

      1

      2

    • Use Both Methods: When selected, both the Characterize Posterior Distribution and Estimate Bayes Factor inference methods as used.
Optionally, you can:
  • Click Criteria to specify the credible interval percentage and numerical method settings.
  • Click Priors to define reference and conjugate prior distribution settings.
  • Click Bayes Factor to specify Bayes factor settings.
  • Click Save to identify which items to save, and save model information to an XML file.
  • Click Predict to specify regressors for Bayesian prediction.
  • Click Plots to plot the posterior distributions of the regression parameters, the variance of error terms, and the predicted values.
  • Click F-tests to compare statistical models in order to identify the model that best fits the population from which is was sampled.
1 Lee, M.D., and Wagenmakers, E.-J. 2013. Bayesian Modeling for Cognitive Science: A Practical Course. Cambridge University Press.
2 Jeffreys, H. 1961. Theory of probability. Oxford University Press.