Stepwise Regression Through Backward Elimination.
The SalesTerritory data set has the goal
of modeling sales with the help of 8 predictor variables.
Since the predictor variables are intercorrelated, a multiple regression model allocates
their shared predictive powerin in unpredictable ways that have little real world
We can see this using the Split Half
method of validation:
Run the regression three times, once with all 25 observations, once
with just half of them, and one with just the other half of the
observations. Then compare the coefficients. The fact that
some of them switch between positive and negative depending on which
half of the data are used is a sure sign of multicollinearity.
The Stepwise Regression approach tries to find a reduced model that
best explains the data. We will focus on "Backward Elimination,"
also known as "Stepwise Deletion. There are three criteria
for eliminating a predictor variable:
1. Eliminating it raises Adjusted R^2
2. Eliminating it lowers the Standard error ofthe estimate
3. The absolute value of the t statistic for the variable is less than
Criteria 2 and 3 are mathematically equivalent. See StepWise.xls
The effectiveness of the reduced model is demonstrated by SplitReduced.xls