Soci709 Module 8 - GENERAL LINEAR TESTS

1.  THE NEED FOR MORE GENERAL TESTS

Given a linear model (omitting the i subscript)
Y = b0 + b1X1 + b2X2 + b3X3 + e
we can estimate the coefficients bk and easily test hypotheses of the form
H0: bk = 0
H1: bk <> 0
for each coefficient by looking on the regression printout at the p-value of the t-ratio  tk* = bk/s{bk} which is distributed as Student t with df=n-p where p is the total number of independent variables (including the constant term).
We can also test whether all bk are jointly equal to zero using the F-test for the regression model as a whole that is usually provided in the regression output.
There are categories of situations where we need more general tests of hypotheses involving the bk .

1.  Testing the Joint Significance of Several bk

There are many situations in which we want to test whether several regression coefficients are simultaneously equal to zero.
For example, X2 and X3 might represent: In all these situations we want to test hypotheses of the form
H0: b2 =  b3 = 0
H1: not both b2 and b3 = 0

2.  Testing That 2 or More bk Are Equal

The hypothesis to be tested is of the form
H0: b1 = b2
H1: b1 <> b2

3.  Testing That Some  Have Specific Values Other Than Zero

The hypothesis to be tested is of the form
H0:  b1 = 3,  b3 = 5
H1: not both equalities in H0 hold

2.  FULL & REDUCED MODELS

1.  Setting Full & Reduced Models

The general approach to testing simultaneous hypotheses on the coefficients is to use an F-test based on contrasting a full model with a reduced (aka restricted, aka constrained) model.  We explain the logic of the test in the case of testing the joint significance of several bk .  The hypothesis is
H0: b2 =  b3 = 0
H1: not both b2 and b3 = 0
To test whether several b2 and b3 are simultaneously zero we contrast
Y = b0 + b1X1 + b2X2 + b3X3  (full model, F)
Y = b0 + b1X1  (reduced model, R)
Q - Why would the reduced model R also be called the constrained model?

The test is based on a comparison of the SSE of the full and reduced models, denoted SSEF and SSER, respectively.
It is always true that SSEF <= SSER, because a model with more parameters always fit the data as well or better.  Thus

Thus the test is based on the difference between SSER and SSEF.
The test statistic is
F* = (SSER - SSEF)/(dfR - dfF) / (SSEF/dfF)  (8.1)
In words, F* is the ratio of the difference in SSE between reduced and full models divided by the difference in degrees of freedom between R and F, to the SSE of the full model divided by the degrees of freedom of F.
There is an equivalent formula for F* in terms of R2F and R2R, the coeffficients of determination of the full and reduced models, respectively.  We know that R2 = 1 - (SSE/SSTO), and that SSTO is the same in the full and reduced models.  From this one can derive from (8.1) the equivalent formula for F*
F* = (R2F - R2R)/(dfR - dfF) / ((1 - R2F)/dfF)   (8.2)
This formula is particularly useful to test hypotheses from published regression results.  (Also, it is the reason why one should not present the adjusted Ra2 alone in published reports, because it makes it more difficult for readers to recover F* if they wish.)

Q - Why is it that SSTO is the same in the full and reduced model?

From the ANOVA table we know that the df of SSE are n-p where p is the total number of variables including the constant.
For the example above

dfF = n-4
dfR = n-2
so
F* = (SSER - SSEF)/((n-2) - (n-4)) / (SSEF/(n-4))
F* = (SSER - SSEF)/2 / (SSEF/(n-4))
Note that the df of the difference between SSER and SSEF is equal to the number of parameters set to zero by the hypothesis.

2.  Carrying Out the Test

From the earlier discussion F* is distributed as F(dfR - dfF, dfF).
As usual one can test the hypothesis using two equivalent approaches: the p-value (Fisher inductive inference) approach and the critical-value (Neyman-Pearson decision theory) approach.
a.  P-value Approach
The p-value associated with F* is the probability of finding a value of F greater than F* if H0 is true or
P-value(F*) = P{F(dfR - dfF, dfF)>F*}
The decision rule is
if p-value(F*) >= a conclude H0      (8.3a)
if p-value(F*) < a conclude H1         (8.3b)
where a is the level of significance chosen.
The p-value approach is easiest when using the computer.
b.  Critical-value Approach
The critical-value associated with a chosen level of significance a is the value of F(dfR - dfF, dfF) such that, if H0 is true, the probability that F is less than the critical value is 1-a.  The critical-value is denoted
F(1-a; dfR - dfF, dfF)
The decision rule is
if F* <= F(1-a; dfR - dfF, dfF) conclude H0      (8.4a)
if F* > F(1-a; dfR - dfF, dfF) conclude H1        (8.4b)
The critical-value approach is easiest when using printed statistical tables of the F distribution.

Q - Why is that?

The strategy for general linear tests is therefore

  1. fit full model and obtain SSEF or R2F
  2. fit reduced model under H0 and obtain SSER or R2R
  3. calculate F* using (8.1) or (8.2)
  4. use decision rule (8.3) or (8.4)

3.  An Alternative Presentation - Extra Sums of Squares

In comparing a full model including X1, X2, and X3 with a reduced model including X1 only, the extra sum of squares of the full model, compared to the reduced model, is denoted SSR(X2, X3 | X1) and defined as
SSR(X2, X3 | X1) = SSE(X1) - SSE(X1, X2, X3)
The extra sum of squares SSR(X2, X3 | X1) is thus the reduction in SSE achieved by including X2 and X3 in a model that already contains X1.
One can derive the F-test in terms of SSR, but I am not using this approach in the course.  See NKNW Sections 7.1 to 7.3 (pp. 260-274) for an exposition emphasizing extra sums of squares.

3.  EXAMPLES OF TESTING JOINT HYPOTHESES

1.  Body Fat Example  (NKNW pp. 260-263)

In the model for body fat (Y) containing X1, X2, and X3 one wants to test the joint significance of X2 & X3.  The hypothesis setup is:
H0: b2 =  b3 = 0
H1: not both b2 and b3 = 0
From the full and reduced models one obtains
 
Full Reduced
SSE 98.404888 143.119703
R2 0.801359 0.711097
df of SSE = (n-p) 16 18

Using formula (8.1) with SSE one gets

F* = (143.119703 - 98.404888)/(18 - 16) / ((98.404888)/16) = 3.635
Equivalently, using formula (8.2) with R2 one gets
F* = (0.801359 - 0.711097)/(18 - 16) / ((1 - 0.801359)/16) = 3.635
a.  P-value Approach
We find the p-value of F* = 3.635 as
>calc 1 - fcf(3.635,2,16)
        0.049956
The p-value is just less than a = .05 so we conclude H1 using decision rule (8.3).  We say "The coefficients of X2 and X3 are jointly significant at the a = .05 level".
b.  Critical-value Approach
We choose a = .05.  To apply the decision rule we need the critical value F(0.95; 2, 16).  Using SYSTAT's calculator we find
>calc fif(0.95,2,16)
        3.633723
Since F* = 3.635 > 3.633723 (again, just barely!) we conclude H1 using decision rule (8.4).  We conclude that b2 and b3 are not both zero at the a = .05 level.

Note that while in the full model each coefficient b2 and b3 is individually non-significant, they are jointly significant.

2.  Testing a Polynomial Function

Table 3 from Nielsen's (1994) article on income inequality is reproduced in the next exhibit. We want to test the joint significance of the second-degree polynomial of energy consumption per capita.  We do this by comparing Model 8 (F) with Model 6 (R).  From the table we have R2F = .818; dfF = 56-9 = 47; R2R = .807; dfR = 49.  Thus
F* = ((.818 - .807)/2) / ((1 - .818)/47) = 1.4203
We find the p-value of F* as
>calc 1 - fcf(1.4203,2,47)
        0.251822
We conclude the polynomial coefficients are jointly non-significant.

3.  Testing Effect of Qualitative Variable with Multiple Indicators

This example uses the Afifi and Claaark data set.  The dependent variable is a depression score calculated as the square root of (total CESD score+1) (to normalize the distribution of the variable).  A multiple regression of depression score on education, logarithm of income, sex (female) and religion is estimated as
depscore = 3.872  +  (-.08)educatn + (-.896)l10inc +  (.376)female +  (.252)cath +  (.785)jewi + (.588)none  n=256 R2=.124
T-ratios are: (10.22)  (-1.26)               (-3.34)                 (2.35)                 (1.19)            (2.79)            (2.88)
Religion is represented by three (0,1) indicators for Catholic, Jewish and None (with Protestant as the omitted category).
1.  From the regression results one can tell that, at the .05 level, there is no significant difference in depression score between Catholic and Protestant, but there are significant differences between Jewish and Protestant and between None and Protestant.  (Q -- How can one tell?)
2.  One also wants to test whether religion (jointly represented by the three indicators) is a significant predictor of depression.  To do this the hypothesis setup is
H0: b4 =  b5 =  b6 = 0
H1: not all three b's = 0
To do this one could estimate the reduced model corresponding to the null hypothesis and use the formula above; one can also use STATA (test cath jewi none) to test the joint significance of the three indicators.  The program calculates the F-test as F* = 4.40, P{F(3, 249) > F} = 0.0049.  Thus one concludes that religion is a significant predictor of the depression score (at the .05 and even .01 level).
3.  Given that their coefficients are both large, one may also want to test whether Jewish and None are equally predisposed to depression.  The hypothesis setup is
H0: b5 =  b6
H1: b5 <>  b6
Using STATA (test none=jewi) yields F* = .4, P{F(1, 249)>F*} = .526.  Thus one concludes that there is no significant difference between categories Jewish and None with respect to depression score.
4.  Suppose one has theological or other reasons to believe that Jewish is more vulnerable to depression than Catholic.  The test for equality of the coefficients for these two categories yields F* = 2.81,  P{F(1, 249)>2.81} = .0949, so the hypothesis of equality cannot be rejected at the .05 level.  Can the hypothesis that Jewish is more vulnerable to depression than Catholic be rejected at the .05 level? 
Exhibit: Stata commands for testing effects of religion on depression score

4.  TESTS ON REGRESSION COEFFICIENTS USING FULL VS. REDUCED MODEL

General linear tests can be cast as a comparison of full and reduced model.

1. Test Whether a Single bk = 0

H0: bk = 0
H1: bk<> 0
This is the usual test reported as the p-value of tk* = bk/s{bk} on the regression printout.  One can show that the corresponding F* from the full vs. reduced model comparison is equal to the square of tk*, i.e., F* = (tk*)2.  Thus the t-test and F-test for a single coefficient are equivalent.

2.  Test Whether All bk = 0

H0: b1 = b2 = ... = bp-1 = 0
H1: not all bk (k = 1, ..., p-1) = 0
This is the usual test reported as the p-value of F* = MSR/MSE on the regression printout.  It follows as a special case of the general formula in which the full model has SSE(X1, X2, ..., Xp-1) with df=n-p and the reduced model has SSE = SSTO with df=n-1.

3.  Test Whether Some bk = 0

H0: bq = bq+1 = ... = bp-1 = 0
H1: not all of the bk in H0 = 0
(The notation assumes that the variables are arranged so that the tested variables have subscripts q to p-1.)  This is the situation discussed earlier.

Other tests can be carried out as a comparison of full & reduced model, using "tricks".

4.  Test Equality of 2 Coefficients

H0: b1 = b2
H1: b1 <> b2
The full model is (omitting the i subscript)
Y = b0 + b1X1 + b2X2 + b3X3 + e
The trick is to define the reduced model as
Y = b0 + bc (X1 + X2) + b3X3 + e
where bc is the "common" regression coefficient of X1 and X2 under H0.  One estimates the reduced model as the regression of Y on a new variable calculated as the sum of X1 and X2.  Then one calculates F* using formula (8.1) or (8.2).  The ful model has df=n-4 and the reduced model has df=n-3 so F* has df=(1, n-4).

5.  Test Whether Some bk Have Specific Values Other than 0

H0:  b1 = 3,  b3 = 5
H1: not both equalities in H0 hold
With the full model as above, one derives the reduced model by replacing b1 and b3 by their assumed values under H0 and removing their effects from the dependent variable, as
W = Y - 3X1 - 5X3 =  b0 + b2X2 + e
where W is the new dependent variable.  The reduced model is estimated as the regression of W on X2.  Then one calculates F* which has df=(2, n-4).

5.  USING THE HYPOTHESIS COMMAND IN SYSTAT

The hypothesis command in SYSTAT implements the general linear test.  To use it one only needs to estimate the full model.  The hypothesis command applies to the last model estimated.  A test begins with the hypothesis command and ends with the test command, with the specifics of the test in between.  For the following examples assume that the full model has 5 independent variables X1 to X5.  First estimate the full model as

model y = constant + x1 + x2 + x3 + x4 + x5
estimate

One way to test the hypothesis that the coefficient of X1 is zero is to use the effect command
hypothesis
effect = x1
test
An alternative method is to use the specify command
hypothesis
specify x1 = 0
test
Note that this test only repeats the test already on the regression printout.

To test the hypothesis that the coefficients of X1, X3, and X4 are simultaneously equal to zero using the effect command
hypothesis
effect = x1&x3&x4
test
Or, using the specify command:
hypothesis
specify x1 = 0; x3 = 0; x4 = 0
test
Note that equalities are listed on the same line separated by semicolons.

Testing more complicated hypotheses involving equality of coefficients or whether a coefficient has a specific nonzero value is done using the specify command.

To test that the coefficients of X2 and X3 are equal, i.e. that b2 - b3 = 0
hypothesis
specify x2 - x3 = 0
test

To test whether the coefficient of X3 is 3.5 times as large as the coefficient of X5, i.e. that b3 - 3.5b5 = 0
hypothesis
specify x3 - 3.5*x5 = 0
test

To test that coefficients have specific values, for example that b1=4 and b3=17, use the commands
hypothesis
specify x1 = 4 ; x3 = 17
test

To test that the difference between coefficient of X2 and X3 is equal to the specific value 20, use the commands
hypothesis
specify x2 - x3 = 20
test

Examples of actual tests are shown in the next exhibits.

6.  MATRIX FORMULATION OF GENERAL LINEAR TEST (OPTIONAL)

General linear hypothesis tests in SYSTAT use an underlying approach based on matrices.  You can see these matrices in the output from the HYPOTHESIS command.  The null hypothesis H0 for any linear hypothesis can be represented by specifying a matrix A and a vector d.  Then the null hypothesis H0 is represented as
H0: Ab = d
where A is sxp, b is px1, and d is sx1; s is the number of constraints on the coefficients.
(In SYSTAT A and d are specified by the commands AMATRIX and DMATRIX, respectively.   See SYSTAT V6/V7 - Statistics,  pp. 284-289.)

Various specifications of A and d are shown in the following examples, based on a full model with a constant term and variables X1, X2, and X3.

EX:  H0: b1 = 0
A = [0 1 0 0]  d = [0]

EX:  H0: b1 = b2 = 0
A =
0
1
0
0
0
0
1
0
d' = [0 0]

EX: H0: b1 = b2
A = [0 1 -1 0]  d = [0]

The curent edition of NKNW no longer presents this material.  The following 3 pages from an older edition (Neter, Wasserman, and Kutner 1990, pp. 306-308) derive the general linear test in matrix notation.  (NWK use the notation C for A and h for d.)



Last modified 20 Mar 2006