39.5. LINEAR REGRESSION 825

Proof: Suppose the corollary is true for m−1, where m−1≥ 2.

A = A1 +(A2 + · · ·+Am)≡ A1 +B.

Then doing the same argument as above, you find that A1B = 0 and B2 = B since XT BXis X 2 (r− r1). Now

B = A2 + · · ·+Am

and there are only m−1 in the sum on the right. By induction, Am is

X 2

(r− r1−

(m−1

∑k=2

rk

))

and XT AkX are independent for k between 2 and m because AkA j = 0 for such k, j. RecallProposition 39.4.2. It only remains to verify that A1Ak for 2 ≤ k ≤ m. You could dothe same argument in the form A = A2 +(A1 +A3 + · · ·+Am) and conclude that A1Ak =0 for 3 ≤ k ≤ m− 1. Then all that is left is A1A2. Just do the argument again for A =A3 +(A1 +A2 +A4 + · · ·+Am) and conclude in particular that A1A2 = 0. Thus all mixedproducts are 0 and so the quadratic forms are independent. ■

Summary 39.4.8 The following are the main ideas in this section. In all of this,

X =(

X1 · · · Xn

)T

where the Xi are independent and n(0,σ2

)and the matrices are symmetric

1. For {Ak} symmetric matrices,{XT AkX

}are independent if and only if AkA j = 0

for each k ̸= j.

2. XT AXσ2 is X 2 (r) if and only if A2 = A and the rank of A is r.

3. If xT A x= ∑mk=1x

T Akx and XT AX is X 2 (r) ,XT AkX is X 2 (rk) where k ≤m−1 and ∑

m−1k=1 rk < r, then XT AmX is X 2

(r−∑

m−1k=1 rk

)and the random variables{

XT AkX}

are independent.

39.5 Linear RegressionThis will be an interesting and important application of the above theory of quadratic forms.The idea is you have finitely many times ti, t1 ≤ t2 ≤ ·· · ≤ tn and there are independentrandom variables X1, · · · ,Xn, corresponding to these ti. More generally, the ti are simplyreal numbers, but often the interpretation is time. Assume the following condition.

Condition 39.5.1 The random variables Xi associated with time tk are

n(α +β (tk− t̄) ,σ2)

where t̄ ≡ 1n ∑

nk=1 tk. There may be many Xi associated with a single tk but it is assumed

that they are independent and normally distributed with a mean which depends on the tkbut the variance is constant. Note that the tk might be repeated in the list. Typically theyare repeated because one is taking a sample larger than one for each tk.