Wyprowadzić wariancję współczynnika regresji w prostej regresji liniowej


38

W prostej regresji liniowej mamy y=β0+β1x+u , gdzie uiidN(0,σ2) . Wyprowadziłem estymator:

β1^=i(xix¯)(yiy¯)i(xix¯)2 ,
gdziex¯ iy¯ to średnie próbkixiy.

Teraz chcę, aby znaleźć wariancję beta 1 . Wyprowadziłem coś takiego: Var ( ^ β 1 ) = σ 2 ( 1 - 1β^1

Var(β1^)=σ2(11n)i(xix¯)2 .

Pochodna jest następująca:

Var(β1^)=Var(i(xix¯)(yiy¯)i(xix¯)2)=1(i(xix¯)2)2Var(i(xix¯)(β0+β1xi+ui1nj(β0+β1xj+uj)))=1(i(xix¯)2)2Var(β1i(xix¯)2+i(xix¯)(uijujn))=1(i(xix¯)2)2Var(i(xix¯)(uijujn))=1(i(xix¯)2)2×E[(i(xix¯)(uijujn)E[i(xix¯)(uijujn)]=0)2]=1(i(xix¯)2)2E[(i(xix¯)(uijujn))2]=1(i(xix¯)2)2E[i(xix¯)2(uijujn)2] , since ui 's are iid=1(i(xix¯)2)2i(xix¯)2E(uijujn)2=1(i(xix¯)2)2i(xix¯)2(E(ui2)2×E(ui×(jujn))+E(jujn)2)=1(i(xix¯)2)2i(xix¯)2(σ22nσ2+σ2n)=σ2i(xix¯)2(11n)

Czy zrobiłem tutaj coś złego?

Var(β1^)=σ2i(xix¯)2. But I am trying to derive the answer without using the matrix notation just to make sure I understand the concepts.


2
Yes, your formula from matrix notation is correct. Looking at the formula in question, 11n=n1n so it rather looks as if you might used a sample standard deviation somewhere instead of a population standard deviation? Without seeing the derivation it's hard to say any more.
TooTone

General answers have also been posted in the duplicate thread at stats.stackexchange.com/questions/91750.
whuber

Odpowiedzi:


35

At the start of your derivation you multiply out the brackets i(xix¯)(yiy¯), in the process expanding both yi and y¯. The former depends on the sum variable i, whereas the latter doesn't. If you leave y¯ as is, the derivation is a lot simpler, because

i(xix¯)y¯=y¯i(xix¯)=y¯((ixi)nx¯)=y¯(nx¯nx¯)=0

Hence

i(xix¯)(yiy¯)=i(xix¯)yii(xix¯)y¯=i(xix¯)yi=i(xix¯)(β0+β1xi+ui)

and

Var(β1^)=Var(i(xix¯)(yiy¯)i(xix¯)2)=Var(i(xix¯)(β0+β1xi+ui)i(xix¯)2),substituting in the above=Var(i(xix¯)uii(xix¯)2),noting only ui is a random variable=i(xix¯)2Var(ui)(i(xix¯)2)2,independence of ui and, Var(kX)=k2Var(X)=σ2i(xix¯)2

which is the result you want.


As a side note, I spent a long time trying to find an error in your derivation. In the end I decided that discretion was the better part of valour and it was best to try the simpler approach. However for the record I wasn't sure that this step was justified

=.1(i(xix¯)2)2E[(i(xix¯)(uijujn))2]=1(i(xix¯)2)2E[i(xix¯)2(uijujn)2] , since ui 's are iid
because it misses out the cross terms due to jujn.

I noticed that I could use the simpler approach long ago, but I was determined to dig deep and come up with the same answer using different approaches, in order to ensure that I understand the concepts. I realise that first juj^=0 from normal equations (FOC from least square method), so u^¯=iuin=0, plus u^¯=y¯y^¯=0, so y¯=y^¯. So there won't be the term jujn in the first place.
mynameisJEFF

ok, in your question the emphasis was on avoiding matrix notation.
TooTone

Yes, because I was able to solve it using matrix notation. And notice from my last comment, I did not use any linear algebra. Thanks for your great answer anyway^.^
mynameisJEFF

sorry are we talking at cross-purposes here? I didn't use any matrix notation in my answer either, and I thought that was what you were asking in your question.
TooTone

sorry for misunderstanding haha...
mynameisJEFF

2

I believe the problem in your proof is the step where you take the expected value of the square of i(xix¯)(uijujn). This is of the form E[(iaibi)2], where ai=xix¯;bi=uijujn. So, upon squaring, we get E[i,jaiajbibj]=i,jaiajE[bibj]. Now, from explicit computation, E[bibj]=σ2(δij1n), so E[i,jaiajbibj]=i,jaiajσ2(δij1n)=iai2σ2 as iai=0.


2

Begin from "The derivation is as follow:" The 7th "=" is wrong.

Because

i(xix¯)(uiu¯)

=i(xix¯)uii(xix¯)u¯

=i(xix¯)uiu¯i(xix¯)

=i(xix¯)uiu¯(ixinx¯)

=i(xix¯)uiu¯(ixiixi)

=i(xix¯)uiu¯0

=i(xix¯)ui

So after 7th "=" it should be:

1(i(xix¯)2)2E[(i(xix¯)ui)2]

=1(i(xix¯)2)2E(i(xix¯)2ui2+2ij(xix¯)(xjx¯)uiuj)

=1(i(xix¯)2)2E(i(xix¯)2ui2)+2E(ij(xix¯)(xjx¯)uiuj)

=1(i(xix¯)2)2E(i(xix¯)2ui2), because ui and uj are independent and mean 0, so E(uiuj)=0

=1(i(xix¯)2)2(i(xix¯)2E(ui2))

σ2(i(xix¯)2)2


1
It might be helpful if you edited your answer to include the correct line.
mdewey

Your answer is being automatically flagged as low quality because it's very short. Please consider expanding on your answer
Glen_b -Reinstate Monica
Korzystając z naszej strony potwierdzasz, że przeczytałeś(-aś) i rozumiesz nasze zasady używania plików cookie i zasady ochrony prywatności.
Licensed under cc by-sa 3.0 with attribution required.