Mean Square Error

[kartik_newpro]

Hello again. This time I have a problem with the mean square error. It starts with biased and unbiased estimators. I am assuming that the purpose of using MLEs and other parameter estimation techniques is to derive a "single value" for the parameter of the concerned distribution.

And the property of unbiasedness precisely gives E[g(X)] = parameter

Then why would a value that is not equal to the true value, even though a small spread, be a better estimate?

The Acted notes say "some estimates are too large and some estimates are too small - but on AVERAGE they give the true value" and true value is what we want right?

And MSE is the measure of the spread of the estimates. Then, in example 10.10 MSE of the Poi(mu) distribution is (mu)/n. What does this tell me about how far I am from the true value?

And I find the notes too technical through these topics (Bias, MSE, CRLB). I am still trying to understand the purpose behind doing all this and its practical application.

 

[kartik_newpro]

POINT ESTIMATION

EXAM TYPE QUESTION

A random sample of x1,x2,.....,xn is taken from a population, which has the probability function F(x) and the density function f(x). The values in the sample are arranged in order and the minimum and maximum values xmin and xmax are recorded.

(i) Show that the distribution function of X max is [F(x)]^n.

Sol.
Consider the value of Xmax. This is will be less than some value x, say, if and only if all the sample values are less than x (Pretty obvious). The probability of this happening is just [F(x)]^n.

How did the notes arrive at this conclusion?

Awaiting reply for both queries. Thanks

 

[freddie]

When we try to estimate the true parameter value (say the population mean mu) we use our sample values (say the sample mean x bar). Every sample is different so x bar is a random variable. Using the central limit theorem x bar will be approximately normally distributed with a mean of mu and a variance (and MSE) of sigma^2/n. (The MSE also equals sigma^2/n since x bar is an unbiased estimator of mu.) This means that on average x bar will equal mu but of course there will be a spread around mu. The higher the sample size n, the smaller the spread around mu, so the more confident we can feel about our estimate.

In your second query, P(X<x) = F(x), so the probability that n independent observations are all less than x = [F(x)]^n.

 

[kartik_newpro]

When we try to estimate the true parameter value (say the population mean mu) we use our sample values (say the sample mean x bar). Every sample is different so x bar is a random variable. Using the central limit theorem x bar will be approximately normally distributed with a mean of mu and a variance (and MSE) of sigma^2/n. (The MSE also equals sigma^2/n since x bar is an unbiased estimator of mu.) This means that on average x bar will equal mu but of course there will be a spread around mu. The higher the sample size n, the smaller the spread around mu, so the more confident we can feel about our estimate.

In your second query, P(X<x) = F(x), so the probability that n independent observations are all less than x = [F(x)]^n.

Thanks freddie. That helps me a lot.

 

[Calum]

The Acted notes say "some estimates are too large and some estimates are too small - but on AVERAGE they give the true value" and true value is what we want right?

Yes, but remember that in practice you only have *one* estimate. If that estimate has huge variance, what information does that value really tell you? In such a case, an estimator which is theoretically biased but much likelier to be close to the population statistic might be a better choice.

I once had a maths lecturer who said that certain topics are "tunnels": you go into them, you can't see daylight, but if you keep on going, you will get through to the other end and you'll be in a place you would never have arrived at otherwise. Stick at it, it will make sense eventually.

 

[kartik_newpro]

Yes, but remember that in practice you only have *one* estimate. If that estimate has huge variance, what information does that value really tell you? In such a case, an estimator which is theoretically biased but much likelier to be close to the population statistic might be a better choice.

I once had a maths lecturer who said that certain topics are "tunnels": you go into them, you can't see daylight, but if you keep on going, you will get through to the other end and you'll be in a place you would never have arrived at otherwise. Stick at it, it will make sense eventually.

He is quite right. I am starting to make some sense. Thanks calum