Maximum likelihood

I'm currently trying to work out how to do MLE's but having problems working out the first part of equation where you need to multiply the probability distribution n times. I just wondered if there was some rule for multiplying these distributions.

Any help really appreciated.

 

Not really a rule....

But the multiplying of distributions to get MLE's is a bit difficult, so instead must people take the log of the ditribution as this is a valid approach.

So say you are looking at Poisson pdf:

pdf = exp(-mu)*mu^X / X!

Then look at the MLE for say n observations - if we multiply this dist n times, we can write this expression using the pie symbol which means multiplication in this case:

π exp(-mu)*mu^Xi / Xi! ...., i = 1,2,3.......,n

Now when we take the log of something, we do it for each part of the eqn on either side of a multiplication or division operator. We then swap any multiplication signs for +'s, and any divisors for -'s.
E.g.

log(pdf) = log( exp(-mu)*mu^X / X!)

= log(exp(-mu)) + log(mu^X) - log(X!)

= -mu + Xlog(mu) - log(X!)

Now if we have put the pie operator inside our log expression, when we log the eqn, any xi's will be summed:


Log(∏ exp(-mu)*mu^X / X!))

= -n*mu + ∑Xilog(mu) - log(∑Xi!)


Then to find the MLE estimator we differentiate w.r.t. the parameter we are interested in, in this case mu :

d(-n*mu + ∑Xilog(mu) - log(∑Xi!)) / dmu = -n + ∑Xi / mu

Then set this eqn equal to zero and solve for mu to get the MLE:

-n + ∑Xi / mu = 0

mu = ∑Xi / n

i.e. MLE(mu) = ∑Xi / n

 

A tip for the exam is once you have worked out what the MLE is, differentiate the eqn you used for solving, again. If the result is an eqn which would yield a value less than zero then you have shown that it is definately a maximum and not a minimum or point of inflection.

E.g.

-n + ∑Xi / mu = 0

If we differentiate this again w.r.t. mu:

- ∑Xi / mu^2 < 0 ....... so implies a maximum


It's usually worth a couple of marks in the exam to point this last part out - and it will nearly always be a maximum too.