• We are pleased to announce that the winner of our Feedback Prize Draw for the Winter 2024-25 session and winning £150 of gift vouchers is Zhao Liang Tay. Congratulations to Zhao Liang. If you fancy winning £150 worth of gift vouchers (from a major UK store) for the Summer 2025 exam sitting for just a few minutes of your time throughout the session, please see our website at https://www.acted.co.uk/further-info.html?pat=feedback#feedback-prize for more information on how you can make sure your name is included in the draw at the end of the session.
  • Please be advised that the SP1, SP5 and SP7 X1 deadline is the 14th July and not the 17th June as first stated. Please accept out apologies for any confusion caused.

Maximum likelihood

M

madge

Member
I'm currently trying to work out how to do MLE's but having problems working out the first part of equation where you need to multiply the probability distribution n times. I just wondered if there was some rule for multiplying these distributions.

Any help really appreciated.
 
Not really a rule....

But the multiplying of distributions to get MLE's is a bit difficult, so instead must people take the log of the ditribution as this is a valid approach.

So say you are looking at Poisson pdf:

pdf = exp(-mu)*mu^X / X!

Then look at the MLE for say n observations - if we multiply this dist n times, we can write this expression using the pie symbol which means multiplication in this case:

π exp(-mu)*mu^Xi / Xi! ...., i = 1,2,3.......,n

Now when we take the log of something, we do it for each part of the eqn on either side of a multiplication or division operator. We then swap any multiplication signs for +'s, and any divisors for -'s.
E.g.

log(pdf) = log( exp(-mu)*mu^X / X!)

= log(exp(-mu)) + log(mu^X) - log(X!)

= -mu + Xlog(mu) - log(X!)

Now if we have put the pie operator inside our log expression, when we log the eqn, any xi's will be summed:


Log( exp(-mu)*mu^X / X!))

= -n*mu + ∑Xilog(mu) - log(∑Xi!)


Then to find the MLE estimator we differentiate w.r.t. the parameter we are interested in, in this case mu :

d(-n*mu + ∑Xilog(mu) - log(∑Xi!)) / dmu = -n + ∑Xi / mu

Then set this eqn equal to zero and solve for mu to get the MLE:

-n + ∑Xi / mu = 0

mu = ∑Xi / n

i.e. MLE(mu) = ∑Xi / n
 
Last edited by a moderator:
A tip for the exam is once you have worked out what the MLE is, differentiate the eqn you used for solving, again. If the result is an eqn which would yield a value less than zero then you have shown that it is definately a maximum and not a minimum or point of inflection.

E.g.

-n + ∑Xi / mu = 0

If we differentiate this again w.r.t. mu:

- ∑Xi / mu^2 < 0 ....... so implies a maximum


It's usually worth a couple of marks in the exam to point this last part out - and it will nearly always be a maximum too.
 
Back
Top