Construction of estimation functions

Maximum Likelihood method

• we estimate an unknown parameter \(\theta\)
• the value of the likelihood function L for given realizations \(x_1, \dots, x_n\) of the sample variables is determined by

\[L(\theta) = f(x_1, \dots, x_n|\theta) = f(x_1|\theta)\dots f(x_n|\theta) = \prod_{i = 1}^{n} f(x_i|\theta)\]

Here \(f(x_1|\theta)\) denotes the value of the probability density f at \(x_i\) under the assumption that the underlying true parameter value is \(\theta\).

To find the ML-estimator we need to derive the likelihood function after \(\theta\) and set to null.

---

ML-estimator for a exponential distribution where \(\lambda\) is searched

\(f(x|\lambda) = \prod_{i = 1}^{n} \lambda e ^{-\lambda x_i} = \lambda^n e^{-\lambda \sum x_i}\)
\(n - \lambda \sum_{i=1}^{n}x_i= 0\)
\(\lambda = \frac{1}{\overline{x}}\)

---

Likelihood for a binomial distribution where \(\pi\) is searched

\(L(\pi) = \pi^x(1-\pi)^{n-x}\)

---

Log-Likelihood method

• we estimate an unknown parameter \(\theta\)
• the value of the log-likelihood function L for given realizations \(x_1, \dots, x_n\) of the sample variables is determined by

\[ln L(\theta) = ln(f(x_1, \dots, x_n|\theta)) = \sum_{i = 1}^{n} ln f(x_i|\theta)\]

Back to Table of Contents