41  GLM. Deviance

Deviance \[D = 2 \left(l(\boldsymbol{\hat \beta_{sat}}) - l(\boldsymbol{\hat \beta}) \right) \phi\]

41.1 Gaussian data

\[Y_i \sim N(\mu_i, \sigma^2),\ \ f(y_i) = \frac{1 }{\sigma \sqrt{2\pi}}exp\left(-(y_i-\mu_i)^2/2\sigma^2 \right)\]

The scaled deviance is

\[D^* = 2 \sum_i \left(log\left(\frac{1}{\sigma\sqrt{2\pi}}\right) - \frac{(y_i-y_i)^2}{2\sigma^2}\right) - 2 \sum_i \left(log\left(\frac{1}{\sigma\sqrt{2\pi}}\right) - \frac{(y_i- \hat \mu_i)^2}{2\sigma^2}\right) = \sum_i \frac{(y_i-\hat \mu_i)^2}{\sigma^2} \]

The deviance is

\[D= \sigma^2 D^* = \sum_i (y_i-\hat \mu_i)^2\]

41.2 Poisson data

\[Y_i \sim Po(\lambda_i),\ \ P(Y_i = y_i) = \frac{exp(-\lambda_i)\lambda_i^{y_i}}{y_i!}\]

Poisson log likelihood

\[l= \sum_{i=1}^n(y_i \log(\lambda_i)-\lambda_i -\log(y_i!))\]

Residual Deviance

\[D = 2 \sum_i \left(y_i \log( y_i) - y_i - \log(y_i!)\right) -2 \sum_i \left(y_i \log(\hat \lambda_i) - \hat \lambda_i - \log(y_i!)\right) = 2 \sum_i \left(y_i (\log(y_i/\hat \lambda_i) - (y_i - \hat \lambda_i)) \right)\]

41.3 Binomial data

\[Y_i \sim Binomial(n_i, \pi_i),\ \ \mu_i = n_i \pi_i ,\ \ P(Y_i = y_i) = {n_i \choose y_i} \pi_i^{y_i} (1-\pi_i)^{n_i-y_i}\]

Binomial log likelihood

\[l(\boldsymbol{\pi}; \boldsymbol{y}) = \sum_{i=1}^n \left(y_i \log(\pi_i) + (n_i - y_i) \log(1-\pi_i) + \log {n_i \choose y_i}\right)\]

\[l(\boldsymbol{\mu}; \boldsymbol{y}) = \sum_{i=1}^n \left(y_i \log\left(\frac{\mu_i}{n_i} \right) + (n_i - y_i) \log\left(\frac{n_i - \mu_i}{n_i} \right) + \log {n_i \choose y_i}\right)\]

Residual Deviance

\[D = 2 \sum \left(y_i \log(y_i/n_i) + (n_i - y_i) \log((n_i-y_i)/n_i) \right) - 2 \sum \left(y_i \log(\hat \mu_i/n_i) + (n_i - y_i) \log((n_i- \hat \mu_i)/n_i) \right)\]

\[D = 2 \sum \left(y_i \log\left(\frac{y_i}{\hat \mu_i}\right) + (n_i - y_i) \log\left(\frac{n_i-y_i}{n_i- \hat \mu_i}\right) \right)\]