Conjugate Prior for Poisson Distribution

 

Conjugate Prior for Poisson

We derive the conjugate prior for a Poisson likelihood and compute the posterior distribution step by step.

---

Likelihood: Poisson Model

Suppose we observe counts

$$x_1,x_2,\dots,x_n \sim \text{Poisson}(\lambda)$$

Poisson PMF:

$$p(x|\lambda)=\frac{\lambda^x e^{-\lambda}}{x!}, \quad x=0,1,2,\dots$$

For independent data:

$$L(\lambda)=\prod_{i=1}^n \frac{\lambda^{x_i} e^{-\lambda}}{x_i!}$$ $$L(\lambda)\propto \lambda^{\sum x_i} e^{-n\lambda}$$

Let

$$S=\sum_{i=1}^n x_i$$

Then

$$L(\lambda)\propto \lambda^{S} e^{-n\lambda}$$

---

Goal: Find Conjugate Prior

A conjugate prior must have the same functional form in $\lambda$ as the likelihood.

Look at likelihood kernel:

$$\lambda^{S} e^{-n\lambda}$$

We want a prior of form:

$$p(\lambda)\propto \lambda^{\alpha-1} e^{-\beta\lambda}$$

This is exactly the Gamma distribution.

---

Choose Prior: Gamma Distribution

Let

$$\lambda \sim \text{Gamma}(\alpha,\beta)$$

PDF:

$$p(\lambda)=\frac{\beta^\alpha}{\Gamma(\alpha)} \lambda^{\alpha-1} e^{-\beta\lambda}$$

Ignoring constants:

$$p(\lambda)\propto \lambda^{\alpha-1} e^{-\beta\lambda}$$

---

Posterior via Bayes Rule

$$p(\lambda|x) \propto L(\lambda)\,p(\lambda)$$

Substitute:

$$p(\lambda|x)\propto \lambda^{S} e^{-n\lambda}\cdot \lambda^{\alpha-1} e^{-\beta\lambda}$$

Combine powers:

$$p(\lambda|x)\propto \lambda^{S+\alpha-1} e^{-(n+\beta)\lambda}$$

---

Recognize Posterior Form

This matches Gamma distribution:

$$\lambda|x \sim \text{Gamma}(\alpha+S,\ \beta+n)$$

---

Final Result

✅ Prior

$$\lambda \sim \text{Gamma}(\alpha,\beta)$$

✅ Posterior

$$\lambda|x \sim \text{Gamma}(\alpha+\sum x_i,\ \beta+n)$$

---

Interpretation

Quantity	Meaning
$\alpha$	prior pseudo-count of events
$\beta$	prior exposure/time
$S=\sum x_i$	observed total events
$n$	number of observations

Posterior = prior + data

$$\text{shape: } \alpha + S$$
$$\text{rate: } \beta + n$$

---

Posterior Mean

Gamma mean:

$$E[\lambda|x]=\frac{\alpha+S}{\beta+n}$$

This equals:

$$\text{weighted average of prior mean and sample mean}$$

---

Numerical Example

Suppose:

• Prior: $\lambda\sim\text{Gamma}(2,1)$
  

• Observations: $x=[3,2,4]$
  

Then:

$$S=3+2+4=9,\quad n=3$$

Posterior:

$$\lambda|x\sim\text{Gamma}(11,4)$$

Posterior mean:

$$11/4=2.75$$

---

Why Gamma is conjugate

Because both prior and likelihood contain:

$$\lambda^{(\cdot)} e^{-(\cdot)\lambda}$$

Multiplying them preserves this form → remains Gamma.

This is the essence of conjugacy.

---

 Summary

For Poisson likelihood, the Gamma prior is conjugate, and the posterior is obtained by simply adding observed counts to the prior parameters.