Beta Distribution

 

Beta Distribution — Detailed Explanation

1. What Is the Beta Distribution?

The Beta distribution is a continuous probability distribution defined on the interval:

$$0 \le x \le 1$$

It is used to model probabilities, proportions, and beliefs about quantities that lie between 0 and 1.

Typical Uses

• Probability of success $p$ in Bernoulli trials

• Click-through rates

• Bias of a coin

• Conversion rates

• Bayesian prior for binomial models

---

2. Why Is It Important?

In Bayesian statistics, the Beta distribution is the conjugate prior for:

• Bernoulli distribution

• Binomial distribution

This makes Bayesian updating analytically simple.

---

3. Probability Density Function (PDF)

A random variable $X\sim \text{Beta}(\alpha ,\beta )$ has the PDF:

$$\boxed{ f(x;\alpha,\beta) = \frac{1}{B(\alpha,\beta)} x^{\alpha-1} (1-x)^{\beta-1}, \quad 0 \le x \le 1 }$$

Where:

• $\alpha > 0$: shape parameter

• $\beta >\mathrm{0:\; shape\; parameter}$

• $B(\alpha,\beta)$: Beta function

---

4. The Beta Function

$$B(\alpha,\beta) = \int_0^1 x^{\alpha-1}(1-x)^{\beta-1}dx$$

It normalizes the distribution so total probability equals 1.

Relation to Gamma function:

$$B(\alpha,\beta) = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$$

---

5. Interpretation of Parameters $\mathrm{\alpha }$and $\beta$

Intuitive Meaning

Parameter	Interpretation
$\alpha - 1$	Number of prior successes
$\beta - 1$	Number of prior failures

📌 The Beta distribution encodes belief about probability.

---

6. Mean and Variance

Mean

$$\boxed{ \mathbb{E}[X] = \frac{\alpha}{\alpha + \beta} }$$

Variance

$$\boxed{ \text{Var}(X) = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)} }$$

---

7. Shapes of the Beta Distribution

$\alpha$	$\beta$	Shape
1	1	Uniform
>1	>1	Bell-shaped
<1	<1	U-shaped
>1	<1	Skewed right
<1	>1	Skewed left

---

8. Special Cases

Uniform Distribution

$$\text{Beta}(1,1) = \text{Uniform}(0,1)$$

---

Coin Toss Prior

$$\text{Beta}(1,1) \Rightarrow \text{no prior bias}$$

---

9. Beta Distribution in Bayesian Inference

Bernoulli Likelihood

$$x_i \sim \text{Bernoulli}(p)$$

Prior

$$p \sim \text{Beta}(\alpha,\beta)$$

Posterior

$$\boxed{ p \mid D \sim \text{Beta}(\alpha + k,\; \beta + n - k) }$$

Where:

• $k$ = number of successes

• $n$ = total trials

📌 This simple update is why Beta is so powerful.

---

10. Worked Example (Coin Toss)

Prior Belief

$$p \sim \text{Beta}(2,2)$$

→ Slight belief that coin is fair

---

Observed Data

• 7 heads

• 3 tails

---

Posterior

$$p \sim \text{Beta}(2+7,\; 2+3) = \text{Beta}(9,5)$$

---

Posterior Mean

$$\mathbb{E}[p] = \frac{9}{14} \approx 0.64$$

---

11. MAP Estimate

$$\boxed{ p_{\text{MAP}} = \frac{\alpha-1}{\alpha+\beta-2} \quad \text{(for } \alpha,\beta > 1) }$$

---

12. Connection to Machine Learning

Regularization

• Beta prior ≈ regularization on probabilities

Logistic Regression (Bayesian)

• Prior on class probability

A/B Testing

• Posterior over conversion rate

---

13. Comparison with Gaussian Distribution

Feature	Beta	Gaussian
Support	[0,1]	$(-\infty,\infty)$
Used for	Probabilities	Real values
Conjugate to	Bernoulli/Binomial	Gaussian

---

14. Key Takeaways (Exam-Friendly)

• Beta distribution models uncertainty over probabilities

• Defined on [0,1]

• Parameters act like pseudo-counts

• Conjugate prior for Bernoulli/binomial models

• Central to Bayesian learning

---

One-Line Intuition for Students

The Beta distribution represents our belief about a probability before and after seeing data