What is a Probability Generating Function?
A probability generating function (PGF) is a power series in a dummy variable t whose coefficients are the probabilities of a discrete random variable X taking non-negative integer values.
Definition — Probability Generating Function
G_X(t) = E[t^X] = Σ P(X=r) · tʳ for r = 0, 1, 2, …
= p₀ + p₁t + p₂t² + p₃t³ + ···
where pᵣ = P(X = r). Valid for |t| ≤ 1 (always converges at t = 1 since Σpᵣ = 1).
The PGF is only defined for discrete random variables taking non-negative integer values (0, 1, 2, …). It is not defined for continuous random variables or for variables taking negative values.
Why is the PGF Useful?
Encodes distribution
The coefficient of tʳ in G(t) is exactly P(X = r). Two different distributions cannot have the same PGF — the PGF uniquely determines the distribution.
Moments via derivatives
E(X) and Var(X) are read off by differentiating G(t) and evaluating at t = 1. No need to sum an infinite series.
Sums of variables
If X and Y are independent, G_{X+Y}(t) = G_X(t) · G_Y(t). Products of PGFs correspond to sums of independent random variables.
Key Values of G(t)
Properties at Special Values of t
G(0)
G(0) = P(X = 0) = p₀
Setting t=0 gives the probability that X takes the value 0.
Setting t=0 gives the probability that X takes the value 0.
G(1)
G(1) = Σ pᵣ = 1
Setting t=1 gives the sum of all probabilities — always equals 1. Use this to find unknown constants.
Setting t=1 gives the sum of all probabilities — always equals 1. Use this to find unknown constants.
G'(1)
G'(1) = E(X) = μ
The first derivative evaluated at t=1 gives the mean.
The first derivative evaluated at t=1 gives the mean.
G''(1)
G''(1) = E[X(X−1)] = E(X²) − E(X)
So E(X²) = G''(1) + G'(1), and Var(X) = G''(1) + G'(1) − [G'(1)]².
So E(X²) = G''(1) + G'(1), and Var(X) = G''(1) + G'(1) − [G'(1)]².
💡 Variance Formula via PGF
Standard form
Var(X) = G''(1) + G'(1) − [G'(1)]²
Memorise this. Cambridge will ask you to derive E(X) and Var(X) from a given PGF by differentiating and evaluating at t = 1.
E
Reading Probabilities from a PGF
A discrete RV X has PGF G(t) = 0.2 + 0.5t + 0.2t² + 0.1t³.
(i) State the distribution of X. (ii) Find E(X). (iii) Verify G(1) = 1.
(i) State the distribution of X. (ii) Find E(X). (iii) Verify G(1) = 1.
Part (i)
Reading coefficients: P(X=0)=0.2, P(X=1)=0.5, P(X=2)=0.2, P(X=3)=0.1
Part (ii)
G'(t) = 0.5 + 0.4t + 0.3t²
E(X) = G'(1) = 0.5 + 0.4 + 0.3 =
E(X) = G'(1) = 0.5 + 0.4 + 0.3 =
1.2
Part (iii)
G(1) = 0.2 + 0.5 + 0.2 + 0.1 = 1 ✓
Key Properties
Extracting Moments — Differentiation Method
Given G(t) = E[tˣ] = Σ pᵣ tʳ, differentiate term by term:
Differentiation — Deriving E(X) and Var(X)
G(t)
p₀ + p₁t + p₂t² + p₃t³ + ···
G'(t)
p₁ + 2p₂t + 3p₃t² + ··· = Σ r·pᵣ·t^(r−1)
At t=1: G'(1) = Σ r·pᵣ = E(X)
At t=1: G'(1) = Σ r·pᵣ = E(X)
G''(t)
2p₂ + 6p₃t + ··· = Σ r(r−1)·pᵣ·t^(r−2)
At t=1: G''(1) = Σ r(r−1)pᵣ = E[X(X−1)] = E(X²) − E(X)
At t=1: G''(1) = Σ r(r−1)pᵣ = E[X(X−1)] = E(X²) − E(X)
Var(X)
Var(X) = E(X²) − [E(X)]² = G''(1) + G'(1) − [G'(1)]²
Uniqueness Property
📌 One-to-One Correspondence
A PGF uniquely determines the probability distribution. If two random variables have the same PGF, they have the same distribution. This is exploited in Cambridge questions: if you can show that G_{X+Y}(t) matches the PGF of a known distribution, you immediately identify the distribution of X+Y.
Linear Transformation
If Y = aX + b (a, b non-negative integers)
G_Y(t) = E[t^(aX+b)] = t^b · G_X(t^a)
For example: if Y = X + 3, then G_Y(t) = t³·G_X(t). If Y = 2X, then G_Y(t) = G_X(t²).
E
Finding E(X) and Var(X) from a PGF
X has PGF G(t) = (0.4 + 0.6t)⁵. Find E(X) and Var(X).
Recognise
G(t) = (0.4 + 0.6t)⁵ is the PGF of B(5, 0.6). We can use derivatives directly.
G'(t)
G'(t) = 5(0.4 + 0.6t)⁴ · 0.6 = 3(0.4+0.6t)⁴
G'(1) = 3(1)⁴ = 3 = E(X)
G'(1) = 3(1)⁴ = 3 = E(X)
G''(t)
G''(t) = 3 · 4(0.4+0.6t)³ · 0.6 = 7.2(0.4+0.6t)³
G''(1) = 7.2(1)³ = 7.2
G''(1) = 7.2(1)³ = 7.2
Var(X)
Var(X) = G''(1) + G'(1) − [G'(1)]²
= 7.2 + 3 − 9
= 7.2 + 3 − 9
= 1.2 [Check: np(1−p) = 5×0.6×0.4 = 1.2 ✓]
Standard PGFs
These PGFs must be known and derivable. Cambridge may ask you to prove them from the definition or to use them directly.
Bernoulli Distribution
X ~ Bernoulli(p)
P(X=0)=q, P(X=1)=p, q=1−p
G(t) = q + pt
G(t) = q + pt
E(X): p Var(X): pq
Binomial Distribution
X ~ B(n, p)
G(t) = (q + pt)ⁿ
where q = 1 − p
[Sum of n independent Bernoulli(p)]
where q = 1 − p
[Sum of n independent Bernoulli(p)]
E(X): np Var(X): npq
Geometric Distribution
X ~ Geo(p), X = 1,2,3,…
P(X=r) = pq^(r−1), r=1,2,…
G(t) = pt / (1 − qt)
Valid for |qt| < 1
G(t) = pt / (1 − qt)
Valid for |qt| < 1
E(X): 1/p Var(X): q/p²
Geometric (alt: X = 0,1,2,…)
X ~ Geo₀(p), X = 0,1,2,…
P(X=r) = pq^r, r=0,1,2,…
G(t) = p / (1 − qt)
[Number of failures before first success]
G(t) = p / (1 − qt)
[Number of failures before first success]
E(X): q/p Var(X): q/p²
Poisson Distribution
X ~ Poisson(λ)
P(X=r) = e^(−λ)λʳ/r!
G(t) = e^(λ(t−1))
= exp(λt − λ)
G(t) = e^(λ(t−1))
= exp(λt − λ)
E(X): λ Var(X): λ
Discrete Uniform
X ~ U{1, 2, …, n}
P(X=r) = 1/n for r=1,…,n
G(t) = t(1−tⁿ) / [n(1−t)]
[For t ≠ 1; G(1) = 1]
G(t) = t(1−tⁿ) / [n(1−t)]
[For t ≠ 1; G(1) = 1]
E(X): (n+1)/2 Var(X): (n²−1)/12
Deriving the Poisson PGF — Full Proof
E
Proof — Poisson PGF
Show that if X ~ Poisson(λ), then G_X(t) = e^(λ(t−1)).
From definition
G(t) = Σ(r=0 to ∞) P(X=r)·tʳ = Σ(r=0 to ∞) [e^(−λ) λʳ/r!] · tʳ
Factor out e^(−λ)
= e^(−λ) Σ(r=0 to ∞) (λt)ʳ/r!
Recognise power series
Σ(λt)ʳ/r! = e^(λt) (the exponential series with argument λt)
Combine
G(t) = e^(−λ) · e^(λt) = e^(λt−λ)
G(t) = e^(λ(t−1)) ✓
Deriving the Binomial PGF
E
Proof — Binomial PGF
Show that if X ~ B(n,p), then G_X(t) = (q + pt)ⁿ where q = 1−p.
From definition
G(t) = Σ(r=0 to n) P(X=r)·tʳ = Σ(r=0 to n) C(n,r) pʳ qⁿ⁻ʳ · tʳ
Rearrange
= Σ(r=0 to n) C(n,r) (pt)ʳ qⁿ⁻ʳ
Recognise binomial theorem
= (q + pt)ⁿ (binomial theorem with x=pt, y=q)
G(t) = (q + pt)ⁿ ✓
Sums of Independent Variables
One of the most powerful applications of PGFs is identifying the distribution of the sum of independent random variables.
Fundamental Result — Sum PGF
If X and Y are independent: G_{X+Y}(t) = G_X(t) · G_Y(t)
More generally: G_{X₁+X₂+···+Xₙ}(t) = G_{X₁}(t) · G_{X₂}(t) · ··· · G_{Xₙ}(t)
This follows directly from E[t^(X+Y)] = E[tˣ · tʸ] = E[tˣ]·E[tʸ] when X and Y are independent.
Key Results from the Sum Property
| Situation | Result | How to Prove |
|---|---|---|
| Sum of n independent B(1,p) variables | B(n, p) | G_{X₁+···+Xₙ}(t) = (q+pt)ⁿ |
| Sum of n independent Poisson(λ) variables | Poisson(nλ) | Product of n copies of e^(λ(t−1)) = e^(nλ(t−1)) |
| Sum of Poisson(λ₁) and Poisson(λ₂) | Poisson(λ₁+λ₂) | e^(λ₁(t−1)) · e^(λ₂(t−1)) = e^((λ₁+λ₂)(t−1)) |
| Sum of B(m,p) and B(n,p) — same p | B(m+n, p) | (q+pt)^m · (q+pt)^n = (q+pt)^(m+n) |
E
Sum of Poisson Variables
X ~ Poisson(2) and Y ~ Poisson(3), independent. Find the distribution of S = X + Y using PGFs.
PGFs
G_X(t) = e^(2(t−1)) G_Y(t) = e^(3(t−1))
Product
G_S(t) = G_X(t)·G_Y(t) = e^(2(t−1)) · e^(3(t−1)) = e^(5(t−1))
Identify
e^(5(t−1)) is the PGF of Poisson(5).
S = X + Y ~ Poisson(5)
Random Sum — Compound Distribution
A more advanced application: if N is a random variable and X₁, X₂, … are i.i.d., then for S = X₁ + X₂ + ··· + X_N (a random number of terms):
Compound distribution PGF
G_S(t) = G_N(G_X(t))
The PGF of S is found by composing the PGF of N with the PGF of X. This is a higher-level Cambridge 9231 result — it may be provided or expected in the hardest questions.
E
Compound PGF — Poisson Number of Bernoulli Trials
N ~ Poisson(λ). Given N=n, X = X₁+X₂+···+Xₙ where Xᵢ ~ Bernoulli(p) independently.
Show that X ~ Poisson(λp).
Show that X ~ Poisson(λp).
PGFs
G_N(t) = e^(λ(t−1)) G_{Xᵢ}(t) = q + pt
Apply compound formula
G_X(t) = G_N(G_{Xᵢ}(t)) = e^(λ((q+pt)−1)) = e^(λ(q+pt−1))
Simplify
q + pt − 1 = (1−p) + pt − 1 = p(t−1)
G_X(t) = e^(λp(t−1))
G_X(t) = e^(λp(t−1))
This is the PGF of Poisson(λp). ✓ (Poisson thinning theorem)
Worked Examples
1
Finding Unknown Constant from G(1) = 1
X has PGF G(t) = k(1 + 2t + 3t²) where k is a constant.
(i) Find k. (ii) State P(X=1). (iii) Find E(X) and Var(X).
(i) Find k. (ii) State P(X=1). (iii) Find E(X) and Var(X).
Part (i) — G(1)=1
G(1) = k(1+2+3) = 6k = 1 →
k = 1/6
Part (ii) — P(X=1)
Coefficient of t¹: P(X=1) = k×2 = 2/6 =
1/3
Part (iii) — Differentiate
G(t) = (1/6)(1 + 2t + 3t²)
G'(t) = (1/6)(2 + 6t)
G'(1) = (1/6)(8) = E(X) = 4/3
G''(t) = (1/6)(6) = 1
G''(1) = 1
Var(X) = G''(1) + G'(1) − [G'(1)]²
= 1 + 4/3 − 16/9 = 9/9 + 12/9 − 16/9
G'(t) = (1/6)(2 + 6t)
G'(1) = (1/6)(8) = E(X) = 4/3
G''(t) = (1/6)(6) = 1
G''(1) = 1
Var(X) = G''(1) + G'(1) − [G'(1)]²
= 1 + 4/3 − 16/9 = 9/9 + 12/9 − 16/9
Var(X) = 5/9
2
Geometric PGF — Deriving Mean and Variance
X ~ Geo(p), so P(X=r) = pq^(r−1) for r=1,2,… Show G(t) = pt/(1−qt) and derive E(X) and Var(X).
Derive G(t)
G(t) = Σ(r=1 to ∞) pq^(r−1) · tʳ = pt Σ(r=1 to ∞) (qt)^(r−1) = pt · 1/(1−qt)
G(t) = pt/(1−qt) (geometric series, |qt|<1)
G'(t) — quotient rule
G'(t) = [p(1−qt) − pt(−q)] / (1−qt)²
= p / (1−qt)²
G'(1) = p/(1−q)² = p/p² = 1/p = E(X)
= p / (1−qt)²
G'(1) = p/(1−q)² = p/p² = 1/p = E(X)
G''(t)
G''(t) = 2pq / (1−qt)³
G''(1) = 2pq/p³ = 2q/p²
Var(X) = 2q/p² + 1/p − 1/p²
= (2q + p − 1)/p² = (2q − q)/p² [since p+q=1 → p−1=−q]
G''(1) = 2pq/p³ = 2q/p²
Var(X) = 2q/p² + 1/p − 1/p²
= (2q + p − 1)/p² = (2q − q)/p² [since p+q=1 → p−1=−q]
Var(X) = q/p²
3
Using PGF to Identify a Distribution
X₁, X₂, X₃ are independent, each ~ Geo(0.3). Using PGFs, find the mean and variance of S = X₁+X₂+X₃.
PGF of each Xᵢ
G_{Xᵢ}(t) = 0.3t / (1 − 0.7t)
PGF of S
G_S(t) = [0.3t/(1−0.7t)]³ = (0.3t)³/(1−0.7t)³
This is the PGF of a Negative Binomial (r=3, p=0.3).
This is the PGF of a Negative Binomial (r=3, p=0.3).
Mean and Variance
E(S) = 3·E(X₁) = 3·(1/0.3) = 3/0.3 =
Var(S) = 3·Var(X₁) = 3·(0.7/0.09) =
10
Var(S) = 3·Var(X₁) = 3·(0.7/0.09) =
70/3 ≈ 23.33
Practice Questions
Question 1 — PGF from Definition
[6 marks]
A discrete RV X has P(X=0) = 1/3, P(X=1) = 1/2, P(X=2) = 1/6.
(i) Write down G_X(t). (ii) Find E(X) and Var(X) using G'(1) and G''(1).
(i) Write down G_X(t). (ii) Find E(X) and Var(X) using G'(1) and G''(1).
G(t) = (1/3) + (1/2)t + (1/6)t². Differentiate twice. G'(1) = E(X), Var = G''(1)+G'(1)−[G'(1)]².
✓ Solution
(i) G(t) = 1/3 + (1/2)t + (1/6)t²Verify: G(1) = 1/3+1/2+1/6 = 2/6+3/6+1/6 = 1 ✓
(ii) G'(t) = 1/2 + (1/3)t → G'(1) = 1/2+1/3 = 5/6 = E(X)
G''(t) = 1/3 → G''(1) = 1/3
Var(X) = 1/3 + 5/6 − (5/6)² = 1/3 + 5/6 − 25/36
= 12/36 + 30/36 − 25/36 =
17/36
Question 2 — Finding k and Distribution
[5 marks]
A random variable X has PGF G(t) = k(2 + t)³ for some constant k.
(i) Find k. (ii) Identify the distribution of X. (iii) Find E(X).
(i) Find k. (ii) Identify the distribution of X. (iii) Find E(X).
Set G(1)=1 to find k. Expand (2+t)³ = (2(1+(t/2)))³ = 8(1+t/2)³ and compare with (q+pt)ⁿ form.
✓ Solution
(i) G(1) = k(2+1)³ = 27k = 1 → k = 1/27(ii) G(t) = (1/27)(2+t)³ = (2/3 + t/3)³ = (2/3 + (1/3)t)³
This matches (q+pt)ⁿ with n=3, p=1/3, q=2/3.
X ~ B(3, 1/3)
(iii) E(X) = np = 3×(1/3) =
1
Or: G'(t)=(1/27)·3(2+t)²=1/9·(2+t)², G'(1)=1/9·9=1 ✓
Question 3 — Sum of Independent Variables
[5 marks]
X ~ Poisson(3) and Y ~ Poisson(5) are independent. Using PGFs, state the distribution of S = X + Y and find P(S = 2).
G_S(t) = G_X(t)·G_Y(t) = e^(3(t−1))·e^(5(t−1)) = e^(8(t−1)). Identify the distribution, then use the Poisson PMF for P(S=2).
✓ Solution
G_S(t) = e^(3(t−1)) · e^(5(t−1)) = e^(8(t−1))This is the PGF of Poisson(8).
S ~ Poisson(8)
P(S=2) = e^(−8)·8²/2! = 64e^(−8)/2 = 32e^(−8)
= 32 × 0.000335 ≈ 0.01073
Question 4 — Deriving PGF and Moments
[8 marks]
A fair die is rolled repeatedly until a 6 appears. Let X be the number of rolls required.
(i) Show that X ~ Geo(1/6) and write down G_X(t).
(ii) Differentiate G_X(t) to find E(X) and Var(X).
(iii) Let S = X₁ + X₂ where X₁ and X₂ are independent rolls-to-six sequences. Write down G_S(t) and state E(S) and Var(S).
(i) Show that X ~ Geo(1/6) and write down G_X(t).
(ii) Differentiate G_X(t) to find E(X) and Var(X).
(iii) Let S = X₁ + X₂ where X₁ and X₂ are independent rolls-to-six sequences. Write down G_S(t) and state E(S) and Var(S).
For (i): P(X=r) = (5/6)^(r−1)·(1/6), which is Geo(p=1/6). G(t) = pt/(1−qt). Differentiate using quotient rule for (ii). For (iii): G_S = [G_X(t)]².
✓ Solution
(i) P(X=r) = (5/6)^(r−1)·(1/6) = geometric with p=1/6, q=5/6. ✓G_X(t) = (1/6)t / (1 − (5/6)t) = t/(6−5t)
(ii) G'(t) = [(6−5t) − t(−5)] / (6−5t)² = 6/(6−5t)²
G'(1) = 6/(6−5)² = 6/1 = 6 = E(X) [Check: 1/p = 6 ✓]
G''(t) = 6·2(6−5t)·5/(6−5t)⁴ = 60/(6−5t)³
G''(1) = 60/1 = 60
Var(X) = 60 + 6 − 36 = 30 [Check: q/p² = (5/6)/(1/36) = 30 ✓]
(iii) G_S(t) = [G_X(t)]² = t²/(6−5t)²
E(S) = 2·E(X) =
12
Var(S) = 2·Var(X) =
60
Interactive PGF Explorer
Select a distribution, adjust parameters, and explore the PGF, its derivatives, and the resulting moments.
PGF Calculator & Moment Extractor
Select a distribution and click Compute.
Formula Sheet — Probability Generating Functions
Definition
G_X(t) = E[t^X]= Σ P(X=r)tʳ
G(0) = P(X=0)—
G(1) = 1Always (use to find k)
P(X=r)Coefficient of tʳ
Moments via Derivatives
E(X) = G'(1)First derivative at 1
G''(1) = E[X(X−1)]—
E(X²) = G''(1)+G'(1)—
Var(X)G''(1)+G'(1)−[G'(1)]²
Standard PGFs
Bernoulli(p)q + pt
Binomial(n,p)(q+pt)ⁿ
Geometric(p) X≥1pt/(1−qt)
Poisson(λ)e^(λ(t−1))
Sums of Variables
X,Y independentG_{X+Y}=G_X·G_Y
n i.i.d. copiesG_S(t)=[G_X(t)]ⁿ
CompoundG_S(t)=G_N(G_X(t))
Key Sum Results
n×Bern(p)~ B(n,p)
Poi(λ₁)+Poi(λ₂)~ Poi(λ₁+λ₂)
B(m,p)+B(n,p)~ B(m+n,p)
Transformations
Y = X + bG_Y(t) = t^b · G_X(t)
Y = 2XG_Y(t) = G_X(t²)
UniquenessPGF → distribution 1-to-1
📋 Cambridge Exam Strategy — PGFs
- Finding k: Always use G(1) = 1. Substitute t=1, set equal to 1, solve. This is the first step in almost every PGF question.
- Extracting probabilities: Read pᵣ as the coefficient of tʳ. If G(t) is given in factored form, expand the first few terms to read off probabilities.
- Differentiating G(t): For products and quotients, use chain/product/quotient rule. Always evaluate at t=1 after differentiating, not before.
- Var(X) formula: Memorise Var(X) = G''(1) + G'(1) − [G'(1)]². Cambridge awards this explicitly — do not try to compute E(X²) and E(X) separately without mentioning G''(1).
- Identifying distributions: After computing G_S(t), compare with the standard forms. If G_S(t) = e^(λ(t−1)) → Poisson(λ). If G_S(t) = (q+pt)ⁿ → B(n,p). State the identification explicitly.
- Proving sum results: Write G_{X+Y}(t) = G_X(t)·G_Y(t), simplify the product, and identify the resulting PGF. This is the standard form of a Cambridge "hence show" question.