Lecture 2 : Confidence Interval Estimators in Normal and Exponential Cases. Optimal Confidence Regions. Basic Lemma.

Confidence Interval (strong) Estimators in Normal Case

Let x be a sample drawn from N(μ, σ) distribution. Confidence interval estimator for scale parameter. Take the statistic

T (x) = \sqrt{n}s_{x},

where

s_{x} = \sqrt{\frac{1}{n} n ; ∑ ; i=1(x_{i} − x̄)^{2}},  x̄ = \frac{1}{n} n ; ∑ ; i=1 x_{i}.

The conditions of Lemma 1 are fulfilled and we can apply Corollary 1.

It is known that (T (y))^{2} = ns_{y}^{2}, when y drawn from N(0, 1) distribution, has χ_{n−1}^{2} distribution, the density function of which is written as

f̃(u) = \frac{1}{2^{(n−1)/2}Γ (\frac{n−1}{2})}u^{\frac{n−3}{2}}e^{−u/2}, u > 0.

For construction of the confidence interval (strong) of level α for σ it is enough to choose χ_{α,1}^{2}, χ_{α,2}^{2} such that

χ_{α,2}^{2} ; ∫ ; χ_{α,1}^{2}f̃(u)du = α.

Values χ_{α,1}^{2}, χ_{α,2}^{2} can be chosen in many ways, but usually they are chosen as

χ_{α,1}^{2} ; ∫ ; 0f̃(u)du = ∞ ; ∫ ; χ_{α,2}^{2} f̃(u)du = \frac{1 − α}{2}.

Then according to Corollary 1,

P_{(μ,σ)}^{(n)} (χ_{α,1}^{2} ≤ \frac{(T (x))^{2}}{σ^{2}} ≤ χ_{α,2}^{2}) = α,

or equivalently,

P_{(μ,σ)}^{(n)} (\frac{\sqrt{n}s_{x}}{χ_{α,2}} ≤ σ ≤ \frac{\sqrt{n}s_{x}}{χ_{α,1}} ) = α.

So, the interval

[\frac{\sqrt{n}s_{x}}{χ_{α,2}}, \frac{\sqrt{n}s_{x}}{χ_{α,1}} ]

is a strong confidence interval of level α for the scale parameter σ.

Confidence interval estimator for location parameter. Consider

T_{1}(x) = x̄, T_{2}(x) = \frac{s_{x}}{\sqrt{n − 1}}.

The conditions of Lemma 2 are fulfilled. But what is the distribution of T_{1}(y)/T_{2}(y) when y drawn from N(0, 1) distribution ?

It is known that in this case T_{1}(y) = ȳ has N(0, 1/\sqrt{n}) distribution, while (n − 1)n(T_{2}(y))^{2} = ns_{y}^{2} has χ_{n−1}^{2} distribution.

It is also known that if ξ has N(0, 1) distribution and η^{2} is independent of ξ and has χ_{n−1}^{2} distribution, then \sqrt{n − 1}ξ/η has Student’s distribution with n − 1 degrees of freedom with the density function

f(u) = \frac{Γ (\frac{n}{2})}{\sqrt{(n − 1)π}Γ (\frac{n−1}{2})} (1 + \frac{u^{2}}{n − 1})^{−n/2} , u ∈ R.

Hence,

\sqrt{n − 1}\frac{\sqrt{n}T_{1}(y)}{\sqrt{(n − 1)n}T_{2}(y)} = \frac{T_{1}(y)}{T_{2}(y)}

has Student’s distribution with n − 1 degrees of freedom.

For construction of the confidence interval (strong) of level α for μ it is enough to choose t_{α} such that

t_{α} ; ∫ ; −t_{α} f(u)du = α.

Then according to Lemma 2,

P_{(μ,σ)}^{(n)} (−t_{α} ≤ \frac{T_{1}(x) − μ}{T_{2}(x)} ≤ t_{α}) = α,

or equivalently,

P_{(μ,σ)}^{(n)} (x̄ − t_{α} \frac{s_{x}}{\sqrt{n − 1}} ≤ μ ≤ x̄ + t_{α} \frac{s_{x}}{\sqrt{n − 1}}) = α.

So, the interval

[x̄ − t_{α} \frac{s_{x}}{\sqrt{n − 1}}, x̄ + t_{α} \frac{s_{x}}{\sqrt{n − 1}}]

is a strong confidence interval of level α for the location parameter μ.

Confidence Interval (strong) Estimators in Exponential Case

Let x be a sample drawn from E(μ, σ) distribution. Confidence interval estimator for scale parameter. Take the statistic

T (x) = 2n(x̄ − x_{(1)}),

where x_{(1)} = min{x_{1}, x_{2}, …, x_{n}}. The conditions of Lemma 1 are fulfilled.

It is known that T (y) = 2n(ȳ − y_{(1)}), when y drawn from E(0, 1) distribution, has χ_{2(n−1)}^{2} distribution with the density function

f(u) = \frac{1}{2^{n−1}Γ (n − 1)}u^{n−2}e^{−u/2}, u > 0.

For construction of the confidence interval (strong) of level α for σ we take χ_{α,1}^{2}, χ_{α,2}^{2} such that

χ_{α,2}^{2} ; ∫ ; χ_{α,1}^{2}f(u)du = α.

Values χ_{α,1}^{2}, χ_{α,2}^{2} are usually chosen as

χ_{α,1}^{2} ; ∫ ; 0f(u)du = ∞ ; ∫ ; χ_{α,2}^{2} f(u)du = \frac{1 − α}{2}.

Then according to Lemma 1,

P_{(μ,σ)}^{(n)} (χ_{α,1}^{2} ≤ \frac{T (x)}{σ} ≤ χ_{α,2}^{2}) = α,

or equivalently,

P_{(μ,σ)}^{(n)} (\frac{2n(x̄ − x_{(1)})}{χ_{α,2}^{2}} ≤ σ ≤ \frac{2n(x̄ − x_{(1)})}{χ_{α,1}^{2}}) = α.

So, the interval

[\frac{2n(x̄ − x_{(1)})}{χ_{α,2}^{2}}, \frac{2n(x̄ − x_{(1)})}{χ_{α,1}^{2}}]

is a strong confidence interval of level α for the scale parameter σ.

Confidence interval estimator for location parameter. Consider

T_{1}(x) = x_{(1)}, T_{2}(x) = \frac{x̄ − x_{(1)}}{n − 1}.

The conditions of Lemma 2 are fulfilled. But what is the distribution of T_{1}(y)/T_{2}(y) when y drawn from E(0, 1) distribution ?

It is known that in this case T_{1}(y) = y_{(1)} has E(0, 1/n) distribution, therefore 2nT_{1}(y) has E(0, 2) = χ_{2}^{2} distribution. On the other hand, 2n(n − 1)T_{2}(y) = 2n(ȳ − y_{(1)}) has χ_{2(n−1)}^{2} distribution.

It is also known that if ξ has χ_{2}^{2} distribution and η is independent of ξ and has χ_{2(n−1)}^{2} distribution, then (n − 1)ξ/η has F (2, 2(n − 1)) (Fisher-Snedecor) distribution with the density function

f(u) = \frac{1}{(1 + \frac{u}{n−1})^{n}}, u ≥ 0.

Hence,

(n − 1)\frac{2nT_{1}(y)}{2n(n − 1)T_{2}(y)} = \frac{T_{1}(y)}{T_{2}(y)}

has F (2, 2(n − 1)) distribution.

For construction of the confidence interval (strong) of level α for μ we take f_{α} such that

f_{α} ; ∫ ; 0f(u)du = α.

Value f_{α} is chosen as

∞ ; ∫ ; f_{α}f(u)du = 1 − α.

Then according to Lemma 2,

P_{(μ,σ)}^{(n)} (0 ≤ \frac{T_{1}(x) − μ}{T_{2}(x)} ≤ f_{α}) = α,

or equivalently,

P_{(μ,σ)}^{(n)} (x_{(1)} − f_{α} \frac{x̄ − x_{(1)}}{n − 1} ≤ μ ≤ x_{(1)}) = α.

So, the interval

[x_{(1)} − f_{α} \frac{x̄ − x_{(1)}}{n − 1}, x_{(1)}]

is a strong confidence interval of level α for the location parameter μ.

Optimal Confidence Regions (general and rectangular)

Let x be a sample drawn from the distribution P_{(μ,σ)}, while T_{1}(x), T_{2}(x) be statistics satisfying the conditions of Lemma 2.

Let α ∈ (0, 1) be a given confidence level and B_{0} ⊂ R^{2} be a Borel set such that

P_{(0,1)}^{(n)} ((\frac{T_{1}(y)}{T_{2}(y)}, \frac{1}{T_{2}(y)}) ∈ B_{0}) = α,  (1)

where y drawn from the distribution P_{(0,1)}.

Then according to Lemma 3,

P_{(μ,σ)}^{(n)} ((\frac{T_{1}(x) − μ}{T_{2}(x)}, \frac{σ}{T_{2}(x)}) ∈ B_{0}) = α,

and the random set

B(x) = (T_{1}(x), 0) + T_{2}(x)B_{0}^{−},  (2)

where

B_{0}^{−} = {(u, v) : (−u, v) ∈ B_{0}},

is the confidence region (strong) of level α for (μ, σ). Furthermore,

R((μ, σ), B) = E_{(μ,σ)}λ(B(x)) = λ(B_{0})E_{(μ,σ)}T_{2}^{2}(x) = σ^{2}λ(B_{0})E_{(0,1)}T_{2}^{2}(y).

So, the set B^{∗}(x) of the form (2) is the optimal confidence region of level α for (μ, σ) if the corresponding set B_{0}^{∗} has the minimal area among all the sets satisfying the condition (1).

Now suppose that we are interested to find the optimal rectangular confidence region of level α for (μ, σ). Let

B_{0} = [a_{1}, a_{2}] × [b_{1}, b_{2}], a_{1} < a_{2}, b_{1} < b_{2},

be a rectangular which satisfies the condition (1).

Similarly as it was done before it can be shown, that the rectangular confidence region (strong) of level α for (μ, σ) has the form

B_{r}(x) = [T_{1}(x) − T_{2}(x)a_{2}, T_{1}(x) − T_{2}(x)a_{1}] × [T_{2}(x)b_{1}, T_{2}(x)b_{2}] (3)

and

R((μ, σ), B_{r}) = σ^{2}(b_{2} − b_{1})(a_{2} − a_{1})E_{(0,1)}T_{2}^{2}(y).

Hence, the rectangular B_{r}^{∗}(x) of the form (3) is the optimal rectangular confidence region of level α for (μ, σ) if the corresponding set B_{0}^{∗} has the minimal area among all the rectangulars satisfying the condition (1).

Basic Lemma (Proposition 2.1 in Einmahl, J.H.J, Mason, D.M. Generalized Quantile Processes. Annals of Statistics, 20 (1992), 2, 1062–1078.)

Let g(u, v), u, v ∈ R, be continuous density function such that λ({(u, v) : g(u, v) = z}) = 0 for any z > 0. Denote

A_{α} = \{B ∈ B^{2} : \begin{aligned}∫ & ∫ \\ & B \end{aligned} g(u, v)dudv = α\}

and

B_{z} = {(u, v) : g(u, v) ≥ z}

given z > 0.

Then the set B_{z_{α}}, where z_{α} is defined by the equation

\begin{matrix}∫ & ∫ \\ B_{zα} \end{matrix} g(u, v)dudv = α,

possesses the property

B_{z_{α}} ∈ arg min_{B∈A_{α}} λ(B).

Remark 1. The statement of Basic Lemma holds for any dimension. Remark 2. If g(u, v) has a single maximum then B_{z_{α}} is a connected set. Remark 3. The set B_{z_{α}} does not necessary be a rectangular.

Making use of Basic Lemma we get the optimal strong confidence region of level α for (μ, σ) of the form

B^{∗}(x) = (T_{1}(x), 0) + T_{2}(x)B_{z_{α}}^{−}.

Questions and Exercises after Lecture 2

1. Construct, please, ”classical” confidence intervals for location and scale parameters separately making use of the formulas given in Lecture 2 (pp. 1–6) in the case when the sample x of size 10 drawn from :

• Normal Distribution N(μ, σ) a) x = (−3.6, −1.0, 2.7, −5.2, −4.3, −2.8, 2.4, −2.7, 2.4, −9.5);

b) x = (2.8, −0.8, −4.8, 0.3, −3.5, 0.9, 3.8, 0.2, 2.5, −0.3);

c) x = (3.9, 2.4, 4.7, −4.2, 1.2, 5.6, 2.5, −1.4, 8.6, 7.1).

• Exponential Distribution E(μ, σ) a) x = (1.1, 4.6, 4.6, 0.3, 2.6, 1.9, 3.3, 3.7, 0.6, 2.3);

b) x = (3.5, 0.1, 0.5, 1.6, 2.1, 1.1, 3.5, 0.2, 1.8, 2.6);

c) x = (0.5, 5.4, 2.5, 1.4, 2.9, 0.1, 3.4, 1.0, 0.1, 1.1).

In each case take, please, as a confidence level α = 0.95.

2. Let x be a sample of size n drawn from the distribution with distribution function F_{ξ}. What is the distribution function of x_{(1)} ?

3. What do you think about the optimality of confidence intervals given in Lecture 2 ?

4. How the optimal confidence intervals should be produced for those cases when the confidence intervals given in Lecture 2 are non-optimal ?

5. Utilize, please, given method for construction of confidence regions (and optimal confidence regions) to construction of confidence intervals separately for location and scale parameters in general case.

6. Does the method of construction of optimal confidence regions given in Lecture 2 always lead to really optimal confidence region in the sense of Definition 3 ?