Lecture 3 : Optimal Confidence Regions in Normal

and Exponential Cases.

Optimal Confidence Regions in Normal Case

Let x be a sample drawn from N(μ, σ) distribution. Consider

T_{1}(x) = x̄, T_{2}(x) = \sqrt{n}s_{x}.

These statistics possess all the properties we need to apply Lemma 3.

It is known that if y is a sample drawn from N(0, 1) distribution then T_{1}(y) = ȳ has N(0, 1/\sqrt{n}) distribution, while T_{2}(y) = \sqrt{n}s_{y} has \sqrt{χ_{n−1}^{2}} distribution, i. e.

f_{1}(u) = \begin{matrix}√ & \overline{n} \\ √ & \overline{2π} \end{matrix}e^{−nu^{2}/2}, u ∈ R,

f_{2}(v) = \frac{1}{2^{(n−3)/2}Γ(\frac{n−1}{2})}v^{n−2}e^{−v^{2}/2}, v > 0.

According to Lemma 3,

P_{(μ,σ)}^{(n)} ((\frac{T_{1}(x) − μ}{T_{2}(x)}, \frac{σ}{T_{2}(x)}) ∈ B) = \begin{aligned}∫ & ∫ \\ & B \end{aligned} \frac{1}{v^{3}}f_{1} (\frac{u}{v}) f_{2} (\frac{1}{v} ) dudv

= c_{n} \begin{aligned}∫ & ∫ \\ & B \end{aligned} v^{−(n+1)}e^{−\frac{1+nu^{2}}{2v^{2}}} dudv, c_{n} = \frac{\sqrt{n}}{2^{(n−2)/2}\sqrt{π}Γ(\frac{n−1}{2})}.

Now we should apply Basic Lemma to the function

g(u, v) = c_{n}v^{−(n+1)}e^{−\frac{1+nu^{2}}{2v^{2}}} , u ∈ R, v > 0,

which satisfies all it’s conditions.

It is worth noting that g(−u, v) = g(u, v).

What can we say about the set B_{z_{α}} which defines the optimal confidence region ? Firstly, B_{z_{α}} = B_{z_{α}}^{−}, therefore it is symmetric with respect to the line u = 0. Secondly,

B_{z_{α}} = \{(u, v) : g(u, v) ≥ z_{α}, v > 0\} = \{(u, v) : v^{n+1}e^{\frac{1+nu^{2}}{2v^{2}}} ≤ z_{α}^{−1}c_{n}, v > 0\},

where z_{α} is defined by the equation

c_{n} \begin{matrix}∫ & ∫ \\ B_{zα} \end{matrix} v^{−(n+1)}e^{−\frac{1+nu^{2}}{2v^{2}}} dudv = α.

We can also rewrite B_{z_{α}} as

B_{z_{α}} = {(u, v) : |u| ≤ u(z_{α}, v), v_{−}(z_{α}) ≤ v ≤ v_{+}(z_{α})},

where

u(z_{α}, v) = \sqrt{\frac{2v^{2} ln (\frac{c_{n}}{z_{α}v^{n+1}} ) − 1}{n}},

while v_{−}(z_{α}), v_{+}(z_{α}) are positive solutions of the equation u(z_{α}, v) = 0, or equivalently, v^{2} ln (\frac{c_{n}}{z_{α}v^{n+1}} ) = \frac{1}{2}, with respect to v. And z_{α} is defined by the equation

c_{n} v_{+}(z_{α}) ; ∫ ; v_{−}(z_{α}) u(z_{α},v) ; ∫ ; 0v^{−(n+1)}e^{−\frac{1+nu^{2}}{2v^{2}}} dudv = \frac{α}{2}.

Finally, the optimal confidence region has the form

B^{∗}(x) = {(u, v) : u = T_{1}(x) + T_{2}(x)z, v = T_{2}(x)t, (z, t) ∈ B_{z_{α}}}

= \{(u, v) : (\frac{u − x̄}{\sqrt{n}s_{x}}, \frac{v}{\sqrt{n}s_{x}} ) ∈ B_{z_{α}} \} ,

or, equivalently,

B^{∗}(x) = \{(u, v) : |\frac{u − x̄}{\sqrt{n}s_{x}}| ≤ u(z_{α}, \frac{v}{\sqrt{n}s_{x}}), v_{−}(z_{α}) ≤ \frac{v}{\sqrt{n}s_{x}} ≤ v_{+}(z_{α})\} .

\begin{matrix}n & z_{α} & v_{−}(z_{α}) & v_{+}(z_{α}) \\ 3 & 0.003470287 & 0.2078085865 & 3.7227073330 \\ 4 & 0.018543099 & 0.2099848336 & 1.9771018340 \\ 5 & 0.050355149 & 0.2074490407 & 1.3765323680 \\ 6 & 0.101721526 & 0.2033948112 & 1.0748488240 \\ 7 & 0.173989752 & 0.1988523390 & 0.8932219526 \\ 8 & 0.265942735 & 0.1941631771 & 0.7721469134 \\ 9 & 0.381452685 & 0.1896892948 & 0.6841293602 \\ 10 & 0.516015059 & 0.1853434028 & 0.6179249525 \\ 12 & 0.857165129 & 0.1774536984 & 0.5229399623 \\ 14 & 1.279721747 & 0.1703733650 & 0.4583900297 \\ 16 & 1.803847657 & 0.1641518650 & 0.4107533442 \\ 18 & 2.397491014 & 0.1585064052 & 0.3744981089 \\ 20 & 3.099278069 & 0.1534917024 & 0.3453906302 \\ 22 & 3.891351043 & 0.1489452671 & 0.3216035423 \\ 24 & 4.766676254 & 0.1447925198 & 0.3017705312 \\ 26 & 5.715582426 & 0.1409741250 & 0.2849556584 \\ 28 & 6.809832452 & 0.1375130376 & 0.2702874890 \\ 30 & 7.927274638 & 0.1342594293 & 0.2576457012 \\ 32 & 9.142895871 & 0.1312522318 & 0.2464620659 \\ 34 & 10.45933434 & 0.1284578836 & 0.2364861824 \\ 36 & 11.87922648 & 0.1258540054 & 0.2275209249 \\ 38 & 13.40521111 & 0.1234193084 & 0.2194118626 \\ 40 & 15.03992493 & 0.1211337737 & 0.2120370840 \end{matrix}

Table 1: Normal distribution, α = 0.95.

Consider the case of construction the optimal rectangular confidence region of level α for (μ, σ) within Normal model.

It is naturally to confine ourselves with the case when the rectangular confidence region is symmetric with respect to the line u = 0, i. e. has the form

B_{r} = [−a, a] × [b_{1}, b_{2}], a > 0, b_{1} < b_{2}.

Let b_{1}, b_{2} are given. Then we should take a = a(b_{1}, b_{2}) such that

a(b_{1},b_{2}) ; ∫ ; −a(b_{1},b_{2}) b_{2} ; ∫ ; b_{1}g(u, v)dudv = α.

Therefore, the rectangular B_{r} is completely defined by b_{1}, b_{2}, and has the area

f(b_{1}, b_{2}) = 2a(b_{1}, b_{2})(b_{2} − b_{1}), b_{1} < b_{2}.

So, in order to minimize the function R((μ, σ), B_{r}) we need to find

(b_{1}, b_{2}) ∈ arg min_{b_{1}<b_{2}} f(b_{1}, b_{2}).

\begin{matrix}n & λ(B_{r}(x)) & λ(B(x)) & \frac{λ(B_{r}(x))}{λ(B(x))} \\ 10 & 2.265476462 & 2.085535842 & 1.086280282 \\ 20 & .8817488104 & .8204335260 & 1.074735225 \\ 30 & .5445196008 & .5084378281 & 1.070965948 \end{matrix}

Table 2: Comparison of areas of optimal regions in D and B_{Θ}

As it was expected, the optimal region in D is not optimal in B_{Θ}, but the ratio of areas of those two regions is quite close to one.

\begin{matrix}n & b_{1} & b_{2} & a(b_{1}, b_{2}) \\ 3 & .2414351643 & 3.699117924 & 2.231709982 \\ 4 & .2376860820 & 1.950761172 & 1.129921458 \\ 5 & .2315128434 & 1.352509220 & .7503595140 \\ 6 & .2243795829 & 1.054525220 & .5609853720 \\ 7 & .2169958266 & .8752155927 & .4480081726 \\ 8 & .2105645841 & .7553869523 & .3730977529 \\ 9 & .2045664980 & .6693606249 & .3195925164 \\ 10 & .1989887757 & .6043522081 & .2794376948 \\ 12 & .1888868339 & .5115477040 & .2234359710 \\ 14 & .1804887729 & .4482285832 & .1863178693 \\ 16 & .1730045953 & .4020723105 & .1595940005 \\ 18 & .1664438606 & .3665430190 & .1396216697 \\ 20 & .1606797536 & .3383176047 & .1240935990 \\ 22 & .1554898248 & .3151673076 & .1116992777 \\ 24 & .1507919660 & .2958328563 & .1015398118 \\ 26 & .1465875724 & .2794232264 & .09309397968 \\ 28 & .1427129459 & .2652655874 & .08592698957 \\ 30 & .1391326003 & .2528394652 & .07981335770 \\ 32 & .1358578462 & .2419444328 & .07448813180 \\ 34 & .1328045414 & .2321976876 & .06985069015 \\ 36 & .1299748379 & .2234610820 & .06576265540 \\ 38 & .1273281799 & .2156360014 & .06208048850 \\ 40 & .1247889296 & .2084051351 & .05882891775 \end{matrix}

Table 3: Normal distribution, α = 0.95

Optimal Confidence Region in Exponential Case

Let x be a sample drawn from E(μ, σ) distribution. Let us take

T_{1}(x) = x_{(1)}, T_{2}(x) = 2n(x̄ − x_{(1)}),

where x_{(1)} = min_{1≤i≤n} x_{i}. These statistics possess all the properties we need to apply Lemma 3.

It is known that if y is a sample drawn from E(0, 1) distribution then T_{1}(y) = y_{(1)} has E(0, 1/n) distribution, while T_{2}(y) = 2n(ȳ − y_{(1)}) has χ_{2(n−1)}^{2} distribution, i. e.

f_{1}(u) = ne^{−nu}, u ≥ 0,

f_{2}(v) = \frac{1}{2^{n−1}Γ(n − 1)}v^{n−2}e^{−v/2}, v > 0.

According to Lemma 3,

P_{(μ,σ)}^{(n)} ((\frac{T_{1}(x) − μ}{T_{2}(x)}, \frac{σ}{T_{2}(x)}) ∈ B) = \begin{aligned}∫ & ∫ \\ & B \end{aligned} \frac{1}{v^{3}}f_{1} (\frac{u}{v}) f_{2} (\frac{1}{v} ) dudv

= c_{n} \begin{aligned}∫ & ∫ \\ & B \end{aligned} v^{−(n+1)}e^{−\frac{1+2nu}{2v}}dudv, c_{n} = \frac{n}{2^{n−1}Γ(n − 1)}.

Now we should apply Basic Lemma to the function

g(u, v) = c_{n}v^{−(n+1)}e^{−\frac{1+2nu}{2v}}, u ≥ 0, v > 0,

which satisfies all it’s conditions.

What can we say about the set B_{z_{α}} which defines the optimal confidence region ? Firstly, it is bounded by the line u = 0. Secondly,

B_{z_{α}} = {(u, v) ∈ [0, ∞) × (0, ∞) : g(u, v) ≥ z_{α}}

= \{(u, v) ∈ [0, ∞) × (0, ∞) : v^{n+1}e^{\frac{1+2nu}{2v}} ≤ z_{α}^{−1}c_{n}\},

where z_{α} is defined by the equation

c_{n} \begin{matrix}∫ & ∫ \\ B_{zα} \end{matrix} v^{−(n+1)}e^{−\frac{1+2nu}{2v}}dudv = α.

We can also rewrite B_{z_{α}} as

B_{z_{α}} = {(u, v) : 0 ≤ u ≤ u(z_{α}, v), v_{−}(z_{α}) ≤ v ≤ v_{+}(z_{α})},

where

u(z_{α}, v) = \frac{v ln (\frac{c_{n}}{z_{α}v^{n+1}} ) − 1/2}{n}

and v_{−}(z_{α}), v_{+}(z_{α}) are positive solutions of the equation g(0, v) = z_{α}, or equivalently, v^{n+1}e^{1/2v} = z_{α}^{−1}c_{n}, with respect to v. And z_{α} is defined by the equation

c_{n} v_{+}(z_{α}) ; ∫ ; v_{−}(z_{α}) u(z_{α},v) ; ∫ ; 0v^{−(n+1)}e^{−\frac{1+2nu}{2v}}dudv = α.

Finally, the optimal confidence region has the form

B^{∗}(x) = {(u, v) : u = T_{1}(x) − T_{2}(x)z, v = T_{2}(x)t, (z, t) ∈ B_{z_{α}}}

= \{(u, v) : (\frac{x_{(1)} − u}{2n(x̄ − x_{(1)})}, \frac{v}{2n(x̄ − x_{(1)})}) ∈ B_{z_{α}} \} ,

or, equivalently,

B^{∗}(x) = \{(u, v) : 0 ≤ \frac{x_{(1)} − u}{2n(x̄ − x_{(1)})} ≤ u(z_{α}, \frac{v}{2n(x̄ − x_{(1)})}),

v_{−}(z_{α}) ≤ \frac{v}{2n(x̄ − x_{(1)})} ≤ v_{+}(z_{α})\} .

\begin{matrix}n & z_{α} & v_{−}(z_{α}) & v_{+}(z_{α}) \\ 3 & 0.035200369 & 0.02904503194 & 2.01951657432 \\ 4 & 0.311009060 & 0.02843873361 & 0.85115634049 \\ 5 & 1.247025364 & 0.02704902501 & 0.49832343241 \\ 6 & 3.414768302 & 0.02550899480 & 0.34005777220 \\ 7 & 7.574477784 & 0.02403930245 & 0.25272258927 \\ 8 & 14.52605930 & 0.02267512208 & 0.19868442238 \\ 9 & 25.13685436 & 0.02142934497 & 0.16241552164 \\ 10 & 40.68355301 & 0.02031226721 & 0.13643614431 \\ 12 & 90.09095340 & 0.01837227746 & 0.10236889528 \\ 14 & 173.0060768 & 0.01677576209 & 0.08113175816 \\ 16 & 300.7691096 & 0.01544211746 & 0.06676963341 \\ 18 & 484.2610759 & 0.01430917288 & 0.05649780727 \\ 20 & 736.5573858 & 0.01333676600 & 0.04881418587 \\ 22 & 1069.088593 & 0.01249163732 & 0.04287761836 \\ 24 & 1504.661211 & 0.01175500821 & 0.03814275862 \\ 26 & 2046.395684 & 0.01110142624 & 0.03431121193 \\ 28 & 2723.429422 & 0.01052159039 & 0.03113510162 \\ 30 & 3543.053905 & 0.01000120162 & 0.02847239626 \\ 32 & 4517.646601 & 0.00953120373 & 0.02621145483 \\ 34 & 5679.651442 & 0.00910608752 & 0.02426381747 \\ 36 & 7025.013759 & 0.00871790384 & 0.02257600927 \\ 38 & 8625.912367 & 0.00836487288 & 0.02109040909 \\ 40 & 10408.27405 & 0.00803814018 & 0.01978750735 \end{matrix}

Table 4: Exponential distribution, α = 0.95.

Exercise after Lecture 3

Take, please, the samples given in Exercise 1 after Lecture 2. Construct optimal confidence regions in all cases and draw them. Could you put forward a hypothesis that all three samples were drawn from the distribution with the same location and scale parameters ?