Dwudzieste dziewia�te Colloquium Biometryczne 1999, str. 17–31

OPTIMAL DESIGNS IN A RANDOM COEFFICIENT REGRESSION MODEL

Aleksander Zaigraev

Institute of Mathematics and Informatics N. Copernicus University Chopina 12/18, 87–100 Toruń e-mail: alzaig@mat.uni.torun.pl

Summary

In a random coefficient linear regression model optimal design problems both for the estimation of the coefficients realization and their mean are considered with respect to the quadratic loss. The mean is assumed to belong to a given linear subspace of Euclidean space of parameters. The case of additional experiments is studied as well. Key words and frases: random coefficient linear regression model, optimal design, best linear unbiased estimator, additional experiments. Classification AMS 1993: 62K05, 62J05.

1. Introduction

Consider the linear regression model

y = Xθ + ξ,  (1)

where y is a p×1 observable random vector, X is a p×n design matrix, ξ is a p×1 random vector of unobservable disturbances having zero mean and covariance matrix σ^{2}I_{p}, σ > 0 is unknown and I_{p} is the p×p unity matrix. Let θ be an n×1 random vector, independent of ξ, with expectation μ ∈ A and covariance matrix σ^{2}S, where A is a given k-dimensional subspace of R^{n}, k ≤ min{n, p}, S is a known matrix. In such a way, the vector y has expectation Xμ and covariance matrix σ^{2}B, where B = I_{p} + XSX^{T} .

The paper deals with the following optimal design problems :

i) under estimation of μ; ii) under estimation of a realization of θ both when μ is known or not. In each case we consider the best linear unbiased estimator with respect to the quadratic loss and our method is based on the singular value decomposition of the design matrix.

Let {f_{i}}_{i=1}^{k} be a set of linearly independent vectors spanned A. Then the vector μ ∈ A admits the representation μ = F β with a vector β ∈ R^{k} and the n × k matrix F whose columns are {f_{i}}_{i=1}^{k}. Throughout the paper we use the usual matrix and vector notation. Among others, if X is a matrix, then R(X) and r(X) denote, respectively, the range (column space) and the rank of X. If x ∈ R^{p} then x_{(i)}, i = 1, …, p, mean its components ordered in ascending order of magnitude, i. e. x_{(1)} ≤ … ≤ x_{(p)}. i) If r(XF) = k then the best linear unbiased estimator μ^{∗} of μ is written as

μ^{∗} = F (F^{T} X^{T} B^{−1}XF)^{−1}F^{T} X^{T} B^{−1}y

with the quadratic loss

R_{μ}(μ^{∗}) := E(μ − μ^{∗})^{T} (μ − μ^{∗}) = σ^{2}trF (F^{T} X^{T} B^{−1}XF)^{−1}F^{T} .  (2)

ii) Unbiasedness of a linear estimator θ̃ of the realization of θ is understood as Eθ̃ = Eθ. The best linear unbiased estimator is given by

θ̂ = μ + SX^{T} B^{−1}(y − Xμ)

when μ is known and by

θ^{∗} = μ^{∗} + SX^{T} B^{−1}(y − Xμ^{∗})

if r(XF) = k, otherwise. Moreover,

R_{θ}(θ̂) = σ^{2}tr(S − SX^{T} B^{−1}XS),  (3)
R_{θ}(θ^{∗}) = R_{θ}(θ̂) + σ^{2}tr (I_{n} − SX^{T} B^{−1}X)F (F^{T} X^{T} B^{−1}XF)^{−1}F^{T} (I_{n} − X^{T} B^{−1}XS). (4)

Let us denote

M_{pn} = {X : X is a p × n matrix, R(X^{T}) ⊂ A},
M′_{pn} = {X ∈ M_{pn} : R(X^{T}) = A},
H = {X ∈ M_{pn} : (XX^{T})_{jj} ≤ a_{j}, j = 1, …, p},
H′ = {X ∈ M′_{pn} : (XX^{T})_{jj} ≤ a_{j}, j = 1, …, p}

given positive numbers {a_{j}}_{j=1}^{p}.

We consider the following optimal design problems :

i) find Arg min_{X∈H′} R_{μ}(μ^{∗}) (estimation of μ); ii) find Arg min_{X∈H} R_{θ}(θ̂) (estimation of the realization of θ, μ is known) or Arg min_{X∈H′} R_{θ}(θ^{∗}) (estimation of the realization of θ, μ is unknown).

Let us note that (3) does not depend on {f_{i}}_{i=1}^{k} and, as a consequence, does not depend on A. On the other hand, (2) and (4) do not depend on the choice of {f_{i}}_{i=1}^{k} spanned A. That is why without loss of generality we can assume that {f_{i}}_{i=1}^{k} are any orthonormal vectors spanned A. In this case it is worth to note that X ∈ M_{pn} implies X = GF^{T} with a p×k matrix G. As a consequence, XF = G, XF F^{T} = X, XX^{T} = GG^{T} and if X, Z ∈ M_{pn} and XF = ZF then X = Z.

The problem under consideration is also an optimal design for additional experiments. It arises when after m experiments one is going to perform r ones more. Such a problem also occurs when only a part of the design matrix, say r×n submatrix, is available for choosing, while the rest m rows are fixed, m + r = p. Thus, the design matrix X is partitioned onto two the parts: X = ( \begin{matrix}X_{1} \\ X_{2} \end{matrix} ) with X_{1} ∈ M′_{mn} and X_{2} ∈ M_{rn}. An optimal design problem in this setting under the estimation problems considered is to choose X_{2} ∈ H_{0} in the best way, where

H_{0} = {X ∈ M_{rn} : (XX^{T})_{ii} ≤ b_{i}, i = 1, …, r}

given positive numbers {b_{i}}_{i=1}^{r}.

A lot of papers were devoted to the problems of unknown parameters estimating in random coefficient linear regression models but only few of them touched optimal designs as well. Below the works close to the subject are indicated.

A survey on random coefficient regression models can be found in Spjotvoll (1977). The emphasis was put there on estimation problems for expectation or covariance matrix of θ. The same model was also studied by Borjas (1982) and Pfeffermann (1984). Borjas (1982) explored different estimators of μ. Pfeffermann (1984) considered problems of the best estimating not only μ but a realization of θ as well both when μ is known or not. Optimal design problems in the specific random coefficient linear regression model allowing to apply the classical equivalence theorem were studied in the particular case of known μ by Gladitz and Pilz (1982). They considered the best linear unbiased estimator of the realization of θ and gave the algorithm for calculation of the optimal design matrix. The picture would be incomplete without mention of the paper written by Chou (1993), where the optimal design problem for non-random θ but additional experiments and the special set of designs was considered.

The paper is organized as follows. Two auxiliary results and an algorithm that turns out to be basical for calculation of the optimal design matrix are presented in Section 2. Our algorithm is simplier than that of Gladitz and Pilz (1982). Main results and their proofs can be found in Section 3. Section 4 consists of an example. At the end we give remarks on how to generalize the results obtained.

2. Auxiliary results

Let e_{i}, i ≥ 1, be the ith unit vector in R^{n}, {t_{i}}_{i=1}^{k} be a set of any orthonormal vectors in R^{n}. Denote by Z = diag (z_{1}, …, z_{n}) the diagonal matrix with positive numbers {z_{i}}_{i=1}^{n} in descending order of magnitude and let {d_{i}}_{i=1}^{k} be any sequence of k ≤ n positive numbers in ascending order of magnitude.

Lemma 1

min_{{t_{i}}_{i=1}^{k}} tr (Z + ∑_{i=1}^{k} d_{i}t_{i}t_{i}^{T})^{−1} = ∑_{i=1}^{n−k} z_{i}^{−1} + ∑_{j=1}^{k}(z_{n−k+j} + d_{j})^{−1},

Arg min_{{t_{i}}_{i=1}^{k}} tr (Z + ∑_{i=1}^{k} d_{i}t_{i}t_{i}^{T})^{−1} = {e_{n−k+i}}_{i=1}^{k}.

Proof. Using known formula for the inverse matrix (see Rao (1973), p. 33) successively k times we get

(Z + ∑_{i=1}^{k} d_{i}t_{i}t_{i}^{T})^{−1} = Z_{k}^{−1} = Z_{k−1}^{−1} − \frac{d_{k}(Z_{k−1}^{−1}t_{k})(t_{k}^{T} Z_{k−1}^{−1})}{1 + d_{k}t_{k}^{T} Z_{k−1}^{−1}t_{k}} = Z^{−1} − ∑_{j=1}^{k} \frac{d_{j}(Z_{j−1}^{−1}t_{j})(t_{j}^{T} Z_{j−1}^{−1})}{1 + d_{j}t_{j}^{T} Z_{j−1}^{−1}t_{j}},

where Z_{j} = Z + ∑_{i=1}^{j} d_{i}t_{i}t_{i}^{T} and Z_{0} = Z. Therefore,

tr (Z + ∑_{i=1}^{k} d_{i}t_{i}t_{i}^{T})^{−1} = tr Z^{−1} − ∑_{j=1}^{k} \frac{d_{j}t_{j}^{T} Z_{j−1}^{−2}t_{j}}{1 + d_{j}t_{j}^{T} Z_{j−1}^{−1}t_{j}} = tr Z^{−1} − ∑_{j=1}^{k} \frac{d_{j}t_{j}^{T} Z^{−2}t_{j}}{1 + d_{j}t_{j}^{T} Z^{−1}t_{j}} (5)

since t_{j}^{T} Z_{j−1}t_{j} = t_{j}^{T} Zt_{j} for j ≥ 1. Hence, what we need is to find

max_{\{t_{j}\}_{j=1}^{k}} ∑_{j=1}^{k} \frac{d_{j}t_{j}^{T} Z^{−2}t_{j}}{1 + d_{j}t_{j}^{T} Z^{−1}t_{j}} = max_{\{t_{j}\}_{j=1}^{k}} ∑_{j=1}^{k} \frac{t_{j}^{T} Z^{−2}t_{j}}{t_{j}^{T} (Z^{−1} + d_{j}^{−1}I_{n})t_{j}} .

Due to the extremal properties of the underlying ratio (see Rao(1973), pp. 63, 74) we obtain

max_{\{t_{j}\}_{j=1}^{k}} ∑_{j=1}^{k} \frac{t_{j}^{T} Z^{−2}t_{j}}{t_{j}^{T} (Z^{−1} + d_{j}^{−1}I_{n})t_{j}} = ∑_{i=1}^{k} \frac{1}{z_{n−k+i}} + ∑_{i=1}^{k} \frac{1}{z_{n−k+i} + d_{i}}  (6)

and the maximum is attained when t_{i} = e_{n−k+i}, i = 1, …, k.

It remains to combine (5) and (6) and Lemma 1 is proved.

Let v ∈ R^{k}, k ≤ p, and λ ∈ R^{p} be two given vectors with positive components, v_{1} ≥ … ≥ v_{k}, λ_{1} ≤ … ≤ λ_{p}. Denote

D_{λ} = {x = (x_{1}, …, x_{p})^{T} : 0 ≤ x_{1} ≤ … ≤ x_{p}, ∑_{i=1}^{j} x_{i} ≤ ∑_{i=1}^{j} λ_{i}, j = 1, …, p}.

It is worth to note here that in terms of weak upper majorization (≺^{w}) for two vectors (see Marshall and Olkin (1979), p. 18) D_{λ} can be rewritten as

D_{λ} = {x ∈ R^{p} : 0 ≤ x_{1} ≤ … ≤ x_{p}, λ ≺^{w} x}.

The next result plays the crucial role for calculation of optimal designs. Lemma 2 There exists a unique vector τ^{v,λ} ∈ D_{λ} defined as

τ^{v,λ} = Arg min_{x∈D_{λ}} ∑_{i=1}^{k}(v_{i} + x_{p−k+i})^{−1}.

It has the following properties:

(i) at least p − k first components are zeros;

(ii) if τ_{p−k+i}^{v,λ} > 0 then v_{i} + τ_{p−k+i}^{v,λ} ≤ v_{i+1} + τ_{p−k+i+1}^{v,λ};

(iii) if τ_{p−k+i}^{v,λ} = 0 then v_{i} ≥ v_{i+1} + τ_{p−k+i+1}^{v,λ};

(iv) if v_{i} + τ_{p−k+i}^{v,λ} < v_{i+1} + τ_{p−k+i+1}^{v,λ} then ∑_{j=1}^{i} τ_{p−k+j}^{v,λ} = ∑_{j=1}^{p−k+i} λ_{j}.

This result can be obtained in the same manner as Theorem 1 in Chan and Li (1989). Follow Lemma 2 we can propose the algorithm for construction of τ^{v,λ}.

1. Define j_{0} = min{1 ≤ j ≤ p−1 : v_{p−j} ≥ min{ϕ_{j1}, …, ϕ_{jj}}}, otherwise j_{0} = p, where ϕ_{jm} = m^{−1}(∑_{i=1}^{m} v_{p−j+i} + ∑_{i=1}^{p−j+m} λ_{i}), m = 1, …, j; j = 1, …, p. Set τ_{i}^{v,λ} = 0, i ≤ p − j_{0}.

2. If ϕ_{j_{0}j_{0}} = min{ϕ_{j_{0}1}, …, ϕ_{j_{0}j_{0}}} then τ_{p−j_{0}+m}^{v,λ} = ϕ_{j_{0}j_{0}} − v_{p−j_{0}+m}, m = 1, …, j_{0}. Otherwise, go to the next step.

3. Let j_{1} = min{1 ≤ m ≤ j_{0} − 1 : ϕ_{j_{0}j_{0}} > ϕ_{j_{0}m}}. Then τ_{p−j_{0}+m}^{v,λ} = ϕ_{j_{0}j_{1}} − v_{p−j_{0}+m}, m = 1, …, j_{1}. Define ϕ′_{m} = m^{−1} ∑_{i=1}^{m}(v_{p−j_{0}+j_{1}+i} + λ_{p−j_{0}+j_{1}+i}), m = 1, …, j_{0} − j_{1}. If ϕ′_{j_{0}−j_{1}} = min{ϕ′_{1}, …, ϕ′_{j_{0}−j_{1}}} then τ_{p−j_{0}+j_{1}+m}^{v,λ} = ϕ′_{j_{0}−j_{1}} − v_{p−j_{0}+j_{1}+m}, m = 1, …, j_{0} − j_{1}. Otherwise, go to the next step.

4. Let j_{2} = min{1 ≤ m ≤ j_{0} − j_{1} − 1 : ϕ′_{j_{0}−j_{1}} > ϕ′_{m}}. Then τ_{p−j_{0}+j_{1}+m}^{v,λ} = ϕ_{j_{2}} − v_{p−j_{0}+j_{1}+m}, m = 1, …, j_{2}. Define ϕ′′_{m} = m^{−1} ∑_{i=1}^{m}(v_{p−j_{0}+j_{1}+j_{2}+i} + λ_{p−j_{0}+j_{1}+j_{2}+i}), m = 1, …, j_{0} − j_{1} − j_{2} and so on.

Obviously, we need only a finite number of steps to construct τ^{v,λ}.

3. Main results

We begin with the optimal design under the estimation of μ. Theorem 1

Arg min_{X∈H′} R_{μ}(μ^{∗}) = ∑_{i=1}^{k}(τ_{p−k+i}^{1_{k},a})^{1/2}u_{i}f_{i}^{T} ,

where {u_{i}}_{i=1}^{k} are orthonormal vectors in R^{p} such that (∑_{i=1}^{k} τ_{p−k+i}^{1_{k},a}u_{i}u_{i}^{T})_{jj} = a_{j}, j = 1, …, p, a = (a_{(1)}, …, a_{(p)})^{T} , 1_{k} = (1, …, 1)^{T} ∈ R^{k}. Proof. According to (2)

R_{μ}(μ^{∗}) = σ^{2}tr (G^{T} (I_{p} + GS_{F} G^{T})^{−1}G)^{−1} = σ^{2}tr(G^{T} G)^{−1} + σ^{2}trS_{F} ,

where G = XF, S_{F} = F^{T} SF. Let {γ_{i}^{2}}_{i=1}^{k} be the eigenvalues of G^{T} G in ascending order of magnitude. Then the problem is reduced to the minimization of ∑_{i=1}^{k} γ_{i}^{−2} provided

(0, …, 0, γ_{1}^{2}, …, γ_{k}^{2})^{T} ∈ D_{a}. Clearly, Arg min_{x∈D_{a}} ∑_{i=1}^{k} x_{p−k+i}^{−1} = Arg min_{x∈D_{a}} ∑_{i=1}^{k}(1+x_{p−k+i})^{−1}

while the solution of the last minimization problem can be found in Lemma 2. Having G, an optimal design matrix is obtained as X = GF^{T} . Theorem 1 is proved.

In the case of additional experiments let {c_{i}}_{i=1}^{k} be the nonzero eigenvalues of X_{1}^{T} X_{1} in descending order of magnitude and {g_{i}}_{i=1}^{k} be the corresponding orthonormal eigenvectors. Further on, we consider only the case r ≤ k (r > k is investigated similarly). Theorem 2

Arg min_{X_{2}∈H_{0}} R_{μ}(μ^{∗}) = ∑_{j=1}^{r}(τ_{j}^{c,b})^{1/2}u_{j}g_{j}^{T} ,

where {u_{j}}_{j=1}^{r} are orthonormal vectors in R^{r} such that (∑_{j=1}^{r} τ_{j}^{c,b}u_{j}u_{j}^{T})_{ii} = b_{i}, i = 1, …, r, c = (c_{1}, …, c_{r})^{T} , b = (b_{(1)}, …, b_{(r)})^{T} . Proof. Denoting G_{1} = X_{1}F, G_{2} = X_{2}F, we get

F^{T} X^{T} B^{−1}XF = ( G_{1}^{T} G_{2}^{T} ) ( \begin{matrix}I_{m} + G_{1}S_{F} G_{1}^{T} & G_{1}S_{F} G_{2}^{T} \\ G_{2}S_{F} G_{1}^{T} & I_{r} + G_{2}S_{F} G_{2}^{T} \end{matrix} )^{−1} ( \begin{matrix}G_{1} \\ G_{2} \end{matrix} ) .

Let T = G_{1}^{T} (I_{m}+G_{1}S_{F} G_{1}^{T})^{−1}G_{1}. Clearly, T^{−1} exists since G_{1} is of full rank. Using formulae for inverse matrices (see Rao (1973), p. 33), we obtain

F^{T} X^{T} B^{−1}XF = T + (I_{n} − T S_{F})G_{2}^{T} (I_{r} + G_{2}(S_{F} − S_{F} T S_{F})G_{2}^{T})^{−1}G_{2}(I_{n} − S_{F} T) (7)

and

R(μ^{∗}) = σ^{2}tr (T^{−1} − (T^{−1} − S_{F})G_{2}^{T} (I_{r} + G_{2}(T^{−1} − S_{F})G_{2}^{T})^{−1}G_{2}(T^{−1} − S_{F})) .

The identity T^{−1} = (G_{1}^{T} G_{1})^{−1} + S_{F} implies

R(μ^{∗}) = σ^{2}tr S_{F} + σ^{2}tr (G_{1}^{T} G_{1})^{−1} − σ^{2}tr (I_{r} + G_{2}(G_{1}^{T} G_{1})^{−1}G_{2}^{T})^{−1}G_{2}(G_{1}^{T} G_{1})^{−2}G_{2}^{T} . (8)

Tedious manipulations yield R(μ^{∗}) = σ^{2}tr S_{F} + σ^{2}tr (G_{1}^{T} G_{1})^{−1} − σ^{2}tr G_{2}^{T} (I_{r} + G_{2}(G_{1}^{T} G_{1})^{−1}G_{2}^{T})^{−1}G_{2}(G_{1}^{T} G_{1})^{−2} = σ^{2}tr S_{F} + σ^{2}tr (G_{1}^{T} G_{1})^{−1} − σ^{2}tr (G_{1}^{T} G_{1})^{−1}G_{2}^{T} G_{2}(I_{r} + (G_{1}^{T} G_{1})^{−1}G_{2}^{T} G_{2})^{−1}(G_{1}^{T} G_{1})^{−1} = σ^{2}tr S_{F} + σ^{2}tr (I_{r} + (G_{1}^{T} G_{1})^{−1}G_{2}^{T} G_{2})^{−1}(G_{1}^{T} G_{1})^{−1} = σ^{2}tr S_{F} + σ^{2}tr (G_{1}^{T} G_{1} + G_{2}^{T} G_{2})^{−1} .

Since X_{2}X_{2}^{T} = G_{2}G_{2}^{T} , it remains to minimize tr (G_{1}^{T} G_{1}+G_{2}^{T} G_{2})^{−1} provided (G_{2}G_{2}^{T})_{jj} ≤ b_{j}, j = 1, …, r.

Using of Lemmas 1 and 2 completes the proof of Theorem 2.

Now we find the optimal design under the estimation of a realization of θ.

It is convenient to rewrite (3) and (4) as

R_{θ}(θ̂) = σ^{2}tr S − σ^{2}tr (I_{p} + XSX^{T})^{−1}XS^{2}X^{T} ,  (9)

and

R_{θ}(θ^{∗}) = R_{μ}(μ^{∗}) + σ^{2} (tr S − 2tr F^{T} SF ) ,  (10)

respectively.

To minimize (10) is the same as (2), therefore, the optimal designs here are the same as those in Theorems 1 and 2. The minimization of (9) with respect to X ∈ H repeats a part of the proof of Theorem 2 (compare the last terms in the right sides of (8) and (9) with X and S instead of G_{2} and (G_{1}^{T} G_{1})^{−1}, respectively). The result is given below in Theorem 3 (cf. Theorem 3 of Gladitz and Pilz (1982)).

Denote by {d_{i}}_{i=1}^{q} the nonzero eigenvalues of S in ascending order of magnitude and by {q_{i}}_{i=1}^{q} the corresponding orthonormal eigenvectors, q ≤ n.

Theorem 3

Arg min_{X∈H} R_{θ}(θ̂) = ∑_{i=1}^{q}(τ_{p−q+i}^{d,a})^{1/2}u_{i}q_{i}^{T} ,

where {u_{i}}_{i=1}^{q} are orthonormal vectors in R^{p} such that (∑_{i=1}^{q} τ_{p−q+i}^{d,a}u_{i}u_{i}^{T})_{jj} = a_{j}, j = 1, …, p, d = (d_{1}^{−1}, …, d_{q}^{−1})^{T} .

The case of additional experiments, i. e. minimization of (9) with respect to X_{2} ∈ H_{0}, is considered in the next theorem. It is not difficult to understand that making use of the known formulae for inverse matrices (see (7)) we can reduce this case to that considered in Theorem 3. Let {h_{i}}_{i=1}^{s} be the nonzero eigenvalues of S − SX_{1}^{T} (I_{n} + X_{1}SX_{1}^{T})^{−1}X_{1}S in ascending order of magnitude and {p_{i}}_{i=1}^{s} be the corresponding orthonormal eigenvectors, s ≤ n. Again for definiteness suppose that r ≤ s.

Theorem 4

Arg min_{X_{2}∈H_{0}} R_{θ}(θ̂) = ∑_{j=1}^{r}(τ_{j}^{h,b})^{1/2}u_{j}p_{j}^{T} ,

where {u_{j}}_{j=1}^{r} are orthonormal vectors in R^{r} such that (∑_{j=1}^{r} τ_{j}^{h,b}u_{j}u_{j}^{T})_{ii} = b_{i}, i = 1, …, r, h = (h_{1}^{−1}, …, h_{r}^{−1})^{T} .

4. Example

Consider a problem that can arise under conducting physical experiments. Suppose we need to extract the information about the signal {θ(t), t ∈ [0, 1]}, that is a Wiener process with a drift μ(t), basing on observations

y_{i} = ∫_{0}^{1} x_{i}(t)θ(t)dt + ξ_{i}, i = 1, …, k.

It is assumed here that we can measure the signal k times on different levels (they are characterized by the functions {x_{i}(t), t ∈ [0, 1]}_{i=1}^{k} which are in our disposal) while {ξ_{i}}_{i=1}^{k} are independent random errors of measurements which are also independent of θ(t) for each t ∈ [0, 1] and such that Eξ_{i} = 0, Eξ_{i}^{2} = σ^{2}, i = 1, …, k. The drift is known to be a linear combination of the functions {ϕ_{i}(t), t ∈ [0, 1]}_{i=1}^{k}, where ϕ_{i}(t) = sin π(k − i + 1/2)t.

We will construct the optimal design based on k observations as well as on r ≤ k additional ones after using the first design. Let

H = {{x_{i}}_{i=1}^{k} ⊂ A : ∫_{0}^{1} x_{i}^{2}(t)dt ≤ 1, i = 1, …, k},
H_{0} = {{x_{j}}_{j=1}^{r} ⊂ A : ∫_{0}^{1} x_{j}^{2}(t)dt ≤ 1, j = 1, …, r},
H′ = {{x_{i}}_{i=1}^{k} ⊂ A, linearly independent : ∫_{0}^{1} x_{i}^{2}(t)dt ≤ 1, i = 1, …, k},

where A is the subspace of L_{2}[0, 1] spanned by {ϕ_{i}}_{i=1}^{k}. Consider two kinds of problems.

1. Drift is unknown and we would like to estimate it or a trajectory of θ based on the functions from A. It is easily seen that τ^{1_{k},1_{k}} = 1_{k}. According to Theorem 1, the best decision is to choose as {x_{i}}_{i=1}^{k} ∈ H′ a set of any orthonormal functions spanned A. Then, in the case of r additional experiments, due to Theorem 2, we should again choose as {x_{j}}_{j=1}^{r} ∈ H_{0} a set of any r orthonormal functions from A. The result obtained is easily tractable: more information about the sygnal can be obtained using orthogonal functions.

2. Drift is known and we would like to estimate a trajectory of θ based on the functions from A. In this case S is of full rank matrix with the eigenvalues d_{i} = π^{−2}(k − i + 1/2)^{−2} and the corresponding orthonormal eigenvectors 2^{1/2}ϕ_{i}(t), t ∈ [0, 1], i = 1, …, k. Therefore, d = π^{2}/4((2k − 1)^{2}, …, 1)^{T} . Let us calculate τ^{d,1_{k}} using the algorithm given in Section 2.

Define j_{0} = min{j ≥ 1 : ∑_{i=1}^{j}(j + i)(j − i + 1) ≥ π^{−2}k}. Then, τ_{i}^{d,1_{k}} = 0, i ≤ k − j_{0},

τ_{k−j_{0}+i}^{d,1_{k}} = \frac{k + π^{2} ∑_{s=1}^{j_{0}}(i − s)(2j_{0} − s − i + 1)}{j_{0}}, i = 1, …, j_{0}.

According to Theorem 3,

Arg min_{{x_{i}}_{i=1}^{k}∈H} R_{θ}(θ̂) = {∑_{s=1}^{j_{0}}(2τ_{k−j_{0}+s}^{d,1_{k}})^{1/2}u_{si} sin π(j_{0} − s + 1/2)t, t ∈ [0, 1]}_{i=1}^{k},

where {u_{1}, …, u_{j_{0}}} are orthonormal vectors in R^{k} such that ∑_{s=1}^{j_{0}} τ_{k−j_{0}+s}^{d,1_{k}}u_{sj}^{2} = 1, j = 1, …, k.

E. g. for k = 20 we have τ_{i} = 0, i ≤ 18, τ_{19} = 10 − π^{2}, τ_{20} = 10 + π^{2}. Then the optimal design is written as {(20−2π^{2})^{1/2}u_{1i} sin 3πt/2+(20+2π^{2})^{1/2}u_{2i} sin πt/2, t ∈ [0, 1]}_{i=1}^{k}, where {u_{1}, u_{2}} are any orthonormal vectors such that (10+π^{2})u_{1i}^{2} +(10−π^{2})u_{2i}^{2} = 1, i = 1, …, k. We can take e. g. u_{1} = 20^{−1/2}1_{k}, u_{2} = 20^{−1/2}(−1, 1, …, −1, 1)^{T} . Then the optimal design is {(1 − π^{2}/10)^{1/2} sin 3πt/2 + (−1)^{i}(1 + π^{2}/10)^{1/2} sin πt/2, t ∈ [0, 1]}_{i=1}^{k}.

In the case of r ≤ k additional experiments, {(d_{i}^{−1} + τ_{i}^{d,1_{k}})^{−1}}_{i=1}^{k} are the eigenvalues of S − SX_{1}^{T} (I_{n} + X_{1}SX_{1}^{T})^{−1}X_{1}S with {2^{1/2}ϕ_{i}(t), t ∈ [0, 1]}_{i=1}^{k} as the corresponding orthonormal eigenvectors. Then the components of the vector h are the r largest numbers in descending order of magnitude from the set {π^{2}(2k − 1)^{2}/4, …, π^{2}(2j_{0} + 1)^{2}/4, π^{2}(2j_{0} − 1)^{2}/4 + τ_{k−j_{0}+1}^{d,1_{k}}, …, π^{2}/4 + τ_{k}^{d,1_{k}}}. Calculating τ^{h,1_{r}} we obtain an optimal design on H_{0} according to Theorem 4. E. g. the solution for k = 20 and r = 10 is {2^{1/2} sin πt/2, …, 2^{1/2} sin πt/2, t ∈ [0, 1]}.

5. Concluding remarks

1. The results obtained could be generalized towards considering instead of the sets H and H′ any compact sets in M_{pn} and M′_{pn}, respectively. Show how to do this in a particular case when we estimate μ. Let H̃ ⊂ M′_{pn} be a compact set. Define the function f : H̃ ⟶ R^{p} as f(X) = (λ_{1}(XX^{T}), …, λ_{p}(XX^{T}))^{T} , X ∈ H̃, where λ_{1}(XX^{T}) ≤ … ≤ λ_{p}(XX^{T}) are the eigenvalues of XX^{T} . Since f is continuous then the set f(H̃) is compact. Therefore, the solution to the problem min_{x∈f(_{H})} ∑_{i=1}^{k} x_{p−k+i}^{−1} exists and it is unique. Denote

(0, …, 0, τ_{1}, …, τ_{k})^{T} = Arg min_{x∈f(_{H})} ∑_{i=1}^{k} x_{p−k+i}^{−1} Then, Arg min_{X∈_{H}} R_{μ}(μ^{∗}) = ∑_{i=1}^{k}(τ_{i})^{1/2}u_{i}f_{i}^{T} ,

where {u_{i}}_{i=1}^{k} are orthonormal vectors in R^{p} providing ∑_{i=1}^{k}(τ_{i})^{1/2}u_{i}f_{i}^{T} ∈ H̃.

2. As one could observe, throughout the paper covariance matrix of θ is assumed to be known until a constant multiplier. Answering possible sceptic notes on this matter we refer to Gladitz and Pilz (1982), where the motivation for such an assumption is given.

References

Borjas, G.J. (1982). On regressing regression coefficients. J. Stat. Plan. Inf., 7, 131-137. Chan, N.N. and Li, Kim-Hung. (1989). Majorization for A-optimal designs. J. Math. Anal. Appl., 142, 101-107. Chou, K.L. (1993). A-optimal designs for supplementary experiments. Theory Probab.

Appl., 37, 1, 126-129. Gladitz, J. and Pilz, J. (1982). Construction of optimal designs in random coefficient regression models. Math.-Oper. und Stat. Ser. Stat., 13, 3, 371-385. Marshall, A.W. and Olkin I. (1979). Inequalities: Theory of Majorization and Its Applications. Academic Press, New York. Pfeffermann, D. (1984). On extensions of the Gauss-Markov theorem to the case of stochastic regression coefficients. J. Roy. Stat. Soc. Ser.B, 46, 139-148. Rao, C.R. (1973). Linear Statistical Inference and Its Applications. Wiley, New York. Spjotvoll, E. (1977). Random coefficients regression models. A review. Math.-Oper. und Stat. Ser. Stat., 8, 1, 69-93.

OPTYMALNE PLANY W MODELU REGRESJI ZE WSPÓLCZYNNIKAMI LOSOWYMI Streszczenie

W modelu regresji liniowej ze wspólczynnikami losowymi rozważone zostały zadania optymalnego, wzgle�dem kwadratowej funkcji strat, planowania doświadczeń przy estymacji realizacji wektora wspólczynników lub ich wektora średnich. Zakladamy że wektor średnich należy do zadanej z góry podprzestrzeni liniowej Euklidesowej przestrzeni parametrów. Rozważony również został przypadek eksperymentów dodatkowych. Slowa kluczowe: model regresji liniowej ze wspólczynnikami losowymi, plan optymalny, najlepszy estymator liniowy nieobcia�żony, eksperymenty dodatkowe. Klasyfikacja AMS 1993: 62K05, 62J05.