Thus

with

j = lim VarVn{ Fn,j (O ü Tn,f\j (s) dAn,j'\j (s) - P0j'ü P0j"} n Jo j"=j' j"

limVarVn{Fn ,j (s~) JJ Fn ,j»\j (s) - F°-(s) JJ j j (s)} dA0' \ j (s) n j"=j' j"=j'

{F°-(s) JJ Ff,\j(s)}2 lim Var^(dAn j\ j(s) - dA°,\ j(s)) j"=j' n fO m n fO —0 t-t —0 n dF0(s)

= I vjf'(s) A\j(s)+ {F°(s) JJ j\j(s)}2 F(Fs )2( ) ' j0 j0 j"=j' n°Gj (s)(Fj'\j)2(s)

= 52 j {II j }2 ^E pjj,j (II pjj, )j, jl j2=jl jl j2=jl j3=jl,j2

and similar calculations give the expression of

j = lim Cov[Vn{ ~Fn,j (O n 'pn,j"\j (s) dAn j/\ j (s) — pjj/Ii pjj", n Jj ■„

References

[ABGK93] Andersen,P.K., Borgan,O, Gill,R.D. and Keiding,N.: Statistical Models Based on Counting Processes. Springer, New York (1993) [BN02] Bagdonavicius,V. and Nikulin,M.N. : Accelerated Life Models, Modeling and Statistical Analysis. Chapman and Hall, London (2002) [GILL80] Gill,R. : Nonparametric estimation based on censored observations of a Markov renewal process. Z. Wahrsch. verw. Gebiete, 53, 97-116 (1980)

[GILL83] Gill,R. : Large sample behaviour of the product-limit estimator on the whole line. Ann. Statist., 11, 49-58 (1983) [HEUT02] Heutte,N. and Huber-Carol,C. : Semi-Markov Models for Quality of Life Data with Censoring. In: Emery, M. (ed)? Statistical Methods for Quality of Life Studies, Kluwer Academic Publishers, Boston (2002) [P0NS04] O. Pons: Estimation of semi-Markov models with right-censored data. In: Balakrishnan, N. and Rao, C.R. (ed) Handbook of Statistics, 23, Advances in Survival Analysis, Elsevier,? (2004)

[LSZ78] S.W. Lagakos, C. Sommer and M. Zelen: Semi-markov models for partially censored data. Biometrika, 65, 311317 (1978)

[PYKE61] R. Pyke: Markov renewal processes: definitions and preliminary properties, Ann. Math. Statist., 32, 1231-1342 (1961)

[TSI75] A. Tsiatis: A nonidentifiability aspect of the problem of competing risks, Proc. Nat. Acad. Sci. USA , 37, 20-22 (1975)

[VC84] J. Voelkel and J. Crowley: Nonparametric inference for a class of semi-markov processes with censored observations, Ann. Statist., 12, 142-160 (1984)

Estimation Of Density For Arbitrarily Censored And Truncated Data

Catherine Huberi, Valentin Solev2, and Filia Vonta3

1 Université René Descartes - Paris 5, 45 rue des Saints-P'eres, 75006 Paris, [email protected]

2 Steklov Institute of Mathematics at St. Petersburg, nab. Fontanki, 27 St.Petersburg 191023 Russia, [email protected]

3 Department of Mathematics and Statistics, University of Cyprus P.O. Box 20537, CY-1678, Nicosia, Cyprus, [email protected]

Summary. We consider survival data that are both interval censored and truncated. Turnbull [Tur76] proposed in 1976 a nice method for nonparametric maximum likelihood estimation of the distribution function in this case, which has been used since by many authors. But, to our knowledge, the consistency of the resulting estimate was never proved. We prove here the consistency of Turnbull's NPMLE under appropriate conditions on the involved distributions: the censoring, truncation and survival distributions.

Key words: incomplete observations, censored and truncated data, non-parametric maximum likelihood estimation, consistency.

Very often in practice, survival data are both interval censored and truncated, as observation of the process is not continuous in time and is done through a window of time which could exclude totally some individuals from the sample. For example, the time of onset of a disease in a patient, like HIV infection or toxicity of a treatment, is not exactly known, but it is usually known to have taken place between two dates ti and Î2] this occurs in particular when the event of interest results in an irreversible change of state of the individual: at time ti, the individual is in state one, while at time t2, he is in state two. Moreover, some people can escape the sample if they are observed during a period of time not including some pair of dates ti, t2 having the above property. Turnbull [Tur76] proposed a nice method for nonparametric maximum

2 The research of the second author was supported by grants RFBR 02-01-00262, grant RFBR-DFG 04-01-04000

likelihood estimation of the distribution function in this case. His method, slightly corrected by Frydman [Fry94], has been used extensively since by several authors, and extended to semi-parametric cases (Alioum and Commenges [ACo96], Huber-Carol and Vonta, [HbV04]). But, to our knowledge, the consistency of the resulting estimates was never proved, even in the simple totally nonparametric case. We give here conditions on the involved distributions, the censoring, truncation and survival distributions, implying the consistency of Turnbull's estimate. The proofs use results of Sara Van de Geer [VdG93], Xiatong Shen [Sh97], Wing Hung Wong and Xiatong Shen [WSh95], Lucien Birge and Pascal Massart [BiM98], Luc Devroye and Gabor Lugosi [DeL01], Nikulin and Solev [NiS02], [NiS04], on non-parametric estimation. In section two, we give a representation of the censoring and truncation mechanisms. As it is due to a non continuous observation of the survival process, the censoring mechanism is represented as a denumerable partition of the total interval of observation time (a,b). Then a truncation is added to the censoring, conditioning the observations both of the survival and the censoring processes. The particular case of right truncation is considered. In the next three sections, three distributions are successively studied, each being conditional on fixed values which become random in the next section. In section three, the distribution associated with a random covering, which is a censoring set conditional on a fixed value x of the survival process. It is considered as the sum of a denumerable number of elementary probabilities, and it is proved to have a density with respect to a baseline probability. In section four, we define the joint distribution of a pair of intervals, a censoring L(x), R(x) and a truncating one L(z), R(z), conditional on fixed values x and z respectively of the survival X and the right truncation Z. Finally, in section five, we consider the distribution of the incomplete observation of X: L(X),R(X),L(z),R(z), conditional on the truncating variable Z = z.

In section six, the non parametric maximum likelihood estimate of the density of the survival is defined in the presence of the nuisance infinite dimensional parameters introduced by the censoring and the truncation laws, using Kull-back and Hellinger distances.

Finally, in the last section, conditions are found on the sets of probabilities that govern the survival process and the censoring and truncation processes that lead to consistency of the NPMLE of the density of the survival process.

2 Partitioning the total observation time 2.1 Random covering.

Let "&(x) = (L(x), R(x)], x € (a,b) C R be a random covering of interval (a, b). That is "&(x) is a process, indexed by x € (a,b), which values are intervals, and such that with probability one x e (L(x),R(x)] C (a, b), (J -d(x) = (a,b).

When it is clear from the context, we shall identify process §(x) with vector valued process v(x) = (L(x),R(x)), whose coordinates are the left and right ends of interval §(x).

In the special case of a random covering §(x) generated by a random partition, for any x,y e (a, b), with probability one

Conversely, let us assume that condition (1) is true. Then, with probability one, the random function R(x) is a left continuous step function. Therefore, there exists a partition t t = {(Yj ,Yj+1], j = 0, ±1,...} , a<...<Y-m < ... <yq <...<Yn ...<b, \J (Yj,Yj+1] = (a,b), (2)

j such that

From now on we assume that the random covering §(x) = (L(x), R(x)], x e (a,b), satisfies condition (1) and hence may be generated by a partition t defined in (2) - (4). Such a random covering will be called a simple random covering. For simplicity we suppose that a = —m, b = to

2.2 Short-cut covering.

Let "&(x) = (L(x), R(x)], x e R, be a simple random covering, t be the partition associated with §(x), t = {(Yj ,Yj+i], j =0, ±1,...} ,

and A = (zi, z2] be an interval, and z = (zi, Z2), zi < Z2 For a fixed value of t = t, t = {(yj ,yj+l], j = 0, ±1,...},

... < y-m < ... <yo < ... <yn ..., define functions

Estimation Of Density For Arbitrarily Censored And Truncated Data 249 k1 = k1 (t,zi) = inf {k : yk > zi} 31 = 31 {t,zi) = YKl

K2 = K2(t,Z2) = sup {k : yk < Z2} 32 = 32(t, Z2) = YK2

The short-cut covering $a(x) = (La(x),Ra(x)], x G A, is defined below. The short-cut covering $a(x) is trivial: $a(x) = (z1, Z2], if

( (L(x),R(x)], if x G (31,32] (La(x),Ra(x)] = < (z1,31], if x G (z1,31]

In the special case when

we shall use notations for corresponding short-cut covering $a(x), x G A, and connected objects dz (x) = &a(x), kz = K(t,z)=sup {k : yk < z} , 3z = 3(t,z)= YK2 (5)

2.3 The mechanism of truncation and censoring

The mechanism of censoring and truncating of a random variable X is defined as follows. Let X be a random variable, A = (Z1,Z2] be a random interval, •&(x) = (L(x), R(x)], x G R, be a random covering , generated by a partition

T = {(Yj, Yj+1 ], j = 0, ±1,...} , ... < Y-m < ... <Yo < ... <Yn

We denote

We suppose that random covering $(■), random variable X and random interval A are independent, but we have not complete observations. More precisely, we suppose that random vector (X, Z1,Z2) is partly observable only in the case when (L(X),R(X)] C A:

In that case the available observations are the interval (L(X),R(X)] of the covering $(■), which contains X, and random interval A* = (R(Z1),L(Z2)].

250 Catherine Huber, Valentin Solev, and Filia Vonta When (L(X), R(X)] C A we have not any observation. We have to think that

1) Conditionally on a fixed value t of t the random interval A is taken from the truncated distribution

Pt {A} = P {A € A | the interval [Z\, Z2] contains at least two points of A} .

In other words, conditionally on fixed values of t = t the random vector Z = (Zi, Z2) is taken from the truncated distribution

2) Conditionally on a fixed value of t = t and A = A = (zi,z2], the random variable X is taken from truncated distribution

In other words conditionally on fixed values of t = t and Zi = zi,Z2 = z2 the random variable X is taken from truncated distribution

P {A | t,zi,z2} = P {X € A | X € (31 (t, zi), i2(t, z2)]} .

We consider the simple case when for a random variable Z random interval A = (-m, Z], and use the notations that were given in (5). We denote Z the random variable

We have to think that

1) The random covering $(■) and the random variable Z are independent.

2) Conditionally on a fixed value of 3 = 3, the random variable X is taken from the truncated distribution

In other words, conditionally on fixed values of t = t and Z = z the random variable X is taken from the truncated distribution

3 The distribution associated with random covering.

Let "&(x) = (L(x),R(x)], x € R, be a simple random covering. The distribution Px of random vector v(x) = (L(x), R(x)) will be called the distribution, associated with random covering §(x).

We assume that for all x the distribution Px has density with respect to Lebesgue measure A2 on the plane R2,

, , dPx rx(u,v) = —, and plan to prove in this case, that there exists a nonnegative function r(u, v) such that for all x rx(u,v) = r(u,v)H(UiV](x) (a.s.)

Function r(u,v) will be called the baseline density of simple random covering •d(x). It is clear that function r(u,v) is the density of a a-finite measure, but, for all x, function r(u,v)H(UtV](x) is the density of a probability measure. It is clear that for all x rx(u,v) = rx(u,v)l(u,v](x).

For positive x < y and nonnegative measurable function '(u,v) such that

'(u, v) = 0, if u < x < v < y or x < u < y < v. (6)

Condition (6) is equivalent to the condition (on function ')

Therefore

= E E, Yj+i) H(Yj,Yi+1](x) = E E, Yj+i) 1IY,Yj+1](y) = kk

Thus, under condition (6) on function '

II'(u,v)rx(u,v)dudv) = II'(u,v)ry(u,v)dudv, u<v u<v and we obtain for all u < x < y < v rx(u,v) = ry (u,v). (7)

From (7) we conclude that there exists a nonnegative function r(u,v), whose support is the set {(u,v) : u < v}, and such that for x rx(u,v) = r(u,v)1l(u,v](x) (a.s.)

It is easy to see that the baseline density r(u,v) depends only on the joint distributions of vectors (Yj, Yj+i)-

Now we prove that measure Px is absolutely continuous with respect to the Lebesgue measure for all x if and only if

(i) for all j the distribution of vector (Yj, Yj+i) has density rj(u,v) with respect to the Lebesgue measure,

(ii) the series rj (u,v) converges a.s. to a function r(u,v), j

(iii) the function r(u,v) satisfies the following condition: for all x rx(u,v) = r(u, v)S(„,v](x).

Indeed, suppose that for all x the distribution Px has density rx(u, v). Let 4(u, v) be a nonnegative function, then for all j

E 4(Yj, Yj+i)H(Y.t Yj+i] (x) < E 4(L(x), R(x)) =

Therefore, for all x the distribution of vector (Yj, Y'+i)H(Yj, Yj+1](x) has a density. Hence, the distribution of vector (Yj, Yj+i) also has a density rj(u, v). We have

Now suppose that (i), (ii) are fulfilled. Then we obtain for a nonnegative measurable function 4(u,v) (by the same way as above)

From this equality we conclude that series r(u,v) ^^ rj(u,v) < to (a.s.), j r and rx(u,v) = r(u,v)l(u,v](x).

4 The distribution of random vector

Now for x < z we denote by Px,z the distribution of random vector (L(x) ,R(x) ,L(z) ,R(z)). Denote by An the Lebesgue measure on Rn. The distribution Px z is not absolutely continuous with respect to the measure on A4. Denote by v the measure, which is defined for continuous nonnegative functions ^(s) = S2, S3, S4) by the relation

We suppose that the distribution Px z is absolutely continuous with respect to the measure v and denote its density qx,z(s):

We suppose that for all n,m > 0 the random vector (Y-m, ...,Yn) has a density with respect to the corresponding Lebesgue measure. For i +1 < j, let function ri,j(V\,V2,V3,V4) be the density of random vector (Yi,Yi+1,Yj,Yj+i), rj(V1,V2,V3) be the density of random vector (Yj-i,Yj,Yj+i),

rj (yi,V2) be the density of the random vector (Yj ,Yj+i).

We assume that d4(yi,y2,y3,y4) = E nj(yi,y2,y3,y4) < to (A4-a.s.), i,T-i+i<j

254 Catherine Huber, Valentin Solev, and Filia Vonta d3(si, S2, S3) = £ rj(s) < to (A3-a.s.), j j and d2(yi,V2) = E rj(yi,y2) < to (A2-a.s.). j

For a nonnegative function 4(x),x = (xi,x2,x3,xa) and x < z we have E 4(L(x),R(x),L(z),R(z)) = = E £ 4(Yi,Yi+i,Yj,Yj+i)iL(YitYi +l](x)HL{Y;,Yj+-\_] (z) =

+ E E 'HYj-i, Y,Yj,Yj+i)l(Yj_i,Yj] (x)H(Yj,Yj+i] (z

Thus,

+ ff 4(si, S2, Si, S2)d2(si, S2)l[(SliS2](x)lI(SliS2] (z) dsids2 +

+ fff ^(s1, s2, s2, s3)d3(su s2, s1,s2] (x)1I(s2,s3] (z) dsids2ds3 +

d4(si, S2, S3, S4)I(SiiS2](x)I(S3iS4](z) dsids2ds3ds4. If we define v—measurable function d(s|x, z), s = (si, S2, S3, S4), by d(s|x,z) = H(Si,S2] (x)A*(s|z), where d2(Si, S2)l(Si,S2](z), if Si = S3 < S2 = S4

d3(Si, S2, S3)II(S2,s3] (z), if Si < S2 = S3 < S4

d4(Si, S2, S3, S4)l(S3,S4](z), if Si < S2 < S3 < S4

Estimation Of Density For Arbitrarily Censored And Truncated Data 255

E ^(L(x),R(x),L(z),R(z)) = JJJJ ^(s)d(s | x,z) dv, and therefore

Was this article helpful?

## Post a comment