About Order Statistic

06 May, 2024

Given random variable $X~f$, we can get a sample of size $n$. At times, the cost of sort can be lower than that of calculation, we can get the so called order statistics by sorting the sample in ascending order:

By this, we lose all information of the sequence of draw.

Cumulative and Distribution Function

Notice that $X_{(k)}\le x$ means that at least $k$ observations of the sample is less than $x$, we have

By differentiating $F_{(k)}(x)$, we can obtain its density function:

which can be simplified into

Heuristically, this can be understood as selecting $k-1$ elements and ensure that they are less than $X_{(k)}$ and then select $n-k$ elements and ensure that they are greater than $X_{(k)}$, and in the end multiply the density of the $k$-th $X$.

$X_{(k)}$ as an Estimator

$Y=F(X)~U(0,1)$ for any continuous distribution $X~f$, where $F={\int }_{\mathbb{R}}fdx$. This is easy to prove: $$ P(Y\le y)=P\left(X\le F^{{-1}(y)\right)=F\left(F}{-1}(y)\right)=y, $$

An interesting property of $X_{(k)}$ is that its an unbiased estimator of the $k$-th $(n+1)$-quantile of $f$. To deal with quantile, a lemma may come handy: thus $Y~U(0,1)$. We now can proceed to prove that $E(Y_{(k)})=\frac{k}{n+1}$:

Now notice that $Y_{(k)}=F(X_{(k)})$,

Admittedly the last step is very clumsy, will need to be fixed.

#"hs-maths"