U-statistics Archives • Page 2 of 2 • Statisticelle

After my last grand slam title, U-, V-, and Dupree statistics I was really feeling the pressure to keep my title game strong. Thank you to my wonderful friend Steve Lee for suggesting this beautiful title.

Overview

A statistical functional is any real-valued function of a distribution function $F$ such that

$\theta = T(F)$

and represents characteristics of the distribution $F$ and include the mean, variance, and quantiles.

Often times $F$ is unknown but is assumed to belong to a broad class of distribution functions $\mathcal{F}$ subject only to mild restrictions such as continuity or existence of specific moments.

A random sample $X_1, …, X_n \stackrel{i.i.d}{\sim} F$ can be used to construct the empirical cumulative distribution function (ECDF) $\hat{F}_n$ ,

$\hat{F}_{n}(x) = \frac{1}{n} \sum_{i=1}^{n} \mathbb{I}(X_i \leq x)$

which assigns mass $\frac{1}{n}$ to each $X_i$ .

$\hat{F}_{n}$ is a valid, discrete CDF which can be substituted for $F$ to obtain $\hat{\theta} = T(\hat{F}_n)$ . These estimators are referred to as plug-in estimators for obvious reasons.

For more details on statistical functionals and plug-in estimators, you can check out my blog post Plug-in estimators of statistical functionals!

Many statistical functionals take the form of an expectation of a real-valued function $\phi$ with respect to $F$ such that for $a \leq n$ ,

$\theta = T(F) = \mathbb{E}_{F}~ \phi(X_1, …, X_a) .$

When $\phi(x_1, …, x_a)$ is a function symmetric in its arguments such that, for e.g. $\phi(x_1, x_2) = \phi(x_2, x_1)$ , it is referred to as a symmetric kernel of degree $a$ . If $\phi$ is not symmetric, a symmetric equivalent $\phi^{*}$ can always be found,

$\phi^{*}(x_1, …, x_a) = \frac{1}{a!} \sum_{\pi ~\in~ \Pi} \phi(x_{\pi(1)}, …, x_{\pi(a)})$

where $\Pi$ represents the set of all permutations of the indices $1, …, a$ .

A statistical functional $\theta = T(F)$ belongs to a special family of expectation functionals when:

$T(F) = \mathbb{E}_F ~\phi(X_1, …, X_a)$ , and
$\phi(X_1, …, X_a)$ is a symmetric kernel of degree $a$ .

Plug-in estimators of expectation functionals are referred to as V-statistics and can be expressed explicitly as,

$V_n = \frac{1}{n^a} \sum_{i_1 = 1}^{n} … \sum_{i_a = 1}^{n} \phi(X_{i_1}, …, X_{i_a})$

so that $V_n$ is the average of $\phi$ evaluated at all possible permutations of size $a$ from $X_1, …, X_n$ . Since the $X_i$ can appear more than once within each summand, $V_n$ is generally biased.

By restricting the summands to distinct indices only an unbiased estimator known as a U-statistic arises. In fact, when the family of distributions $\mathcal{F}$ is large enough, it can be shown that a U-statistic can always be constructed for expectation functionals.

Since $\phi$ is symmetric, we can require that $1 \leq i_1 < ... < i_a \leq n$ , resulting in ${n \choose a}$ combinations of the subscripts $1, ..., a$ . The U-statistic is then the average of $\phi$ evaluated at all ${n \choose a}$ distinct combinations of $X_1, ..., X_n$ ,

$U_n = \frac{1}{{n \choose a}} \mathop{\sum … \sum} \limits_{1 \leq i_1 < ... < i_a \leq n} \phi(X_{i_1}, ..., X_{i_a}).$

While $i_j \neq i_k$ within each summand now, each $X_i$ still appears in multiple summands, suggesting that $U_n$ is the sum of correlated terms. As a result, the central limit theorem cannot be relied upon to determine the limiting distribution of $U_n$ .

For more details on expectation functionals and their estimators, you can check out my blog post U-, V-, and Dupree statistics!

This blog post provides a walk-through derivation of the limiting, or asymptotic, distribution of a single U-statistic $U_n$ .

Continue reading Getting to know U: the asymptotic distribution of a single U-statistic