One, Two, U: Examples of common one- and two-sample U-statistics

My previous two blog posts revolved around derivation of the limiting distribution of U-statistics for one sample and multiple independent samples.

For derivation of the limiting distribution of a U-statistic for a single sample, check out Getting to know U: the asymptotic distribution of a single U-statistic.

For derivation of the limiting distribution of a U-statistic for multiple independent samples, check out Much Two U About Nothing: Extension of U-statistics to multiple independent samples.

The notation within these derivations can get quite complicated and it may be a bit unclear as to how to actually derive components of the limiting distribution.

In this blog post, I provide two examples of both common one-sample U-statistics (Variance, Kendall’s Tau) and two-sample U-statistics (Difference of two means, Wilcoxon Mann-Whitney rank-sum statistic) and derive their limiting distribution using our previously developed theory.

Asymptotic distribution of U-statistics

One sample

For a single sample, , the U-statistic is given by

where is a symmetric kernel of degree .

For a review of what it means for to be symmetric, check out U-, V-, and Dupree Statistics.

In the examples covered by this blog post, , so we can re-write as,

Alternatively, this is equivalent to,

The limiting variance of is given by,

where

or equivalently,

Note that when , .

For , these expressions reduce to

where

and

The limiting distribution of for is then,

For derivation of the limiting distribution of a U-statistic for a single sample, check out Getting to know U: the asymptotic distribution of a single U-statistic.

Two independent samples

For two independent samples denoted and , the two-sample U-statistic is given by

where is a kernel that is independently symmetric within the two blocks and .

In the examples covered by this blog post, , reducing the U-statistic to,

The limiting variance of is given by,

where

and

Equivalently,

and

For , these expressions reduce to

where

and

The limiting distribution of for and is then,

For derivation of the limiting distribution of a U-statistic for multiple independent samples, check out Much Two U About Nothing: Extension of U-statistics to multiple independent samples.

Much Two U About Nothing: Extension of U-statistics to multiple independent samples

Thank you very much to the lovely Feben Alemu for pointing me in the direction of https://pungenerator.org/ as a means of ensuring we never have to go without a brilliant title! With great power comes great responsibility.

Review

Statistical functionals are any real-valued function of a distribution function , . When is unknown, nonparametric estimation only requires that belong to a broad class of distribution functions , typically subject only to mild restrictions such as continuity or existence of specific moments.

For a single independent and identically distributed random sample of size , , a statistical functional is said to belong to the family of expectation functionals if:

1. takes the form of an expectation of a function with respect to ,

2. is a symmetric kernel of degree .

A kernel is symmetric if its arguments can be permuted without changing its value. For example, if the degree , is symmetric if .

If is an expecation functional and the class of distribution functions is broad enough, an unbiased estimator of can always be constructed. This estimator is known as a U-statistic and takes the form,

such that is the average of evaluated at all distinct combinations of size from .

For more detail on expectation functionals and their estimators, check out my blog post U-, V-, and Dupree statistics.

Since each appears in more than one summand of , the central limit theorem cannot be used to derive the limiting distribution of as it is the sum of dependent terms. However, clever conditioning arguments can be used to show that is in fact asymptotically normal with mean

and variance

where

The sketch of the proof is as follows:

1. Express the variance of in terms of the covariance of its summands,

1. Recognize that if two terms share common elements such that,

conditioning on their shared elements will make the two terms independent.

2. For , define

such that

and

Note that when , and , and when , and .

3. Use the law of iterated expecation to demonstrate that

and re-express as the sum of the ,

Recognizing that the first variance term dominates for large , approximate as

4. Identify a surrogate that has the same mean and variance as but is the sum of independent terms,

so that the central limit may be used to show

5. Demonstrate that and converge in probability,

and thus have the same limiting distribution so that

For a walkthrough derivation of the limiting distribution of for a single sample, check out my blog post Getting to know U: the asymptotic distribution of a single U-statistic.

This blog post aims to provide an overview of the extension of kernels, expectation functionals, and the definition and distribution of U-statistics to multiple independent samples, with particular focus on the common two-sample scenario.

Getting to know U: the asymptotic distribution of a single U-statistic

After my last grand slam title, U-, V-, and Dupree statistics I was really feeling the pressure to keep my title game strong. Thank you to my wonderful friend Steve Lee for suggesting this beautiful title.

Overview

A statistical functional is any real-valued function of a distribution function such that

and represents characteristics of the distribution and include the mean, variance, and quantiles.

Often times is unknown but is assumed to belong to a broad class of distribution functions subject only to mild restrictions such as continuity or existence of specific moments.

A random sample can be used to construct the empirical cumulative distribution function (ECDF) ,

which assigns mass to each .

is a valid, discrete CDF which can be substituted for to obtain . These estimators are referred to as plug-in estimators for obvious reasons.

For more details on statistical functionals and plug-in estimators, you can check out my blog post Plug-in estimators of statistical functionals!

Many statistical functionals take the form of an expectation of a real-valued function with respect to such that for ,

When is a function symmetric in its arguments such that, for e.g. , it is referred to as a symmetric kernel of degree . If is not symmetric, a symmetric equivalent can always be found,

where represents the set of all permutations of the indices .

A statistical functional belongs to a special family of expectation functionals when:

1. , and
2. is a symmetric kernel of degree .

Plug-in estimators of expectation functionals are referred to as V-statistics and can be expressed explicitly as,

so that is the average of evaluated at all possible permutations of size from . Since the can appear more than once within each summand, is generally biased.

By restricting the summands to distinct indices only an unbiased estimator known as a U-statistic arises. In fact, when the family of distributions is large enough, it can be shown that a U-statistic can always be constructed for expectation functionals.

Since is symmetric, we can require that , resulting in combinations of the subscripts . The U-statistic is then the average of evaluated at all distinct combinations of ,

While within each summand now, each still appears in multiple summands, suggesting that is the sum of correlated terms. As a result, the central limit theorem cannot be relied upon to determine the limiting distribution of .

For more details on expectation functionals and their estimators, you can check out my blog post U-, V-, and Dupree statistics!

This blog post provides a walk-through derivation of the limiting, or asymptotic, distribution of a single U-statistic .

U-, V-, and Dupree statistics

To start, I apologize for this blog’s title but I couldn’t resist referencing to the Owen Wilson classic You, Me, and Dupree ā wow! The other gold-plated candidate was U-statistics and You. Please, please, hold your applause.

My previous blog post defined statistical functionals as any real-valued function of an unknown CDF, , and explained how plug-in estimators could be constructed by substituting the empirical cumulative distribution function (ECDF) for the unknown CDF . Plug-in estimators of the mean and variance were provided and used to demonstrate plug-in estimators’ potential to be biased.

Statistical functionals that meet the following two criteria represent a special family of functionals known as expectation functionals:

1) is the expectation of a function with respect to the distribution function ; and

2) the function takes the form of a symmetric kernel.

Expectation functionals encompass many common parameters and are well-behaved. Plug-in estimators of expectation functionals, named V-statistics after von Mises, can be obtained but may be biased. It is, however, always possible to construct an unbiased estimator of expectation functionals regardless of the underlying distribution function . These estimators are named U-statistics, with the āUā standing for unbiased.

This blog post provides 1) the definitions of symmetric kernels and expectation functionals; 2) an overview of plug-in estimators of expectation functionals or V-statistics; 3) an overview of unbiased estimators for expectation functionals or U-statistics.