As an Amazon Associate, I earn from qualifying purchases. For more practice problems like this, see Schaum’s Outline of Statistics, 6th Edition.

1. What is being asked?

We have an i.i.d. sample

[ X_1,\dots ,X_n\stackrel{\text{i.i.d.}}{\sim}F!\left(\frac{x}{\sigma}\right),\qquad \sigma>0, ]

i.e. a scale family.
A statistic (S(X_1,\dots ,X_n)) is ancillary if its distribution does not depend on the unknown scale parameter (\sigma).

The post claims that

“(S) is ancillary iff it depends on the data only through the ratios
(\displaystyle \frac{X_1}{X_n},\dots ,\frac{X_{n-1}}{X_n}).”

The questions are

  1. Is the if part (sufficiency) true?
  2. Does an analogous statement exist for a general exponential family that is not a pure scale family?

We are asked to give a rigorous answer, with references, and to explain the geometry behind it.

2. Preliminaries

2.1 Ancillary statistics

A statistic (S=S(X)) is ancillary for a model ({P_\theta:\theta\in\Theta}) if

[ \forall \theta_1,\theta_2\in\Theta,\qquad P_{\theta_1}^{S}=P_{\theta_2}^{S}, ]

i.e. the distribution of (S) is the same for every parameter value.

2.2 Scale families and the scale group

Define the scale group

[ G={g_c(x)=c\,x : c>0}, ]

acting on the sample space ({\cal X}^n) by

[ g_c(x_1,\dots ,x_n)=(c x_1,\dots ,c x_n). ]

If (X\sim F(\cdot/\sigma)) then (g_{1/\sigma}X) has distribution (F) that does not involve (\sigma).
Hence the model is invariant under the group (G).

A maximal invariant for a group action is a measurable function (M) such that

  • (M(gX)=M(X)) for every (g\in G) (invariance), and
  • if (M(X)=M(Y)) then there exists a (g\in G) with (Y=gX) (maximality).

For the scale group the vector of ratios

[ M(X)=\bigl( X_1/X_n,\; X_2/X_n,\dots ,X_{n-1}/X_n \bigr) \tag{1} ]

is a maximal invariant (any two samples having the same ratios differ only by a common multiplicative constant).

2.3 Basu’s theorem

If (T) is complete sufficient for (\theta) and (A) is ancillary, then (T) and (A) are independent.

In a full regular exponential family the minimal sufficient statistic is complete, so any non‑trivial ancillary must be independent of it.

3. Scale families – proof of the claimed result

3.1 Statement

Theorem (Scale‑family ancillarity).
Let (X_1,\dots ,X_n) be i.i.d. from the scale family ({F(\cdot/\sigma),\ \sigma>0}).
A statistic (S=S(X_1,\dots ,X_n)) is ancillary iff there exists a Borel function (h) such that
[ S(X)=h!\bigl( X_1/X_n,\dots ,X_{n-1}/X_n \bigr)\quad\text{a.s.} \tag{2} ]

3.2 Proof – “only if’’ (necessity)

  1. Group‑invariance argument.
    The family is invariant under the scale group (G).
    If (S) is ancillary, then for every (\sigma>0)

    [ \mathcal L_\sigma(S)=\mathcal L_1(S), ]

    where (\mathcal L_\sigma) denotes the law under scale (\sigma).

  2. Transform the data.
    Write (Y_i = X_i/\sigma). Then (Y_i\stackrel{\text{i.i.d.}}{\sim}F) (parameter‑free).
    Hence

    [ S(X)=S(\sigma Y)=\tilde S(Y),\qquad \tilde S(Y)=S(\sigma Y), ]

    and (\tilde S(Y)) has a distribution that does not involve (\sigma).

  3. Invariance of the σ‑algebra generated by (S).
    For any (c>0),

    [ S(cX)=S(X) \quad\text{a.s.} ]

    (otherwise the distribution would change when we replace (\sigma) by (c\sigma)).
    Thus (S) is invariant under the action of (G).

  4. Maximal invariance.
    By the standard result from group theory, any (G)–invariant measurable function is a measurable function of a maximal invariant.
    Since (M) in (1) is a maximal invariant, there exists a measurable (h) with (2). ∎

3.3 Proof – “if’’ (sufficiency)

Assume (S=h(M)) for some measurable (h).

Because (M) is invariant under scaling, for any (\sigma>0),

[ M\bigl(X_1,\dots ,X_n\bigr) =M\bigl(\sigma Y_1,\dots ,\sigma Y_n\bigr) =M\bigl(Y_1,\dots ,Y_n\bigr), ]

where (Y_i=X_i/\sigma).
Consequently the distribution of (M) (hence of (S)) under any (\sigma) coincides with its distribution when (\sigma=1).
Therefore (S) is ancillary. ∎

3.4 References for the theorem

  • Lehmann & Casella, Theory of Point Estimation, 2nd ed., §4.5 (maximal invariants).
  • Eaton, Group Invariant Statistical Inference, Ch. 2 (Theorem 2.1).
  • Barndorff‑Nielsen (1978), Information and Exponential Families, §3.2.

4. What about a general exponential family?

4.1 Exponential family set‑up

A (full‑rank) regular exponential family on ({\cal X}) can be written as

[ p_\theta(x)=\exp!\bigl{\theta^\top T(x)-\psi(\theta)\bigr}h(x),\qquad \theta\in\Theta\subset\mathbb R^k, \tag{3} ]

where (T(x)) is the canonical statistic (dimension (k)), (\psi) the log‑partition, and (h) a carrier measure.

The minimal sufficient statistic for (\theta) is (T_n=\sum_{i=1}^n T(X_i)).
When the family is full (i.e. the natural parameter space has non‑empty interior) (T_n) is also complete.

4.2 Consequence of completeness

Basu’s theorem ⇒ any ancillary statistic must be independent of (T_n).
But (T_n) is complete; the only random variables that are independent of a complete sufficient statistic are constants (or functions of a statistic that is ancillary and independent of (T_n)).

Hence for a full regular exponential family there are no non‑trivial ancillary statistics.

Result. In a full‑rank exponential family the only ancillary statistics are trivial (a.s. constant).

This already tells us that a universal expression analogous to the ratios in the scale case cannot exist for the whole class.

4.3 When non‑trivial ancillaries do appear

Non‑trivial ancillaries arise in two main situations:

Situation Reason for ancillarity Example
Curved exponential families (dimension of (\theta) < dimension of (T)) The model lives on a lower‑dimensional submanifold of the full exponential family; directions orthogonal to the submanifold are nuisance directions that generate ancillaries. Normal model with unknown mean (\mu) but known variance (\sigma^2) – the sample variance is ancillary for (\mu).
Group‑invariant submodels (e.g. location, scale, shape groups) The group action provides a maximal invariant that is ancillary. Gamma(shape‑scale) family: (X_i/\bar X) (or ratios) are ancillary for the scale parameter.
Mixture of exponential families The mixing variable creates a separate “parameter” that only affects the mixing distribution, producing an ancillary component. Poisson–Gamma mixture (negative binomial) – the proportion of zero counts can be ancillary for the Poisson mean.

Thus, ancillarity is a model‑specific property, not a universal functional form.

4.4 Geometric view

Statistical manifolds: The family (3) is a smooth (k)-dimensional submanifold ({\cal M}) of the space of all densities (the “exponential‑family manifold”).

Normal bundle: At a point (\theta) the tangent space (T_\theta{\cal M}) consists of scores (\partial\log p_\theta/\partial\theta). Directions orthogonal (with respect to the Fisher information metric) to the tangent space are precisely those that do not change the likelihood to first order.

When the model is a group orbit (e.g. a scale family), the orbit direction is a one‑dimensional subspace; the orthogonal complement is the space of maximal invariants, and the statistic built from that complement is ancillary.

In a full exponential family the normal bundle is trivial (its dimension is zero) because the family already fills the ambient space; consequently there is no non‑trivial ancillary submanifold.

References for the geometry:

  • Amari & Nagaoka, Methods of Information Geometry, Ch. 3.

Original question: Is there a general expression for ancillary statistics in exponential families? on Cross Validated (Stats Stack Exchange), licensed CC BY-SA.