The Shape of What's Allowed: How Support Constraints Determine Universality Classes

1. Introduction

The central limit theorem (CLT) is one of the most powerful results in probability theory. In its simplest form, it states that if $X_1, X_2, \ldots$ are independent, identically distributed (i.i.d.) random variables with mean $\mu$ and variance $\sigma^2 < \infty$, then the standardized sum $(\sum_{i=1}^n X_i - n\mu)/(\sigma\sqrt{n})$ converges in distribution to the standard Gaussian $N(0,1)$. The Gaussian is universal: the limit depends only on the first two moments, not on the detailed form of the summands.

But the CLT makes no assumption about the support of the $X_i$. What happens when the summands are constrained to lie in a subset $S \subsetneq \mathbb{R}^d$? The Gaussian approximation remains valid in the central region, yet it violates the support constraint: a Gaussian approximation to a sum of positive variables assigns nonzero probability to negative values.

It has long been recognized, in various specific contexts, that support-respecting approximations outperform the Gaussian. Castro and Cuesta [1] recently proved that sums of independent positive random variables are better approximated by the Gamma distribution than by the Gaussian. Barthe et al. [2] showed that marginals of uniform distributions on $\ell^p$ balls converge to the generalized Gaussian $\propto e^{-|t|^p}$. Classical results connect the Dirichlet distribution to Pólya urns on the simplex [3], and the Beta distribution to various bounded-support limits.

The purpose of this paper is to argue that all of these are instances of a single principle:

The support of the summands determines the universality class. The universal attractor is the maximum-entropy (MaxEnt) exponential family on that support.

Table 1 summarizes the known instances.

What is new. The individual entries of Table 1 are established or known results. What is new in this paper is threefold. First, we state a general theorem (Theorem 1) that unifies all cases: for any closed convex support $S$ equipped with a natural exponential family, the I-projection of the empirical measure onto that family yields the universal attractor, with $O(n^{-1})$ convergence in KL divergence. The proof combines Csiszár's conditional limit theorem [4] with the multivariate saddlepoint approximation. Second, we propose the Barrier Function Conjecture (Conjecture 1), which asserts that the logarithmic barrier of $S$ determines the sufficient statistics and hence the universality class. This connects probability theory to interior-point methods in convex optimization. Third, we assemble evidence from five independent mathematical perspectives—information geometry, reflecting diffusions, free probability, Schrödinger bridges, and transform methods—all of which point to the same principle, creating what we call an "overdetermination argument" for its correctness.

2. Preliminaries

2.1 Notation

Let $S \subseteq \mathbb{R}^d$ be a closed convex set with nonempty interior relative to its affine hull. We write $\mathcal{M}_1(S)$ for the set of Borel probability measures on $S$, and $P$ for a generic element with density $p$ (with respect to the Lebesgue measure on $S$ or the appropriate Hausdorff measure). The Kullback–Leibler divergence is $D_{\mathrm{KL}}(Q \| P) = \int \log(dQ/dP)\, dQ$ when $Q \ll P$, and $+\infty$ otherwise. The cumulant generating function (CGF) of a random variable $X$ is $\Lambda(\theta) = \log \mathbb{E}[e^{\theta \cdot X}]$, with Fenchel–Legendre transform $\Lambda^*(x) = \sup_\theta [\theta \cdot x - \Lambda(\theta)]$.

2.2 I-projection and exponential families

2.3 MaxEnt on constrained supports

For each support $S$ in Table 1, the MaxEnt distribution subject to appropriate moment constraints is a well-known exponential family. We record the key cases.

3. Main Results

3.1 General theorem

We impose the following conditions on the support $S$, the sufficient statistics $T$, and the base distribution $P$.

Conditions.

3.2 Corollaries for specific supports

4. The Barrier Function Conjecture

Theorem 1 takes the sufficient statistics $T$ as given. But which $T$ is "natural" for a given support $S$? On $[0,\infty)$, why $(x, \log x)$ and not $(x, x^2)$? We propose that the answer is determined by the boundary geometry of $S$.

4.1 Verification for known cases

$S = [0,\infty)$. Here $g(x) = x$ and $\phi(x) = -\log x$. The barrier-exponential family is $p(x) \propto x^{\alpha-1} e^{-\lambda x}$, i.e., the Gamma family. Confirmed.

$S = [0,1]$. Here $g_1(x) = x$, $g_2(x) = 1{-}x$, and $\phi(x) = -\log x - \log(1{-}x)$. The barrier-exponential family is $p(x) \propto x^{\alpha-1}(1{-}x)^{\beta-1}$, i.e., the Beta family. Confirmed.

$S = \Delta_{k-1}$. Here $g_j(\boldsymbol{x}) = x_j$ and $\phi(\boldsymbol{x}) = -\sum_j \log x_j$. The family is $p(\boldsymbol{x}) \propto \prod_j x_j^{\alpha_j - 1}$, i.e., the Dirichlet. Confirmed.

$S = \{x : \|x\|_2 \le R\}$ (Euclidean ball). Here $g(x) = R^2 - \|x\|^2$ and the barrier-exponential family is $p(x) \propto (R^2 - \|x\|^2)^{\alpha-1} \exp(\lambda \cdot x)$, a generalized matrix-variate Beta-type density. This is consistent with the known marginal structure from uniform distributions on Euclidean balls.

$S = \ell^p$ ball. The barrier is $\phi(x) = -\log(1 - \sum_j |x_j|^p)$. The barrier-exponential family with $\lambda = 0$ gives $p(x) \propto (1 - \sum |x_j|^p)^{\alpha-1}$, whose one-dimensional marginals converge to $\propto e^{-c|t|^p}$ as the dimension $n \to \infty$ [2]. Confirmed.

4.2 Connection to self-concordant barriers

In the theory of interior-point methods for convex optimization [26], the self-concordant barrier is the central object. A barrier $\phi$ is self-concordant if $|D^3\phi(x)[h,h,h]| \le 2(D^2\phi(x)[h,h])^{3/2}$ for all $h$.

The logarithmic barrier is self-concordant for polyhedral $S$ (this is the starting point of Nesterov and Nemirovskiĭ [26]). The number of constraint functions $|J|$ is the barrier parameter, which controls the complexity of the interior-point method.

We observe the following parallel. In constrained optimization, the logarithmic barrier provides a smooth interior path to the constrained optimum. In constrained universality, the same barrier function provides the sufficient statistics for the smooth exponential family that serves as the universal attractor. Both uses of the barrier encode the same information: the geometry of the boundary $\partial S$.

A full proof of Conjecture 1 would require showing that the steep, regular exponential family on $S$ with sufficient statistics adapted to the boundary is unique (up to reparametrization) and equals the barrier-exponential family. This is plausible for polyhedral $S$ but may require additional structure for curved boundaries.

5. Alternative Perspectives

The I-projection proof of Theorem 1 is our primary argument. We briefly summarize five alternative perspectives, each providing independent evidence for the constrained universality principle.

5.1 Reflecting diffusions

Consider the reflecting stochastic differential equation on $S$:

$$dX_t = -\nabla V(X_t)\, dt + \sqrt{2}\, dW_t + d\ell_t,$$

where $\ell_t$ is a boundary local-time process enforcing reflection at $\partial S$. By the Lions–Sznitman theory [21], for convex $S$ and mild growth conditions on $V$, this process has a unique stationary measure

$$\pi_V(dx) \propto e^{-V(x)} \mathbf{1}_S(x)\, dx.$$

Setting $V(x) = -\eta \cdot T(x)$ reproduces the MaxEnt exponential family exactly. The boundary local time $\ell_t$ acts as an automatic Lagrange multiplier enforcing the support constraint, just as $\eta$ enforces the moment constraint.

The Fokker–Planck equation with no-flux boundary conditions confirms this directly. On $[0,\infty)$ with $V(x) = x/\beta - (\alpha{-}1)\log x$, the stationary density is $\mathrm{Gamma}(\alpha,\beta)$. On $[0,1]$ with $V(x) = -(\alpha{-}1)\log x - (\beta{-}1)\log(1{-}x)$, it is $\mathrm{Beta}(\alpha,\beta)$.

This perspective explains why the boundary matters: the reflecting boundary condition forces the stationary measure to be an exponential family restricted to $S$, rather than on all of $\mathbb{R}^d$.

5.2 Free probability

Voiculescu's free probability theory provides an independent confirmation. The free CLT gives the Wigner semicircle law as the universal limit for sums of freely independent variables [24]. The semicircle maximizes the free entropy $\chi(\mu) = \iint \log|x{-}y|\, d\mu(x)\, d\mu(y)$ among measures with given mean and variance.

On the positive half-line, the free analog is the Marchenko–Pastur (free Poisson) law [25], which maximizes $\chi$ among positive measures with given moments. The structural pattern is identical:

The fact that constrained universality holds in both frameworks—with the same structure but different entropy functionals—suggests a principle more general than either theory alone.

5.3 Optimal transport and Wasserstein gradient flows

The Jordan–Kinderlehrer–Otto (JKO) framework [22] interprets the Fokker–Planck equation as the Wasserstein gradient flow of the free energy functional $\mathcal{F}(\rho) = \int \rho \log \rho\, dx + \int V\rho\, dx$ on $\mathcal{P}_2(S)$. Setting $V = -\eta \cdot T$, the MaxEnt distribution is the unique minimizer of $\mathcal{F}$ and hence the unique fixed point of the Wasserstein flow on $\mathcal{P}_2(S)$.

A Barron-type argument [18] using the relative Fisher information and the log-Sobolev inequality for the Gamma distribution [23] gives $D_{\mathrm{KL}}(S_n \| G_n) \le C/n$, providing quantitative convergence without the saddlepoint machinery.

5.4 The overdetermination argument

Including the Schrödinger bridge and transform-method perspectives, the MaxEnt distribution on $S$ is simultaneously:

1. the I-projection (information geometry),
2. the stationary measure of a reflecting diffusion (stochastic dynamics),
3. the Wasserstein gradient-flow equilibrium (optimal transport),
4. the free-entropy maximizer (free probability),
5. the Padé fixed point of the cumulant generating function (transform theory).

This overdetermination—five independent characterizations yielding the same family—constitutes strong circumstantial evidence, even though no single perspective provides a complete standalone proof of the general conjecture.

6. The Renormalization Group Perspective

The CLT admits an elegant renormalization group (RG) formulation [17]. Define the convolution-and-rescale map $\mathcal{R}[P] = \mathrm{Law}((X_1 + X_2 - 2\mu)/(\sigma\sqrt{2}))$ with $X_1, X_2 \stackrel{\text{iid}}{\sim} P$. The Gaussian is the unique fixed point. At the level of characteristic functions, $\hat{\mathcal{R}}[\varphi](t) = \varphi(t/\sqrt{2})^2$, and the fixed-point equation $\varphi^*(t) = \varphi^*(t/\sqrt{2})^2$ with $\varphi^*(0) = 1$, $(\varphi^*)''(0) = -1$ has the unique solution $e^{-t^2/2}$.

Can the Gamma play an analogous role on $[0,\infty)$?

The Gamma family is nonetheless an invariant manifold: sums of Gammas with common rate remain Gamma. This suggests a two-timescale picture. Under iterated convolution of positive random variables:

1. Fast timescale: the distribution rapidly approaches the Gamma manifold (constrained universality).
2. Slow timescale: the distribution drifts along the Gamma manifold as the shape parameter $\alpha$ grows, eventually approaching the Gaussian (classical CLT, since $\mathrm{Gamma}(n\alpha, \beta) \to N(n\alpha\beta, n\alpha\beta^2)$).

The crossover occurs at $n \sim \alpha = \mu^2/\sigma^2$, which is the Gamma's "distance from Gaussianity." For small $n$, the Gamma is the better approximation; for large $n$, the Gaussian takes over.

We are not aware of prior work on constrained RG transformations (i.e., RG flows restricted to distributions on a proper subset of $\mathbb{R}$). Formalizing the Gamma as a "quasi-fixed-point" or "slow manifold" of such a constrained RG is an interesting open problem.

7. Numerical Evidence

We tested the constrained universality predictions numerically using Monte Carlo simulation. For each support $S$, we generated $10^5$ replications of the aggregate statistic $\bar{T}_n$ for several values of $n$ and source distributions, then compared the moment-matched MaxEnt family against the moment-matched Gaussian using the Kolmogorov–Smirnov (KS) statistic and an estimated KL divergence.

Positive sums ($S = [0,\infty)$). We tested with $X_i \sim \mathrm{Exp}(1)$, $\mathrm{Weibull}(0.5, 1)$, and $\mathrm{Uniform}(0,2)$. For the exponential and Weibull sources (high skewness), the Gamma outperforms the Gaussian at all tested $n$ ($n = 2, 5, 10, 20, 50$), with the advantage most pronounced at small $n$. The KS statistic for the Gamma is 2–10 times smaller than for the Gaussian. For $\mathrm{Uniform}(0,2)$ (low skewness, $\kappa_3/\sigma^3 = 0$), the Gaussian and Gamma perform comparably, consistent with Remark 2. In all cases, the Gaussian assigns nonzero probability to negative values, while the Gamma does not.

Bounded averages ($S = [0,1]$). We tested with $X_i \sim \mathrm{Beta}(2,5)$ and $X_i \sim \mathrm{Uniform}(0,1)$. The moment-matched Beta consistently outperforms the Gaussian for $n \ge 5$, with the advantage growing as $n$ increases (the Gaussian's support violation worsens relative to the concentration of $\bar{X}_n$). At $n = 50$, the Beta KS statistic is approximately 3 times smaller.

Simplex averages ($S = \Delta_2$). We tested with $X_i \sim \mathrm{Dir}(1,1,1)$ (uniform on the 2-simplex) and $X_i \sim \mathrm{Dir}(2,3,5)$. The moment-matched Dirichlet outperforms the multivariate Gaussian in terms of marginal KS statistics. The Gaussian assigns nonzero mass outside the simplex for all tested $n$.

Convergence rates. In all cases, the KL divergence between the empirical distribution and the MaxEnt approximation decreases as $O(n^{-1})$, consistent with Theorem 1(d).

8. Quantitative Beta Convergence via Stein's Method

The saddlepoint framework of Theorem 1 establishes $D_{\mathrm{KL}}(\bar{X}_n \| B_n) = O(n^{-1})$ for the Beta case but leaves open the question of convergence in stronger metrics—Wasserstein and total variation. We now address this using Stein's method for the Beta distribution, following the framework of Goldstein and Reinert [35] and Döbler [36].

8.1 Setup

Let $X_1, \ldots, X_n$ be i.i.d. on $[0,1]$ with mean $\mu \in (0,1)$, variance $\sigma^2 > 0$, and third absolute moment $\gamma_3 = \mathbb{E}[|X_i|^3] < \infty$ (automatically satisfied since $X_i \in [0,1]$). Set $W_n = \bar{X}_n$ and let $Z_n \sim \mathrm{Beta}(a_n, b_n)$ where the parameters are moment-matched:

$$a_n = \mu \nu_n, \quad b_n = (1{-}\mu)\nu_n, \quad \nu_n = \frac{n\mu(1{-}\mu)}{\sigma^2} - 1.$$

Write $\kappa = \mu(1{-}\mu)/\sigma^2 \geq 1$, so $\nu_n = n\kappa - 1$ and $a_n, b_n \to \infty$ as $n \to \infty$.

8.2 Exchangeable pairs and linear regression

Define the exchangeable pair $(W_n, W_n')$ by replacing a uniformly chosen summand: let $I \sim \mathrm{Uniform}\{1, \ldots, n\}$ independent of everything, $X_I'$ an independent copy of $X_I$, and $W_n' = W_n + (X_I' - X_I)/n$. A symmetry argument shows $(W_n, W_n')$ is exchangeable, and the linear regression property holds:

$$\mathbb{E}[W_n' - W_n \mid W_n] = \frac{\mu - W_n}{n} = -\frac{1}{n}(W_n - \mu).$$

The conditional second moment is

$$\mathbb{E}[(W_n' - W_n)^2 \mid W_n] = \frac{1}{n^2}\bigl(\sigma^2 + \mu^2 - 2\mu W_n + \mathbb{E}[Q_n \mid W_n]\bigr),$$

where $Q_n = \frac{1}{n}\sum_{i=1}^n X_i^2$.

8.3 Results

8.4 Comparison of approaches

9. Discussion

9.1 What is proved versus conjectured

We are explicit about the logical status of each claim.

Proved (under conditions (C1)–(C5)): The I-projection framework (Theorem 1) yields the MaxEnt exponential family as the saddlepoint approximation to the density of the aggregate statistic, with $O(n^{-1})$ error in KL divergence. This follows from Csiszár's theorems and standard saddlepoint analysis.

Conjectured: The Barrier Function Conjecture (Conjecture 1), which asserts that the logarithmic barrier of $S$ determines the sufficient statistics. This is verified for all known cases but not proved in general. The key difficulty is establishing uniqueness of the "natural" exponential family on $S$.

Proved (Section 8): The Beta convergence rate $d_W = O(n^{-1})$ in Wasserstein distance, via Stein's method with exchangeable pairs following Goldstein–Reinert [35] and Döbler [36]; and $d_{\mathrm{TV}} = O(n^{-1/2})$ in total variation via local CLT comparison. The $O(n^{-1/2})$ TV rate is sharp when the third cumulant of the source does not match that of the Beta target.

Open: Whether the constrained universality principle extends to non-convex supports (where the I-projection may not be unique), discrete supports (where local limit theorems replace density convergence), and whether $d_{\mathrm{TV}} = O(n^{-1})$ holds for all absolutely continuous sources on $[0,1]$ (Conjecture 2).

9.2 Open problems

1. Non-convex supports. If $S$ is not convex, the I-projection onto $\{Q : \mathrm{supp}(Q) \subseteq S,\; \mathbb{E}_Q[T] = \tau\}$ may not be unique. What replaces the exponential family in this case?
2. Discrete supports. For $S = \{0, 1, \ldots, N\}$, the MaxEnt family with mean constraint is the Binomial. Is there a "discrete Barrier Function Conjecture"? This connects to grammar-constrained decoding in large language models, where sampling is restricted to outputs conforming to a formal grammar.
3. Convergence rates. We prove $O(n^{-1})$ for KL divergence. Can this be improved to an Edgeworth-type expansion with explicit dependence on the boundary geometry?
4. The Beta gap (partially resolved). Section 8 establishes Beta convergence as a standalone limit theorem in Wasserstein distance at rate $O(1/n)$ and in total variation at rate $O(1/\sqrt{n})$. The remaining gap is whether $d_{\mathrm{TV}} = O(1/n)$ holds generally, or whether the $O(1/\sqrt{n})$ rate from third-cumulant mismatch is intrinsic to two-parameter moment matching. A "zero-bias" coupling for the Beta distribution, or a direct tilted Edgeworth expansion comparing to Beta rather than Gaussian, may close this gap.
5. Constrained RG. Formalizing the Gamma as a quasi-fixed-point or slow manifold of an RG flow on $\mathcal{M}_1([0,\infty))$ would complete the RG picture of constrained universality.