class

ExponentialFamily

extendsDistribution

ExponentialFamily(batch_shape: tuple[int, ...] = (), event_shape: tuple[int, ...] = (), validate_args: bool | None = None)

source edit

Mix-in for exponential-family distributions.

An exponential-family distribution has a density of the form

p(x \mid \eta) = h(x)\, \exp\!\bigl(\eta^\top T(x) - A(\eta)\bigr)

where:

$\eta$ are the natural parameters (also called canonical parameters),
$T(x)$ is the vector of sufficient statistics,
$A(\eta) = \log Z(\eta)$ is the log-partition function (log-normaliser), and
$h(x)$ is the base measure.

This family encompasses most common distributions: Normal, Bernoulli, Gamma, Beta, Poisson, Dirichlet, and many more.

Subclasses provide _natural_params and _log_normalizer so that entropy can be derived from the standard exponential-family identity

H[X] = A(\eta) - \eta^\top \nabla_\eta A(\eta)

Per-distribution overrides of Distribution.entropy remain available for closed-form efficiency.

Parameters

batch_shapetuple[int, ...]= ()

Shape of independent, non-identical draws. For a scalar parameter this is (), for a vector parameter of length n it is (n,). Default is ().

event_shapetuple[int, ...]= ()

Shape of a single event (observation). Univariate distributions have event_shape = (). Multivariate distributions such as lucid.distributions.Dirichlet have a non-empty event_shape. Default is ().

validate_argsbool or None= None

If True, the parameters are checked against arg_constraints at construction, and the value given to log_prob, cdf and icdf is checked before it is scored; a violation raises ValueError. None takes the default, which is on and is changed for every distribution by set_default_validate_args. Default is None.

Notes

The exponential family structure implies:

The log-normalizer $A(\eta)$ is convex in $\eta$ .
$\nabla_\eta A(\eta) = \mathbb{E}[T(X)]$ (the mean parameter).
The Hessian $\nabla^2_\eta A(\eta)$ equals the covariance matrix of $T(X)$ , hence is always positive semi-definite.

These properties underpin many elegant results in information geometry, variational inference, and natural gradient methods.

Members of this family in lucid.distributions include lucid.distributions.Normal, lucid.distributions.Bernoulli, lucid.distributions.Categorical, lucid.distributions.Gamma, lucid.distributions.Beta, lucid.distributions.Dirichlet, lucid.distributions.Exponential, and lucid.distributions.Poisson. Heavy-tailed families such as lucid.distributions.Cauchy and Student's-t are not members.

Examples

>>> import lucid
>>> from lucid.distributions import Bernoulli
>>> d = Bernoulli(probs=0.7)  # ExponentialFamily subclass
>>> isinstance(d, lucid.distributions.distribution.ExponentialFamily)
True
>>> d.entropy()
tensor(0.6109)

Used by 7

Instance methods

entropy

→Tensor

entropy()

source edit

Compute Shannon entropy via the Bregman divergence of _log_normalizer.

H[p] = A(\eta) - \langle \eta,\, \nabla_\eta A(\eta) \rangle - \mathbb{E}_p[k(X)]

Returns

Tensor

Entropy tensor of shape batch_shape.

>>> import lucid >>> from lucid.distributions import Bernoulli >>> d = Bernoulli(probs=0.7) # ExponentialFamily subclass >>> isinstance(d, lucid.distributions.distribution.ExponentialFamily) True >>> d.entropy() tensor(0.6109)