class

Multinomial

extendsDistribution

Multinomial(total_count: Tensor | int = 1, probs: Tensor | None = None, logits: Tensor | None = None, validate_args: bool | None = None)

source edit

Multinomial distribution over $K$ categories.

Multivariate generalisation of the lucid.distributions.Bernoulli /Binomial: the joint distribution of the counts $(k_1, \ldots, k_K)$ obtained when $n = \text{total\_count}$ independent draws are made from a Categorical with probabilities $(p_1, \ldots, p_K)$ . The $k_i$ are non-negative integers summing to $n$ .

The event dimension is the last axis of probs / logits; all preceding axes form the batch shape.

Parameters

total_countint or Tensor= 1

Total number of trials

n \geq 0

. Default 1 (a one-hot Categorical sample).

probsTensor= None

Unnormalised category probabilities of shape (..., K) — normalised internally to sum to 1 along the last axis. Mutually exclusive with logits.

logitsTensor= None

Unnormalised log-probabilities of shape (..., K), converted via softmax. Mutually exclusive with probs.

validate_argsbool= None

If True, validate parameter constraints at construction time.

Notes

Probability mass function (joint over the count vector $\mathbf{k}$ with $\sum_i k_i = n$ ):

P(\mathbf{K} = \mathbf{k} \mid n, \mathbf{p}) = \binom{n}{k_1, \ldots, k_K} \prod_{i=1}^{K} p_i^{k_i}, \qquad \binom{n}{k_1, \ldots, k_K} = \frac{n!}{\prod_i k_i!}

Moments (per category):

\mathbb{E}[K_i] = n p_i, \qquad \mathrm{Var}[K_i] = n p_i (1 - p_i), \qquad \mathrm{Cov}[K_i, K_j] = -n p_i p_j \;\; (i \neq j)

Special cases:

$K = 2$ → $\mathrm{Binomial}(n, p_1)$
$n = 1$ → one-hot lucid.distributions.Categorical

Conjugate prior: lucid.distributions.Dirichlet — observing counts $\mathbf{k}$ updates Dirichlet(α) → Dirichlet(α + k).

Sampling is non-reparameterised (has_rsample = False) and implemented by summing $n$ one-hot Categorical draws.

Examples

>>> import lucid
>>> from lucid.distributions import Multinomial
>>> d = Multinomial(total_count=10, probs=lucid.tensor([0.2, 0.3, 0.5]))
>>> d.mean  # n * p
tensor([2., 3., 5.])
>>> x = d.sample((4,))
>>> x.shape
(4, 3)
>>> bool((x.sum(dim=-1) == 10).all())  # every draw places all n trials
True

Used by 1

lucid.distributions

Constructors

dunder

init

→None

__init__(total_count: Tensor | int = 1, probs: Tensor | None = None, logits: Tensor | None = None, validate_args: bool | None = None)

source edit

Initialise a Multinomial distribution.

Parameters

total_countTensor | int= 1

Total number of draws

n \geq 0

. Must be a scalar. Default is 1 (reduces to a one-hot Categorical).

probsTensor | None= None

The category weights, as probabilities or as log-probabilities, of shape (..., K) — exactly one of the two. Either is normalised over the last axis before it is stored.

logitsTensor | None= None

The category weights, as probabilities or as log-probabilities, of shape (..., K) — exactly one of the two. Either is normalised over the last axis before it is stored.

validate_argsbool | None= None

Whether to check the parameters here and the counts later scored.

Raises

ValueError

If both or neither of probs and logits are given, or — under validation — a normalised parameter is not a probability vector or total_count is not a non-negative whole number.

Properties

prop

mean

→Tensor

mean: Tensor

source edit

n · p (element-wise).

prop

support

→Constraint

support: Constraint

source edit

Count vectors of total_count draws.

Returns

Constraint

multinomial(total_count) — non-negative entries summing to at most total_count, the reference framework's support.

prop

total_count

→Tensor

total_count: Tensor

source edit

Total number of trials $n$ per Multinomial draw.

Returns

Tensor

The integer-valued n parameter. Each independent Multinomial sums to this many trials across the K categories.

prop

variance

→Tensor

variance: Tensor

source edit

n · p · (1 − p) (element-wise).

Instance methods

entropy

→Tensor

entropy()

source edit

Shannon entropy of the Multinomial distribution (in nats).

H = n H(\mathbf p) - \ln n! + \sum_{i=1}^{K} \sum_{j=1}^{n} \binom{n}{j} p_i^j (1 - p_i)^{n-j} \ln j!

where $H(\mathbf p)$ is the entropy of one categorical draw. From $-\mathbb E[\ln P(\mathbf K)]$ with $\ln P = \ln n! - \sum_i \ln K_i! + \sum_i K_i \ln p_i$ : each count $K_i$ is Binomial(n, p_i), so the expectation of $\ln K_i!$ is a sum over its $n + 1$ values — the reference framework's formula.

Returns

Tensor

Entropy in nats, shape batch_shape.

Examples

>>> import lucid
>>> from lucid.distributions import Multinomial
>>> Multinomial(2, probs=lucid.tensor([0.5, 0.5])).entropy()  # ln 4 - ln 2/2
tensor(1.04)

log_prob

→Tensor

log_prob(value: Tensor)

source edit

log C(n; k₁,…,kK) + Σ kᵢ log pᵢ with $n = Σ kᵢ$ .

The coefficient counts the arrangements of the counts given, as the reference framework's does, so a vector summing to less than total_count is scored as a multinomial of its own total. A category with $k_i = 0$ contributes 0 even where $p_i = 0$ (a zero in probs or a logit of -inf), rather than 0 · (-inf) = NaN.

logits

→Tensor

logits()

source edit

Log-probabilities — the logits argument shifted by its logsumexp, or derived from probs on access.

Derived as log(probs) with probs clamped one epsilon inside [0, 1], as the reference framework derives it.

probs

→Tensor

probs()

source edit

Category probabilities, normalised along the last axis.

sample

→Tensor

sample(sample_shape: tuple[int, ...] = ())

source edit

Draw samples by summing one-hot Categorical draws.

>>> import lucid >>> from lucid.distributions import Multinomial >>> d = Multinomial(total_count=10, probs=lucid.tensor([0.2, 0.3, 0.5])) >>> d.mean # n * p tensor([2., 3., 5.]) >>> x = d.sample((4,)) >>> x.shape (4, 3) >>> bool((x.sum(dim=-1) == 10).all()) # every draw places all n trials True

dunder

init

→None

__init__(total_count: Tensor | int = 1, probs: Tensor | None = None, logits: Tensor | None = None, validate_args: bool | None = None)

source edit

Initialise a Multinomial distribution.

Parameters

total_countTensor | int= 1

Total number of draws

n \geq 0

. Must be a scalar. Default is 1 (reduces to a one-hot Categorical).

probsTensor | None= None

The category weights, as probabilities or as log-probabilities, of shape (..., K) — exactly one of the two. Either is normalised over the last axis before it is stored.

logitsTensor | None= None

The category weights, as probabilities or as log-probabilities, of shape (..., K) — exactly one of the two. Either is normalised over the last axis before it is stored.

validate_argsbool | None= None

Whether to check the parameters here and the counts later scored.

Raises

ValueError

If both or neither of probs and logits are given, or — under validation — a normalised parameter is not a probability vector or total_count is not a non-negative whole number.