normal

→Tensor

normal_(tensor: Tensor, mean: float = 0.0, std: float = 1.0, generator: _C_engine.Generator | None = None)

source edit

Implementing kernel

C++normal_opfree fn

Initialise tensor in-place with samples from a Gaussian distribution.

Each entry is drawn independently from $\mathcal{N}(\text{mean}, \text{std}^2)$ . This is the simplest initialisation scheme and serves as the building block for xavier_normal_ / kaiming_normal_, which choose std so that the activation variance is preserved layer-by-layer.

Parameters

tensorTensor

Tensor to fill in place; any shape is accepted.

meanfloat= 0.0

Mean of the Gaussian distribution. Default 0.0.

stdfloat= 1.0

Standard deviation of the Gaussian distribution. Must be non-negative. Default 1.0.

generatorGenerator= None

PRNG for Gaussian draws.

Returns

Tensor

tensor after Gaussian fill.

Notes

The samples have

\mathbb{E}[W] = \text{mean}, \qquad \mathrm{Var}(W) = \text{std}^2.

For unbounded tails consider trunc_normal_ instead, which truncates extreme draws and is often preferred in transformer pre-training recipes.

Examples

>>> import lucid
>>> from lucid.nn.init import normal_
>>> w = lucid.empty(64, 32)
>>> lucid.manual_seed(0)
>>> _ = normal_(w, mean=0.0, std=0.02)
>>> abs(w.std().item() - 0.02) < 0.002       # sample std close to the target
True

>>> import lucid >>> from lucid.nn.init import normal_ >>> w = lucid.empty(64, 32) >>> lucid.manual_seed(0) >>> _ = normal_(w, mean=0.0, std=0.02) >>> abs(w.std().item() - 0.02) < 0.002 # sample std close to the target True