spectral_norm(module: Module, name: str = 'weight', n_power_iterations: int = 1, eps: float = 1e-12, dim: int | None = None)Constrain a weight's spectral norm via power iteration (Miyato et al. 2018).
Reparametrises module.<name> as
where is the largest singular value
(operator 2-norm) of the weight viewed as a matrix. This bounds the
layer's Lipschitz constant by 1 and is the canonical
stabilisation trick for GAN discriminators; it also sees use in
certified-robust classifiers and contractive Transformer variants.
Parameters
moduleModulenamestr= 'weight'"weight".n_power_iterationsint= 1u / v vectors
performed before each training-mode forward. Defaults to 1
— sufficient because the state is preserved between calls and
converges over the course of training.epsfloat= 1e-12u and v. Default 1e-12.dimint= NoneNone (the default), uses 0 for everything except
ConvTranspose* modules, which use 1 (so output channels
end up as rows).Returns
ModuleThe same module, with <name>_orig (the trainable underlying
matrix), <name>_u / <name>_v (buffers), and a forward
pre-hook performing the normalisation.
Notes
Each forward in training mode runs n_power_iterations updates
estimates ,
and exposes as the active
weight. In eval mode no updates occur — the cached buffers are used
as-is for deterministic inference. Invert with
remove_spectral_norm.
Examples
>>> import lucid.nn as nn
>>> from lucid.nn.utils import spectral_norm
>>> disc = spectral_norm(nn.Conv2d(3, 64, 3, padding=1))