feature_alpha_dropout(x: Tensor, p: float = 0.5, training: bool = True, inplace: bool = False)Channel-wise alpha dropout for SELU convolutional networks.
Combines the channel-granularity of dropout2d /
dropout3d with the variance-preserving substitution of
alpha_dropout. Whole channels are dropped (replaced
rather than zeroed) and the result is affine-rescaled to keep
the self-normalising property — appropriate for SELU-activated
CNNs where spatially-correlated activations would make per-
element alpha dropout ineffective.
For inputs of rank less than 2, falls back to per-element
alpha_dropout.
Parameters
pfloat= 0.50.5).trainingbool= TrueFalse, identity (default True).inplacebool= FalseTrue, x is overwritten with the result and
returned (default False).Returns
TensorSame shape and dtype as x.
Notes
A Bernoulli mask of shape is drawn and
broadcast across the spatial axes. A dropped channel takes the
SELU saturation value in every position, and the
result goes through the same affine map as alpha_dropout:
a kept channel becomes , a dropped one the constant
. Variance preservation holds in expectation
when the input statistics already satisfy the SELU fixed-point
assumptions.
Examples
>>> import lucid
>>> from lucid.nn.functional import feature_alpha_dropout
>>> x = lucid.randn(2, 4, 3, 3)
>>> y = feature_alpha_dropout(x, p=0.25, training=True)