Compute with the convention .
Frequently used to compute cross-entropy losses and log-densities where the limit must be honoured to avoid NaN contamination from probability values that are exactly zero.
Parameters
Returns
TensorElement-wise with the x == 0 convention
applied.
Notes
Mathematical definition:
The convention covers only x == 0: with x != 0 the plain
definition applies, so xlogy(2, 0) is -inf and a negative
other gives NaN.
Gradients are and
, except where x == 0 and
is not finite (y <= 0 or other infinite): there
both are 0, so the backward never forms 0 \cdot \infty. At
x == 0 with 0 < y < \infty the input gradient stays
. Two cells differ from the reference framework on
purpose: at (0, 0) its other gradient is 0 / 0 = NaN and
Lucid's is 0, and at (0, +\infty) its input gradient is
and Lucid's is 0 (a composite cannot give a
value of 0 a gradient of without forming
0 \cdot \infty in the forward pass).
Examples
>>> import lucid
>>> x = lucid.tensor([0.0, 1.0, 2.0])
>>> y = lucid.tensor([0.0, 2.0, 3.0])
>>> lucid.xlogy(x, y)
tensor([0., 0.6931, 2.197])
>>> lucid.xlogy(2.0, 0.0)
tensor(-inf)