ctc_loss

→Tensor

ctc_loss(log_probs: Tensor, targets: Tensor, input_lengths: Tensor | Sequence[int], target_lengths: Tensor | Sequence[int], blank: int = 0, reduction: Reduction = 'mean', zero_infinity: bool = False)

source edit

Implementing kernel

C++ctc_loss_opfree fn

Connectionist Temporal Classification (CTC) loss.

The standard training objective for unaligned sequence prediction — used in speech recognition, handwriting recognition, and any task where the input sequence is much longer than the target and no per-frame alignment is provided. Introduced by Graves et al. 2006.

Internally marginalises over every valid alignment of a $T$ -frame prediction onto an $S$ -symbol target by inserting "blank" symbols and allowing each target symbol to span one or more frames, computing the negative log of the total path probability via dynamic programming.

Parameters

log_probsTensor

Log-probabilities of shape

(T, N, C)

where

T

is the input sequence length,

N

is the batch size, and

C

is the number of classes (including the blank). Typically produced by lucid.nn.functional.log_softmax over the class axis.

(T, C)

is one unbatched sequence, with a

(S,)

target, 0-d lengths, and a 0-d loss under "none".

targetsTensor

Target indices, shape

(N, S)

(padded) or

(\sum_i \text{target\_lengths}_i,)

(concatenated). int32. Every label read must lie in

[0, C)

, else IndexError. This is stricter than the reference, which reads past the class axis. The padding past a row's target length is not read and may hold anything.

input_lengthsTensor

Effective input lengths

(N,)

, int32. Enables padding-aware batching. Each must lie in

[1, T]

, else IndexError. A length of 0 raises NotImplementedError for now.

target_lengthsTensor

Effective target lengths

(N,)

, int32. Each must lie in

[0, S]

for padded targets, and they must add up to the number of concatenated targets. A length outside its range raises IndexError; a count that does not match raises ValueError.

blankint= 0

Index of the blank symbol (default 0), in

[0, C)

, else IndexError.

reductionstr= 'mean'

"mean" (default), "sum", or "none". Under "mean", each sample's loss is divided by its target length (at least 1) and the results are averaged across the batch, as the reference framework defines it.

zero_infinitybool= False

When True, infinite losses (which arise when a target cannot fit in the available input frames) and their gradients are set to zero, effectively skipping those samples (default False).

Returns

Tensor

Scalar ("mean" / "sum") or per-sample tensor of shape $(N,)$ .

Notes

The CTC objective is the negative log of the total alignment probability:

L = -\log \sum_{\pi \in \mathcal{B}^{-1}(\mathbf{y})} \prod_{t=1}^{T} p_t(\pi_t),

where $\mathcal{B}$ is the "many-to-one" alignment map that collapses repeats and removes blanks. The forward and backward recursions run in the log domain on the CPU; on metal the inputs make a round trip. The gradient with respect to log_probs follows the reference framework's formula, which is the true gradient with respect to the logits once a lucid.nn.functional.log_softmax sits in front — the way the loss is meant to be fed. It is differentiable once.

Examples

>>> import lucid
>>> from lucid.nn.functional import ctc_loss, log_softmax
>>> # T=4 frames, N=1 batch, C=3 classes (blank=0)
>>> logits = lucid.randn(4, 1, 3)
>>> log_p = log_softmax(logits, dim=2)
>>> targets = lucid.tensor([[1, 2]], dtype=lucid.int32)
>>> il = lucid.tensor([4], dtype=lucid.int32)
>>> tl = lucid.tensor([2], dtype=lucid.int32)
>>> ctc_loss(log_p, targets, il, tl)
Tensor(...)

Used by 2

>>> import lucid >>> from lucid.nn.functional import ctc_loss, log_softmax >>> # T=4 frames, N=1 batch, C=3 classes (blank=0) >>> logits = lucid.randn(4, 1, 3) >>> log_p = log_softmax(logits, dim=2) >>> targets = lucid.tensor([[1, 2]], dtype=lucid.int32) >>> il = lucid.tensor([4], dtype=lucid.int32) >>> tl = lucid.tensor([2], dtype=lucid.int32) >>> ctc_loss(log_p, targets, il, tl) Tensor(...)