hessian

→Tensor or tuple of tuple of Tensor

hessian(func: Callable[..., Tensor], inputs: Tensor | tuple[Tensor, ...], create_graph: bool = False, strict: bool = False, vectorize: bool = False)

source edit

Compute the Hessian matrix of a scalar-valued func.

The Hessian of a scalar function $f : \mathbb{R}^n \to \mathbb{R}$ is

H_{ij} = \frac{\partial^2 f}{\partial x_i \, \partial x_j}, \qquad H \in \mathbb{R}^{n \times n}.

Implemented as the jacobian of the gradient of func — one forward pass, one backward pass that records its own graph, and one backward pass through that graph per gradient coordinate. Cost is therefore $O(n \cdot \text{cost}(\nabla f))$ .

Parameters

funccallable

Function of one or more Tensor inputs returning a tensor of one element.

inputsTensor or tuple of Tensor

Inputs at which

H

is evaluated. They are left as they are: the Hessian is taken at copies that require grad, so the inputs' requires_grad and .grad do not change.

create_graphbool= False

If True the Hessian itself is differentiable with respect to the inputs that require grad (third-order derivatives). Defaults to False.

strictbool= False

If True, refuse a gradient of func that does not depend on the inputs (a linear func, say) and a Hessian block of an input the gradient does not depend on, instead of filling them with zeros. Defaults to False.

vectorizebool= False

Reserved for a future vmap-based implementation. Currently unused.

Returns

Tensor or tuple of tuple of Tensor

For a single input the returned tensor has shape x.shape + x.shape. For a tuple of inputs a nested tuple of cross-Hessian blocks is returned, with H[i][j] of shape x_i.shape + x_j.shape containing $\partial^2 f / (\partial x_i \, \partial x_j)$ .

Raises

RuntimeError

func does not return a single tensor of one element.

Notes

Symmetry $H_{ij} = H_{ji}$ holds in exact arithmetic when $f$ is $C^2$ . In floating-point the result is only approximately symmetric; symmetrize as $\tfrac{1}{2}(H + H^\top)$ if a strictly symmetric matrix is required.

Examples

>>> import lucid
>>> from lucid.autograd import hessian
>>> x = lucid.tensor([1.0, 2.0])
>>> def f(x):
...     return (x ** 3).sum()
>>> hessian(f, x).tolist()
[[6.0, 0.0], [0.0, 12.0]]

Used by 2