hvp

→tuple of (Tensor, Tensor or tuple of Tensor)

hvp(func: Callable[..., Tensor], inputs: Tensor | tuple[Tensor, ...], v: Tensor | tuple[Tensor, ...] | None = None, create_graph: bool = False, strict: bool = False)

source edit

Hessian-vector product $H v$ of a scalar-valued func.

For $f : \mathbb{R}^n \to \mathbb{R}$ with Hessian $H$ and a vector $v \in \mathbb{R}^n$ , returns

H v = \left.\frac{d}{dt} \nabla_x f(x + t v)\right|_{t=0}

along with $f(x)$ , without forming $H$ — the directional derivative of the gradient, taken exactly as jvp takes one.

Parameters

funccallable

Function of the inputs returning one single-element tensor, as for vhp.

inputsTensor or tuple of Tensor

Where

H

is taken; the caller's tensors are not modified.

vTensor or tuple of Tensor= None

The direction, one tensor per input, packed like inputs. Optional only for one single-element input (seeded with 1).

create_graphbool= False

Keep the output and the product differentiable. Defaults to False.

strictbool= False

Refuse what vhp refuses under strict. Defaults to False.

Returns

tuple of (Tensor, Tensor or tuple of Tensor)

(output, hvp), the product packed like inputs and both detached unless create_graph — as vhp returns them.

Raises

(RuntimeError, ShapeMismatch, DeviceMismatch)

For the reasons vhp gives.

Notes

Three reverse passes against vhp's two; prefer vhp when func is twice continuously differentiable, where the Hessian is symmetric and the two agree up to rounding.

Examples

>>> import lucid
>>> from lucid.autograd import hvp
>>> x = lucid.tensor([1.0, 2.0])
>>> def f(x):
...     return (x ** 3).sum()
>>> out, product = hvp(f, x, lucid.tensor([1.0, 0.0]))
>>> product
tensor([6., 0.])

Used by 1

lucid.autograd

>>> import lucid >>> from lucid.autograd import hvp >>> x = lucid.tensor([1.0, 2.0]) >>> def f(x): ... return (x ** 3).sum() >>> out, product = hvp(f, x, lucid.tensor([1.0, 0.0])) >>> product tensor([6., 0.])