grid_sample

→Tensor

grid_sample(x: Tensor, grid: Tensor, mode: str = 'bilinear', padding_mode: str = 'zeros', align_corners: bool | None = None)

source edit

Implementing kernel

Sample an input feature map at flow-field coordinates.

For each output location, looks up a value in x at the (sub-pixel) location specified by grid, using bilinear, nearest or bicubic interpolation. Forms the second half of a spatial transformer network (STN) when paired with affine_grid.

The grid uses normalised coordinates in $[-1, 1]$ : $(-1, -1)$ denotes the top-left corner of x and $(+1, +1)$ the bottom-right corner (with the align_corners=False convention placing corners just outside the edge pixels).

Parameters

xTensor

Source feature map of shape (N, C, H_in, W_in) (4-D) or (N, C, D_in, H_in, W_in) (5-D for volumetric sampling).

gridTensor

Sampling grid of shape (N, H_out, W_out, 2) for 2-D input or (N, D_out, H_out, W_out, 3) for 3-D input. Last axis stores (x, y) (or (x, y, z)) in

[-1, 1]

modestr= 'bilinear'

Interpolation mode: "bilinear" (default; trilinear for 5-D input), "nearest", or "bicubic" (4-D input only).

padding_modestr= 'zeros'

Behaviour for grid locations outside

[-1, 1]

: "zeros" (default), "border" (clamp to edge), or "reflection" (mirror at the image's edges, then clamp).

align_cornersbool= None

Controls whether ±1 denotes the centre of the corner pixels (True) or the outer edge of the image (False, default).

Returns

Tensor

Sampled output of shape (N, C, H_out, W_out) (or 5-D analogue), with the same dtype as x.

Raises

ValueError

Unknown mode / padding_mode, an input that is not 4-D or 5-D, "bicubic" with a 5-D input, or a grid whose shape does not fit the input.

Notes

grid_sample is differentiable w.r.t. both x and grid. The gradient w.r.t. grid is what makes STN-style learned warps trainable end-to-end (Jaderberg et al., 2015). Under "nearest" the grid's gradient is zero: the sample does not move with the grid.

4-D bilinear and nearest sampling run on one engine kernel; reflection padding mirrors the grid in pixel space first and then samples with border padding. Bicubic and 5-D sampling are composed from gather and elementwise ops, so they cost more memory per output point than the kernel. Coordinates far outside the image, and NaN or infinite ones, land where the reference framework's do; the few 5-D cases where its forward and backward disagree with each other are listed in lucid/nn/functional/_grid_sampling.py.

Examples

>>> import lucid
>>> from lucid.nn.functional import affine_grid, grid_sample
>>> x = lucid.randn(1, 3, 32, 32)
>>> theta = lucid.eye(2, 3).unsqueeze(0)        # identity
>>> g = affine_grid(theta, (1, 3, 32, 32), align_corners=False)
>>> y = grid_sample(x, g, align_corners=False)
>>> y.shape
(1, 3, 32, 32)

Used by 1

lucid.nn.functional

>>> import lucid >>> from lucid.nn.functional import affine_grid, grid_sample >>> x = lucid.randn(1, 3, 32, 32) >>> theta = lucid.eye(2, 3).unsqueeze(0) # identity >>> g = affine_grid(theta, (1, 3, 32, 32), align_corners=False) >>> y = grid_sample(x, g, align_corners=False) >>> y.shape (1, 3, 32, 32)