class

RASampler

extendsSampler

RASampler(dataset: Dataset, num_replicas: int = 1, rank: int = 0, shuffle: bool = True, seed: int = 0, num_repeats: int = 3)

source edit

Repeated-augmentation sampler (Hoffer et al., 2020 — arXiv:1901.09335).

Each dataset sample appears num_repeats times consecutively in the index stream, so that — when combined with a stochastic augmentation pipeline — the model sees several different augmented views of the same image inside each batch. Empirically lifts ImageNet accuracy at no extra labelled-data cost.

The sampler is also distributed-aware (single-process here, but sharding by rank is preserved for parity with the reference framework — see DistributedSampler). Per-epoch shuffling is deterministic given seed + the epoch counter (set_epoch).

Parameters

datasetDataset

Dataset whose __len__ defines the unique-index range.

num_replicasint= 1

Number of distributed replicas. 1 is the single-process case (the default Lucid context).

rankint= 0

Replica id in [0, num_replicas).

shufflebool= True

If True, the unique indices are shuffled before being repeated. Identical seeds reproduce the same epoch order.

seedint= 0

Base seed for the per-epoch RNG (combined with the epoch counter via set_epoch).

num_repeatsint= 3

How many consecutive copies of each index to emit. The original paper recommends 3 for ImageNet.

Notes

Two derived sizes (matching the reference-framework convention):

num_samples = ceil(len(dataset) * num_repeats / num_replicas) — number of indices this replica's slab contains before truncation. Used as total_size = num_samples * num_replicas for the full repeated-and-padded index list.
num_selected_samples = floor(len(dataset) / num_replicas) — number of indices this replica actually yields per epoch. Iteration truncates the slab to this length so the effective epoch size (per rank) matches an un-repeated pass — repetition diversifies within the epoch rather than extending it.

Examples

>>> import lucid
>>> from lucid.utils.data import RASampler, DataLoader, TensorDataset
>>> dataset = TensorDataset(lucid.randn(12, 3), lucid.randint(0, 2, (12,)))
>>> sampler = RASampler(dataset, num_repeats=3, shuffle=True, seed=0)
>>> loader = DataLoader(dataset, batch_size=4, sampler=sampler)
>>> for x, y in loader:                      # each batch may contain
...     pass                                 # ≤3 augmented views of
...     # ...train step...                   # the same underlying image
>>> len(sampler)                             # an un-repeated pass per epoch
12

Used by 1

lucid.utils.data

Constructors

dunder

init

→None

__init__(dataset: Dataset, num_replicas: int = 1, rank: int = 0, shuffle: bool = True, seed: int = 0, num_repeats: int = 3)

source edit

Store data_source for length introspection.

Parameters

data_sourceDataset

Dataset to be iterated sequentially.

Instance methods

set_epoch

→None

set_epoch(epoch: int)

source edit

Set the epoch — affects the shuffling RNG seed.

Dunder methods

dunder

iter

→Iterator[int]

__iter__()

source edit

Yield integer sample indices.

Returns

Iterator[int]

Iterator over 0 <= i < len(data_source) values, in an order defined by the concrete sampler.

dunder

len

→int

__len__()

source edit

Return per-replica yield count — floor(len(dataset) / num_replicas).

>>> import lucid >>> from lucid.utils.data import RASampler, DataLoader, TensorDataset >>> dataset = TensorDataset(lucid.randn(12, 3), lucid.randint(0, 2, (12,))) >>> sampler = RASampler(dataset, num_repeats=3, shuffle=True, seed=0) >>> loader = DataLoader(dataset, batch_size=4, sampler=sampler) >>> for x, y in loader: # each batch may contain ... pass # ≤3 augmented views of ... # ...train step... # the same underlying image >>> len(sampler) # an un-repeated pass per epoch 12