Per-worker context published inside a DataLoader worker process.
Returned by get_worker_info when called from within a
worker's __getitem__ / __iter__ / worker_init_fn. Lets
user code shard work, seed RNGs differently per worker, or open
per-worker file handles.
Parameters
idint[0, num_workers).num_workersintDataLoader.seedintbase_seed + id, where
base_seed is drawn once per iterator from the loader's
generator (or the global generator). The loader seeds
Lucid's generator (lucid.manual_seed), Python random
and (when available) numpy with this value before invoking
worker_init_fn, so random augmentations differ between
workers and between epochs.datasetDatasetspawn is
used, this is a deep-copied instance — mutations in one worker
are not visible to others.Notes
Use get_worker_info rather than constructing this dataclass
directly; the per-thread storage is what actually wires it up.
Examples
>>> def worker_init_fn(worker_id):
... info = get_worker_info()
... # Shard an IterableDataset across workers:
... info.dataset.start = info.id * shard_sizeUsed by 2
Constructors
1Configure a DataLoader; see the class docstring for parameter
semantics.
Raises
ValueErrorbatch_size or prefetch_factor that is not a positive
integer (or None); a num_workers that is not an integer
>= 0; a negative or NaN timeout; prefetch_factor,
multiprocessing_context or persistent_workers given
with num_workers=0; or an unknown start-method name.TypeErrortimeout is not a number, collate_fn or
worker_init_fn is not callable, multiprocessing_context
is neither a name nor a context, or generator is neither a
lucid.Generator, a seed, nor None (a bool is
not a seed).Notes
sampler / batch_sampler / shuffle / batch_size /
drop_last interact: passing batch_sampler precludes the
other four; passing sampler precludes shuffle. When no
sampler is supplied, a SequentialSampler (shuffle=False)
or RandomSampler (shuffle=True) is constructed
automatically. persistent_workers requires num_workers > 0;
batch_size=None precludes drop_last.