ChainedScheduler
ChainedScheduler(schedulers: list[_LRScheduler], optimizer: Optimizer | None = None)Apply multiple schedulers simultaneously, compounding their LR changes.
At every step call each scheduler in the list is stepped in
order. Because each scheduler writes its own learning rates back into
the shared optimizer, the net effect is the composition of all
individual schedule factors:
where is the multiplicative change applied by scheduler at epoch .
Parameters
schedulerslist of _LRSchedulerAttributes
schedulerslist of _LRSchedulerNotes
ChainedScheduler differs from SequentialLR in that all
schedulers are active at every step rather than one at a time. This is
useful when you want, for example, a warmup schedule and an exponential
decay to both apply simultaneously.
get_last_lr returns the composed rate, which is also what is
stored in the optimizer.
Examples
>>> import lucid.nn as nn
>>> model = nn.Linear(4, 2)
>>> import lucid.optim as optim
>>> optimizer = optim.SGD(model.parameters(), lr=0.1)
>>> warmup = optim.LinearLR(optimizer, start_factor=0.1, total_iters=5)
>>> decay = optim.ExponentialLR(optimizer, gamma=0.95)
>>> scheduler = optim.ChainedScheduler([warmup, decay])
>>> for epoch in range(50):
... optimizer.step()
... scheduler.step()Used by 1
Constructors
1Instance methods
4Return the composed learning rates the chain last wrote.
Restore state produced by state_dict.
Serialisable state, including each wrapped scheduler's own state.
Step all chained schedulers simultaneously.
Calls _LRScheduler.step on every scheduler in order.
Each scheduler writes its updated learning rates into the shared
optimizer.
Examples
>>> import lucid.nn as nn
>>> import lucid.optim as optim
>>> model = nn.Linear(4, 2)
>>> optimizer = optim.SGD(model.parameters(), lr=0.1)