class

SequentialLR

SequentialLR(optimizer: Optimizer, schedulers: list[_LRScheduler], milestones: list[int], last_epoch: int = -1)

source edit

Chain multiple schedulers and switch between them at epoch milestones.

SequentialLR runs its list of schedulers one at a time, activating the next scheduler whenever the epoch counter crosses a milestone. Before the first milestone, schedulers[0] is active; between milestone k-1 and k, schedulers[k] is active.

Parameters

optimizerOptimizer

The optimizer shared by all schedulers.

schedulerslist of _LRScheduler

Ordered list of scheduler instances to activate in sequence. Must have len(schedulers) == len(milestones) + 1.

milestoneslist of int

Sorted list of epoch indices at which to switch to the next scheduler.

last_epochint= -1

The index of the last epoch (default: -1).

Attributes

optimizerOptimizer

The shared optimizer.

schedulerslist of _LRScheduler

All schedulers in activation order.

milestoneslist of int

Epoch indices marking scheduler transitions.

last_epochint

Number of step calls completed.

Notes

A common pattern is to combine a short warmup with a long cosine decay:

warmup   = optim.LinearLR(optimizer, start_factor=0.1, total_iters=5)
cosine   = optim.CosineAnnealingLR(optimizer, T_max=95)
combined = optim.SequentialLR(
    optimizer,
    schedulers=[warmup, cosine],
    milestones=[5],
)

Examples

>>> import lucid.nn as nn
>>> model = nn.Linear(4, 2)
>>> import lucid.optim as optim
>>> optimizer = optim.SGD(model.parameters(), lr=0.1)
>>> s1 = optim.ConstantLR(optimizer, factor=0.1, total_iters=5)
>>> s2 = optim.ExponentialLR(optimizer, gamma=0.9)
>>> scheduler = optim.SequentialLR(optimizer, schedulers=[s1, s2], milestones=[5])
>>> for epoch in range(50):
...     optimizer.step()
...     scheduler.step()

Used by 1

lucid.optim

Constructors

dunder

init

→None

__init__(optimizer: Optimizer, schedulers: list[_LRScheduler], milestones: list[int], last_epoch: int = -1)

source edit

Initialise the SequentialLR. See the class docstring for parameter semantics.

Instance methods

get_last_lr

→list of float

get_last_lr()

source edit

Return the last learning rates from the currently active scheduler.

Returns

list of float

Current learning rate of each optimizer param group, as reported by the active scheduler.

load_state_dict

→None

load_state_dict(state_dict: dict[str, object])

source edit

Restore state produced by state_dict.

schedulers must have the same length and order as when it was saved — the children themselves are not reconstructed here.

state_dict

→dict[str, object]

state_dict()

source edit

Serialisable state, including each wrapped scheduler's own state.

The children are live scheduler objects, so they are replaced by their state_dict() under "schedulers"; load_state_dict feeds them back into the corresponding children.

step

→None

step()

source edit

Advance the active scheduler by one epoch, switching if a milestone is reached.

Increments the internal epoch counter, checks whether any milestone has been crossed, and delegates to the currently active scheduler's _LRScheduler.step.

Examples

>>> import lucid.nn as nn
>>> import lucid.optim as optim
>>> model = nn.Linear(4, 2)
>>> optimizer = optim.SGD(model.parameters(), lr=0.1)
>>> scheduler = optim.SequentialLR(
...     optimizer,
...     schedulers=[optim.ConstantLR(optimizer, factor=0.1),
...                 optim.ExponentialLR(optimizer, gamma=0.9)],
...     milestones=[5],
... )
>>> for epoch in range(100):
...     optimizer.step()
...     scheduler.step()

warmup = optim.LinearLR(optimizer, start_factor=0.1, total_iters=5) cosine = optim.CosineAnnealingLR(optimizer, T_max=95) combined = optim.SequentialLR( optimizer, schedulers=[warmup, cosine], milestones=[5], )

>>> import lucid.nn as nn >>> model = nn.Linear(4, 2) >>> import lucid.optim as optim >>> optimizer = optim.SGD(model.parameters(), lr=0.1) >>> s1 = optim.ConstantLR(optimizer, factor=0.1, total_iters=5) >>> s2 = optim.ExponentialLR(optimizer, gamma=0.9) >>> scheduler = optim.SequentialLR(optimizer, schedulers=[s1, s2], milestones=[5]) >>> for epoch in range(50): ... optimizer.step() ... scheduler.step()

>>> import lucid.nn as nn >>> import lucid.optim as optim >>> model = nn.Linear(4, 2) >>> optimizer = optim.SGD(model.parameters(), lr=0.1) >>> scheduler = optim.SequentialLR( ... optimizer, ... schedulers=[optim.ConstantLR(optimizer, factor=0.1), ... optim.ExponentialLR(optimizer, gamma=0.9)], ... milestones=[5], ... ) >>> for epoch in range(100): ... optimizer.step() ... scheduler.step()