A value the package carries from one prediction to the next.
Core ML keeps it: the caller neither passes it in nor gets it back, and each prediction sees what the last one wrote. A decoder's key-value cache is the case this exists for.
Lucid's side has to be a pair — an input the model reads and an output
it returns — rather than a buffer it mutates. The tracer records a
pure graph, so an in-place buffer write does not appear in it at all;
a package built from that would agree on the first call and stop
accumulating on every one after, which export refuses.
The state begins at zero. Nothing carries the example's values into it, so a model that needs a different starting point has to be given one through an ordinary input.
Attributes
inputstrFeature name of the input the state replaces.
outputstrOutput field whose value is written back into it.
Examples
The cache stops being an input the caller passes; Core ML keeps it
and each prediction sees what the last one wrote:
>>> import shutil, tempfile
>>> import lucid, lucid.nn as nn, lucid.coreml as cml
>>> class Decoder(nn.Module):
... def forward(self, x, cache):
... cache = cache + x # what this step writes back
... return cache, cache * 2.0
>>> token, room = lucid.ones(1, 4) * 3.0, tempfile.mkdtemp()
>>> package = cml.export(
... Decoder().eval(), {"x": token, "cache": lucid.zeros(1, 4)},
... f"{room}/decoder.mlpackage", precision=cml.Precision.FLOAT16,
... state=[cml.State(input="cache", output="output_0")],
... )
>>> package.input_names
['x']
>>> package.predict(token).tolist(), package.predict(token).tolist()
([[6.0, 6.0, 6.0, 6.0]], [[12.0, 12.0, 12.0, 12.0]])
>>> package.reset_state() # start a fresh sequence
>>> package.predict(token).tolist()
[[6.0, 6.0, 6.0, 6.0]]
>>> package.close()
>>> shutil.rmtree(room)