Auto-dispatching wrapper that loads a language model.
Covers the masked, causal and sequence-to-sequence objectives, which were three tasks until they were recognised as one — each trains a language model, and what differs is which positions are visible when a token is predicted.
Notes
The three objectives keep their own output types
(MaskedLMOutput, CausalLMOutput,
Seq2SeqLMOutput), and the forward signatures are not
interchangeable — an encoder-decoder model requires
decoder_input_ids as a second positional argument where a causal
model takes only input_ids. This class resolves a name to a
class; it does not promise you can call two of them the same way.
See lucid.models._tasks.
Examples
>>> from lucid.models import AutoModelForLanguageModeling
>>> model = AutoModelForLanguageModeling.from_pretrained("gpt_lm")
>>> type(model).__name__
'GPTLMHeadModel'