Pretrained weight tags for lucid.models.gpt2_xlarge_lm.
GPT-2 XLarge trunk + tied lm_head over the 50 257-token BPE vocab.
Attributes
WEBTEXTWeightEntryWebText pre-training checkpoint (trunk + LM head) sourced from
transformers/gpt2-xl.DEFAULTWeightEntryAlias for
WEBTEXT.Notes
Reference: Radford, Wu, Child, Luan, Amodei, Sutskever, "Language Models are Unsupervised Multitask Learners", OpenAI 2019.
Examples
>>> from lucid.models import gpt2_xlarge_lm
>>> model = gpt2_xlarge_lm(pretrained=True).eval()