bert_large_qa

→BERTForQuestionAnswering

bert_large_qa(pretrained: bool | str = False, weights: BERTLargeQAWeights | None = None, overrides: object = {})

source edit

Construct a BERT-Large extractive-QA model (whole-word-masking, SQuAD v1.1).

Same trunk as bert_large (L=24, H=1024, A=16, ~335M parameters), augmented with a 2-way span head.

Model Size

Parameters

pretrainedbool or str= False

Weight selector. False → random init; True → the official Google whole-word-masking SQuAD v1.1 checkpoint (BERTLargeQAWeights.SQUAD_V1, from google-bert/bert-large-uncased-whole-word-masking-finetuned-squad) — an inference-ready extractive-QA model (EM ≈ 86.9 / F1 ≈ 93.2), the strongest BERT SQuAD result. Tokenize with the uncased BERTTokenizer.

weights(BERTLargeQAWeights, optional, keyword - only)= None

Explicit weights enum member; takes precedence over pretrained.

**overridesobject= {}

Optional BERTConfig field overrides forwarded into the underlying config. num_labels is ignored by this head. Overrides that change the encoder shape are incompatible with the pretrained checkpoint.

Returns

BERTForQuestionAnswering

BERT-Large wrapped with the span-prediction head (fully fine-tuned on SQuAD when pretrained is requested).

Notes

Reference: Devlin, Chang, Lee, and Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", NAACL 2019 (arXiv:1810.04805) §4.2. Whole-word masking is the pre-training tweak from the BERT repository's later release.

Examples

>>> import lucid
>>> from lucid.models.text.bert import bert_large_qa
>>> model = bert_large_qa().eval()
>>> input_ids = lucid.tensor([[101, 2040, 102, 1045, 2572, 102]])
>>> out = model(input_ids)
>>> out.start_logits.shape, out.end_logits.shape
((1, 6), (1, 6))

Used by 1

lucid.models.text.bert