CoAtNet-5 backbone (Dai et al., 2021).
Widest paper-cited variant: blocks_per_stage=(2, 12, 28, 2),
dims=(256, 512, 1280, 2048), stem_width=192,
attn_heads=(20, 32). Approximately 688M parameters, 85.8%
ImageNet-1k top-1 at 224×224 (Table 5). With JFT-3B pretraining
the paper reports 89.0% at 512x512.
Model Size
Parameters
pretrainedbool= FalseNo checkpoint is published for
coatnet_5, so True raises
NotImplementedError rather than return randomly initialised weights.**overridesobject= {}Keyword overrides applied on top of the canonical CoAtNet-5
config; each must name a field of
CoAtNetConfig (e.g.
image_size=384, in_channels=1).Notes
Memory footprint at default image_size=224 is non-trivial — a
forward pass of a single 3×224×224 input materialises stage-3/4
activations on the order of B × D × H/32 × W/32 channels with
D=2048. Instantiating + running this variant on a 16 GB host is
near-the-edge; prefer measuring on a larger GPU.
Examples
>>> import lucid
>>> from lucid.models.vision.coatnet import coatnet_5
>>> model = coatnet_5()
>>> out = model(lucid.randn(1, 3, 224, 224))
>>> out.last_hidden_state.shape
(1, 2048, 7, 7)