ZFNet

ZFNet is the ILSVRC-2013 winner and is best understood as an AlexNet that was tuned by looking at what the network had learned. Zeiler and Fergus introduced the deconvolutional network visualisation technique — projecting individual feature- map activations back to the input pixel space — and used the resulting diagnostic images to reshape the early layers of the network.

Two specific changes distinguish ZFNet from AlexNet. The first convolution was reduced from an $11\times11$ stride-4 kernel to a $7\times7$ stride-2 kernel, addressing the observation that AlexNet's first layer learned a mixture of extremely high-frequency and dead filters. §4.1 attributes both changes to layer 1 ("we (i) reduced the 1st layer filter size from 11x11 to 7x7 and (ii) made the stride of the convolution 2, rather than 4"). §5.1 then describes the model as having "stride 2 convolutions in layers 1 & 2" — conv2's stride was raised from AlexNet's 1 to 2, which is also what Figure 3's 55 -> 26 transition requires arithmetically with a 5x5 kernel. The remaining layers retain AlexNet's topology.

These modest topological changes — together with the methodology of using visualisation to drive architectural choices — were influential out of proportion to their parameter cost. ZFNet reports 16.5% top-5 validation error for the Figure 3 model (Table 2), and the deconvolutional-network analysis it popularised is the conceptual ancestor of every later interpretability tool from Grad-CAM to feature inversion.