yoklainterview sim
ProgressHistory

ML Engineer Dl Normalization Regularization Interview Questions

75 verified ML Engineer Dl Normalization Regularization interview questions — solve with answers, learn from explanations, test yourself in a real simulation.

Try the real simulation →

Sample questions

Dl Normalization RegularizationDifficulty 1
A BatchNorm layer is applied to an activation tensor of shape (N, C) during training, where N is the batch dimension and C is the feature dimension. Which statistics does the layer compute in order to standardize this tensor?
  • aOne global mean and one global variance over all N x C entries, so the whole tensor is standardized by a single pair of numbers
  • bOne mean and one variance per feature, each estimated across the N samples in the batch✓
  • cOne mean and one variance per sample, each estimated across the C features of that row
  • dA full C x C covariance matrix, so correlated features are whitened together
Explanation:BatchNorm keeps one statistic pair per feature channel and estimates it by reducing over the batch dimension. The option that reduces over features within a row describes LayerNorm instead, and the single global pair would erase per-feature scale differences. Whitening with a covariance matrix is a different, far more expensive operation that BatchNorm does not perform.
Dl Normalization RegularizationDifficulty 2
In PyTorch 2.8:

bn = nn.BatchNorm2d(3)   # default affine=True
bn.train()
x = torch.randn(4, 3, 5, 5) * 2 + 1
y = bn(x)
print(y.mean(dim=(0, 2, 3)), y.std(dim=(0, 2, 3), unbiased=False))


What does this print, to floating-point tolerance?
  • aMeans near 1 and standard deviations near 2, because the layer keeps the incoming location and scale and only learns a correction
  • bMeans near 0 and standard deviations near 2, because only centering is applied on the first forward pass
  • cMeans near 1 and standard deviations near 1, because the learnable shift is initialised to the batch mean
  • dMeans near 0 and standard deviations near 1, because gamma and beta start at 1 and 0✓
Explanation:In training mode the layer subtracts the batch mean and divides by the batch standard deviation of each channel, then applies the affine parameters, which are initialised to gamma=1 and beta=0. With those defaults the affine step is the identity, so the printed statistics are the standardized ones. The incoming location of 1 and scale of 2 are removed, not preserved.
Dl Normalization RegularizationDifficulty 2
A convolutional network with BatchNorm layers is switched to evaluation mode before scoring a single image. Which quantities does each BatchNorm layer use to standardize its input in this call?
  • aThe stored running mean and running variance buffers, which were accumulated across training batches✓
  • bThe mean and variance of the current input, exactly as during training, since the layer has no mode-dependent branch
  • cThe mean and variance of the last training batch that was seen, which the layer caches verbatim
  • dThe learned gamma and beta only, since standardization is skipped once training ends
Explanation:In evaluation mode the layer stops estimating statistics from the incoming tensor and reads the buffers it accumulated while training, which is what makes single-image inference well defined. Standardization itself is not skipped; the affine parameters are applied on top of it. The buffers hold a moving estimate over many batches rather than a verbatim copy of the last one.
Dl Normalization RegularizationDifficulty 2
A BatchNorm layer is freshly constructed, so its running mean buffer holds 0. Its momentum hyperparameter is 0.1. The very first training forward pass sees a batch whose mean for one channel is 5.0. What does that channel's running mean hold afterwards, in PyTorch 2.8?
  • a5.0, because the first observation has nothing to blend with and replaces the initial value
  • b4.5, because the buffer keeps a weight of 0.1 on its old value and 0.9 on the new one
  • c0.5, because the new batch statistic enters with weight 0.1 and the old buffer keeps weight 0.9✓
  • d0.05, because the momentum is applied twice, once to the batch statistic and once to the update
Explanation:The buffer is an exponential moving average, and on this first pass the update evaluates 0.9 0 + 0.1 5.0. Overwriting the stored value with the batch mean would give 5.0, and exchanging the two coefficients would give 4.5. The coefficient enters the rule once rather than twice, so nothing shrinks the result a second time.
Dl Normalization RegularizationDifficulty 2
A service scores the same request twice through a trained network that contains Dropout layers, and gets two different prediction vectors. The weights were not updated between the two calls. What is the most likely cause?
  • aThe dropout probability is annealed automatically as more requests arrive, so later calls keep more units
  • bThe module is still in training mode, so each forward pass samples a fresh dropout mask✓
  • cDropout rounds its output to a random precision, so repeated calls differ in the low-order bits
  • dDropout retains the mask of the previous request and inverts it on the next one
Explanation:Dropout is stochastic only while the module is in training mode; each forward pass then draws an independent Bernoulli mask, so identical inputs give different outputs. Once evaluation mode is set the layer becomes the identity and repeated calls agree exactly. Nothing about the probability changes on its own, and no mask is carried over between calls.
Dl Normalization RegularizationDifficulty 1
In PyTorch 2.8:

d = nn.Dropout(p=0.5)
d.train()
y = d(torch.ones(10000))
print(torch.unique(y))


Which distinct values appear in y?
  • aOnly 0. and 2., because surviving entries are divided by the keep probability✓
  • bOnly 0. and 0.5, because surviving entries are multiplied by the keep probability
  • cOnly 0. and 1., because the mask zeroes entries and leaves the survivors untouched
  • dA continuum between 0. and 1., because each entry is multiplied by its own Bernoulli sample
Explanation:The implementation is inverted dropout: dropped entries become zero and surviving entries are divided by 1 - p, which is 0.5 here, giving 2. The mask itself is binary, so no intermediate values are produced. Multiplying the survivors by the keep probability would print 0.5 instead of 2., and leaving them untouched would print 1.

By specialization

Classical ML 450Deep Learning 450

By seniority

ML Engineer JuniorML Engineer MidML Engineer Senior

Other fields

BackendFrontendFullstackDevOps / CloudAI EngineerDatabaseMobileSecurityQA / Test AutomationData EngineerGame Development

Test yourself against the 1500-question ML Engineer bank.

Start interview