Sample questions
Dl Training Dynamics BackpropDifficulty 2
A feedforward network computes h1 = f1(x), h2 = f2(h1), ... , h5 = f5(h4), and a scalar loss L is computed from h5. When backpropagation forms the gradient of L with respect to a weight inside the first layer f1, what is that gradient made of?
- aOnly the derivative of L with respect to h5, since the loss is the single source of the training signal.
- bOnly the derivative of h1 with respect to that weight; the later layers govern their own weights.
- cA chained product of the local derivative factors of every layer between the loss and f1, multiplied together with the local derivative of h1 with respect to that weight.✓
- dAn average of the per-layer derivatives, weighted by how many parameters each layer holds.
Explanation:The chain rule turns a deep composition into a product: the derivative of the loss with respect to an early weight is the derivative of the loss with respect to h5, times the derivative of each layer's output with respect to its input all the way back, times the local derivative of h1 with respect to the weight. This multiplicative structure is exactly why depth changes gradient magnitude so aggressively, in either direction.
Dl Training Dynamics BackpropDifficulty 2
The logistic sigmoid s(z) = 1/(1+e^-z) has derivative s(z)(1-s(z)). In a network that stacks many sigmoid layers, why does this derivative alone already push early-layer gradients toward zero as depth grows?
- aIts largest value anywhere is 0.25, reached at z = 0, so every layer multiplies the backward signal by at most a quarter.✓
- bThe derivative turns negative for negative pre-activations, so signs cancel across layers.
- cThe sigmoid output is bounded in the open unit interval, which makes the derivative exactly zero outside that range and cuts the chain at the first saturated layer.
- dThe derivative grows with |z|, so deep layers keep multiplying by ever larger factors.
Explanation:s(z)(1-s(z)) is maximized when s(z) = 0.5, giving 0.25 (confirmed by evaluating the derivative on a grid in PyTorch 2.8). Since backpropagation multiplies one such factor per sigmoid layer, ten stacked layers cap the signal at 0.25^10, which is under one in a million even before the weight matrices are taken into account.
Dl Training Dynamics BackpropDifficulty 2
The backward step of a linear layer y = xW^T is written out by hand (PyTorch 2.8, CPU):
W = torch.tensor([[1., 2., 3.], [0., 1., -1.]])
x = torch.tensor([[1., 0., 2.], [3., 1., 0.]])
delta = torch.ones(2, 2) # dL/dy for L = sum of all outputs
gW = delta.T @ x
What is gW?
- a[[7.5, -2.5], [5.5, 0.5]] — the layer outputs, which the loss adds up.
- b[[1., 0., 2.], [3., 1., 0.]] — the input batch itself, since the loss is linear.
- c[[1., 2., 3.], [0., 1., -1.]] — the weight matrix, because the derivative of a linear map is that map.
- d[[4., 1., 2.], [4., 1., 2.]] — each row is the column-wise sum of the input batch.✓
Explanation:For a linear layer the weight gradient is the outer product of the backpropagated signal with the layer's input, summed over the batch. With delta all ones, delta.T @ x collapses to the column sums of x repeated for each output unit, giving [[4,1,2],[4,1,2]]; the same values come back from an autograd run on the identical setup. The layer's own outputs never enter the expression.
Dl Training Dynamics BackpropDifficulty 3
An engineer trains a 9-layer sigmoid MLP whose hidden layers all have the same shape, and prints the gradient norm of each weight matrix after one backward pass (PyTorch 2.8, CPU, seed 0): layer 1 ~ 2.6e-7, layer 2 ~ 4.8e-6, layer 3 ~ 3.1e-5, ... , layer 8 ~ 5.8e-1, layer 9 ~ 4.0. What does this profile say?
- aThe backward signal shrinks by a roughly constant factor per layer, so the early layers receive almost no training signal.✓
- bThe learning rate is too small, which is why every layer's gradient is small.
- cThe last layers are exploding and will be the first to diverge.
- dGradient norms from different layers cannot be compared, so the profile carries no information.
Explanation:The norms fall by nearly an order of magnitude per layer going backwards, which is the signature of a multiplicative per-layer factor well below one — here the sigmoid derivative combined with the weight matrices. The deepest layer still gets a usable gradient, so this is a depth-dependent attenuation of the backward signal, not a globally small step size. Layers of identical shape make the norms directly comparable.
Dl Training Dynamics BackpropDifficulty 2
A hidden unit uses tanh, whose derivative is 1 - tanh(z)^2. Logging shows its pre-activation z sits near 3 for almost every sample; measured in PyTorch 2.8, the derivative there is about 0.0099 while at z = 0 it is 1.0. What does this mean for the weights feeding that unit?
- aThey receive a full-strength gradient, because tanh is differentiable at every point and its derivative never reaches zero.
- bThey receive under 1% of the signal that reaches the unit, so they move extremely slowly.✓
- cThey receive a gradient with the opposite sign, because tanh bends downward in that region.
- dThey receive exactly zero gradient, because the tanh derivative vanishes once |z| passes 2 and the unit dies outright.
Explanation:The gradient arriving at the unit is multiplied by the local tanh derivative before it reaches the incoming weights, so an operating point at z = 3 attenuates it by about a hundredfold. The derivative is still strictly positive, so learning is slow rather than dead, and the sign of the signal is preserved.
Dl Training Dynamics BackpropDifficulty 3
A training run prints finite losses, then inf at one step, then NaN at every step afterwards — including long after the offending batch has passed. Which mechanism explains why it never recovers?
- aThe training loop caches the last printed loss and keeps reprinting it.
- bNaN is only a printing artifact; the parameters keep improving underneath.
- cOne update with an infinite gradient pushed parameters to +/-inf, and every later computation on an infinite parameter yields inf or NaN.✓
- dThe optimizer stops stepping once it sees a NaN, so the model is frozen at its last healthy parameters.
Explanation:Verified in PyTorch 2.8: after a single SGD step with an infinite gradient a parameter becomes -inf, and subsequent finite gradients leave it at -inf because inf minus a finite number is still inf. Forward passes through an infinite weight then produce inf and inf-minus-inf products, which are NaN, so the loss stays NaN forever regardless of the data.