Sample questions
Aipy Tensor Ops AutogradDifficulty 1
In a PyTorch-style tensor library, what is the 'computation graph' that autograd builds during the forward pass?
- aA record of the operations run, used afterward to compute gradients via the chain rule.✓
- bA cache of previously computed gradients so repeated backward calls skip recomputation entirely.
- cA static schedule that decides which device (CPU or GPU) each operation runs on.
- dA compressed copy of the model's weights created before training starts.
Explanation:Autograd records each differentiable operation as it runs (a directed graph of tensors and the functions that produced them), and backward() walks that graph applying the chain rule to compute gradients. (b) describes caching, not graph construction. (c) describes device placement/scheduling, unrelated to gradient tracking. (d) describes weight compression, which autograd does not perform.
Aipy Tensor Ops AutogradDifficulty 1
When you create a new tensor directly from data (e.g., torch.tensor([1.0, 2.0])) without specifying anything else, what is the default value of requires_grad?
- aTrue, since every floating-point tensor tracks gradients by default.
- bFalse, so the tensor does not participate in autograd unless requires_grad is set to True.✓
- cTrue only if the tensor has more than one element.
- dIt depends on whether a GPU is available at tensor creation time.
Explanation:Tensors created from plain data default to requires_grad=False; you must opt in explicitly (requires_grad=True at creation, or requires_grad_() afterward) to have that tensor tracked by autograd. (a) and (c) invent conditions that don't govern the default. (d) confuses device availability with the requires_grad default, which are unrelated.
Aipy Tensor Ops AutogradDifficulty 1
What is a 'leaf tensor' in an autograd computation graph?
- aAny tensor produced as the final output of the loss function.
- bAny tensor that has been moved to a GPU device.
- cA tensor created directly by the user, not as the result of a tracked operation, such as a model parameter or input.✓
- dAny tensor whose value never changes during training.
Explanation:Leaf tensors sit at the 'edges' of the graph: they were created directly by the user rather than produced by a tracked operation, and their .grad attribute is populated after backward() when requires_grad=True. Model parameters and directly created inputs are typical leaves. (a) describes the graph's root/output, not a leaf. (b) confuses device placement with graph position. (d) describes constants, not the leaf concept specifically.
Aipy Tensor Ops AutogradDifficulty 2
In a typical training loop, why must you call optimizer.zero_grad() (or param.grad = None) before each loss.backward() call?
- aBecause backward() overwrites .grad by default, so zero_grad() is only a defensive habit with no real effect.
- bBecause forgetting it causes backward() to raise an exception immediately.
- cBecause zero_grad() resets the model's weights to their initial values before the next step.
- dbackward() accumulates gradients into .grad by adding to the existing value each time it runs.✓
Explanation:Autograd accumulates: each backward() call adds newly computed gradients onto whatever is already in .grad, rather than replacing it. If you skip zero_grad(), gradients from earlier steps silently add into the current step's gradients, corrupting the update direction. (a) reverses the actual accumulation behavior. (b) is false — no exception is raised. (c) confuses gradient reset with weight reset, which zero_grad() does not do.
Aipy Tensor Ops AutogradDifficulty 2
A leaf tensor x has requires_grad=True and starts with x.grad = None. You run y = x**2 and call y.backward() twice in a row (without zero_grad in between, and with retain_graph=True passed to avoid the graph-freed error). If dy/dx at the current x gives grad value 6.0 each time it's computed, what is x.grad after the second backward() call?
- a6.0, since .grad always stores only the most recent gradient contribution.
- b12.0, since each backward() call adds its computed gradient (6.0) onto the existing .grad value.✓
- cNone, since calling backward() twice on the same graph clears .grad back to None.
- d0.0, since the two calls cancel each other out.
Explanation:Gradient accumulation means .grad starts at None, becomes 6.0 after the first backward(), and becomes 6.0 + 6.0 = 12.0 after the second, confirmed by how the accumulation loop behaves. (a) ignores accumulation entirely. (c) and (d) describe behaviors the accumulation mechanism does not exhibit.
Aipy Tensor Ops AutogradDifficulty 1
What does x.detach() return, given a tensor x that requires grad?
- aA new tensor sharing x's data but disconnected from the graph.✓
- bA deep copy of x with its own separate storage, still tracked by autograd.
- cThe gradient of x with respect to the loss, computed immediately.
- dA tensor moved to CPU, regardless of which device x was on.
Explanation:detach() gives you a view sharing x's storage, but it is cut off from the computation graph: it has requires_grad=False and no grad_fn, so operations on it won't be tracked and gradients won't flow back through it into x's graph. (b) wrongly claims a data copy and continued tracking. (c) confuses detaching with gradient computation. (d) confuses detach() with a device-transfer call like .cpu().