yoklainterview sim
ProgressHistory

ML Engineer Ti Throughput Profiling Bottlenecks Interview Questions

75 verified ML Engineer Ti Throughput Profiling Bottlenecks interview questions — solve with answers, learn from explanations, test yourself in a real simulation.

Try the real simulation →

Sample questions

Ti Throughput Profiling BottlenecksDifficulty 2
A profiler breaks one training step (average of many steps) into phases:
input_wait   : 2 ms
forward      : 18 ms
backward     : 34 ms
collective    : 6 ms
optimizer    : 4 ms
-------------------
step total   : 64 ms

Which phase is the best target for the first optimization attempt?
  • ainput_wait, since any nonzero wait means the pipeline is starving the GPU
  • bcollective, because inter-rank communication tends to dominate hidden costs in a synchronous step
  • cbackward, since it is by far the largest phase✓
  • doptimizer, because its 4 ms is pure overhead, and overhead should be eliminated before real compute
Explanation:backward (34 ms) is more than half the step and larger than every other phase combined; a fixed percentage improvement there yields the largest absolute time saved. input_wait is only 2 ms (not a bottleneck), collective is a modest 6 ms here, and optimizer's 4 ms is normal, not 'pure overhead'.
Ti Throughput Profiling BottlenecksDifficulty 1
In a per-step time breakdown (input wait, forward, backward, collective, optimizer, host overhead), which phase represents time the process spends stalled waiting for the next batch to become available?
  • ainput wait✓
  • bforward
  • ccollective
  • dhost overhead
Explanation:input wait is defined as the time the training loop is blocked waiting for the data pipeline to hand it the next batch. Forward and collective are active compute/communication phases, and host overhead refers to CPU-side dispatch cost, not waiting on data.
Ti Throughput Profiling BottlenecksDifficulty 3
Two profiled steps from the same job, both with global batch size and model unchanged:
Step A: input_wait=1ms, forward=20ms, backward=38ms, collective=5ms, total=64ms.
Step B (taken 500 steps later): input_wait=27ms, forward=20ms, backward=38ms, collective=5ms, total=90ms.
Forward, backward, and collective are unchanged between the two steps. What most plausibly explains step B being 26ms slower?
  • athe model became larger between the two steps
  • bthe collective became slower because more ranks joined
  • cthe optimizer switched to a more expensive update rule
  • dthe data pipeline fell behind✓
Explanation:Every compute/communication phase (forward, backward, collective) is identical between A and B; only input_wait grew, by almost exactly the 26ms gap. That isolates the slowdown to the data pipeline falling behind, not to compute, communication, or the model changing (which forward/backward would reflect).
Ti Throughput Profiling BottlenecksDifficulty 2
times = []
for i in range(10):
    t0 = time.perf_counter()
    loss = train_step(batch)
    t1 = time.perf_counter()
    times.append(t1 - t0)
print(sum(times) / len(times))

On a GPU where train_step launches asynchronous kernels and returns before they finish, what is the main problem with this timing code?
  • at1 is captured right after launch, before the kernels finish, so the measured duration understates GPU work✓
  • bperf_counter() is not precise enough to time single steps
  • c10 iterations is not enough samples to compute any average
  • dthe loop should switch from time.perf_counter() to time.time(), which is the timer built specifically for GPU-launched code
Explanation:GPU kernel launches are asynchronous: the Python call returns as soon as the kernel is queued, not when it finishes. Without an explicit synchronization point before t1, the wall-clock delta mostly measures dispatch overhead, not actual compute time. The other options are not the actual defect here.
Ti Throughput Profiling BottlenecksDifficulty 2
A team times the first 5 training steps of a fresh process to estimate steady-state throughput: [420ms, 38ms, 36ms, 37ms, 36ms]. Using the mean of these 5 numbers as 'the' step time, what mistake are they making?
  • athe first step includes one-time warm-up cost, which drags the mean upward✓
  • b5 samples is a statistically invalid sample size for any average
  • cthe mean is the wrong summary statistic and standard deviation should be reported instead
  • dstep time naturally decreases over the life of a run, so early steps are meaningless anyway
Explanation:The 420ms first step is an outlier caused by one-time costs that don't recur (thread pool/allocator warm-up, kernel selection, JIT/autotune). Including it in a 5-sample mean pulls the average far above the steady-state value (~37ms) that will hold for the rest of the run.
Ti Throughput Profiling BottlenecksDifficulty 3
30 consecutive step times were recorded (in ms), the first being a one-time warm-up outlier:
[178, 21, 20, 22, 21, 20, 21, 22, 21, 20,
  21, 20, 22, 21, 20, 21, 22, 21, 20, 21,
  22, 21, 20, 21, 22, 21, 20, 21, 22, 21]

Which of these best describes the effect of computing the mean vs. the median over all 30 values?
  • amean and median will be nearly identical because both are equally sensitive to a single outlier
  • bthe median will be pulled upward by the outlier, while the mean stays close to the typical step time
  • cthe mean gets pulled up by the 178ms outlier; the median stays near ~21ms✓
  • dneither statistic is affected, since one outlier out of 30 values is too small a fraction to matter
Explanation:One outlier among 30 values shifts the mean by roughly (178-21)/30 ≈ 5ms, a real and measurable inflation. The median, being the middle-ranked value, is unaffected by the size of a single extreme value as long as it's still just one outlier — it stays at the typical ~21ms.

By specialization

Classical ML 450Deep Learning 450Feature Engineering 450Training Infrastructure 450

By seniority

ML Engineer JuniorML Engineer MidML Engineer Senior

Other fields

BackendFrontendFullstackDevOps / CloudAI EngineerDatabaseMobileSecurityQA / Test AutomationData EngineerGame Development

Test yourself against the 2400-question ML Engineer bank.

Start interview