Sample questions
Aipy Async Concurrency Ml ServingDifficulty 1
In standard CPython, what does the Global Interpreter Lock (GIL) guarantee about threads running inside a single process?
- aThreads are forbidden from calling into any C extension, since the GIL blocks all native code paths.
- bEach thread gets its own dedicated CPU core, so bytecode execution is fully parallel across cores.
- cOnly one thread executes Python bytecode at a time, even if the machine has multiple CPU cores.✓
- dThe GIL only exists during process startup and is dropped once the first thread finishes importing modules.
Explanation:The GIL is a mutex that lets only one thread execute Python bytecode at a time in a standard CPython process, regardless of how many CPU cores are available. (b) describes true multi-core parallelism, which the GIL specifically prevents for Python bytecode. (c) is false — C extensions can and often do release the GIL. (d) invents a startup-only lifetime that doesn't match how the GIL works.
Aipy Async Concurrency Ml ServingDifficulty 1
A team runs a CPU-bound tokenization/preprocessing function on 4 threading.Thread instances, hoping it will finish about 4x faster on a 4-core machine. What actually happens in standard CPython?
- aIt gives little to no speedup, because the GIL still lets only one thread run Python bytecode at a time.✓
- bIt runs almost exactly 4x faster, since each thread claims one core for pure Python computation.
- cIt runs slower than 1 thread by roughly 4x, since the GIL fully serializes all 4 threads end to end with no overlap.
- dThe threads raise a RuntimeError at creation time, since CPython refuses more than one CPU-bound thread.
Explanation:For CPU-bound, pure-Python work, the GIL means the 4 threads still take turns executing bytecode on effectively one core, so there is little to no real speedup — this is exactly why multiprocessing is preferred for CPU-bound Python work. (a) describes true parallelism the GIL prevents. (c) overstates the slowdown; there's some minor GIL-switching overhead, not a 4x penalty. (d) invents a restriction that doesn't exist — threads are created and run fine, just not in parallel for bytecode.
Aipy Async Concurrency Ml ServingDifficulty 2
A model-serving endpoint makes a network call to a remote feature store and waits ~50ms for the response before running inference. Why does using await on that network call (instead of a blocking call) help serve more concurrent requests on the same single thread?
- aIt converts the network call into a CPU-bound operation, so the GIL no longer applies to it at all.
- bIt causes the feature store to respond faster, since awaited calls are prioritized over blocking ones on the wire, similar to how QoS tagging reorders packets.
- cIt spawns a new OS thread behind the scenes for every awaited call, so the wait happens in parallel automatically.
- dWhile waiting on I/O, the coroutine yields control back to the event loop, letting other requests run.✓
Explanation:await on an I/O operation suspends that coroutine and returns control to the event loop, which can then run other pending coroutines while the network response is in flight — this is the core benefit of async for I/O-bound waiting. (a) misdescribes network I/O as CPU-bound. (b) invents network-level prioritization that doesn't exist. (d) describes a thread-per-call model, not how asyncio's single-threaded event loop works.
Aipy Async Concurrency Ml ServingDifficulty 2
Inside an async def request handler, a developer calls a synchronous, CPU-heavy inference function directly (no await, no offloading) that takes 200ms to run. What is the effect on other concurrent requests handled by the same event loop during that 200ms?
- aOther coroutines keep running normally, since
async def handlers are always non-blocking by definition. - bThe event loop is blocked for the full 200ms, so no other coroutine — including unrelated requests — can make progress.✓
- cThe event loop pauses accepting new connections but keeps processing already-scheduled coroutines at full speed.
- dOnly requests from other client IP addresses are delayed; requests from the same client continue unaffected.
Explanation:A synchronous, CPU-bound call inside a coroutine runs on the same single thread as the event loop and never yields control, so it blocks the entire event loop for its whole duration — every other pending coroutine stalls too. (a) is false; declaring a function async def doesn't make blocking code inside it non-blocking. (b) and (c) invent partial-blocking behaviors that don't reflect how a single-threaded event loop works.
Aipy Async Concurrency Ml ServingDifficulty 1
In the context of a Python model-serving API, what best distinguishes 'I/O-bound' work from 'CPU-bound' work?
- aI/O-bound work always runs faster than CPU-bound work, regardless of what the code actually does.
- bI/O-bound work spends most of its time waiting on external systems; CPU-bound work spends most of its time computing.✓
- cCPU-bound work is any code inside an
async def function, while I/O-bound work is any code inside a regular def function. - dI/O-bound work only applies to reading files from disk; any network call is classified as CPU-bound instead, because sockets are managed by kernel threads.
Explanation:The distinction is about where time is actually spent: I/O-bound work (e.g. waiting on a network response or a disk read) is mostly idle waiting, while CPU-bound work (e.g. tensor math, tokenization loops) is mostly active computation — this distinction drives whether async, threads, or processes help. (b) makes a false universal speed claim. (c) confuses the async def/def syntax with the actual nature of the work. (d) wrongly excludes network calls from I/O-bound work.
Aipy Async Concurrency Ml ServingDifficulty 2
import asyncio
async def handle_request(req):
result = run_cpu_heavy_inference(req) # sync, ~150ms CPU work
return result
async def main():
await asyncio.gather(*(handle_request(r) for r in requests))
With 10 requests in
requests, roughly how long does
asyncio.gather take, and why?
- aAbout 150ms total, since
asyncio.gather runs all 10 coroutines' CPU work truly in parallel on separate cores. - bIt hangs indefinitely, since
run_cpu_heavy_inference is not itself declared with async def. - cAbout 15ms total, since
asyncio.gather splits the CPU work evenly across all requests automatically. - dAbout 1500ms total, since each
handle_request call blocks the event loop for its full 150ms with nothing overlapping.✓
Explanation:Calling a synchronous, CPU-bound function directly inside a coroutine (no await, no offloading to a thread/process) blocks the single event loop thread for the full duration of each call; with 10 requests run back to back with no overlap, total time is roughly 10 x 150ms = 1500ms. (a) describes true parallel execution across cores, which asyncio.gather alone does not provide for blocking sync calls. (c) invents automatic work-splitting that doesn't exist. (d) is wrong — calling a plain sync function from a coroutine works fine, it's just blocking.