yoklainterview sim

Data Engineer St Kafka Broker Storage Replication Interview Questions

75 verified Data Engineer St Kafka Broker Storage Replication interview questions — solve with answers, learn from explanations, test yourself in a real simulation.

Try the real simulation →

Sample questions

St Kafka Broker Storage ReplicationDifficulty 1
On a Kafka broker's disk, how is a single partition physically stored?
  • aAs one continuously growing file, regardless of data volume
  • bAs rows inside a relational database table the broker manages
  • cAs a sequence of log segment files, one of which is active
  • dEntirely in memory, flushed to disk only at broker shutdown
Explanation:A partition's log is split into segment files on disk. Only one segment (the active segment) accepts new appends at a time; older segments are closed (rolled) and stay read-only until deleted or compacted.
St Kafka Broker Storage ReplicationDifficulty 1
What causes Kafka to roll a partition's active log segment (close it, start a new one)?
  • aReaching segment.bytes in size or segment.ms in age, whichever comes first
  • bA consumer group finishing all reads of the active segment
  • cThe broker's JVM running a garbage collection cycle
  • dThe partition leader moving to a different broker
Explanation:Segment rolling is driven purely by size (segment.bytes) and age (segment.ms) of the active segment; whichever threshold is hit first triggers the roll. Consumer progress and leader changes don't roll segments.
St Kafka Broker Storage ReplicationDifficulty 3
A topic-partition has segment.bytes=1073741824 (1 GiB) and segment.ms=604800000 (7 days). It steadily receives about 50 MB of data per day. After 20 days of continuous production, roughly how many times has the active segment been rolled, and which threshold caused it?
  • aAbout 20 rolls, all triggered by segment.bytes filling up daily
  • b2 rolls, both triggered by segment.ms, not by size
  • cExactly 1 roll, triggered by segment.bytes at day 20
  • dZero rolls, since segments only roll on broker restart
Explanation:At 50 MB/day, 7 days of writes accumulate only ~350 MB, well under the 1 GiB segment.bytes limit, so segment.ms fires first. Rolls occur at days 7 and 14; the third would be at day 21. Thus there are 2 rolls in 20 days, both driven by time rather than size.
St Kafka Broker Storage ReplicationDifficulty 1
Each Kafka log segment (.log file) is paired with .index and .timeindex files. What are these used for?
  • aStoring the replica assignment for the partition
  • bStoring the consumer group's committed offsets
  • cHolding a compressed backup copy of the segment
  • dMapping offsets/timestamps to approximate byte positions
Explanation:The .index file maps offsets to byte positions and .timeindex maps timestamps to offsets, both sparsely. Kafka uses them to jump close to the target position and then scan a small range, instead of reading the segment from the start.
St Kafka Broker Storage ReplicationDifficulty 2
A topic has cleanup.policy=delete and retention.ms=604800000 (7 days). A log segment was closed (rolled) 8 days ago, and its newest message is 7.5 days old. A newer, still-active segment holds only the last few hours. What happens to the closed segment?
  • aThe whole closed segment becomes eligible for segment-level deletion
  • bOnly the individual messages older than 7 days are removed
  • cNothing is deleted until the active segment also ages out
  • dThe segment is compacted to keep the latest value per key
Explanation:Deletion under cleanup.policy=delete happens at segment granularity, not per message. Once every message in a closed segment is older than the retention threshold, that segment becomes eligible and is selected at a later retention check; physical file removal may be delayed further. The active segment is ineligible while it accepts writes.
St Kafka Broker Storage ReplicationDifficulty 2
A topic is configured with cleanup.policy=compact. What does log compaction do, at its core?
  • aRemoves every message older than a fixed age, like delete does
  • bKeeps at least the latest value for each distinct message key
  • cCompresses payloads with gzip/zstd without removing records
  • dMerges every partition of the topic into a single partition
Explanation:Compaction is per-key retention, not time-based deletion or payload compression: the log cleaner rewrites segments so that only the latest record for each key survives (plus recent, not-yet-cleaned records), letting the topic act like a changelog of current state.

Test yourself against the 1950-question Data Engineer bank.

Start interview