Sample questions
Cml Probabilistic Models CalibrationDifficulty 1
Naive Bayes is called "naive" because of one specific modelling assumption. Which statement expresses that assumption?
- aOnce the class label is known, every feature is independent of every other feature✓
- bBefore the class label is known, the features are independent of one another
- cOnce the class label is known, every feature follows the same distribution
- dEvery feature contributes the same amount of evidence to the class score
Explanation:The assumption is conditional independence: P(x_i | y, x_1, ..., x_n) = P(x_i | y) for every feature, which is what turns the joint likelihood into a product of one-dimensional terms. It says nothing about the features being independent when the class is unknown, and it does not require them to share a distribution or carry equal weight.
Cml Probabilistic Models CalibrationDifficulty 2
Naive Bayes ranks classes with P(y) times the product of P(x_i | y) and simply drops the denominator P(x_1, ..., x_n). Why can it be dropped when choosing a class?
- aIt equals exactly 1 once the feature vector has been normalised
- bIt has the same value for every class, so it cannot change which class wins✓
- cIt is absorbed into the smoothing parameter alpha during fitting
- dIt only carries information when the class priors are unequal
Explanation:The evidence term depends on the sample but not on the candidate class, so it is a common positive factor across all classes and the argmax is unaffected. It is still needed to turn the scores into probabilities, which is why predict_proba renormalises the per-class scores so they sum to one.
Cml Probabilistic Models CalibrationDifficulty 2
In scikit-learn 1.6, class 0 has a single training document with the count vector [3, 2, 0] over a three-word vocabulary. What does MultinomialNB(alpha=1.0) estimate for the probability of the third word in class 0?
- a0.0, because the word never occurs in that class during training
- b0.125, because count 0 becomes 1 and total 5 becomes 8✓
- c0.2, because the smoothed count 1 is divided by the raw class total 5
- d0.1667, because the smoothed count 1 is divided by the class total 5 plus alpha
Explanation:MultinomialNB uses theta_yi = (N_yi + alpha) / (N_y + alpha n_features), so here (0 + 1) / (5 + 1 3) = 0.125. The vocabulary size multiplies alpha in the denominator, which is what keeps the row a valid distribution; forgetting that factor is the usual source of the other values.
Cml Probabilistic Models CalibrationDifficulty 2
A multinomial naive Bayes model is fitted with no smoothing at all. A test document contains one word that never appeared in class 0 during training. What happens to that class's score?
- aThe unseen word is skipped and the remaining words are left to decide the outcome
- bThe class falls back to its own prior probability as the final score it reports
- cThe unseen word gets the average probability of the words seen in that class
- dThe whole product collapses to zero, whatever the other words say about class 0✓
Explanation:Every per-word factor is multiplied together, so a single zero factor annihilates the class score regardless of the other evidence. Run with alpha=0.0 in scikit-learn 1.6 the log of that zero probability becomes -inf and predict_proba returns nan, which is exactly why a smoothing prior is the default.
Cml Probabilistic Models CalibrationDifficulty 2
On a four-word vocabulary a team raises MultinomialNB's alpha from 1.0 to 500. The estimated per-class word probabilities move from [0.6, 0.2, 0.1, 0.1] to [0.252, 0.250, 0.249, 0.249]. What is happening?
- aThe counts are being rescaled so that the rarest words come to dominate the class score
- bThe estimates are being pushed toward 0 and 1, which makes the classifier more decisive
- cThe smoothing mass now dominates the counts, pulling every word toward the uniform 1/n✓
- dThe class priors are being flattened, and the word probabilities simply follow them
Explanation:In (N_yi + alpha) / (N_y + alpha * n) a large alpha swamps both N_yi and N_y, so every entry tends to 1/n and the words stop distinguishing the classes. Smoothing is a prior toward the uniform distribution, so its strength trades bias for protection against zero counts.
Cml Probabilistic Models CalibrationDifficulty 2
Why do naive Bayes implementations accumulate log P(x_i | y) as a sum instead of multiplying the probabilities directly?
- aMultiplying hundreds of small probabilities underflows to exactly 0 in float64✓
- bTaking logarithms makes the independence assumption hold more accurately
- cSummation is required because the individual likelihood terms may turn out negative
- dThe logarithm converts the fitted class prior into a uniform prior over the classes
Explanation:Four hundred factors drawn between 0.001 and 0.05 multiply out to exactly 0.0 in float64, while the corresponding sum of logs is a perfectly usable -1542. Since the logarithm is strictly increasing, the argmax over classes is unchanged, and the scores are exponentiated and renormalised only at the very end.