ExamGecko
Question list
Search
Search

List of questions

Search

Related questions











Question 137 - Professional Machine Learning Engineer discussion

Report
Export

You work for a magazine publisher and have been tasked with predicting whether customers will cancel their annual subscription. In your exploratory data analysis, you find that 90% of individuals renew their subscription every year, and only 10% of individuals cancel their subscription. After training a NN Classifier, your model predicts those who cancel their subscription with 99% accuracy and predicts those who renew their subscription with 82% accuracy. How should you interpret these results?

A.
This is not a good result because the model should have a higher accuracy for those who renew their subscription than for those who cancel their subscription.
Answers
A.
This is not a good result because the model should have a higher accuracy for those who renew their subscription than for those who cancel their subscription.
B.
This is not a good result because the model is performing worse than predicting that people will always renew their subscription.
Answers
B.
This is not a good result because the model is performing worse than predicting that people will always renew their subscription.
C.
This is a good result because predicting those who cancel their subscription is more difficult, since there is less data for this group.
Answers
C.
This is a good result because predicting those who cancel their subscription is more difficult, since there is less data for this group.
D.
This is a good result because the accuracy across both groups is greater than 80%.
Answers
D.
This is a good result because the accuracy across both groups is greater than 80%.
Suggested answer: B

Explanation:

This is not a good result because the model is performing worse than predicting that people will always renew their subscription. This option has the following reasons:

It indicates that the model is not learning from the data, but rather memorizing the majority class. Since 90% of the individuals renew their subscription every year, the model can achieve a 90% accuracy by simply predicting that everyone will renew their subscription, without considering the features or the patterns in the data. However, the model's accuracy for predicting those who renew their subscription is only 82%, which is lower than the baseline accuracy of 90%. This suggests that the model is overfitting to the minority class (those who cancel their subscription), and underfitting to the majority class (those who renew their subscription).

It implies that the model is not useful for the business problem, as it cannot identify the customers who are at risk of churning. The goal of predicting whether customers will cancel their annual subscription is to prevent customer churn and increase customer retention. However, the model's accuracy for predicting those who cancel their subscription is 99%, which is too high and unrealistic, as it means that the model can almost perfectly identify the customers who will churn, without any false positives or false negatives. This may indicate that the model is cheating or exploiting some leakage in the data, such as a feature that reveals the outcome of the prediction. Moreover, the model's accuracy for predicting those who renew their subscription is 82%, which is too low and unreliable, as it means that the model can miss many customers who will churn, and falsely label them as renewing customers. This can lead to losing customers and revenue, and failing to take proactive actions to retain them.

How to Evaluate Machine Learning Models: Classification Metrics | Machine Learning Mastery

Imbalanced Classification: Predicting Subscription Churn | Machine Learning Mastery

asked 18/09/2024
Kris Dayananda
36 questions
User
Your answer:
0 comments
Sorted by

Leave a comment first