ExamGecko
Question list
Search
Search

List of questions

Search

Related questions











Question 75 - Professional Machine Learning Engineer discussion

Report
Export

You work for a gaming company that manages a popular online multiplayer game where teams with 6 players play against each other in 5-minute battles. There are many new players every day. You need to build a model that automatically assigns available players to teams in real time. User research indicates that the game is more enjoyable when battles have players with similar skill levels. Which business metrics should you track to measure your model's performance? (Choose One Correct Answer)

A.
Average time players wait before being assigned to a team
Answers
A.
Average time players wait before being assigned to a team
B.
Precision and recall of assigning players to teams based on their predicted versus actual ability
Answers
B.
Precision and recall of assigning players to teams based on their predicted versus actual ability
C.
User engagement as measured by the number of battles played daily per user
Answers
C.
User engagement as measured by the number of battles played daily per user
D.
Rate of return as measured by additional revenue generated minus the cost of developing a new model
Answers
D.
Rate of return as measured by additional revenue generated minus the cost of developing a new model
Suggested answer: C

Explanation:

The best business metric to track to measure the model's performance is user engagement as measured by the number of battles played daily per user. This metric reflects the main goal of the model, which is to enhance the user experience and satisfaction by creating balanced and fair battles. If the model is successful, it should increase the user retention and loyalty, as well as the word-of-mouth and referrals. This metric is also easy to measure and interpret, as it can be directly obtained from the user activity data.

The other options are not optimal for the following reasons:

A) Average time players wait before being assigned to a team is not a good metric, as it does not capture the quality or outcome of the battles. It only measures the efficiency of the model, which is not the primary objective. Moreover, this metric can be influenced by external factors, such as the availability and demand of players, the network latency, and the server capacity.

B) Precision and recall of assigning players to teams based on their predicted versus actual ability is not a good metric, as it is difficult to measure and interpret. It requires having a reliable and consistent way of estimating the player's ability, which can be subjective and dynamic. It also requires having a ground truth label for each assignment, which can be costly and impractical to obtain. Moreover, this metric does not reflect the user feedback or satisfaction, which is the ultimate goal of the model.

D) Rate of return as measured by additional revenue generated minus the cost of developing a new model is not a good metric, as it is not directly related to the model's performance. It measures the profitability of the model, which is a secondary objective. Moreover, this metric can be affected by many other factors, such as the market conditions, the pricing strategy, the marketing campaigns, and the competition.

Professional ML Engineer Exam Guide

Preparing for Google Cloud Certification: Machine Learning Engineer Professional Certificate

Google Cloud launches machine learning engineer certification

How to measure user engagement

How to choose the right metrics for your machine learning model

asked 18/09/2024
mark anthony sampayan
34 questions
User
Your answer:
0 comments
Sorted by

Leave a comment first