ExamGecko
Question list
Search
Search

List of questions

Search

Related questions











Question 120 - DP-203 discussion

Report
Export

You have an Azure Databricks workspace named workspace1 in the Standard pricing tier. You need to configure workspace1 to support autoscaling all-purpose clusters. The solution must meet the following requirements:

Automatically scale down workers when the cluster is underutilized for three minutes. Minimize the time it takes to scale to the maximum number of workers. Minimize costs. What should you do first?

A.
Enable container services for workspace1.
Answers
A.
Enable container services for workspace1.
B.
Upgrade workspace1 to the Premium pricing tier.
Answers
B.
Upgrade workspace1 to the Premium pricing tier.
C.
Set Cluster Mode to High Concurrency.
Answers
C.
Set Cluster Mode to High Concurrency.
D.
Create a cluster policy in workspace1.
Answers
D.
Create a cluster policy in workspace1.
Suggested answer: B

Explanation:

For clusters running Databricks Runtime 6.4 and above, optimized autoscaling is used by all-purpose clusters in the Premium plan Optimized autoscaling:

Scales up from min to max in 2 steps.

Can scale down even if the cluster is not idle by looking at shuffle file state. Scales down based on a percentage of current nodes.

On job clusters, scales down if the cluster is underutilized over the last 40 seconds. On all-purpose clusters, scales down if the cluster is underutilized over the last 150 seconds. The spark.databricks.aggressiveWindowDownS Spark configuration property specifies in seconds how often a cluster makes down-scaling decisions. Increasing the value causes a cluster to scale down more slowly. The maximum value is 600.

Note: Standard autoscaling

Starts with adding 8 nodes. Thereafter, scales up exponentially, but can take many steps to reach the max. You can customize the first step by setting the spark.databricks.autoscaling.standardFirstStepUp Spark configuration property. Scales down only when the cluster is completely idle and it has been underutilized for the last 10 minutes. Scales down exponentially, starting with 1 node.

Reference:

https://docs.databricks.com/clusters/configure.html

asked 02/10/2024
Jana Rutrich
29 questions
User
Your answer:
0 comments
Sorted by

Leave a comment first