List of questions
Related questions
Question 280 - Professional Data Engineer discussion
You are running a Dataflow streaming pipeline, with Streaming Engine and Horizontal Autoscaling enabled. You have set the maximum number of workers to 1000. The input of your pipeline is Pub/Sub messages with notifications from Cloud Storage One of the pipeline transforms reads CSV files and emits an element for every CSV line. The Job performance is low. the pipeline is using only 10 workers, and you notice that the autoscaler is not spinning up additional workers. What should you do to improve performance?
A.
Use Dataflow Prime, and enable Right Fitting to increase the worker resources.
B.
Update the job to increase the maximum number of workers.
C.
Enable Vertical Autoscaling to let the pipeline use larger workers.
D.
Change the pipeline code, and introduce a Reshuffle step to prevent fusion.
Your answer:
0 comments
Sorted by
Leave a comment first