Question 142 - MLS-C01 discussion

To improve the training speed of a time-series forecasting model using TensorFlow, the Machine Learning Specialist should change the TensorFlow code to implement a Horovod distributed framework supported by Amazon SageMaker.Horovod is a free and open-source software framework for distributed deep learning training using TensorFlow, Keras, PyTorch, and Apache MXNet1.Horovod can scale up to hundreds of GPUs with upwards of 90% scaling efficiency2.Horovod is easy to use, as it requires only a few lines of Python code to modify an existing training script2.Horovod is also portable, as it runs the same for TensorFlow, Keras, PyTorch, and MXNet; on premise, in the cloud, and on Apache Spark2.

Amazon SageMaker is a fully managed service that provides every developer and data scientist with the ability to build, train, and deploy machine learning models quickly3.Amazon SageMaker supports Horovod as a built-in distributed training framework, which means that the Machine Learning Specialist does not need to install or configure Horovod separately4.Amazon SageMaker also provides a number of features and tools to simplify and optimize the distributed training process, such as automatic scaling, debugging, profiling, and monitoring4. By using Amazon SageMaker, the Machine Learning Specialist can parallelize the training to as many machines as needed to achieve the business goals, while minimizing coding effort and infrastructure changes.

References:

1: Horovod (machine learning) - Wikipedia

2: Home - Horovod

3: Amazon SageMaker -- Machine Learning Service -- AWS

4: Use Horovod with Amazon SageMaker - Amazon SageMaker