ExamGecko
Question list
Search
Search

List of questions

Search

Related questions











Question 70 - DEA-C01 discussion

Report
Export

A data engineer needs to maintain a central metadata repository that users access through Amazon EMR and Amazon Athena queries. The repository needs to provide the schema and properties of many tables. Some of the metadata is stored in Apache Hive. The data engineer needs to import the metadata from Hive into the central metadata repository.

Which solution will meet these requirements with the LEAST development effort?

A.

Use Amazon EMR and Apache Ranger.

Answers
A.

Use Amazon EMR and Apache Ranger.

B.

Use a Hive metastore on an EMR cluster.

Answers
B.

Use a Hive metastore on an EMR cluster.

C.

Use the AWS Glue Data Catalog.

Answers
C.

Use the AWS Glue Data Catalog.

D.

Use a metastore on an Amazon RDS for MySQL DB instance.

Answers
D.

Use a metastore on an Amazon RDS for MySQL DB instance.

Suggested answer: C

Explanation:

The AWS Glue Data Catalog is an Apache Hive metastore-compatible catalog that provides a central metadata repository for various data sources and formats. You can use the AWS Glue Data Catalog as an external Hive metastore for Amazon EMR and Amazon Athena queries, and import metadata from existing Hive metastores into the Data Catalog. This solution requires the least development effort, as you can use AWS Glue crawlers to automatically discover and catalog the metadata from Hive, and use the AWS Glue console, AWS CLI, or Amazon EMR API to configure the Data Catalog as the Hive metastore. The other options are either more complex or require additional steps, such as setting up Apache Ranger for security, managing a Hive metastore on an EMR cluster or an RDS instance, or migrating the metadata manually.Reference:

Using the AWS Glue Data Catalog as the metastore for Hive(Section: Specifying AWS Glue Data Catalog as the metastore)

Metadata Management: Hive Metastore vs AWS Glue(Section: AWS Glue Data Catalog)

AWS Glue Data Catalog support for Spark SQL jobs(Section: Importing metadata from an existing Hive metastore)

AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide (Chapter 5, page 131)

asked 29/10/2024
Brad Jarrett
42 questions
User
Your answer:
0 comments
Sorted by

Leave a comment first