A company is designing a process for importing a large amount of loT JSON data from cloud storage into Snowflake. New sets of loT data get generated and uploaded approximately every 5 minutes.
Once the loT data is in Snowflake, the company needs up-to-date information from an external vendor to join to the data. This data is then presented to users through a dashboard that shows different levels of aggregation. The external vendor is a Snowflake customer.
What solution will MINIMIZE complexity and MAXIMIZE performance?

Question

A company is designing a process for importing a large amount of loT JSON data from cloud storage into Snowflake. New sets of loT data get generated and uploaded approximately every 5 minutes.

Once the loT data is in Snowflake, the company needs up-to-date information from an external vendor to join to the data. This data is then presented to users through a dashboard that shows different levels of aggregation. The external vendor is a Snowflake customer.

What solution will MINIMIZE complexity and MAXIMIZE performance?

Martin White · Accepted Answer

1. Create a Snowpipe to bring the JSON data into Snowflake. 2. Use streams and tasks to trigger a transformation procedure when new JSON data arrives. 3. Ask the vendor to create a data share with the required data that is then imported into the Snowflake account. 4. Join the vendor's data back to the loT data in a transformation procedure 5. Create materialized views over the larger dataset to perform the aggregations required by the dashboard. 6. Give the materialized views access to the dashboard tool.

Martin White · Answer

1. Create an external table over the JSON data in cloud storage. 2. Create a task that runs every 5 minutes to run a transformation procedure on new data, based on a saved timestamp. 3. Ask the vendor to expose an API so an external function can be used to generate a call to join the data back to the loT data in the transformation procedure. 4. Give the transformed table access to the dashboard tool. 5. Perform the aggregations on the dashboard tool.

Martin White · Answer

1. Create an external table over the JSON data in cloud storage. 2. Create a task that runs every 5 minutes to run a transformation procedure on new data based on a saved timestamp. 3. Ask the vendor to create a data share with the required data that can be imported into the company's Snowflake account. 4. Join the vendor's data back to the loT data using a transformation procedure. 5. Create views over the larger dataset to perform the aggregations required by the dashboard. 6. Give the views access to the dashboard tool.

Martin White · Answer

1. Create a Snowpipe to bring the JSON data into Snowflake. 2. Use streams and tasks to trigger a transformation procedure when new JSON data arrives. 3. Ask the vendor to expose an API so an external function call can be made to join the vendor's data back to the loT data in a transformation procedure. 4. Create materialized views over the larger dataset to perform the aggregations required by the dashboard. 5. Give the materialized views access to the dashboard tool.

Question list

List of questions

Question 1

(0)

Question 2

(0)

Question 3

(0)

Question 4

(0)

Question 5

(0)

Question 6

(0)

Question 7

(0)

Question 8

(0)

Question 9

(0)

Question 10

(0)

Related questions

Question 137 - ARA-C01 discussion

Suggested answer: D

0 comments