Exam Data-Engineer-Associate Topic 1 Question 122 Discussion

Actual exam question for Amazon's Data-Engineer-Associate exam
Question #: 122
Topic #: 1

A company uploads .csv files to an Amazon S3 bucket. The company's data platform team has set up an AWS Glue crawler to perform data discovery and to create the tables and schemas.
An AWS Glue job writes processed data from the tables to an Amazon Redshift database. The AWS Glue job handles column mapping and creates the Amazon Redshift tables in the Redshift database appropriately.
If the company reruns the AWS Glue job for any reason, duplicate records are introduced into the Amazon Redshift tables. The company needs a solution that will update the Redshift tables without duplicates.
Which solution will meet these requirements?

A. Modify the AWS Glue job to copy the rows into a staging Redshift table. Add SQL commands to update the existing rows with new values from the staging Redshift table. B. Modify the AWS Glue job to load the previously inserted data into a MySQL database. Perform an upsert operation in the MySQL database. Copy the results to the Amazon Redshift tables. C. Use Apache Spark's DataFrame dropDuplicates() API to eliminate duplicates. Write the data to the Redshift tables. D. Use the AWS Glue ResolveChoice built-in transform to select the value of the column from the most recent record.

Suggested Answer: A Vote an answer

To avoid duplicate records in Amazon Redshift, the most effective solution is to perform the ETL in a way that first loads the data into astaging tableand then uses SQL commands like MERGE or UPDATE to insert new records and update existing records without introducing duplicates.
* Using Staging Tables in Redshift:
* The AWS Glue job can write data to astaging tablein Redshift. Once the data is loaded, SQL commands can be executed to compare the staging data with the target table andupdate or insert records appropriately. This ensures no duplicates are introduced during re-runs of the Glue job.
Reference:Amazon Redshift Best Practices
Alternatives Considered:
B (MySQL upsert): This introduces unnecessary complexity by involving another database (MySQL).
C (Spark dropDuplicates): While Spark can eliminate duplicates, handling duplicates at the Redshift level with a staging table is a more reliable and Redshift-native solution.
D (AWS Glue ResolveChoice): The ResolveChoice transform in Glue helps with column conflicts but does not handle record-level duplicates effectively.
References:
Amazon Redshift MERGE Statements
Staging Tables in Amazon Redshift

by Gordon at Apr 07, 2026, 02:36 AM

Limited Time Offer

15%

Off

Get Premium Data-Engineer-Associate Questions as Interactive Self Test Engine or PDF

Comments

Here are all the actual test exam dumps for IT exams. Most people prepare for the actual exams with our test dumps to pass their exams. So it's critical to choose and actual test pdf to succeed.

RECENT DISCUSSIONS

Useful Links

Contact Us

Our Working Time: ( GMT 0:00-15:00 )
From Monday to Saturday

Support: Contact now

If you have any question please leave me your email address, we will reply and send email to you in 12 hours.

Disclaimer:
Actual4test doesn't offer Real SANS and GIAC Exam Questions.
Oracle and Java are registered trademarks of Oracle and/or its affiliates
Actual4test material do not contain actual actual Oracle Exam Questions or material.
Actual4test doesn't offer Real Microsoft Exam Questions.
Microsoft®, Azure®, Windows®, Windows Vista®, and the Windows logo are registered trademarks of Microsoft Corporation
Actual4test Materials do not contain actual questions and answers from Cisco's Certification Exams. The brand Cisco is a registered trademark of CISCO, Inc
CFA Institute does not endorse, promote or warrant the accuracy or quality of these questions. CFA® and Chartered Financial Analyst® are registered trademarks owned by CFA Institute.
Actual4test does not offer exam dumps or questions from actual exams. We offer learning material and practice tests created by subject matter experts to assist and help learners prepare for those exams. All certification brands used on the website are owned by the respective brand owners. Actual4test does not own or claim any ownership on any of the brands.