Duplicates

Select one or more columns from the input dataset to check for duplicates. If duplicate records are found based on the selected column(s), the test case will be marked as failed. The results will show the total count of duplicate records, along with a sample of the duplicate entries for review.

In the screenshot below, the user has created a rule to detect duplicate records based on the combination of the Name and City columns in the User table.

Duplicates Data Quality UI

If there are any with the same Name and City, they would be shown in the results screen.