Skip to main content

Identify Duplicates

The Identify Duplicates feature allows you to identify and manage duplicate records in your query results. This feature helps you find records that share the same value in a specified field (or formula) and provides options to mark records for deletion while retaining others.

Accessing Identify Duplicates​

  1. Execute a SOQL query in Brobench to view records in the datagrid
  2. Click on the More Actions menu (three dots icon) in the datagrid toolbar
  3. Select Identify Duplicates from the menu

The Identify Duplicates dialog will open, displaying all records with duplicate identification columns.

Dialog Overview​

The Identify Duplicates dialog consists of:

  • Control Panel: Configuration options at the top
  • Data Grid: Shows all records with duplicate identification information at the bottom

Configuration Options​

Match Field​

Select the field to use for identifying duplicate records. Records with the same value in this field will be considered duplicates.

Options:

  • Brobench Formula: Use a formula to generate the match value (see Match Field Formula below)
  • Any field from your query: Select from the available fields in your query results

Default: The first available field (excluding "Brobench Formula" and "Id")

Match Field Formula​

This field appears only when "Brobench Formula" is selected as the Match Field.

Enter a Brobench formula to generate the match value. The formula will be evaluated for each record. You can use any of the Brobench Formula functions and expressions to build the Match Value.

Example formulas:

  • FirstName + LastName - Combine two fields
  • UPPER(Email) - Convert email to uppercase for case-insensitive matching
  • IF(IsActive, AccountId, null) - Conditional matching

Refer to the Formula documentation for details on how to use formula functions.

Case Sensitive​

Toggle switch to control whether duplicate matching is case-sensitive.

  • Unchecked (default): Values are compared case-insensitively (e.g., "John" = "john")
  • Checked: Values are compared case-sensitively (e.g., "John" ≠ "john")

Sort Field​

Select the field to use for sorting records within each duplicate group. Records are always sorted in ascending order.

Default Priority:

  1. CreatedDate (if available)
  2. LastModifiedDate (if available)
  3. First available field (excluding "Brobench Formula" and "Id")

Delete Duplicates​

Select which records to mark for deletion from each duplicate group.

Options:

  • None (default): No records are marked for deletion; all records are marked as "Retain"
  • All Records: Mark all records in the duplicate group for deletion
  • All Except First Record: Mark all records except the first one (after sorting) for deletion
  • All Except Last Record: Mark all records except the last one (after sorting) for deletion
  • First n Records: Mark the first n records for deletion
  • Last n Records: Mark the last n records for deletion
  • Except First n Records: Mark all records except the first n records for deletion
  • Except Last n Records: Mark all records except the last n records for deletion

When you select "First n Records", "Last n Records", "Except First n Records", or "Except Last n Records", an additional Records to Delete field appears where you can specify the number (n).

Identify Duplicates Button​

Click the Identify Duplicates button to process the records and identify duplicates based on your current settings. The grid will update to show the duplicate identification results.

Data Grid Columns​

The data grid displays all records with the following special columns at the beginning:

Dupe Status​

Shows the duplicate status of each record:

  • Duplicate: Record is part of a duplicate group (2 or more records with the same match value)
  • Not Duplicate: Record is unique (only one record with this match value)

The status is displayed using a color-coded status badge for easy visual identification.

Dupe Group​

A running number assigned to each group of records that share the same match value. All records in a duplicate group have the same group number.

Dupe Key​

The actual match value that was used to identify the duplicate group. This is the value from the Match Field (or the result of the Match Field Formula).

Dupe Count​

The number of records in the duplicate group. For duplicate groups, this will be 2 or more. For non-duplicate records, this will be 1.

Change​

Indicates the action to be taken for each record:

  • Retain: Record will be kept (not deleted)
  • Delete: Record is marked for deletion

This column is automatically populated based on the Delete Duplicates setting you selected.

Error​

Displays any errors that occur when processing or saving the record. This column is typically empty unless there's an issue.

Original Columns​

All original columns from your query are displayed after the duplicate identification columns.

Filtering Records​

Use the View dropdown in the datagrid actions area to filter the displayed records:

  • -View- (default): Show all records
  • Duplicates: Show only records that are part of duplicate groups
  • Non-Duplicates: Show only unique records (not part of any duplicate group)

Status Information​

The datagrid status bar displays summary information:

  • (No Duplicates): No duplicate records were found
  • (X Duplicate Groups with total Y Duplicate Records): Shows the number of duplicate groups and total duplicate records found

How It Works​

  1. Grouping: Records are grouped by their match value (from Match Field or Match Field Formula)
  2. Status Assignment:
    • Groups with 2+ records are marked as "Duplicate"
    • Groups with 1 record are marked as "Not Duplicate"
  3. Sorting: Within each duplicate group, records are sorted by the Sort Field (ascending)
  4. Action Assignment: Based on the "Delete Duplicates" setting, records are marked as "Delete" or "Retain"
  5. Group Numbering: Each group (duplicate or not) is assigned a sequential group number

Saving Results​

Click Save to apply the duplicate identification results. The results will be applied to the main query datagrid, and the duplicate identification columns will be added to the main grid if they don't already exist.

Best Practices​

  1. Choose the Right Match Field: Select a field that uniquely identifies duplicates (e.g., Email, Phone, or a combination of fields using a formula)

  2. Use Formulas for Complex Matching: When you need to match on multiple fields or apply transformations, use " Brobench Formula" with a formula like:

    • TRIM(UPPER(Email)) - Normalize email addresses
    • FirstName + "|" + LastName + "|" + Phone - Multi-field matching

  3. Sort Field Selection: Choose a sort field that helps you determine which records to keep (e.g., CreatedDate to keep the oldest record, or LastModifiedDate to keep the most recently updated record)

  4. Review Before Saving: Always review the duplicate identification results before clicking Save, especially when using "All Records" or other aggressive deletion options

  5. Case Sensitivity: Consider your use case - for email addresses, you typically want case-insensitive matching, but for some fields, case sensitivity may be important

Example Use Cases​

Finding Duplicate Contacts by Email​

  1. Query contacts: SELECT Id, Name, Email FROM Contact
  2. Open Identify Duplicates
  3. Select Email as Match Field
  4. Select CreatedDate as Sort Field (to keep oldest contact)
  5. Select All Except First Record as Delete Duplicates
  6. Click Identify Duplicates
  7. Review results and click Save

Finding Duplicate Accounts by Name and Billing City​

  1. Query accounts: SELECT Id, Name, BillingCity FROM Account
  2. Open Identify Duplicates
  3. Select Brobench Formula as Match Field
  4. Enter formula: Name + "|" + BillingCity
  5. Select LastModifiedDate as Sort Field
  6. Select All Except Last Record as Delete Duplicates (keep most recently modified)
  7. Click Identify Duplicates
  8. Review results and click Save

Case-Insensitive Name Matching​

  1. Query records with a Name field
  2. Open Identify Duplicates
  3. Select Brobench Formula as Match Field
  4. Enter formula: UPPER(Name) to normalize case
  5. Ensure Case Sensitive is unchecked
  6. Configure other settings as needed
  7. Click Identify Duplicates