Configuring Schema Drift Policies

The schema drift policy detects changes to a schema or table between the previously crawled and currently crawled data sources.

note

If no modifications are made to a table or schema, it is assumed to have passed or succeeded the schema drift policy.

Create a Schema Drift Policy

To create a Schema Drift policy, perform the following:

  1. Click Discover from the left navigation menu. The Discover window is displayed.

  2. Filter and search for schemas and tables by entering their names into the search bar.

  3. Select the asset name for which you want to build a schema drift policy from the search results. This opens the Asset Details page for that asset.

  4. Click the Add Schema Drift Policy link. The Create Schema Drift Policy window is displayed.

    add schema drift policy

  5. Fill in values for the following properties for each tab:

    TabsProperty Description
    Asset InfoDisplays the asset type. Click Add Tag to add a tag for the asset.
    InfoEnter a name and description for the policy in the respective fields provided.
    Alert ConfigurationClick the Alert On drop-down list and select one of the following: ERROR, SUCCESS, ALL.
    Choose a notification channel, such as email or a web-hook, to receive alerts when the schema drift policy execution fails or succeeds. The View details button navigates you to the schema changes tab in the asset details page of the table.

    schema drift notifications

  6. Click the Enable Policy toggle button.

  7. Click the Save Policy button, to save your settings.

Crawl the Data Source

  1. Click Data Sources from the left navigation menu. The Data Source window is displayed.
  2. Search for the data source for which the schema drift policy was created.
  3. On finding the data source, click the ellipsis icon.

start crawlers

  1. From the drop-down list, click Start Crawler. The status of the data source crawler changes from inactive to active.

crawler status

View the Schema Drift Results

After crawling the data source, do the following to see the schema drift policy results:

  1. Click the Data Quality tab from the left navigation menu. The Data Quality window is displayed.

  2. Click the Policy Type drop-down button and select Schema Drift.

  3. From the filtered list, you can view the all the schema drift policies that have been executed. Click a schema drift policy name to view its details.

  4. A panel is displayed with the following details:

    PropertiesDescription
    Policy NameName of the schema drift policy.
    Policy TypeSchema Drift
    Enable toggle button means the policy is enabled and means the policy is disabled.
    Click the toggle button to enable or disable the policy.
    Number of executionsDisplays the count of executions performed on the policy.
    Vertical ellipsis icon Click the ellipsis icon, to perform the following actions: view, edit, delete, and execute
    Policy StatusSuccessful or Errored.
    Manual Execution icon Click the user icon to manually execute the policy.
  5. To view the policy's execution summary, click the tile that includes the date and time the policy was executed. The execution summary for the schema or table is shown. The table below shows the execution summary of a schema drift policy that was created on a table.

    schema drift results

    Column NameDescription
    Column(s) AddedNumber of columns added to the table
    Column(s) ModifiedNumber of columns updated with new data in the table
    Column(s) DeletedNumber of columns deleted from the table
    Last SnapshotDate & time of last snapshot of the crawled data source
    Current SnapshotDate & time of the current snapshot of the crawled data source
    ActionsThe See details links navigates you to the schema changes tab in the asset details page of the table.