The schema drift policy detects changes to a schema or table between the previously crawled and currently crawled data sources.
If no modifications are made to a table or schema, it is assumed to have passed or succeeded the schema drift policy.
Create a Schema Drift Policy
To create a Schema Drift policy, perform the following:
Click Discover from the left navigation menu. The Discover window is displayed.
Filter and search for schemas and tables by entering their names into the search bar.
Select the asset name for which you want to build a schema drift policy from the search results. This opens the Asset Details page for that asset.
Click the Add Schema Drift Policy link. The Create Schema Drift Policy window is displayed.
Fill in values for the following properties for each tab:
Tabs Property Description Asset Info Displays the asset type. Click Add Tag to add a tag for the asset. Info Enter a name and description for the policy in the respective fields provided. Alert Configuration Click the Alert On drop-down list and select one of the following: ERROR, SUCCESS, ALL.
Choose a notification channel, such as email or a web-hook, to receive alerts when the schema drift policy execution fails or succeeds. The View details button navigates you to the schema changes tab in the asset details page of the table.
Click the Enable Policy toggle button.
Click the Save Policy button, to save your settings.
Crawl the Data Source
- Click Data Sources from the left navigation menu. The Data Source window is displayed.
- Search for the data source for which the schema drift policy was created.
- On finding the data source, click the ellipsis icon.
- From the drop-down list, click Start Crawler. The status of the data source crawler changes from inactive to active.
View the Schema Drift Results
After crawling the data source, do the following to see the schema drift policy results:
Click the Data Quality tab from the left navigation menu. The Data Quality window is displayed.
Click the Policy Type drop-down button and select Schema Drift.
From the filtered list, you can view the all the schema drift policies that have been executed. Click a schema drift policy name to view its details.
A panel is displayed with the following details:
Properties Description Policy Name Name of the schema drift policy. Policy Type Schema Drift Enable toggle button means the policy is enabled and means the policy is disabled.
Click the toggle button to enable or disable the policy.
Number of executions Displays the count of executions performed on the policy. Vertical ellipsis icon Click the ellipsis icon, to perform the following actions: view, edit, delete, and execute Policy Status Successful or Errored. Manual Execution icon Click the user icon to manually execute the policy.
To view the policy's execution summary, click the tile that includes the date and time the policy was executed. The execution summary for the schema or table is shown. The table below shows the execution summary of a schema drift policy that was created on a table.
Column Name Description Column(s) Added Number of columns added to the table Column(s) Modified Number of columns updated with new data in the table Column(s) Deleted Number of columns deleted from the table Last Snapshot Date & time of last snapshot of the crawled data source Current Snapshot Date & time of the current snapshot of the crawled data source Actions The See details links navigates you to the schema changes tab in the asset details page of the table.