Data Source Auto Profiling

Tables from a data source can be scheduled for profiling by configuring the data source.

Profiling large assets or data sets instantaneously could be time-consuming. Auto profiling is important because it saves you a lot of time. Once a data source is configured for auto profiling, you will always be able to view the most recently updated data on the asset details page.

Add Configuration

To configure a cron based profiling job for a data source, follow the given procedure.

  1. Navigate to the Settings tab from the left navigation menu bar. The Settings page is displayed.

  2. Click Auto Profile from the left menu bar. The Data Source Auto Profiling page is displayed. ![auto profile](/assets/auto profile.png)

  3. Click Add Configuration. The Create Auto Profile Config wizard is displayed. create auto profile

  4. Fill in the following properties

    Field NameDescription
    Select Data SourceFrom the drop-down list, select a data source to configure and start auto profiling.
    Execution ScheduleSelect one of the following time tags: Minute, Hour, Day, Week, Month, and Year.
    Include TablesWrite a comma-separated regex query to include the tables that need to be auto profiled.
    Exclude TablesWrite a comma-separated regex query to exclude the tables from being auto profiled.
    Show AssetsOn including or excluding tables, click show assets to view the assets that belong to the tables.
    Parallelization CountParallelization count allows you to select a number of tables that can be auto profiled simultaneously.
  5. Click the toggle button to enable or disable the configuration.

  6. Click Save.

View Configurations

Once a data source is configured, it is displayed in a table along with the below properties:

Column NameDescription
Data SourceName of the data source that was configured.
ScheduledTime that was scheduled for the cron job.
Parallelization CountNumber of tables selected for parallelization.
Enabled for disabled and for enabled.
Created AtDate and time at which the configuration happened.
Updated AtDate and time at which the configuration was last updated.

view auto profile configurations

Delete Configurations

To delete a configuration, follow the given procedure:

  1. Click the three vertical ellipsis icon .
  2. Click Delete. A confirmation dialog box is displayed.

delete configurations

  1. Click Ok.

Edit Configurations

To edit a configuration, follow the given procedure:

  1. Click the name of a data source from the data source auto profile configuration page or click the three vertical ellipsis icon and click Edit .
  2. The Edit Auto Profile Configuration wizard is displayed.
  3. Make your changes to the configuration.
  4. Click Save.

edit autoprofile configuration