Auto Actions

Actions are a unique feature which enables operators to run workflows to perform some activity. These can be run at various modes.

  • One off - An operator can trigger an action from UI when ever required.
  • Scheduled - An operator can schedule an action to execute it at varying interval. This can also be configured to run infinitely or till a certain time period.
  • Alerts - An action can also be configured to run when an incident is raised for an alert.

Important concepts for Actions

ConceptDescription
StepMost granular entity which wraps an executor. One or more steps are executed when a workflow is run.
WorkflowWorkflow consists of a bag of steps, their configurations and dependency information between them. Esentially, the workflow is defined as a DAG of steps.
ActionActions wraps a workflow and its optional schedule. It is what is fired by a user or a system.
ScheduleSimilar to a crontab configuration, this determines at what interval and how this action will be fired.

Out of the box actions

Kill rougue Yarn application action

  • On a Node
  • On a Queue
  • Resource Utilization Based
  • Application Priority

HDFS Actions

  • Clean Trash Files / User
  • Clean Trash Files / Service
  • Log Compression - Ranger, Sentry

Hbase Actions

  • Trigger Hbck (1&2)
  • Trigger Hbase Rebalance

Service Disruption and auto-restarts

  • DWH Auto Actions
  • Table level compaction
  • Leaf/Partition level compaction

ETL Integration (post data-load, stats gather)

  • Stats Generation

Custom actions

We provide Steps which can be used to execute arbitary shell scripts, python scripts or Java functions. Operations team will model their custom runbooks as workflows and execute them manually or in a scheduled manner.