Capacity Management


The following components are present in the Yarn Capacity Analyser. These are based on the needs of enterprises during the planning and subsequent operational phases.


Calendar Date Selection

Select the Calendar interval from the top-right of the screen. Acceldata stores historical capacity utilization for a configurable duration.


Queue Filtering

On the left pane of navigation, the queues can be selected. The overall cluster view is then filtered and information related to the queue is only shown. Currently the default queue is slected.

Child queues are also represented, along with their hierarchies upon selection of specific queues.


Application Status Summary

At the top of the page one can see the status of the applications in various stages of their lifecycle. The following stages are displayed:

  • Succeeded
  • Failed
  • Killed
  • Undefined

Additionally, the count of applications of various types with their cumulative numbers is displayed:

  • Yarn
  • Spark
  • Tez
  • MR

Query Metric Distribution

Queue Usage

The usage of the queue from a historical perspective is displayed as well. The total available capacity along with the used capacity is displayed. Also represented is the relationship between parent and child queues, accounting for the hierarchies as well.

Memory & VCore Usage

The following is the representation of usage of Memory and VCore during the selected calendar interval, represented by vertical bar charts.


Top K Users (By Memory & VCore)

Top K Users by way of resource consumption is represented in the bar charts. Some users need greater resources than others, which occasionally represents trouble as well.


CPU, Memory Utilization & Container Allocation

The following graphs represent the overall CPU & Memory utilization. This visualization is very useful in presenting the state of resources in terms of shortage or wastage.


Resource Pool Selection

Resource pools are separated in order to allow different kinds of workloads to be submitted on different queues in question. Typically enterprises use these resource pools for:

  • Ingestion
  • Access

Specific monitoring of capacity on these resource pools is available.