Optimized HBase Usage / Cluster Monitoring in HBase

Acceldata Pulse provides visualization of Hbase cluster load, HBase latencies, and HBase region sizes.

Region Server JVM

Region server Heap holds the memstore of Hbase Region. A high heap utilization can cause frequent flushes impacting write latencies directly as the heap utilization involves disk i/o. With Pulse, you can visualize cluster-level heap utilization of all region servers.

Cluster Load Visualization: Region Server Level

Most of the cluster faces the latency spike for a couple of minutes and then returns back to normal trend. The actual cause might be increase in cluster traffic due to some promotions, campaigns, or other related events. Using Region Server load visualization in Pulse, you can spot traffic spikes in seconds.

Operation Wise Visualization: Reads Vs Writes

The operation wise visualization helps in identifying the actual use case of the cluster and gives you, as an administrator, more insight to tune the cluster. In case of a read-intensive cluster, you can tune the read buffers. You can tune the cluster to use block cache effectively, enable call queue handlers and tune the queues to have more read queues.

Operation Wise Visualization

Table Level Visualizations

Once you identify the region server which impacts the performance of the cluster, the table level visualizations help to identify the table causing the load or inconsistency.

Hotspot Visualizer - Region Server Level

Ad-Pulse lets your visualize the hotspot at Region Server Level and table level. Visualizing hotspots, in general, at these levels is hard to get.

You can also identify hotspots based on read requests, write requests, memstore size, and store file size.

In the following image, for table level hotspotting, Table H has high read requests between 1700 hours and 2000 hours whereas Table S and Table E are consistent within the timeframe.

Region Server Level

Hotspot Visualizer - Table Level

Region Size Visualizer

Architecturally, Hbase splits the table into regions and shares the regions among all available region servers that serve user requests. All region servers must be used effectively in any cluster. To achieve this in HBase, region sizes and number of regions a particular region server serves, must be balanced.

For example, in a cluster of 10 region servers, HBase table having 100 regions and each region server hosts having 10 regions is balanced.

AD-Pulse table region visualizer helps in identifying a further level of hotspots, that is, the actual region size. You can identify the hotspot when a particular region gets more writes because of incorrectly designed RowKey.

Dashboard

HBase Proctor

Pulse HBase Proctor is a real time dashboard that displays the P95 metrics of all HBase operations throughout the cluster in a single view.

You as an administrator can identify the anomalies in the cluster by monitoring the proctor.