Hive on Spark Query Details

The Hive on Spark Query Details page contains the following panels.

  • Summary
  • Query Trends
  • Recommendations
  • Query
  • YARN Diagnostics
  • MapReduce Stats
  • Query Execution Metrics
  • Query Plan and DAG

The summary panel displays the following information.

Field NameDescription
UserThe name of the user that executed the job.
StateThe state of the job that can be one of the following: Created, Initialized, Compiled, Running, Finished, Exception, or Unknown.
DurationThe duration of the query execution.
Start TimeThe time at which the query execution started.
End TimeThe time at which the query execution ended.
Bytes ReadThe number of bytes read while executing the query across a given number of stages.
Bytes WrittenThe number of bytes written to a file format while executing the query across a given number of stages.
Shuffle Bytes ReadThe number of shuffle bytes read (in bytes) across a given number of stages.
Shuffle Bytes WrittenThe number of bytes written in shuffle operations across a given number of stages.
jvmGCTimeTime spent by the JVM in garbage collection while executing a task.

Query Trends

The Query Trends panel displays a chart showing the pattern of jobs running at a particular time, based on the following factors.

MetricDescription
Elapsed TimeThe time taken to run the jobs at a particular time.
VCoresThe number of VCores consumed to execute the query within a timeframe.
MemoryThe amount of memory used to execute the query within a timeframe.

Comparing Runs

Click Compare Runs to compare different runs of the query. Select the runs that you want to compare. You can choose from upto 10 previous runs of the query. The metrics that are different are highlighted and displayed at the very top of the comparison result.

Query

The Query panel displays the query along with the Joins and Table Name used in the query.

Core Usage by Locality

The Core Usage by Locality chart displays the core usage by the following locality types. The chart also displays Core Used and Core Wasted values (in%).

  • Process Local: The tasks in this locality are run within the same process as the source data.
  • Node Local: The tasks in this locality are run on the same machine as the source data.
  • Rack Local: The tasks in this locality are run in the same rack as the source data.
  • Any: The tasks in this locality are run anywhere else but not on the same node or rack.
  • No pref: The tasks in this locality have no locality preference.
  • Idle: The tasks in this locality that are idle.

Query Plan

Query Plan is a logical representation of how MapReduce executes the query, where a query is broken into different stages. The query plan is displayed in JSON format.