Hive on Spark Query Details
The Hive on Spark Query Details page contains the following panels.
- Summary
- Query Trends
- Recommendations
- Query
- YARN Diagnostics
- MapReduce Stats
- Query Execution Metrics
- Query Plan and DAG
The summary panel displays the following information.
| Field Name | Description |
|---|---|
| User | The name of the user that executed the job. |
| State | The state of the job that can be one of the following: Created, Initialized, Compiled, Running, Finished, Exception, or Unknown. |
| Duration | The duration of the query execution. |
| Start Time | The time at which the query execution started. |
| End Time | The time at which the query execution ended. |
| Bytes Read | The number of bytes read while executing the query across a given number of stages. |
| Bytes Written | The number of bytes written to a file format while executing the query across a given number of stages. |
| Shuffle Bytes Read | The number of shuffle bytes read (in bytes) across a given number of stages. |
| Shuffle Bytes Written | The number of bytes written in shuffle operations across a given number of stages. |
| jvmGCTime | Time spent by the JVM in garbage collection while executing a task. |
Query Trends
The Query Trends panel displays a chart showing the pattern of jobs running at a particular time, based on the following factors.
| Metric | Description |
|---|---|
| Elapsed Time | The time taken to run the jobs at a particular time. |
| VCores | The number of VCores consumed to execute the query within a timeframe. |
| Memory | The amount of memory used to execute the query within a timeframe. |
Comparing Runs
Click Compare Runs to compare different runs of the query. Select the runs that you want to compare. You can choose from upto 10 previous runs of the query. The metrics that are different are highlighted and displayed at the very top of the comparison result.
Query
The Query panel displays the query along with the Joins and Table Name used in the query.
Core Usage by Locality
The Core Usage by Locality chart displays the core usage by the following locality types. The chart also displays Core Used and Core Wasted values (in%).
- Process Local: The tasks in this locality are run within the same process as the source data.
- Node Local: The tasks in this locality are run on the same machine as the source data.
- Rack Local: The tasks in this locality are run in the same rack as the source data.
- Any: The tasks in this locality are run anywhere else but not on the same node or rack.
- No pref: The tasks in this locality have no locality preference.
- Idle: The tasks in this locality that are idle.
Query Plan
Query Plan is a logical representation of how MapReduce executes the query, where a query is broken into different stages. The query plan is displayed in JSON format.