Impala Query Details

The Impala Query Details page contains the following panels:

  • Query Trends
  • Recommendations
  • Query
  • Query Timeline
  • Query Execution Metrics
  • Error Details
  • Query Fragment Stats
  • Query Fragment Details
  • Query Plan

The summary panel displays the following information.

Field NameDescription
UserThe name of the user that ran the job.
StateStatus of the job. The state can be one of the following: Created, Initialized, Compiled, Running,Finished,Exception, and Unknown.
DurationThe total time taken by the query to run.
DatabaseThe database on which the query was run.
CoordinatorThe host coordinating this query.
# FragmentsThe number of fragments of the Impala query.
Start TimeThe time at which the query started to run.
End TimeThe time at which the query stopped running.

Query Trends

The Query Trends panel displays a chart showing the pattern of jobs running at a particular time, based on the following factors.

MetricDescription
Elapsed TimeThe time taken to run the jobs at a particular time.
Peak Memory UsageThe highest amount of memory allocated by the query.
HDFS Bytes ReadThe sum of HDFS bytes read from memory by the query.
# of FragmentsThe number of fragments of the Impala query.

Query Trend

Click Compare Runs to compare different runs of the query. Select the runs that you want to compare. You can choose from upto 10 previous runs of the query. The metrics that are different are highlighted and displayed at the very top of the comparison result.

Recommendations

The Recommendations panel displays recommendations that you can use to improve the performance of the SQL Query.

Query

The Query panel displays the query along with the Join Columns and Table.

Query Timeline

The Query Timeline panel displays the stages of the query execution or compilation with the time taken for each stage. You can view the time line by filtering by Execution Time or by Compilation Time. The series of steps for each process is displayed along with the time taken by each step. Hover over a section to view the time taken by the specific step.

Query Timeline

Query Execution Metrics

The query execution metrics are categorized into the following groups - General, Thread Metrics, HDFS Metrics, and Other Metrics.

Metric TypeMetric NameDescription
GeneralQuery IDThe ID of the query.
DatabaseThe database on which the query was run.
Query TypeThe type of the query SQL statement, whether DML, DDL, or Query.
Rows ProducedThe number of rows produced by the query.
Admission ResultThe result of admission, whether admitted immediately, queued, rejected, or timed out.
Admission Waiting TimeThe time from submission for admission to completion of admission.
Per Node Peak Mem UsageThe peak memory usage per node.
Client Fetch Wait TimeThe total amount of time the query spent waiting for the client to fetch row data
Client Fetch Wait Time PercentageThe total amount of time the query spent waiting for the client to fetch row data divided by the query duration, expressed as a percentage.
Estimated per Node Peak MemoryThe planning process's estimate of per-node peak memory usage for the query
Total File FormatsA list of all the file formats used in the query.
Out Of MemoryWhether the query ran out of memory. The value can be True or False.
Thread MetricsAggregate Total Threads CPU TimeThe sum of the CPU time used by all threads of the query.
Aggregate Total Threads CPU Time PercentageThe sum of the CPU time used by all threads of the query divided by the total thread time, expressed as a percentage.
Aggregate Total Network Receive Wait TimeThe sum of the time spent waiting to receive data over the network by all threads of the query.
Aggregate Total Network Receive Wait Time PercentageThe sum of the time spent waiting to receive data over the network by all threads of the query divided by the total thread time, expressed as a percentage.
Aggregate Total Network Send Wait TimeThe sum of the time spent waiting to send data over the network by all threads of the query.
Aggregate Total Network Send Wait Time PercentageThe sum of the time spent waiting to send data over the network by all threads of the query divided by the total thread time, expressed as a percentage.
Aggregate Total Storage Wait TimeThe sum of the time spent waiting for storage by all threads of the query.
Aggregate Total Storage Wait Time PercentageThe sum of the time spent waiting for storage by all threads of the query divided by the total thread time, expressed as a percentage.
Threads Total TimeThe sum of thread CPU, storage wait and network wait times used by all threads of the query.
Aggregate Total Threads System TimeThe total system time taken by all threads for this query.
Aggregate Total Threads Wall Clock TimeThe total wallclock time taken by all threads for this query.
HDFS MetricsHDFS Aggregate Per Host Peak Memory UsageThe sum of peak memory usage of all the nodes for the query.
HDFS Aggregate Peak Memory UsageThe highest amount of memory allocated by this query at a particular time across all nodes.
HDFS Aggregate Bytes ReadThe total number of bytes read from HDFS by this query.
HDFS Aggregate Bytes Read LocalThe total number of local bytes read from HDFS by this query. This is only for completed queries.
HDFS Aggregate Bytes Read RemoteThe total number of remote bytes read from HDFS by this query. This is only for completed queries.
HDFS Aggregate Bytes Read Data Node CacheThe total number of bytes read from HDFS that were read from the HDFS cache. This is only for completed queries.
HDFS Aggregate Bytes Read Short CircuitThe total number of bytes read from HDFS by this query that used short-circuit reads. This is only for completed queries.
Avg HDFS Thread Read ThroughputThe average HDFS scanner read throughput for this query. This is calculates by dividing the total bytes read from HDFS by the total time spent reading by all HDFS scanners.
Other MetricsNetwork AddressThe network address that issued this query.
Node with Peak Memory UsageThe node with the highest peak memory usage for this query.
Planning Wait TimeThe total amount of time the query spent waiting for planning to complete
Planning Wait Time PercentageThe total amount of time the query spent waiting for planning to complete divided by the query duration, expressed as a percentage.
Number Of BackendsThe number of backend connections from this Impala Daemon to other Impala Daemons.
Number of Query Fragments InstancesThe number of query fragment instances.
PoolThe name of the resource pool in which this query executed.
Statistics CorruptWhether the query was flagged with any corrupt blocks warning during the planning process
Statistics MissingWhether the query was flagged with missing table or column statistics warning during the planning process.
Statistics Missing TablesThe name of the missing table.

Error Details

The error panel is displayed only if there is an error during the running of the query. It displays the error encountered during the run.

Query Fragment Stats

The Query Fragments Stats panel displays statistics about the fragments of the query. The fragments are represented by dots and are spread across a scale of lowest to highest value. You can choose the scale as Execution Time, Peak Memory Usage, or HDFS Bytes Read. Hover over a dot to view the fragment name. Click a fragment to view its details.

Query Fragment Details

You can search for a fragment by name. Click a fragment to view its details along with its operators. Click an operator to view it in the Query Plan

ColumnDescription
Fragment NameThe ID of the stage.
TypeThe number of tasks in the stage.
Execution TimeThe graphical representation of the duration of the tasks.
Peak Mem Per HostThe time taken to complete tasks in that stage.
HDFS Bytes ReadThe maximum memory occupied by tasks.
# of RowsThe rate of input/output operations (in %).

Query Plan

The Query Plan displays a pictorial representation of the query. You can also view the text of the fragment or operators of the query in the Query Text tab.

Query Timeline