The problem of conducting SQL query on Hive is its high latency. This is because Hive is based on MapReduce. The Impala is trying to reduce this latency.
Impala is consisted of 3 main services: Impala daemon (impalad), statestore daemon, Catalog daemon.
Impala daemon: run on each machine in the cluster. It is used for accepting queries and manage executing across the cluster.
Statestore daemon: metadata publish-subscribe service. It transmits cluster-wide metadata to all impala processes.
Catalog daemon: serve as Impala's catalog repository and metadata gateway.
I think the most important contribution of Impala is to translate SQL into query plans, without leveraging. In addition, since the intermediate results are stored in memory instead of writing back to disk. I think it is a good way to do so.
Compared with Hive, Impala do not need to use MapReduce, which has long latency for processing SQL query. In addition, in memory processing is also a difference compared with other related works.
For the tradeoff, Impala do not have fault tolerance. However, since the time overhead is low. Impala can re-do the failed query.
For the influence, I think Impala is kind of a Hadoop version of Dremel. So for the novelty, I do not give it much credits.
No comments:
Post a Comment