1) Is the problem real?
The problem is MapReduce's high latency (minute-level) is not acceptable in interactive data analysis at scale. Dremel wants to have second-level latency for queries over trillion row table.
2) What is the solution’s main idea (nugget)?
It has two aspects.
1. Its novel columnar storage format. That is to use replication and definition level to translate nested record with column-striped representation. And then propose automaton for assembling records.
2. leverage serving tree for assembling query results by aggregating relies from low-levels of the tree.
3) Why is solution different from previous work?
Compared with its competitive schemes like Pig or Hive, Dremel does not execute query by translating them into MapReduce jobs.
4) Does the paper (or do you) identify any fundamental/hard trade-offs?
Currently, for ad-hoc query using Dremel, I have not found any trade-off here. However, it could not support for data update. In addition, it does not have fault tolerance.
5) Do you think the paper will be influential in 10 years? Why or why not?
I think Dremel maybe the future. It will definitely have impacts in 10 years. It is because Dremel can really query on trillion-level data in second-level, which is amazing. Dremel maybe the next generation of Hive.
No comments:
Post a Comment