Graph-structure computation is very popular and important in data processing. However, the real-world graph have highly skewed power-law degree. More precisely, a small amount of vertex have connections with a large fraction of nodes within the graph.
Therefore, Power graph tries to provide great parallelism and reduce the network connections in the graph processing. The key concept of PowerGraph is to exploring the factor computation over edges instead of vertices.
Different from previous works like Pregel or GraphLab, these kind of system cannot deal with the high-skewed power-law degree problem. Whereas PowerGraph is designed for deal with this problem.
The nugget of PowerGraph is to "Think like a vertex". Basically, it tries to split the high-degree vertex and design a new abstraction for the spliced vertices. Therefore, PowerGraph can run multiple vertex programs in parallel in a cluster instead of a single node.
The high parallelization is based on increasing the storage overhead. I think it maybe a trade-off here for the power graph.
Graph problem is a main challenge in nowadays data processing. In addition, the increasing using of large-scale cluster computing is a big trend. To processing graph problem based on these big data processing system is a important way to achieve high performance. I think it will be influential within 10 years.
No comments:
Post a Comment