The problem is that, the implementation of new statistical technique into RDBMS is a huge and complicated development process. The authors want to propose a unified architecture for in-database analytics. Basically, they want to leverage existing code path as much as possible when developing the new platforms.
The main difference of this paper compared with other related works is that, Bismarck is a general statistical technique that can be span to many different models. In short, it is a more general one.
The authors find out that many common data analytics problem can be formalized as convex programming problem. They propose a unified architecture "Bismarck". The key idea of BISMARCK is to formulate many data analytics as incremental gradient descent (IDG) into RDBMS. The main component of BISMARCK is the IDG with a data access pattern similar to SQL aggregate query.
In addition, the authors also made several optimization in BISMARCK. For example, it changes the per epoch shuffle into a one-time shuffle, etc.
The trade-off is that, even the one-time shuffle reduce the computational complexity, it may lead to bad performance if the rendering order is not good for the IDG.
From my perspective, the general model/system is much more suited than specified ones. Therefore, I think it will still be useful in the future.
No comments:
Post a Comment