Review for Omega (Eurosys'13)

1) Is the problem real?
Running a mix of data processing workloads on large scale cluster introduces a high workload on scheduler. And in the future, the scheduler may be the bottleneck. Therefore, it is a must to design a new framework for scheduler on cluster.

2) What is the solution’s main idea (nugget)?

Omega grants each scheduler full access to the entire cluster, and offers the resources allocation decision by the framework schedulers themselves.

When there is conflict between framework schedulers. Omega scheduler will conduct an all-or-nothing gang scheduling to solve that.

3) Why is solution different from previous work?

Previous works are mainly separated in two categories. First, Monolithic scheduler uses a centralized scheduling algorithm for all jobs. It is not scalable  The second kind is called two-level scheduler, which has a single resource manager offers resource among different frameworks in parallel. This kind of scheme cannot support preemption or policies requiring access to the whole cluster state. It is because each framework scheduler cannot have a view of overall cluster.

Omega offers each scheduler full access to the entire cluster. Therefore, it can be scalable. In addition, compared with 2-level scheduling, this idea also reduce the scheduler workload, since there is no need for the Omega scheduler to make the resource allocation decision.

4) Does the paper (or do you) identify any fundamental/hard trade-offs? 


Omega gives each framework scheduler full access to the entire cluster, it may improve cluster utilization efficiency. However, this full-access offer also gives scheduler opportunity to devour all the computing resources or even preempt other framework's resources. Thus there is no fairness at all.

5) Do you think the paper will be influential in 10 years? Why or why not?

Yeah, I think it is good way to reduce workloads on scheduler. Simple and efficient scheduler is the need for running mixed workload on large scale cluster scheduling. However, it is a prototype, without considering the issue of fair-sharing, etc. Hope a more sophisticated version may be proposed instead of this one.