Review for "pFabric: Minimal Near-Optimal Datacenter Transport"

In existing TCP-based fabrics, it introduce high latency in short flow processing. The key reason for this is that the short flows are often get queued behind the large amount of packets from large flow of the co-existing network workload. Therefore, this queue latency significantly introduce more latency on the short flows.

The most important concept here is that, rate control is a poor and ineffective scheme for flow control. The key design of pFabric is a priority-based packet scheduling. More precisely, each host put a priority number in the header of each packet. And the switch can determine which packet should in the queue and which should be scheduled strictly by its priority. When the switch buffer is full, the switch can compare the priority level to decide which packed should be dropped. For the flow control, all the flows start with line-speed and being limited only when there is high loss rate.

Previous works that try to solve this short flow's high latency problem by using rate control. The first kind of works is to reduce the queue length (i.e. keep it nearly empty). Therefore, with less queue, there is less latency, which improves FCT (flow completion times). The second kind of approach is to repeatedly calculate and assign rate to each flow in the network. However, it is complicated and impractical in real-world deployment.

The trade off here is since pFabric guarantee the short flow to have short FCT, it may leads to starvation on the large flows.

pFabric is essentially useful for interactive short flow transmission in the real-world scenarios. I think it will be influential with 10 years.