Batch latency analysis and phase transitions for a tandem of queues with exponentially distributed service times

Abstract: We analyze the latency or sojourn time L(m,n) for the last customer in a batch of n customers to exit from the m-th queue in a tandem of m queues in the setting where the queues are in equilibrium before the batch of customers arrives at the first queue. We first characterize the distribution of L(m,n) exactly for every m and n, under the assumption that the queues have unlimited buffers and that each server has customer independent, exponentially distributed service times with an arbitrary, known rate. We then evaluate the first two leading order terms of the distributions in the large m and n limit and bring into sharp focus the existence of phase transitions in the system behavior. The phase transition occurs due to the presence of either slow bottleneck servers or a high external arrival rate. We determine the critical thresholds for the service rate and the arrival rate, respectively, about which this phase transition occurs; it turns out that they are the same. This critical threshold depends, in a manner we make explicit, on the individual service rates, the number of customers and the number of queues but not on the external arrival rate.