Two-layer representation of the maximum of five numbers

Determine whether two hidden layers suffice to represent the function $x\mapsto\max\{0,x_1,x_2,x_3,x_4\}$ by a ReLU network, without imposing restrictions beyond those specified for the network model under consideration.

Background

The maximum function is used in the paper as the central test case for lower bounds on exact ReLU depth. The case $\max\{0,x_1,x_2\}$ is known not to be representable with one hidden layer, while the paper focuses on determining depth lower bounds for larger maxima.

At this point in the introduction, the authors identify the five-term maximum as an unresolved two-layer case. Later, the paper resolves this question under the $B_d^0$-conformity assumption by proving that three hidden layers are necessary, but the unrestricted version stated here remains an open problem.