Strict growth of ReLU expressivity with every hidden layer

Prove or disprove that, for every integer $\ell\leq\lceil\log_2(d+1)\rceil$, the inclusion $ReLU_d(\ell-1)\subsetneq ReLU_d(\ell)$ is strict, equivalently determining whether successive hidden layers strictly increase the class of exactly representable CPWL functions up to logarithmic depth.

Background

Here $ReLU_d(\ell)$ denotes the set of CPWL functions on R^d computable with $\ell$ hidden ReLU layers. The conjecture would establish that the logarithmic-depth upper bound is tight layer by layer, rather than merely showing that some function requires a particular depth.

The paper states that this conjecture has been confirmed only in special cases and remains unresolved in general. It further identifies the maximum function as a key candidate for separating consecutive depth classes, but the general conjecture is not settled.