Convexity of the density function over the FM-tetrahedron domain

Establish whether the density function δ(T), mapping each FM-tetrahedron (specified by its six edge lengths subject to the FM-tetrahedron constraints) to the proportion of its volume covered by the four incident spheres of radii 1 or r, is convex on the six-dimensional domain of FM-tetrahedra for radii 1 and r = √2 − 1.

Background

Maximizing density over the compact but non-polyhedral domain of FM-tetrahedra is formulated as a challenging optimization problem involving sums of arctangent expressions and algebraic constraints (Cayley–Menger positivity, existence of a support sphere with radius ≤ r).

Convexity of the objective would enable powerful optimization tools and reductions, but the authors note that, while plausible, verifying convexity analytically or computationally is difficult.