Canonical Hexagons and the PSL(2,C) Discreteness Problem

Abstract: The discreteness problem, that is, the problem of determining whether or not a given finitely generated group G of orientation preserving isometries of hyperbolic three-space is discrete as a subgroup of the whole isometry group of hyperbolic three space, is a challenging problem that has been investigated for more than a century and is still open. It is known that G is discrete if, and only if, every non-elementary two generator subgroup is. Several sufficient conditions for discreteness are also known as are some necessary conditions, though no single necessary and sufficient condition is known. There is a finite discreteness algorithm for the two generator subgroups of the isometry group of hyperbolic two-space. But the situation in three dimensions is more delicate because there are geometrically infinite groups. We present a semi-algorithm, that is, a procedure that terminates sometimes but not always. There is no standard way to find an infinite sequence of distinct elements that converges to the identity to show that a group is not discrete. Our semi-algorithm either produces such an infinite sequence or finds a finite sequence that produces a right angled hexagon in hyperbolic three-space which has a special property that is a generalization of the notion of convexity. We call it a canonical hexagon. If the group is discrete, free and geometrically finite, it always has an essentially unique canonical hexagon which the procedure finds in a finite number of steps.