A family of non-Cayley cores based on vertex-transitive or strongly regular self-complementary graphs

Abstract: Given a finite simple graph $\Gamma$ on $n$ vertices its complementary prism is the graph $\Gamma\bar{\Gamma}$ that is obtained from $\Gamma$ and its complement $\bar{\Gamma}$ by adding a perfect matching, where each its edge connects two copies of the same vertex in $\Gamma$ and $\bar{\Gamma}$. It generalizes the Petersen graph, which is obtained if $\Gamma$ is the pentagon. The automorphism group of $\Gamma\bar{\Gamma}$ is described for arbitrary graph $\Gamma$. In particular, it is shown that the ratio between the cardinalities of the automorphism groups of $\Gamma\bar{\Gamma}$ and $\Gamma$ can attain only values $1$, $2$, $4$, and $12$. It is shown that the Cheeger number of $\Gamma\bar{\Gamma}$ equals either 1 or $1-\frac{1}{n}$, and the two corresponding classes of graphs are fully determined. It is proved that $\Gamma\bar{\Gamma}$ is vertex-transitive if and only if $\Gamma$ is vertex-transitive and self-complementary. In this case the complementary prism is Hamiltonian-connected whenever $n>5$, and is not a Cayley graph whenever $n>1$. The main results involve endomorphisms of graph $\Gamma\bar{\Gamma}$. It is shown that $\Gamma\bar{\Gamma}$ is a core, i.e. all its endomorphisms are automorphisms, whenever $\Gamma$ is strongly regular and self-complementary. The same conclusion is obtained for many vertex-transitive self-complementary graphs. In particular, it is shown that if there exists a vertex-transitive self-complementary graph $\Gamma$ such that $\Gamma\bar{\Gamma}$ is not a core, then $\Gamma$ is neither a core nor its core is a complete graph.