Higher identities for the ternary commutator

Abstract: We use computer algebra to study polynomial identities for the trilinear operation [a,b,c] = abc - acb - bac + bca + cab - cba in the free associative algebra. It is known that [a,b,c] satisfies the alternating property in degree 3, no new identities in degree 5, a multilinear identity in degree 7 which alternates in 6 arguments, and no new identities in degree 9. We use the representation theory of the symmetric group to demonstrate the existence of new identities in degree 11. The only irreducible representations of dimension < 400 with new identities correspond to partitions 2^5 1 and 2^4 1^3 and have dimensions 132 and 165. We construct an explicit new multilinear identity for partition 2^5 1 and we demonstrate the existence of a new non-multilinear identity in which the underlying variables are permutations of a^2 b^2 c^2 d^2 e^2 f.