What does mathoverflow tell us about the production of mathematics?

Abstract: The highest level of mathematics research is traditionally seen as a solitary activity. Yet new innovations by mathematicians themselves are starting to harness the power of social computation to create new modes of mathematical production. We study the effectiveness of one such system, and make proposals for enhancement, drawing on AI and computer based mathematics. We analyse the content of a sample of questions and responses in the community question answering system for research mathematicians, math-overflow. We find that mathoverflow is very effective, with 90% of our sample of questions answered completely or in part. A typical response is an informal dialogue, allowing error and speculation, rather than rigorous mathematical argument: 37% of our sample discussions acknowledged error. Responses typically present information known to the respondent, and readily checked by other users: thus the effectiveness of mathoverflow comes from information sharing. We conclude that extending and the power and reach of mathoverflow through a combination of people and machines raises new challenges for artificial intelligence and computational mathematics, in particular how to handle error, analogy and informal reasoning.