Quantifying imperfect cognition via achieved information gain

Abstract: Cognition, information processing in form of inference, communication, and memorization, is the central activity of any intelligence. Its physical realization in a brain, computer, or in any other intelligent system requires resources like time, energy, memory, bandwidth, money, and others. Due to limited resources, many real world intelligent systems perform only imperfect cognition. To understand the trade-off between accuracy and resource investments in existing systems, e.g. in biology, as well as for the resource-aware optimal design of information processing systems, like computer algorithms and artificial neural networks, a quantification of information obtained in an imperfect cognitive operation is desirable. To this end, we propose the concept of the achieved information gain (AIG) of a belief update, which is given by the amount of information obtained by updating from the initial state of knowledge to the ideal state, minus the amount that a change from the imperfect to the ideal state would yield. AIG has many desirable properties for quantifying imperfect cognition. The ratio of achieved to ideally obtainable information measures cognitive fidelity and that of AIG to the necessary cognitive effort measures cognitive efficiency. We provide an axiomatic derivation of AIG, relate it to other information measures, illustrate its application to common scenarios of posterior inaccuracies, and discuss the implication of cognitive efficiency for sustainable resource allocation in computational inference.