Information-Theoretical Quantifiers

• Information-Theoretical Quantifiers are rigorous mathematical constructs that extend classical Boolean quantification by integrating entropy, diversity indices, and softmax operators.
• They use a continuum of p-means—from harmonic to arithmetic to geometric—to smoothly generalize existential and universal quantifiers for graded logical aggregation.
• These quantifiers enable practical applications in machine learning and statistical physics by leveraging additive and multiplicative semantics through softmax operators and Napierian duality.

Information-theoretical quantifiers are rigorous mathematical constructs that generalize the concept of quantification in logic, enabling a seamless connection between logic, probability, and statistical information measures. They provide a unifying perspective that subsumes traditional Boolean quantification as well as well-known information-theoretic functionals such as entropy, diversity indices, and softmax–type operators. Recent developments have positioned these quantifiers at the core of quantitative logic, statistical physics, and machine learning, offering robust tools for aggregating and reasoning about numerical predicates over structured domains.

1. Quantitative Logic: Predicate Connectives and Quantifiers

Quantitative predicate logic extends traditional first-order logic by assigning to each predicate a value in a commutative quantale, typically $[0, \infty]$. This algebraic structure supports three generations of logical connectives:

• Non-linear connectives: Analogues of classical ∧ (“and”) and ∨ (“or”), realized as min and max on the reals.
• Linear additive connectives: Operations such as log-sum-exp (softplus), capturing normalized summation in the additive real semiring.
• Linear multiplicative connectives: Operations leveraging the multiplicative monoid structure, including products and their inverses, forming a ∗-autonomous quantale.

Quantification in this setting is interpreted via generalized means rather than strict set-theoretic existence or universality. Existential and universal quantifiers are lifted to soft or “graded” means and harmonic means, establishing a direct route to information-theoretic aggregation.

2. p-Means as Generalized Quantifiers

Quantification over a domain $I$ is executed not by simple minima or maxima, but by a continuum of p–means:

• p–sum: For $p \neq 0$, $$\mathrm{sum}_p = \left( \sum_{i \in I} a_i^p \right)^{1/p}$$.
• p–mean: $$\mathrm{mean}_p = \left( \frac{1}{|I|} \sum_{i \in I} a_i^p \right)^{1/p}$$.
• Soft quantifiers:
  • Existential (∃): $p$–mean for $p \to +\infty$ approximates max; for $p=1$ yields arithmetic mean.
  • Universal (∀): Harmonic $p$–mean for $p \to -\infty$ approximates min.

This spectrum interpolates continuously between min, harmonic mean, geometric mean ($p=0$), arithmetic mean ($p=1$), and max, covering classical quantifiers as limiting cases. These soft quantifiers are central in smoothing logical aggregation, as in the softmax operator.

$p$-Value	Operation	Quantifier Analogue
$p = -\infty$	min	∀ (strict universal)
$p \in (-\infty,0)$	harmonic mean	soft universal
$p = 0$	geometric mean	soft average (neutral)
$p \in (0, +\infty)$	arithmetic mean	soft existential
$p = +\infty$	max	∃ (strict existential)

3. Softmax, Argmax, and Quantifier Semantics

The softmax operator is the canonical quantitative semantics for argmax within this framework. For $f: X \to [0,\infty]$, softmax is given as:

$$(\mathrm{softmax}~f)(x^*) = \frac{f(x^*)}{\sum_{x \in X} f(x)}$$

or in the general p–mean setting, normalization is achieved by dividing by the $p$–mean. In the limit $p \to \infty$, the softmax recovers the indicator function for the true argmax:

$$\lfloor (\mathrm{softmax}[f]~p \to \infty)(x^*) \rfloor = (\mathrm{argmax}~f)(x^*)$$

This formalism allows interpolation between “soft” statistical softmax and “hard” Boolean argmax by tuning $p$. It further demonstrates the suitability of means and p–means as graded quantifiers, directly generalizing existential and universal quantification in a rigorous, quantitative sense.

4. Additive vs. Multiplicative Semantics and Napierian Duality

A foundational insight is that information quantities exhibit either additive or multiplicative structure, often determined by their behavior under transformations (e.g., logarithm or exponential).

• Multiplicative semantics operate in $([0, \infty], \times, 1)$, directly aggregating probabilities or likelihoods.
• Additive semantics leverage $([-\infty, \infty], +, 0)$, typically after a logarithmic transformation.

Napierian duality ($-\log \dashv 1/\exp$) mediates between these worlds: passing from multiplicative to additive quantities corresponds to applying $-\log$, and vice versa. For a multiplicative predicate $\varphi$, the additive quantifier is:

$$\#\llbracket \varphi \rrbracket_{\mathrm{add}} = -\log (\# \llbracket 1/\exp(\varphi) \rrbracket_{\mathrm{mult}})$$

This duality is exemplified in the relationship between Rényi entropy (additive, log-domain) and Hill numbers (multiplicative, exponential-domain):

• Rényi entropy of order $p$:

$$H_p(\varphi) = \frac{1}{1-p} \log \int_{i \in I} \varphi(i)^p \, di$$

• Hill number (p–diversity):

$$D_p(\varphi) \equiv \#\llbracket (p/(1-p)) \cdot (\forall^{(p)}_{i \in I}\varphi(i)^*) \rrbracket$$

Thus, both are seen as additive and multiplicative semantics of the same underlying logical quantifier formula.

5. Categorification and the Limits of Enriched Hyperdoctrines

Attempts at providing a categorical semantics for soft quantifiers (enriched hyperdoctrines) face fundamental obstacles:

• When entailment is defined via quantitative universal quantifiers, basic order-theoretic properties such as reflexivity or transitivity can fail unless the domain is a probability space.
• Indexed monoidal categories enriched over quantales of measurable functions with quantification as pushforward (Radon–Nikodym) lack the lax monoidal structure needed for quantifier adjunction properties.

Integration and “almost-everywhere” equivalence introduce metatheoretic subtleties (non-idempotency of identity/cut rules) not captured by classical hyperdoctrines. This suggests that although categorical semantics provide strong intuition, conventional Lawvere-style hyperdoctrines do not fully capture the semantics of quantitative logic with soft quantifiers.

6. Significance for Information Theory, Machine Learning, and Logic

Information–theoretical quantifiers underpin diverse applications by unifying logical and probabilistic aggregation:

• Classical information quantities such as entropy, diversity, and statistical complexity are directly expressible as quantitative quantifiers.
• In machine learning, softmax and related quantifiers enable differentiable relaxations of hard assignment rules and support gradient-based optimization in large structured spaces.
• The continuous spectrum of p–means encodes tradeoffs between averaging, selection, and extremizing, allowing adaptation to diverse modeling contexts from statistical physics (e.g., free energy minimization) to compositional semantics in natural language processing.
• The explicit separation of additive and multiplicative semantics clarifies the dual behavior of statistical and information-theoretic quantities, offering a precise framework for the translation between frequency and information domains.

In summary, information-theoretical quantifiers provide a foundational generalization of logical quantification, embedding classical, soft, and information–measures within a unified algebraic and analytic architecture. This framework not only recasts standard entropies and diversity indices as quantifier semantics but also exposes deep connections between logic, probability, and measure theory, driving new formulations and applications in contemporary theoretical and applied research (Capucci, 7 Jun 2024).