Task Entropy Discrete Coding

• Task Entropy Discrete Coding is a fixed-length coding framework where ambiguity is intrinsic and measured by the ρ-th moment of the partition sizes.
• It leverages Rényi entropy of order 1/(1+ρ) to set sharp performance thresholds, contrasting with Shannon entropy’s role in traditional coding.
• The unified approach connects task partitioning with related problems like guessing and source coding, highlighting design heuristics and mismatch penalties.

Task entropy discrete coding denotes a class of fixed-length coding and partitioning problems in which a random task drawn from a finite set is described using a limited number of labels or bits, and all tasks sharing that description must be performed. The central object is therefore not decoding error or expected code length, but the multiplicity or ambiguity induced by the description, typically measured by a $\rho$-th moment such as $\mathbb E[|f^{-1}(f(X))|^\rho]$ or $\mathbb E[A(X)^\rho]$. Across task encoding, task partitioning, and their asymptotic extensions, the governing entropy is Rényi entropy of order $\alpha=\frac{1}{1+\rho}$; in the limit $\rho\to 0$, Shannon entropy reappears (Bunte et al., 2013, Bunte et al., 2014, Kumar et al., 2019).

1. Task encoding as fixed-length coding with ambiguity

The basic model starts with a finite task set $\mathcal X$ and a random task $X\sim P$. An encoder uses a fixed description alphabet, either as a map

$f:\mathcal X\to\{1,\dots,M\},$

or, equivalently, as a partition of $\mathcal X$ into $M$ cells. If task $\mathbb E[|f^{-1}(f(X))|^\rho]$0 is assigned description $\mathbb E[|f^{-1}(f(X))|^\rho]$1, then all tasks in the inverse-image set

$\mathbb E[|f^{-1}(f(X))|^\rho]$2

must be performed. The induced multiplicity is therefore

$\mathbb E[|f^{-1}(f(X))|^\rho]$3

or, in the partition formulation,

$\mathbb E[|f^{-1}(f(X))|^\rho]$4

The performance criterion is the $\mathbb E[|f^{-1}(f(X))|^\rho]$5-th moment

$\mathbb E[|f^{-1}(f(X))|^\rho]$6

with $\mathbb E[|f^{-1}(f(X))|^\rho]$7 in the original task-encoding formulation and $\mathbb E[|f^{-1}(f(X))|^\rho]$8 in the later unified treatment (Bunte et al., 2013, Bunte et al., 2014, Kumar et al., 2019).

This formulation differs sharply from ordinary almost-lossless source coding. The decoder does not output a unique symbol, and ambiguity is not treated as an error event. Instead, ambiguity is intrinsic to the model and is optimized directly. The ideal value of the multiplicity is $\mathbb E[|f^{-1}(f(X))|^\rho]$9, meaning that no superfluous tasks are performed, but when the number of descriptions is smaller than $\mathbb E[A(X)^\rho]$0, several tasks must share a label (Bunte et al., 2014).

A key combinatorial identity makes the coding structure explicit. If $\mathbb E[A(X)^\rho]$1 is the size of the partition cell containing $\mathbb E[A(X)^\rho]$2, then

$\mathbb E[A(X)^\rho]$3

In the task-partitioning notation, if a partition has size $\mathbb E[A(X)^\rho]$4 and block-size function $\mathbb E[A(X)^\rho]$5, then

$\mathbb E[A(X)^\rho]$6

This is the analog of Kraft’s inequality for partitions of finite sets and is the bridge between task encoding and discrete coding with a finite description alphabet (Bunte et al., 2013, Kumar et al., 2019).

2. Rényi entropy as the operational quantity

The entropy measure governing the multiplicity moment is Rényi entropy. For order $\mathbb E[A(X)^\rho]$7,

$\mathbb E[A(X)^\rho]$8

In task encoding the relevant order is

$\mathbb E[A(X)^\rho]$9

Thus $\alpha=\frac{1}{1+\rho}$0 corresponds to $\alpha=\frac{1}{1+\rho}$1, while the unified formulation also allows $\alpha=\frac{1}{1+\rho}$2, giving $\alpha=\frac{1}{1+\rho}$3 (Bunte et al., 2013, Bunte et al., 2014, Kumar et al., 2019).

The one-shot converse for fixed-length task encoding states that for every positive integer $\alpha=\frac{1}{1+\rho}$4 and every encoder $\alpha=\frac{1}{1+\rho}$5,

$\alpha=\frac{1}{1+\rho}$6

There is also a matching achievability statement up to finite-size slack: for all integers $\alpha=\frac{1}{1+\rho}$7, there exists an encoder $\alpha=\frac{1}{1+\rho}$8 such that

$\alpha=\frac{1}{1+\rho}$9

So, up to the explicit slack term $\rho\to 0$0, the optimal moment behaves like

$\rho\to 0$1

The same form appears for task partitioning with $\rho\to 0$2 replaced by $\rho\to 0$3 and $\rho\to 0$4 replaced by $\rho\to 0$5 (Bunte et al., 2013, Bunte et al., 2014, Kumar et al., 2019).

The operational interpretation is that Rényi entropy, not Shannon entropy, is the correct quantity when the coding criterion is a moment of ambiguity/list size. A useful design heuristic also emerges from the converse: equality would require block sizes roughly proportional to

$\rho\to 0$6

or, in partition form,

$\rho\to 0$7

This means more probable tasks should be placed in smaller bins or smaller ambiguity sets, but with Rényi-order scaling rather than the scaling associated with ordinary source coding (Bunte et al., 2014, Kumar et al., 2019).

3. Asymptotic thresholds and entropy rates

For a general source $\rho\to 0$8 over a finite alphabet, the blocklength-$\rho\to 0$9 encoder

$\mathcal X$0

jointly describes $\mathcal X$1, and the decoder performs all sequences in the cell $\mathcal X$2. The asymptotic criterion remains

$\mathcal X$3

The threshold is the Rényi entropy rate of order $\mathcal X$4, when it exists: $\mathcal X$5 The coding theorem is sharp. If

$\mathcal X$6

then there exist encoders $\mathcal X$7 such that

$\mathcal X$8

If

$\mathcal X$9

then for every coding sequence,

$X\sim P$0

For IID sources, $X\sim P$1, so the threshold reduces to the single-letter Rényi entropy (Bunte et al., 2013, Bunte et al., 2014).

The task-partitioning formulation gives the same threshold statement with partitions of $X\sim P$2 of size at most $X\sim P$3. If $X\sim P$4, there exists a sequence of partitions such that

$X\sim P$5

If $X\sim P$6, then for any such partitions,

$X\sim P$7

Operationally, $X\sim P$8 is therefore the critical fixed rate for asymptotically vanishing task ambiguity under a moment criterion (Kumar et al., 2019).

A recurring misconception is that the critical rate should be Shannon entropy because the descriptions are discrete. The asymptotic theorem shows otherwise: Shannon entropy is recovered only in the limiting regime $X\sim P$9, whereas for $f:\mathcal X\to\{1,\dots,M\},$0 the correct threshold is the Rényi entropy rate of order $f:\mathcal X\to\{1,\dots,M\},$1 (Bunte et al., 2014, Kumar et al., 2019).

4. Unified optimization framework and relation to coding, guessing, and partitioning

A later synthesis places task partitioning inside a single variational problem. The abstraction is: choose a nonnegative function $f:\mathcal X\to\{1,\dots,M\},$2 under the linear constraint

$f:\mathcal X\to\{1,\dots,M\},$3

and minimize

$f:\mathcal X\to\{1,\dots,M\},$4

The main theorem is

$f:\mathcal X\to\{1,\dots,M\},$5

with optimizer

$f:\mathcal X\to\{1,\dots,M\},$6

As $f:\mathcal X\to\{1,\dots,M\},$7, the criterion becomes logarithmic and Shannon entropy appears: $f:\mathcal X\to\{1,\dots,M\},$8 achieved by $f:\mathcal X\to\{1,\dots,M\},$9 (Kumar et al., 2019).

This theorem unifies several discrete problems by different choices of $\mathcal X$0.

Problem	Choice of $\mathcal X$1	Constraint
Source coding	$\mathcal X$2	$\mathcal X$3
Guessing	$\mathcal X$4	$\mathcal X$5
Memoryless guessing	$\mathcal X$6	$\mathcal X$7
Task partitioning	$\mathcal X$8	$\mathcal X$9

For task partitioning, the choice

$M$0

and the partition identity

$M$1

yield the one-shot converse

$M$2

This gives task partitioning a clean coding interpretation: labels are compressed descriptions, and $M$3 is the ambiguity or list size induced by the description (Kumar et al., 2019).

The same framework also explains why source coding, guessing, and task encoding are mathematically parallel. Only the operational meaning of $M$4 changes: inverse code mass, inverse guessing rank, randomized guessing distribution, or inverse ambiguity size (Kumar et al., 2019).

5. Side information, mismatch, and distributed task encoding

The single-source model extends in two important directions: side information and mismatch. With side information $M$5 available to both encoder and performer, the encoder becomes

$M$6

and the performed set is

$M$7

The moment criterion is

$M$8

and the governing quantity is Arimoto’s conditional Rényi entropy of order $M$9: $\mathbb E[|f^{-1}(f(X))|^\rho]$00 The one-shot bounds mirror the unconditional case, with $\mathbb E[|f^{-1}(f(X))|^\rho]$01 replaced by $\mathbb E[|f^{-1}(f(X))|^\rho]$02, and the corresponding asymptotic threshold is the conditional Rényi entropy rate (Bunte et al., 2014).

Mismatch introduces a second family of operational quantities. In task encoding designed for the wrong law $\mathbb E[|f^{-1}(f(X))|^\rho]$03, the excess performance is measured by a divergence identified by Sundaresan. In the task-encoding papers this appears as $\mathbb E[|f^{-1}(f(X))|^\rho]$04, and the mismatched achievability bound becomes

$\mathbb E[|f^{-1}(f(X))|^\rho]$05

In the IID case, the penalty is additive: $\mathbb E[|f^{-1}(f(X))|^\rho]$06 In the unified task-partitioning framework, the corresponding penalty is written as Sundaresan’s divergence $\mathbb E[|f^{-1}(f(X))|^\rho]$07, and for a fixed partition $\mathbb E[|f^{-1}(f(X))|^\rho]$08 with partition function $\mathbb E[|f^{-1}(f(X))|^\rho]$09,

$\mathbb E[|f^{-1}(f(X))|^\rho]$10

Thus mismatch contributes exactly an excess ambiguity term on top of the intrinsic task entropy and the rate offset (Bunte et al., 2014, Kumar et al., 2019).

The distributed version replaces a single encoder by two separate encoders

$\mathbb E[|f^{-1}(f(X))|^\rho]$11

with decoder list

$\mathbb E[|f^{-1}(f(X))|^\rho]$12

The achievable region is characterized by Rényi marginals plus a dependence penalty $\mathbb E[|f^{-1}(f(X))|^\rho]$13: $\mathbb E[|f^{-1}(f(X))|^\rho]$14

$\mathbb E[|f^{-1}(f(X))|^\rho]$15

For IID sources this becomes single-letter. This is a notable point of contrast with Slepian–Wolf coding: the individual-rate constraints depend on the marginals, and correlation enters through the sum-rate penalty $\mathbb E[|f^{-1}(f(X))|^\rho]$16, not through conditional entropies (Bracher et al., 2017).

6. Extensions, interpretation, and conceptual boundaries

The task-encoding program also includes two IID extensions solved in the same Rényi framework. The first is a rate-distortion-flavored model in which the decoder outputs a subset $\mathbb E[|f^{-1}(f(X))|^\rho]$17 and every source sequence must have at least one reproduction in the subset within distortion $\mathbb E[|f^{-1}(f(X))|^\rho]$18. The relevant threshold is

$\mathbb E[|f^{-1}(f(X))|^\rho]$19

with the same dichotomy: if $\mathbb E[|f^{-1}(f(X))|^\rho]$20, the multiplicity moment can be driven to $\mathbb E[|f^{-1}(f(X))|^\rho]$21; if $\mathbb E[|f^{-1}(f(X))|^\rho]$22, it diverges. At $\mathbb E[|f^{-1}(f(X))|^\rho]$23, this reduces to the lossless threshold

$\mathbb E[|f^{-1}(f(X))|^\rho]$24

The second extension assigns nonnegative costs $\mathbb E[|f^{-1}(f(X))|^\rho]$25 to tasks and, for $\mathbb E[|f^{-1}(f(X))|^\rho]$26, yields threshold $\mathbb E[|f^{-1}(f(X))|^\rho]$27 in the IID case (Bunte et al., 2014).

These developments sharpen the conceptual meaning of “task entropy.” In this literature, task entropy is not a separate primitive but the operational quantity that governs fixed-length coding with ambiguity. When the performance criterion is expected code length or vanishing error, the operative quantity is Shannon entropy. When the criterion is a moment of multiplicity, list size, or ambiguity under a fixed description alphabet, the operative quantity is Rényi entropy of order $\mathbb E[|f^{-1}(f(X))|^\rho]$28 (Bunte et al., 2014, Kumar et al., 2019).

A second misconception is to treat task encoding as a form of ordinary list decoding. The operational structure is different. The performer executes all tasks in the induced bin, so the cost is literally the number of performed tasks, not a decoding-list size used only for post-processing. This is why the central metric is

$\mathbb E[|f^{-1}(f(X))|^\rho]$29

rather than error probability or expected length (Bunte et al., 2014).

The broader significance of the unified line of work is that source coding, guessing, memoryless guessing, and task partitioning all reduce to the same constrained moment minimization, with Rényi entropy and, in the limit, Shannon entropy as exact solutions. Within that class, task partitioning supplies a particularly direct operational interpretation: under a finite description alphabet, the minimum achievable moment of task ambiguity is characterized by Rényi entropy, and under mismatch the penalty is quantified by Sundaresan’s divergence (Kumar et al., 2019).