Evaluate privacy leakage under additional sparsity schemes and deployed codecs

Investigate privacy leakage and communication rates for structured sparsity schemes beyond global top-k sparsification, including channel pruning, block sparsity, N:M sparsity, transformer token pruning, and deployed entropy codecs, and measure codec and end-to-end wire rates rather than relying solely on analytical upper bounds.

Background

The study evaluates sparse collaborative inference primarily with global top-k sparsification and estimates transmission cost analytically using an upper bound based on the entropy of the position mask and the precision of retained values. The authors note that several practically relevant settings are not covered, including alternative structured sparsity patterns and transformer token pruning, whose transmitted positions may themselves form a privacy-sensitive channel.

Because activation positions can be correlated, actual compression may be more efficient than the analytical rate bound. Consequently, privacy leakage per transmitted bit may differ from the paper’s estimates when real entropy codecs and complete end-to-end communication pipelines are used. The open problem is therefore to extend the privacy analysis and rate measurement to these deployment-relevant schemes and codecs.