Designing architectures and local losses for truncated credit assignment

Determine architectures and per-layer local loss functions that effectively guide truncated credit assignment in deep neural networks, so that global backpropagation can be replaced by layer-wise or truncated variants without degrading performance.

Background

The thesis discusses alternatives to global backpropagation for credit assignment, noting recent success with truncated layer-wise backpropagation approaches. However, achieving comparable performance with truncated methods requires appropriate architectural choices and local objectives that steer learning effectively.

This open question targets the co-design of network topology and local losses to enable scalable training without full global error propagation, aligning with the thesis’ broader theme of locality and hardware-awareness.

References

Nevertheless, designing the right architecture and local loss functions to guide the truncated credit assignment is still an open question.

— Analog Alchemy: Neural Computation with In-Memory Inference, Learning and Routing  (2412.20848 - Demirag, 2024) in Introduction, footnote 5

First, while numerical ill conditioning spans both CNNs and Transformers and the stable solution rescues the solver on ResNet-50, applying the complete layer wise framework to CNNs remains an open challenge: module level compensation does not yet stably outperform the block level baseline, which we attribute to the error propagation structure of convolutional layers.

— QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncation  (2608.14149 - Lee et al., 14 Aug 2026) in Section Conclusion and Future Work