Designing architectures and local losses for truncated credit assignment

Determine architectures and per-layer local loss functions that effectively guide truncated credit assignment in deep neural networks, so that global backpropagation can be replaced by layer-wise or truncated variants without degrading performance.

Background

The thesis discusses alternatives to global backpropagation for credit assignment, noting recent success with truncated layer-wise backpropagation approaches. However, achieving comparable performance with truncated methods requires appropriate architectural choices and local objectives that steer learning effectively.

This open question targets the co-design of network topology and local losses to enable scalable training without full global error propagation, aligning with the thesis’ broader theme of locality and hardware-awareness.

References

Nevertheless, designing the right architecture and local loss functions to guide the truncated credit assignment is still an open question.

— Analog Alchemy: Neural Computation with In-Memory Inference, Learning and Routing  (2412.20848 - Demirag, 2024) in Introduction, footnote 5

First, while numerical ill conditioning spans both CNNs and Transformers and the stable solution rescues the solver on ResNet-50, applying the complete layer wise framework to CNNs remains an open challenge: module level compensation does not yet stably outperform the block level baseline, which we attribute to the error propagation structure of convolutional layers.

— QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncation  (2608.14149 - Lee et al., 14 Aug 2026) in Section Conclusion and Future Work

however, how local objectives should be organized to encourage feature learning remains an open question.

— A Study of Hidden-State Optimization Order in Predictive Coding Networks  (2609.00686 - Li et al., 1 Sep 2026) in Introduction

We conjecture that local optimization makes these shortcuts harder to learn because they require an earlier step to overwrite its recurrent state with the inputs needed by a later step.

— Thinking with Looped Flows  (2609.11801 - Suleymanzade et al., 10 Sep 2026) in Appendix, Section “Possibility of shortcuts,” paragraph “Training”