Cause of ESP32-S3 two-core execution failures

Determine the cause of corrupted activations, incorrect transcripts, and intermittent crashes produced by the two-core Oído inference schedule on the ESP32-S3 at 240 MHz, in order to establish a correct and potentially faster parallel execution mode.

Background

Oído’s two-core schedule is functionally correct in the emulator and was expected to approach real-time performance, but the corresponding implementation on physical ESP32-S3 hardware produces random activation corruption, incorrect transcripts, and occasional IllegalInstruction crashes at 240 MHz. The failures occur primarily in certain K=704 matrix products, especially within the feed-forward blocks of the Conformer layers.

The authors report that lowering the clock to 160 MHz eliminates the errors, while changing memory placement, cache configuration, synchronization, interrupt handling, instruction placement, and workload partitioning does not. The underlying mechanism therefore remains unresolved, and identifying it is necessary for exploiting the second core to reduce recognition latency.