Binary Autoencoder Fundamentals
- Binary Autoencoder is an unsupervised neural model that constrains its hidden layer to binary values (0/1 or -1/1), facilitating compact storage and fast Hamming-space retrieval.
- The design encompasses various formulations—such as hashing, pairwise-correlation, and sparse coding—that leverage discrete latent representations for improved image retrieval, continual learning, and optimization.
- Optimization challenges due to non-differentiable binary constraints are addressed via techniques like auxiliary coordinates, surrogate gradients, and biconvex alternating minimization for effective reconstruction and performance.
to=arxiv_search.12query12^ 天天爱彩票怎么json {"12query12 OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12", "12max_results12 12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12max_results12} to=arxiv_search.12query12^ 大发快三开奖结果ായjson {"12query12 with binary autoencoders12\12 OR 12\12 Binary Autoencoding with Pairwise Correlations12\12 OR 12\12 autoencoder with random binary weights12\12 OR 12\12 A Binary Latent Autoencoder for Generative Replay Continual Learning12\12 OR 12\12 of Binary Autoencoders for QUBO-Based Optimization Problems12\12 OR 12\12 Sparse Coding for Interpretability12\12 OR 12\12 Autoencoder for Mechanistic Interpretability of LLMs12\12 OR 12\12 Binary Autoencoder-Based Codes with Progressive Training12\12 "12max_results12 12max_results12query12} Binary autoencoder denotes an autoencoder in which the hidden or code representation is constrained to be binary, typically PRESERVED_PLACEHOLDER_12query12^ or PRESERVED_PLACEHOLDER_12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12. In the hashing formulation of "Hashing with binary autoencoders" (&&&12query12&&&), the encoder maps an image PRESERVED_PLACEHOLDER_12max_results12^ to a short binary code and the decoder reconstructs the image from that code; related work uses the same term for binary vector compression, binary latent replay, binary sparse coding, binary feature extraction, and binary channel coding (&&&12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12&&&, &&&12query12&&&, &&&12 OR \12&&&). Across these settings, the defining property is not a single architecture but the presence of discrete latent states during representation learning, which makes the model simultaneously attractive for compact storage, Hamming-space search, discrete optimization, and sparse feature discovery, while also making training non-smooth and combinatorial.
12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12. Concept and design space
In the canonical hashing setting, a binary autoencoder consists of an encoder PRESERVED_PLACEHOLDER_12query12^ that maps PRESERVED_PLACEHOLDER_12\12^ and a decoder PRESERVED_PLACEHOLDER_12 OR \12^ that reconstructs PRESERVED_PLACEHOLDER_12 OR \12^ from PRESERVED_PLACEHOLDER_12 OR \12, with the code layer constrained to be binary. For hashing, the encoder is thresholded linear, PRESERVED_PLACEHOLDER_12 OR \12, where PRESERVED_PLACEHOLDER_12 OR \12^ if PRESERVED_PLACEHOLDER_12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12query12^ and PRESERVED_PLACEHOLDER_12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12^ otherwise, and the decoder is often linear, PRESERVED_PLACEHOLDER_12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12max_results12^ (&&&12query12&&&). Other formulations retain the encoder–decoder template but move the binary restriction to different objects: visible and hidden bits, sparse latent activations, precomputed replay codes, or transmitted codewords.
The literature therefore contains several non-identical but related meanings of binary autoencoding. Some works emphasize strict binary bottlenecks for reconstructing real-valued inputs; some study binary vector data with worst-case reconstruction guarantees; some analyze binary activations and even binary random weights; and some adopt binary latent variables because the downstream task itself is discrete, as in QUBO optimization or digital communication (&&&12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12&&&, &&&12max_results12&&&, &&&12\12&&&).
12max_results12. Canonical objectives and mathematical formulations
The best-known objective is reconstruction under an explicit binary bottleneck. In the hashing model, the binary autoencoder minimizes
PRESERVED_PLACEHOLDER_12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12 OR \12^
with PRESERVED_PLACEHOLDER_12max_results12query12. This is presented as a wrapper method: it optimizes the hash function itself under binary constraints rather than solving a relaxed continuous problem and binarizing afterward (&&&12query12&&&). The same work also introduces an entropy-based code-utilization statistic. If PRESERVED_PLACEHOLDER_12max_results12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12^ is the fraction of data points mapped to code PRESERVED_PLACEHOLDER_12max_results12max_results12, then
PRESERVED_PLACEHOLDER_12max_results12query12^
so the entropy of the empirical code distribution is interpreted as an effective number of bits.
A different formalization appears in "Optimal Binary Autoencoding with Pairwise Correlations" (&&&12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12&&&). There, the data are binary vectors PRESERVED_PLACEHOLDER_12max_results12\12^ or randomized versions in PRESERVED_PLACEHOLDER_12max_results12 OR \12, the encodings are PRESERVED_PLACEHOLDER_12max_results12 OR \12^ or PRESERVED_PLACEHOLDER_12max_results12 OR \12, and the only retained statistics are the pairwise correlations
PRESERVED_PLACEHOLDER_12max_results12 OR \12^
The problem is posed as a minimax reconstruction game under bitwise cross-entropy. For cross-entropy, the minimax value reduces to independent convex problems through the slack function
PRESERVED_PLACEHOLDER_12max_results12 OR \12^
and the optimal decoder emerges as a single layer of logistic neurons,
PRESERVED_PLACEHOLDER_12query12query12^
A third formulation, intended more as analysis than training, uses a thresholded random binary projection with binary activations and binary random weights: PRESERVED_PLACEHOLDER_12query12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12^ where PRESERVED_PLACEHOLDER_12query12max_results12, PRESERVED_PLACEHOLDER_12query12query12, and PRESERVED_PLACEHOLDER_12query12\12. Reconstruction error is the normalized Hamming distance
PRESERVED_PLACEHOLDER_12query12 OR \12^
This model is used to study sparsity, similarity preservation, and mutual information rather than gradient-based learning (&&&12max_results12&&&).
12query12. Optimization under discrete constraints
Binary autoencoders are difficult to optimize because the binary bottleneck is discontinuous and non-smooth. In the hashing model, gradients with respect to the encoder parameters are zero almost everywhere, and the resulting problem is described as NP-hard / combinatorial because of the binary constraints (&&&12query12&&&). The central algorithmic response in that work is the method of auxiliary coordinates (MAC), which introduces auxiliary codes PRESERVED_PLACEHOLDER_12query12 OR \12^ and rewrites the problem as
PRESERVED_PLACEHOLDER_12query12 OR \12^
A quadratic-penalty relaxation then yields
PRESERVED_PLACEHOLDER_12query12 OR \12^
with PRESERVED_PLACEHOLDER_12query12 OR \12^ increased over iterations. The optimization alternates between a decoder step, which is ordinary least squares for a linear decoder, an encoder step, implemented as PRESERVED_PLACEHOLDER_12\12query12^ independent linear SVMs, and per-image code updates that can use exact enumeration for small PRESERVED_PLACEHOLDER_12\12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12, groupwise alternating optimization, relaxed QP initialization, and greedy refinement. The paper reports that the algorithm usually converges in about 12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12query12–12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12 OR \12^ iterations (&&&12query12&&&).
The pairwise-correlation formulation replaces direct discrete backpropagation with biconvex alternating minimization. Given the current decoder weights PRESERVED_PLACEHOLDER_12\12max_results12, each example is encoded independently by solving the convex problem
PRESERVED_PLACEHOLDER_12\12query12^
and given the current encodings PRESERVED_PLACEHOLDER_12\12\12, each visible-bit decoder PRESERVED_PLACEHOLDER_12\12 OR \12^ is updated by convex optimization. The resulting algorithm, called Pairwise Correlation Autoencoder (PC-AE), is therefore biconvex rather than jointly convex, but each subproblem is convex (&&&12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12&&&).
Several later variants return to direct neural training and rely on surrogate gradients. Binary sparse coders use a sigmoid-based straight-through estimator for the discontinuous binarization step, with a temperature parameter of 12max_results12^ reported to improve training stability, while the QUBO-oriented bAE uses stochastic binarization PRESERVED_PLACEHOLDER_12\12 OR \12^ with PRESERVED_PLACEHOLDER_12\12 OR \12^ and a straight-through estimator because binarization is non-differentiable (&&&12 OR \12&&&, &&&12\12&&&). By contrast, "Learning Binary Autoencoder-Based Codes with Progressive Training" (&&&12 OR \12&&&) proposes a two-stage procedure consisting of continuous pretraining, direct binarization via PRESERVED_PLACEHOLDER_12\12 OR \12, and fine-tuning without gradient approximation techniques; in the PRESERVED_PLACEHOLDER_12\12 OR \12^ setting over a BSC, this is reported to recover a rotated version of the optimal Hamming code.
12\12. Theoretical interpretations: optimality, information, and geometry
The pairwise-correlation literature gives binary autoencoding a minimax interpretation. Among all algorithms that use only the encodings PRESERVED_PLACEHOLDER_12 OR \12query12^ and the correlation matrix PRESERVED_PLACEHOLDER_12 OR \12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12, the derived decoder attains the minimum possible worst-case reconstruction loss, and the logistic single-layer decoder is not postulated in advance but emerges from the minimax solution (&&&12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12&&&). The same framework also shows that if the correlation constraints are relaxed to
PRESERVED_PLACEHOLDER_12 OR \12max_results12^
then the dual decoding problem acquires an PRESERVED_PLACEHOLDER_12 OR \12query12^ penalty, so PRESERVED_PLACEHOLDER_12 OR \12\12^ correlation uncertainty corresponds exactly to PRESERVED_PLACEHOLDER_12 OR \12 OR \12^ regularization.
The random-weight theory emphasizes a different set of principles. Sparse activation of the hidden layer is reported to arise naturally in order to preserve information between layers; with a large enough hidden layer, zero reconstruction error is possible for any input just by varying the thresholds of neurons; and the model preserves the similarity of inputs at the hidden layer that is maximal for the dense hidden layer activation (&&&12max_results12&&&). The same analysis connects sparsity and mutual information to a memory-computation trade-off: the sparsity that minimizes reconstruction error is not the same as the sparsity that maximizes similarity preservation or encoder mutual information. In the main experiments of that paper, reconstruction error is typically minimized when about PRESERVED_PLACEHOLDER_12 OR \12 OR \12–PRESERVED_PLACEHOLDER_12 OR \12 OR \12^ of hidden units are active, whereas mutual information PRESERVED_PLACEHOLDER_12 OR \12 OR \12^ and mean average precision peak near PRESERVED_PLACEHOLDER_12 OR \12 OR \12.
The optimization-oriented bAE for QUBO problems interprets the binary latent space geometrically. On a fully enumerable 12 OR \12-city TSP, the learned latent Hamming geometry aligns tour distances with latent distances better than rank-based Log, rank-based Gray, or random label encodings at similar compression (&&&12\12&&&). The neighborhood distance characteristic
PRESERVED_PLACEHOLDER_12 OR \12query12^
increases with PRESERVED_PLACEHOLDER_12 OR \12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12^ for the learned bAE, which is the desired locality behavior, and the local optimum ratio
PRESERVED_PLACEHOLDER_12 OR \12max_results12^
is lowest for the bAE at 12query12.12query12max_results12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12 OR \12^, compared with 12query12.12query12\12 OR \12max_results12^ for rank-based Log, 12query12.12query12 OR \12 OR \12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12^ for rank-based Gray, and 12query12.12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12 OR \12 OR \12^ for random label encoding. This suggests that a useful binary code is not merely compact: it also induces a smoother, less trap-prone search landscape.
12 OR \12. Applications and empirical behavior
Binary autoencoders were first developed in this corpus primarily for hashing and image retrieval. The 12max_results12query12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12 OR \12^ hashing study evaluates on CIFAR, NUS-WIDE, NUS-WIDE-LITE, and SIFT-12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12M, using precision, recall, precision/recall curves, precision at fixed Hamming radii, precision for PRESERVED_PLACEHOLDER_12 OR \12query12-nearest neighbors in Hamming space, and code utilization via PRESERVED_PLACEHOLDER_12 OR \12\12^ (&&&12query12&&&). The main empirical conclusions reported there are that the resulting hash function outperforms or is competitive with state-of-the-art binary hashing methods, often beats thresholded PCA, ITQ, SH, KLSH, AGH, and SPH, and that respecting binary constraints during optimization helps: BA PRESERVED_PLACEHOLDER_12 OR \12 OR \12^ ITQ PRESERVED_PLACEHOLDER_12 OR \12 OR \12^ tPCA in reconstruction error and usually in precision.
In continual learning, BinPlay introduces a binary latent space autoencoder architecture for generative replay. Binary codes are deterministically generated from the chronological indices of training samples, using modular arithmetic over powers of primes, so that past samples can later be replayed by recomputing their codes and decoding them on demand without storing the images (&&&12query12&&&). In the class-incremental five-batch protocol PRESERVED_PLACEHOLDER_12 OR \12 OR \12, the reported final accuracies are 12 OR \12 OR \12.12max_results12^ ± 12query12.12 OR \12^ on MNIST, 12 OR \12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12.12\12^ ± 12query12.12 OR \12^ on Fashion-MNIST, and 12 OR \12query12.12query12^ ± 12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12.12\12^ on CIFAR-12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12query12. The same study reports a CIFAR-12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12query12^ ablation in which the reference configuration scores 12max_results12query12.12 OR \12^, adding processing of new samples through the autoencoder raises it to 12 OR \12\12.12query12^, and adding soft targets raises it to 12 OR \12query12.12query12^.
For black-box combinatorial optimization, the bAE+FMQA pipeline learns a compact binary latent representation of feasible solutions, trains a factorization machine surrogate on the latent codes, converts that surrogate into a QUBO, and optimizes it on an Ising machine or quantum annealer (&&&12\12&&&). On the 12 OR \12-city TSP testbed, using PRESERVED_PLACEHOLDER_12 OR \12 OR \12^ and PRESERVED_PLACEHOLDER_12 OR \12 OR \12, the bAE reconstructs feasible tours with final average reconstruction accuracy of about 12 OR \12query12%, reduces the approximation ratio PRESERVED_PLACEHOLDER_12 OR \12query12^ fastest, and maintains feasible-sample probability PRESERVED_PLACEHOLDER_12 OR \12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12^ at 12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12.12query12^ in all trials. This is presented as evidence that latent geometry matters under limited black-box evaluation budgets.
Interpretability-oriented work gives a more mixed empirical picture. Binary sparse autoencoders and binary transcoders trained on SmolLM12max_results12-12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12query12 OR \12M and SmolLM12max_results12-12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12.12 OR \12B improve unweighted auto-interpretability scores and monosemanticity, but they also increase reconstruction error, explain less variance than continuous sparse coders, and create many ultra-high-frequency uninterpretable features; after frequency adjustment, continuous sparse coders are reported to be slightly better (&&&12 OR \12&&&). A different BAE for LLM hidden states discretizes activations to 12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12-bit and minimizes minibatch entropy plus a covariance penalty; on Llama 12query12.12max_results12 OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12B layer 12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12, the paper reports 12 OR \12\12 OR \12\12^ activated features and 12query12 OR \12 OR \12max_results12^ interpretable features, exceeding the compared baselines in feature count, while also using the same binary representation to characterize layer bandwidth and an information-reduction view of in-context learning (&&&12 OR \12&&&).
Binary autoencoder-based channel coding provides yet another application. In the PRESERVED_PLACEHOLDER_12 OR \12max_results12^ block configuration over a binary symmetric channel, continuous pretraining followed by direct binarization and fine-tuning yields a learned encoder–decoder pair that learns a rotated version, or coset code, of the optimal Hamming code (&&&12 OR \12&&&). The learned codebook is reported to recover linearity, the Hamming PRESERVED_PLACEHOLDER_12 OR \12query12^ distance spectrum, minimum distance PRESERVED_PLACEHOLDER_12 OR \12\12, and the same BLER as maximum-likelihood decoding of the classical Hamming code.
12 OR \12. Limitations, misconceptions, and acronym ambiguity
A persistent misconception is that a binary bottleneck is merely a benign postprocessing step. The hashing literature states the opposite: replacing the true binary objective by a relaxed continuous objective and binarizing afterward is generally suboptimal, because it is not the same as optimizing under binary constraints (&&&12query12&&&). The communication-coding literature makes the related point that enforcing binary codewords directly inside differentiable autoencoders is difficult because discretization breaks gradient flow and often leads to unstable convergence (&&&12 OR \12&&&). These works converge on the same technical lesson: the central difficulty of binary autoencoding is not only representation capacity, but optimization under discontinuous constraints.
Another common misconception is that binarization automatically improves interpretability. The sparse-coding results do not support that as a universal claim. Binarization improves unweighted interpretability and monosemanticity, but increases reconstruction error, creates ultra-high-frequency uninterpretable features, and loses its advantage when interpretability scores are frequency-adjusted; the paper concludes that polysemanticity may be an ineliminable property of neural activations (&&&12 OR \12&&&). This suggests that binary codes can remove one channel for hiding information—continuous activation magnitude—without eliminating the underlying pressure to redistribute information elsewhere.
The acronym itself is also ambiguous. In adjacent literatures, BAE can denote Boosting-based Autoencoder Ensemble for unsupervised outlier detection (&&&12 OR \12&&&), Branched Autoencoder in BAE-NET for shape co-segmentation (&&&12 OR \12&&&), Bayesian Autoencoder for explainable unsupervised deep learning (&&&12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12&&&), and Benign Autoencoder as a theoretical notion of an optimal encoder–decoder pair (&&&12id:(Carreira-Perpiñán et al., 2015) OR id:(Balsubramani, 2016) OR id:(Osaulenko, 2020) OR id:(Deja et al., 2020) OR id:(Abe et al., 10 Feb 2026) OR id:(Quirke et al., 29 Sep 2025) OR id:(Cho et al., 25 Sep 2025) OR id:(Ninkovic et al., 12 Nov 2025) OR id:(Sarvari et al., 2019) OR id:(Chen et al., 2019) OR id:(Malamud et al., 2022) OR id:(Yong et al., 2021)12query12&&&). These usages are distinct from binary autoencoder and should not be conflated with it.