--- title: Boolean Operations on Conceptors url: https://www.emergentmind.com/topics/boolean-operations-on-conceptors type: topic --- # Boolean Operations on Conceptors A conceptor is a symmetric positive semidefinite matrix with eigenvalues in $[0,1]$, functioning as a soft projection operator onto the characteristic linear subspace of a dataset or neural activation pattern. Boolean operations on conceptors—negation (NOT), conjunction (AND), and disjunction (OR)—systematically extend logical calculus into the nonlinear, dynamical setting of neural representations and abstract vector spaces. These operations have been foundational in recurrent neural network dynamics [1403.3369], continual representation learning [1904.09187], neural memory modeling [2003.11640], and activation engineering in large language models [2410.16314]. ## 1. Definition of Conceptor Matrices Given a collection of vectors $\{x_i\} \subset \mathbb{R}^N$, the corresponding conceptor $C \in \mathbb{R}^{N \times N}$ minimizes the regularized reconstruction loss: \[ \frac{1}{n}\sum_{i=1}^n \|x_i - C x_i\|_2^2 + \alpha^{-2}\|C\|_F^2, \] where $\alpha > 0$ is the aperture parameter controlling the subspace's width. The closed-form solution is: \[ C = R (R + \alpha^{-2}I)^{-1}, \quad \text{where} \; R = \frac{1}{n}X X^\top, \] with $X = [x_1, ..., x_n]$. The eigenvalues of $C$ satisfy $0 \leq \mu_i < 1$, ensuring that $C$ is a soft, rather than hard, projector. In spectral terms, $C$ shrinks each principal direction according to the data variance and aperture. ## 2. Formalism of Boolean Operations Boolean operations on conceptor matrices closely parallel standard set algebra, mapping geometric and logical relationships in activation space to closed algebraic forms. ### Negation (NOT) \[ \neg C = I - C \] This operator captures the orthogonal complement to the subspace encoded by $C$, softly projecting onto directions not present in the original manifold [1904.09187, 1406.2671, 1403.3369, 2410.16314]. ### Conjunction (AND) \[ C_1 \wedge C_2 = (C_1^{-1} + C_2^{-1} - I)^{-1} \] $C_1 \wedge C_2$ projects onto components simultaneously present in $C_1$ and $C_2$, corresponding to the intersection of their ellipsoidal regions [1406.2671]. The Moore–Penrose pseudoinverse is applied when $C_i$ is singular. ### Disjunction (OR) Multiple mathematically equivalent definitions are recognized: \[ C_1 \vee C_2 = \neg(\neg C_1 \wedge \neg C_2) = I - \left[(I-C_1)^{-1} + (I-C_2)^{-1} - I\right]^{-1} \] Alternately, for data sharing principal axes, the union formula becomes: \[ C_1 \vee C_2 = C_1 + C_2 - (C_1 \wedge C_2) \] OR spans all directions captured by either conceptor, yielding the union ellipsoid [1403.3369, 1904.09187]. These operations preserve positive semidefiniteness and ensure eigenvalues remain in $[0,1]$. ## 3. Algebraic Properties and Geometric Intuition Boolean operations on conceptors form an algebra satisfying most classical Boolean identities, modulo the non-commutativity and dimensionality constraints inherent in matrix algebra [1406.2671, 1403.3369]: - **Commutativity:** $C_1 \wedge C_2 = C_2 \wedge C_1, \;\; C_1 \vee C_2 = C_2 \vee C_1$ - **Associativity:** $(C_1 \wedge C_2) \wedge C_3 = C_1 \wedge (C_2 \wedge C_3)$, analogously for $\vee$ - **Absorption:** $C_1 \vee (C_1 \wedge C_2) = C_1$, $C_1 \wedge (C_1 \vee C_2) = C_1$ - **Idempotence:** $C \wedge C = C$, $C \vee C = C$ - **De Morgan’s Laws:** $\neg(C_1 \wedge C_2) = \neg C_1 \vee \neg C_2$, $\neg(C_1 \vee C_2) = \neg C_1 \wedge \neg C_2$ Geometrically, negation flips the passage versus suppression of principal directions; conjunction isolates the intersection ellipsoid (shared directions), and disjunction creates the convex hull or union of the patterns' supported directions [1403.3369]. Spectrum-wise, for commuting conceptors with $\text{diag}(s_i), \text{diag}(t_i)$, AND/OR operate elementwise: \[ s_i(C \wedge D) = \frac{s_i t_i}{s_i + t_i - s_i t_i}, \quad s_i(C \vee D) = s_i + t_i - s_i t_i \] ## 4. Applications in Neural Network Architectures ### Continual Representation Learning In continual sentence representation learning, Boolean disjunction enables accumulation of all "common discourse directions" seen across sequential corpora. At stage $i$, after processing a new corpus $D^i$ and computing its temporary conceptor $C^{\mathrm{temp}}$, the running conceptor is updated by: \[ C^i = C^{\mathrm{temp}} \vee C^{i-1} \] This strategy preserves encoded knowledge from preceding corpora, in contrast to retraining from scratch, which exhibits catastrophic forgetting. Sentence embeddings are then formed by projecting new data away from the accumulated subspaces: \[ f_s = q_s - C^M q_s \] Zero-shot modes—setting $M=0$—utilize an initial conceptor computed on stop-word vectors alone. Empirically, this algorithm achieves significant stability in semantic textual similarity tasks and robust retention of prior knowledge [1904.09187]. ### Activation Steering in Large Language Models Boolean operations on conceptors facilitate advanced activation engineering in transformer-based LLMs. Given conceptor matrices $C_1, C_2$ for distinct behaviors (e.g., "antonym" and "capitalize"), AND-combined conceptors (${C_1 \wedge C_2}$) more robustly encode multiple steering constraints than additive combinations of steering vectors: \[ h'_\ell = \beta_c (C^{\mathrm{(combined)}} h_\ell) \] Boolean operations allow for precise geometric control in activation space—intersections enforce simultaneous constraints, while unions aggregate features [2410.16314]. Empirically, AND-combined conceptors consistently outperform additive baselines on composite function tasks, even surpassing conceptors trained directly on the function's union in certain cases. ### Neural Memory and Dynamical Pattern Manipulation Boolean operations enable construction, intersection, and suppression of stored memory patterns within reservoir computing frameworks. For example, OR combines two stored patterns into a memory capable of replaying either, AND focuses on overlapping structure, and NOT supports targeted erasure (e.g., $C \leftarrow C \wedge \neg C_{v_1}$ forgets pattern $v_1$) [2003.11640, 1403.3369]. ## 5. Examples and Implementation Notes For diagonal conceptors $C = \mathrm{diag}(0.8, 0.2)$, $D = \mathrm{diag}(0.5, 0.7)$, the Boolean operators behave as follows [1406.2671]: | Operation | Formula | Result | |-----------------|------------------------------------------------------|----------------------------| | Negation | $I - C$ | $\mathrm{diag}(0.2, 0.8)$ | | Conjunction | $(C^{-1} + D^{-1} - I)^{-1}$ | $\approx \mathrm{diag}(0.44, 0.18)$ | | Disjunction | $I - ((I-C)^{-1} + (I-D)^{-1} - I)^{-1}$ | $\approx \mathrm{diag}(0.83, 0.72)$ | Efficient implementation requires careful treatment of pseudoinverses and low-rank cases. For large $N$, SVD or eigendecomposition is recommended to manage numerical instabilities and regularization [1403.3369, 2410.16314]. In high dimensions, Boolean operations are most stable when performed in a basis where all relevant conceptors are diagonal. ## 6. Empirical Impact and Limitations Empirical studies demonstrate that Boolean conceptor operations: - Improve stability and retention in continual learning for sentence representations, outperforming retrain-from-scratch methods susceptible to forgetting [1904.09187]. - Enable reliable compositional control in large language models, with AND-combined conceptors achieving superior performance on composite activation steering tasks compared to additive or union-trained baselines [2410.16314]. - Support memory retrieval, pattern blending, abstraction, and selective deletion within recurrent neural systems, providing a unified algebraic structure for cognitive manipulation [2003.11640, 1403.3369]. Limitations include computational costs ($O(N^3)$ for matrix inversion), the need for adequate data samples to avoid rank-deficient covariance, and aperture hyperparameter tuning. Boolean operations assume compatible apertures and sufficient subspace overlap for invertibility. ## 7. Significance in Neural Representation and Abstraction The Boolean algebra of conceptors extends classical logical calculus to high-dimensional, distributed neural representations. This framework enables rich programmatic manipulation of neural activations—merging, intersecting, abstracting, and suppressing patterns—while maintaining mathematical tractability. The lattice structure introduced by AND/OR supports compositionality and abstraction, with direct application to continual learning, memory management, and controllable generation in deep neural models [1403.3369, 2410.16314, 1904.09187, 2003.11640]. Source: https://www.emergentmind.com/topics/boolean-operations-on-conceptors