--- title: Code Concept Analysis (CoCoA) url: https://www.emergentmind.com/topics/code-concept-analysis-cocoa type: topic --- # Code Concept Analysis (CoCoA) Searching arXiv for the core CoCoA paper and closely related code-concept analysis works. Code Concept Analysis (CoCoA) denotes a global, post-hoc interpretability framework for code language models that uncovers emergent lexical, syntactic, and semantic structures in a model’s representation space by clustering contextualized token embeddings into human-interpretable concept groups [2510.00476]. In the code-model literature, the term also connects to adjacent lines of work on neuron-level concept localization in code-trained language models [2305.00875] and concept-supervised reasoning for software engineering tasks [2603.23470]. In its most specific current usage, however, CoCoA refers to the framework introduced for analyzing latent concepts in code language models, combining unsupervised concept discovery, hybrid annotation with static analysis and large language models, and concept-grounded local explanations [2510.00476]. ## 1. Definition and scope CoCoA is a *global, post-hoc interpretability framework* for code language models that operates on hidden representations rather than on surface tokens alone [2510.00476]. Its central premise is that contextualized token embeddings in code models contain latent clusters corresponding to recurring code concepts, and that these clusters can be recovered, labeled, and analyzed without modifying model training [2510.00476]. The recovered concepts span multiple abstraction levels, including lexical regularities, syntactic roles, and higher-level semantic functions [2510.00476]. This framing distinguishes CoCoA from local attribution methods such as Integrated Gradients, SHAP, or LIME, which typically highlight individual tokens but often do not explain *why* a token matters in code-specific terms [2510.00476]. It also differs from conventional probing, which requires predefined labels and primarily tests whether a property can be decoded from hidden states rather than how the model organizes its internal concept space [2510.00476]. CoCoA instead treats the model’s representation geometry itself as the primary object of analysis [2510.00476]. The term has acquired broader resonance because related work addresses “code concepts” from complementary angles. “Redundancy and Concept Analysis for Code-trained Language Models” studies neuron-level traceability of code-related concepts such as “number” neurons, “string” neurons, and higher-level task features [2305.00875]. “ConceptCoder” defines code concepts as human-understandable semantic properties of code and uses them as supervised intermediate representations for vulnerability detection and branch prediction [2603.23470]. This suggests a broader research area in which CoCoA can denote concept-centric analysis of code models, but the canonical framework name corresponds to the latent-concept clustering approach introduced in 2025 [2510.00476]. ## 2. Core pipeline CoCoA consists of three main components: concept discovery, concept alignment and annotation, and concept-based local explanations [2510.00476]. The pipeline begins by extracting contextualized token embeddings from each layer of a code language model for a large corpus of code, using approximately 310k token occurrences per layer for clustering [2510.00476]. The reported models include CodeBERT, UniXCoder, and DeepSeekCoder V2 Lite Instruct, and the data source is approximately 30k snippets from Project CodeNet [2510.00476]. Concept discovery is performed per layer using K-means with a fixed number of clusters \(K = 350\) [2510.00476]. The optimization objective is $$ \min_{\{C_1,\dots,C_K\}} \sum_{k=1}^K \sum_{z \in C_k} \|z - \mu_k\|^2 $$ where \(C_k\) is cluster \(k\), \(\mu_k\) its centroid, and \(z\) are contextualized token embeddings [2510.00476]. Before clustering, highly frequent tokens with frequency \(> 15k\) are discarded, and clusters with \(> 15k\) members are later removed as too broad or uninformative [2510.00476]. This yields a layer-wise inventory of latent concepts defined by representation-space proximity rather than by programmer-specified taxonomies [2510.00476]. Concept alignment and annotation then convert clusters into interpretable units. Static analysis via Tree-sitter ASTs is used to quantify syntactic alignment, while a hybrid manual-plus-LLM annotation process generates lexical, syntactic, and semantic labels [2510.00476]. The resulting labeled collection is curated as CodeConceptNet (CoCoNet) [2510.00476]. Finally, CoCoA integrates the discovered concepts with local attribution by mapping salient tokens to latent concept clusters and using cluster labels and descriptions to produce concept-grounded explanations [2510.00476]. A plausible implication is that CoCoA treats concept analysis as a layer between raw representations and human interpretation: clustering provides the latent structure, static analysis constrains syntactic plausibility, and LLM annotation supplies descriptive semantics [2510.00476]. ## 3. Mathematical formulation and annotation methodology The formalization of concept alignment in CoCoA is based on overlaps between discovered clusters and ground-truth syntactic categories [2510.00476]. Let \(\mathcal{T}\) denote the set of token occurrences, \(\mathcal{C}_E = \{C_e\}\) the discovered clusters, and \(\mathcal{C}_H = \{C_h\}\) the set of syntactic categories derived from Tree-sitter [2510.00476]. For a cluster \(C_e\) and category \(C_h\), the token-overlap ratio is $$ \frac{|C_e \cap C_h|}{|C_e|}. $$ A cluster is aligned at threshold \(\theta\) if $$ \alpha_\theta(C_e) = \begin{cases} 1, & \text{if } \exists\ C_h \in \mathcal{C}_H : \dfrac{|C_e \cap C_h|}{|C_e|} \geq \theta \\ 0, & \text{otherwise.} \end{cases} $$ Coverage of a ground-truth category is defined as $$ \kappa_\theta(C_h) = \begin{cases} 1, & \text{if } \exists\ C_e \in \mathcal{C}_E : \dfrac{|C_e \cap C_h|}{|C_e|} \geq \theta \\ 0, & \text{otherwise.} \end{cases} $$ [2510.00476] These definitions support aggregate measures such as “Clusters Labeled,” “Tag Coverage,” and an “Overall Alignment Score” [2510.00476]. For CodeBERT at layer 12 with 350 clusters and a 90% threshold, the reported values are 328/350 clusters labeled, 50% tag coverage, and overall alignment 0.719 in the pre-trained setting; after AST-node-classification fine-tuning, these become 341/350, 76.4%, and 0.869, respectively [2510.00476]. Such numbers indicate that many latent clusters are syntactically coherent, and that structure-aware fine-tuning sharpens this coherence [2510.00476]. Annotation proceeds in two stages. First, senior computer science students with Java experience manually labeled approximately 500 clusters using token lists and example code contexts [2510.00476]. Second, LLM-based labeling scaled this process using prompts that included token lists, context sentences, and examples of desired output format [2510.00476]. The reported LLMs include GPT-4o and Gemini-2.0-Flash, and the final annotation scheme consolidates 1,724 raw labels into 43 canonical semantic tags, with “Unclear Behavioral Role” assigned to residual clusters that resist stable interpretation [2510.00476]. With the improved prompt, 95.8% of LLM-generated labels were rated acceptable; 38.4% of syntactic labels and 83.2% of semantic labels were judged superior to human counterparts [2510.00476]. The hybrid annotation design is notable because it does not require the latent concept inventory to be predefined. This suggests that CoCoA occupies a middle ground between unsupervised discovery and ontology-based interpretation: clusters are model-induced, but their final descriptions are constrained by static syntax and curated semantic labeling [2510.00476]. ## 4. Concept types and internal organization CoCoA reports that discovered clusters capture lexical, syntactic, and semantic regularities [2510.00476]. Lexical clusters include variable-name patterns such as suffix-sharing tokens, numerics, and casing regularities such as CamelCase and PascalCase [2510.00476]. Syntactic clusters group items such as access modifiers (`public`, `private`, `protected`), comparison operators (`<`, `==`, `!=`), parentheses, and the method invocation operator (`.`) [2510.00476]. Semantic clusters capture broader code functions such as mathematical operations (`abs`, `max`, `floor`, `pow`), error logging, data-structure operations, and function definition keywords across languages [2510.00476]. Layer-wise analyses indicate that lexical patterns are strongest in early layers but persist into deeper layers, where they are embedded in more structured or compositional clusters [2510.00476]. Syntactic alignment is high in early and middle layers and then plateaus or slightly declines in later layers, consistent with a transition toward more semantic or task-specialized concepts [2510.00476]. Fine-tuning modulates this structure. AST node classification increases lexical and structural coherence, compile-error detection produces more unaligned clusters suggestive of higher-level semantics, and language classification reduces syntactic alignment while promoting cross-lingual abstractions [2510.00476]. The framework’s examples emphasize ambiguity resolution. A salient token such as `<` may belong to different clusters corresponding to logical comparison, generic type delimiters, or PHP opening tags, depending on context [2510.00476]. CoCoA’s cluster assignment and annotation disambiguate these roles by grounding each occurrence in a concept group rather than treating the token type as intrinsically meaningful [2510.00476]. This directly addresses a common limitation of token-level saliency, where punctuation often appears important but its role is underdetermined [2510.00476]. Related work complements this cluster-centric notion of concept organization with neuron-centric evidence. Neuron-level analysis identifies “number” neurons, “string” neurons, “identifier” neurons, and higher-level “text” neurons in code-trained language models, and reports that over 95% of neurons are redundant with respect to several code intelligence tasks [2305.00875]. ConceptCoder, by contrast, defines explicit code concepts such as null assignment, null check, pointer dereference, memory allocation, buffer access, bounds check, and memory free for vulnerability detection, as well as abstract-value concepts such as `True`, `False`, `NULL`, `Not NULL`, and numeric range abstractions for branch prediction [2603.23470]. Together, these studies suggest that code concepts can be localized at multiple representational scales: clusters of token embeddings, subsets of neurons, and supervised intermediate labels [2510.00476][2305.00875][2603.23470]. ## 5. Robustness, attribution, and human interpretability A central contribution of CoCoA is the Cluster Sensitivity Index (CSI), which measures how stable latent concept clusters remain under semantic-preserving perturbations [2510.00476]. If \(\mathcal{C} = \{c_1,\dots,c_K\}\) and \(\mathcal{C}' = \{c'_1,\dots,c'_K\}\) are clusterings before and after perturbation, CoCoA computes pairwise Jaccard similarities $$ \text{Jaccard}(c_i, c'_j) = \frac{|c_i \cap c'_j|}{|c_i \cup c'_j|} $$ and uses the Hungarian algorithm to find a bijection maximizing total similarity [2510.00476]. The average matched Jaccard is then converted into $$ \text{CSI} = 1 - \text{Average Jaccard}. $$ Lower CSI indicates greater stability [2510.00476]. Reported examples include deterministic identifier renaming with Average Jaccard 0.6751 and CSI 0.3249, identifier casing variation with 0.6723 and 0.3277, minimal casing perturbation with 0.6657 and 0.3343, and statement order randomization with 0.5315 and 0.4685 [2510.00476]. The paper reports average CSI \(= 0.288\), indicating that discovered concepts remain moderately stable under semantics-preserving transformations [2510.00476]. CoCoA’s local explanation mechanism, termed Latent Concept Attribution, combines Integrated Gradients with concept mapping [2510.00476]. A model prediction is first explained at token level; top tokens are selected until they account for 50% of cumulative attribution mass [2510.00476]. Each salient token’s contextual embedding is then mapped to a latent concept cluster using a logistic regression classifier trained to predict cluster IDs from embeddings [2510.00476]. Cluster labels, word clouds, and LLM-generated descriptions are finally used to express why the salient token mattered in concept terms [2510.00476]. This changes the explanatory unit from isolated symbols to structured code roles. In a programming-language-classification example, the token `<` is not merely highlighted as important; it is mapped to a concept cluster associated with PHP opening tags such as `