---
title: 'LogicCAR: Neural-Symbolic ZS Action Recognition'
url: https://www.emergentmind.com/topics/logiccar
type: topic
---

# LogicCAR: Neural-Symbolic ZS Action Recognition

Searching arXiv for LogicCAR and the related baseline methods mentioned in the provided data.
arXiv search: LogicCAR main paper.
LogicCAR is a neural-symbolic framework for zero-shot compositional action recognition (ZS-CAR) that introduces first-order logic constraints over verb-object compositions and over hierarchical verb and object categories, then encodes those constraints as differentiable fuzzy-logic regularizers within a video-text contrastive model. The framework is presented in “Zero-shot Compositional Action Recognition with Neural Logic Constraints” [2508.02320]. Its stated objective is to address two failure modes in ZS-CAR: missing compositional structure constraint, which leads to spurious correlations between primitives, and neglect of semantic hierarchy constraint, which leads to semantic ambiguity and impairs training. LogicCAR addresses these issues through two components—Explicit Compositional Logic and Hierarchical Primitive Logic—and evaluates them on the Sth-com benchmark, where the reported result is state-of-the-art performance among the listed baselines [2508.02320].

## 1. Problem setting and motivation

Zero-shot compositional action recognition aims to identify unseen verb-object compositions in videos by exploiting knowledge of verb and object primitives learned during training [2508.02320]. In the formulation used for LogicCAR, the unseen classes are not arbitrary new primitives; rather, they are unseen pairings of verbs and objects that are individually observed during training but never paired together in the training set [2508.02320].

The framework is motivated by two specific deficiencies in naïve compositional models. The first is the absence of compositional structure constraints, which permits spurious primitive combinations. The provided example is the prediction of an implausible category such as “napkin fall like a rock,” arising from a high-frequency verb and a high-frequency object being combined without structural restriction [2508.02320]. The second is the absence of semantic hierarchy constraints, which introduces semantic ambiguity among primitives and weakens the training process [2508.02320].

LogicCAR argues that human-like symbolic reasoning provides a principled solution because it explicitly models compositional and hierarchical structured abstraction [2508.02320]. This suggests a neural-symbolic design in which symbolic constraints are not used as post hoc filters but are integrated into optimization as soft training constraints.

## 2. Explicit Compositional Logic

The Explicit Compositional Logic component formalizes structural constraints over composition labels and their constituent primitives using first-order logic. Every valid composition category $c=(v,o)$ must imply its verb category $v$ and its object category $o$:

$$
\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad
\forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].
$$

Here $c(\cdot)$, $v(\cdot)$, and $o(\cdot)$ are 0–1 predicates indicating whether a sample $x$ belongs to the corresponding label [2508.02320]. These are referred to as “Composed” (subordinate) relationships.

LogicCAR also imposes “Exclusive” relationships. Different verb categories are pairwise mutually exclusive, and likewise for objects. For a fixed verb predicate $v$ and all other verbs $v^1_v,\dots,v^{M_1}_v$:

$$
\forall x\;\bigl(v(x)\;\Rightarrow\;\neg v^1_v(x)\;\wedge\;\dots\;\wedge\;\neg v^{M_1}_v(x)\bigr),
$$

with an analogous form for objects $o,o^1_o,\dots,o^{M_2}_o$ [2508.02320]. These axioms specify that a sample should not simultaneously satisfy incompatible primitive labels.

To make these logical constraints trainable, the paper relaxes each first-order logic axiom into a soft numeric consistency score ${\cal G}\in[0,1]$ using Gödel/Goguen fuzzy operators. The mappings given are conjunction $\phi\wedge\varphi\mapsto\phi\cdot\varphi$, implication $\phi\Rightarrow\varphi\mapsto1-\phi+\phi\,\varphi$, and universal quantification $\forall x.\,\phi(x)\mapsto1-\bigl(\frac1K\sum_k(1-\phi(x_k))^q\bigr)^{1/q}$ [2508.02320]. If $\mathbf{s}_k[c]\in[0,1]$ denotes the model confidence that sample $x_k$ belongs to composition class $c$, and $\mathbf{s}_k[v], \mathbf{s}_k[o]$ denote the corresponding verb and object confidences, then the differentiable form of the composed rule is

$$
{\cal G}_{C1}(c)\;=\;1
-\Bigl(\tfrac1K\sum_{k=1}^K(\mathbf{s}_k[c]-\mathbf{s}_k[c]\!\cdot\!\mathbf{s}_k[v])^q\Bigr)^{1/q}
-\Bigl(\tfrac1K\sum_{k=1}^K(\mathbf{s}_k[c]-\mathbf{s}_k[c]\!\cdot\!\mathbf{s}_k[o])^q\Bigr)^{1/q},
$$

and the exclusive term for verbs is

$$
{\cal G}_{C2}(v)\;=\;1\;-\;\tfrac1{M_1}\sum_{m=1}^{M_1}
\Bigl(\tfrac1K\sum_{k=1}^K(\mathbf{s}_k[v]\cdot\mathbf{s}_k[v^m_v])^q\Bigr)^{1/q},
$$

with an analogous formula ${\cal G}_{C2}(o)$ for objects [2508.02320].

The resulting explicit-compositional logic regularizer aggregates over all composition predicates $c\in\mathcal N_c$, verbs $v\in\mathcal N_v$, and objects $o\in\mathcal N_o$:

$$
\mathcal L_{ECL}
\;=\;\frac1{|\mathcal N_c|}\sum_{c}\bigl[1-{\cal G}_{C1}(c)\bigr]
\;+\;\frac1{|\mathcal N_v|}\sum_{v}\bigl[1-{\cal G}_{C2}(v)\bigr]
\;+\;\frac1{|\mathcal N_o|}\sum_{o}\bigl[1-{\cal G}_{C2}(o)\bigr].
$$

This regularizer is minimized jointly with the ordinary classification loss [2508.02320]. In functional terms, it enforces that composition predictions remain consistent with their primitives while suppressing mutually incompatible verb and object assignments.

## 3. Hierarchical Primitive Logic

The Hierarchical Primitive Logic component adds a fine-to-coarse semantic hierarchy over verbs and objects. Verbs are grouped by their root action word; the provided examples are “fall like a feather” and “fall like a rock,” which share the coarse category “fall” [2508.02320]. Objects are grouped into broad semantic categories such as “napkin” $\rightarrow$ “tableware” and “hat” $\rightarrow$ “clothing,” and these object groupings are obtained via in-context prompting of a large language model [2508.02320].

Let $\hat v$ denote the coarse verb category containing a verb $v$, and let $\hat o$ denote the coarse object category containing an object $o$. The logical requirements are

$$
\forall x\,[v(x)\Rightarrow\hat v(x)],\qquad
\forall x\,[o(x)\Rightarrow\hat o(x)].
$$

Exclusivity is then imposed among distinct coarse classes as well: $\forall x\;[\hat v(x)\Rightarrow \neg\hat v^1\wedge\dots]$ and similarly for $\hat o$ [2508.02320]. These axioms enforce a fine-to-coarse implication structure together with mutual exclusivity at the coarse level.

The differentiable fuzzy-logic relaxation produces the “composed” scores

$$
{\cal G}_{H1}(v)
=1 - \Bigl(\tfrac1K\sum_{k=1}^K(\mathbf{s}_k[v]-\mathbf{s}_k[v]\!\cdot\!\mathbf{s}_k[\hat v])^q\Bigr)^{1/q},
$$

$$
{\cal G}_{H1}(o)
=1 - \Bigl(\tfrac1K\sum_{k=1}^K(\mathbf{s}_k[o]-\mathbf{s}_k[o]\!\cdot\!\mathbf{s}_k[\hat o])^q\Bigr)^{1/q},
$$

and the “exclusive” term

$$
{\cal G}_{H2}(\hat v)
=1 - \tfrac1{N_1}\sum_{n=1}^{N_1}
\Bigl(\tfrac1K\sum_{k=1}^K(\mathbf{s}_k[\hat v]\!\cdot\!\mathbf{s}_k[\hat v^n])^q\Bigr)^{1/q},
$$

plus the analogous ${\cal G}_{H2}(\hat o)$ [2508.02320].

Over all base verbs and objects and over all coarse verbs and objects, the hierarchical-logic loss is

$$
\mathcal L_{HPL}
=\frac1{|\mathcal N_v|}\sum_v[1-{\cal G}_{H1}(v)]
+\frac1{|\mathcal N_o|}\sum_o[1-{\cal G}_{H1}(o)]
+\frac1{|\mathcal N_{\hat v}|}\sum_{\hat v}[1-{\cal G}_{H2}(\hat v)]
+\frac1{|\mathcal N_{\hat o}|}\sum_{\hat o}[1-{\cal G}_{H2}(\hat o)].
$$

The intended effect is fine-to-coarse reasoning capacity: primitive predictions are encouraged to align with broader semantic groupings, while coarse categories remain semantically distinct [2508.02320]. A plausible implication is that such hierarchical consistency reduces semantic confusion among nearby classes even when the primitive space is large.

## 4. Neural architecture and training objective

LogicCAR extends a standard video-text contrastive model, exemplified by CLIP adapted to video, by adding the two logic-driven regularizers [2508.02320]. The visual encoder uses a frozen, framewise backbone such as ViT-B/32 to extract per-frame features $\mathbf F_x\in\mathbb R^{T\times D}$ [2508.02320]. A Dynamic Module pools and MLP-projects $\mathbf F_x$ into verb-related features for motion pattern detection, while a Static Module uses temporal convolutions and pooling to produce object-related features for shape and appearance detection [2508.02320].

On the text side, LogicCAR uses two lightweight branches with learnable prompts or adapters, one for verbs and one for objects. The prompts are explicitly phrased as “a verb of X” and “an object of Y,” and are tuned in a CoOp-style manner so that the verb branch specializes in dynamic semantics and the object branch specializes in static semantics [2508.02320; 2109.01134]. This dual-branch text design parallels the decomposition imposed by the symbolic constraints.

For classification, the model computes cosine similarity between the video representation of sample $x_k$ and the prompt embedding of composition $c$, denoted $\bm s_k[c]$ [2508.02320]. A multi-label softmax over all seen compositions yields the ordinary cross-entropy loss

$$
\mathcal L_c=\frac1K\sum_k\mathrm{CE}(\bm s_k,\bm y_k).
$$

The logic modules enter training as soft constraints. Explicit Compositional Logic contributes $\mathcal L_{ECL}$, and Hierarchical Primitive Logic contributes $\mathcal L_{HPL}$. The final training objective is

$$
\boxed{
\mathcal L
= \mathcal L_c
\;+\;\alpha\;\bigl(\mathcal L_{ECL} \;+\;\beta\,\mathcal L_{HPL}\bigr)
}
$$

where $\alpha,\beta$ weight the logic penalties, typically $\alpha\approx0.04,\beta\approx0.06$ [2508.02320]. The paper states that these constraints integrate seamlessly with standard contrastive video-text learning and incur negligible inference overhead [2508.02320]. Since the logic is realized as a training-time regularization mechanism rather than a separate reasoning engine, the architectural extension remains tightly coupled to the base embedding model.

## 5. Experimental protocol and empirical results

The reported experiments use the Sth-com dataset, which is based on Something-Something V2 [2508.02320]. Sth-com contains 161 verbs, 248 objects, and 5 124 feasible verb-object compositions, with 79 465 videos [2508.02320]. The training split contains 3 451 seen compositions. The validation and test splits each contain approximately 733 seen and approximately 717 unseen constitution classes, where the verbs and objects are observed in training but never paired during training [2508.02320].

Evaluation uses top-1 verb accuracy, top-1 object accuracy, top-1 seen composition accuracy, top-1 unseen composition accuracy, the harmonic mean

$$
HM = 2\,(\mathrm{Seen}\times\mathrm{Unseen})/(\mathrm{Seen}+\mathrm{Unseen}),
$$

and AUC, defined as the area under the seen-unseen accuracy trade-off curve [2508.02320].

When built on CLIP (ViT-B/32) with CoOp prompts, LogicCAR is reported to achieve new state-of-the-art results across all metrics among the baselines listed in the paper [2508.02320]. The summary values given are as follows.

| Method | HM | AUC |
|---|---:|---:|
| Base CLIP | ≈24.6% | ≈9.1% |
| CoOp | ≈36.6% | ≈18.1% |
| CSP | ≈36.0% | ≈18.0% |
| DFSP | ≈35.8% | ≈17.9% |
| C2C | ≈44.8% | ≈25.9% |
| LogicCAR | ≈45.2% | ≈27.0% |

The comparative baselines named in the data are CLIP [2103.00020], CoOp [2109.01134], CSP [2204.03575], DFSP [2207.11427], and C2C [2407.20132]. Within the scope of the reported numbers, LogicCAR improves over the strongest listed baseline, C2C, by approximately $+0.4\%$ HM and $+1.1\%$ AUC [2508.02320]. This suggests that the primary gain is not only in balanced seen-unseen performance but also in the calibration trade-off captured by AUC.

## 6. Ablations, qualitative analysis, and interpretation

The ablation studies separate the contributions of the two logic components. Explicit Compositional Logic only adds $+0.5\%$ HM and $+0.4\%$ AUC over C2C. Hierarchical Primitive Logic only adds $+0.2\%$ AUC. Using both constraints yields the best overall result, including the reported $+1.1\%$ AUC over C2C [2508.02320]. This indicates that the two regularizers are complementary rather than redundant.

Coefficient sweeps over $\alpha$ and $\beta$ show an “inverted-U” dependence, which the paper interprets as validation of the weighting strategy [2508.02320]. In practical terms, this means that underweighting the logic losses weakens their effect, while overweighting them can be detrimental. The result is consistent with the broader design choice of imposing symbolic knowledge as soft regularization rather than as a hard constraint.

The qualitative analyses further align with the formal objective. Top-1 examples on seen and unseen compositions confirm that impossible compositions such as “napkin fall like a rock” are suppressed [2508.02320]. Confusion matrices show that LogicCAR better separates semantically similar actions [2508.02320]. These analyses do not introduce new metrics, but they illustrate the intended role of the logic terms: to suppress structurally invalid primitive pairings and to refine semantic discrimination among related categories.

A common misconception in compositional recognition is that better primitive representations alone are sufficient for robust zero-shot composition. LogicCAR directly contests that assumption by treating primitive recognition as necessary but insufficient unless compositional restrictions and semantic hierarchy are explicitly modeled [2508.02320]. Another possible misconception is that symbolic structure must be imposed through discrete inference at test time; the LogicCAR formulation instead embeds symbolic constraints into the training objective through differentiable fuzzy logic [2508.02320].

## 7. Significance within neural-symbolic video understanding

LogicCAR is situated at the intersection of compositional recognition, contrastive video-text learning, and neural-symbolic integration. Its core claim is that encoding human-like compositional rules—every composition must imply its verb and object categories—together with fine-to-coarse hierarchies—verbs and objects belong to exactly one coarse class—systematically prevents spurious correlations and enforces semantic coherence [2508.02320].

The framework’s technical significance lies in how symbolic constraints are operationalized. Rather than replacing a contrastive backbone, the method augments a standard architecture with differentiable fuzzy-logic regularizers [2508.02320]. This allows the model to retain the standard video-text embedding pipeline while incorporating formal structure into optimization. The paper further states that the constraints incur negligible inference overhead and yield consistent improvements in zero-shot generalization [2508.02320].

Within the benchmarked setting, LogicCAR therefore functions as a logic-driven extension of CLIP-style compositional recognition rather than as a separate symbolic reasoner. A plausible implication is that the framework can be viewed as a template for integrating explicit logical priors into other recognition pipelines where label spaces are factorized and hierarchically organized. The paper characterizes this direction as bridging symbolic transparency with deep representation learning for robust compositional reasoning in video understanding [2508.02320].

Source: https://www.emergentmind.com/topics/logiccar