LogicCAR: Neural-Symbolic ZS Action Recognition

• LogicCAR is a neural-symbolic framework that applies zero-shot compositional action recognition using differentiable fuzzy logic to impose first-order constraints.
• It integrates explicit compositional logic and hierarchical primitive logic to reduce spurious correlations and semantic ambiguity in video-text models.
• State-of-the-art results on the Sth-com benchmark demonstrate its effectiveness in enforcing structural consistency and boosting unseen composition accuracy.

Searching arXiv for LogicCAR and the related baseline methods mentioned in the provided data. arXiv search: LogicCAR main paper. LogicCAR is a neural-symbolic framework for zero-shot compositional action recognition (ZS-CAR) that introduces first-order logic constraints over verb-object compositions and over hierarchical verb and object categories, then encodes those constraints as differentiable fuzzy-logic regularizers within a video-text contrastive model. The framework is presented in “Zero-shot Compositional Action Recognition with Neural Logic Constraints” (Ye et al., 4 Aug 2025). Its stated objective is to address two failure modes in ZS-CAR: missing compositional structure constraint, which leads to spurious correlations between primitives, and neglect of semantic hierarchy constraint, which leads to semantic ambiguity and impairs training. LogicCAR addresses these issues through two components—Explicit Compositional Logic and Hierarchical Primitive Logic—and evaluates them on the Sth-com benchmark, where the reported result is state-of-the-art performance among the listed baselines (Ye et al., 4 Aug 2025).

1. Problem setting and motivation

Zero-shot compositional action recognition aims to identify unseen verb-object compositions in videos by exploiting knowledge of verb and object primitives learned during training (Ye et al., 4 Aug 2025). In the formulation used for LogicCAR, the unseen classes are not arbitrary new primitives; rather, they are unseen pairings of verbs and objects that are individually observed during training but never paired together in the training set (Ye et al., 4 Aug 2025).

The framework is motivated by two specific deficiencies in naïve compositional models. The first is the absence of compositional structure constraints, which permits spurious primitive combinations. The provided example is the prediction of an implausible category such as “napkin fall like a rock,” arising from a high-frequency verb and a high-frequency object being combined without structural restriction (Ye et al., 4 Aug 2025). The second is the absence of semantic hierarchy constraints, which introduces semantic ambiguity among primitives and weakens the training process (Ye et al., 4 Aug 2025).

LogicCAR argues that human-like symbolic reasoning provides a principled solution because it explicitly models compositional and hierarchical structured abstraction (Ye et al., 4 Aug 2025). This suggests a neural-symbolic design in which symbolic constraints are not used as post hoc filters but are integrated into optimization as soft training constraints.

2. Explicit Compositional Logic

The Explicit Compositional Logic component formalizes structural constraints over composition labels and their constituent primitives using first-order logic. Every valid composition category $c=(v,o)$ must imply its verb category $v$ and its object category $o$:

$$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$

Here $c(\cdot)$, $v(\cdot)$, and $o(\cdot)$ are 0–1 predicates indicating whether a sample $x$ belongs to the corresponding label (Ye et al., 4 Aug 2025). These are referred to as “Composed” (subordinate) relationships.

LogicCAR also imposes “Exclusive” relationships. Different verb categories are pairwise mutually exclusive, and likewise for objects. For a fixed verb predicate $v$ and all other verbs $v^1_v,\dots,v^{M_1}_v$:

$v$0

with an analogous form for objects $v$1 (Ye et al., 4 Aug 2025). These axioms specify that a sample should not simultaneously satisfy incompatible primitive labels.

To make these logical constraints trainable, the paper relaxes each first-order logic axiom into a soft numeric consistency score $v$2 using Gödel/Goguen fuzzy operators. The mappings given are conjunction $v$3, implication $v$4, and universal quantification $v$5 (Ye et al., 4 Aug 2025). If $v$6 denotes the model confidence that sample $v$7 belongs to composition class $v$8, and $v$9 denote the corresponding verb and object confidences, then the differentiable form of the composed rule is

$o$0

and the exclusive term for verbs is

$o$1

with an analogous formula $o$2 for objects (Ye et al., 4 Aug 2025).

The resulting explicit-compositional logic regularizer aggregates over all composition predicates $o$3, verbs $o$4, and objects $o$5:

$o$6

This regularizer is minimized jointly with the ordinary classification loss (Ye et al., 4 Aug 2025). In functional terms, it enforces that composition predictions remain consistent with their primitives while suppressing mutually incompatible verb and object assignments.

3. Hierarchical Primitive Logic

The Hierarchical Primitive Logic component adds a fine-to-coarse semantic hierarchy over verbs and objects. Verbs are grouped by their root action word; the provided examples are “fall like a feather” and “fall like a rock,” which share the coarse category “fall” (Ye et al., 4 Aug 2025). Objects are grouped into broad semantic categories such as “napkin” $o$7 “tableware” and “hat” $o$8 “clothing,” and these object groupings are obtained via in-context prompting of a LLM (Ye et al., 4 Aug 2025).

Let $o$9 denote the coarse verb category containing a verb $$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$0, and let $$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$1 denote the coarse object category containing an object $$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$2. The logical requirements are

$$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$3

Exclusivity is then imposed among distinct coarse classes as well: $$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$4 and similarly for $$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$5 (Ye et al., 4 Aug 2025). These axioms enforce a fine-to-coarse implication structure together with mutual exclusivity at the coarse level.

The differentiable fuzzy-logic relaxation produces the “composed” scores

$$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$6

$$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$7

and the “exclusive” term

$$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$8

plus the analogous $$\forall x\,[\,c(x)\;\Rightarrow\;v(x)\,],\quad \forall x\,[\,c(x)\;\Rightarrow\;o(x)\,].$$9 (Ye et al., 4 Aug 2025).

Over all base verbs and objects and over all coarse verbs and objects, the hierarchical-logic loss is

$c(\cdot)$0

The intended effect is fine-to-coarse reasoning capacity: primitive predictions are encouraged to align with broader semantic groupings, while coarse categories remain semantically distinct (Ye et al., 4 Aug 2025). A plausible implication is that such hierarchical consistency reduces semantic confusion among nearby classes even when the primitive space is large.

4. Neural architecture and training objective

LogicCAR extends a standard video-text contrastive model, exemplified by CLIP adapted to video, by adding the two logic-driven regularizers (Ye et al., 4 Aug 2025). The visual encoder uses a frozen, framewise backbone such as ViT-B/32 to extract per-frame features $c(\cdot)$1 (Ye et al., 4 Aug 2025). A Dynamic Module pools and MLP-projects $c(\cdot)$2 into verb-related features for motion pattern detection, while a Static Module uses temporal convolutions and pooling to produce object-related features for shape and appearance detection (Ye et al., 4 Aug 2025).

On the text side, LogicCAR uses two lightweight branches with learnable prompts or adapters, one for verbs and one for objects. The prompts are explicitly phrased as “a verb of X” and “an object of Y,” and are tuned in a CoOp-style manner so that the verb branch specializes in dynamic semantics and the object branch specializes in static semantics (Ye et al., 4 Aug 2025, Zhou et al., 2021). This dual-branch text design parallels the decomposition imposed by the symbolic constraints.

For classification, the model computes cosine similarity between the video representation of sample $c(\cdot)$3 and the prompt embedding of composition $c(\cdot)$4, denoted $c(\cdot)$5 (Ye et al., 4 Aug 2025). A multi-label softmax over all seen compositions yields the ordinary cross-entropy loss

$c(\cdot)$6

The logic modules enter training as soft constraints. Explicit Compositional Logic contributes $c(\cdot)$7, and Hierarchical Primitive Logic contributes $c(\cdot)$8. The final training objective is

$c(\cdot)$9

where $v(\cdot)$0 weight the logic penalties, typically $v(\cdot)$1 (Ye et al., 4 Aug 2025). The paper states that these constraints integrate seamlessly with standard contrastive video-text learning and incur negligible inference overhead (Ye et al., 4 Aug 2025). Since the logic is realized as a training-time regularization mechanism rather than a separate reasoning engine, the architectural extension remains tightly coupled to the base embedding model.

5. Experimental protocol and empirical results

The reported experiments use the Sth-com dataset, which is based on Something-Something V2 (Ye et al., 4 Aug 2025). Sth-com contains 161 verbs, 248 objects, and 5 124 feasible verb-object compositions, with 79 465 videos (Ye et al., 4 Aug 2025). The training split contains 3 451 seen compositions. The validation and test splits each contain approximately 733 seen and approximately 717 unseen constitution classes, where the verbs and objects are observed in training but never paired during training (Ye et al., 4 Aug 2025).

Evaluation uses top-1 verb accuracy, top-1 object accuracy, top-1 seen composition accuracy, top-1 unseen composition accuracy, the harmonic mean

$v(\cdot)$2

and AUC, defined as the area under the seen-unseen accuracy trade-off curve (Ye et al., 4 Aug 2025).

When built on CLIP (ViT-B/32) with CoOp prompts, LogicCAR is reported to achieve new state-of-the-art results across all metrics among the baselines listed in the paper (Ye et al., 4 Aug 2025). The summary values given are as follows.

Method	HM	AUC
Base CLIP	≈24.6%	≈9.1%
CoOp	≈36.6%	≈18.1%
CSP	≈36.0%	≈18.0%
DFSP	≈35.8%	≈17.9%
C2C	≈44.8%	≈25.9%
LogicCAR	≈45.2%	≈27.0%

The comparative baselines named in the data are CLIP (Radford et al., 2021), CoOp (Zhou et al., 2021), CSP (Bergermann et al., 2022), DFSP (Collaboration et al., 2022), and C2C (Timoshuk et al., 2024). Within the scope of the reported numbers, LogicCAR improves over the strongest listed baseline, C2C, by approximately $v(\cdot)$3 HM and $v(\cdot)$4 AUC (Ye et al., 4 Aug 2025). This suggests that the primary gain is not only in balanced seen-unseen performance but also in the calibration trade-off captured by AUC.

6. Ablations, qualitative analysis, and interpretation

The ablation studies separate the contributions of the two logic components. Explicit Compositional Logic only adds $v(\cdot)$5 HM and $v(\cdot)$6 AUC over C2C. Hierarchical Primitive Logic only adds $v(\cdot)$7 AUC. Using both constraints yields the best overall result, including the reported $v(\cdot)$8 AUC over C2C (Ye et al., 4 Aug 2025). This indicates that the two regularizers are complementary rather than redundant.

Coefficient sweeps over $v(\cdot)$9 and $o(\cdot)$0 show an “inverted-U” dependence, which the paper interprets as validation of the weighting strategy (Ye et al., 4 Aug 2025). In practical terms, this means that underweighting the logic losses weakens their effect, while overweighting them can be detrimental. The result is consistent with the broader design choice of imposing symbolic knowledge as soft regularization rather than as a hard constraint.

The qualitative analyses further align with the formal objective. Top-1 examples on seen and unseen compositions confirm that impossible compositions such as “napkin fall like a rock” are suppressed (Ye et al., 4 Aug 2025). Confusion matrices show that LogicCAR better separates semantically similar actions (Ye et al., 4 Aug 2025). These analyses do not introduce new metrics, but they illustrate the intended role of the logic terms: to suppress structurally invalid primitive pairings and to refine semantic discrimination among related categories.

A common misconception in compositional recognition is that better primitive representations alone are sufficient for robust zero-shot composition. LogicCAR directly contests that assumption by treating primitive recognition as necessary but insufficient unless compositional restrictions and semantic hierarchy are explicitly modeled (Ye et al., 4 Aug 2025). Another possible misconception is that symbolic structure must be imposed through discrete inference at test time; the LogicCAR formulation instead embeds symbolic constraints into the training objective through differentiable fuzzy logic (Ye et al., 4 Aug 2025).

7. Significance within neural-symbolic video understanding

LogicCAR is situated at the intersection of compositional recognition, contrastive video-text learning, and neural-symbolic integration. Its core claim is that encoding human-like compositional rules—every composition must imply its verb and object categories—together with fine-to-coarse hierarchies—verbs and objects belong to exactly one coarse class—systematically prevents spurious correlations and enforces semantic coherence (Ye et al., 4 Aug 2025).

The framework’s technical significance lies in how symbolic constraints are operationalized. Rather than replacing a contrastive backbone, the method augments a standard architecture with differentiable fuzzy-logic regularizers (Ye et al., 4 Aug 2025). This allows the model to retain the standard video-text embedding pipeline while incorporating formal structure into optimization. The paper further states that the constraints incur negligible inference overhead and yield consistent improvements in zero-shot generalization (Ye et al., 4 Aug 2025).

Within the benchmarked setting, LogicCAR therefore functions as a logic-driven extension of CLIP-style compositional recognition rather than as a separate symbolic reasoner. A plausible implication is that the framework can be viewed as a template for integrating explicit logical priors into other recognition pipelines where label spaces are factorized and hierarchically organized. The paper characterizes this direction as bridging symbolic transparency with deep representation learning for robust compositional reasoning in video understanding (Ye et al., 4 Aug 2025).