CoPINet: Contrastive Perceptual Inference Network

• CoPINet is a permutation-invariant reasoning model that employs contrast modules and a Noise-Contrastive Estimation loss to rank RPM answer candidates.
• It combines a ResNet-style perception backbone with an inference module, using contrast mechanisms to extract candidate-specific features from aggregated contexts.
• Empirical results show CoPINet outperforms baselines on RAVEN and PGM benchmarks, achieving high accuracy with significantly fewer training samples.

CoPINet, short for Contrastive Perceptual Inference network, is a permutation-invariant neural architecture for Raven’s Progressive Matrices (RPM) that was introduced to improve machine spatial-temporal reasoning from pixel-level inputs. It combines a ResNet-style perception backbone with an inference module, instantiates contrast both in model topology and in the training objective, and treats RPM as a ranking problem over answer candidates rather than as a conventional classification problem. In "Learning Perceptual Inference by Contrasting" (Zhang et al., 2019), CoPINet is presented as borrowing the idea of “contrast effects” from psychology, cognition, and education and as setting the new state-of-the-art for permutation-invariant models on RAVEN and PGM.

1. Problem setting and formal task

CoPINet was developed for RPM-style abstract visual reasoning, a setting in which eight observed panels $\mathcal{O}$ define a latent relational pattern and a model must select the correct completion from a candidate set $\mathcal{A}$. The paper frames this domain as “thinking in pictures,” that is, spatial-temporal reasoning, and uses two major benchmarks: RAVEN and PGM. Within CoPINet, the prediction problem is reduced to ranking candidate completions by a scalar score rather than assigning a class label to a fixed output space.

Formally, for each candidate $a$, the model computes a scalar “negative potential” $f(\mathcal{O}\cup a)$ and defines

$$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$

with the desired ordering

$$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$

This formulation is important because it aligns the learning problem with the structure of RPM instances, where the core task is to compare alternative completions under a shared context rather than to recognize a single object class. The authors further argue that spatial-temporal reasoning depends on envisaging the possibilities consistent with the relations between objects and can be solved from pixel-level inputs (Zhang et al., 2019).

2. Network organization

CoPINet consists of two jointly trained branches: a perception backbone and an inference module. The perception backbone is a ResNet-style CNN encoder that processes each $3\times 3$ problem panel independently. After the initial convolutional layers, features from the first two rows and first two columns are summed to form row-wise and column-wise context embeddings, which is used to ensure invariance to swapping rows or columns. These context embeddings are then passed through a stack of residual blocks, each preceded by a contrast module. A small MLP on top of the final feature produces the scalar score $f(\mathcal{O}\cup a)$ for each candidate.

The inference module shares the same CNN encoder as the perception branch and extracts a global context embedding from the eight observed panels $\mathcal{O}$. On this embedding, it places $N$ independent SoftMax, or Gumbel-SoftMax, heads, one per attribute, each over $\mathcal{A}$0 possible rule types. A single sample $\mathcal{A}$1 is drawn once per forward pass and injected, via learned $\mathcal{A}$2 convolution transforms, into every contrast module in the perception stream.

The reported implementation uses a 4-stage ResNet perception backbone with channels $\mathcal{A}$3. The inference branch uses the same ResNet encoder, global pooling, and $\mathcal{A}$4 independent linear heads of size $\mathcal{A}$5. The contrast module $\mathcal{A}$6 is a $\mathcal{A}$7 convolution followed by BatchNorm, and the MLP head is a two-layer ReLU MLP with hidden size $\mathcal{A}$8 and output dimension $\mathcal{A}$9 (Zhang et al., 2019).

3. Contrastive mechanism and objective

A defining property of CoPINet is that contrast appears at two levels: in the model architecture and in the loss. At the model level, each residual block in the perception backbone is prefaced by a contrast module that enforces a common-versus-residual decomposition across all answer candidates. If $a$0 denotes the raw feature map associated with candidate $a$1, CoPINet first computes a common feature

$a$2

and then forms the candidate-specific contrastive representation

$a$3

This subtraction biases the network toward features that distinguish one candidate from the set-level common structure.

At the objective level, CoPINet does not use a straight cross-entropy loss. Instead, it employs a sigmoid-based Noise-Contrastive Estimation-style objective. Let $a$4 be the correct answer and let $a$5 be a constant, or fixed-random, baseline. Training encourages

$a$6

by minimizing

$a$7

The paper’s ablations separate the effects of the contrast module and the contrastive objective, making clear that the architectural contrast and the NCE-style loss are distinct contributions. A later summary of CoPiNet’s mechanism restates the contrastive representation as subtraction of a pooled candidate summary and describes the loss in terms of positive and negative samples with a compatibility score, preserving the same basic contrastive logic (Zhang et al., 2019).

4. Permutation invariance and inference-by-basis

Permutation invariance is central to CoPINet’s design. Because every observation-candidate pair $a$8 is processed with shared weights, and because the contrast module only depends on the set of candidate features through an aggregation operation, CoPINet is strictly invariant to permuting the two governed rows or columns and to permuting the answer-choice order. This is not a peripheral implementation convenience; it is part of the model’s formalization of RPM, whose relational structure should not depend on arbitrary candidate ordering.

The inference module implements what the paper describes as permutation-invariant inference-by-basis. Classical RPM solvers posit that each attribute follows one of a finite set of relation types. CoPINet operationalizes this by assuming $a$9 possible attributes, each drawn from a dictionary of $f(\mathcal{O}\cup a)$0 rule-bases, and by modeling

$f(\mathcal{O}\cup a)$1

A single sampled hypothesis $f(\mathcal{O}\cup a)$2 is linearly projected and added into every contrast module, conditioning candidate comparison on the inferred relational hypothesis. In a later proof sketch used to explain the same family of ideas, the pooled feature is written as $f(\mathcal{O}\cup a)$3, and invariance follows because $f(\mathcal{O}\cup a)$4 is unchanged under any permutation of its arguments; shared parameters then preserve invariance through the remaining computation (Zhang et al., 2019).

5. Empirical results, ablations, and data efficiency

CoPINet was evaluated on RAVEN, with 70K problems, and PGM, with 1.4 M problems. The reported test-set results place it substantially above prior permutation-invariant baselines on both benchmarks.

Benchmark	Prior reference point	CoPINet
RAVEN	WReN-NoTag-Aux: 17.6%; ResNet + DRT: 59.6%	91.4%
PGM (neutral split)	WReN-NoTag-Aux: 49.1%	56.4%

The RAVEN result is also reported against human and oracle reference points: human $f(\mathcal{O}\cup a)$5, oracle $f(\mathcal{O}\cup a)$6. The paper’s ablations show how performance changes as components are added. On RAVEN, a backbone only model with CE loss achieves $f(\mathcal{O}\cup a)$7; adding the contrast module with CE loss raises performance to $f(\mathcal{O}\cup a)$8; adding the contrastive NCE loss yields $f(\mathcal{O}\cup a)$9; and the full CoPINet with contrast and inference reaches $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$0. On PGM, the analogous progression is $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$1, $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$2, $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$3, and $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$4.

The data-efficiency study is also notable. On RAVEN, CoPINet reaches human-level, defined there as $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$5, with only 21K training samples, approximately one half of the full set, and beats all prior baselines with as few as 660 samples, approximately $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$6 of full. On PGM, even 75K samples, approximately $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$7 of full, yield approximately $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$8 accuracy, described as competitive with CNN and LSTM baselines. Training is reported in PyTorch with Adam, $$p(a\mid \mathcal{O})=\frac{1}{Z}\exp\bigl(f(\mathcal{O}\cup a)\bigr),$$9, $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$0, weight decay $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$1, learning rate $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$2 with decay on plateau, batch size $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$3 on four Titan RTX GPUs, early stopping on validation loss, and convergence typically in 100–200 epochs (Zhang et al., 2019).

6. Interpretation, later extensions, and nomenclatural disambiguation

The paper grounds CoPINet in several cognitive motifs. “Contrast effects” from psychology, cognition, and education motivate the subtraction of common candidate structure so that the model emphasizes relational residuals. Structure-mapping theory motivates the inference-by-basis component, in which abstract rule-bases inferred from the observed panels condition the perception stream. Noise-Contrastive Estimation provides the template for the objective-level contrast. Within this framing, CoPINet is intended to be simultaneously perceptually grounded, abstractly inferential, and contrast-driven.

A later extension, "ViTCN: Vision Transformer Contrastive Network For Reasoning" (Song et al., 2024), preserves CoPINet’s contrastive module and loss while replacing the CNN encoder with a Vision Transformer. In that formulation, each $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$4 panel is split into non-overlapping patches, processed by stacked Multi-Head Self-Attention and MLP layers, and the resulting global feature is fed into the same contrastive machinery. The reported overall accuracy on RAVEN rises from $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$5 for CoPiNet to $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$6 for ViTCN, with the largest gains on the more complex $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$7Grid and $$p(a_\star\mid \mathcal{O}) \ge p(a'\mid \mathcal{O}) \quad \forall\, a'\neq a_\star.$$8Grid configurations. This suggests that later work treated CoPINet less as a closed architecture than as a contrastive, permutation-invariant reasoning framework whose perceptual front end can be replaced.

The acronym can be confused with unrelated models. CoPINet is distinct from Competitive Physics-Informed Neural Networks, abbreviated CPINNs, which use an adversarial min-max formulation for partial differential equations, and from Competitive Pathway Networks, abbreviated CoPaNet, which use parallel residual-type subnetworks followed by an element-wise max operation for object recognition (Zeng et al., 2022, Chang et al., 2017). The similarity in naming reflects a shared use of “competitive” or “contrastive” vocabulary rather than a shared problem setting or shared architecture.