---
title: Perfection Gap Factor in VLM Transfer
url: https://www.emergentmind.com/topics/perfection-gap-factor-pgf
type: topic
---

# Perfection Gap Factor in VLM Transfer

Searching arXiv for the named PGF paper and closely related uses of “PGF” to ensure current, source-grounded coverage.
to=arxiv_search.search ทะเบียนฟรี  尚度 json
{"query":"id:2511.18787 OR title:\"Understanding Task Transfer in Vision-Language Models\" OR \"Perfection Gap Factor\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}【อ่านข้อความเต็มjson
[{"arxiv_id":"2511.18787","version":"v1","title":"Understanding Task Transfer in Vision-Language Models","authors":["Aman Chaddha","Kartik Gupta","Dmitry Lagun","Amit Das"],"categories":["cs.CV","cs.AI"],"published":"2025-11-24","updated":"2025-11-24","pdf_url":"http://arxiv.org/pdf/2511.18787v1","abs_url":"https://arxiv.org/abs/2511.18787v1"}]

to=arxiv_search.search  彩神争霸 彩票直属 json
{"query":"\"projectively coresolved Gorenstein flat\" PGF arXiv","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}
to=arxiv_search.search 
Searching for the established homological-algebra meaning of PGF to handle acronym disambiguation accurately.
to=arxiv_search.search  大发时时彩怎么 ҳәоит json
{"query":"\"projectively coresolved Gorenstein flat\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}
to=arxiv_search.search  สำนักเลขานุการองค์กร 】【。】【”】【json
{"query":"id:2408.07232 OR title:\"Relative PGF modules and dimensions\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}
Perfection Gap Factor (PGF) is a metric introduced to quantify zero-shot cross-task transfer in vision-language models (VLMs) after single-task fine-tuning. It is designed for settings in which fine-tuning on one perception task alters performance on other tasks, sometimes positively and sometimes negatively, and it measures these effects in a headroom-normalized way so that transfer is comparable across heterogeneous task difficulties and baselines. In the study that introduced it, PGF is used to analyze task transferability across 13 perception tasks and to construct directed task-transfer graphs for open-weight VLMs [2511.18787].

## 1. Concept and motivation

1. **Definition of the problem space**

Perception-oriented fine-tuning of a VLM rarely affects only the fine-tuned task. It reshapes representations and skills that also matter for other perception tasks, sometimes helping and sometimes hurting them. PGF is introduced to quantify these cross-task transfer effects in a way that is comparable across heterogeneous task difficulties and baselines [2511.18787].

The central motivation is that raw averages of score deltas conflate tasks with different difficulty and ceiling effects. A \(+2\%\) gain on a task already at \(90\%\) accuracy closes a much larger fraction of the remaining headroom than a \(+2\%\) gain on a task at \(40\%\). PGF therefore measures improvement or deterioration relative to the remaining “perfection gap,” rather than in raw score units alone [2511.18787].

2. **What PGF is designed to capture**

PGF captures both magnitude and breadth. Magnitude refers to how much a target task’s score changes after fine-tuning on a source task. Breadth refers to how many target tasks are helped or harmed. The headroom normalization means that identical raw gains are scored differently depending on how close the baseline already is to a target task’s ceiling [2511.18787].

A direct implication of this design is that PGF emphasizes ceiling-level effects. Near-perfect baselines amplify the metric: a small absolute gain or loss can correspond to a large fraction of the remaining gap. The study explicitly treats this as a feature rather than an error, while also recommending separate reporting of positive and negative aggregates because the metric is asymmetric [2511.18787].

## 2. Formal definition and variants

1. **Pairwise PGF**

Let \(\mathcal{M}\) denote a base VLM, let \(T_j\) be a target task, and let \(\mathcal{M}(T_i)\) denote the same model after fine-tuning on a source task \(T_i\). Let \(\mathrm{Acc}(\mathcal{M}, T_j)\) be the model’s zero-shot score on \(T_j\), and let \(U_j\) be a ceiling performance for \(T_j\). The PGF from source task \(T_i\) into target task \(T_j\) is defined by [2511.18787]
\[
\mu_{i \to j} =
\frac{
\mathrm{Acc}\!\left(\mathcal{M}(T_i),\, T_j\right)
-
\mathrm{Acc}\!\left(\mathcal{M},\, T_j\right)
}{
U_j - \mathrm{Acc}\!\left(\mathcal{M},\, T_j\right) + \epsilon
},
\]
where \(\epsilon = 10^{-6}\) for numerical stability.

The numerator is the change in target-task performance induced by fine-tuning on \(T_i\). The denominator is the baseline headroom to the ceiling. Consequently, \(\mu_{i \to j}=1\) means that fine-tuning completely closes the remaining gap, \(\mu_{i \to j}=0.5\) closes half the gap, and \(\mu_{i \to j}<0\) indicates negative transfer [2511.18787].

2. **Normalization across tasks and metrics**

In the original study, all BLINK tasks are evaluated with multiple-choice accuracy in percent, so scores are directly comparable and \(U_j=100\) is natural. For metrics where lower values are better, such as MAE or RMSE, the paper proposes a metric-mapped generalization based on a monotone affine transformation to a normalized higher-is-better score in \([0,100]\) [2511.18787]:
\[
\mathrm{Score}_j = 100 \cdot \frac{U_j - E}{U_j - L_j}.
\]
The main PGF formula is then applied with \(\mathrm{Score}\) replacing \(\mathrm{Acc}\).

3. **Best-bound PGF**

The paper also defines a best-bound variant in which \(U_j\) is replaced by the best observed score on task \(T_j\) among all fine-tunes or models in the study:
\[
U_j^{\mathrm{best}} = \max_k \mathrm{Acc}\!\left(\mathcal{M}(T_k),\, T_j\right),
\]
\[
\mu_{i \to j}^{\mathrm{best}} =
\frac{
\mathrm{Acc}\!\left(\mathcal{M}(T_i),\, T_j\right)
-
\mathrm{Acc}\!\left(\mathcal{M},\, T_j\right)
}{
U_j^{\mathrm{best}} - \mathrm{Acc}\!\left(\mathcal{M},\, T_j\right) + \epsilon
}.
\]
Empirically, this produces a PGF of \(1\) along the diagonal, because direct supervision on \(T_j\) gives its best score in the study [2511.18787].

4. **Bounds and sign conventions**

PGF has a positive upper bound of \(1\): gains cannot exceed the remaining headroom. Its negative side is asymmetric. With \(m\) evaluation items, if \(x=100(1-\tfrac{1}{m})\), then the worst deterioration for a baseline strictly below perfection is [2511.18787]
\[
\mu_{\min} = \frac{-x}{100-x} = -(m-1).
\]
Thus positive and negative transfer are not symmetric, which is why the study reports separate aggregates for positive and negative effects.

## 3. Aggregation, transferability, and graph construction

1. **Breadth-weighted source-side aggregates**

PGF is defined pairwise, but the paper also introduces source-level aggregates that combine magnitude and breadth. Let \(N\) be the total number of target tasks, let
\[
p = \bigl|\{j:\mu_{i \to j}>0\}\bigr|,
\qquad
n = \bigl|\{j:\mu_{i \to j}<0\}\bigr|.
\]
Positive and negative task transferability of a source task \(T_i\) are defined by [2511.18787]
\[
\Delta(i)^+ =
\begin{cases}
\left(\dfrac{1-e^{-p/N}}{p}\right)\sum_{j=1}^{N}\mu_{i \to j}\,\mathbf{1}\{\mu_{i \to j}>0\}, & p>0,\\[0.75em]
0, & p=0,
\end{cases}
\]
\[
\Delta(i)^- =
\begin{cases}
\left(\dfrac{1-e^{-n/N}}{n}\right)\sum_{j=1}^{N}\mu_{i \to j}\,\mathbf{1}\{\mu_{i \to j}<0\}, & n>0,\\[0.75em]
0, & n=0.
\end{cases}
\]

These forms average the positive or negative PGFs over affected targets and multiply by a saturating breadth factor \(1-e^{-\text{fraction}}\), which rewards broader effects without allowing unbounded linear growth [2511.18787].

2. **Target-side malleability**

The target-side analog aggregates over sources for a fixed target \(T_j\). With
\[
p = \bigl|\{i:\mu_{i \to j}>0\}\bigr|,
\qquad
n = \bigl|\{i:\mu_{i \to j}<0\}\bigr|,
\]
positive and negative malleability are defined as [2511.18787]
\[
\Theta(j)^+ =
\begin{cases}
\left(\dfrac{1-e^{-p/N}}{p}\right)\sum_{i=1}^{N}\mu_{i \to j}\,\mathbf{1}\{\mu_{i \to j}>0\}, & p>0,\\[0.75em]
0, & p=0,
\end{cases}
\]
\[
\Theta(j)^- =
\begin{cases}
\left(\dfrac{1-e^{-n/N}}{n}\right)\sum_{i=1}^{N}\mu_{i \to j}\,\mathbf{1}\{\mu_{i \to j}<0\}, & n>0,\\[0.75em]
0, & n=0.
\end{cases}
\]

These quantities measure how sensitive a task is to many different source fine-tunes, again separating positive from negative effects [2511.18787].

3. **Computation and visualization**

The paper gives an explicit workflow. For each source-target pair, one chooses a ceiling \(U_j\), computes baseline and post-fine-tune scores, forms the gap \(G(j)=U_j-A_{\text{base}}(j)+\epsilon\), and then computes \(\mu_{i\to j}\). Missing scores are set to NaN and excluded from aggregates. A task-transfer graph is then constructed from the directed weighted adjacency matrix \(W\) with entries \(W_{i\to j}=\mu_{i\to j}\) [2511.18787].

For visualization, the study keeps the top \(q\)-percentile by absolute value of edges; direction runs from source task \(T_i\) to target task \(T_j\); color is determined by sign and thickness by \(|\mu|\). If many edges share the cutoff, all are included to avoid bias. Positive cliques are subsets of tasks with \(\mu_{i\to j}>0\) for every ordered pair \(i\neq j\), and negative cliques are defined analogously [2511.18787].

4. **Personas**

The study defines node-level “personas” by comparing aggregates with model-level averages. A donor has \(\Delta(i)^+\) consistently above the model’s mean positive transferability; a pirate has \(|\Delta(i)^-|\) consistently above the model’s mean negative transferability. Sponge and sieve are defined analogously for targets using \(\Theta(j)^+\) and \(\Theta(j)^-\) [2511.18787].

## 4. Empirical patterns in task transfer

1. **Model scale and transfer structure**

Across Qwen-2.5-VL 3B, 7B, and 32B, positive transferability and positive malleability increase from 3B through 7B to 32B, while negative transfer shows no monotone trend. This makes PGF a vehicle for describing not only whether transfer occurs, but how transfer structure changes with scale [2511.18787].

2. **Task families with strong positive influence**

Low-level perception tasks are both strong donors and strong sponges. Relative Depth (RD), Relative Reflectance (RR), and Visual Correspondence (VC) frequently yield high positive \(\mu\) toward many targets and exhibit high positive malleability. The study also reports that image-level tasks tend to be more positively transferable than crop-level tasks, with pixel- and image-level targets showing high malleability [2511.18787].

Art Style (AS), Counting (CN), and Forensics Detection (FD) are reported as image-level tasks with notable influence patterns. This suggests that transfer is structured by task type rather than being a purely idiosyncratic artifact of individual fine-tunes, although the paper states its claims in empirical rather than mechanistic terms [2511.18787].

3. **Cliques and personas**

In Qwen-2.5-VL 32B, the largest positive clique comprises nine tasks:
\[
\{ \mathrm{AS}, \mathrm{CN}, \mathrm{FC}, \mathrm{JG}, \mathrm{RD}, \mathrm{RR}, \mathrm{SR}, \mathrm{VC}, \mathrm{VS} \}.
\]
Smaller positive and negative cliques are present in the 3B and 7B variants [2511.18787].

The persona analysis identifies Semantic Correspondence (SC) as a statistically significant donor task across models by unpaired \(t\)-tests across seeds (\(p<0.01\)). Functional Correspondence (FC) is a significant pirate in 3B and 7B (\(p<0.05\)). Visual Similarity (VS), Relative Depth (RD), and Relative Reflectance (RR) are significant sponges (\(p<0.001\)), while Forensics Detection (FD) is the strongest sieve (\(p<0.005\) in 3B and 32B) [2511.18787].

4. **Extension to video tasks**

The study reports that transfer also generalizes to spatio-temporal tasks on VSI Bench. Relative Reflectance again emerges as a donor, Forensics Detection as a pirate, and Counting as a sponge, while Object Appearance Order and Object Relative Distance behave as sieves [2511.18787].

## 5. Experimental setting and operational use

1. **Benchmark and training protocol**

The PGF study evaluates three open-weight VLMs: Qwen-2.5-VL 3B, 7B, and 32B. It uses 13 BLINK perception tasks: Art Style (AS), Counting (CN), Forensics Detection (FD), Functional Correspondence (FC), Jigsaw (JG), Multi-view Reasoning (MR), Object Localization (OL), Relative Depth (RD), Relative Reflectance (RR), Semantic Correspondence (SC), Spatial Reasoning (SR), Visual Correspondence (VC), and Visual Similarity (VS) [2511.18787].

All tasks are evaluated with multiple-choice accuracy in percent. Evaluation uses the official BLINK code with consistent prompting and response extraction, and GPT-4.1 is used for parsing responses. Fine-tuning is performed independently per task using LoRA with rank \(8\), \(\alpha=16\), cosine learning-rate schedule with \(10^{-4}\) learning rate, batch size \(16\), and mixed precision; QLoRA and DeepSpeed ZeRO are used depending on model size, with ZeRO-2 for 3B and 7B and ZeRO-3 for 32B. Four random seeds are used for robustness [2511.18787].

The source datasets are task-specific. The paper gives examples including TallyQA for Counting, Depth-in-the-Wild plus human annotations for Relative Depth, SPair-71k for Semantic Correspondence, HPatches for Visual Correspondence, LVIS for Object Localization, and WikiArt for Art Style [2511.18787].

2. **Data selection guided by PGF**

The paper proposes a concrete use case: when supervised data for a target task \(T\) is scarce or unavailable, choose source tasks with high positive \(\mu_{i\to T}\) and form a multi-task mixture. A simple rule is to sample source-task training examples with probability proportional to \(\max(\mu_{i\to T},0)\):
\[
\alpha_i =
\frac{\max(\mu_{i\to T},0)}
{\sum_k \max(\mu_{k\to T},0)}.
\]
Empirically, PGF-informed mixtures consistently outperform random mixtures, and in two cases even surpass direct supervised fine-tuning on the target task [2511.18787].

This use of PGF is explicitly outcome-based: it relies only on before-and-after evaluation scores, not on access to gradients or internal representations. A plausible implication is that it is operationally attractive in settings where architecture internals are unavailable or expensive to analyze, although the paper presents this point through comparison rather than formal complexity claims [2511.18787].

## 6. Related metrics, limitations, and terminological ambiguity

1. **Relation to other transfer measures**

The paper contrasts PGF with average transfer gain, task-affinity or Taskonomy-style scores, and gradient-interference or representational-overlap measures. Average transfer gain is simpler but ignores headroom. Taskonomy-style scores often involve fine-tuning on both source and target with task-specific decoders, whereas PGF isolates zero-shot cross-task transfer induced by a single-task fine-tune in foundation VLMs. Gradient and representation measures require model internals; PGF is architecture-agnostic and outcome-based [2511.18787].

2. **Limitations and robustness**

PGF assumes a meaningful ceiling \(U_j\). With multiple-choice accuracy, \(U_j=100\) is straightforward; for other metrics, careful normalization is required. The asymmetry of the metric means that positive and negative aggregates should be reported separately. Near-perfect baselines amplify \(|\mu|\), which the paper describes as by design, but which also requires careful interpretation [2511.18787].

The study also notes benchmark-format dependence. Many BLINK tasks are multiple-choice, which constrains failure modes relative to open-ended generation. The findings are obtained primarily on the Qwen-2.5-VL family with LoRA fine-tunes, though the paper reports qualitative generalization to LLaVA-1.5-13B and to VSI video tasks. Robustness analyses show that PGF heatmaps are stable across seeds, step ablations preserve qualitative transfer structure while increasing magnitudes, multi-image tasks such as VS, JG, and AS induce similar LoRA parameter updates, and task-transfer graphs retain structure across percentile thresholds such as \(25\%\), \(50\%\), \(75\%\), and \(100\%\) [2511.18787].

3. **Acronym disambiguation**

In the VLM literature covered here, PGF means **Perfection Gap Factor**. In homological algebra, however, PGF has an established and unrelated meaning: **projectively coresolved Gorenstein flat** modules. That usage appears in work on model structures and Gorenstein dimensions, where “PGF” does not stand for “Perfection Gap Factor” [2302.12905]. A relative extension of that terminology introduces \({\rm PG_CF}\) or PGcF for “projectively coresolved Gc-flat,” and when \(C=R\) or \(C\) is a projective generator, PGcF\((R)\) coincides with the absolute PGF\((R)\) of Šaroch–Šťovíček [2408.07232].

Because the two meanings belong to distinct research programs, acronym expansion is essential in cross-disciplinary contexts. In contemporary VLM transfer analysis, PGF denotes a headroom-normalized metric of cross-task transfer; in relative and absolute Gorenstein homological algebra, PGF denotes a module class and associated dimension theory [2511.18787].

Source: https://www.emergentmind.com/topics/perfection-gap-factor-pgf