DeFacto: Diverse Research Artifacts in AI

• DeFacto represents distinct research artifacts focused on evidence-grounded methods in crisis response, summarization, and multimodal reasoning.
• In crisis response, DEFACTO employs multi-agent simulation with human-agent proxies and 3D visualization for situational awareness.
• In summarization and multimodal reasoning, DeFacto leverages human feedback and counterfactual training to enhance factual consistency and accuracy.

Searching arXiv for papers explicitly named “DeFacto” to ground the article. Search query: DeFacto arXiv titles and abstracts. DeFacto is a label attached to multiple distinct research artifacts in the arXiv literature, rather than a single unified method or benchmark. In the material considered here, it denotes: a user-centric multi-agent crisis response system named DEFACTO (“Demonstrating Effective Flexible Agent Coordination of Teams through Omnipresence”) for training, simulation, and human–agent coordination in disaster scenarios (0903.2543); a human-annotated dataset and experimental framework for improving factual consistency in summarization through natural language feedback (Liu et al., 2022); and a counterfactual multimodal reasoning framework for enforcing evidence-grounded and faithful reasoning in multimodal LLMs (Xu et al., 25 Sep 2025). The same string also appears in unrelated papers only as the ordinary phrase “de facto,” for example in discussions of public-key security notions, XML, Simulink, and JPEG, and those usages are terminologically distinct from the named systems and datasets (Rastaghi, 2012).

1. Scope and nomenclature

The term appears in at least three technically separate senses. In crisis response research, DEFACTO is a large-scale multi-agent simulation and modeling system built on Machinetta proxy architecture and designed for training, situational awareness, and flexible human–agent teamwork in fire evacuation and disaster rescue scenarios (0903.2543). In summarization, DeFacto is a dataset centered on factual consistency, containing source articles, machine-generated summaries, human explanations, evidence sentences, corrective instructions, and human-edited corrections (Liu et al., 2022). In multimodal reasoning, DeFacto is a counterfactual training framework that constructs positive, counterfactual, and random-masking image variants and optimizes multimodal LLMs with GRPO-based reinforcement learning to improve answer accuracy and reasoning faithfulness (Xu et al., 25 Sep 2025).

A useful disambiguation is that these artifacts share a concern with reliability under imperfect information, but they do so in different problem settings. This suggests a family resemblance at the level of research motivation rather than a common implementation lineage. By contrast, several other papers use the lower-case phrase “de facto” only descriptively, as in “de facto notion of security,” “de facto standard for data transfer,” “de facto CPS simulator,” or “de facto standard for image compression”; those are ordinary linguistic uses and do not designate a research system named DeFacto (Kalman et al., 2013).

Artifact	Research area	Core role
DEFACTO	Multi-agent crisis response	Training, simulation, situational awareness, human–agent coordination
DeFacto	Summarization factuality	Dataset and framework for feedback-driven factual correction
DeFacto	Multimodal reasoning	Counterfactual training framework for evidence-grounded reasoning

2. DEFACTO in multi-agent crisis response

DEFACTO is presented as a user-centric multi-agent crisis response system whose primary purposes are training and rehearsal in complex crisis environments, simulation and decision support, improved situational awareness, and human–agent team coordination (0903.2543). It is described as “a multi-agent simulation and modeling system based on Machinetta proxy architecture,” with a scalable architecture at $O(10{,}000)$ agents, and it consists of a simulator, a 3D omni-viewer, Machinetta proxy-based teamwork infrastructure, and an analysis tool.

Its simulator executes dynamic crisis scenarios, including fire evacuation and disaster rescue. The Machinetta proxy layer associates each human or software agent with a proxy responsible for transfer-of-control over decisions, management of local team beliefs, proxy-to-proxy communication, proxy-to-team-member communication, coordination, and task allocation. The system also supports team-oriented plans, joint activities, and conflict resolution. The 3D omni-viewer provides an omnipresent, high-fidelity view of the environment for human responders and crisis managers, while the analysis tool is used to analyze the impact of teamwork interaction strategies, particularly different transfer-of-control policies and team coordination schemes.

A distinctive feature of DEFACTO is its explicit treatment of transfer-of-control. The paper states that “an optimal transfer-of-control strategy balances the risk of high quality decision made by human against the risk of costs incurred due to a delay in getting the decision from agent,” and the supplied conceptual formalization expresses this as a policy choice between human control and agent-team control: $$\pi^* = \arg\min_{\pi} \sum_{t} \Big( C(d_{\pi(t)}) - U(q_{\pi(t)}) \Big).$$ Within the same framework, team-oriented plans organize goals at strategic, tactical, and operational levels, and conflict resolution is treated as an allocation problem over tasks and agents subject to incompatibility constraints.

The paper’s comparative analysis identifies several requirements well addressed by DEFACTO: decentralized control, coordination and task allocation, improved situational awareness through the 3D omni-viewer, flexible human–agent interaction, and structured strategic–tactical–operational decomposition. It also identifies specific limitations: “System does not support fault-tolerance,” “System does not support plug-and-play capability,” “System does not support adaptive planning,” and “System does not provide learning from experience strategies.” Additional limitations include high bandwidth communication requirements, limited configurability in terms of agent profile and scenarios, and the observation that building 3D models for the omni-viewer can require “months or even years” of manual modeling effort. These limitations position DEFACTO as a strong platform for studying human–multi-agent coordination, but not as a complete realization of all crisis response domain requirements.

3. DeFacto as a summarization factuality dataset

In summarization research, DeFacto is a high-quality, human-annotated dataset designed to study whether informational natural language feedback can improve factual consistency in abstractive summarization (Liu et al., 2022). Each example contains a source news article from XSum, an initial PEGASUS-generated summary, human feedback about factual consistency, and a human-edited summary that corrects factual errors using minimal edits. The paper defines factual consistency as the requirement that “All the information of the summary can be supported by the source document,” and adopts the distinction between intrinsic and extrinsic errors.

The annotation protocol is six-step. Annotators decide whether the summary is factually consistent, categorize errors as intrinsic, extrinsic, or both, provide a free-form explanation, select an evidence sentence from the source document, write corrective instructions using templated operations such as Remove, Add, Replace, Modify, Rewrite, or Other, and then edit the summary to make it factually consistent with minimal necessary modifications. The average number of instructions is 1.52 per example. The dataset contains approximately 1,500 examples from the XSum validation set and approximately 1,000 from the test set, with one annotation per document-summary pair in the main release and three annotations for 100 test examples used for inter-annotator agreement.

The paper reports that 71.1% of annotated summaries contain at least one factual error, 58.8% contain extrinsic errors, 22.0% contain intrinsic errors, and 9.63% contain both. Krippendorff’s alpha on 100 test examples is reported as approximately 0.5552 for extrinsic errors, 0.1899 for intrinsic errors, and 0.5260 for any factual errors. For explanation, instruction, and edited-summary agreement, ROUGE-1 F is approximately 30.52, 50.96, and 71.77, respectively. Evidence-selection match rate, using ROUGE-1 at least 90 as the equivalence criterion, is approximately 0.4403.

DeFacto defines three generation tasks. The Editing task maps article, candidate summary, and human feedback to a corrected summary. The Critic task maps article and candidate summary to corrective instructions. The Editor task jointly generates feedback and an edited summary. Fine-tuned encoder–decoder models are trained with maximum likelihood: $$\mathcal{L}_{\text{MLE}} = - \sum_{t=1}^{T} \log p_\theta(y_t \mid y_{<t}, X).$$ The evaluation uses DAE and QAFactEval for factuality and ROUGE for textual similarity. Human-edited summaries substantially improve factuality relative to both XSum references and initial PEGASUS outputs, with DAE scores of 0.6176 for references, 0.6904 for system summaries, and 0.8975 for human-edited summaries, and QAFactEval scores of 1.549, 1.826, and 2.540, respectively.

The dataset supports a broader conclusion: natural language feedback is not merely evaluative but operational. Instructions and explanations both improve editing performance, and jointly conditioning on document, summary, instructions, and explanation yields the strongest reported editing results among the variants in Table 5. The paper also reports that LLMs in zero-shot settings can increase factuality but lack the controllable editing ability needed for minimal, targeted correction. A common misconception would be to treat DeFacto here as a factuality metric; in the paper, it is instead a dataset and experimental framework for feedback-conditioned revision.

4. DeFacto as a multimodal counterfactual reasoning framework

In multimodal reasoning, DeFacto addresses a different problem: multimodal LLMs can produce “thinking with images” traces while remaining unfaithful to the actual visual evidence (Xu et al., 25 Sep 2025). The paper identifies two failure modes. In Mislocalized Failure, the model selects regions that do not contain the relevant evidence and answers incorrectly. In Spurious Correctness, the model answers correctly while selecting irrelevant regions, implying reliance on priors, shortcuts, or dataset biases rather than genuine visual evidence. DeFacto is proposed to enforce both accurate answering and faithful, evidence-grounded reasoning.

The framework is built around three complementary training paradigms. In the positive setting, the relevant evidence is present and the model must select correct evidence regions, reason coherently, and answer correctly. In the counterfactual setting, evidence regions are masked out and the model should abstain by outputting a designated token such as “Unknown.” In the random-masking setting, irrelevant regions are masked while all necessary evidence remains visible, so the model should still answer normally and should not use the mere presence of masks as a cue to abstain.

Evidence localization begins with candidate regions from a Region Proposal Network and OCR, written as

$\mathcal{R} = \{ r_1, r_2, \dots, r_n \}.$

Qwen2.5-VL extracts descriptors of relevant visual concepts,

$\text{MLLM}(I,q) = \{ d_1, d_2, \dots, d_m \},$

and DINO-X grounds those descriptors onto regions, partitioning candidates into evidence regions $\mathcal{R}^+$ and irrelevant regions $\mathcal{R}^-$: $$\mathcal{R}^+ = \{ r \in \mathcal{R} \mid \max_{k \in \mathcal{K}(q)} \mathsf{Det}(r, k) > \tau \}, \qquad \mathcal{R}^- = \mathcal{R} \setminus \mathcal{R}^+.$$ From these, the framework constructs positive, counterfactual, and random-masking instances. The resulting dataset is built from diverse benchmarks and is described as a ~100k-image counterfactual dataset.

Training uses GRPO-based reinforcement learning with group size $M=4$. The objective is

$$\mathcal{L}_{\pi}(\theta) = \mathbb{E}_{i}\left[ \frac{\pi_\theta(\tau^{(i)})}{\pi_{\theta_{\text{old}}}(\tau^{(i)})} \left( R(\tau^{(i)}) - \frac{1}{M}\sum_{j=1}^M R(\tau^{(j)}) \right) \right],$$

and the total reward is

$$\pi^* = \arg\min_{\pi} \sum_{t} \Big( C(d_{\pi(t)}) - U(q_{\pi(t)}) \Big).$$0

Answer correctness rewards correct answers in positive and random instances and rewards “Unknown” in counterfactual instances, while explicitly penalizing guessing in counterfactual cases, even if the guess is correct. Format consistency enforces >, <bbox>, and <answer> structure. Region selection coherence rewards overlap with $$\pi^* = \arg\min_{\pi} \sum_{t} \Big( C(d_{\pi(t)}) - U(q_{\pi(t)}) \Big).$$1 and penalizes overlap with $$\pi^* = \arg\min_{\pi} \sum_{t} \Big( C(d_{\pi(t)}) - U(q_{\pi(t)}) \Big).$$2, using IoU-based quantities $$\pi^* = \arg\min_{\pi} \sum_{t} \Big( C(d_{\pi(t)}) - U(q_{\pi(t)}) \Big).$$3 and $$\pi^* = \arg\min_{\pi} \sum_{t} \Big( C(d_{\pi(t)}) - U(q_{\pi(t)}) \Big).$$4.

The reported quantitative results are substantial. On general VQA, DeFacto improves Qwen2.5-VL-7B from 57.3 to 79.7 on VQAv2, 54.5 to 68.0 on OKVQA, 41.3 to 70.1 on GQA, 85.3 to 88.2 on ScienceQA, 37.7 to 64.5 on VizWiz, and 2.2 to 70.3 on VSR. On document and text-centric tasks, it improves DocVQA from 84.4 to 85.8, ChartQA from 77.8 to 82.4, InfoVQA from 66.0 to 76.9, DeepForm from 30.3 to 51.8, KLC from 35.9 to 37.6, WTQ from 63.9 to 74.5, STVQA from 64.9 to 74.1, TextVQA from 71.0 to 73.4, and AI2D from 71.2 to 79.0. Ablation studies show that full DeFacto outperforms SFT without counterfactuals, SFT with counterfactual alignment, and GRPO without counterfactual reward, supporting the claim that counterfactual rewards and region-level selection are crucial for both accuracy and faithfulness.

A plausible implication is that DeFacto’s novelty lies less in bounding-box prediction by itself than in using counterfactual evidence removal as a training signal that changes the desired answer behavior from answering to abstaining. That is the key contrast with prior “thinking with images” methods described in the paper.

5. Shared themes and major contrasts

Across the three named artifacts, a recurring concern is faithful dependence on the right evidence source. In DEFACTO for crisis response, the central issue is how to balance human decision quality and delay against autonomous agent action, with proxies mediating transfer-of-control and maintaining local team beliefs (0903.2543). In the summarization DeFacto dataset, the central issue is whether every summary claim is supported by the source document and whether corrective instructions can drive minimal edits that remove unsupported content (Liu et al., 2022). In multimodal DeFacto, the central issue is whether the reasoning trace and selected image regions correspond to the actual evidence used to answer the question (Xu et al., 25 Sep 2025).

The three artifacts also differ sharply in methodological substrate. DEFACTO is a multi-agent simulation and coordination infrastructure with 3D visualization, team-oriented plans, and conflict resolution. The summarization DeFacto is a human-annotation resource and supervised generation benchmark built around explanations, evidence sentences, instructions, and edits. The multimodal DeFacto is a synthetic-data and reinforcement-learning framework combining evidence localization, masking-based intervention, structured outputs, and composite rewards. These are not successive versions of one project.

Another contrast concerns the role of humans. In the crisis-response system, humans are first-class team members and decision makers, with explicit opportunities for intervention through the omni-viewer and proxy-mediated control transfer. In the summarization dataset, humans provide the gold-standard feedback and revisions used to train and evaluate models. In the multimodal framework, humans are largely absent from the core optimization loop; the emphasis is instead on automated evidence localization and reward shaping. This suggests that “DeFacto” has been attached to projects in which reliability is sought either by human guidance, by human annotation, or by counterfactual training signals, depending on the domain.

A common misconception would be to read the name as denoting a single benchmark family spanning crisis response, summarization, and multimodal reasoning. The provided literature does not support that interpretation. It supports only that multiple independent research groups have used the same or similar name for distinct systems and datasets.

6. Limitations and prospective directions

Each DeFacto artifact is accompanied by explicit limitations. For DEFACTO in crisis response, the paper states that the system does not support fault tolerance, plug-and-play capability, adaptive planning, or learning from experience, and that it requires high bandwidth communication and intensive 3D modeling effort (0903.2543). These limitations leave open directions such as improved robustness under degraded communications, lower-cost environment modeling, and more adaptive planning mechanisms. That implication is directly aligned with the paper’s own discussion of unmet crisis-response domain requirements.

For the summarization DeFacto dataset, the main constraints are scale, domain concentration on XSum news articles, and the difficulty of annotation (Liu et al., 2022). The paper also notes that extending the methodology to more subjective preferences, such as tone or fairness, is non-trivial. This suggests that DeFacto is strongest where the target preference is well defined and can be tied to concrete evidence in the source document.

For the multimodal DeFacto framework, limitations include dependence on the quality of automatic evidence localization via Qwen2.5-VL descriptors, DINO-X detection, and OCR; the expense of generating multiple masked variants and running GRPO; and the difficulty of handling questions whose evidence is diffuse rather than localized (Xu et al., 25 Sep 2025). The paper explicitly points to future directions such as video understanding, embodied AI, better localization, and explicit faithfulness metrics beyond reward-based evaluation.

Taken together, these works show that DeFacto has become associated, in several independent strands of arXiv research, with the enforcement of grounded behavior under uncertainty: grounded action in human–agent teams, grounded revision in summarization, and grounded reasoning in multimodal models. That synthesis is interpretive, but it is consistent with the technical emphasis of the cited works.