Network Structure Inference via ChatGPT

• The paper demonstrates a hybrid approach combining ChatGPT’s language reasoning with classical structure learning to refine causal and dynamic networks.
• It applies tailored prompt engineering and iterative user feedback to resolve ambiguous edges and incorporate latent variables, improving accuracy.
• Empirical evaluations reveal reduced misdirected edges and enhanced prediction performance in domains like finance and public health.

Network structure inference via ChatGPT refers to the leveraging of LLMs such as GPT-4, often through a conversational interface, to extract, refine, or generate the structure of networks—including causal graphs and dynamic relation networks—from structured data, expert inputs, or unstructured text such as news headlines. This paradigm enables the integration of linguistic, domain, and commonsense knowledge encoded within the LLM with classical data-driven or algorithmic network inference, producing systems that are interactive, interpretable, and, in some settings, empirically superior to fully automated or purely manual approaches (Zhang et al., 2024, Chen et al., 2023).

1. System Architectures and Workflows

Two principal system types have been demonstrated: causal network inference (CausalChat) and dynamic entity-relation inference from text (ChatGPT-informed GNNs).

• CausalChat (Zhang et al., 2024):
  • Integrates a Structure Learning Algorithm (SLA; e.g., GES, PC/FCI) to generate an initial CPDAG from user-supplied data.
  • Augments this skeleton with a ChatGPT-powered edge commentary module that, upon user interaction with ambiguous or undirected edges, generates bespoke prompts and interprets LLM responses.
  • Embeds user-in-the-loop iterative refinement, where visual analytics dashboards link model structure, GPT-4 justifications, and actionable interface elements.
• ChatGPT-Informed Graph Neural Networks (Chen et al., 2023):
  • Utilizes ChatGPT to process temporally indexed news headline corpora, extracting dynamic relation graphs at daily resolution.
  • Feeds these inferred edge sets into graph neural networks for downstream predictive tasks, such as next-day stock movement forecasting.

A summary table of main pipeline components:

Paper	Input Type	LLM Role	Output Network	Downstream Task
CausalChat	Variable data	Causal reasoning	CPDAG/DAG	Expert-guided structure ref.
GNN for Stocks	News headlines	Entity relation	Dynamic co-impact cliques	Financial prediction

2. Prompt Engineering and Template Strategies

Network structure inference via ChatGPT is driven by systematic prompt templates tailored to the inference objectives.

• CausalChat (Zhang et al., 2024):
  • For each selected variable pair, the system issues a decuple of directional causality prompts (five A→B; five B→A), requesting numerical significance ratings.
  • Additional prompts elicit sets of confounders (name, strength, justification), mediators (with conditions and direction), and latent factors (strength, sign, explanation).
  • This recursive interrogation enables local enrichment of edge directionality, causality type, and latent variable augmentation.
• ChatGPT-Informed GNNs (Chen et al., 2023):
  • Employ a daily zero-shot prompt instructing ChatGPT to extract affected companies and their sentiment from news headlines in strict JSON format.
  • No elaborate preprocessing or weighting—edges are undirected, forming a clique over the affected subset for that day.

This design of prompts, tightly aligned with both human reasoning and LLM supervisions, is central to extracting high-fidelity information from black-box models.

3. Mathematical Foundations and Representation

Both domains reconstruct network adjacency structures, but with context-specific graph semantics and inference procedures.

• Graph Representation:
  • For a graph $G = (V, E)$, the adjacency matrix $A_{ij}$ is defined by $A_{ij}=1$ if $i \to j \in E$, $0$ otherwise.
  • In temporal entity-relation graphs, $A^{(t)}_{ij}$ encodes dynamic, time-indexed connectivity.
• CausalChat Back-end:
  • Conditional independence testing and Bayesian Information Criterion (BIC)-based score search, as in GES or PC, are employed for initial structure learning.
  • SEM coefficients are estimated by regressing each variable onto its parents, and the total log-likelihood is computed as $-\sum_i [N_i/2 \ln(\operatorname{RSS}_i/N_i)]$.
• Iterative Update Algorithm (Zhang et al., 2024):
  • Edge direction is assigned by comparing average ChatGPT ratings across bidirectional prompt sets.
  • Cycles triggered by user selection are resolved using a Model Tree, splitting the structure into acyclic variants.
  • New confounders, mediators, or latents suggested by ChatGPT are added as nodes and edges, marked as “dotted” if lacking data, with SEM re-estimated as necessary.
• Financial News Graph (Chen et al., 2023):
  • Each day’s co-affected companies define node sets and edge cliques $E_t$.
  • This evolving graph provides the connectivity for a GNN (Graph SAGE or GCN layer), whose outputs are fused with LSTM-based time series features for final stock movement classification.

4. Concrete Case Studies and Applications

• Automotive Causal Inference Example (Zhang et al., 2024):
  • Given an ambiguous edge between Cylinders and Displacement, ChatGPT's ratings (e.g., "Does higher Cylinders cause higher Displacement?" → 4) resolved the direction CL→DS and justified edge orientation via summary visualization.
  • In another scenario, a query on CarWeight→Time0-60 led ChatGPT to propose “Torque” as a strong mediator, added to the graph as a latent path.
• Public Health Causal Modeling (Zhang et al., 2024):
  • Analysts used confounder and mediator prompts to surface and add missing factors (e.g., SocioeconomicStatus, Access to Alternative Treatments) to opioid-related DAGs, improving both graph completeness and explanatory value.
• Financial Entity-Relation Graphs (Chen et al., 2023):
  • For each daily window, headlines generated inferred company impact cliques via ChatGPT, constructing $G_t$ for input into a GNN.
  • The system produced superior prediction metrics and risk-adjusted returns versus ARIMA, LSTM, and pure embedding baselines.

5. Empirical Evaluations and Performance

CausalChat was evaluated via both ablation and expert studies (Zhang et al., 2024):

• Non-expert Study (Life Expectancy):
  • Three stages compared BIC-only, pure LLM text, and CausalChat Lite (with charts). CausalChat yielded $1/18$ misdirected edges (5%) vs $A_{ij}$0 (28%) for BIC and $A_{ij}$1 (39%) for LLM-only, discovering more true confounders and mediators and reducing edge resolution time by ~3.7$A_{ij}$2 versus LLM text.
  • SUS usability score reached 79.17 (85-89th percentile).
• Expert Public Health Assessment:
  • All experts successfully corrected edge directionality and hypothesized new factors; CausalChat was noted for rapid domain-access and personalized DAG support.
  • Noted limitations included absence of citation and explicit disclaimer mechanisms for potential hallucinations.

In financial prediction (Chen et al., 2023):

• The ChatGPT→GNN→LSTM→MLP approach attained a test-set weighted F1 of 0.4133 (≥1.8% absolute improvement over all baselines) and decreased volatility (14.06% vs 23.61%) and drawdown in backtesting.

6. Limitations, Challenges, and Future Prospects

Identified challenges include:

• Justification Transparency: GPT-4 explanations lack source citations; hallucinations or unsupported claims require explicit disclaimers and possibly auxiliary validation (Zhang et al., 2024).
• Edge Weighting and Granularity: No edge weighting in text-derived graphs (sentiment is logged but not encoded); a plausible implication is that finer-grained prompt engineering or post-processing could further refine relation strength (Chen et al., 2023).
• Integration Bottlenecks: Addition of LLM-suggested variables may not be actionable without data coverage; “dotted” edges/nodes visually distinguish unsupported inferences but do not resolve the data deficit.

The described systems furnish a new class of hybrid interactive tools for network structure inference, coupling LLM reasoning with visualization and classical data analytics. The demonstrable improvement in discovery rates, efficiency, and prediction accuracy signals widespread applicability, but mandates continued work on provenance, citation, and hallucination management for deployment in mission-critical domains (Zhang et al., 2024, Chen et al., 2023).