SWE-chat: Swedish Conversational & SWE Agents

• SWE-chat is a family of systems, datasets, and evaluation methodologies designed for Swedish-language conversation and real-world software engineering agent interactions.
• It builds on large-scale Transformer models and refined transfer-learning techniques to optimize dialogue human-likeness, informativeness, and coding performance.
• Empirical evaluations reveal operational trade-offs in agent code survival, safety, and ethical deployment, driving continuous refinement in low-resource dialogue settings.

SWE-chat refers to a family of systems, datasets, and evaluation methodologies for open-domain conversational and software engineering (SWE) agents, with a particular emphasis on Swedish-language dialogue generation and realistic developer–AI coding interactions. Originating from the first large-scale Swedish generative conversational model (Flashback-GPT) and evolving through large-scale empirical coding datasets, advanced user-modeling architectures, and purpose-built agent benchmarks, SWE-chat delineates both the state-of-the-art in dialogue generation for low-resource languages and the empirical foundation for measuring coding agents in practical software engineering workflows. It also encompasses best practices for humanlikeness, informativeness, safety, and ethical deployment.

1. Model Architectures and Swedish Language Pretraining

SWE-chat initially denoted large-scale Swedish open-domain conversational models, most notably Flashback-GPT (Norlund et al., 2021). This model employs an auto-regressive Transformer decoder architecture at the GPT-2 1.5B scale:

• 48 layers, 1,600 hidden dimension, 5,400 feed-forward/intermediate dimension, 16 attention heads, and a 52,000 BPE token vocabulary (custom for Swedish).
• Joint probability modeled as $$p(x_1…x_n) = p(x_1) \prod_{i=2}^n p(x_i | x_1…x_{i-1})$$.
• Total parameter count: approximately 1.4 billion.

Parallel work on Swedish transfer learning used DialoGPT-medium (345M parameters) fine-tuned on multiple Swedish conversational corpora without structural modification, demonstrating effective cross-lingual adaption (Adewumi et al., 2021).

These architectures were trained on extremely high-coverage Swedish sources: Flashback-GPT used 1.96 million threads and 55.1 million posts (23.5 GB formatted text), while Småprat experiments used a range of forums and dialogue corpora (Reddit, Familjeliv.se, GDC), employing consistent preprocessing and 80/10/10 data splits.

2. Training Regimens, Data Processing, and Objectives

For large conversational models:

• Training objective: next-token cross-entropy

$$L = -\sum_{i=1}^n \sum_{v \in V} y_i(v) \cdot \log p_\text{model}(x_i = v \mid x_1...x_{i-1})$$

• Optimizer: Adam, with learning-rate warm-up (first 5,000 steps to $5 \times 10^{-5}$), standard Transformer dropout, and block 3-gram repetition filtering during decoding.
• Data ingestion on multi-GPU infrastructure (e.g., 16 × V100 SXM2 for 7 days, 86,250 gradient updates).

Discussion forum structure was leveraged by preserving "Citat" blocks (quoted speech), but the initial data ingested was unrestricted—no explicit filtering for toxicity or topic except quote reinsertion for explicit context. Conversation truncation at 400 tokens was used, with longer threads split.

For transfer-learning approaches, all weights were fine-tuned; context windows were limited (up to 7 utterances); and batch sizes and learning rates matched available framework defaults.

3. Evaluation Methodologies and Empirical Outcomes

Human-Likeness and Informativeness (Flashback-GPT Pilot)

• Human evaluation on 120 Flashback threads, stratified across 12 forums.
• Annotators asked:
  1. Was there any indication the last message was not human-written?
  2. Did the last message add information to the discussion?

• Metrics: majority agreement per thread.

	Humanlike	Informative	Both Qualities
Human	95% (79%)	83% (74%)	83%
Flashback-GPT	68% (48%)	48% (52%)	46%

• Forum-level performance was slightly lower for the largest and most active domains (Society, Politics, Culture & Media). Common failures included brief "safe" answers, topic drift, and misunderstanding.

For Småprat (DialoGPT-Sv), human-likeness on the largest dataset reached 57.3% (Familjeliv 1M+), with perplexity (dev/test) as low as 7.15, comparable to high-resource English fine-tuning (Adewumi et al., 2021). Annotator agreement was high (0.78), and human-written responses scored 94.9% human-like.

Coding Agent Interaction and Code Survival (Dataset SWE-chat)

Later, SWE-chat expanded to denote a living dataset of coding agent/human interaction sessions (Baumann et al., 22 Apr 2026):

• 6,000 sessions, 63,000+ user prompts, 355,000+ agent tool calls.
• Bimodal coding patterns: 41% sessions with nearly all code by agents ("vibe coding"), 23% solely by humans.
• Survival rate of agent-produced code:

$$\mathrm{SurvivalRate} = \frac{\text{agent lines survived}}{\text{agent lines in final state}} \times 100\%$$

Averaged 44.3%, with 64.6% for vibe coding. Collaborative modes achieved better resource efficiency.

• Pushback against agent outputs occurred in 44% of all turns, with common failures relating to parameter misunderstanding, repetitive incorrect fixes, and irrelevant commits.
• Agent code introduced higher vulnerability rates: 0.76 / 1,000 lines in agent-dominated sessions vs 0.08 for human-only sessions.

4. Safety, Ethical Risk, and Mitigation Practices

Forum-derived chat models and code assistants ingest data containing biased, toxic, or illegal content. Identified risks:

• Reproduction of hate speech, stereotypes, or misinformation.
• Misuse for automation of trolling, disinformation, or impersonation.
• Increased security vulnerability rate in agent-written code in practical deployments (Baumann et al., 22 Apr 2026).

Recommended mitigations:

• Active bias audits, toxicity filters, guardrails for both generative dialogue and code.
• Open release for public scrutiny paired with bias documentation.
• Grounding of responses in verified knowledge bases for factuality.
• Real-time content filtering and robust user monitoring for deployment.
• Continuous evaluation of compliance and refusal rates to profanity or unsafe content, detailed in benchmarked enterprise safety frameworks such as SweEval (Patel et al., 22 May 2025).

5. Methodological Advances and Evaluation Paradigms

SWE-chat contributed to advancing SWE agent evaluation via multiple paradigms:

• Dialogue SWE-Bench: Introduces persona-grounded simulated users and schema-guided agents, showing that dialogue resolve-rate (up to +14%) is not strictly tied to code-generation performance; dialogue management is a distinct axis (King et al., 12 Jun 2026).
• SWE-Interact: Benchmarks multi-turn, user-driven, long-horizon workflows, with resolve rates dropping ∼10–25 percentage points versus single-turn settings (Raghavendra et al., 29 Jun 2026). Capability measured across requirement discovery, ambiguity resolution, and iterative refinement.
• Mutation-based benchmark evaluation: Mutation frameworks replace GitHub-issue-style prompts with telemetry-derived real queries; resolve rates fall by 20–50%, revealing systematic overestimation in prior benchmarks (Garg et al., 10 Oct 2025).
• Empirical agent studies: Code survival, pushback, and efficiency data, as in the SWE-chat dataset, provide quantitative backgrounds for agent failure analysis and future agent refinement.

6. Key Limitations and Future Directions

Empirical studies demonstrate that while SWE-chat systems can perform diverse conversational and coding tasks in Swedish, substantial gaps remain in informativeness, dialogue robustness, handling ambiguous or under-specified user intent, and security posture in real-world deployment scenarios:

• Human-likeness plateaus at ∼68% (Flashback-GPT) or ∼57% (Småprat), and informativeness lags further behind.
• Survival of agent-proposed code in committed repositories is less than half, with the risk of increased vulnerabilities (Baumann et al., 22 Apr 2026).
• Practical deployment necessitates biases and toxicity mitigation, persistent memory management, continuous user feedback incorporation, and multi-agent architectural separation for theory-of-mind modeling (Zhou et al., 24 Oct 2025).
• Multi-turn, ambiguous task settings reveal deficits in planning, memory, and context tracking; addressable by user simulators, specialized test suites, and hierarchical memory structures.
• Ongoing research is focused on improved language-specific pretraining, fact-grounded response models, toolkit benchmarking via realistic session logs, and agent safety in code production.

In summary, SWE-chat encompasses both a technical lineage of Swedish open-domain conversational LLMs and a rich empirical foundation for real-world software engineering agent evaluation. It is central to both the advancement of dialogue agents in low-resource languages and the practical, safety-critical evolution of coding agents in developer workflows (Norlund et al., 2021, Adewumi et al., 2021, Baumann et al., 22 Apr 2026, King et al., 12 Jun 2026, Garg et al., 10 Oct 2025, Patel et al., 22 May 2025, Zhou et al., 24 Oct 2025).