---
title: 'TMAX-15K Dataset: Multi-Domain Benchmark'
url: https://www.emergentmind.com/topics/tmax-15k-dataset
type: topic
---

# TMAX-15K Dataset: Multi-Domain Benchmark

TMAX-15K refers to distinct, large-scale datasets spanning diverse domains, each designed for nuanced, high-value research in their respective fields: (1) computer vision for autonomous driving, (2) language-model-driven terminal agent RL, and (3) social network analysis of conspiracy-related behavior on Twitter. Each TMAX-15K dataset defines new methodological standards and benchmarks, characterized by unique collection, annotation, and benchmarking strategies.

## 1. Computer Vision Dataset for Autonomous Driving

TMAX-15K ("Traffic-weather and light-level AUTOMotive eXperience, 15,000 video sequences") is a benchmark dataset for weather, light-level, and street-surface classification aimed at robustifying vision algorithms in harsh driving scenarios [2104.14042]. It specifically addresses the lack of public, multi-label, automotive datasets for visual perception in Level-2+ ADAS and autonomous driving.

- **Dataset Design**: Recorded over 24 months in Germany, utilizing an industrial-grade front-windshield RCCC camera ($1024\times1084$, 10-bit, 25 fps), TMAX-15K consists of 15,000 video clips (5–15 s each), from which 60,000 stratified key frames are sampled.
- **Labeling**: Each image is annotated with three orthogonal labels: weather (clear, rain, snow), light level (bright, moderate, low), and street type (asphalt, grass, cobblestone), yielding $3\times3\times3=27$ possible label combinations, but the dataset focuses on 9 "physical" classes.
- **Distillation & Active Learning**: Redundancy is minimized via an active learning protocol using an MLP loss prediction module $\mathcal{L}_{\rm loss} = \tfrac1N\sum_{i=1}^N |g_\phi(f_\theta(x_i)) - l_i|$. Iterative selection identifies high-uncertainty frames for manual annotation, resulting in a distilled set of 1,165 de-identified images retaining balanced distribution across all label axes.

| Label category | Class        | Count | % of 1,165 |
|---------------|--------------|-------|------------|
| Weather       | clear        | 390   | 33.5%      |
|               | rain         | 385   | 33.1%      |
|               | snow         | 390   | 33.5%      |
| Light         | bright       | 390   | 33.5%      |
|               | moderate     | 385   | 33.1%      |
|               | low          | 390   | 33.5%      |
| Street        | asphalt      | 583   | 50.0%      |
|               | grass        | 291   | 25.0%      |
|               | cobblestone  | 291   | 25.0%      |

- **Privacy**: All faces and license plates are blurred; no GPS or other personal data is shared.
- **Format**: 224$\times$224 px grayscale JPEGs (RCCC→luminance); metadata stored as JSON.
- **Benchmarking**: ResNet-18 with multi-task heads achieves an overall F₁=0.772 (weather and light labels), indicating the difficulty and unsaturated nature of the benchmark.

## 2. RL Terminal Agent Environment Suite

TMAX-15K ("TMAX: Terminal-agent environments, 14,600 tasks") is a synthetic, structured reinforcement learning environment suite curated to advance RL-based language model agents for Unix command-line tasks [2606.23321].

- **Dataset Construction**: Each environment is a Docker container seeded with artifacts (source files, binaries, images, audio) and auto-generated initialization instructions. Tasks span nine axes: domain, skill type, primitive skills, persona, language, task complexity, command complexity, fixture kind, and verifier kind—with only 14,600 realized task signatures from an O($10^7$) combinatorial space.
- **Interaction Protocol**: Agents interact by emitting shell commands, code snippets, or launching services in the sandbox; at episode end, a task-specific verifier returns a scalar reward in $[0, 1]$.
- **Axes Sampling**:
  - Domains: e.g., security, debugging, data_science, etc.
  - Persona: e.g., “bioinformatics analyst,” “DevOps engineer”
  - Task/Command complexity: granularity from 3–60 command steps, with variable composition
  - Verifier kind: exact_text, metric_threshold, adversarial_corpus, fuzz_equivalence, multi_protocol
- **Balance & Coverage**: Domain balance = 0.998 (ideal 1.0), skill-type balance = 0.732.
- **Benchmark Comparison**: Pass@1 for Gemini-3-Flash-Preview on 250 held-out tasks is 42%, underscoring the suite’s difficulty; pass@8 is 53%.
- **De-contamination Testing**: 0% n-gram (n=13) overlap with prior public benchmarks (Terminal-Bench 2.0, TB-Lite).
- **Release**: Full public access to data, code, environments, and evaluation tools under a permissive MIT-style license. JSON+tarball per-task format, Dockerfile reproducibility.

## 3. Twitter Social Network and Behavioral Analysis Dataset

TMAX-15K in social analysis is an anonymized Twitter dataset for comparative study of conspiracy-related activity, comprising 15,000 users and ~37.5M tweets [2308.15154].

- **Collection Method**:
  - *Conspiratorial users*: Identified via a query-independent like-expansion from 26 MBFC-tagged “conspiracy” website accounts; strict filtering retains users who liked $\geq25$ posts across $\geq4$ distinct seed accounts.
  - *Controls*: Randomly sampled but hashtag-matched, time-matched, and purged of overlap or seed-connected users.
- **Dataset Composition**: 7,394 conspiracy and 7,394 control users; up to 3,200 tweets/user; tweets span 2008-2022.
- **Botometer Filtering**: Both groups are overwhelmingly organic ($\ll2\%$ bots).
- **Feature Extraction**:
  - Profile (credibility): $16$ scalars (e.g., followers/age, verification)
  - Activity (initiative): reply/retweet/tweet ratios, word entropy
  - Adaptability: vocabulary novelty, temporal dynamics
  - Psycholinguistics: emotion/sentiment lexicons, Big Five, LIWC categories
  - Derivative topic modeling (CorEx with hashtag anchoring)
- **Key Results**:
  - Classifier (LightGBM, all features): $F_1=0.94$ (Precision=$0.94$, Recall=$0.94$)
  - Top discriminators: bio length, unique-word entropy, reply ratio, #URLs in bio, followings
  - Psycholinguistic: conspirators display lower “need for stability,” higher “need for curiosity,” greater informal language.
- **Release & Ethics**: Released as tweet and user IDs only, for Twitter API rehydration; CC BY 4.0; full compliance with Twitter Developer Policy.

## 4. Comparative Summary Table

| Dataset Area | Core Focus                                   | Size/Content                           |
|--------------|----------------------------------------------|----------------------------------------|
| Autonomous driving vision [2104.14042]   | Scene/weather/light/street classification | 1,165 images (distilled from 60k frames, 15k videos) |
| RL terminal environments [2606.23321]    | LLM-driven shell/code RL tasks           | 14,600 Docker-based environments       |
| Twitter social behavior [2308.15154]     | Conspiracy/control user comparison      | 15,000 users, 37.5M tweets             |

## 5. Research Utility and Impact

Each TMAX-15K dataset fulfills a critical gap in its domain:

- The automotive vision TMAX-15K sets a new standard for comprehensive, multi-label testing of machine perception under adverse driving conditions, demonstrating that conventional architectures (e.g., ResNet-18) remain unsaturated when confronted by real-world edge cases.
- The terminal-RL TMAX-15K suite enables large-scale, diversified, and fine-grained curriculum learning for LM agents in command-line and software environments, facilitating new research into complex, multi-modal and long-horizon RL.
- The social-Twitter TMAX-15K delivers query-independent, high-fidelity behavioral comparisons between conspiratorial and control groups, supporting reproducible supervised learning and providing rich feature benchmarks for social-linguistic inference.

A plausible implication is that the TMAX-15K datasets will serve as foundation benchmarks for both methodological and empirical investigations across machine perception, language-agent RL, and behavioral social computing.

## 6. Access, Licensing, and Data Format

- **Autonomous driving**: 224$\times$224 JPEGs + JSON annotation; $\approx$200 MB. Download and license specifics in [2104.14042].
- **RL terminal environments**: Full metadata in JSON, task files in tarballs, playground Docker images; $\approx$10 GB. MIT-style license.
- **Twitter social corpus**: Released as tweet and user IDs (for rehydration), CC BY 4.0 license.

All three datasets provide standardized, privacy-compliant data releases with public documentation, catalyzing reproducibility and extensibility for future research.

Source: https://www.emergentmind.com/topics/tmax-15k-dataset