Papers
Topics
Authors
Recent
Search
2000 character limit reached

TMAX-15K Dataset: Multi-Domain Benchmark

Updated 27 June 2026
  • TMAX-15K is a multi-domain benchmark offering uniquely curated datasets for computer vision, RL terminal environments, and social behavior analysis.
  • The autonomous driving dataset delivers 1,165 annotated frames with detailed weather, light, and street labels to challenge real-world perception algorithms.
  • The RL and Twitter datasets leverage advanced sampling, active learning, and filtering techniques to establish new standards in language-driven tasks and behavioral inference.

TMAX-15K refers to distinct, large-scale datasets spanning diverse domains, each designed for nuanced, high-value research in their respective fields: (1) computer vision for autonomous driving, (2) language-model-driven terminal agent RL, and (3) social network analysis of conspiracy-related behavior on Twitter. Each TMAX-15K dataset defines new methodological standards and benchmarks, characterized by unique collection, annotation, and benchmarking strategies.

1. Computer Vision Dataset for Autonomous Driving

TMAX-15K ("Traffic-weather and light-level AUTOMotive eXperience, 15,000 video sequences") is a benchmark dataset for weather, light-level, and street-surface classification aimed at robustifying vision algorithms in harsh driving scenarios (Dhananjaya et al., 2021). It specifically addresses the lack of public, multi-label, automotive datasets for visual perception in Level-2+ ADAS and autonomous driving.

  • Dataset Design: Recorded over 24 months in Germany, utilizing an industrial-grade front-windshield RCCC camera (1024×10841024\times1084, 10-bit, 25 fps), TMAX-15K consists of 15,000 video clips (5–15 s each), from which 60,000 stratified key frames are sampled.
  • Labeling: Each image is annotated with three orthogonal labels: weather (clear, rain, snow), light level (bright, moderate, low), and street type (asphalt, grass, cobblestone), yielding 3×3×3=273\times3\times3=27 possible label combinations, but the dataset focuses on 9 "physical" classes.
  • Distillation & Active Learning: Redundancy is minimized via an active learning protocol using an MLP loss prediction module Lloss=1Ni=1Ngϕ(fθ(xi))li\mathcal{L}_{\rm loss} = \tfrac1N\sum_{i=1}^N |g_\phi(f_\theta(x_i)) - l_i|. Iterative selection identifies high-uncertainty frames for manual annotation, resulting in a distilled set of 1,165 de-identified images retaining balanced distribution across all label axes.
Label category Class Count % of 1,165
Weather clear 390 33.5%
rain 385 33.1%
snow 390 33.5%
Light bright 390 33.5%
moderate 385 33.1%
low 390 33.5%
Street asphalt 583 50.0%
grass 291 25.0%
cobblestone 291 25.0%
  • Privacy: All faces and license plates are blurred; no GPS or other personal data is shared.
  • Format: 224×\times224 px grayscale JPEGs (RCCC→luminance); metadata stored as JSON.
  • Benchmarking: ResNet-18 with multi-task heads achieves an overall F₁=0.772 (weather and light labels), indicating the difficulty and unsaturated nature of the benchmark.

2. RL Terminal Agent Environment Suite

TMAX-15K ("TMAX: Terminal-agent environments, 14,600 tasks") is a synthetic, structured reinforcement learning environment suite curated to advance RL-based LLM agents for Unix command-line tasks (Ivison et al., 22 Jun 2026).

  • Dataset Construction: Each environment is a Docker container seeded with artifacts (source files, binaries, images, audio) and auto-generated initialization instructions. Tasks span nine axes: domain, skill type, primitive skills, persona, language, task complexity, command complexity, fixture kind, and verifier kind—with only 14,600 realized task signatures from an O(10710^7) combinatorial space.
  • Interaction Protocol: Agents interact by emitting shell commands, code snippets, or launching services in the sandbox; at episode end, a task-specific verifier returns a scalar reward in [0,1][0, 1].
  • Axes Sampling:
    • Domains: e.g., security, debugging, data_science, etc.
    • Persona: e.g., “bioinformatics analyst,” “DevOps engineer”
    • Task/Command complexity: granularity from 3–60 command steps, with variable composition
    • Verifier kind: exact_text, metric_threshold, adversarial_corpus, fuzz_equivalence, multi_protocol
  • Balance & Coverage: Domain balance = 0.998 (ideal 1.0), skill-type balance = 0.732.
  • Benchmark Comparison: Pass@1 for Gemini-3-Flash-Preview on 250 held-out tasks is 42%, underscoring the suite’s difficulty; pass@8 is 53%.
  • De-contamination Testing: 0% n-gram (n=13) overlap with prior public benchmarks (Terminal-Bench 2.0, TB-Lite).
  • Release: Full public access to data, code, environments, and evaluation tools under a permissive MIT-style license. JSON+tarball per-task format, Dockerfile reproducibility.

3. Twitter Social Network and Behavioral Analysis Dataset

TMAX-15K in social analysis is an anonymized Twitter dataset for comparative study of conspiracy-related activity, comprising 15,000 users and ~37.5M tweets (Gambini et al., 2023).

  • Collection Method:
    • Conspiratorial users: Identified via a query-independent like-expansion from 26 MBFC-tagged “conspiracy” website accounts; strict filtering retains users who liked 25\geq25 posts across 4\geq4 distinct seed accounts.
    • Controls: Randomly sampled but hashtag-matched, time-matched, and purged of overlap or seed-connected users.
  • Dataset Composition: 7,394 conspiracy and 7,394 control users; up to 3,200 tweets/user; tweets span 2008-2022.
  • Botometer Filtering: Both groups are overwhelmingly organic (2%\ll2\% bots).
  • Feature Extraction:
    • Profile (credibility): $16$ scalars (e.g., followers/age, verification)
    • Activity (initiative): reply/retweet/tweet ratios, word entropy
    • Adaptability: vocabulary novelty, temporal dynamics
    • Psycholinguistics: emotion/sentiment lexicons, Big Five, LIWC categories
    • Derivative topic modeling (CorEx with hashtag anchoring)
  • Key Results:
    • Classifier (LightGBM, all features): 3×3×3=273\times3\times3=270 (Precision=3×3×3=273\times3\times3=271, Recall=3×3×3=273\times3\times3=272)
    • Top discriminators: bio length, unique-word entropy, reply ratio, #URLs in bio, followings
    • Psycholinguistic: conspirators display lower “need for stability,” higher “need for curiosity,” greater informal language.
  • Release & Ethics: Released as tweet and user IDs only, for Twitter API rehydration; CC BY 4.0; full compliance with Twitter Developer Policy.

4. Comparative Summary Table

Dataset Area Core Focus Size/Content
Autonomous driving vision (Dhananjaya et al., 2021) Scene/weather/light/street classification 1,165 images (distilled from 60k frames, 15k videos)
RL terminal environments (Ivison et al., 22 Jun 2026) LLM-driven shell/code RL tasks 14,600 Docker-based environments
Twitter social behavior (Gambini et al., 2023) Conspiracy/control user comparison 15,000 users, 37.5M tweets

5. Research Utility and Impact

Each TMAX-15K dataset fulfills a critical gap in its domain:

  • The automotive vision TMAX-15K sets a new standard for comprehensive, multi-label testing of machine perception under adverse driving conditions, demonstrating that conventional architectures (e.g., ResNet-18) remain unsaturated when confronted by real-world edge cases.
  • The terminal-RL TMAX-15K suite enables large-scale, diversified, and fine-grained curriculum learning for LM agents in command-line and software environments, facilitating new research into complex, multi-modal and long-horizon RL.
  • The social-Twitter TMAX-15K delivers query-independent, high-fidelity behavioral comparisons between conspiratorial and control groups, supporting reproducible supervised learning and providing rich feature benchmarks for social-linguistic inference.

A plausible implication is that the TMAX-15K datasets will serve as foundation benchmarks for both methodological and empirical investigations across machine perception, language-agent RL, and behavioral social computing.

6. Access, Licensing, and Data Format

  • Autonomous driving: 2243×3×3=273\times3\times3=273224 JPEGs + JSON annotation; 3×3×3=273\times3\times3=274200 MB. Download and license specifics in (Dhananjaya et al., 2021).
  • RL terminal environments: Full metadata in JSON, task files in tarballs, playground Docker images; 3×3×3=273\times3\times3=27510 GB. MIT-style license.
  • Twitter social corpus: Released as tweet and user IDs (for rehydration), CC BY 4.0 license.

All three datasets provide standardized, privacy-compliant data releases with public documentation, catalyzing reproducibility and extensibility for future research.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to TMAX-15K Dataset.