FLAME Dataset Series

• FLAME dataset series are curated collections designed for supervised learning, benchmarking, and method development in high-stakes domains.
• The wildfire branch includes three releases featuring multimodal UAV data enabling fire detection, segmentation, and spatiotemporal analysis.
• The financial branch provides Chinese-language benchmarks via FLAME-Cer and FLAME-Sce to evaluate LLM performance in certification and business scenarios.

The FLAME dataset series refers to a family of curated, annotated datasets specifically designed to support supervised learning, benchmarking, and method development for high-stakes domains. Distinct FLAME datasets address separate scientific domains, including the analysis of wildland fire imagery via unmanned aerial vehicle (UAV) sensors and the evaluation of LLMs in the financial sector. The most established and technically mature branch focuses on UAV-based wildland fire detection and segmentation, with three main releases (FLAME 1, FLAME 2, FLAME 3); an emergent branch (FLAME for Financial LLM Assessment, 2025) serves as a comprehensive evaluation suite for Chinese financial LLMs.

1. FLAME Wildland Fire UAV Imagery Dataset Series

1.1 Evolution and Scope

The core FLAME wildfire dataset family comprises sequential, publicly released collections capturing prescribed burns across multiple environments, sensor payloads, and annotation schemas. Each major version augments previous releases in data richness and processing capabilities. The following table summarizes the technical characteristics of the three generations:

FLAME Release	Modalities and Key Attributes	Year (Paper)
FLAME 1	RGB + non-radiometric thermal (JPEG; white-hot, green-hot, fusion); 1 burn, ~47k frames, ~2k masks	2020 (Shamsoshoara et al., 2020)
FLAME 2	Paired RGB/thermal (JPEG); 5 burns, ~53k pairs; pre-burn orthomosaics, weather data, 3D point clouds	2022 (as cited (Hopkins et al., 2024))
FLAME 3	Nadir/oblique RGB, radiometric thermal (TIFF), standard thermal JPEG; 6 burns; automation pipeline; 14k+ images	2024 (Hopkins et al., 2024)

Editor’s term: “FLAME-wildfire series” refers collectively to FLAME 1, 2, and 3.

1.2 Data Collection and Modalities

FLAME datasets were acquired using commercial UAV platforms (DJI Matrice, Autel EVO II, Mavic 2 EA, etc.) equipped with co-mounted RGB and thermal cameras:

• FLAME 1 captured aerial RGB and color-mapped non-radiometric thermal video over a single forest slash pile burn, using DJI Matrice platforms and FLIR Vue Pro R for thermal data.
• FLAME 2 expanded to side-by-side, temporally synchronized RGB and thermal interfaces over five prescribed burns, recording ~53k paired examples for image-based tasks and supplementing with environmental metadata.
• FLAME 3 introduced true radiometric thermal imaging (GeoTIFF with Planck calibration constants) captured simultaneously with high-resolution RGB (4000×3000 to 8000×6000 px), enabling per-pixel temperature mapping and advanced environmental georeferencing via RTK GNSS. Multiple drones and multi-burn, multi-environment deployments (pine, grass, sagebrush, oak woodland) were captured.

Calibration, photogrammetric control, field-of-view alignment, and automated preprocessing (frame extraction, radiometric conversion) are fully documented (Hopkins et al., 2024).

1.3 Annotation Schema and Machine Learning Tasks

• Frame-level binary classification: All generations offer fire/no-fire frame labels. These are algorithmically seeded (e.g., via temperature threshold in FLAME 3) and human-verified.
• Pixel-wise fire segmentation: FLAME 1 provides 2,003 high-resolution manual fire masks (3480×2160 px), while FLAME 3 enables generation of ground truth from radiometric thresholds.
• Paired/multimodal inputs: FLAME 2/3 provide precisely aligned RGB and thermal (or radiometric) data for multimodal learning.
• Temporal modeling: Especially in FLAME 3, nadir plot time series (0.2 Hz) and oblique video (25 Hz) support spatiotemporal model development (e.g., rate-of-spread estimation).

1.4 Baseline Methods and Performance Metrics

Baseline experiments leverage convolutional (Xception, U-Net, MobileNet-v2) and recurrent (GRU) neural architectures, with task-specific design:

• Binary frame or clip classification is evaluated using accuracy, precision, recall, F1-score, and area under curve (AUC);
  • FLAME 1: 76.23% test accuracy; up to 91.99% precision for segmentation (Shamsoshoara et al., 2020)
  • FLAME 3: 90.95% accuracy, F1 ≈ 91.7% using RGB+TIFF inputs (Hopkins et al., 2024)
• Semantic segmentation is assessed with mean IoU, specificity, and manual ground-truth masks.
• Frame subset efficiency (see Section 1.5) using AccSampler (Zhao et al., 2024): with salient frame selection, only 8 out of 64 frames (87.5% reduction) retain 87.3% accuracy, and 20 frames support 91.0%, outperforming the full-clip baseline.

2. Dataset Processing, Policy Networks, and Efficient Inference

2.1 Frame Reduction and Redundancy Management

AccSampler (Zhao et al., 2024) is a lightweight video understanding framework that compresses UAV video streams to minimal, information-rich subsets using sequential policy networks:

• Station-point mechanism: Uniformly samples M context frames per clip, extracting future context features via MobileNet-v2.
• ClipMixup compresses temporally adjacent frames using mixed-sample interpolation ($\alpha=0.3$, Eqns. 1–2).
• Policy Network employs differentiable action selection (Gumbel-softmax, Eqns. 6–7) to adaptively determine k-tuple aggregations, balancing accuracy and computational cost.
• Multi-loss objective: Combines cross-entropy, action-space balancing, and GFLOP penalty terms (Eqn. 10; $\beta=0.3$, $\gamma=0.1$).

The methodology reduces video-level floating-point operation count (GFLOPs) by 9.4–13× and increases test accuracy by 3–3.7 percentage points versus recurrent-only baselines. The refined dataset is suitable for more efficient downstream training (Zhao et al., 2024).

2.2 Downstream and Temporal Tasks

FLAME 3 enables temperature-field regression, time-series analysis of burn plots, and rapid energy release estimation, supported by high-precision georeferencing and per-pixel thermal calibration (Hopkins et al., 2024). The per-frame radiometric temperature derivation follows:

$T_{ij} = \frac{K_2}{\ln(\frac{K_1}{L_{ij} + 1})}$

where $K_1$, $K_2$ are Planck constants from TIFF metadata and $L_{ij}$ is spectral radiance extracted via camera gain/offset.

3. FLAME for Financial LLM Benchmarking

3.1 Domain and Purpose

The FLAME dataset for financial LLM evaluation is distinct from the wildfire imagery series. It consists of two comprehensive Chinese-language benchmarks: FLAME-Cer (qualification certification) and FLAME-Sce (scenario-based business applications) (Guo et al., 3 Jan 2025).

3.2 FLAME-Cer: Certification Evaluation

Covers 14 financial certifications, spanning accounting, auditing, risk management, regulation, insurance, and economics. It comprises ≈16,000 multiple-choice questions, all manually reviewed for authenticity, accuracy, and syllabus coverage. Each question is graded for professional significance (core, significant, auxiliary), with 40% designated as core by industry experts.

3.3 FLAME-Sce: Scenario-Based Evaluation

Encapsulates a multi-level scenario taxonomy—10 primary, 21 secondary, ≈100 tertiary business application tasks for LLM ability measurement, totaling >5,000 evaluation items. Scenarios include document generation, customer profiling, compliance Q&A, financial risk controls, and data computation, each with manual, multi-dimensional scoring guidelines (accuracy, completeness, relevance, instruction compliance).

4. Access, Licensing, and Usage Guidance

• FLAME-wildfire datasets: Public subsets are available via IEEE DataPort (DOI: 10.21227/qad6-r683), Kaggle (FLAME 3 CV subset), or upon direct request for full sets (Shamsoshoara et al., 2020, Hopkins et al., 2024).
• Annotation protocols: Manual (SME) or algorithmic (temperature threshold plus review), with binary fire/no-fire as the primary label type. Segmentation masks exist for a subset (notably in FLAME 1).
• Financial FLAME: Available on GitHub (https://github.com/FLAME-ruc/FLAME/) with documentation, annotation standards, evaluation scripts, and ongoing updates. Usage is subject to the stated repository license (Guo et al., 3 Jan 2025).

5. Limitations and Research Considerations

• Wildland fire FLAME datasets are primarily focused on prescribed, pile-burn scenarios; free-burning, crown, or urban fire events are not included (Shamsoshoara et al., 2020, Hopkins et al., 2024).
• Annotation for fire-front segmentation is limited in scale (2,003 masks in FLAME 1), with plans for algorithmic mask generation using radiometric TIFFs in FLAME 3.
• Environmental diversity (weather, vegetation type) increases through the series, but temporal length, number of distinct burns, and frame rates are not always explicitly reported—this limits replicability and generalization analysis (Zhao et al., 2024).
• Financial FLAME is centered on Chinese certifications and workflows; generalizability to other regulatory or linguistic contexts is not addressed (Guo et al., 3 Jan 2025).

6. Research Impact and Future Directions

The FLAME dataset series enables:

• High-fidelity, open-source benchmarks for aerial wildfire detection, segmentation, and assessment adaptable to efficient deep learning techniques.
• Automated frame selection and distillation pipelines (e.g., via AccSampler), reducing annotation and computational costs while preserving model accuracy (Zhao et al., 2024).
• Radiometric datasets (FLAME 3) that stimulate research in per-pixel thermal modeling, multi-modal fusion, and robust geospatial surveillance (Hopkins et al., 2024).
• Comprehensive, expert-verified benchmarks for financial LLMs, directly informing regulatory-compliant deployment strategies and task-specific NLP evaluation in finance (Guo et al., 3 Jan 2025).

A plausible implication is that further expansion into diverse burn types, unstructured wildfires, and multi-institutional financial tasks could broaden applicability and optimize model generalization, while harmonized annotation protocols would promote cross-domain benchmarking and transfer learning.