---
title: 'KITTI-A: Diverse Benchmark Interpretations'
url: https://www.emergentmind.com/topics/kitti-a
type: topic
---

# KITTI-A: Diverse Benchmark Interpretations

Searching arXiv for the cited papers to ground the terminology and references.
{"query":"id:2008.08063 OR id:1904.01416 OR id:2507.09214 OR id:2109.12979 OR id:2509.08982", "max_results": 10}
KITTI-A is not a single canonical benchmark designation in the arXiv literature. Across papers that mention the expression directly or are queried about it, the label denotes different technical objects depending on context: in AB3DMOT it refers to the “Average” tracking metrics introduced for KITTI, not to a dataset or split; in SemanticKITTI it can denote the annotated semantic LiDAR extension of KITTI; in recent anomaly-detection work it corresponds to KITTI-AR, an augmented-reality stereo dataset; and in some odometry or registration contexts it is used as shorthand for KITTI autonomous-driving protocols rather than for a distinct corpus [2008.08063; 1904.01416; 2507.09214; 2109.12979; 2509.08982]. The term is therefore polysemous and must be interpreted from the surrounding benchmark, metric, and task definitions.

## 1. Terminological status in the literature

Several papers explicitly indicate that “KITTI-A” is absent from their formal nomenclature and that results are instead reported on the official KITTI benchmarks. MonoATT evaluates the standard KITTI 3D object detection benchmark and states that the term is not used in the paper [2303.13018]. Uni3D likewise states that there is no dataset or setting named “KITTI-A” in that work [2303.06880]. UnFlow targets KITTI 2012 and KITTI 2015 optical-flow benchmarks and also does not use the expression [1711.07837]. Taken together, this suggests that “KITTI-A” is not a universally standardized KITTI sub-benchmark, but a context-dependent label that appears in paper-specific or community-specific usage.

| Usage of “KITTI-A” | Technical referent | Representative source |
|---|---|---|
| “Average” evaluation on KITTI | AMOTA, AMOTP, sAMOTA for 3D MOT | AB3DMOT [2008.08063] |
| Annotated semantic KITTI | SemanticKITTI semantic LiDAR dataset | SemanticKITTI [1904.01416] |
| Naming variant | KITTI-AR augmented-reality stereo dataset | S3AD [2507.09214] |
| Autonomous-driving shorthand | KITTI odometry setting | CT-ICP [2109.12979] |
| Informal protocol shorthand | KITTI registration protocol with 10 m-separated scan pairs | iMatcher [2509.08982] |

This variation is the central fact about the term. Any technical reading of “KITTI-A” that omits task context—tracking, semantic segmentation, anomaly detection, odometry, or registration—is underspecified.

## 2. KITTI-A as the “Average” metrics of AB3DMOT

In AB3DMOT, “KITTI-A” does not denote a dataset or a special split of KITTI. The “A” refers to the Average metrics introduced by the authors and applied on KITTI: AMOTA, AMOTP, and sAMOTA [2008.08063]. These metrics were proposed because KITTI traditionally evaluates trackers at a single confidence threshold, does not consider confidence scores, and only supports 2D evaluation. The AB3DMOT evaluation tool instead matches tracks and ground truth directly in 3D space using 3D IoU with a threshold of \(0.25\), and it summarizes performance across a 40-point recall sweep.

The core quantities are defined as recall-averaged extensions of classic MOT metrics:
$$
\mathrm{AMOTA}=\frac{1}{R}\sum_{r\in\mathcal R}\mathrm{MOTA}(r),\qquad
\mathrm{AMOTP}=\frac{1}{R}\sum_{r\in\mathcal R}\mathrm{MOTP}(r),
$$
with \(R=|\mathcal R|=40\). The scaled variant sAMOTA maps AMOTA into \([0\%,100\%]\) for easier interpretability. In this usage, “KITTI-A” therefore means “KITTI evaluated with Average metrics,” not “KITTI-A” as a corpus.

AB3DMOT couples a 3D Kalman filter with the Hungarian algorithm. The tracker consumes 3D detections from LiDAR point clouds, uses a constant-velocity 3D state with position, size, velocity, and orientation, and performs data association using a geometric overlap cost based on 3D IoU [2008.08063]. On the KITTI validation set, car subset, under 3D evaluation, the paper reports for AB3DMOT: sAMOTA \(=93.28\), AMOTA \(=45.43\), AMOTP \(=77.41\), MOTA \(=86.24\), MOTP \(=78.43\), IDS \(=0\), FRAG \(=15\), and FPS \(=207.4\). A recurrent misconception is thus directly contradicted by the paper itself: in this line of work, KITTI-A is not a split.

## 3. KITTI-A as SemanticKITTI

SemanticKITTI is described as being informally referred to as “KITTI-A” when the intended meaning is the KITTI annotated semantic LiDAR dataset [1904.01416]. In this interpretation, the term refers neither to AB3DMOT’s metrics nor to an odometry protocol, but to a semantic-scene-understanding resource built on top of the KITTI Vision Odometry Benchmark.

SemanticKITTI annotates all 22 sequences of the KITTI Vision Odometry dataset, uses training sequences 00–10 and test sequences 11–21, and provides dense point-wise labels for the complete \(360^\circ\) field of view of a Velodyne HDL-64E [1904.01416]. The dataset contains 23,201 training scans and 20,351 test scans, totaling 43,552 scans and approximately 4,549 million labeled points overall. Annotation required more than 1,700 hours and covers 518 spatial tiles. The label taxonomy contains 28 classes, including moving versus non-moving variants for six movable categories.

This usage is tied to three benchmark tasks: single-scan semantic segmentation, multi-scan semantic segmentation, and semantic scene completion. For single-scan evaluation, moving classes are merged with static counterparts and three classes are ignored, yielding 19 evaluation classes. For multi-scan segmentation, models predict 25 evaluation classes. Semantic scene completion uses a \(256\times256\times32\) voxel grid at \(0.2\,\mathrm{m}\) resolution over a volume extending \(51.2\,\mathrm{m}\) ahead, \(25.6\,\mathrm{m}\) to each side, and \(6.4\,\mathrm{m}\) in height [1904.01416]. In this semantic interpretation, “KITTI-A” is best understood as a shorthand for a densely annotated LiDAR benchmark derived from KITTI odometry data.

## 4. KITTI-A as KITTI-AR in stereo 3D anomaly detection

In the S3AD paper, the clarification is explicit: “KITTI-A” in the query corresponds to KITTI-AR, and the paper consistently uses the name KITTI-AR rather than KITTI-A [2507.09214]. Here the “A” stands for Augmented Reality. This is a different meaning again, centered on open-world 3D anomaly detection rather than on semantic labeling or tracking metrics.

KITTI-AR extends KITTI by adding 97 new categories and about 6,000 pairs of stereo images. It is divided into KITTI-AR-ExD, containing 39 common categories used as extra training data, and KITTI-AR-OoD, containing 58 rare categories held out from training and used only for zero-shot evaluation [2507.09214]. The ExD training set has 4,038 samples, while the OoD evaluation set has 2,347 samples. The background images follow the commonly used KITTI split of 3,712 training samples and 3,769 validation samples. Experimental preprocessing crops 100 pixels from the top and resizes stereo inputs to \(288\times1280\).

The accompanying S3AD method decouples 2D and 3D supervision, introduces a class-agnostic foreground branch based on stereo disparity features, and defines target-level anomaly scores such as
$$
\operatorname{MSPF}(x)=c_{fg}-\max_{n=1}^{N}(\operatorname{softmax}(c_{norm})),
$$
and
$$
\operatorname{RbAF}(x)=c_{fg}-\frac{1}{N}\sum_{n=1}^{N}\sigma(c_{norm}),
$$
with \(\sigma(\cdot)=\operatorname{sigmoid}(\cdot)\) in the proposed method. Stereo depth follows the standard relation
$$
Z=\frac{f\,B}{d}.
$$
The paper reports that, on KITTI-AR-OoD, training with KITTI only yields \(\mathrm{AP3D}_{OoD}=9.09\); adding KITTI-AR-ExD 2D labels raises this to \(21.05\); and adding KITTI-AR-ExD 3D boxes raises it to \(74.35\) [2507.09214]. Under this usage, “KITTI-A” should be read as a naming variant for KITTI-AR rather than as an official KITTI benchmark name.

## 5. KITTI-A as autonomous-driving odometry or registration shorthand

A further usage occurs in odometry and registration contexts, where “KITTI-A” refers not to a new dataset but to KITTI autonomous-driving evaluation settings. In CT-ICP, the context and scope explicitly state that KITTI-A refers to autonomous driving odometry on the KITTI benchmark [2109.12979]. CT-ICP is a LiDAR-only odometry and SLAM method with a continuous-time formulation within scans and discontinuity between scans, together with pure LiDAR loop detection based on elevation-image 2D matching. On the KITTI odometry benchmark, it reports average Relative Translation Error of approximately \(0.59\%\) and runtime of approximately \(60\,\mathrm{ms}\) per scan on a single-thread CPU [2109.12979].

In point-cloud registration, iMatcher notes that the paper itself does not use the name “KITTI-A,” but that some papers informally use it for the standard Predator/GeoTransformer KITTI odometry protocol [2509.08982]. That protocol uses train sequences 0–5, validation sequences 6–7, and test sequences 8–10; evaluates only scan pairs separated by at least \(10\,\mathrm{m}\) along the trajectory; and refines the provided poses with ICP before evaluation. Under that setup, iMatcher reports, for GeoTransformer with iMatcher, RTE \(=4.8\,\mathrm{cm}\), RRE \(=0.20^\circ\), and RR \(=99.8\%\), with inlier ratios of \(95.1\%\), \(97.2\%\), and \(97.7\%\) at 5000, 1000, and 250 correspondences, respectively [2509.08982].

This protocol-centered usage is narrower than the semantic or anomaly-detection meanings. It denotes a task configuration—autonomous-driving odometry or registration on KITTI—rather than a separately distributed dataset.

## 6. Interpretation, misconceptions, and citation practice

The most persistent misconception is to treat KITTI-A as a single public split or formally standardized benchmark. The cited literature does not support that interpretation. In AB3DMOT, the term explicitly denotes Average metrics on KITTI and not a split [2008.08063]. In S3AD, it corresponds to KITTI-AR [2507.09214]. In SemanticKITTI, it can refer informally to the annotated semantic LiDAR dataset [1904.01416]. In CT-ICP and iMatcher, it can denote autonomous-driving odometry or a specific registration protocol [2109.12979; 2509.08982].

A second source of confusion is that several KITTI papers use only the official benchmark names and reject the term entirely. MonoATT reports on the official KITTI 3D object detection benchmark, specifically for the car class in the monocular image-only setting, and states that there is no use of the term “KITTI-A” in the paper [2303.13018]. Uni3D states that there is no dataset or setting named “KITTI-A” in that work [2303.06880]. UnFlow likewise evaluates KITTI 2012 and KITTI 2015 rather than any “KITTI-A” benchmark [1711.07837]. This suggests that precise citation should always resolve the ambiguity explicitly.

The technically correct practice is therefore to name the intended referent rather than rely on the bare label. If the subject is 3D multi-object tracking evaluation, the intended object is AB3DMOT’s Average metrics—AMOTA, AMOTP, and sAMOTA—on KITTI. If the subject is LiDAR semantics, the intended object is SemanticKITTI. If the subject is open-world stereo 3D anomaly detection, the intended object is KITTI-AR. If the subject is LiDAR odometry or registration, the intended object is the specific KITTI protocol being used. Without that disambiguation, “KITTI-A” is technically incomplete.

Source: https://www.emergentmind.com/topics/kitti-a