--- title: CompACT Pedestrian Detection url: https://www.emergentmind.com/topics/complexity-aware-pedestrian-detection-compact type: topic --- # CompACT Pedestrian Detection Complexity-Aware Pedestrian Detection (CompACT) is a framework for constructing cascaded pedestrian detectors that explicitly optimize a trade-off between detection accuracy and computational complexity. CompACT enables the seamless integration of features possessing widely varying computational costs, including both classical image features and deep convolutional neural network (CNN) activations, within a single learned detector. By penalizing unnecessary computation, CompACT allocates inexpensive features to the early cascade stages and reserves complex, high-capacity features for later stages, when only a small subset of ambiguous input windows remains. This approach yields a detector that is both accurate and computationally efficient across challenging pedestrian detection benchmarks [1507.05348]. ## 1. Lagrangian Formulation: Joint Optimization of Accuracy and Complexity CompACT is motivated by the need to balance detection accuracy against computational constraints. Classical AdaBoost minimizes the empirical classification risk: \[ R_E[F] \simeq \frac{1}{|S|}\sum_{i=1}^{|S|} e^{-y_i F(x_i)}, \] where $F(x)$ is the score function, $y_i \in \{\pm 1\}$ are labels, and $S$ is the training set. CompACT introduces a second risk term, “complexity risk” $R_C[F]$, to quantify average computational cost. Detector learning is thus cast as: \[ \min_{F} R_E[F] \quad \text{s.t.} \; R_C[F] \leq \gamma \] which is equivalent, via a Lagrange multiplier $\eta \ge 0$, to minimizing the composite objective: \[ \mathcal{L}[F] = R_E[F] + \eta R_C[F]. \tag{1} \] $R_E[F]$ measures classification error, and $R_C[F]$ penalizes excessive computation, making the trade-off explicit. The complexity risk is defined analogously to $R_E[F]$: \[ R_C[F] \simeq \frac{1}{|S|}\sum_{i=1}^{|S|} \tau(\kappa(y_i, F(x_i))), \] with $\kappa(y, F(x)) = y\,\Omega(F(x))$ as the signed “complexity margin” and $\tau(v) = \max(0, -v)$ as a hinge-style loss. The per-sample implementation complexity $\Omega(F(x))$ is the average cost per window: \[ \Omega(F(x)) = \frac{1}{m}\sum_{k=1}^m r_k(x)\, \Omega(f_k), \quad r_k(x) = \prod_{j0), \tag{2} \] where $f_k$ are weak learners, $F_k = \sum_{j=1}^k f_j$, $T_j$ are thresholds, and $m$ is the number of cascade stages. ## 2. CompACT Boosting Algorithm and Learning Dynamics Learning proceeds via functional gradient descent on $\mathcal{L}[F]$. The negative gradient for a weak learner $g$ combines classification and computational cost: \[ D[g] = \frac{1}{|S|} \sum_{i=1}^{|S|} y_i \left[ \omega_i g(x_i) + \frac{\eta r_i \psi_i}{m+1}\Omega(g(x_i)) \right], \tag{3} \] where $\omega_i = e^{-y_i F(x_i)}$ and $\psi_i = -\tau'(y_i\,\Omega(F(x_i)))$. At each boosting round, the weak learner maximizing $D[g]$ is selected: - Compute $C_g = \sum_i y_i \omega_i g(x_i)$ - Compute $K_g = \sum_i y_i r_i \psi_i \Omega(g(x_i))$ - Score: $D[g] = (C_g + \eta K_g/(m+1))/|S|$ - Select $g^* = \arg\max_g D[g]$ - Find optimal step size $\alpha^*$ via 1-D line search minimizing $\mathcal{L}[F+\alpha g^*]$ - Update: $F \leftarrow F + \alpha^* g^*$ The algorithm penalizes complex features during early stages (when more windows remain), and relaxes this constraint in later stages as $m$ increases. This scheduling is a principled consequence of the joint loss in (1) [1507.05348]. ## 3. Modeling Feature Complexity CompACT accommodates features with heterogeneous computational costs, assigning each a cost $\Omega$ entering directly into the risk function: - **Pre-computed features**: Aggregate Channel Features (ACF), $\Omega=1$. - **Just-in-time (JIT) features**: Evaluated only on surviving windows at a given stage. - Self-Similarity (SS): $\Omega=2$ - Checkerboard (CB): $2\times2$ on 10 ACF channels, $\Omega=4$ - LDA basis filters: $3\times3$ PCA-like on ACF, $\Omega=9$ - Small CNN conv5 channels: $\Omega_{\text{CNN}}$ for the initial trigger (entire CNN forward-pass), then 1 per JIT feature. - CNN-Checkerboard (CNNCB): on conv5 feature maps, $\Omega=4$ per feature For CNNs, a “trigger” cost $\Omega_{\text{CNN}}$ is charged on first use per window, and zero for subsequent conv5 feature extractions within that window. This cost model enables complexity-aware scheduling of diverse feature types. ## 4. Embedded Cascade Structure and Feature Scheduling CompACT learns an embedded cascade of $M$ stages, each a depth-2 decision tree $f_k$ with threshold $T_k$. The cumulative score at stage $k$ is $F_k(x) = \sum_{j=1}^k f_j(x)$. A window is rejected if $F_k(x) + T_k \leq 0$: \[ \mathrm{DECIDE}(x) = \begin{cases} \text{reject} & \exists\,k:\; F_k(x)+T_k \le 0, \\ \text{accept} & F_M(x) + T_M > 0. \end{cases} \] Empirically, early cascade stages rely on cheap ACF features, intermediate stages incorporate SS and CB, while expensive CNNCB and CNN features are reserved for final stages. As most windows are eliminated early, only a small fraction incurs the cost of high-complexity features. ## 5. Integration of Deep Convolutional Neural Networks Deep CNN conv5-layer activations are integrated as another JIT feature family with a per-feature cost structure as previously described. CompACT typically refrains from selecting raw conv5 CNN features early in the cascade, instead favoring CNNCB filters when advantageous. To incorporate a large, ImageNet-pretrained CNN (e.g., AlexNet or VGG), the network is embedded as the final weak learner $g^*$ at stage $M$. Boosting computes the optimal step size $\alpha^*$ for this feature. During inference, the cascade processes early stages using fast features at approximately 4 fps (CPU), while only the ~10% of windows surviving after NMS propagate to the large CNN (adding about 2 fps processing on GPU). This architecture generalizes the combination of a proposal stage with a CNN, but operates as a unified single-pass system. ## 6. Empirical Results and Comparative Analysis CompACT was extensively evaluated on Caltech and KITTI pedestrian datasets using the log-average miss-rate (MR) over $[10^{-2}, 10^0]$ FPPI and AUC for KITTI. Runtime measurements employed a 2.1 GHz Intel Xeon (CPU) and NVIDIA K40M (GPU). **Table 1: Single-feature Cascades vs. CompACT on Caltech** | Method | log-avg MR [%] | time (s) | |----------------|---------------|----------| | ACF-only | 42.6 | 0.07 | | SS-only | 34.3 | 0.08 | | CB-only | 37.9 | 0.23 | | LDA-only | 37.2 | 0.16 | | CNN-only | 28.1 | 0.87 | | CNNCB-only | 26.9 | 2.05 | | CompACT-ACF | 32.2 | 0.11 | | CompACT-CNN | **23.8** | 0.28 | **Table 2: Large CNN as Final Stage (Caltech)** | Method | AlexNet MR | Δtime | VGG MR | Δtime | |------------------------------|------------|-------|--------|-------| | CompACT-small-CNN only | 23.8 | — | 23.8 | — | | + embedded large CNN-Alex | 15.0 | +0.10 | 14.8 | +0.10 | | + embedded large CNN-VGG | 11.8 | +0.25 | 11.8 | +0.25 | **Table 3: State-of-the-Art Comparison on Caltech "Reasonable"** | Detector | MR [%] | time (s) | |-------------------------------|----------|----------| | SpatialPooling+ (ECCV’14) | 25.4 | 5.0† | | Checkerboards (CVPR’15) | 25.6 | 4.0† | | R‐CNN (arXiv’15) | 32.9 | 2.0† | | Katamari (ECCV) | 24.8 | 0.5† | | CompACT‐Deep (embed VGG) | **11.8** | 2.0 | **Table 4: KITTI (Moderate) AUC and Timing** | Method | AUC [%] | time (s) | |----------------|---------|----------| | FilteredICF | 54.0 | 0.40 | | pAUCEnsT | 54.5 | 0.60 | | R‐CNN (KITTI) | 50.1 | 4.0 | | Regionlets | 61.2 | 1.0† | | CompACT‐Deep | **58.7**| 1.0 | †Excludes proposal-generation time or is reported end-to-end. Key observations: - CompACT's Lagrangian framework governs an explicit, tunable trade-off between accuracy and complexity via $\eta$. - Boosting ranks weak learners by a composite classification-plus-computation score, naturally allocating feature types across stages. - Embedding AlexNet or VGG as the final stage yields single-pass detectors that significantly outperform separate proposal+CNN pipelines in both accuracy and speed. ## 7. Implications and Significance CompACT operationalizes the notion of budgeted detection, allowing large, heterogeneous feature pools and optimizing for both accuracy and computational efficiency. The explicit complexity-awareness advances the state of the art on pedestrian detection benchmarks and enables practical, real-time detectors. A plausible implication is that similar frameworks could be leveraged for other detection tasks requiring heterogeneous feature integration and complexity management [1507.05348]. Source: https://www.emergentmind.com/topics/complexity-aware-pedestrian-detection-compact