--- title: 'ALLabel: Active Learning for Label Optimization' url: https://www.emergentmind.com/topics/allabel type: topic --- # ALLabel: Active Learning for Label Optimization ALLabel denotes a family of research formulations in which labels are treated as the primary object of optimization rather than as passive annotations. In the cited literature, the term appears explicitly as a three-stage active learning framework for LLM-based entity recognition [2509.07512], while related summaries also use it to describe multi-label active learning in remote sensing [2212.01165], label-plus-explanation acquisition pipelines [2305.12710], active label correction for semantic segmentation [2403.10820], and several automatic label placement systems in augmented reality, visualization, and cartography [1912.07105], [2405.10953], [2308.13540], [2507.22952]. This suggests that ALLabel is best understood as a research umbrella spanning label acquisition, label correction, label representation learning, and spatial label layout. ## 1. Scope and terminology Across the available papers, ALLabel is not a single standardized algorithm. It names, or is used to organize, several technically distinct problems: selecting which samples should be labeled, deciding how noisy or partial labels should be corrected, constructing high-value demonstration corpora for in-context learning, and placing textual labels in spatial media. The common thread is that the design variable is the label itself—its content, reliability, representation, or position. | Usage | Core mechanism | Representative paper | |---|---|---| | Multi-label active learning | Uncertainty plus clustering-based diversity | [2212.01165] | | Label-plus-explanation acquisition | Explanation generator, rationale-aware predictor, explanation-informed sampler | [2305.12710] | | Active label correction | Correction queries with superpixel look-ahead | [2403.10820] | | LLM demonstration selection | Three-stage diversity–similarity–uncertainty sampling | [2509.07512] | | Automatic label placement | Guidance-map, bitmap, RL, or LLM-driven spatial layout | [1912.07105], [2405.10953], [2308.13540], [2507.22952] | A recurrent misconception is to treat ALLabel as a single method family with a single objective. The literature instead partitions into at least four technical regimes. In active learning papers, the goal is annotation efficiency. In correction and refinement papers, the goal is robustness to noisy supervision. In label-placement papers, the goal is legibility and non-occlusion. In representation-learning work, the goal is to replace one-hot targets with learned label geometries. The shared emphasis on labels does not imply interchangeable formulations. ## 2. Multi-label active learning in remote sensing In remote sensing multi-label classification, ALLabel is instantiated as a two-stage active learning pipeline in which uncertainty scoring is followed by diversity enforcement [2212.01165]. The setting assumes an archive \(X=\{X_1,\dots,X_N\}\), a labeled set \(T^\tau\), an unlabeled pool \(U^\tau=X\setminus T^\tau\), and a deep neural network \(F\) with sigmoid outputs \(p^j=F(X_j)=[p_1^j,\dots,p_C^j]\). Training uses binary cross-entropy, \[ \mathcal{L}_{BCE}\big(F(X_j),y_j\big) = -\sum_{i=1}^C \Big[y_i^j \log(p_i^j) + (1-y_i^j)\log(1-p_i^j) \Big]. \] Three uncertainty criteria are introduced. Learning multi-label loss ordering (LL) attaches an auxiliary head \(F_L\) that predicts a scalar loss-rank score \(\hat l_j\) from hidden features and is trained with a margin-based pairwise ranking loss. Temporal prediction discrepancy (TPD) measures instability between consecutive active learning iterations through \[ \text{Unc}^{TPD}(X_j)=\|F^\tau(X_j)-F^{\tau-1}(X_j)\|_2. \] Magnitude of approximated gradient embeddings (MGE) constructs a pseudo-label \(\hat y_j\) by thresholding at \(0.5\), forms \[ G_j=(p^j-\hat y_j)h_j^\top \in \mathbb{R}^{C\times d}, \] and scores samples by \(\|g_j\|_2=\|(p^j-\hat y_j)\otimes h_j\|_2\). Diversity is imposed after uncertainty filtering: the top \(mb\) uncertain samples are clustered into \(b\) clusters with KMeans++, and the highest-uncertainty sample from each cluster is queried. The resulting query functions—LL+Clustering, TPD+Clustering, and MGE+Clustering—are evaluated on UCMerced and TreeSatAI with DenseNet-121, SGD, BCE, and simple geometric augmentations. On UCMerced, all three outperform random sampling, and MGE+Clustering is reported as slightly best overall; MGE+Clustering vs random gives \(+>2\%\) micro-F1 at 200 labeled \((79.87\%)\) and \(+>1\%\) at 400 labeled \((85.81\%)\), while LL+Clustering reaches \(86.29\%\) micro-F1 at 400 labeled. On TreeSatAI, LL+Clustering exceeds random by \(+>2\%\) micro-F1 at 18,000 labeled \((69.90\%)\), and for macro-F1 it reaches \(59.05\%\) at 14,000 labeled, \(+8\%\) over random. The study also reports that smaller query batch size \(b\) improves sample efficiency but increases total training time, and that warm-start is beneficial on UCMerced while cold-start is better overall on TreeSatAI [2212.01165]. The technical significance of this formulation is that it generalizes active learning from single-label and segmentation settings to remote-sensing multi-label outputs without collapsing the label space into a scalar confidence. Uncertainty is defined over multi-label loss ordering, temporal inconsistency, or gradient geometry, and diversity is enforced in deep feature space rather than in raw pixel space. ## 3. Label-plus-explanation active learning A second ALLabel usage treats explanations as first-class annotations, alongside labels, within the active learning loop [2305.12710]. The architecture contains an explanation-generation model \(g\), a rationale-aware prediction model \(f\), and an active learning sampler \(S\). The explanation generator is a T5-based sequence-to-sequence model trained on \((x,e)\) pairs using standard sequence negative log-likelihood, \[ L_{\mathrm{exp}}(\phi)=-\sum_{t=1}^{T}\log p_\phi(w_t\mid x,w_{0\), each unlabeled example \(u\) is scored by the average similarity of its text to previously collected explanations, \[ s(u)=\frac{1}{|D_{\mathrm{prev}}|}\sum_{(x_p,y_p,e_p)\in D_{\mathrm{prev}}}\mathrm{sim}(\mathrm{emb}(x_u),\mathrm{emb}(e_p)), \] using Sentence-Transformers embeddings and cosine similarity. At \(t=0\), when no explanations exist, the score is based on text-to-text similarity over the unlabeled pool. After ranking by \(s(u)\), the algorithm selects \(k\) items by equal spacing across the ranked list, a lightweight dispersion heuristic intended to preserve coverage. The empirical protocol on e-SNLI uses 80 trials, 9,000 unlabeled candidates per trial, and two budgets: \(k=9\) for 20 iterations, yielding 180 annotations, and \(k=30\) for 15 iterations, yielding 450 annotations. Under these settings, the explanation-informed selector consistently outperforms random and input-only similarity in every iteration, while least-confidence also beats random but remains weaker than explanation-informed sampling. Average accuracy reaches around \(55\%\) by the end of 20 iterations in the 180-label setting and around \(72\%\) by the end of 15 iterations in the 450-label setting. On ECQA, the same qualitative pattern holds, and input-only similarity performs close to random [2305.12710]. The human evaluation further distinguishes label accuracy from explanation utility. With 180 training examples, the dual-model AL system yields 64 “Yes” ratings out of 90 for prediction correctness versus 42 for the few-shot self-rationalization baseline, and 35 versus 21 for “Would you trust and use this AI in real decision-making?”, both statistically significant. By contrast, differences in explanation validity and “explanation supports prediction” are not statistically significant. The paper therefore operationalizes ALLabel as a system in which explanations affect all three stages of the loop: they are annotated by humans, generated by a model, and used as the basis for future sample selection [2305.12710]. ## 4. Label correction and noisy supervision In semantic segmentation, ALLabel is formulated as Active Label Correction (ALC), where the core query is not “what is the class of this pixel?” but “give the correct label only if the pseudo label is incorrect” [2403.10820]. Theoretical analysis compares classification-query cost \[ C_{\mathrm{cls}}(L)=\log_2 L \] with correction-query cost \[ C_{\mathrm{cor}}(L,p)=p+(1-p)\log_2 L, \] yielding savings rate \[ 1-\frac{C_{\mathrm{cor}}}{C_{\mathrm{cls}}}=\left(1-\frac{1}{\log_2 L}\right)p. \] With \(L=20\) and \(p=0.5\), the theoretical cost ratio is approximately \(0.62\). A user study with 20 annotators at \(p=0.5\) reports total time \(126.1\pm19.8\) s for classification versus \(95.1\pm9.0\) s for correction, time per query \(6.31\pm0.99\) s versus \(4.76\pm0.45\) s, and \(95\%\) accuracy for both. ALC combines foundation-model pseudo labels from Grounded-SAM, superpixel-based diversified candidate pools, and a look-ahead acquisition \[ a_{\mathrm{SIM}}(x;s,\theta):=\sum_{x_i\in s}\cos(f_\theta(x),f_\theta(x_i))\cdot a_{\mathrm{CIL}}(x_i;\theta), \] where \(a_{\mathrm{CIL}}(x;\theta)=1-f_\theta(y;x)\). It reaches \(95\%\) of fully supervised performance with 6K clicks on PASCAL and 150K clicks on Cityscapes, and produces a corrected PASCAL+ set with approximately 2.6 million corrected pixels [2403.10820]. A different correction-oriented line addresses label noise in image classification through Adaptive Label Refinement (ALR) [2502.00386]. After warm-up for \(m\) epochs on original labels, ALR updates the target for sample \(i\) at epoch \(e\) by temporal ensembling, \[ t_i(e)=\alpha t_i(e-1)+(1-\alpha)p_i(e), \] and optimizes \[ L_{\mathrm{ALR}}(e)=L_{\mathrm{ce}}(e)+\lambda L_e(e), \] where \(L_e\) is the prediction entropy. The design decouples avoidance of noisy labels from thorough learning of clean samples: soft-label refurbishment buffers incorrect supervision, while entropy minimization progressively hardens high-confidence predictions. The method requires no prior noise-rate knowledge or auxiliary clean data. On CIFAR-10 with \(80\%\) symmetric noise, ALR reports \(78.47\pm0.60\) top-1 accuracy versus \(73.86\pm0.61\) for ELR and \(53.82\pm1.04\) for CE; on ANIMAL-10N it reports \(85.79\pm0.05\), and on Clothing1M \(74.22\) [2502.00386]. A third noisy-supervision formulation extends ALLabel to the case of imperfect annotators through Optimal Labeler Assignment and Sampling (OLAS) [2512.12870]. Here, uncertainty is classifier entropy, \[ e(x)=-\sum_{i=1}^{C}P_\theta(y_i\mid x)\ln P_\theta(y_i\mid x), \] while label noise is modeled as \(\epsilon(a,e)\), decreasing in labeler accuracy \(a\) and increasing in uncertainty \(e\). The assignment problem minimizes maximum per-cycle noise: \[ \min_z \max_j \sum_{i=1}^{M}\epsilon(a_i,e_j)z_{ij}, \] subject to single-assignment and capacity constraints. The constructive optimum pairs highest-entropy points with highest-accuracy labelers. Sampling is then constrained by a noise bound \(\beta\). On UCI datasets under two simulated noise models, OLAS substantially exceeds the best baseline ES+OLA; for example, on Spambase under noise model 1, ES+OLA reports \(0.342\pm0.103\) F1 while OLAS reports \(0.785\pm0.050\) [2512.12870]. Taken together, these correction-oriented formulations redefine ALLabel from a query policy into a supervision-management problem. The label may be corrected relative to a pseudo label, refined from hard to soft form, or routed to a specific annotator under a noise-aware assignment policy. ## 5. Three-stage demonstration selection for LLM-based entity recognition The paper titled "ALLabel: Three-stage Active Learning for LLM-based Entity Recognition using Demonstration Retrieval" gives the most explicit canonical use of the name [2509.07512]. The task is entity recognition in scientific texts using in-context learning rather than fine-tuning. Given an unlabeled dataset \(D=\{x_1,\dots,x_N\}\), an annotation budget \(M\), and a demonstration corpus \(S=A\), ALLabel selects which samples should be annotated so that the resulting retrieval corpus maximizes LLM performance. The framework is single-pass and deterministic, with a fixed budget split of \(1{:}3{:}1\) across diversity, similarity, and uncertainty stages. Stage 1 performs warm-start core-set selection over a similarity matrix \(U\in\mathbb{R}^{N\times N}\). The seed is the sample with lowest average similarity, \[ u=\arg\min_{i\in D}\frac{1}{|D|-1}\sum_{j\in D,j\neq i}U(x_i,x_j), \] after which greedy additions maximize minimum distance to the selected set. Stage 2 computes a global usefulness score \(\mathrm{sum\_rank}[b]\) for each candidate demonstration \(b\), based on how frequently and how highly it appears in ranked similarity lists across queries. For query \(a\), if \(b\) has rank \(\mathrm{rank}\), the increment is \(1\) when \(\mathrm{rank}\le k\), \(1/(\mathrm{rank}-k+1)\) when \(k<\mathrm{rank}\le x\), and \(0\) otherwise. Stage 3 defines uncertainty as weak retrievability: queries whose best current demonstrations have low similarity are treated as uncertain, and the similarity-stage procedure is reapplied on the filtered subset. Retrieval uses BM25 in the main experiments, with \[ \mathrm{Score}(p,d)=\sum_{i=1}^{n}\mathrm{IDF}(p_i)\cdot \frac{f(p_i,d)(k_1+1)}{f(p_i,d)+k_1\left(1-b+b\cdot|d|/\mathrm{avg\_dl}\right)}, \] though Sentence-BERT is also evaluated. GPT-4o is the primary LLM, DeepSeek-V3 appears in additional experiments, and retrieved demonstrations are formatted as structured JSON entity lists. The datasets are CSD-MOFs, NC 2024 General, and USPTO, with pool sizes from 10 to 60 and \(k=3\) shots in the main setting [2509.07512]. Empirically, ALLabel consistently outperforms Random, Core-set, Perplexity-based selection, and BATCHER across all three datasets. Average F1 gains over Random are approximately \(5.5\%\) on CSD-MOFs, \(5.8\%\) on NC 2024 General, and \(5.1\%\) on USPTO. At pool size 60, ALLabel reports F1 values of \(93.3\) on CSD-MOFs, \(84.8\) on NC 2024 General, and \(89.7\) on USPTO, each within about \(1\)–\(2\%\) of full-corpus F1. The convergence analysis is central: performance within \(2\%\) of full-corpus retrieval is achieved with only \(5.0\%\) of the data on CSD-MOFs, \(9.1\%\) on NC 2024 General, and \(8.0\%\) on USPTO. Component ablations show that removing the similarity stage causes the largest degradation, and order ablations support the paper’s D–S–U sequence over S–D–U and S–U–D [2509.07512]. In this formulation, ALLabel is not querying labels to retrain model weights. It is constructing an annotation budget allocation policy for retrieval-augmented in-context learning. Demonstration quality, rather than classifier update dynamics, becomes the primary optimization target. ## 6. Label placement in augmented reality, visualization, and maps A separate research branch applies ALLabel to the spatial placement of text labels. In AR street-view scenes, semantic-aware placement is driven by a guidance map \[ G(i,j)=\frac{H'(i,j)\cdot S(i,j)}{\max(S(i,j))}, \qquad H'(i,j)=c_{ij}\cdot H(i,j), \] where saliency comes from DSS, semantic segmentation from DeepLabv3, and \(c\) is a task-specific importance prior learned from user placements [1912.07105]. Layout minimizes an energy combining label overlap with the guidance map, edge penalties, pairwise label intersections, leader-line guidance penalties, line intersections, line length, and line orientation. On a Cityscapes-derived dataset of 300 images and 48,000 user placements, the proposed method reports \(\mu_{\mathrm{centroid}}=61.78\), \(\mu_{\mathrm{over}}=8.50\), \(\mu_{\mathrm{int}}=12\), and \(\mu_{\mathrm{len}}=26.99\), outperforming Naive, Height Separation, Planar Separation, and Grasset baselines [1912.07105]. For static data visualization, a bitmap-based label-placement algorithm replaces point-sampling overlap tests with a raster occupancy bitmap [2405.10953]. Marks are rasterized into a 2D bit matrix aligned to chart pixels, and label rectangles are checked through bitwise operations over fully and partially covered machine words. Because overlap cost depends on label area and bitmap resolution rather than on the number or geometry of existing marks, the method accelerates large-mark and large-chart cases. On a benchmark built from a US airports map with 3,320 airports and 56 routes, the bitmap-based method is reported as at least \(22\%\) faster than original and Improved Particle-Based Labeling across chart sizes, while placing a similar number of labels; at 8,000 px width it places only \(0.8\%\) fewer non-overlapping labels than Improved Particle-Based Labeling [2405.10953]. Dynamic AR requires temporal rather than purely static optimization. RL-LABEL formulates label placement as an actor-critic problem with PPO, where the policy maps state \(s^i\) to acceleration action \(a^i\) in an \(x\)–\(z\) plane and maximizes \[ r(s^i,a^i)+\gamma V^*(s^i,a^i). \] The reward combines occlusion count, leader-line intersections, and acceleration regularization. On NBA and STU motion datasets, RL-LABEL reports lower occlusion, fewer line intersections, and lower movement distance than both a no-management baseline and a force-based baseline; on STU, for example, OCC/INT/DIST are \(0.06/0.02/+2.58\) for RL-LABEL versus \(0.07/0.06/+11.46\) for Force and \(0.18/0/0\) for No. A user study with 18 participants also reports lower completion times and mental load in several tasks [2308.13540]. Map finishing introduces yet another formulation. "Automated Label Placement on Maps via Large Language Models" treats label placement as structured spatial data editing and uses retrieval-augmented generation over NGA Data Product Specification sections to produce label anchor coordinates in pixel space [2507.22952]. The MAPLE dataset contains 100 map images and 1,276 labeled landmarks, with train/validation/test splits of 883/126/267. Four open-source LLMs—Llama 3.1, Gemma 2, Qwen 3, and Phi-4—are evaluated in zero-shot and QLoRA-tuned settings. Instruction tuning sharply reduces RMSE; for example, Phi-4 improves from \(81.2\) zero-shot to \(28.4\) tuned in the best reported format, and Gemma 2 reaches \(30.1\). Landmark-type analysis shows Office and Shop are easier than Leisure and Building, and neighbor-feature injection usually does not help. This line of work replaces hard geometric optimization with prompt conditioning, retrieved guidelines, and structured coordinate generation [2507.22952]. These placement systems share the term “label,” but their optimization objects differ fundamentally from active learning. The label is now a text box or anchor position in screen or map coordinates, and the key constraints are visibility, non-overlap, readability, temporal stability, and adherence to cartographic or interface conventions. ## 7. Adaptive label representations and recurring principles ALLabel also connects to research on learned label representations. "Improving Model Training via Self-learned Label Representations" introduces Learning with Adaptive Labels (LwAL), which replaces one-hot targets by class prototypes in latent space [2209.04528]. For class \(c_j\), the adaptive label is the centroid \[ \hat{\mathbf y}_{c_j}=\frac{1}{m_{c_j}}\sum_{y^{(i)}=c_j}\mathbf z^{(i)}, \] and class probabilities are derived from negative Euclidean distances, \[ p_j^{(i)}=\mathrm{Softmax}\!\left(-\|\mathbf z^{(i)}-\hat{\mathbf y}_{c_j}\|_2\right). \] Training then uses cross-entropy over these distance-based probabilities, optionally with a repel regularizer over cosine similarity between embeddings of different classes. Reported effects include training-time reductions usually by more than \(50\%\), sometimes up to \(80\%\), while often improving test accuracy; for example, on CIFAR-100 with DenseNet121, LwAL10+rpl reports \(48.0\pm0.0\) versus \(44.6\pm0.8\) for standard one-hot training [2209.04528]. Across the literature, several recurring principles emerge. First, uncertainty is rarely used alone. Remote sensing AL couples it to clustering [2212.01165], explanation-driven AL couples semantic similarity to evenly spaced selection [2305.12710], and OLAS constrains high-entropy selection by labeler noise bounds [2512.12870]. Second, labels are increasingly enriched beyond hard categorical assignments: explanations become supervision objects [2305.12710], soft targets are iteratively refined [2502.00386], pseudo labels become correction references [2403.10820], and demonstrations become retrievable labeled exemplars for ICL [2509.07512]. Third, priors are central. These include foundation-model pseudo labels and superpixels [2403.10820], retrieved guidelines for cartography [2507.22952], saliency and semantic maps for AR layout [1912.07105], and latent prototype geometry for adaptive classification targets [2209.04528]. The main limitations are equally consistent. Many methods are domain-specific: aerial remote sensing [2212.01165], NLI and QA with explanations [2305.12710], semantic segmentation with Grounded-SAM priors [2403.10820], scientific NER with BM25-based retrieval [2509.07512], and map or AR placement under specialized geometric assumptions [1912.07105], [2308.13540], [2507.22952]. Several formulations also leave important issues open: the explanation-based system does not introduce a formal faithfulness metric [2305.12710], ALC depends on the quality of pseudo labels and superpixels [2403.10820], ALR is weaker than ELR on WebVision top-1 under severe class imbalance [2502.00386], and the LLM-based map labeler does not include explicit collision resolution [2507.22952]. In that sense, ALLabel is less a singular algorithmic object than a naming locus for research that elevates labels from fixed outputs to actively designed structures. Depending on the problem, the label may be an annotation candidate, an explanation, a pseudo label to be corrected, a latent prototype, a retrieved demonstration, or a spatially placed text element. The technical diversity of these formulations is not incidental; it is the defining feature of the term’s current use in the literature. Source: https://www.emergentmind.com/topics/allabel