--- title: Progressive Explicit-to-Latent Curriculum url: https://www.emergentmind.com/topics/progressive-explicit-to-latent-curriculum type: topic --- # Progressive Explicit-to-Latent Curriculum Progressive Explicit-to-Latent Curriculum denotes a class of curriculum-learning strategies in which optimization begins with signals that are direct, overt, or strongly supervised and then shifts toward signals that are hidden, compositional, weakly supervised, or internal to the model. In the medical-imaging formulation, training starts from clear, unobstructed images and progressively increases occlusion so that the model must rely on latent or hidden anatomical structure [2306.15574]. In latent-reasoning work, the transition runs from explicit Chain-of-Thought tokens to continuous latent tokens or latent hidden-state rollouts [2602.10229][2604.02073]. In multimodal learning, explicit signals are redundant or modality-unique cues, whereas latent signals are synergistic interactions that become usable only after unimodal representations mature [2606.16639]. Related formulations also appear in robust visual question answering, web navigation, molecular generation, human-object interaction video generation, motion generation, curriculum search, and manifold-structured bandits [2411.17292][2604.12666][2511.11894][2512.22854][2504.16722][2106.08569][2606.19750]. ## 1. Conceptual definition and scope The defining feature of this curriculum family is not merely staged training, but a directed progression from directly observable cues to internally inferred structure. In "See Through the Fog," the mapping is literal: early stages exploit explicit features such as clear anatomical boundaries and high-contrast lesions, whereas higher occlusion forces inference from contextual cues, global shape priors, and learned invariances [2306.15574]. In LT-Tuning and PLUME, the mapping is representational: explicit textual reasoning is used as a temporary scaffold and is then rewritten into latent tokens or latent hidden-state computation [2602.10229][2604.02073]. In SPICE, explicit signals are redundant/shared cues and unique modality-specific cues, while latent interactions are synergistic components revealed only jointly [2606.16639]. In TPCL, explicit subtasks are question-type partitions, and the latent objective is the full VQA problem over the entire training distribution [2411.17292]. A concise cross-domain mapping is as follows. | Paper/domain | Explicit regime | Latent regime | |---|---|---| | Medical imaging [2306.15574] | Clear, unobstructed images | Increasingly occluded images | | LT-Tuning [2602.10229] | Explicit CoT tokens | Continuous latent tokens | | SPICE [2606.16639] | Redundant and unique cues | Synergistic interactions | | TPCL [2411.17292] | Question-type subtasks | Full VQA objective | | Web navigation [2604.12666] | Supervised imitation | Preference and policy optimization | | CoG molecular design [2511.11894] | Explicit prompt segments | Latent denoising trajectory | | PLUME [2604.02073] | Verbalized CoT scaffold | Latent rollout with hidden states | | ProMoGen SAP-CL [2504.16722] | Dense anchor supervision | Sparse-anchor latent motion prior | This scope indicates that the phrase refers to a design pattern rather than a single algorithm. A plausible implication is that explicit-to-latent curricula are best understood as curriculum policies over what information is exposed, rather than only over how difficult examples are. That interpretation is directly supported by settings in which the curriculum controls occlusion ratio, reasoning-token replacement, anchor density, task composition, or sampling geometry rather than conventional easy-to-hard ranking alone [2306.15574][2602.10229][2504.16722][2606.19750]. ## 2. Core mechanisms of progression A recurring mechanism is stage-wise control over observable information. In the medical-imaging formulation, occlusion is applied as $x' = x \odot m$ with occlusion ratio $\alpha = 1 - |m|/(h \cdot w)$, and the ordered curriculum is implemented by sorting samples by increasing occlusion and defining $S_t = \{x'_i \mid i \le n_t\}$ with $n_t = \lceil t \cdot n/T \rceil$ [2306.15574]. The same paper adds three regulators for stable progression: Wasserstein Curriculum Learning, which minimizes $W_1(P_t, P_{t+1})$ between stagewise occlusion histograms; Information Adaptive Learning, which maintains $I(Y;\hat{Y})$ as occlusion rises; and Geodesic Curriculum Learning, which regularizes the parameter trajectory by a geodesic-length term [2306.15574]. A second mechanism is progressive replacement of explicit tokens by latent computation. LT-Tuning uses a three-stage pipeline: Stage 1 trains explicit CoT with standard supervised fine-tuning; Stage 2 inserts `` tokens at low-confidence positions using $g_t = \mathbb{1}[p_\theta(y_t \mid y_{, ^K, `, and the explicit suffix remains teacher-forced until the final fully latent stage [2604.02073]. A third mechanism is online re-estimation of what counts as explicit versus latent complexity. SPICE computes per-sample surrogate PID scores for redundancy $R_i$, uniqueness $U_i$, and synergy $S_i$, refreshes them every $k=5$ epochs, and progresses from redundancy-focused sampling to unique-pattern exposure and then to synergy-focused training [2606.16639]. TPCL computes per-task loss histograms and ranks tasks by the Wasserstein Optimal Transport distance between consecutive histogram states, then expands the curriculum set according to an incremental pacing function while preserving a hard-to-easy ordering [2411.17292]. BMC similarly begins with prompt-level reward-variance signals $y_i = \phi(\mathrm{Var}([r_{i1},\dots,r_{ik}]))$ and then organizes prompts into a Latent Task Tree, enabling top-down hierarchical Thompson sampling over latent regions rather than independent arms [2606.19750]. These mechanisms show that explicit-to-latent progression is frequently stabilized by continuity constraints. The continuity may be distributional, as in $W_1(P_t,P_{t+1})$ [2306.15574]; historical, as in TPCL’s consolidated OT difficulty over $B=5$ cycles with $\alpha=[0.10,0.10,0.30,0.50]$ [2411.17292]; geometric, as in geodesic regularization [2306.15574]; or structural, as in PLUME’s stagewise rewrite and BMC’s tree-based belief propagation [2604.02073][2606.19750]. ## 3. Task- and sample-level curricula In perception and supervised multimodal learning, explicit-to-latent curricula often operate by reordering samples or task partitions rather than by modifying model architecture. The medical-imaging curriculum is the clearest sample-level instance: the model is trained first on low-occlusion images and later on higher-occlusion variants generated by PROS or PBOS, with WCL, IAL, and GCL available as integrated strategies for smoother distributional shifts and information adequacy [2306.15574]. On Br35H, the reported binary results are Baseline Accuracy 99.50, ROC-AUC 99.67, F1 99.50; PROS Accuracy 99.83, ROC-AUC 100, F1 99.83; PBOS Accuracy 99.33, ROC-AUC 99.67, F1 99.33. On Brain Multi-Class, the reported values are Baseline Accuracy 96.11, ROC-AUC 96.88, F1 96.03; PROS Accuracy 97.41, ROC-AUC 98.19, F1 97.37; PBOS Accuracy 98.02, ROC-AUC 98.64, F1 97.96 [2306.15574]. SPICE instantiates a multimodal version of the same principle. The curriculum begins with high-redundancy samples, adds high-unique samples, and finally emphasizes high-synergy samples, with two realizations: SPICE-S via stage-wise binning and SPICE-E via entire-dataset ordering [2606.16639]. The training schedule is a 30-epoch warm-up followed by 3 stages of 50 epochs each, with PID scores recomputed every $k=5$ epochs [2606.16639]. On CREMA-D, SPICE-E reaches 83.06% ACC and 89.07% mAP; on Kinetics-Sounds, 73.99% ACC and 79.48% mAP; on NVGesture, 87.14% ACC and 87.36% F1; on VGGSound, 54.98% ACC and 56.63% mAP [2606.16639]. The paper explicitly states that redundant and unique components correspond to explicit signals, whereas synergy corresponds to latent interactions [2606.16639]. TPCL operationalizes explicit-to-latent at the task level. The dataset is partitioned into $T=65$ question types, task difficulty is measured by OT between consecutive loss histograms, and the curriculum progressively enlarges the active task set under $p^\uparrow(r)=\min(1,\lambda_0+(1-\lambda_0)\cdot r/\lambda_{grow})$ with $\lambda_0=0.1$ and $\lambda_{grow}=4.5$ [2411.17292]. The dynamic LXMERT results are 77.23 on VQA-CP v2, 76.15 on VQA-CP v1, and 78.03 on VQA v2; the fixed variant reaches 78.42 on VQA v2 [2411.17292]. A common misconception is that curriculum learning here is easy-to-hard; TPCL explicitly reports that the progression is hard-to-easy while gradually increasing the set size, and that the “backward” hard-to-easy ordering generally gives better OOD generalization in low-data regimes [2411.17292]. TSO and BMC extend the same family into curriculum discovery and online RL sampling. TSO begins with explicit permutations of training batches, embeds them with an encoder-decoder-predictor assembly, and then performs gradient ascent in the latent representation $z \leftarrow z + \eta \nabla_z p_\psi(z)$ before decoding back to explicit curricula [2106.08569]. Reported gains include 47.92 to 48.97 on Sampled-CIFAR-100 with N4 and 50.39 to 51.93 with N5, alongside the summary claim of an empirical gain of 2AP over random strategies [2106.08569]. BMC begins with explicit reward-variance modeling and then transitions to latent-geometry-aware sampling via a Latent Task Tree; curriculum-based strategies are reported to achieve roughly 40% effective-ratio improvement over uniform sampling [2606.19750]. ## 4. Explicit-to-latent reasoning and representation transfer The most literal use of the term in contemporary language-model work concerns the migration from explicit reasoning tokens to latent hidden-state computation. LT-Tuning begins with explicit CoT grounding, then inserts latent `` tokens at uncertain positions, and finally constructs those latent tokens with Context-Prediction-Fusion to mitigate feature collapse, distribution mismatch, and alignment issues [2602.10229]. The core claim is that latent tokens should not be raw recycled hidden states alone; they should fuse contextual history with predictive semantic guidance from the model’s own vocabulary distribution [2602.10229]. Reported average accuracies are 36.4% for 1B, 52.4% for 3B, and 68.8% for 8B, compared with best baselines of 33.2%, 50.5%, and 66.0%; with adapter, the 8B model reaches 70.3%, including a gain on MultiArith from 92.8% to 96.1% [2602.10229]. The ablations identify the curriculum as structurally important: w/o Stage 2 is −3.9% for 3B and −6.7% for 8B, and w/o Stage 3 is −8.0% for 3B and −23.5% for 8B [2602.10229]. PLUME applies the same transfer principle to universal multimodal embedding. Its training schedule is Stage 0 fully explicit CoT for 3 epochs, Stages 1–3 progressive partial latentization for 1 epoch total, and Stage 4 fully latent for 1 epoch, with $K=8$ latent steps throughout [2604.02073]. The latent rollout is guided by a semantic-anchor-conditioned routed adapter, with one shared expert, $M_e=4$ specialized experts, Top-$K_r=2$ routing, and an auxiliary balance loss $\mathcal{L}_{bal}$ [2604.02073]. On the 78-task MMEB-v2 benchmark, PLUME reports All 61.6, Image 66.3, Video 44.1, and VisDoc 67.5, compared with UME-R1 at All 60.1, Image 66.6, Video 42.2, and VisDoc 63.9 [2604.02073]. The efficiency comparison is central to the explicit-to-latent claim: PLUME uses 8 latent steps versus 403 reasoning tokens for UME-R1, with latency 298±12 ms/sample versus 9023±187 ms/sample, yielding a 30.3× speedup [2604.02073]. The curriculum ablation is decisive: w/o Curriculum drops All from 61.6 to 54.8 [2604.02073]. A notable shared property of LT-Tuning and PLUME is that neither paper introduces an explicit distillation loss from explicit CoT into latent states. LT-Tuning states that no additional KL, contrastive, or reconstruction losses are introduced beyond standard supervised fine-tuning losses on CoT traces and answers [2602.10229]. PLUME likewise states that it does not introduce an explicit KL or hidden-state consistency loss and instead effects transfer through the curriculum design itself [2604.02073]. This suggests that explicit-to-latent transfer can be realized by structural supervision and staged rewriting alone when the latent interface is sufficiently well aligned. ## 5. Generative, control, and decision-making instantiations In generative modeling, explicit-to-latent curricula frequently operate by decomposing conditioning signals and then progressively shifting generation burden onto latent priors. CoG for text-guided molecular design decomposes a prompt into explicit semantic segments, orders them coarse-to-fine, and then performs multi-stage latent diffusion with cumulative sub-prompts $T_{1:k}$ and partial denoising starting at approximately $\rho \approx 0.8$ of the schedule [2511.11894]. The first stage fixes the scaffold, and later stages add functional groups and modifiers while preserving prior structure through cumulative conditioning and partial denoising [2511.11894]. On ChEBI-20, GraphLDM + CoG reports BQI 60.47, Q-Cov 45.58, Q-Nov 11.10, Validity 100, compared with GraphLDM at BQI 59.54 and Q-Cov 43.19; on PubChem, GraphLDM + CoG reports BQI 58.07 and Q-Cov 43.13 [2511.11894]. The anti-curriculum result is much worse, with GPT-4o, One-shot$(T_l)$ + CoG$(T_m)$ + CoG$(T_s)$ at BQI 38.18 and Q-Cov 10.77 [2511.11894]. In embodied and structured-control settings, the transition commonly reduces explicit control density over time. ProMoGen’s SAP-CL starts with dense anchor supervision and then progressively reduces the number of anchors by setting $K_{min}(1)=20$ and $K_{min}(E_{stage})=1$ across $E_{total}=100$ epochs and $E_{stage}=4$ stages, while keeping $K_{max}=30$ [2504.16722]. This turns sparse anchor motion generation into a progressive explicit-to-latent curriculum: early dense anchors provide precise explicit guidance, and late sparse anchors require the model to rely increasingly on the latent motion prior [2504.16722]. The curriculum-versus-regular comparison reports, for 9 anchors, MPJPE 3.101 to 2.130 and FID 0.319 to 0.124; for 1 anchor, MPJPE 6.119 to 5.929 and FID 1.127 to 1.014 [2504.16722]. ByteLoom uses a related staged progression for human-object interaction video generation, but the explicit-to-latent transition is formulated as a reduction in reliance on fine-grained hand annotations and an increase in reliance on learned HOI priors. Curriculum I trains on human pose conditioning without objects; Curriculum II trains on hand-object interaction with textured object meshes, estimated 6-DoF object poses, per-frame RCM, and RCM-cache; Curriculum III fine-tunes on full HOI data [2512.22854]. The reported full model, I+II+III, achieves Obj-IoU 0.8288, Obj-CLIP 0.9100, Face-Cos 0.8891, LMD 0.1427, and T-SSIM 0.5682 on Mani4D-Test [2512.22854]. The ablations show that adding the hand-object datasets of Curriculum II is key: I+III gives Obj-IoU 0.7627 and T-SSIM 0.4812, whereas I+II+III gives Obj-IoU 0.8288 and T-SSIM 0.5682 [2512.22854]. Robust web navigation provides a decision-making instance of the same pattern. Triton first trains Triton-SFT-32B on explicit imitation, then Triton-ORPO-32B on pairwise preference signals via Odds Ratio Preference Optimization, and finally Triton-GRPO-32B on long-horizon policy optimization via Group Relative Policy Optimization [2604.12666]. The paper explicitly defines Stage I as explicit and Stages II–III as latent, because preference and policy objectives shape discrimination and consistency without directly imitating labels [2604.12666]. On Mind2Web, Step Success Rate rises from 47.6% for Triton-SFT-32B to 53.2% for Triton-ORPO-32B and 58.7% for Triton-GRPO-32B [2604.12666]. ## 6. Empirical patterns, limitations, and open directions Across these papers, the principal empirical pattern is that explicit-to-latent curricula improve robustness when direct cues become unreliable or insufficient. In medical imaging, progressively occluded training improves performance on occluded brain-tumor classification [2306.15574]. In VQA, hard-to-easy task growth reduces shortcut learning and improves OOD robustness, with LXMERT rising from 48.66 to 77.23 on VQA-CP v2 under TPCL_Dyn↑ [2411.17292]. In latent reasoning and multimodal retrieval, staged transfer from explicit CoT to latent computation improves both accuracy and efficiency [2602.10229][2604.02073]. In web navigation, the progression from imitation to discrimination to policy optimization yields additive gains in Step Success Rate [2604.12666]. In HOI video generation and motion generation, staged conditioning improves geometric consistency, anchor adherence, and realism under sparse or incomplete controls [2512.22854][2504.16722]. Several misconceptions are directly contradicted by the record. First, explicit-to-latent does not mean the late stages are less supervised in every respect: TPCL enlarges the active task set while remaining fully supervised [2411.17292], and SPICE never adds an explicit PID regularizer, using only data ordering under the loss $L = CE(p^{multi}, y) + \sum_m CE(p^{(m)}, y)$ [2606.16639]. Second, explicit-to-latent does not require token reasoning: medical imaging, multimodal interaction learning, HOI video generation, and motion generation implement it through occlusion, PID-based sample reordering, RCM-conditioned staged training, or sparse-anchor schedules [2306.15574][2606.16639][2512.22854][2504.16722]. Third, it is not synonymous with easy-to-hard: TPCL explicitly reports hard-to-easy progression, and BMC shows that productivity, diversity, and utility need not be aligned under a single difficulty axis [2411.17292][2606.19750]. The main limitations are also consistent across domains. Smooth progression is often expensive or estimator-dependent. Medical imaging adds OT overhead and relies on accurate mutual-information estimation and suitable metric tensors for geodesic regularization [2306.15574]. SPICE uses surrogate PID scores derived from model predictions rather than exact PID measures and is sensitive to probability calibration and weak-synergy regimes [2606.16639]. LT-Tuning is sensitive to distribution mismatch in large untied models, with a −23.5% ablation when Stage 3 fusion is removed at 8B scale [2602.10229]. PLUME is sensitive to the curriculum itself, with a −6.8 overall drop when Stages 1–3 are skipped [2604.02073]. ByteLoom reports that object-only NVS gives only marginal benefit and may introduce distribution mismatch [2512.22854]. CoG depends on segmentation quality and can struggle with conflicting or noisy textual constraints [2511.11894]. BMC assumes that latent embeddings reflect behaviorally meaningful structure and notes that tree reconstruction may be needed if drift is suspected [2606.19750]. The future directions named in these papers are largely extensions of the same design principle. Medical imaging proposes adaptive schedules, semi-supervised learning, generative occluders, and latent-space curricula [2306.15574]. SPICE suggests that the curriculum can be layered on top of balancing or robustness methods and generalized to more than two modalities [2606.16639]. PLUME points toward structured latent computation as a practical replacement for explicit rationale generation in retrieval [2604.02073]. BMC adds the utility-aware BMC-T variant, in which latent geometry is combined with target-aware bonuses when productivity and evaluation relevance diverge [2606.19750]. Taken together, these directions indicate that Progressive Explicit-to-Latent Curriculum has become a unifying strategy for transferring learning from overt supervision or evidence into internalized inference, structured priors, and latent computation. Source: https://www.emergentmind.com/topics/progressive-explicit-to-latent-curriculum