LeMix: Mixed Methods in Diverse Domains

• LeMix is a context-dependent term encompassing methods for dataset mixture selection via model merging, embedded program rehosting on Linux, multi-GPU LLM scheduling, and neural audio remixing.
• Each variant applies distinct techniques: parameter-space averaging for fine-tuning data selection, RTOS-to-Linux retargeting for security, and stage-level prediction for GPU scheduling.
• Across domains, LeMix improves efficiency and performance—reducing computation through surrogate models, enhancing bug detection in embedded testing, and optimizing resource use in LLM systems.

Across recent arXiv literature, LeMix is not a single standardized research term. Instead, the name and closely related variants denote several technically distinct proposals: dataset-mixture selection for language-model fine-tuning via model merging; dynamic analysis of embedded applications rehosted as Linux executables; unified scheduling of concurrent LLM training and inference on multi-GPU systems; and, in related but terminologically distinct audio work, neural remixing of music and soundscapes (Tao et al., 21 May 2025, Tanksalkar et al., 22 Mar 2025, Li et al., 28 Jul 2025, Yang et al., 2021, Jiang et al., 2024). The term therefore has to be interpreted from context rather than by name alone.

1. Nomenclature and scope

The main usages associated with LeMix or closely related naming differ in domain, objective, and mathematical structure. In the dataset-mixing setting, the authors use the name “Merge to Mix” rather than LeMix; the supplied details state that if LeMix is used elsewhere in that sense, it is only conceptually synonymous and not an alias adopted by the authors. In the embedded-systems paper, LEMIX is the explicit framework name. In the multi-GPU scheduling paper, LeMix is likewise the explicit system name. The audio papers do not adopt the term LeMix, but they are often relevant because they center on learning to remix mixtures directly (Tao et al., 21 May 2025, Tanksalkar et al., 22 Mar 2025, Li et al., 28 Jul 2025, Yang et al., 2021, Jiang et al., 2024).

Usage	Domain	Defining mechanism
Merge to Mix	LM fine-tuning and dataset selection	Evaluate candidate dataset mixtures by merging individually fine-tuned models
LEMIX	Embedded software security	Rehost RTOS applications as native x86 Linux binaries for fuzzing and sanitization
LeMix	Multi-GPU LLM systems	Co-locate and schedule inference and training with prediction- and memory-aware policies
Learn-to-remix / text-guided remixing	Audio and music	Directly edit mixtures through joint optimization or text-conditioned latent masking

This terminological fragmentation matters because the same surface name can imply parameter-space averaging, RTOS-to-Linux retargeting, GPU scheduling, or waveform-level remixing. Common secondary signals are usually decisive: references to $\theta_0$, $\Delta \theta_i$, and candidate subsets point to dataset mixing; RTOS, MMIO, and LPL point to LEMIX; and $\Delta_F$, $M_{\text{threshold}}$, TTFT/TBT, or KV cache point to the multi-GPU scheduler.

2. Dataset-mixture selection via model merging

When LeMix is used to mean dataset mixing by model merging, it refers to the method introduced as “Merge to Mix: Mixing Datasets via Model Merging” (Tao et al., 21 May 2025). The problem is fine-tuning-data composition: given datasets $D_1,\dots,D_N$, a base model $\theta_0 \in \mathbb{R}^d$, and a target task $T$, the goal is to choose a subset whose fine-tuning distribution best matches downstream evaluation. The paper argues that common practice relies on heuristics, similarity measures, or “use all data,” and that exhaustive evaluation of candidate mixtures can require up to $2^N-1$ full fine-tuning runs.

The method constructs a surrogate for mixture fine-tuning through model merging. For each dataset $D_i$, one first fine-tunes from the same initialization $\theta_0$ to obtain $\Delta \theta_i$0 and defines $\Delta \theta_i$1. For a candidate mixture $\Delta \theta_i$2 with $\Delta \theta_i$3, the merged surrogate is

$\Delta \theta_i$4

or, equivalently,

$\Delta \theta_i$5

The surrogate objective replaces the intractable search over fully mixture-fine-tuned models,

$\Delta \theta_i$6

with

$\Delta \theta_i$7

The justification is not exact equality but positive correlation: $\Delta \theta_i$8 so that the ranking of candidate mixtures by merged-model validation performance approximates the ranking obtained by actual mixture fine-tuning.

The algorithmic pipeline is correspondingly simple. It requires $\Delta \theta_i$9 individual fine-tunes, then cheap arithmetic merges and validations over candidate subsets, and finally an optional final fine-tune on the selected mixture for deployment. The reported complexity reduction is substantial: for $\Delta_F$0–$\Delta_F$1, the method uses 6–7 fine-tunes instead of 63–127 mixture fine-tunes. The paper intentionally uses simple parameter averaging rather than weight matching, Fisher weighting, sign or magnitude filtering, or TIES-Merging, so as to avoid extra hyperparameters and preserve speed. Weighted generalizations are described, including

$\Delta_F$2

for continuous nonnegative mixture weights.

The experiments cover both vision and language. Vision uses CLIP ViT-B/32 with full-model fine-tuning for 10 epochs, batch size 128, and learning rate $\Delta_F$3 across eight classification tasks: Cars, DTD, EuroSAT, GTSRB, MNIST, RESISC45, SUN397, and SVHN. Language uses Llama-3-8B-Instruct, predominantly pretrained on English, with IA3 fine-tuning for 1 epoch, learning rate 0.01, and context length 6000, over six Japanese skill-building datasets and nine evaluation areas including translation, information extraction, mathematical reasoning, entity extraction, question answering, English understanding, semantic analysis, syntactic analysis, and commonsense morality understanding.

Empirically, the central assumption is supported by average Pearson correlations of 0.78 in vision and 0.57 in language between merged-model accuracy and mixture-fine-tuned accuracy. Selection performance also exceeds heuristic baselines. In vision, reported average test accuracy is 0.587 for Merge to Mix (merged), 0.524 for Merge to Mix with a final fine-tune on the selected mixture, 0.482 for all datasets, 0.483 for the best similarity baseline, 0.417 for random selection, and 0.568 for oracle selection. In language, the corresponding values are 0.491, 0.489, 0.403, 0.394, 0.395, and 0.512. The paper further reports that similarity metrics correlate weakly with downstream performance, with cosine average correlations around 0.17–0.19 and $\Delta_F$4 often negative. Reported limitations include nonlinear interactions among datasets, scaling problems when exhaustive enumeration over $\Delta_F$5 becomes infeasible, weakened local-linearity assumptions after long training, and the requirement of consistent architectures and identical initialization.

3. LEMIX as an embedded-systems rehosting framework

In embedded-systems security, LEMIX denotes “Enabling Testing of Embedded Applications as Linux Applications” (Tanksalkar et al., 22 Mar 2025). Its premise is that high-fidelity firmware emulation is often unnecessary for bug exposure. The paper argues that most relevant vulnerabilities arise at higher abstraction levels—application logic, HAL code, and RTOS services—rather than in inline assembly or exact peripheral timing. A study of 84 publicly reported memory corruption vulnerabilities is cited as motivation: none required precise peripheral models, and none depended on inline assembly. The paper formalizes this standpoint as Bug Manifestation Fidelity (BMF): enough fidelity to execute the relevant module and feed inputs may suffice to trigger most bugs.

The framework transforms an embedded program $\Delta_F$6 into a Linux-hosted program $\Delta_F$7 with a peripheral stub/model set $\Delta_F$8: $\Delta_F$9 The transformation replaces the MCU-specific RTOS port and board-specific layers with the RTOS’s Native Portable Layer, specifically the Linux Portable Layer, while preserving the application and HAL/driver layers. Kernel services such as tasks, timers, scheduling, and interrupts are mapped to Linux primitives via LPL: tasks to pthreads, interrupts to signals plus a dispatcher, and timers to POSIX timers. Peripheral interaction is then emulated at low fidelity. MMIO loads are detected through constant-address analysis and replaced with reads from stdin, MMIO writes are ignored, and an interrupt dispatcher task randomly invokes identified ISRs. Predicates dependent on MMIO values are weakened by toggling their truth value with 50% probability, so that busy-wait and peripheral-state blockers do not prevent coverage growth.

The conversion pipeline has two phases. Phase 1 generates an analysis-friendly Linux application by swapping in LPL, interactively retargeting GCC/ARM-GCC-centric code to Clang/LLVM and x86, resolving linker and symbol-alias issues, and applying LLVM-based instrumentation for MMIO and condition weakening. Phase 2 performs testing through either whole-program fuzzing in AFL++ persistent mode or function-level fuzzing for selected risky functions, with AddressSanitizer and other sanitizers available because the output is a native x86 executable. The framework uses build capture via Bear, bitcode production via wllvm, Python tooling, LLVM 10+ passes, and a runtime library for MMIO hooks.

The evaluation covers 18 real-world applications across four RTOSes—FreeRTOS, NuttX, Zephyr, and ThreadX—on an AMD EPYC 7543P host with 32 cores, 64 threads, and 128 GB RAM. Whole-program fuzzing runs for 48 hours per application, while function-level fuzzing runs for approximately 10 minutes per function. The reported outcomes are substantial: 21 previously unknown bugs were found across 12 applications and all 4 RTOS kernels; the framework achieved approximately 2x more coverage than existing techniques and found 18 more bugs than the compared state of the art. Bug counts are reported as 7 OOB writes, 2 OOB reads, 2 divide-by-zero, 3 NULL dereferences, 2 DoS, 2 assert failures, and 3 build-related. Condition weakening improves whole-program coverage by about 2x on average, with M2 covering ~860 versus M1 ~345 unique basic blocks per application. Against baselines, LEMIX detected 21 bugs total, compared with 1 for Fuzzware and 3 for MultiFuzz.

The framework’s limitations are equally explicit. It depends on the availability and maturity of LPL or WPL support; ISR modeling is incomplete and does not capture nested interrupts; memory-layout-dependent initialization patterns can require manual handling; inline assembly is commented out and replaced with randomized safe values, so ISA-specific bugs may be missed; and function-level fuzzing currently focuses on C rather than large C++ targets. Even so, the paper positions LEMIX as a pragmatic alternative to emulator-centric rehosting: lower fidelity than full hardware modeling, but high throughput and materially better bug exposure for many vulnerability classes.

4. LeMix as a multi-GPU scheduler for LLM serving and retraining

In distributed LLM systems, LeMix refers to “Unified Scheduling for LLM Training and Inference on Multi-GPU Systems” (Li et al., 28 Jul 2025). The target setting is co-location of online inference and continuous retraining on shared multi-GPU nodes. The paper argues that separate deployments waste resources because inference nodes experience low utilization under variable arrivals and training nodes suffer pipeline bubbles and imbalance. Empirical observations in the paper include 13.62% GPU utilization on inference nodes and 29.16% on training nodes under separate setups, while a naive co-location baseline reaches only about 31.88% and remains vulnerable to interference and SLO violations.

LeMix combines three subsystems: offline profiling, execution prediction, and runtime scheduling. Offline profiling learns stage-level forward and backward latency coefficients and safe memory thresholds. Stage latency is modeled as

$M_{\text{threshold}}$0

where $M_{\text{threshold}}$1 is sequence length, $M_{\text{threshold}}$2 is batch size, and $M_{\text{threshold}}$3 are hardware-dependent coefficients for node $M_{\text{threshold}}$4. Memory management uses

$M_{\text{threshold}}$5

together with a profiled waiting tolerance $M_{\text{threshold}}$6 beyond which response latency degrades sharply. Execution planning simulates stage timelines to estimate Idleness increased ($M_{\text{threshold}}$7) and Response time ($M_{\text{threshold}}$8), including interference from overlapping backward passes. Runtime scheduling then assigns tasks to nodes according to a multi-objective fitness: $M_{\text{threshold}}$9

$D_1,\dots,D_N$0

$D_1,\dots,D_N$1

Training can be deprioritized when an imminent inference request would otherwise violate the response-time SLO $D_1,\dots,D_N$2, and memory-aware execution either waits until memory becomes available or offloads KV cache to CPU when waiting reaches $D_1,\dots,D_N$3.

The model explicitly targets the two failure modes identified in the paper: dynamic arrivals during serving and heterogeneity in pipeline-parallel training. Arrivals are modeled as Poisson in the experiments, with FCFS continuous batching and a maximum waiting time $D_1,\dots,D_N$4. Pipeline-parallel training is treated with stage-level timeline simulation that respects forward/backward dependencies and far dependencies induced by interference. Rather than using a closed-form queueing model for scheduling, the system relies on custom execution planning and per-stage calibration with realized traces. The scheduling complexity is reported as $D_1,\dots,D_N$5.

The evaluation spans GPT-400M, GPT-1.4B, GPT-2.5B, and Llama-8B, Llama-13B, Llama-70B, on four 2-GPU servers per experiment. GPT systems use RTX 6000 Ada hardware; Llama systems use AWS EC2 servers with A100 80 GB GPUs and NVLink. Workloads combine synthetic Poisson arrivals and LMSYS Chatbot Arena traces, with HH-RLHF and SHP used for online training and evaluation. The main reported gains are up to 3.53x throughput, up to 0.61x inference loss, and up to 2.12x higher response time SLO attainment relative to separate setups. Node-level end-to-end latency is reduced by up to 0.43x, and TTFT/TBT by up to 0.43x/0.50x for GPT-2.5B. Reported overheads are small: execution planning about 0.14 ms per task, resource allocation 0.014 ms, memory-aware scheduling 0.056 ms, with about 9.6 MB memory for prediction and <4 MB for other modules, versus 727.5 MB for Mix-LUF utilization tracking.

The paper’s ablations show that removing profiling lowers throughput by up to 0.82x and worsens inference loss under high training rates; removing prioritization causes SLO attainment to collapse from about 95% to <30% at 100 rps and to 0% at 150 rps; and removing memory awareness reduces SLO attainment from 95% to 55% at 100 rps. Limitations include degraded prediction under highly bursty arrivals, sensitivity to extreme length heterogeneity, dependence on accurate profiling, temporary fairness trade-offs when training is deprioritized, and the need to reprofile after major architectural changes.

5. Audio-domain remixing lineages

Two audio papers are often associated with LeMix only in a broad conceptual sense: both learn to edit or remix mixtures directly, but neither adopts the name as an official method. The first is “Don’t Separate, Learn to Remix: End-to-End Neural Remixing with Joint Optimization” (Yang et al., 2021). It considers a $D_1,\dots,D_N$6-source music mixture

$D_1,\dots,D_N$7

a user-specified gain vector $D_1,\dots,D_N$8, and a target remix

$D_1,\dots,D_N$9

with gains mapped from loudness values by $\theta_0 \in \mathbb{R}^d$0. The paper argues that a two-step separator-then-remix pipeline is artifact-prone because leakage and artifact terms in $\theta_0 \in \mathbb{R}^d$1 are scaled into the final remix. It therefore repurposes Conv-TasNet into two jointly optimized architectures. Model-I applies gains after source estimation and trains with

$\theta_0 \in \mathbb{R}^d$2

Model-II applies gains in latent space before decoding,

$\theta_0 \in \mathbb{R}^d$3

with an analogous joint objective. Training uses classical scale-sensitive SNR/SDR rather than SI-SDR, Adam with initial learning rate $\theta_0 \in \mathbb{R}^d$4, 1-second segments, and random gains in $\theta_0 \in \mathbb{R}^d$5 dB on Slakh and MUSDB18. Reported gains are especially large in harder settings. On Slakh with $\theta_0 \in \mathbb{R}^d$6, the baseline records minSDR/LD = -4.08 / 11.31, versus 12.20 / 3.20 for Model-I and 12.44 / 3.15 for Model-II. On MUSDB18 with $\theta_0 \in \mathbb{R}^d$7, the baseline records -9.16 / 10.10, versus 11.01 / 2.85 and 10.95 / 3.00. The paper also reports that joint remix loss improves separation as a byproduct, especially SAR.

The second is “Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experience” (Jiang et al., 2024). Here the input mixture is

$\theta_0 \in \mathbb{R}^d$8

and the edited target is

$\theta_0 \in \mathbb{R}^d$9

where actions are chosen from removing, keeping, increasing volume, and decreasing volume, with scaling factors $T$0. The system, implemented as CocktailChat, has two modules. PromptReader maps a natural-language prompt $T$1 to an embedding

$T$2

using GPT-2 or LLaMA2. SoundEditor maps the mixture and text embedding to an edited waveform

$T$3

through a learnable encoder, a single text-conditioned editing mask in latent space, FiLM conditioning, and a decoder. Crucially, the model does not explicitly separate all sources before remixing; it directly outputs a single edited mixture. The dataset contains 160 hours and over 100k mixtures, each 5 s at 16 kHz, spanning speech and non-speech sources, 16 tasks, and five text rephrasings per mixture. Training runs for 100 epochs on 4× NVIDIA L40 GPUs with bfloat16. On the in-domain 2 Speech + 2 Audio test set, the best reported model—SepFormer + LLaMA2 (LoRA)—achieves 10.4 dB average SNRi, with task-level results including TSE 13.1, TAE 10.1, SE 10.7, SR 11.1, MVC 6.3, Others 6.8, and OVC 44.2. Across all 16 tasks, 89.0% to 99.8% of mixtures show positive SNRi. The system also reports robust zero-shot behavior across unseen source counts and unseen audio classes.

These two audio lineages differ from the systems papers in both representation and objective. Their central problem is waveform-level or latent-waveform-level edit control; their key abstractions are gain vectors, source-conditioned or text-conditioned masks, and scale-sensitive remix-quality losses rather than model-parameter arithmetic, RTOS replacement, or pipeline-scheduling prediction.

6. Disambiguation, misconceptions, and conceptual relations

Several misconceptions follow directly from the naming overlap. First, LeMix is not a universally accepted alias for “Merge to Mix”; the details explicitly state that the paper does not use the term and that any such equivalence is only conceptual (Tao et al., 21 May 2025). Second, LEMIX in embedded security is unrelated to LeMix in multi-GPU LLM scheduling; one operates over RTOS ports, MMIO hooks, and Linux-native fuzzing (Tanksalkar et al., 22 Mar 2025), while the other operates over stage-level latency models, queue prioritization, and KV-cache-aware co-execution (Li et al., 28 Jul 2025). Third, the audio papers are not nomenclatural instances of LeMix at all, but separate “learn to remix” programs organized around joint remix optimization or text-guided latent editing (Yang et al., 2021, Jiang et al., 2024).

The relationships among these usages are therefore conceptual rather than technical. A plausible interpretation is that the naming convergence reflects a shared emphasis on mixing heterogeneous components—datasets, workloads, or audio sources—but the underlying objects being mixed are different in every case: model deltas in parameter space, embedded execution environments, training and inference tasks on GPUs, or sources in a waveform mixture. As a result, transfer of intuition across usages is limited. Correlation between merged and fine-tuned validation accuracy in dataset selection does not imply anything about BMF in embedded analysis or about SLO-aware co-scheduling, and vice versa.

In practice, contextual disambiguation is straightforward. References to $T$4, $T$5, and candidate subset selection identify the dataset-mixing method. References to RTOSes, LPL, MMIO, AFL++, and ASan identify the embedded rehosting framework. References to $T$6, $T$7, $T$8, $T$9, $2^N-1$0, or KV cache identify the multi-GPU LLM scheduler. References to Conv-TasNet, SepFormer, PromptReader, FiLM, SNRi, or minSDR/LD identify the audio-remixing literature. Under that reading, LeMix is best treated not as one encyclopedia entry for one method, but as a context-dependent label spanning multiple independent research programs.