SignalLLM: LLM Framework for Signal Processing

• SignalLLM is a modular, general-purpose LLM framework designed to automate and generalize signal processing tasks using in-context learning and hierarchical planning.
• It employs a two-stage architecture that decomposes high-level objectives into structured subtasks and refines solutions through adaptive retrieval-augmented generation.
• The framework demonstrates strong performance in few-shot and zero-shot scenarios, excelling in applications like radar detection, human activity recognition, and text compression.

SignalLLM is a modular, general-purpose LLM agent framework designed for automating and generalizing signal processing (SP) tasks. It integrates in-context learning, hierarchical planning, adaptive retrieval-augmented generation (RAG), cross-modal reasoning, code synthesis, and LLM-assisted modeling to address limitations of fragmented, expert-dependent, and inflexible SP pipelines. The framework is notable for its structured architectural approach and versatility across a diverse set of SP applications, with especially strong performance in low-data regimes such as few-shot and zero-shot settings (Ke et al., 21 Sep 2025).

1. Architectural Principles and System Components

SignalLLM is built on a modular and agentic architecture that explicitly decomposes high-level SP objectives into structured subtasks. The overall process is organized into two sequential stages:

• Stage 1: Tailored Planning
  • SP Task Decomposition: Utilizes in-context learning and domain-specific retrieval (via a Web Searcher component, akin to Toolformer), converting a user's natural language request into an explicit chain of structured subtasks.
  • SP Subtask Planning: Employs a complexity-aware, adaptive RAG mechanism. Depending on subtask difficulty, planning proceeds via direct LLM generation, single-round retrieval, or iterative multi-hop retrieval, with iterative context updates:

  $c_{i+1} = (d_1, \ldots, d_i, a_1, \ldots, a_i)$

  where $d_j$ are retrieved domain items and $a_j$ are intermediate LLM-generated answers. - Solution Refining Module: Aggregates and compares intermediate solutions stored in agent memory, using comparative evaluation to select the highest-quality approach.

• Stage 2: Adaptive Execution

  • LLM-Assisted SP Reasoning Module: Solves subtasks involving logical, algorithmic, or cross-modal reasoning using prompt engineering, code synthesis (with capability for Python/MATLAB invocation), or direct LLM inference.
  • LLM-Assisted SP Modeling Module: Engages for data-driven tasks, such as parameter tuning or algorithm adaptation, leveraging frozen pre-trained models with minimal fine-tuning as required. Execution strategy is selected dynamically based on subtask nature.

This multi-stage modular design allows SignalLLM to sequence and blend prompt-based, algorithmic, and data-driven components according to task composition and resource constraints.

2. Hierarchical Task Decomposition and Planning Methodology

SignalLLM’s planning process operationalizes decomposition as a series of retrieval-augmented in-context learning steps, governed by subtask complexity:

• Decomposition: Starting with a natural language SP request, in-context learning and web search retrieval break down the task into granular subtasks featuring explicit chains of logic linked to domain knowledge.
• Adaptive Retrieval-Augmented Generation (RAG): The planning module adaptively decides between direct LLM answers, retrieval-augmented single shot solutions, or multi-hop iterative refinement for more ambiguous/complex cases. The multi-hop process accumulates evidence and intermediate results:

$c_{i+1} = (d_1, ..., d_i, a_1, ..., a_i)$

• Refinement: All candidate solutions are persisted, and the refinement step makes comparative assessments over this persistent workspace to optimize the final output.

This approach enables efficient, contextually-aware navigation of open-ended, under-specified, or data-limited SP tasks.

3. Execution Strategies: Reasoning, Modeling, and Cross-Modal Fusion

SignalLLM routes subtasks to execution modules based on modality and problem type:

• Prompt-Based Reasoning: For structured queries, the system builds instruction-centric prompts, integrating explicit components (instructions, expert knowledge, examples, response format) to maximize reasoning fidelity.
• Cross-Modal Reasoning: For modalities such as radar or human activity recognition, SignalLLM combines visual representations (e.g., STFT plots, sensor data images) and textual description in prompts to enable LLM-driven interpretation and classification.
• Code Synthesis: In numerically precise or algorithmic domains, SignalLLM generates executable code artifacts (Python/MATLAB), optionally interacting with external solvers. Specific algorithms, such as text source coding, are realized as LLM-driven code synthesis pipelines.
• LLM-Assisted Modeling: For optimization or adaptation (e.g., hyperparameter search), SignalLLM leverages its reasoning capabilities to recommend model adjustments, evaluate performance with scoring pairs $(\theta_i, \mathcal{M}(\theta_i))$, and iteratively adapt using hybrid methods (e.g., alternating with Differential Evolution).

This mixed-strategy execution enables robust adaptation to SP domains with differing characteristic data types and solution requirements.

4. Versatility, Modalities, and Empirical Effectiveness

SignalLLM is deployed and evaluated across a spectrum of communication and sensing tasks:

• Few-Shot Radar Target Detection: Achieves superior detection accuracy and F1-score compared to state-of-the-art handcrafted and agent-based baselines with minimal training samples, leveraging cross-modal fusion of Doppler entropy, STFT, and textual features.
• Zero-Shot Human Activity Recognition: Outperforms prior frameworks by combining knowledge retrieval, signal visualizations, and structured prompts for zero-shot classification based solely on generic domain knowledge.
• Text Signal Source Coding: Implements lossless text compression as probabilistic model inference; experiments confirm higher compression efficiency relative to classical coding techniques.
• Handcrafted Feature Optimization: Uses agent reasoning to guide hyperparameter tuning for features such as Frequency Peak-to-Average Ratio, outperforming conventional optimization methods on stability and accuracy.
• Modulated Signal Recognition Under Resource Constraints: Maintains high recognition accuracy under limited resource conditions by fine-tuning pre-trained CNNs with LLM-guided preprocessing.

Performance across these domains demonstrates that SignalLLM is agnostic to input modality (text, time-series, image-derived), task type (reasoning, modeling), and can deliver substantial improvement in data-scarce scenarios.

5. Representative Technical Approaches and Algorithms

SignalLLM operationalizes and combines leading technical strategies:

• Adaptive Use of Retrieval-Augmented Generation (RAG): Planning is tuned to task uncertainty and complexity, shifting between single-retrieval, multi-hop, or direct LLM inference, and formalizing context updates per

$c_{i+1} = (d_1, ..., d_i, a_1, ..., a_i)$

• Cross-Modal Prompt Engineering: Integration of visual and textual features in prompts for LLMs, leveraging signal feature images alongside domain knowledge for reasoning.
• LLM-Driven Algorithmic Synthesis: Chain-of-thought code synthesis for source coding and hyperparameter optimization, optionally deploying external solvers when required for computational efficiency.
• Fine-Tuning and Model Selection: For data-driven modules, SignalLLM invokes frozen pre-trained models (transformers, CNNs), fine-tuning only where beneficial, minimizing resource demands.
• Refinement via Comparative Evaluation: Multiple candidate solutions are maintained in agent memory and compared for efficacy before choosing or synthesizing the optimal approach.

These strategies allow flexible and robust adaptation to open-ended SP workflows.

6. Applications, Impact, and Limitations

SignalLLM is demonstrated on multiple communication and sensing scenarios. Experimental evidence supports:

• Superior accuracy and F1-score in few-shot radar detection relative to state-of-the-art handcrafted and agent-based systems.
• Enhanced zero-shot activity recognition through retrieval-augmented, cross-modal LLM prompts.
• More efficient text compression and improved hyperparameter optimization guided by LLM inference.
• Superior modulated signal recognition, especially when training data or computational resources are limited.

Current limitations primarily stem from model and API size for real-time and resource-constrained deployment, and the potential for further gains in RAG/memory mechanisms. Ongoing development directions include expansion into additional SP domains (audio, biomedical, geophysical), refining adaptive planning with advanced memory and reinforcement learning, and reducing model size/cost for embedded and real-time applications.

7. Future Prospects and Research Directions

Planned advances to SignalLLM include:

• Domain Expansion: Support for a broader array of SP tasks spanning audio, biomedical, and geophysical signals.
• Enhanced Adaptive Planning: Refined retrieval-augmented generation and memory integration, reinforcement learning for improved decision-making.
• Resource-Constrained Optimization: Lighter-weight model variants for edge and real-time processing to address computation and API cost.
• Extensible Toolkit: Addition of domain-specific modules, solvers, and pre-trained models for dynamic action space extension.

These developments will further position SignalLLM as a foundational framework for fully automated, general-purpose, and adaptive signal processing pipelines.