---
title: 'ICRL-JSA: Underwater MARL & LLM Adaptation'
url: https://www.emergentmind.com/topics/icrl-jsa
type: topic
---

# ICRL-JSA: Underwater MARL & LLM Adaptation

ICRL-JSA is an overloaded acronym rather than a single standardized term in recent arXiv literature. In one explicit usage, it denotes a deep multi-agent reinforcement learning optimizer for **joint link scheduling and power allocation** in **imperfect and energy-constrained underwater wireless sensor networks (IC-UWSNs)** [2508.07679]. In another, it denotes **In-Context Representation Learning – Joint Space Adaptation**, a training-free procedure for injecting non-text foundation-model representations into text-based large language models [2509.17552]. The neighboring acronyms are likewise polysemous: **ICRL** appears as inverse constrained reinforcement learning, in-context reinforcement learning, and learning to internalize self-critique with reinforcement learning, while **JSA** in several latent-variable learning papers denotes joint stochastic approximation [2409.07569][2506.06303][2605.15224][2207.12235].

## 1. Terminological scope

Within the cited literature, the acronym has at least two explicit expansions.

| Usage | Expansion | Core role |
|---|---|---|
| ICRL-JSA [2508.07679] | Deep MARL-based optimizer for joint link scheduling and power allocation | Solves the FER-communication optimization problem in IC-UWSNs |
| ICRL-JSA [2509.17552] | In-Context Representation Learning – Joint Space Adaptation | Training-free mapping of non-text FM representations into LLM input space |

This ambiguity is amplified by the surrounding literatures. The survey on inverse constrained reinforcement learning uses **ICRL** for the problem of inferring implicit constraints from demonstrations [2409.07569]. The prompting-based work “Reward Is Enough” uses **ICRL** for in-context reinforcement learning in fixed-parameter LLMs [2506.06303]. The self-critique paper defines **ICRL** as “learning to internalize self-critique with reinforcement learning” [2605.15224]. Separately, **JSA** denotes joint stochastic approximation in semi-supervised task-oriented dialogue, retrieval-augmented generation, and directed latent-variable models [2207.12235][2508.18168][2505.18558].

## 2. ICRL-JSA in imperfect and energy-constrained underwater wireless sensor networks

In the underwater networking paper, ICRL-JSA is a deep multi-agent reinforcement learning scheme for **Joint Link Scheduling and Power Allocation** that targets **fair, efficient, and reliable communication** in **imperfect and energy-constrained underwater wireless sensor networks** [2508.07679]. The system is a slotted underwater acoustic network with half-duplex, omnidirectional modems, a single carrier frequency \(f\), and bandwidth \(B\). Nodes are deployed in 3D underwater space, undergo passive mobility induced by water currents, and communicate over distance- and frequency-dependent acoustic channels. The model also includes two nonidealities that define the “IC-UWSN” setting: severely limited battery energy and irreversible node malfunctions.

The physical layer is parameterized by the received SINR
$$
\gamma_{i,\tilde{i}} =
\frac{
\eta_0\, p_i\, A(d_{i,\tilde{i}}, f)^{-1}
}{
\eta_0 \sum_{j\in \mathcal{N}_{send}^{-i}} p_j A(d_{j,\tilde{i}}, f)^{-1} + I_s + I_a
},
$$
where \(A(d,f)\) follows the Urick attenuation model, \(I_s\) denotes interference from other acoustic entities, and \(I_a\) denotes ambient noise [2508.07679]. Successful reception requires \(\gamma_{i,\tilde{i}}^t \ge \gamma^{th}\), and the corresponding slot-level capacity is
$$
c_{i,\tilde{i}}^t =
\begin{cases}
B \log_2(1+\gamma_{i,\tilde{i}}^t), & \gamma_{i,\tilde{i}}^t \ge \gamma^{th} \\
0, & \text{otherwise.}
\end{cases}
$$

Energy and reliability constraints are built directly into the network model. Each node starts with battery capacity \(E^0\), consumes \(e_i^t = p_i^t T_{tran}\) energy in slot \(t\), and accumulates
$$
E_i^t = \sum_{\tau=1}^t e_i^\tau.
$$
The network lifetime is defined over intelligent nodes, and the optimization imposes a target lifetime \(\delta^0\). Malfunctions are modeled by a Bernoulli indicator \(\kappa_i\in\{0,1\}\), with \(\kappa_i=1\) denoting an irrecoverably malfunctioning node that stops transmission [2508.07679].

The optimization target is the **FER-communication optimization problem (FERCOP)**. Its utility combines three indices: spatial reuse, communication fairness, and ineffective communication. The resulting network utility is
$$
\mathcal{U}_{\mathcal{N}} =
\alpha \mathcal{U}_{\mathcal{N}}^{(Spa)}
+ \beta \mathcal{U}_{\mathcal{N}}^{(Fair)}
+ \mu \mathcal{U}_{\mathcal{N}}^{(Ief)}.
$$
The spatial reuse index measures the fraction of scheduled links that succeed, the fairness index is Jain’s fairness index over successful receptions, and the ineffective communication index lies in \([-1,0]\), with \(-1\) meaning all scheduled transmissions failed and \(0\) meaning none were ineffective [2508.07679]. This formulation makes explicit that the scheme is not simply maximizing throughput; it is balancing spatial reuse, fairness across transmitters, and reliability under lifetime constraints.

## 3. RL formulation, decentralized control, and advanced training

ICRL-JSA models the control problem as a **Decentralized Partially Observable Markov Decision Process (Dec-POMDP)** and uses **centralized training, decentralized execution (CTDE)** [2508.07679]. Each intelligent node is an agent. The environment comprises acoustic propagation, interference, residual energies, node positions, passive mobility, and stochastic malfunctions. During execution, each node acts from a local observation; during training, a virtual centralized controller aggregates rewards and computes joint TD targets.

The local observation of node \(i\) includes self information, intended receiver positions, and interferer information. The self component contains the node position, residual-energy ratio, last-slot reception indicator, previous transmission ratio, previous transmit power, and node identifier. The action is discrete power selection,
$$
\mathcal{A} = \mathcal{P} \cup \{0\},
$$
where \(0\) means the node does not transmit in the slot. Scheduling is therefore implicit: a node is effectively unscheduled when its action is zero [2508.07679].

The slot-level team reward is
$$
r^t =
\begin{cases}
\alpha \mathcal{I}_{\mathcal{N}}^{(Spa)}(t)
+ \beta \mathcal{I}_{\mathcal{N},h}^{(Fair)}(t)
- \mu \varsigma(t), & \delta_{\mathcal{N}} \ge \delta^0 \\
-100, & \delta_{\mathcal{N}} < \delta^0,
\end{cases}
$$
so the learned behavior is driven directly by the same fairness, efficiency, and reliability structure used in FERCOP [2508.07679]. If the lifetime requirement is violated, the episode terminates early with a hard penalty.

At the function-approximation level, each agent uses a deep recurrent Q-network with two fully connected ReLU layers of width 64, a GRU with 64 hidden units, and an output layer of size \(|\mathcal{A}|\) [2508.07679]. Parameter sharing is used across agents. Multi-agent coordination is handled with **Value Decomposition Networks (VDN)**, which define
$$
Q_{tot}(\boldsymbol{\tau},\mathbf{a}) = \sum_{i=1}^N Q_i(\tau_i,a_i),
$$
thereby satisfying the individual-global-maximization property and permitting decentralized greedy action selection at execution time [2508.07679].

The paper further introduces an **advanced training mechanism** tailored to underwater uncertainty and device failures. The mechanism contains a **Performance Evaluation (PE-module)** and a **Malfunction Rate Adjustment (MRA-module)** [2508.07679]. The PE-module estimates \(u^{max}\) by training in a perfect UWSN and estimates \(u^{min}\) using random power allocation, then defines a threshold \(u^{th}\in[u^{min},u^{max}]\). Model selection also accounts for malfunction rate: among candidate policies, those that sustain high reward under higher malfunction rates are preferred, which makes robustness to node failure an explicit training objective rather than an afterthought.

## 4. Reported behavior and empirical claims in the underwater literature

The underwater paper states that conventional RL methods are unable to address the challenges posed by underwater environments and IC-UWSNs, specifically citing complex acoustic channels, limited energy supplies, and unexpected node malfunctions [2508.07679]. Its simulations therefore evaluate not just nominal throughput but the FER objective under those combined constraints.

The reported empirical conclusion is qualitative but unambiguous: simulation results demonstrate the superiority of the proposed ICRL-JSA scheme with the advanced training mechanism compared to various benchmark algorithms [2508.07679]. Because the extracted record does not enumerate the benchmark list or numerical margins, the literature-supported statement is that the contribution is comparative and benchmarked, but not that a particular gain or ranking can be reproduced from the abbreviated metadata alone.

This suggests a specific methodological role for ICRL-JSA within underwater networking research. Rather than separating scheduling and power control, the method treats them as a single cooperative control problem under partial observability. Rather than assuming perfect nodes, it trains directly against stochastic malfunction processes. The paper’s terminology therefore binds “ICRL-JSA” tightly to the FER objective and to the IC-UWSN system model, not merely to a generic MARL scheduler [2508.07679].

## 5. ICRL-JSA as In-Context Representation Learning – Joint Space Adaptation

A second explicit use of the same acronym appears in multimodal LLM research. There, **ICRL-JSA** means **In-Context Representation Learning – Joint Space Adaptation**, a training-free method for allowing a text-only LLM to use representations from a non-text foundation model in few-shot inference [2509.17552]. The motivating question is whether a text-only LLM can exploit non-text representations at test time without any supervised adapter training.

In this setting, a foundation model \(\phi\) produces vectors in \(\mathbb{R}^{d_{FM}}\), while the LLM consumes embeddings in \(\mathbb{R}^{d_{LLM}}\). ICRL-JSA introduces a projector
$$
P : \mathbb{R}^{d_{FM}} \to \mathbb{R}^{d_{LLM}},
$$
and studies both text-level and embedding-level injection schemes [2509.17552]. The implementation space includes PCA serialization, Zero-Pad, Random Projection, and two alignment procedures, **OT-Embed** and **OT-PCA**, based on affine matching of mean and variance between projected FM embeddings and LLM-side target distributions. The paper gives theoretical support for linear random projection, including norm concentration and cosine-similarity preservation, and reports that nonlinear activations such as ReLU or GELU degrade performance [2509.17552].

The empirical domain is molecular property prediction, with additional preliminary vision and audio experiments. In this usage, JSA means **Joint Space Adaptation**, not joint stochastic approximation. The method is explicitly training-free and is positioned as a proof-of-concept for adaptable multimodal generalization by frozen text LLMs [2509.17552]. The overlap in acronym with the underwater MARL method is terminological only.

## 6. Broader lineages of “ICRL” and “JSA”

The acronym collision is easier to understand once the adjacent literatures are laid out. Inverse constrained reinforcement learning defines **ICRL** as the problem of inferring implicit constraints from expert demonstrations in constrained Markov decision processes [2409.07569]. In-context reinforcement learning uses **ICRL** for fixed-parameter agents whose performance improves as interaction history grows; this includes prompting-based LLM studies and transformer pretraining on RL histories [2506.06303][2505.15143][2509.18389]. A separate line defines **ICRL** as “learning to internalize self-critique with reinforcement learning,” in which a solver and critic are jointly trained from a shared backbone and the critic is rewarded by the solver’s subsequent improvement [2605.15224].

The **JSA** lineage is more consistent: in latent-variable learning, it denotes **joint stochastic approximation**. JSA-TOD applies it to semi-supervised task-oriented dialogue with discrete latent belief states and acts, and reports that using 20% labels performs close to the full-supervised baseline on MultiWOZ2.1 [2207.12235]. JSA-KRTOD extends the paradigm to retrieval-based knowledge selection in noisy real-life task-oriented dialogue, using latent knowledge snippets and system actions on the MobileCS dataset [2305.13199]. JSA-RAG treats retrieved passages as discrete latent variables in end-to-end retrieval-augmented generation and reports gains over vanilla RAG and variational RAG on five datasets [2508.18168]. JSA autoencoders formulate deep directed generative modeling as direct log-likelihood maximization plus minimization of the inclusive KL divergence between the posterior and the inference model [2505.18558].

These threads are methodologically distinct. The underwater ICRL-JSA is a deep MARL controller for acoustic networks [2508.07679]. The multimodal ICRL-JSA is a training-free representation-alignment method for LLM prompting [2509.17552]. JSA in dialogue and RAG denotes a stochastic-approximation estimator for discrete latent-variable models [2207.12235][2508.18168]. The shared acronym does not imply shared algorithmic structure.

## 7. Disambiguation and significance

Because the cited literature uses the same string for distinct methods, “ICRL-JSA” is best interpreted only together with its domain and source. In underwater wireless networking, it refers to a FER-oriented MARL optimizer that jointly learns link scheduling and power allocation under energy and malfunction constraints [2508.07679]. In multimodal LLM prompting, it refers to In-Context Representation Learning – Joint Space Adaptation, where frozen LLMs ingest adapted non-text representations without fine-tuning [2509.17552].

This multiplicity also reflects a wider naming pattern in current arXiv research. “ICRL” can denote inverse constrained reinforcement learning, in-context reinforcement learning, or internalized self-critique [2409.07569][2506.06303][2605.15224]. “JSA” can denote joint stochastic approximation in probabilistic latent-variable learning [2207.12235][2505.18558]. The acronym therefore functions less as a stable field-wide term than as a local label attached to separate research programs.

A plausible implication is that precise citation practice is essential for this topic. In technical writing, identifying the intended expansion together with the paper identifier—rather than relying on the bare acronym—avoids conflating underwater MARL control, training-free multimodal prompting, latent-variable stochastic approximation, and several unrelated forms of “ICRL.”

Source: https://www.emergentmind.com/topics/icrl-jsa