--- title: 'LACMAS: Learning to Act and Cooperate' url: https://www.emergentmind.com/topics/learning-to-act-and-cooperate-lacmas type: topic --- # LACMAS: Learning to Act and Cooperate Learning to Act and Cooperate (LACMAS), written as **LAC-MAS** in the named framework introduced for distributed black-box consensus optimization, denotes a trajectory-driven approach in which agents adapt both their **internal action behaviors** and their **external cooperation patterns** from historical optimization trajectories [2605.00691]. A broader reading suggested by adjacent literature is that LACMAS also names a research agenda in which decentralized agents must learn not only what to do locally, but also how, when, and with whom to cooperate under partial observability, partner uncertainty, heterogeneous goals, imperfect communication, and adaptive social response [2311.14770]. ## 1. Named framework and conceptual scope In its narrow, explicit sense, LAC-MAS is a framework for **distributed black-box consensus optimization** on a fixed connected communication graph \(G=(V,E)\), where each agent \(i\in V\) has a local black-box objective \(f_i:\mathbb{R}^D\to\mathbb{R}\), can communicate only with neighbors \(N_i\), and must improve the global objective \[ f(x)=\frac{1}{N}\sum_{i=1}^{N} f_i(x) \] while enforcing consensus. The equivalent constrained form is \[ \min \frac{1}{N}\sum_{i=1}^{N} f_i(x_i)\quad \text{s.t. } x_i=x_j,\ \forall i,j. \] The defining constraints are that only local function queries are allowed, communication is limited to neighbors in a fixed sparse graph, agents lack global state access, and the setting is often heterogeneous and nonconvex [2605.00691]. The phrase “learning to act and cooperate” also aligns with several neighboring lines of work. Those lines formulate the same underlying challenge in different mathematical languages: **POSGs** under unreliable communication, **Dec-POMDPs** for ad hoc teamwork with unseen partners, repeated games with adaptive opponents or partners, and heterogeneous-goal teamwork in which cooperation itself becomes a meta-level decision [2111.03431]. This suggests that LACMAS is not a single problem class with one canonical formalization, but a family of settings in which action selection and cooperation policy must be learned jointly. ## 2. Core mechanism of LAC-MAS The official LAC-MAS framework separates adaptation into **learning to act** and **learning to cooperate**. Learning to act concerns local search behavior in each agent’s own black-box space. Learning to cooperate concerns how much weight an agent assigns to each neighbor during decentralized consensus formation. The architecture is deliberately hybrid: a low-level execution layer runs swarm-style optimization continuously, while a high-level guidance layer uses large language models to reshape internal search coefficients and neighbor-aggregation weights from trajectory summaries rather than from end-to-end control [2605.00691]. Each agent \(i\) maintains a local particle population \(\{x_{i,p}^{(t)}\}_{p=1}^{P}\), with centroid \[ \mu_i^{(t)} = \frac{1}{P}\sum_{p=1}^{P} x_{i,p}^{(t)} \] and particle divergence \[ D_i^{(t)} = \sum_{p=1}^{P} \left\|x_{i,p}^{(t)}-\mu_i^{(t)}\right\|^2. \] Divergence functions as a regime indicator: large \(D_i^{(t)}\) corresponds to exploration, while small \(D_i^{(t)}\) corresponds to convergence or local exploitation. Each agent carries a discrete internal coefficient set \[ W_i = (w_{i,1}, w_0, w_{i,2}), \] and selects the active coefficient according to thresholds \(d_1 d_2. \end{cases} \] The subsequent update is \[ v_{i,p}^{(t+1)} = w_i^{(t)} \odot A_{i,p}^{(t)},\qquad x_{i,p}^{(t+1)} = x_{i,p}^{(t)} + v_{i,p}^{(t+1)}, \] where \(A_{i,p}^{(t)}\) is the stochastic modulation vector from the swarm backbone. The intended effect is to improve the balance between exploration, convergence, and local escape without abandoning decentralization [2605.00691]. Cooperation is likewise made adaptive. For each neighbor \(k\in N_i\), the framework summarizes trajectory information by \[ s_k^{(t)} = \big[f_k^{(t)},\, D_k^{(t)},\, |\Delta x_k|^{(t)}\big], \] encoding recent objective quality, particle divergence, and state variation. The LLM receives only local and neighbor trajectory summaries and outputs candidate cooperation weights over \(N_i\cup\{i\}\). After normalization and projection to ensure nonnegativity, graph compatibility, and unit sum, agent \(i\) performs the external update \[ x_i^{(t+1)} = \sum_{k\in N_i\cup\{i\}} a_{ik}^{(t)} x_k^{(t+1)}. \] Cooperation therefore becomes trajectory-aware and utility-sensitive rather than a fixed uniform average over neighbors [2605.00691]. ## 3. Trajectory guidance, scheduling, and consensus guarantees A distinctive feature of LAC-MAS is that the LLM is not the optimizer itself. It is a **sparse high-level guidance module** invoked only at scheduled times. Internal-action adaptation is useful when local search regimes shift, whereas cooperation adaptation is refreshed more regularly as neighbor usefulness changes. To regulate this, the framework introduces **Phased Cognitive Guidance (PCG)** based on a calibrated horizon \(T\). The cooperation-refresh gate is \[ g_{\text{ext}}(t)=\mathbb{I}\{t\in T_{\text{ext}}\}, \quad T_{\text{ext}}=\{[\!m p_{\text{ext}} T\!]\}_{m\ge 1}, \] while the internal-action refresh gate is \[ g_{\text{int}}(t)=\mathbb{I}\{t\in T_{\text{int}}\}, \quad T_{\text{int}}=\{[\!p_1T\!],[\!p_2T\!]\},\quad 0T. \] The paper interprets this as a four-stage progression: **trajectory accumulation**, **learning to act**, **learning to act and cooperate**, and **consensus-oriented cooperation** [2605.00691]. This schedule is resource-aware in two senses. First, it reduces expensive LLM calls. Second, it matches the form of adaptation to the optimization phase: early trajectory collection, mid-course internal adaptation, combined adaptation, and finally consensus-biased coordination. Algorithmically, the loop computes centroid and divergence, summarizes local and neighbor trajectories, conditionally queries the LLM, selects the divergence-triggered internal coefficient, updates particles, normalizes cooperation weights, performs the weighted consensus update, and appends to history [2605.00691]. The framework also supplies a consensus analysis. Under a fixed connected graph, normalized cooperation weights, bounded internal coefficients and perturbations, a consensus-capable swarm backbone, and asymptotically vanishing late-stage perturbations, the closed-loop dynamics are written as \[ x^{(t+1)} = A^{(t)}x^{(t)} + \varepsilon^{(t)}. \] Theorem 4.1 states \[ \lim_{t\to\infty}\|x_i^{(t)}-x_j^{(t)}\| = 0,\quad \forall i,j. \] Within the paper’s assumptions, adaptation of acting and cooperating is therefore designed not to destroy the consensus structure it seeks to improve [2605.00691]. ## 4. Adjacent multi-agent paradigms A major adjacent paradigm studies cooperation through **explicit communication** rather than through consensus weighting. In a cooperative POSG with private observations, decentralized execution, and a limited unreliable channel, agents select both an environment action and a message size \(\varphi\), with message-size selection learned by independent DQNs and message content produced by size-specific encoders. In the proposed pipeline, agents observe, receive prior messages, encode local information, choose action, choose message size, generate a message, transmit through a slotted stochastic limited-capacity channel, and then use surviving messages at the next step. In **POMNIST**, a two-step cooperative digit-prediction benchmark, adaptive communication improves over non-adaptive baselines; continuous communication performs best, pseudo-gradient discrete communication is next, and DRU performs worst among the tested message types. In **traffic junction**, communication still improves success over no communication, but adaptive message-size selection is less stable and sometimes worse than fixed-size communication [2311.14770]. A second paradigm concerns **ad hoc teamwork with unseen partners**. One meta-RL formulation treats the partner distribution as the meta-distribution and trains recurrent policies whose hidden state acts as the fast adaptation mechanism. In a Dec-POMDP with shared reward, the recurrent models receiving reward input, especially **r-RNN** and **ar-RNN**, perform best on unseen partners. Reported average test performance for **ar-RNN** is around **35.03** in the Lever Coordination Game and **34.94** in the Discrete Speaker-Listener Game. The same work reports robustness to much longer horizons (**500 steps vs. 50**) and to partner switching within an episode, and also strong ad hoc performance against independently learned partners, about **32.62** in LC and **25.63** in DSL [2111.03431]. A third paradigm addresses **completely unknown teammates** rather than merely unseen teammates. The method **OTLPP** combines **PLASTIC-Policy** with transfer learning via parameter sharing. After identifying the most similar prior teammate model, it initializes a fresh DQN for the current teammate from the selected source DQN rather than learning from scratch. In **Half Field Offense**, a \(2\) vs. \(2\) setting with five independently developed teammates—**agent2d**, **aut**, **gliders**, **helios**, and **receptivity**—the reported 100,000-episode scores are: agent2d **0.37** vs. **0.40**, helios **0.61** vs. **0.62**, aut **0.55** vs. **0.58**, gliders **0.53** vs. **0.51**, and receptivity **0.47** vs. **0.52** for PLASTIC-Policy and OTLPP respectively [2205.03289]. A fourth paradigm generalizes ad hoc teamwork to **heterogeneous goals**. In that setting, an agent must decide whether a goal should be pursued jointly, alone, or not at all, because teammate goals may only partially overlap. The method **GRILL** separates high-level goal selection by PPO from low-level goal-conditioned control learned by offline imitation, while **GRILL-M** augments this with an auxiliary teammate-modeling module. Across extended **Cooperative Reaching** and **Level-Based Foraging**, GRILL and GRILL-M outperform PPO, LIAM, and OMG. The effect of teammate modeling is explicitly tied to goal observability: at \(\sigma^2 = 0.05\), GRILL-M is **5.5%** better; at \(\sigma^2 = 1.0\), **95.2%** better; and with the cue \(\phi\) removed entirely, **142.6%** better [2603.07253]. ## 5. Incentives, social structure, and theoretical boundaries Not all act-and-cooperate systems begin from a fixed reward. One strand evolves the reward signal itself. In Pong, reward mutation is restricted to three predefined screen regions \(a_1,a_2,a_3\), each taking value \(0\) or \(1\), yielding \(2^3=8\) possible reward signals. Optimizing the goals of **winning**, **losing**, and **cooperation** leads to final surviving reward signals **100**, **000**, and **110**, respectively. The paper reports that evolved rewards can be more stable than score-based baselines, with **110** especially notable as a non-obvious but strong cooperative signal [2105.08187]. Another strand models cooperation through **reputation**. In the **Clean Up** social dilemma, five agents interact in a \(23\times 16\) world containing an orchard and a polluted river; agents collect apples for individual reward, clean pollution with a water-beam tool, and can punish others with a ticketing tool. Cooperation depends on where and when agents clean, not just how much they contribute. The model uses independent multi-agent A2C with heterogeneous intrinsic reputational terms, with \(\alpha \sim \mathcal{U}(2.4, 3.0)\), \(\beta \sim \mathcal{U}(0.16, 0.20)\), temporal smoothing \(c_j^t = \lambda c_j^{t-1} + q_j^t\) with \(\lambda = 0.97\), and anonymous visibility limited by \(R=9\) under the \(\ell^\infty\) norm. Under identifiability, both humans and agents contribute more, obtain higher collective return, exhibit less territoriality, and show more turn-taking than under anonymity [2103.04982]. A central theoretical boundary is that learning to cooperate with **adaptive partners** is not generically solvable. In infinitely repeated matrix games, adaptive regret compares the learner’s long-run payoff with that of the best expert strategy against the partner’s actual adaptive response. The paper proves that for any learning algorithm, there exists an open-ended adaptive partner strategy that induces nontrivial adaptive regret, even in an \(N\times N\) coordination game with fixed-action experts. The underlying dilemma is structural: a learner that eventually commits can be exploited after commitment, whereas a learner that keeps exploring can be denied sufficiently long evaluation windows for any expert [2206.10614]. A related population-level result studies finitely repeated two-player general-sum matrix games with private utilities. There, a target population is assumed to be **consistent** in the sense of bounded external regret and **compatible** in the sense that pairwise interaction reaches at least some Pareto-optimal Nash equilibrium with high probability. Those assumptions are shown to be insufficient for zero-shot cooperation by an outsider. Offline observations are therefore introduced, yielding a dataset \(\mathcal{D}=\{(\theta_1^j,\theta_2^j,h_T^j)\mid j\in[n]\}\), lower bounds showing that naive imitation can require sample complexity \(\Omega(N^{3k})\), and an **imitate-then-commit** upper bound whose dependence is on a coordination prefix \(\tilde T\) rather than the full horizon \(T\) [2407.00419]. ## 6. Empirical domains, interpretations, and unresolved questions Across the literature, LACMAS-related work is tested in markedly different domains. | Setting | Representative environments | Primary cooperation issue | |---|---|---| | Distributed optimization | \(F1\)–\(F10\), wireless sensor network localization | Consensus, trajectory-aware neighbor weighting [2605.00691] | | Communication-constrained MARL | POMNIST, traffic junction | Message content and message-size adaptation under drops and collisions [2311.14770] | | Ad hoc teamwork | Lever Coordination, DSL, HFO, Cooperative Reaching, LBF | Unseen teammates, unknown teammates, heterogeneous goals [2111.03431] | Within distributed optimization proper, LAC-MAS is evaluated on **10 distributed black-box benchmark functions** \(F1\)–\(F10\) with **100 decision variables** and **20 agents**, against **MASOIE**, **GFPDO**, **RGF**, and **DA-PSO**, using **final fitness**, **cumulative communication cost until convergence**, and **disagreement over iterations**. Convergence is declared when disagreement drops below \(10^{-7}\). The paper also reports a transfer study on **wireless sensor network multi-target localization**, in which LAC-MAS attains the lowest estimation error across different numbers of targets [2605.00691]. Several misconceptions are corrected by the broader record. LACMAS is not reducible to a single communication protocol, a single MARL architecture, or a single assumption of shared reward. Some work assumes full cooperation and shared returns; some assumes heterogeneous private goals; some assumes explicit communication; some addresses cooperation without prior coordination; and some proves that unrestricted adaptive cooperation is impossible. Cooperation is also not uniformly desirable: one paper makes the decision of **when to cooperate** the central problem, rather than assuming that teamwork is always beneficial [2603.07253]. The limitations are similarly heterogeneous. In LAC-MAS itself, the communication graph is fixed and connected, cooperation only reweights existing neighbors, the schedule depends on a calibrated horizon \(T\), and the theoretical guarantee relies on boundedness and vanishing-perturbation assumptions [2605.00691]. In adaptive communication over imperfect channels, the message-size value target can have high variance, performance degrades when reliability is poor and coordination demand is high, and the channel model remains simplistic [2311.14770]. In heterogeneous-goal teamwork, the experiments are limited to fully observable environments, two agents, and an offline-first low-level policy [2603.07253]. In unknown-teammate transfer, performance still depends on identifying a meaningful source policy [2205.03289]. In meta-RL ad hoc teamwork, generalization improves with both the quantity and diversity of training partners, implying that narrow partner distributions sharply limit robustness [2111.03431]. Taken together, the literature supports a precise interpretation. In the narrow sense, LAC-MAS is a trajectory-driven framework that couples adaptive local search with adaptive decentralized cooperation in distributed black-box consensus optimization [2605.00691]. In the broader sense suggested by adjacent work, LACMAS denotes a research program on how agents learn action, communication, coordination, adaptation, and social response jointly—under exactly those conditions where cooperation is useful, difficult, and often only partially identifiable from interaction alone. Source: https://www.emergentmind.com/topics/learning-to-act-and-cooperate-lacmas