NanoMsg: Brokerless Messaging Library

• NanoMsg is a brokerless messaging library re-imagined from ZeroMQ, offering low memory footprint and direct peer-to-peer communication.
• Benchmark studies show NanoMsg excels in small-message latency, CPU efficiency, and memory usage, particularly in IPC transport.
• Within FairMQ and ALICE-O2, NanoMsg acts as a pluggable transport backend enabling polyglot interoperability and modular process design.

Searching arXiv for papers on NanoMsg, FairMQ, and fer.

NanoMsg, written as both “nanomsg” and “NanoMsg” in the cited literature, is a brokerless messaging library and, in the FairMQ ecosystem, a pluggable transport backend for message-based communication among distributed processes. It is presented as a complete rewrite in C that re-imagines ZeroMQ with a lower memory footprint, improved algorithmic efficiency, and full POSIX compliance, while preserving a simple, transport-agnostic socket API for direct peer-to-peer communication without an intermediary broker. In high-energy physics software, particularly within the ALICE collaboration’s transition to ALICE-O2, nanomsg appears as one of the interchangeable wire transports beneath FairMQ’s device-and-channel abstraction, including polyglot interoperability with the Go-based toolkit fer (Corte et al., 11 Aug 2025, Binet, 2019).

1. Conceptual position and design goals

NanoMsg is classified as a brokerless messaging library. In the cited benchmarking taxonomy, this means that it omits an intermediary broker and instead provides direct communication between publisher and subscriber endpoints. The immediate consequence is a reduced hop count and potentially lower latency, while reliability, retransmission, and delivery guarantees are left to the application if needed (Corte et al., 11 Aug 2025).

The library is explicitly characterized as a re-imagining of ZeroMQ. The benchmark study describes it as a complete rewrite in C intended to improve maintainability and development through full POSIX compliance, while also targeting a lower memory footprint and improved algorithmic efficiency. At the API level, it retains the same general style of simple, transport-agnostic socket API associated with ZeroMQ-like brokerless systems (Corte et al., 11 Aug 2025).

A recurrent point in the literature is that NanoMsg is best understood as a transport substrate, not as a full application framework. In FairMQ, application code is structured around devices and channels, whereas nanomsg sits beneath that abstraction as an interchangeable communication backend. This distinction matters because it separates application semantics from transport choice: the program is written against a message-passing model, and the transport library provides the concrete data movement.

2. NanoMsg in FairMQ and ALICE-O2

Within the ALICE software stack, nanomsg is presented in relation to FairMQ, a distributed processing toolkit written in C++. FairMQ models applications as UNIX processes connected by message queues, with those processes potentially residing in the same address space, on the same machine, or on different machines. The paper states that the nodes of these pipelines are processes exchanging data via message queues either through ZeroMQ or nanomsg, and later describes FairMQ as having pluggable transports (ZeroMQ, nanomsg, shmem, InfiniBand) (Binet, 2019).

This transport role is central to ALICE-O2. To meet Run 3 data rates and volumes, the ALICE collaboration is merging online and offline infrastructures into a common framework, ALICE-O2, which is based on FairRoot and FairMQ. In that setting, nanomsg functions as one of the interchangeable mechanisms by which device-to-device communication is realized across process and machine boundaries (Binet, 2019).

FairMQ does not present direct socket programming as its primary programming model. Instead, it exposes message queues / channels between devices. The application is therefore expressed as a pipeline of devices, and nanomsg is the implementation choice underneath those queues. A common misconception is to treat nanomsg in this context as the application framework itself; the paper instead places it strictly at the transport layer beneath the FairMQ abstraction.

3. Messaging abstraction, channels, and topology

FairMQ’s device model is defined by a C++ base class whose public communication primitives are Send(...) and Receive(...), while lifecycle hooks include Init(), InitTask(), Run(), ConditionalRun(), Pause(), Reset(), and ResetTask() (Binet, 2019). This makes the transport backend largely invisible to application logic: devices receive messages from upstream, process them, send messages downstream, or react to control commands.

The communication semantics are explicit. Messages are concrete objects sent or received on named channels, with an integer index for channel or socket selection:

int Send(const std::unique_ptr<FairMQMessage>& msg, const std::string& chan, const int i) const;
int Receive(const std::unique_ptr<FairMQMessage>& msg, const std::string& chan, const int i) const;

  











  
The paper also identifies the media that can be chosen between devices as `tcp`, `udp`, `ipc`, `inproc`, and `shared-memory` [1901.04834]. In this formulation, nanomsg is one element of the transport layer behind channels, whereas the higher-level topology is specified independently. Supported communication patterns explicitly include `router/dealer`, `request/reply`, `publish/subscribe`, and `client/server` [1901.04834].

Configuration is expressed in JSON and includes **device IDs**, **channel names**, **socket types**, **binding method**, and **transport address**. The example provided uses values such as `"type": "push"`, `"method": "bind"`, and `"address": "tcp://*:5555"`, together with `"sndBufSize": 1000`, `"rcvBufSize": 1000`, and `"rateLogging": 0` [1901.04834]. This is significant because the same channel/socket configuration mechanism governs nanomsg-based deployments as well as other FairMQ transports.

## 4. Polyglot interoperability through fer

The principal contribution of the ALICE paper is **fer**, a **Go-based toolkit** designed to interoperate with C++ FairMQ processes. Its relevance to NanoMsg is that interoperability is achieved through the **same transport layer** used by FairMQ. Since FairMQ channels may be implemented through nanomsg, fer can participate in C++ pipelines as long as both sides agree on **channel names**, **topology**, **socket/transport type**, and **configuration** [1901.04834].

On the Go side, a device implements:


go
type Device interface { Run(ctl Controler) error }

  



  
and receives named channels through a `Controler`:


go
type Controler interface {
    Logger
    Chan(name string, i int) (chan Msg, error)
    Done() chan Cmd
}

type Msg struct {
    Data []byte // Data is the message payload.
    Err  error  // Err indicates whether an error occured.
}
type Cmd byte

A Go device obtains channels such as ctl.Chan("data1", 0) and ctl.Chan("data2", 0), then uses a receive-transform-send loop that is a direct analogue of the FairMQ device loop (Binet, 2019).

NanoMsg had a particularly important implementation role in this setting because, at the time fer was developed, there was already a pure Go implementation for nanomsg, but not for ZeroMQ. The paper identifies three consequences: it avoided cgo overhead, preserved Go’s easy deployment model, and enabled pure-Go integration with a distributed messaging backend (Binet, 2019). This makes nanomsg not merely a transport option in principle, but a practical enabler of polyglot process composition in the specific FairMQ/fer context.

5. Quantitative performance profile

A later benchmark study evaluates ZeroMQ, NanoMsg, and NNG under an extensive open-source benchmarking suite focused exclusively on PUB/SUB. The quantitative comparison uses libzmq v4.3.4 plus CZMQ v4.2.1, libnanomsg v1.1.5, and libnng v1.9.0 on a single host with an Intel Xeon w3-2435, 8 cores @ 3.1 GHz, and 64 GB RAM. The testbed varies the number of subscribers among 1, 2, 4, 8, payload size from 1 KB to 512 KB, and publishing interval $\mathcal{T}$ between 1000 \ \mu s for latency and 0 \ \mu s for throughput and CPU analysis, with $\mathcal{C} = 5000$ messages and publisher startup delay $\mathcal{D} = 1000$ ms (Corte et al., 11 Aug 2025).

Across that study, NanoMsg is described as showing best or near-best latency for small messages, strong CPU efficiency across many scenarios, lower throughput than ZeroMQ, worse scaling than ZeroMQ for larger payloads and many subscribers, best memory efficiency, and good jitter, comparable to ZeroMQ. The paper also reports a clear performance threshold around 32–64 KB or 128 KB, depending on transport, after which ZeroMQ often overtakes it (Corte et al., 11 Aug 2025).

The following summary condenses the main results already reported in the study.

Transport regime	NanoMsg strengths	Reported comparison
In-Process	Lowest latency for small payloads; best CPU efficiency up to 32 KB; peak throughput ~3.8 GB/s	ZeroMQ reaches ~4.8 GB/s and dominates larger payloads
IPC	Best latency up to 128 KB; lowest CPU for all message sizes; jitter around 1 µs; most memory-efficient	ZeroMQ performs considerably better beyond 128 KB and has highest throughput ~2.9 GB/s
TCP	Lowest latency up to 8 KB; generally lowest CPU usage; peak throughput around 1.5 GB/s at 128 KB	ZeroMQ peak throughput ~2.9 GB/s; NanoMsg shows “strange behavior” from 8 KB to 64 KB

The IPC results are especially favorable to NanoMsg. With 32 KB payload and 1 subscriber, it has the best average latency, minimum latency, p90, and p99, while ZeroMQ has the best maximum latency (Corte et al., 11 Aug 2025). In TCP, however, NanoMsg exhibits an explicitly noted anomaly: between 8 KB and 64 KB, the paper reports “strange behavior” or a possible internal limitation, and for payloads greater than 128 KB, ZeroMQ is best for latency (Corte et al., 11 Aug 2025).

The study also defines jitter as

$$\text{jitter} = \frac{1}{M-1} \sum_{i=1}^{M-1} |l_i - l_{i-1}|$$

where $l_i$ is the one-way latency of the $i$-th received message (Corte et al., 11 Aug 2025). NanoMsg’s reported jitter remains around 1 \ \mu s in IPC and is described as comparable to ZeroMQ. Memory usage is reported as below 0.01\% in most cases, with NanoMsg being the most memory-efficient among the three libraries (Corte et al., 11 Aug 2025).

6. Trade-offs, maintenance status, and comparison points

The literature presents NanoMsg as a library with a sharply defined trade-off profile. It is strongest when workloads emphasize small payloads, low-to-moderate subscriber counts, latency sensitivity, CPU efficiency, IPC transport, and memory-constrained settings. It tends to degrade when payloads become large, especially 128 KB and above, when subscriber counts rise substantially, and when the workload is throughput-oriented rather than latency-oriented (Corte et al., 11 Aug 2025).

Relative to ZeroMQ, NanoMsg is reported to win on small-message latency, average latency in IPC, CPU efficiency in many scenarios, memory efficiency, and roughly tie on jitter. ZeroMQ, by contrast, wins on throughput almost everywhere, large-payload latency, worst-case latency in IPC, and scalability with increasing subscribers, especially throughput (Corte et al., 11 Aug 2025). In the FairMQ paper, the two are also treated as alternative off-the-shelf transport libraries that can serve the same message-passing abstraction, with FairMQ remaining agnostic to the choice (Binet, 2019).

Relative to NNG, NanoMsg usually outperforms it in average latency, CPU efficiency, memory efficiency, and jitter, while NNG can outperform NanoMsg in some larger-payload throughput cases and certain TCP latency regions (Corte et al., 11 Aug 2025). The study further notes that NNG is the actively developed successor family, whereas NanoMsg is not in active development and is in sustaining mode (Corte et al., 11 Aug 2025).

A second comparison point arises from the fer interoperability work. There, the relevant distinction is not NanoMsg versus ZeroMQ at the message-level API alone, but pure-Go transport versus cgo-bound integration. In a simple benchmark that sends a single uint64 token through a three-stage pipeline—source, processor, sink—the paper reports about 70 $\mu s$ mean time of flight for the czmq C++ version called from Go, versus about 45 $\mu s$ for the pure Go nanomsg and ZeroMQ transports, with MaxRSS around 22 MB across all configurations (Binet, 2019). This suggests that, in the tested setup, implementation strategy and language binding overhead can matter as much as library family.

7. Interpretation and usage domains

Taken together, the cited studies place NanoMsg in two overlapping but distinct roles. First, it is a general-purpose brokerless messaging library with a profile optimized toward low overhead and efficient small-message handling. Second, in the FairMQ ecosystem, it is an interchangeable transport backend that enables process isolation, language agnosticism, and a POD-oriented data model because communication is forced through explicit message queues (Binet, 2019).

The ALICE paper emphasizes architectural consequences of this design. Since each device is a separate process, process boundaries prevent one device from corrupting another’s memory, and the requirement to exchange data through message queues “healthily constrains” the data model toward plain-old-data style payloads (Binet, 2019). NanoMsg participates in that architecture by providing the communication substrate for inter-process and inter-machine links.

The benchmark paper, by contrast, frames NanoMsg as a resource-efficient, small-message specialist. Its practical guidance is explicit: choose NanoMsg when very good latency for small messages, strong CPU efficiency, the lowest memory usage, and a lightweight brokerless API are needed, provided the workload is not dominated by very large payloads or many subscribers. Prefer ZeroMQ when throughput, large payloads, or subscriber scaling are the top priorities. Consider NNG when active maintenance is a central concern, even though it is usually not the top performer in those benchmarks (Corte et al., 11 Aug 2025).

An objective synthesis is therefore straightforward. NanoMsg is neither a universal performance winner nor a legacy artifact without contemporary relevance. In the literature, it is a transport and messaging substrate whose value depends strongly on workload regime: particularly effective for small to medium payloads, latency-sensitive paths, CPU- and memory-constrained environments, and, in the ALICE/FairMQ setting, polyglot distributed frameworks built around message queues rather than shared-memory application logic (Corte et al., 11 Aug 2025, Binet, 2019).