NbQA: Polysemy in Speech, QA & Data Analysis

• NbQA is a polysemous term used in speech processing, fairness-focused QA, and notebook-based multi-step data analysis, each with unique evaluation criteria.
• In speech applications, NbQA employs a BLSTM architecture with frame-level averaging of magnitude spectrograms to approximate PESQ scores from noisy inputs.
• For non-biased QA and data analysis, NbQA frameworks leverage bias mitigation and executable notebook datasets to improve model fairness and multi-step reasoning.

Searching arXiv for recent and relevant papers on "NbQA" and its distinct usages. {"query":"NbQA arXiv speech quality assessment no-reference Quality-Net bias question answering Jupiter notebook data analysis", "max_results": 10} {"query":"(Fu et al., 2018, Ma et al., 2023, Li et al., 11 Sep 2025)", "max_results": 10} In recent arXiv usage, NbQA is a context-dependent acronym rather than a single standardized term. The papers represented by "Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM" (Fu et al., 2018), "Mitigating Bias for Question Answering Models by Tracking Bias Influence" (Ma et al., 2023), and "Jupiter: Enhancing LLM Data Analysis Capabilities via Notebook and Inference-Time Value-Guided Search" (Li et al., 11 Sep 2025) use NbQA in three distinct senses: non-intrusive / no-reference speech quality assessment, non-biased question answering, and a notebook-based question answering dataset for multi-step data analysis. The acronym therefore denotes different technical problems depending on disciplinary context, with correspondingly different supervision regimes, architectures, evaluation criteria, and deployment objectives.

1. Terminological scope

The three documented usages of NbQA occupy different parts of machine learning and speech processing. In speech processing, NbQA refers to single-ended quality estimation: predicting speech quality without access to a clean reference signal (Fu et al., 2018). In fairness-oriented NLP, NbQA refers to non-biased QA, where the objective is to reduce stereotype-aligned behavior in question answering systems while preserving task accuracy (Ma et al., 2023). In LLM-based data analysis, NbQA is the name of a large-scale dataset of standardized task–solution pairs extracted from real Jupyter notebooks and associated data files, intended to support notebook reasoning with tool use (Li et al., 11 Sep 2025).

Usage of NbQA	Domain	Central object
Non-intrusive / no-reference speech quality assessment	Speech processing	Single-ended quality predictor
Non-biased question answering	Fairness in NLP	Bias-mitigation framework for QA
Notebook-based question answering	LLM data analysis	Dataset of executable notebook tasks

This polysemy is not superficial. Each usage defines a different notion of what must be inferred from incomplete information. In speech NbQA, the missing element is the clean “golden” reference. In non-biased QA, the central issue is controlling stereotype transfer in end-user-facing answers. In notebook NbQA, the challenge is reconstructing and executing multi-step reasoning over files, code, and intermediate outputs rather than producing only a final response.

2. NbQA as non-intrusive speech quality assessment

In the speech literature represented by Quality-Net, NbQA denotes non-intrusive / no-reference speech quality assessment: the prediction of speech quality from the observed utterance alone, without access to the corresponding clean signal (Fu et al., 2018). The problem is motivated by the limitations of intrusive metrics such as PESQ, which require comparison against a clean reference and are therefore difficult to use in deployment scenarios such as online monitoring of call quality, deciding when to activate speech enhancement, evaluating enhancement output in real time, or quality-aware control in ASR or mobile communication systems.

Quality-Net formulates the task as a mapping from a variable-length utterance $u \in \mathbb{R}^{T(u)}$ to a scalar quality estimate $Q \in \mathbb{R}$. The input feature is the magnitude spectrogram. The architecture used in the experiments consists of 1 bidirectional LSTM layer with 100 nodes, followed by 2 fully connected layers with 50 ELU units each, followed by 1 linear output node interpreted as the frame-level quality score, and finally a global average layer that outputs the utterance-level estimate. The utterance score is computed by averaging frame scores,

$Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$

This design is intended both to handle variable-length inputs and to induce intermediate frame-level quality estimates.

A central technical difficulty is that supervision is available only at the utterance level. Quality-Net addresses this with a conditional frame-level constraint added to the utterance-level regression loss. The regularization weight is defined as $\alpha(Q_s)=10^{(Q_s-Q_{\max})}$, with $Q_{\max}=5$ for MOS and $Q_{\max}=4.5$ for PESQ. As described in the paper, this makes the frame constraint stronger for high-quality utterances and weaker for low-quality utterances, allowing frame scores to remain relatively uniform in clean speech while permitting temporal heterogeneity in degraded speech. The paper further manipulates the forget gate bias of the LSTM to improve locality of frame-level estimates, finding the best setting at $F_{gb}=-3$.

The experiments use the TIMIT corpus. All 4620 utterances from the TIMIT training set are used to prepare training data, including a clean set of 250 utterances, a noisy set corrupted by 90 noise types at 8 SNR levels $-10, -5, 0, 5, 10, 15, 20, 25$ dB, and an enhanced set produced by a separate BLSTM-based speech enhancement model. Evaluation uses 100 utterances randomly selected from the TIMIT test set, with unseen noises and a total of 4900 test utterances. Training targets are PESQ scores, not MOS, and optimization uses RMSprop.

Reported performance indicates strong approximation of PESQ from a single-ended signal. For clean + noisy speech, the paper reports MSE = 0.1255, LCC = 0.9054, and SRCC = 0.9065. For enhanced speech, it reports MSE = 0.1277, LCC = 0.8161, and SRCC = 0.8444. The frame constraint also improves global performance relative to training without it: MSE drops from 0.1441 to 0.1266, LCC rises from 0.8559 to 0.8749, and the variance of frame quality in clean speech drops from 4.1128 to 0.2468. The paper compares Quality-Net against an autoencoder + neural network baseline, reporting better MSE, LCC, and SRCC.

A critical interpretive point is that this NbQA formulation does not directly predict human MOS. It learns a non-intrusive approximation to PESQ, and the paper explicitly presents this as a pilot study toward eventual MOS prediction. The frame-level outputs are also only weakly supervised: they are intended as context-aware local quality indicators within an utterance, not as globally calibrated frame-level perceptual labels.

3. NbQA as non-biased question answering

In fairness-oriented NLP, the paper on Bmbi uses NbQA to denote bias-aware or non-biased QA, specifically the effort to reduce harmful stereotypes in question answering systems whose outputs may be directly consumed by end users (Ma et al., 2023). The focus is on multiple-choice QA and on cases where a model, when faced with an underspecified question, selects an answer aligned with a societal stereotype, especially a negative association involving a demographic group.

The formal QA instance is written as $Q=(ct, q, A; a)$, where $ct$ is the context passage, $Q \in \mathbb{R}$0 the question, $Q \in \mathbb{R}$1 the answer candidate set, and $Q \in \mathbb{R}$2 the correct answer. The paper studies both classification-based and generation-based formulations. It uses DeBERTaV3-large as the classification backbone and UnifiedQA-large as the generation backbone. The proposed method is Bmbi, “Bias Mitigation by tracking Bias Influence.”

Bmbi rests on the intuition that if a model learns from a biased example, that example should make another controlled example more likely to receive a biased answer. Rather than requiring instance-level bias labels, the method uses a small reference dataset $Q \in \mathbb{R}$3 for each bias category. Each reference instance has answer choices

$Q \in \mathbb{R}$4

where $Q \in \mathbb{R}$5 is the stereotyped protected group, $Q \in \mathbb{R}$6 the non-stereotyped group on the same axis, and $Q \in \mathbb{R}$7 a neutral option such as “not sure.” For each query instance, Bmbi samples a neutral reference and a ruler reference. It then compares the ruler’s answer distribution when preceded by the query instance with the ruler’s distribution when preceded by the neutral instance. The resulting bias level is defined as the shift in the probability of the stereotyped option among the non-unknown options. If the shift is positive, the query is treated as negatively biased toward $Q \in \mathbb{R}$8.

This bias estimate becomes an auxiliary optimization target. The paper defines the bias-mitigation loss as

$Q \in \mathbb{R}$9

Only positive stereotype-aligned bias is penalized; movement toward the anti-stereotyped direction is clipped to zero. Operationally, the method is trained in a multi-task setting in addition to the original QA task. The authors also introduce a new bias evaluation metric defined only over incorrectly predicted test examples and using predicted probabilities rather than binary indicators. The text states that this score ranges from $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$0 to $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$1, with $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$2 as the ideal aggregate neutrality.

The experiments are centered on BBQ, which covers 9 categories: Age, Disability status, Gender identity, Nationality, Physical appearance, Race/ethnicity, Religion, Socio-economic status, and Sexual orientation. The underlying QA models are trained on RACE, while a small reference dataset is sampled from BBQ and evaluation is conducted on the remaining BBQ examples. Implementation uses PyTorch with Hugging Face Transformers, on a single Nvidia A6000 GPU, with AdamW at learning rate $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$3, maximum 20 epochs, QA batch size 3, bias-mitigation batch size 2, beam search with 4 beams for generation, and 5 reference pairs as the best setting.

The paper reports that Bmbi significantly reduces the bias level in all 9 bias categories while maintaining comparable QA accuracy. On ambiguous BBQ instances, accuracy often increases, particularly for the generative model. The reported gains include more than 48% and 38% accuracy improvements for Sexual orientation and Physical appearance in the generative setting on ambiguous instances. Reported bias-score reductions include 21.8 points on ambiguous Socio-economic status, 15.0 on ambiguous Nationality, 22.8 on disambiguated Gender identity, and 15.7 on disambiguated Religion. The paper further reports low variance across sampled reference sets, with average variances around 0.45–0.53 in key debiased bias-score columns.

The paper also states several limitations. The bias axis is unidirectional and simplified to $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$4 versus $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$5, which cannot capture richer or intersectional patterns. The detector has low recall, so the method uses a high threshold to preserve precision. Performance depends on the quality and topical relevance of the reference instances. The paper is explicit that the influence-based bias level is a training signal from a controlled probing setup, not a standalone instance-level bias detector.

4. NbQA as a notebook-based data-analysis dataset

In the LLM data-analysis literature, NbQA is the name of a dataset: a large-scale dataset of standardized task–solution pairs extracted from real-world Jupyter notebooks and their associated data files (Li et al., 11 Sep 2025). The stated objective is to improve LLM performance on notebook-based, multi-step data analysis questions with tool use. The dataset is paired with Jupiter, a framework that treats data analysis as search over executable notebook states and uses inference-time value-guided search.

NbQA is built from a large-scale pipeline over public GitHub notebooks. The collection stage gathers about 1.6 million .ipynb notebooks and 3.2 million associated raw data files from roughly 47,000 repositories. Structural and execution filtering require valid JSON, executed code-cell order, no unexecuted cells, and no unhandled execution errors. The pipeline also excludes many common educational or competition datasets, filters out notebooks with fewer than 20 rows of data or fewer than 40 lines of code, and uses GPT-4o mini to score notebook quality from 1 to 5, retaining only notebooks with score ≥ 3. Additional processing retains at most one notebook from a repository if data files are incomplete, or at most two notebooks if complete, and removes notebooks involving neural networks, pretrained models, or GPU computation.

The resulting tasks are standardized as self-contained natural-language problems with constraints, expected output format, local data files, and an executable multi-step solution trajectory consisting of multiple thought-code-output blocks and a final thought-label block. Final answers are normalized with forms such as @answer name[answer]. The tasks span eight predefined categories: Summary Statistics, Distribution Analysis, Correlation Analysis, Outlier Detection / Anomaly Detection, Comprehensive Data Preprocessing, Feature Engineering, Machine Learning, and Visualization. The main released dataset used in experiments excludes visualization tasks because the downstream evaluations are text-based.

After filtering, the main NbQA dataset contains 38,635 task–solution pairs. Of these, 6,845 have complete, fully accessible data dependencies and low randomness, making them suitable for interactive execution, supervised fine-tuning, and value-model training; the remaining 31,790 have incomplete dependencies or some randomness. The paper also states that 12,976 visualization tasks were removed from the main text-based dataset and will be released separately. For supervised fine-tuning, the authors sample 8,975 instances from NbQA and convert them into a multi-turn ReAct-style interaction format executed by the Jupyter kernel of SandboxFusion.

Jupiter then uses the executable subset to formulate notebook reasoning as a search problem. The root node is the initial notebook state; subsequent nodes represent the accumulated thought–action history plus code execution results. During trajectory collection, search uses up to 50 iterations, maximum tree depth 10, and 3 sampled candidate branches per expansion. Terminal reward is +1.0 for fully correct answers and -1.0 for invalid output, repeated errors, or max-depth failures. A branch is terminated after 3 consecutive code execution failures, and each code execution has a 3-minute timeout. The value model is trained from search trajectories on the 6,845 executable NbQA tasks, using mean squared error against normalized MCTS $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$6-values, with LoRA over Qwen2.5-7B-Instruct and Qwen2.5-14B-Instruct backbones.

The headline results are on InfiAgent-DABench. After supervised fine-tuning on NbQA, Qwen2.5-7B-Instruct rises from 43.97% to 68.09%, and Qwen2.5-14B-Instruct rises from 69.65% to 77.04%. With Jupiter and value-guided search, the best reported results are 77.82% for the 7B model and 86.38% for the 14B model, both with the value model and no exploration. The paper reports that the 14B result surpasses Taskweaver with GPT-4o (82.49%) in the same table. Additional evaluations report transfer to DSBench, where value-guided search reaches 98.65% task completion for Qwen2.5-14B-Instruct and 89.19% for Qwen2.5-7B-Instruct at 50 iterations, and to AIME 2025, where Jupiter with a value model gives 33.3% OR / 20.0% Vote for the Qwen2.5-7B model in Python-assisted problem solving.

5. Shared methodological themes and major contrasts

Although the three usages of NbQA are unrelated as terminological origins, they share a family resemblance at the level of problem structure. Each replaces a costly or unavailable supervision signal with an indirect one. Speech NbQA removes the clean reference at inference time and learns to approximate PESQ from the degraded signal alone (Fu et al., 2018). Non-biased QA removes the need for instance-level bias labels and instead derives a debiasing signal from how a query shifts the answer distribution of a controlled ruler example (Ma et al., 2023). Notebook NbQA replaces final-answer-only supervision with executable trajectories extracted from real notebooks and then further augments inference with value-guided search over notebook states (Li et al., 11 Sep 2025).

The contrasts are equally sharp. Speech NbQA is a regression problem over acoustic sequences, with evaluation by MSE, LCC, and SRCC against PESQ. Non-biased QA is a fairness-aware multiple-choice QA problem, with evaluation by QA accuracy and a confidence-sensitive bias score defined over wrong predictions. Notebook NbQA is a dataset and systems framework for multi-step data analysis, with performance measured by benchmark task completion under tool use. The architectural substrates are correspondingly different: BLSTM sequence modeling over magnitude spectrograms in speech, DeBERTaV3-large and UnifiedQA-large in fairness-oriented QA, and Qwen2.5-Instruct models coupled to notebook execution and MCTS-derived value learning in data-analysis QA.

A plausible implication is that the acronym’s reuse reflects a recurring research pattern rather than a shared lineage: the design of systems that must make meaningful judgments when the most direct form of supervision or comparison is unavailable, partial, or impractical.

6. Limitations, misconceptions, and interpretive boundaries

A recurring misconception is that NbQA names a single subfield. The literature represented here does not support that reading. Instead, it shows three unrelated but technically substantive usages of the same acronym, each with its own problem definition and evaluation regime (Fu et al., 2018, Ma et al., 2023, Li et al., 11 Sep 2025).

Within the speech usage, an important boundary is that Quality-Net predicts PESQ scores, not MOS. It therefore inherits the biases and limitations of PESQ, and its evaluation is confined to TIMIT-derived conditions. The paper also states that frame-level quality is only weakly supervised and should be interpreted as a context-aware local estimate within an utterance rather than as an independently validated frame-level perceptual label.

Within the fairness usage, Bmbi is aimed at stereotype-aligned negative social associations with protected groups, but its bias axis is simplified to $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$7 versus $Q = \frac{1}{T(u)} \sum_{t=1}^{T(u)} q_t .$8. The paper explicitly notes low recall in the detector, dependence on the quality of reference instances, and the absence of validation beyond multiple-choice QA. Its contribution is therefore best understood as a targeted bias-mitigation framework rather than a general theory of fairness in QA.

Within the notebook usage, NbQA’s strengths in realism and executability are coupled to substantial curation choices. The entire construction pipeline relies heavily on GPT-4o and GPT-4o mini for scoring, task extraction, normalization, and review; source notebooks come from public GitHub repositories; deep learning and GPU-heavy workflows are excluded; and only 6,845 examples are fully executable for search and value training. The paper also notes missing metadata such as exact notebook counts after each stage and train/validation/test splits for NbQA itself.

Taken together, these boundaries show that NbQA is best treated as a polysemous technical acronym whose meaning must be resolved from context. In speech processing it denotes no-reference speech quality assessment; in fairness-oriented NLP it denotes non-biased QA; and in LLM data analysis it denotes a notebook-derived task–solution dataset coupled to a search framework. The substantive unity lies not in nomenclature but in the specific technical programs attached to each usage.