BanglaCodeAct: NL2Code & Mixed-Language Tech

Updated 5 July 2026

BanglaCodeAct is a multi-agent framework integrating execution-guided feedback to convert Bangla instructions into effective Python code.
The framework also acts as a blueprint for Bangla NLP tasks including POS tagging, sentiment analysis, and offensive language detection in code-mixed texts.
Iterative execution and multilingual prompting in BanglaCodeAct yield performance improvements over static methods despite challenges like transliteration variability and low-resource data.

BanglaCodeAct denotes, in its explicit arXiv usage, a ReAct-style, multi-agent, self-correcting framework for Bangla-to-Python code generation that replaces task-specific fine-tuning with multilingual prompting and iterative execution feedback (Islam et al., 27 Nov 2025). In adjacent Bangla NLP work, the same label is also used prospectively or retrospectively for systems and benchmarks centered on Bangla–English code-mixing, Romanized Bangla, dialectal variation, and low-resource supervision, including word-level language identification, customer-service retrieval-augmented generation, sentiment analysis, POS tagging, and offensive-language detection (Mandal et al., 2018). Taken together, these usages suggest an umbrella interpretation in which BanglaCodeAct names both a concrete NL2Code framework and a broader engineering pattern for Bangla-centric, code-mixed language technologies (Kruk et al., 28 Mar 2025).

1. Scope, nomenclature, and research setting

The named formulation of BanglaCodeAct appears in the Bangla-to-Python code-generation literature, where it is introduced as an agent-based framework for executing, testing, and revising Python programs generated from Bangla instructions (Islam et al., 27 Nov 2025). In other works, the term functions as a blueprint or adaptation target rather than as the original name of the system being described. A clear example is the POS-tagging literature, which explicitly states that “BanglaCodeAct” does not appear in the paper even though the pipeline is directly applicable to Bangla code-mixed NLP (Raha et al., 2020).

Context	Task	Representative source
Explicit BanglaCodeAct	Bangla-to-Python NL2Code	(Islam et al., 27 Nov 2025)
BanglaCodeAct as blueprint	Bengali-English customer-service RAG	(Kruk et al., 28 Mar 2025)
BanglaCodeAct-style antecedents	LID, sentiment, POS, offensive-language modeling in Romanized or code-mixed Bangla	(Mandal et al., 2018, Mandal et al., 2018, Raha et al., 2020, Raihan et al., 2023)

Across these settings, several constraints recur. Bangla data are often low-resource; Bangla tokens are frequently transliterated into Roman script with inconsistent phonetic spellings; English lexical mixing is common; and many tasks are complicated by noise such as elongated spellings, numerals, special characters, dialectal variants, or mixed morphology (Mandal et al., 2018). In the customer-service setting, these constraints extend to mixed-script queries, South Asian Englishes, sparse non-parallel enterprise knowledge bases, and hallucination risk in generative systems (Kruk et al., 28 Mar 2025). In offensive-language modeling, Romanized Bangla and code-mixing are treated as first-class phenomena rather than as annotation noise (Raihan et al., 2023).

2. BanglaCodeAct as a Bangla-to-Python NL2Code framework

In its formal definition, BanglaCodeAct is a multi-agent architecture organized around a Thought–Code–Observation loop (Islam et al., 27 Nov 2025). The framework instantiates four role-specific agents. The Planner/Reasoner interprets the Bangla instruction and emits a structured <thought> in Bangla; the Coder converts that plan into Python within <code>; the Tester executes the code in a sandboxed Python REPL and returns <observation>; and the Fixer analyzes failures, revises the plan, and hands a new thought back to the Coder. A controller manages retries, iteration limits, and stopping criteria.

The loop is execution-guided rather than purely generative. Code is run against the unit tests supplied by the mHumanEval dataset, with stdout, stderr, exceptions, and pass/fail state summarized as observation. The framework stops when all tests pass or when max_iters is reached; it also supports early stopping when the same error recurs without meaningful plan change. The reported implementation uses timeout=5s for sandbox execution, retries=25 for malformed or empty LLM outputs, seed=42, max_tokens=8192, temperature=0.7, top_p=0.9, best_of=1, repetition_penalty=1.05, and num_paths=16 for the CodeAct loop, with vLLM as the inference engine (Islam et al., 27 Nov 2025).

The framework is evaluated on Bangla NL2Code using mHumanEval. The primary metric is pass@1,

$\text{pass@1}=\frac{\sum_{i=1}^{N}\mathbf{1}[c_i\ \text{passes tests}]}{N},$

with the paper also reporting the unbiased estimator for pass@k. Among the evaluated models, Qwen3-8B with BanglaCodeAct achieved pass@1 of 94.0\% on the development set and 71.6\% on the blind test set, ranking 8th on development and 17th on test. The same base model under self-consistency reached 88.0\%, under majority voting 66.0\%, under few-shot prompting 46.0\%, and under zero-shot prompting 36.0\% (Islam et al., 27 Nov 2025). This establishes the central empirical claim of the framework: iterative execution-guided self-correction is substantially stronger than static prompting for Bangla instructions.

The reported failure modes are technically specific. They include Bangla semantic ambiguities, implicit constraints, case-sensitivity expectations, empty-string handling, off-by-one errors, inclusive-index mistakes, type mismatches, Unicode handling, and cycling on repeated errors. The “remove first and last occurrence” example shows the mechanism directly: an initial implementation removed only the first occurrence, the observation exposed the failure on remove_Occ("banana", "a"), and the Fixer introduced rfind plus single-occurrence handling before the corrected code passed all tests (Islam et al., 27 Nov 2025).

3. Romanized Bangla, token-level language identification, and POS processing

A major precursor to BanglaCodeAct-style systems is token-level processing of Bangla–English code-mixed text in Roman script. For word-level language identification, one influential formulation uses binary BN-versus-EN classification without sentence context, operating on transliterated social-media tokens and explicitly targeting low-resource conditions (Mandal et al., 2018). The reported dataset is disjointly split into train 6632×2, dev 300×2, and test 700×2, with lowercasing, removal of tokens shorter than three characters or containing special or numeric characters, elongation normalization, and exclusion of mixed-morpheme tokens such as English stems combined with Bengali suffixes.

Two encodings are defined. The character-based representation maps a–z to indices 1–26, pads or truncates each word to length 15, and converts sequences to one-hot vectors. The root-phone-based representation uses a Bengali-oriented root-phone inventory of 31 phones and similar-phone groups; a variable-length sliding window over Roman graphemes with n \in \{3,2,1\} maps matched n-grams to root-phone indices and unmatched spans to an unknown index set to 35 (Mandal et al., 2018). This design is intended to collapse transliteration variants such as khabar and khbr into the same phonetic sequence.

The underlying classifiers are LSTMs trained in Keras with sigmoid outputs, Adam, binary cross-entropy, 500 epochs, batch size 1658, and fixed input length 15. The character LSTM uses layer sizes 15–35–25–1; the phone LSTM uses 15–15–40–1. Individually, the character model reached 91.71\% test accuracy under threshold calibration and the phone model 90.42\%. Two ensembles were then reported: stacking at 91.78\% and a threshold-based mean ensemble at 92.35\% on test, compared with a linear-kernel SVM baseline with character bi/tri/quadgrams at 83.64\% (Mandal et al., 2018). The threshold-based decision rule uses the mean probability

$\bar p=\frac{p_{\text{char}}+p_{\text{phone}}}{2}$

with a tuned threshold $\theta \le 0.9$ to decide BN versus EN.

A distinct but related modular approach addresses POS tagging for English–Bengali code-mixed tweets where Bengali appears in Roman script (Raha et al., 2020). It first performs token-level LID with a character-embedding stacked LSTM trained on 11,060 Romanized Bengali words and 7,223 English words, reporting 91% validation accuracy. Bengali segments are then transliterated by a character-level sequence-to-sequence model trained on 22,781 Romanized–target-script word pairs, with 87% validation accuracy. English segments are tagged with the Stanford POS Tagger, Bengali segments with an earlier Bengali POS tagger, and the outputs are merged and mapped into a universal tag set. On 100 manually POS-tagged code-mixed sentences, the system reported 75.29% accuracy and Krippendorff’s Alpha 0.7522 (Raha et al., 2020).

These token-level systems expose the main structural difficulty of Bangla code-mixed NLP: errors are often induced not by the final classifier alone but by Romanization variability, phonetic reduction, ambiguous lexical items such as to, transliteration mismatches, and code-switch boundaries (Mandal et al., 2018). This suggests that BanglaCodeAct-style systems outside NL2Code benefit from explicit subword or phonetic normalization before higher-level inference.

4. Corpus creation, annotation schemes, and benchmark diagnostics

BanglaCodeAct-style research depends heavily on curated corpora with explicit annotation of code-mixing. A foundational resource for sentiment analysis is a gold-standard Bengali–English Twitter corpus with token-level language tags BN, EN, and UN, plus tweet-level polarity labels Positive, Negative, and Neutral (Mandal et al., 2018). Data collection began with approximately 89,000 raw tweets obtained through the Twitter public streaming API via Twitter4j, filtered to approximately 10,500, and manually reduced to approximately 5,000 code-mixed tweets for annotation. The released gold-standard splits contain 2,500 training tweets and 2,460 testing tweets.

The annotation process combines hybrid pre-annotation with human review. Language tagging uses a two-step hybrid of lexicon matching and a LinearSVC trained on character bi-grams and tri-grams; sentiment tagging combines rule-based checks for “feeling” expressions, emoticons, and hashtags with supervised classification, where SGDClassifier performed best on a held-out 600-tweet test with F1=78.70 and accuracy=78.66. On the final expanded corpus, the sentiment tagger reached accuracy=80.97% and F1=81.2%, while the language tagger achieved 81% accuracy (Mandal et al., 2018). Human agreement was strong: for language tagging, Cohen’s $\kappa$ was 0.83 between annotator groups A and B; for sentiment tagging, it was 0.94 between A and B.

The same work formalizes code-mixing intensity through Code-Mixing Index and Complexity Factor. The canonical CMI formula reported for code-mixing work is

$CMI(u)=100 \times \left(1-\frac{\max_{L \in \mathcal{L}} n_L(u)}{n(u)-n_{other}(u)}\right),$

where $n_L(u)$ counts tokens of language $L$ in utterance $u$ , $n(u)$ is total length, and $n_{other}(u)$ counts tokens tagged as Other or Unknown (Mandal et al., 2018). The corpus reports classwise CMI and CF statistics, with Positive tweets showing the highest average CMI in training and testing.

For offensive-language identification, TB-OLID provides a separate but complementary benchmark built from 5,000 manually annotated Latin-script Bangla Facebook comments, split 4,000 train and 1,000 test (Raihan et al., 2023). Its annotation has three levels: Level 1 distinguishes transliterated Bangla T from transliterated code-mixed Bangla C; Level 2 labels offensive O versus not-offensive N; and Level 3 marks the offense target as I, G, or U. Class distributions are reported exactly: T=2,959, C=2,041, O=2,381, N=2,619, I=1,192, G=954, U=235. Pairwise Cohen’s $\bar p=\frac{p_{\text{char}}+p_{\text{phone}}}{2}$ 0 on 1,000 instances was 0.77 for Level 1, 0.72 for Level 2, and 0.66 for Level 3 (Raihan et al., 2023).

The empirical modeling results on TB-OLID are notable for their tokenizer and pretraining implications. For offensive-language identification, fBERT and HateBERT achieved the best overall F1 on the full test set at 0.72, with HateBERT also leading target-type classification at 0.68 F1 overall. Bangla-BERT performed markedly worse on offense detection at 0.42 F1, while XLM-R and roBERTa reached 0.41 (Raihan et al., 2023). The paper attributes this pattern partly to script mismatch and reports that 38.42% of all tokens in the code-mixed class are English, which helps explain why English pre-trained, offense-specialized transformers transfer well to Romanized and code-mixed Bangla.

5. Customer-service retrieval-augmented generation and code-switching support

A different interpretation of BanglaCodeAct appears in multilingual customer-service design. In this setting, the term is used as a blueprint for a Bengali–English code-switching assistant that combines retrieval-augmented generation, prompt engineering, query normalization, and language/script mirroring (Kruk et al., 28 Mar 2025). The architecture centers on GPT-4o with temperature=0, an enterprise FAQ corpus containing question-answer pairs in English and Bengali with approximately 61% English and 39% Bengali content, dense retrieval via text-embedding-3-large, cosine similarity scoring, and reranking with bge-reranker-v2-m3.

The retrieval pipeline is technically explicit. FAQ entries are indexed as combined question-plus-answer strings; five documents are retrieved initially; the reranker selects the top three; and an exact-match shortcut compares the query embedding against FAQ question embeddings, returning the corresponding FAQ answer directly when cosine similarity is at least 0.8 (Kruk et al., 28 Mar 2025). Query normalization proceeds by translating any user query—Bangla script, Romanized Bangla, mixed Banglish, or South Asian English—to English before retrieval. The reported rationale is empirical: translation to English improved retrieval in code-switching contexts because multilingual embeddings underperformed on Banglish, with Appendix B showing failure to cluster Banglish sentence variants even when monolingual Bengali and English variants clustered correctly.

The evaluation is preliminary but specific. On 20 real customer queries—6 Bengali-script, 9 Banglish Romanized, and 5 English or South Asian English—retrieval at k=5 before reranking reached Precision=0.57, Recall=0.42, and [MRR](https://www.emergentmind.com/topics/memory-reuse-rate-mrr)=0.85; retrieval at k=3 before reranking reached Precision=0.74, Recall=0.35, and MRR=0.85; and retrieval at k=3 after reranking reached Precision=0.69, Recall=0.35, and MRR=0.86 (Kruk et al., 28 Mar 2025). Generation achieved 1.00 language/script matching across Bengali, Banglish, and English, with per-language context-adherence accuracy of 0.92 for Bengali, 0.84 for Banglish, and 0.68 for English; the overall accuracy is reported as 0.81 in the abstract.

This line of work broadens the BanglaCodeAct idea from token or sentence labeling to grounded interactive systems. Its defining constraints are not only code-switching and Romanization, but also enterprise approval, escalation logic, cost control, and latency. The prescribed guardrails are correspondingly strict: answer only from the provided context, mirror the user’s language and script, ask clarifying questions when context is insufficient, or connect the user to a human operator (Kruk et al., 28 Mar 2025).

6. Limitations, misconceptions, and research trajectory

A recurrent misconception is that BanglaCodeAct refers to a single, historically continuous benchmark across the entire Bangla NLP literature. The evidence is narrower. The explicit named framework is the Bangla-to-Python system of 2025, whereas earlier work on POS tagging states directly that the term is absent from that paper, even if the methods are adaptable to such a setting (Raha et al., 2020). The broader use of the label is therefore best treated as an overview or adaptation layer rather than as original nomenclature.

Several technical limitations recur across the literature. Word-level LID without context remains vulnerable to ambiguous homographs such as choke, borrowed words, and mixed morphemes; the phone-based encoder helps with transliteration variance such as khbr versus khabar, but fails on highly lossy forms such as krci (Mandal et al., 2018). POS pipelines suffer from error propagation across LID, transliteration, and tagging stages, as well as non-standard mapping choices such as INTJ→NOUN and NEG→ADV (Raha et al., 2020). Customer-service RAG evaluations remain preliminary, with only 20 queries and explicit trade-offs between reranking gains in MRR and drops in precision@3 for multi-document answers (Kruk et al., 28 Mar 2025). In offensive-language modeling, performance is consistently worse on pure transliterated Bangla than on code-mixed text, and Bangla-script-pretrained models are hindered by Romanized input (Raihan et al., 2023). In Bangla NL2Code, persistent ambiguity, conflicting cues, and iteration-cap limits can prevent convergence even with execution feedback (Islam et al., 27 Nov 2025).

Future directions are correspondingly concrete. For code-mixed LID, proposed extensions include multi-class softmax heads for BN/EN/Other, per-class calibration, BiLSTM-CRF sequence decoding, attention over characters or phones, and transformer encoders pretrained on larger unlabeled code-mixed corpora (Mandal et al., 2018). For customer-service systems, the recommended agenda includes broader datasets, human evaluations, multiple retrievers, threshold sweeps, and systematic study of meaning preservation under translation-to-English normalization (Kruk et al., 28 Mar 2025). For offensive-language identification, the natural next step is stronger normalization for non-standard Romanization and possibly Romanized-Bangla-specific modeling (Raihan et al., 2023). For BanglaCodeAct in the strict NL2Code sense, the paper itself proposes richer agent roles such as a Validator or Retriever, better test generation, hybrid retrieval, lightweight fine-tuning on curated Bangla NL2Code pairs, and adaptive iteration control (Islam et al., 27 Nov 2025).

The overall trajectory is technically consistent. BanglaCodeAct, whether understood narrowly as execution-guided Bangla NL2Code or more broadly as Bangla-focused code-mixed system design, is organized around the same fundamental problem: Bangla language technology must operate under transliteration noise, mixed-language inputs, low-resource supervision, and domain-specific correctness constraints. The literature indicates that robust progress usually comes not from a single model family, but from careful decomposition—phonetic or character encoders for Romanized text, hybrid lexicon-plus-model pipelines for annotation, grounded retrieval for enterprise settings, and executable feedback for program synthesis.