---
title: 'NovaLAD: Dual Systems in Optimization & Parsing'
url: https://www.emergentmind.com/topics/novalad
type: topic
---

# NovaLAD: Dual Systems in Optimization & Parsing

Searching arXiv for "NovaLAD" to identify the relevant papers and disambiguate the term.
{"query":"NovaLAD", "max_results": 10}
to=arxiv_search  เดิมพันฟรี json
{"query":"NovaLAD", "max_results": 10}
NovaLAD is a name used in recent arXiv literature for two technically unrelated systems. In convex optimization, it denotes the New Fast Adaptive Linearized Alternating Direction Multiplier Method, an adaptive linearized ADMM variant for separable convex programs with linear constraints [2404.11435]. In document AI, it denotes a CPU-first document-extraction engine that converts PDFs or scans into structured JSON, Markdown, RAG-ready chunks, and knowledge graphs through concurrent detection, OCR, and optional Vision-LLM enrichment [2603.00122]. The shared name therefore identifies two distinct research lines rather than a single framework.

## 1. Nomenclature and scope

The optimization paper, "A New Fast Adaptive Linearized Alternating Direction Multiplier Method for Convex Optimization" [2404.11435], presents NovaLAD as an adaptive linearized alternating direction multiplier method for convex optimization. Its core claim is that iterative step-size adjustment improves the convergence rate of LADMM-based algorithms while ensuring convergence. The document-analysis paper, "NovaLAD: A Fast, CPU-Optimized Document Extraction Pipeline for Generative AI and Data Intelligence" [2603.00122], presents NovaLAD as a comprehensive parsing system for document extraction before retrieval-augmented generation, knowledge bases, and downstream generative AI.

The duplication of terminology is significant because the two systems share neither objective, data model, nor evaluation protocol. One addresses constrained convex minimization with augmented-Lagrangian machinery; the other addresses page-image parsing with YOLOv10 detection, ViT filtering, hybrid OCR, and optional Vision-enabled LLM calls. A common misconception is therefore that NovaLAD designates a single method spanning optimization and document parsing. In the arXiv record, it does not.

## 2. NovaLAD in convex optimization

In the optimization setting, NovaLAD is defined for the convex composite problem
$$
\min \{\, \theta_1(x) + \theta_2(y) \mid Ax + By = b,\; x \in X,\; y \in Y \,\},
$$
where $\theta_1:\mathbb{R}^{n_1}\to\mathbb{R}\cup\{+\infty\}$ and $\theta_2:\mathbb{R}^{n_2}\to\mathbb{R}\cup\{+\infty\}$ are proper, closed, convex and not necessarily smooth, $X\subseteq\mathbb{R}^{n_1}$ and $Y\subseteq\mathbb{R}^{n_2}$ are closed convex sets, and $A\in\mathbb{R}^{m\times n_1}$ and $B\in\mathbb{R}^{m\times n_2}$ have full column rank [2404.11435]. No Lipschitz-gradient assumption is required on $\theta_1$ or $\theta_2$; the standing assumptions are convexity and separability.

The augmented Lagrangian is
$$
L_\beta(x,y,\lambda)=\theta_1(x)+\theta_2(y)-\lambda^\top(Ax+By-b)+(\beta/2)\|Ax+By-b\|^2.
$$
Classical ADMM alternately minimizes $L_\beta$ in $x$ and $y$ and then updates $\lambda$. When the $y$-subproblem is expensive, the quadratic term is linearized. Writing
$$
H(y)=\tfrac12\|Ax^{k+1}+By-b\|^2,
$$
the paper approximates around $y^k$ by
$$
H(y)\approx H(y^k)+\nabla H(y^k)^\top(y-y^k)+(\mu_k/2)\|y-y^k\|^2,
$$
with
$$
\nabla H(y^k)=B^\top(Ax^{k+1}+By^k-b).
$$
The constant $\mu_k$ is chosen large enough to dominate $\|B^\top B\|$.

NovaLAD’s distinctive feature is the replacement of the fixed linearization weight $r$ by an adaptive sequence $\delta_k$. The paper motivates this by the standard LADMM trade-off: larger regular-term coefficients ensure convergence but tend to produce small step sizes, whereas smaller coefficients enlarge the iterative step size but tend toward non-convergence. NovaLAD addresses this balance through adaptive parameter selection.

## 3. Adaptive updates, convergence theory, and numerical behavior

Algorithmically, NovaLAD begins by choosing $\tau>1$, $\eta>1$, $\beta>0$, $\delta_{\min}>0$, and $\epsilon\in(0,\tfrac12)$, initializing $\delta_{-1}=\delta_0=0.75\|B^\top B\|$, and selecting $(x^0,y^0,\lambda^0)\in X\times Y\times\mathbb{R}^m$ [2404.11435]. The iteration computes
$$
x^{k+1}=\arg\min_{x\in X}\{\theta_1(x)-(\lambda^k)^\top Ax+(\beta/2)\|Ax+By^k-b\|^2\},
$$
then updates $y^{k+1}$ through the linearized subproblem
$$
y^{k+1}=\arg\min_{y\in Y}\Bigl\{\theta_2(y)-(\lambda^k)^\top By+\beta\langle B^\top(Ax^{k+1}+By^k-b),\,y-y^k\rangle+(\beta\delta_k/2)\|y-y^k\|^2\Bigr\},
$$
and finally
$$
\lambda^{k+1}=\lambda^k-\beta(Ax^{k+1}+By^{k+1}-b).
$$

The adaptive mechanism is driven by a local curvature test. The inner loop increases $\delta_k$ until either
$$
\delta_k\|y^{k+1}-y^k\|^2 > \frac{1}{2\epsilon}\|B(y^{k+1}-y^k)\|^2
$$
or $y^{k+1}=y^k$ holds; otherwise $\delta_k\leftarrow \tau\delta_k$ and the $y$-subproblem is resolved. If $\delta_k>\delta_{k-1}$, the lower bound is enlarged by $\delta_{\min}\leftarrow \eta\,\delta_{\min}$. The next parameter is then set by
$$
\delta_{k+1}=\max\{\, h_{k+1},\; \min\{\delta_{\min},\|B^\top B\|\}\,\},
$$
where
$$
h_{k+1}=
\begin{cases}
\delta_k, & \text{if } y^{k+1}=y^k,\\
\|B(y^{k+1}-y^k)\|^2/\|y^{k+1}-y^k\|^2, & \text{otherwise.}
\end{cases}
$$

The convergence analysis proceeds through a variational-inequality reformulation of the KKT system, an intermediate point
$$
\tilde w^k=(x^{k+1},y^{k+1},\lambda^k-\beta(Ax^{k+1}+By^k-b)),
$$
and weighted descent estimates in the $(y,\lambda)$ variables [2404.11435]. Theorem 3.1 establishes one-step descent with
$$
H_{k+1}=\operatorname{diag}(\beta\delta_k I_{n_2},(1/\beta)I_m)
$$
and
$$
D_{k+1}=\delta_k I_{n_2}-(1/(2\epsilon))B^\top B \succ 0,
$$
the latter positivity being guaranteed by the curvature condition. Theorem 3.2 allows the weighting matrix to grow slowly by introducing $\xi_k=\max\{0,\delta_k-\delta_{k-1}\}$. Because $\sum_k \xi_k<\infty$ by construction, the Robbins-Siegmund lemma yields convergence of $\|v^k-v^*\|_{H_k}$ together with $\|y^{k+1}-y^k\|\to 0$ and $\|\lambda^{k+1}-\lambda^k\|\to 0$. Theorem 3.4 then gives global convergence of the full iterate $(x^k,y^k,\lambda^k)$ to some saddle-point $w^\infty\in\Omega^*$.

The paper is explicit about what is not proved: it establishes global convergence but does not supply an explicit $O(1/k)$ or $O(1/k^2)$ rate. Numerically, it reports linear residual decay, but a complexity bound is left for future work. On the LASSO benchmark
$$
\min \{ \tfrac12\|Ay-b\|^2+\sigma\|y\|_1\},
$$
reformulated as
$$
\min_{x,y}\{\tfrac12\|x-b\|^2+\sigma\|y\|_1 \;\text{ subject to }\; x=Ay\},
$$
the data are generated with $A\in\mathbb{R}^{m\times n}$ having i.i.d. $N(0,1)$ columns that are $\ell_2$-normalized, $y_{\text{true}}$ with $100/n$ density, and $b=Ay_{\text{true}}+\text{noise}$ with $\sigma=10^{-3}$. The regularization parameter is $\sigma=0.1\|A^\top b\|_\infty$. Against “OLADMM” from [28], NovaLAD uses $\delta_0=0.75\|A^\top A\|$, $\delta_{\min}=0.05\|A^\top A\|$, $\tau=1.1$, $\eta=1.1$, $\beta=1$, and $\epsilon=5/11$, while OLADMM uses $r=0.75\|A^\top A\|$ and $\beta=1$. With stopping criteria based on the primal residual $\|x^{k+1}-Ay^{k+1}\|<\epsilon_{\text{pri}}$ and dual residual $\|\beta A(y^{k+1}-y^k)\|<\epsilon_{\text{dual}}$ with $\epsilon_{\text{abs}}=10^{-6}$ and $\epsilon_{\text{rel}}=10^{-4}$, the reported result for $m=4000$, $n=5500$ is that NovaLAD requires fewer iterations and lower CPU time than OLADMM.

## 4. NovaLAD in document extraction: architecture and data flow

In document AI, NovaLAD is a fully-featured, CPU-first document-extraction engine that turns page images from PDFs or scans into richly structured representations—JSON, Markdown, RAG-ready chunks, and knowledge graphs—while sustaining state-of-the-art accuracy on DP-Bench and sub-10 s per-page latency on commodity CPUs [2603.00122]. The end-to-end pipeline has six stages coordinated for parallel CPU utilization and minimal redundant work.

Stage 1 performs parallel detection in a `ThreadPoolExecutor` using two concurrent YOLOv10 models. The element detector operates at threshold $0.30$ and the layout detector at threshold $0.20$. Stage 2 applies image classification with a ViT: all `image` elements are cropped to $224\times224$, passed through ViT-Base/patch16-224, and filtered so that “Useful” images and all tables are retained. Stage 3 merges layout and element detections by assigning every element box whose midpoint lies in a layout box to that layout region, producing a `PageRepresentation` with `Groups` and `Non-groups`. Stage 4 performs content extraction through a hybrid OCR strategy: PyMuPDF is used for PDF text layers on text-like elements, EasyOCR (English) is applied to cropped images for tables and figures, and the resulting text is normalized with NFKC, bullets, and punctuation handling. Stage 5 optionally invokes a Vision-enabled LLM for each retained image or table, using a structured prompt to obtain `{title, summary, data (rows as list of dicts)}`. Stage 6 emits JSON and then, in parallel, Markdown, RAG chunks, a knowledge graph, and document category classification.

This workflow is explicitly parallel. Pages are rasterized once at 300 DPI with `pypdfium2`, both YOLO inferences run concurrently on CPU, ViT classification is dispatched across image crops, PDF extraction and EasyOCR are parallelized across elements, and export tasks run simultaneously. In production, each page yields one JSON, one Markdown document, a set of RAG-ready chunks, and a compact knowledge graph, with wall-clock time dominated by the slowest export sub-task rather than the sum of all export times.

## 5. Detection, grouping, OCR, Vision-LLM enrichment, and benchmark metrics

The element detection model is YOLOv10 (Ultralytics), with input $640\times640$ RGB, trained from scratch for 50 epochs to detect 11 classes: `{title, header, section, page_header, page_footer, text, list_item, table_of_content, table, image, table_caption, image_caption}` [2603.00122]. Its reported inference quality is typical `mAP50 = 0.859`, `mAP50–95 = 0.670`, `precision 0.806`, and `recall 0.813`. The layout detection model is also YOLOv10, trained for 30 epochs on a dedicated layout-annotated corpus for 6 structural region types: `{layout_box, column_group, column_text, multi_column, row_group, group}`. It reports `mAP50 = 0.567`, `mAP50–95 = 0.512`, `precision 0.619`, and `recall 0.530`. These layout regions drive rule-based grouping and reading order.

The ViT-based image relevance classifier uses `google/vit-base-patch16-224-in21k` with 12 layers, `hidden_dim = 768`, 12 heads, `patch_size = 16`, and `input_size = 224`. It is fine-tuned with binary cross-entropy on “Useful” versus “Useless” crops for 4 epochs, reaching final evaluation accuracy `98.53 %` and loss `0.0851`. Thresholding is by `argmax` over two logits. Useless images are stored in `skipped_images` and excluded from downstream LLM calls and exports.

Grouping is rule-based. For `multi_column` regions, x-centers are normalized with `MinMaxScaler`, then `DBSCAN(eps=0.3,min_samples=2)` is applied and members are sorted by descending `y_center`. For `row_group`, elements are sorted by `(x_center, -y_center)` and adjacent pairs are swapped whenever `line_angle(e_i,e_{i+1}) \ge 50^\circ` to respect left-to-right visual order. Generic groups are sorted by descending `y_center`. At page level, all entities are sorted by key `(-y_center, x_center)`, duplicates are resolved with `keep_highest_confidence()`, groups and non-groups are merged by vertical ordering, and insertion order is rebuilt to produce stable reading order. Header/footer correction is then performed by collecting candidate strings across pages and fuzzy matching with `thefuzz.ratio > 95`.

The Vision-LLM stage is optional and deliberately gated. Only images labeled “Useful” and all tables are forwarded to a Vision-enabled LLM such as GPT-4-vision or GPT-4o-mini. The prompt requests a one-line title, a 2–3 sentence summary, and tabular data when applicable. The inference parameters are `temperature = 0.0`, `max_tokens = 512`, and `top_p = 1.0`, and calls are batched per page in parallel. Returned JSON is merged into each `Entity.value` under `{title, summary, text/data}`.

Evaluation on DP-Bench uses NID (Normalized Indel Distance) for layout and reading order and TEDS (Tree Edit Distance Similarity) for tables [2603.00122]. NID is defined as
$$
\mathrm{NID} = 1 - \frac{\mathrm{ins}+\mathrm{del}}{|\mathrm{reference}|+|\mathrm{prediction}|},
$$
with tables and figures excluded. TEDS-S omits cell content and measures structure only. On Table 6, NovaLAD reports `96.49% TEDS`, `98.51% NID`, and `8.50 s` average time, compared with `93.48%`, `97.02%`, `3.79 s` for Upstage; `88.05%`, `96.71%`, `14.47 s` for AWS Textract; `87.19%`, `87.69%`, `4.44 s` for Microsoft AI; `74.57%`, `92.82%`, `4.14 s` for Llamaparse; `65.56%`, `91.18%`, `13.14 s` for Unstructured; and `66.13%`, `90.86%`, `5.85 s` for Google DocAI. The reported interpretation is that NovaLAD leads in both table-structure and layout-order accuracy while running entirely on CPU.

## 6. Practical interpretation and recurring misconceptions

The two NovaLAD systems illustrate two different uses of adaptivity and systems integration. In the optimization paper, adaptivity is mathematical: the proximal coefficient $\delta_k$ is adjusted to satisfy a curvature condition while preserving global convergence [2404.11435]. In the document-extraction paper, adaptivity is architectural: the pipeline uses concurrent detectors, ViT-based gating, hybrid OCR, and optional Vision-LLM enrichment to manage speed, cost, and semantic detail [2603.00122]. This suggests that the shared name reflects separate local naming decisions rather than a shared technical lineage.

Several misconceptions are directly addressed by the underlying papers. For optimization, NovaLAD is not presented with an explicit nonasymptotic complexity rate; the proved result is global convergence to a saddle-point, and the observation of linear residual decay is numerical rather than a formal rate theorem. For document extraction, CPU-first does not mean rule-only parsing or the absence of modern deep models; the system combines two YOLOv10 detectors, a ViT classifier, EasyOCR, and an optional Vision-enabled LLM. The deployment profile is also concrete: average per-page profiling on an 8-core CPU gives `2.1 s` for dual YOLO, `0.4 s` for ViT image classification with average 5 images per page, `1.8 s` for EasyOCR on tables and images, `1.0 s` for two LLM calls per page when enabled, and `1.2 s` for exports, for totals of approximately `7.5 s` without LLM and `8.5 s` with LLM. Reported memory footprint is approximately `1.2 GB` for the shared YOLOv10 models, `0.9 GB` for ViT, `0.5 GB` for EasyOCR, and `2.6 GB` total peak RAM. Disabling Vision-LLM insights saves approximately `1 s/page` with only marginal loss in semantic richness, while reducing YOLO resolution to `512×512` yields a `20%` speedup at an approximately `2–3%` drop in mAP.

Taken together, the term NovaLAD currently identifies two distinct contributions: an adaptive linearized ADMM variant for constrained convex optimization, and a CPU-optimized document extraction pipeline for generative-AI and data-intelligence workflows. Any technical discussion of NovaLAD therefore requires domain disambiguation at the outset.

Source: https://www.emergentmind.com/topics/novalad