Ben-Util: Beyond Benford Analysis

• Ben-Util is an R package that compares real-world digit distributions using Newcomb–Benford’s Law and the BDS model with upper-bound constraints.
• It employs chi-square goodness-of-fit tests to objectively assess which digit distribution model best matches the observed data.
• The package provides user-friendly functions for digit extraction, theoretical probability computation, and visual comparison to support model selection.

Ben-Util (BeyondBenford) is an R package designed to compare the empirical digit distributions of real-world datasets with two theoretical models: Newcomb–Benford’s Law and the Blondeau Da Silva (BDS) model. Its primary utility is for researchers and practitioners evaluating whether observed digit frequencies—particularly leading or p-th digits—are better explained by the widely cited logarithmic Benford distribution or the parameterized, data-bound BDS model (Silva, 2019).

1. Decision Problem: Modeling Digit Distributions

Empirical investigations consistently demonstrate that leading digits in many natural datasets are not uniformly distributed, instead often exhibiting a logarithmic decay described by Newcomb–Benford’s Law. The BDS model extends this, formalizing how an upper-bound constraint on the data induces fluctuations around Benford frequencies. Ben-Util operationalizes this comparative inquiry by offering a workflow to assess, for any dataset of positive real numbers, whether the Benford or BDS distribution more appropriately models its observed digits, using goodness-of-fit metrics (Silva, 2019).

2. Explicit Theoretical Formulations

Let $d$ be the digit of interest (first digit $d \in \{1, \ldots, 9\}$; in general, the $p$th digit $d \in \{0,\ldots,9\}$).

Newcomb–Benford’s Law:

$P_B(d) = \log_{10}(1 + 1/d)$

Blondeau Da Silva’s (BDS) Model:

If the integer upper bound for the data is $u_b$ and focusing on the $p$th digit,

$$P_{BDS}(d; p, u_b) = \frac{1}{u_b - 10^{p-1} + 1} \sum_{i = d \cdot 10^{p-1}}^{u_b} \frac{1}{i}$$

For first digit $(p=1)$:

$$P_{BDS}(d; u_b) = \frac{1}{u_b} \sum_{i=d}^{u_b} \frac{1}{i}$$

These formulas codify the two candidate distributions underlying Ben-Util comparisons. The Benford model is parameter-free; the BDS model depends explicitly on $d \in \{1, \ldots, 9\}$0 matched to the dataset’s observed maximum (Silva, 2019).

3. Core Functionality and Workflow

Ben-Util (BeyondBenford) provides a suite of R functions facilitating full digit-distribution analysis:

• obs.numb.dig(dat, dig): Extracts observed frequencies for each digit (positions 0–9) at digit position dig from a vector or data-frame dat.
• Benf.val(fig, dig): Computes Benford’s theoretical probability for digit fig at position dig.
• Blon.val(fig, dig, upbound): Computes BDS’s probability for digit fig at position dig, with parameter upbound.
• dat.distr(dat, dig=1, upbound=ceiling(max(dat)), ...): Plots histograms of observed digit frequencies, optionally overlaying theoretical curves and providing $d \in \{1, \ldots, 9\}$1 summaries.
  • Key arguments:
  • nclass: Number of bins,
  • theor: Whether to plot the BDS curve,
  • nchi: If $d \in \{1, \ldots, 9\}$2, merges bins for robust $d \in \{1, \ldots, 9\}$3.
• digit.distr(dat, dig, upbound, mod="ben"|"BDS"|"benblo", ...): Plots comparisons between the observed, Benford, and/or BDS digit distributions.
• chi2(dat, dig, upbound=…, mod="ben"|"BDS", pval=0|1): Computes Pearson’s $d \in \{1, \ldots, 9\}$4 and optionally $d \in \{1, \ldots, 9\}$5-value for goodness-of-fit to the chosen model (Silva, 2019).

This workflow enables detailed side-by-side quantitative and graphical comparisons between candidate distributions.

4. Observed Versus Expected Frequencies

Frequencies are computed as follows:

• Observed counts:

$d \in \{1, \ldots, 9\}$6

For first digit analyses, focus on $d \in \{1, \ldots, 9\}$7.

• Expected counts:
  • Benford:

  $d \in \{1, \ldots, 9\}$8 - BDS:

  $d \in \{1, \ldots, 9\}$9

with $p$0 indicating records with at least dig digits (Silva, 2019).

These serve as empirical-theoretical pairs input to the $p$1 test.

5. Statistical Testing: Pearson’s $p$2 Test

For bins $p$3 (typically digits 1–9 or 0–9):

$p$4

Under the null hypothesis ($p$5), $p$6 with $p$7 if no parameters are estimated from the data. The $p$8-value is

$p$9

In Ben-Util, bins are merged if necessary to ensure all expected frequencies $d \in \{0,\ldots,9\}$0, following the standard Pearson’s rule. This procedure provides a statistical measure of the adequacy of each model’s fit (Silva, 2019).

6. Example Analysis Pipeline

The illustrative R workflow using Ben-Util includes:

$P_B(d) = \log_{10}(1 + 1/d)$5

Reported outputs for this example indicate strong fits for both models:

• Benford $d \in \{0,\ldots,9\}$1, $d \in \{0,\ldots,9\}$2
• BDS $d \in \{0,\ldots,9\}$3, $d \in \{0,\ldots,9\}$4 The lower $d \in \{0,\ldots,9\}$5 and higher $d \in \{0,\ldots,9\}$6 indicate BDS provides a marginally better fit (Silva, 2019).

7. Model Selection and Interpretation

Assessment proceeds by juxtaposing $d \in \{0,\ldots,9\}$7 statistics and $d \in \{0,\ldots,9\}$8-values. The preferred model is the one yielding the smaller $d \in \{0,\ldots,9\}$9 and larger $P_B(d) = \log_{10}(1 + 1/d)$0-value:

• If both $P_B(d) = \log_{10}(1 + 1/d)$1, either model may be considered plausible.
• If only one model yields $P_B(d) = \log_{10}(1 + 1/d)$2, preference should be given to that model.
• For closely matching $P_B(d) = \log_{10}(1 + 1/d)$3-values, differences in the $P_B(d) = \log_{10}(1 + 1/d)$4 statistic may provide additional guidance (Silva, 2019).

A plausible implication is that, in datasets subject to explicit upper bounds or selection effects, BDS should be favored over Benford when it offers a superior empirical fit. However, with both models highly plausible, interpretative caution and context-specific considerations remain essential.

---

This framework enables rigorous, reproducible comparison of empirical digit distributions in diverse disciplines, using the operational tools provided by the Ben-Util (BeyondBenford) package (Silva, 2019).