Improving Minority Population Sampling with BISG Probabilities: Evidence from a Survey of Jewish Americans

Abstract: Sampling geographically dispersed minority populations poses substantial challenges when individual group membership cannot be directly observed. Although stratified sampling can offer efficiency gains, these gains are typically modest unless the minority population is highly concentrated within a small number of strata. In this paper, we propose using Bayesian Improved Surname Geocoding (BISG) to enhance the efficiency of minority population sampling. BISG generates individual-level probabilities of minority group membership based on names and residential addresses. We incorporate these probabilities into a stratified Poisson probability sampling design. Applying the proposed approach to a national survey of Jewish Americans, we find that our estimates closely align with those from a large-scale Pew Research Center survey of the same population, which relied on a substantially more expensive sampling strategy involving geographic stratification and screening. At a fraction of the cost, our survey reproduces nearly identical patterns observed by Pew, including estimates of religious denominations and participation in specific religious activities.

• The paper introduces an innovative BISG-based stratified Poisson framework that efficiently targets minority populations.
• The study employs a hierarchical Dirichlet-multinomial Bayesian model to stabilize surname-geography probability estimates amidst sparse data.
• Empirical results show improved response yield and cost efficiency, closely aligning survey outcomes with established benchmarks.

Efficient Minority Population Sampling Using BISG: A Technical Synthesis

Problem Context and Motivation

Sampling small, geographically diffuse minority populations in the absence of direct membership labels within sampling frames presents severe efficiency barriers for survey research. Conventional approaches—such as screening-based designs or stratified sampling based solely on geography—incur high costs and generally yield only incremental improvements unless the minority group is highly concentrated. The paper "Improving Minority Population Sampling with BISG Probabilities: Evidence from a Survey of Jewish Americans" (2605.05384) proposes leveraging Bayesian Improved Surname Geocoding (BISG) for constructing individual-level stratification, offering a substantial gain in efficiency while maintaining unbiasedness under probability sampling, even when estimated probabilities are imperfect.

Methodological Innovation

The central methodological contribution is the fusion of individual-level BISG scores—estimating the probability of minority group membership from surnames and geographic context—into a stratified Poisson probability sampling framework. The approach can be summarized as follows:

• Surname- and Geography-Informed Probabilities: Assemble $P(R=1|S=s,G=g)$ using minority-group surname lists (here, extracted from >230,000 Jewish funeral home obituaries), external geographic prevalence estimates (Brandeis AJPP), and the overall surname/geography joint distribution (national voter file).
• Hierarchical Estimation of Surname Distributions: To overcome sparse data and geographical imbalance, a hierarchical Dirichlet-multinomial Bayesian model is applied, inducing appropriate shrinkage toward the overall surname distribution as needed for rare surnames and low-frequency states. The degree of shrinkage per state is adaptively tuned via a hierarchical prior on the concentration parameter. This addresses instability of raw sample proportions, especially for under-observed surname-state pairs.
• Sampling Design: Poisson sampling is performed with selection probabilities proportional to the BISG-estimated probability of inclusion in the minority population. Per-state allocations are derived to minimize the variance of the weighted estimator for population means, explicitly accounting for the informativeness of surnames by state.
• Screening for Uninformative Surnames: The sampling frame is conservatively filtered to include only those surnames observed within the obituary data (yielding a frame of ~120 million sampled from the full voter file), thereby improving targeting efficiency while mitigating selection bias found in the traditional approach of relying on highly distinctive surname lists.

  Figure 1: Sampling allocations by state (right) and modeled/proposed sampling yields (left); proposed Poisson/BISG allocation achieves notably higher expected sampling yield across all states.

Empirical Application: Survey of U.S. Jewish Adults

The methodology is validated via a large-scale national survey of the U.S. Jewish population, drawn from the L2 voter file. Key features include:

• Sampling Output and Response: From 49,546 postcards, the response rate was 3.6% (1,765 respondents), with 56.9% self-identified Jewish—a stark improvement over traditional random or geographic stratification methods, where yields are below 4%.
• Cost Structure: The campaign cost was $22,833 (≈$23 per Jewish respondent), an order of magnitude less than the Pew 2020 Jewish American study, whose unit costs exceeded $200 per Jewish respondent due to massive screening overhead.
• Accuracy: Weighted survey results replicate the distributional statistics from Pew for key variables such as denominational affiliation (Figure 2), Jewish social networks (Figure 3), and ritual practice measures (Figure 4), with the only detectable deviation being in the bar/bat mitzvah item—a result not robust after multiple testing correction.

  Figure 2: Distribution of survey respondents' self-identified Jewish religious denominations, showing close agreement with Pew benchmarks.

  

  Figure 3: Distribution of responses to “How many of your close friends are Jewish?”, demonstrating comparable network structures to the Pew survey.

  

  Figure 4: Multiple indicators of Jewish practice; weighted means generally indistinguishable from Pew, with only the Bar/Bat Mitzvah item reaching marginal statistical significance.

Theoretical Considerations and Robustness

The proposed Poisson/BISG procedure maintains unbiasedness for design-based inference so long as the estimated probabilities for all potential minority members are nonzero. The estimator's efficiency—measured by the yield of actual minority respondents per fielded contact—is solely a function of how well name and geography signal group membership, rather than of perfect accuracy in the probabilities themselves.

The hierarchical surname model is justified under partial pooling theory: it enables borrowing of strength from larger states or more common surnames, reducing variance for low-information strata. Posterior means of the shrinkage parameter $\rho_g$ are near zero for high-count states, indicating estimates are data-driven there, and approach one for low-count states, indicating reliance on prior/global surname patterns.

Figure 5: Posterior means for the geographical shrinkage parameter $\rho_g$; nonrepresented states utilize only the overall surname distribution.

Figure 6: Distribution of estimated $P(S=s|R=1)$ for 500 representative surnames, illustrating skew and long-tail patterns.

Figure 7: Joint distributions of $P(S=s|R=1)$ and $P(S=s|R=0)$, highlighting signal separation achieved for high-informativeness surnames.

Given real-world constraints—imperfect external counts, undercoverage in obituary-derived lists—the framework empirically delivers close agreement with gold-standard benchmarks, indicating robustness to moderate misestimation.

Contrasts with Prior Art and Practical Implications

Relative to binary-thresholded or highly selective surname lists, the proposed method:

• Substantially broadens the eligible frame, reducing selection bias associated with distinctive-surname approaches while dramatically raising sampling efficiency.
• Eliminates the need for direct, labeled training data with full population labels, a critical advantage where such data are structurally unavailable (as for religious identification in U.S. administrative records).
• Demonstrates that modest per-unit costs can generate sufficiently large, demographically and behaviorally representative minority samples to support high-fidelity social inference without sacrificing design-based validity.

In application, the method enables community organizations, public health researchers, and political scientists to generate reliable, population-based data for smaller or hard-to-label minority groups at scale, with orders-of-magnitude cost reduction.

Limitations and Potential Extensions

Key limitations include:

• Minority Coverage: Surname-based methods will always risk undercoverage for group members whose names are non-informative or misclassified.
• Frame Definition: Use of the voter file as a base frame can omit non-registered adults, non-citizens, and children, restricting external validity to the voting-age citizen population.
• Data Source Biases: Obituary data, by nature, reflect age and regional biases; future improvements may integrate additional sources or use structural models to further correct such biases.

Potential extensions include application to regional surveys, adaptation to other groups with available surname/geographic data, and methodological development for purely list-based (rather than distributional) surname priors.

Conclusion

The paper rigorously demonstrates that sampling efficiency for rare, label-difficult populations can be dramatically improved by leveraging BISG-derived probabilities within a stratified Poisson framework. Hierarchical Bayesian estimation of joint surname-geography distributions enables robust probability assignment despite sparse data, and downstream survey statistics are empirically validated to match gold-standard reference values. The proposed methodology addresses a longstanding challenge in minority sampling, with substantial practical utility for survey research, public policy, and applied social science.